2024-04-23T14:14:06.436 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T14:14:06.436 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569 --owner scheduled_rishabh@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} --verbose --name rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi -- /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569/orig.config.yaml 2024-04-23T14:14:06.494 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T14:14:06.581 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569 branch: wip-rishabh-testing-20240416.193735-3 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} email: null first_in_suite: false job_id: '7669569' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735-3 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd heartbeat grace: 60 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request - SLOW_OPS - slow request sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735-3 sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7954 sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735-3 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOJKqACtIvRJOSpWkZ2GCQaksijfrwGP67CDNXeRoTCGUvgv2WtsWa5yRAoa/rR6n1D8NUQx0we9O6O7w5S9njE= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIxtn2o/mrPu1GucuiHWSWJIR0rGPngFs0WNIXdLVpxCwpbhetQOBVJ57z8Ciy4t0NJyG82VYYdcnsAIs5qASq8= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEk+B8YSfkKV1bLDhgD+UbEpqvPgJV4Nd6crtUgg+mmFFCWmDQXiT52cTXhTVeien6aQYUmTYsGz7GVQv1m4vKQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_14:04:26 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-23T14:14:06.581 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa; will attempt to use it 2024-04-23T14:14:06.582 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks 2024-04-23T14:14:06.582 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T14:14:06.584 INFO:teuthology.task.internal:Checking packages... 2024-04-23T14:14:06.607 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd44fddd910b5e94da0f2357af4be37d7bd495ae0' 2024-04-23T14:14:06.607 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T14:14:06.607 INFO:teuthology.packaging:ref: None 2024-04-23T14:14:06.607 INFO:teuthology.packaging:tag: None 2024-04-23T14:14:06.607 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735-3 2024-04-23T14:14:06.607 INFO:teuthology.packaging:sha1: d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:14:06.607 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735-3 2024-04-23T14:14:06.875 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3126.gd44fddd9 2024-04-23T14:14:06.876 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T14:14:06.882 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T14:14:06.882 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T14:14:06.905 INFO:teuthology.task.internal:Saving configuration 2024-04-23T14:14:06.926 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T14:14:06.932 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T14:14:06.952 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:35.710731', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOJKqACtIvRJOSpWkZ2GCQaksijfrwGP67CDNXeRoTCGUvgv2WtsWa5yRAoa/rR6n1D8NUQx0we9O6O7w5S9njE='} 2024-04-23T14:14:06.975 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:35.712293', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIxtn2o/mrPu1GucuiHWSWJIR0rGPngFs0WNIXdLVpxCwpbhetQOBVJ57z8Ciy4t0NJyG82VYYdcnsAIs5qASq8='} 2024-04-23T14:14:06.996 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:35.713353', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEk+B8YSfkKV1bLDhgD+UbEpqvPgJV4Nd6crtUgg+mmFFCWmDQXiT52cTXhTVeien6aQYUmTYsGz7GVQv1m4vKQ='} 2024-04-23T14:14:06.996 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T14:14:07.002 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-23T14:14:07.002 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-23T14:14:07.003 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-23T14:14:07.003 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T14:14:07.095 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f309c2c9160>, signals=[15]) 2024-04-23T14:14:07.096 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T14:14:07.102 INFO:teuthology.task.internal:Opening connections... 2024-04-23T14:14:07.102 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-04-23T14:14:07.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:14:07.182 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2024-04-23T14:14:07.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:14:07.259 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-04-23T14:14:07.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:14:07.333 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T14:14:07.340 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-04-23T14:14:07.358 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-04-23T14:14:07.359 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-04-23T14:14:07.416 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-04-23T14:14:07.416 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-04-23T14:14:07.416 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-04-23T14:14:07.416 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-04-23T14:14:07.416 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-04-23T14:14:07.416 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-04-23T14:14:07.416 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T14:14:07.416 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-04-23T14:14:07.416 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-04-23T14:14:07.416 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T14:14:07.416 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-04-23T14:14:07.416 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T14:14:07.417 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T14:14:07.417 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T14:14:07.417 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-04-23T14:14:07.449 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2024-04-23T14:14:07.467 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2024-04-23T14:14:07.467 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2024-04-23T14:14:07.523 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="CentOS Stream" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="9" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:ID="centos" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="rhel fedora" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="9" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el9" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:LOGO="fedora-logo-icon" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://centos.org/" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T14:14:07.524 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T14:14:07.525 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2024-04-23T14:14:07.549 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-04-23T14:14:07.568 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-04-23T14:14:07.568 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-04-23T14:14:07.625 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="CentOS Stream" 2024-04-23T14:14:07.626 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="9" 2024-04-23T14:14:07.626 INFO:teuthology.orchestra.run.smithi203.stdout:ID="centos" 2024-04-23T14:14:07.626 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE="rhel fedora" 2024-04-23T14:14:07.626 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="9" 2024-04-23T14:14:07.626 INFO:teuthology.orchestra.run.smithi203.stdout:PLATFORM_ID="platform:el9" 2024-04-23T14:14:07.626 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T14:14:07.626 INFO:teuthology.orchestra.run.smithi203.stdout:ANSI_COLOR="0;31" 2024-04-23T14:14:07.626 INFO:teuthology.orchestra.run.smithi203.stdout:LOGO="fedora-logo-icon" 2024-04-23T14:14:07.626 INFO:teuthology.orchestra.run.smithi203.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T14:14:07.626 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://centos.org/" 2024-04-23T14:14:07.626 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T14:14:07.626 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T14:14:07.627 INFO:teuthology.orchestra.run.smithi203.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T14:14:07.627 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-04-23T14:14:07.649 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T14:14:07.660 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T14:14:07.668 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T14:14:07.669 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-04-23T14:14:07.671 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2024-04-23T14:14:07.673 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-04-23T14:14:07.689 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T14:14:07.695 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T14:14:07.696 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-04-23T14:14:07.727 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2024-04-23T14:14:07.730 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-04-23T14:14:07.814 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T14:14:07.887 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T14:14:07.901 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-23T14:14:07.901 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-23T14:14:07.901 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-23T14:14:07.902 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T14:14:07.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:14:07.919 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-23T14:14:08.177 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:14:08.177 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-23T14:14:08.177 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-23T14:14:08.194 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-437.el9.x86_64 2024-04-23T14:14:08.194 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:14:08.195 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-23T14:14:08.195 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum upgrade -y linux-firmware 2024-04-23T14:14:11.646 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-04-23T14:14:15.872 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 19 MB 00:03 2024-04-23T14:14:22.389 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.2 MB 00:02 2024-04-23T14:14:24.959 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-23T14:14:27.194 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 21 MB 00:02 2024-04-23T14:14:32.333 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-23T14:14:34.306 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-23T14:14:34.307 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-04-23T14:14:34.307 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-23T14:14:34.427 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-23T14:14:34.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-23T14:14:34.656 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-23T14:14:34.657 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-23T14:14:34.659 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi083.front.sepia.ceph.com to complete... 2024-04-23T14:22:17.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23 14:22:17 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-23T14:22:17.523 INFO:teuthology.orchestra.run.smithi083.stderr:FINISHED --2024-04-23 14:22:17-- 2024-04-23T14:22:17.523 INFO:teuthology.orchestra.run.smithi083.stderr:Total wall clock time: 7m 43s 2024-04-23T14:22:17.523 INFO:teuthology.orchestra.run.smithi083.stderr:Downloaded: 1 files, 341M in 7m 42s (755 KB/s) 2024-04-23T14:22:17.528 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-23T14:22:17.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-23T14:22:19.672 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying... ######################################## 2024-04-23T14:22:19.808 INFO:teuthology.orchestra.run.smithi083.stdout:Preparing... ######################################## 2024-04-23T14:22:19.921 INFO:teuthology.orchestra.run.smithi083.stdout:Updating / installing... 2024-04-23T14:22:27.838 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-23T14:23:05.343 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi083.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-23T14:23:05.344 DEBUG:teuthology.orchestra.run.smithi083:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-23T14:23:05.425 INFO:teuthology.orchestra.run.smithi083.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-23T14:23:05.425 INFO:teuthology.orchestra.run.smithi083.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-23T14:23:05.425 INFO:teuthology.orchestra.run.smithi083.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-23T14:23:05.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-23T14:23:05.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-23T14:23:05.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-23T14:23:05.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-23T14:23:05.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-23T14:23:05.425 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-23T14:23:05.426 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-23T14:23:05.427 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-23T14:23:05.428 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-23T14:23:05.429 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-23T14:23:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-23T14:23:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-23T14:23:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-23T14:23:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-23T14:23:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-23T14:23:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-23T14:23:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-23T14:23:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-23T14:23:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-23T14:23:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-23T14:23:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-23T14:23:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-23T14:23:05.431 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-23T14:23:05.432 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-23T14:23:05.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-23T14:23:05.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-23T14:23:05.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-23T14:23:05.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-23T14:23:05.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-23T14:23:05.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-23T14:23:05.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-23T14:23:05.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-23T14:23:05.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-23T14:23:05.433 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-23T14:23:05.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-23T14:23:05.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-23T14:23:05.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-23T14:23:05.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-23T14:23:05.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-23T14:23:05.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-23T14:23:05.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-23T14:23:05.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-23T14:23:05.434 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-23T14:23:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-23T14:23:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-23T14:23:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-23T14:23:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-23T14:23:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-23T14:23:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-23T14:23:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-23T14:23:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-23T14:23:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-23T14:23:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-23T14:23:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-23T14:23:05.435 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-23T14:23:05.436 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-23T14:23:05.437 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-23T14:23:05.438 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-23T14:23:05.439 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-23T14:23:05.440 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-23T14:23:05.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-23T14:23:05.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-23T14:23:05.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-23T14:23:05.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-23T14:23:05.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-23T14:23:05.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-23T14:23:05.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-23T14:23:05.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-23T14:23:05.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-23T14:23:05.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-23T14:23:05.441 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-23T14:23:05.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-23T14:23:05.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-23T14:23:05.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-23T14:23:05.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-23T14:23:05.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-23T14:23:05.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-23T14:23:05.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-23T14:23:05.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-23T14:23:05.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-23T14:23:05.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-23T14:23:05.442 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-23T14:23:05.443 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-23T14:23:05.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-23T14:23:05.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-23T14:23:05.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-23T14:23:05.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-23T14:23:05.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-23T14:23:05.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-23T14:23:05.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-23T14:23:05.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-23T14:23:05.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-23T14:23:05.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-23T14:23:05.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-23T14:23:05.444 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-23T14:23:05.445 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-23T14:23:05.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-23T14:23:05.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-23T14:23:05.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-23T14:23:05.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-23T14:23:05.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-23T14:23:05.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-23T14:23:05.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-23T14:23:05.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-23T14:23:05.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-23T14:23:05.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-23T14:23:05.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-23T14:23:05.446 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-23T14:23:05.447 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-23T14:23:05.448 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-23T14:23:05.449 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-23T14:23:05.450 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-23T14:23:05.451 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-23T14:23:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-23T14:23:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-23T14:23:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-23T14:23:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-23T14:23:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-23T14:23:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-23T14:23:05.452 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-23T14:23:05.453 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-23T14:23:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-23T14:23:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-23T14:23:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-23T14:23:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-23T14:23:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-23T14:23:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-23T14:23:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-23T14:23:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-23T14:23:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-23T14:23:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-23T14:23:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-23T14:23:05.454 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-23T14:23:05.455 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-23T14:23:05.456 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-23T14:23:05.457 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-23T14:23:05.458 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-23T14:23:05.459 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-23T14:23:05.460 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-23T14:23:05.461 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-23T14:23:05.462 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-23T14:23:05.463 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-23T14:23:05.464 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-23T14:23:05.465 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-23T14:23:05.466 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-23T14:23:05.467 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-23T14:23:05.468 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-23T14:23:05.469 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-23T14:23:05.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-23T14:23:05.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-23T14:23:05.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-23T14:23:05.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-23T14:23:05.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-23T14:23:05.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-23T14:23:05.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-23T14:23:05.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-23T14:23:05.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-23T14:23:05.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-23T14:23:05.470 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-23T14:23:05.471 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-23T14:23:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-23T14:23:05.473 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-23T14:23:05.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-23T14:23:05.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-23T14:23:05.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-23T14:23:05.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-23T14:23:05.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-23T14:23:05.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-23T14:23:05.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-23T14:23:05.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-23T14:23:05.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-23T14:23:05.474 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-23T14:23:05.475 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-23T14:23:05.476 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-23T14:23:05.477 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-23T14:23:05.478 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-23T14:23:05.479 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-23T14:23:05.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-23T14:23:05.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-23T14:23:05.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-23T14:23:05.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-23T14:23:05.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-23T14:23:05.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-23T14:23:05.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-23T14:23:05.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-23T14:23:05.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-23T14:23:05.480 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-23T14:23:05.481 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-23T14:23:05.481 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-23T14:23:05.481 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-23T14:23:05.481 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-23T14:23:05.481 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-23T14:23:05.481 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-23T14:23:05.481 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-23T14:23:05.481 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-23T14:23:05.481 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-23T14:23:05.481 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-23T14:23:05.481 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-23T14:23:05.482 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-23T14:23:05.483 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-23T14:23:05.484 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-23T14:23:05.485 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-23T14:23:05.486 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-23T14:23:05.487 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-23T14:23:05.488 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-23T14:23:05.489 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-23T14:23:05.490 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-23T14:23:05.491 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-23T14:23:05.492 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-23T14:23:05.493 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-23T14:23:05.494 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-23T14:23:05.495 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-23T14:23:05.496 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-23T14:23:05.497 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-23T14:23:05.498 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-23T14:23:05.499 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-23T14:23:05.500 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-23T14:23:05.501 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-23T14:23:05.502 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-23T14:23:05.503 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-23T14:23:05.504 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-23T14:23:05.505 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-23T14:23:05.506 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-23T14:23:05.507 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-23T14:23:05.508 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-23T14:23:05.509 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-23T14:23:05.510 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-23T14:23:05.511 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-23T14:23:05.512 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-23T14:23:05.513 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-23T14:23:05.514 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-23T14:23:05.515 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-23T14:23:05.516 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-23T14:23:05.517 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-23T14:23:05.518 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-23T14:23:05.519 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-23T14:23:05.520 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-23T14:23:05.521 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-23T14:23:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-23T14:23:05.523 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-23T14:23:05.524 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-23T14:23:05.525 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-23T14:23:05.526 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-23T14:23:05.527 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-23T14:23:05.528 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-23T14:23:05.529 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-23T14:23:05.530 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-23T14:23:05.531 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-23T14:23:05.532 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-23T14:23:05.533 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-23T14:23:05.534 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-23T14:23:05.535 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-23T14:23:05.536 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-23T14:23:05.537 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-23T14:23:05.538 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-23T14:23:05.539 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-23T14:23:05.540 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-23T14:23:05.541 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-23T14:23:05.542 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-23T14:23:05.543 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-23T14:23:05.544 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-23T14:23:05.545 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-23T14:23:05.546 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-23T14:23:05.546 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-23T14:23:05.546 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-23T14:23:05.546 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-23T14:23:05.546 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-23T14:23:05.546 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-23T14:23:05.546 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-23T14:23:05.546 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-23T14:23:05.546 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-23T14:23:05.546 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-23T14:23:05.546 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-23T14:23:05.546 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-23T14:23:05.547 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-23T14:23:05.548 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-23T14:23:05.549 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-23T14:23:05.550 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-23T14:23:05.551 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-23T14:23:05.552 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-23T14:23:05.553 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-23T14:23:05.554 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-23T14:23:05.555 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-23T14:23:05.556 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-23T14:23:05.557 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-23T14:23:05.558 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-23T14:23:05.559 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-23T14:23:05.561 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-23T14:23:05.561 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-23T14:23:05.561 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-23T14:23:05.561 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-23T14:23:05.562 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-23T14:23:05.563 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-23T14:23:05.564 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-23T14:23:05.565 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-23T14:23:05.566 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-23T14:23:05.567 INFO:teuthology.orchestra.run.smithi083.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-23T14:23:05.572 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-23T14:23:05.572 DEBUG:teuthology.orchestra.run.smithi083:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-23T14:23:05.595 INFO:teuthology.orchestra.run.smithi083.stdout:postinstall scriptlet (using /bin/sh): 2024-04-23T14:23:05.595 INFO:teuthology.orchestra.run.smithi083.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-23T14:23:05.595 INFO:teuthology.orchestra.run.smithi083.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-23T14:23:05.595 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-04-23T14:23:05.595 INFO:teuthology.orchestra.run.smithi083.stdout:for file in vmlinuz System.map config; do 2024-04-23T14:23:05.595 INFO:teuthology.orchestra.run.smithi083.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-23T14:23:05.595 INFO:teuthology.orchestra.run.smithi083.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-23T14:23:05.595 INFO:teuthology.orchestra.run.smithi083.stdout: fi 2024-04-23T14:23:05.595 INFO:teuthology.orchestra.run.smithi083.stdout:done 2024-04-23T14:23:05.595 INFO:teuthology.orchestra.run.smithi083.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-23T14:23:05.596 INFO:teuthology.orchestra.run.smithi083.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-23T14:23:05.596 INFO:teuthology.orchestra.run.smithi083.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-23T14:23:05.596 INFO:teuthology.orchestra.run.smithi083.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-23T14:23:05.596 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-04-23T14:23:05.596 INFO:teuthology.orchestra.run.smithi083.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-23T14:23:05.596 INFO:teuthology.orchestra.run.smithi083.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-23T14:23:05.596 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-04-23T14:23:05.596 INFO:teuthology.orchestra.run.smithi083.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-23T14:23:05.596 INFO:teuthology.orchestra.run.smithi083.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-23T14:23:05.596 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-23T14:23:05.596 INFO:teuthology.orchestra.run.smithi083.stdout:fi 2024-04-23T14:23:05.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo rpm -qi grub2-tools 2024-04-23T14:23:05.677 INFO:teuthology.orchestra.run.smithi083.stdout:Name : grub2-tools 2024-04-23T14:23:05.678 INFO:teuthology.orchestra.run.smithi083.stdout:Epoch : 1 2024-04-23T14:23:05.678 INFO:teuthology.orchestra.run.smithi083.stdout:Version : 2.06 2024-04-23T14:23:05.678 INFO:teuthology.orchestra.run.smithi083.stdout:Release : 68.el9 2024-04-23T14:23:05.678 INFO:teuthology.orchestra.run.smithi083.stdout:Architecture: x86_64 2024-04-23T14:23:05.678 INFO:teuthology.orchestra.run.smithi083.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-23T14:23:05.678 INFO:teuthology.orchestra.run.smithi083.stdout:Group : Unspecified 2024-04-23T14:23:05.678 INFO:teuthology.orchestra.run.smithi083.stdout:Size : 8264254 2024-04-23T14:23:05.678 INFO:teuthology.orchestra.run.smithi083.stdout:License : GPLv3+ 2024-04-23T14:23:05.678 INFO:teuthology.orchestra.run.smithi083.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-23T14:23:05.678 INFO:teuthology.orchestra.run.smithi083.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-23T14:23:05.678 INFO:teuthology.orchestra.run.smithi083.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-23T14:23:05.679 INFO:teuthology.orchestra.run.smithi083.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-23T14:23:05.679 INFO:teuthology.orchestra.run.smithi083.stdout:Packager : builder@centos.org 2024-04-23T14:23:05.679 INFO:teuthology.orchestra.run.smithi083.stdout:Vendor : CentOS 2024-04-23T14:23:05.679 INFO:teuthology.orchestra.run.smithi083.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-23T14:23:05.679 INFO:teuthology.orchestra.run.smithi083.stdout:Summary : Support tools for GRUB. 2024-04-23T14:23:05.679 INFO:teuthology.orchestra.run.smithi083.stdout:Description : 2024-04-23T14:23:05.679 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:23:05.679 INFO:teuthology.orchestra.run.smithi083.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-23T14:23:05.679 INFO:teuthology.orchestra.run.smithi083.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-23T14:23:05.679 INFO:teuthology.orchestra.run.smithi083.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-23T14:23:05.679 INFO:teuthology.orchestra.run.smithi083.stdout:hardware devices. 2024-04-23T14:23:05.679 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:23:05.680 INFO:teuthology.orchestra.run.smithi083.stdout:This subpackage provides tools for support of all platforms. 2024-04-23T14:23:05.681 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-23T14:23:05.681 INFO:teuthology.task.kernel:Updating grub on smithi083 to boot 6.9.0-rc3-g806ec36b2112 2024-04-23T14:23:05.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-23T14:23:06.233 INFO:teuthology.orchestra.run.smithi083.stderr:Generating grub configuration file ... 2024-04-23T14:23:07.659 INFO:teuthology.orchestra.run.smithi083.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-23T14:23:07.693 INFO:teuthology.orchestra.run.smithi083.stderr:done 2024-04-23T14:23:07.696 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-04-23T14:23:07.713 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.iMjq41xh7Y 2024-04-23T14:23:07.714 DEBUG:teuthology.orchestra.run.smithi083:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.iMjq41xh7Y 2024-04-23T14:23:07.779 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.iMjq41xh7Y 2024-04-23T14:23:07.919 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.iMjq41xh7Y is 6KB 2024-04-23T14:23:07.969 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.iMjq41xh7Y 2024-04-23T14:23:07.984 DEBUG:teuthology.orchestra.run.smithi083:> sudo /bin/ls /boot/loader/entries || true 2024-04-23T14:23:08.047 INFO:teuthology.orchestra.run.smithi083.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-23T14:23:08.047 INFO:teuthology.orchestra.run.smithi083.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-23T14:23:08.047 INFO:teuthology.orchestra.run.smithi083.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-23T14:23:08.047 INFO:teuthology.orchestra.run.smithi083.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-23T14:23:08.049 DEBUG:teuthology.orchestra.run.smithi083:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-23T14:23:08.201 DEBUG:teuthology.orchestra.run.smithi083:> sudo shutdown -r now 2024-04-23T14:23:38.204 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-23T14:23:38.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-23T14:23:38.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:23:56.703 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-23T14:24:05.704 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-23T14:24:05.705 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:24:08.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-23T14:24:20.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-23T14:24:20.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:24:39.202 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.83 2024-04-23T14:24:54.203 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-23T14:24:54.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:24:54.624 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-23T14:24:55.268 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-23T14:24:55.268 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-23T14:24:55.269 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-23T14:24:55.306 INFO:teuthology.orchestra.run.smithi083.stdout:6.9.0-rc3-g806ec36b2112 2024-04-23T14:24:55.306 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-23T14:24:55.306 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-23T14:24:55.306 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-23T14:24:56.307 DEBUG:teuthology.parallel:result is None 2024-04-23T14:24:56.307 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T14:24:56.314 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T14:24:56.315 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T14:24:56.322 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T14:24:56.324 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T14:24:56.361 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T14:24:56.369 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T14:24:56.379 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T14:24:56.380 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T14:24:56.407 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T14:24:56.410 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T14:24:56.534 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T14:24:56.542 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T14:24:56.545 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T14:24:56.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:24:56.562 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T14:24:56.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:24:56.579 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T14:24:56.585 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T14:24:56.604 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T14:24:56.640 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:24:56.660 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:24:56.664 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:24:56.688 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:24:56.717 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:24:56.768 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T14:24:56.773 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T14:24:56.782 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T14:24:56.783 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T14:24:56.787 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T14:24:56.789 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T14:24:56.857 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T14:24:56.867 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T14:24:56.867 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T14:24:56.872 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T14:24:56.879 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T14:24:56.913 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T14:24:57.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T14:24:57.095 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T14:24:57.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T14:24:57.253 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:24:57.253 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T14:24:57.342 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T14:24:57.389 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T14:24:57.443 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T14:24:57.531 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T14:24:57.575 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T14:24:57.575 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T14:24:57.641 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T14:24:57.686 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T14:24:57.749 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T14:24:57.791 DEBUG:teuthology.orchestra.run.smithi203:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T14:24:57.835 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-23T14:24:57.835 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T14:24:57.902 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-04-23T14:24:57.910 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2024-04-23T14:24:57.913 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-04-23T14:24:57.945 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T14:24:57.989 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T14:24:58.029 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T14:24:58.388 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T14:24:58.396 INFO:teuthology.task.internal:Starting timer... 2024-04-23T14:24:58.396 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T14:24:58.408 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T14:24:58.418 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-23T14:24:58.419 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-04-23T14:24:58.513 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-04-23T14:24:58.518 DEBUG:teuthology.orchestra.run.smithi173:> sudo service auditd rotate 2024-04-23T14:24:58.577 INFO:teuthology.orchestra.run.smithi173.stdout:Rotating logs: 2024-04-23T14:24:58.579 DEBUG:teuthology.orchestra.run.smithi203:> sudo service auditd rotate 2024-04-23T14:24:58.640 INFO:teuthology.orchestra.run.smithi203.stdout:Rotating logs: 2024-04-23T14:24:58.642 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T14:24:58.642 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-04-23T14:24:58.694 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-04-23T14:24:58.695 DEBUG:teuthology.orchestra.run.smithi173:> /usr/sbin/getenforce 2024-04-23T14:24:58.719 INFO:teuthology.orchestra.run.smithi173.stdout:Permissive 2024-04-23T14:24:58.719 DEBUG:teuthology.orchestra.run.smithi203:> /usr/sbin/getenforce 2024-04-23T14:24:58.755 INFO:teuthology.orchestra.run.smithi203.stdout:Permissive 2024-04-23T14:24:58.755 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi083.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi173.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi203.front.sepia.ceph.com': 'permissive'} 2024-04-23T14:24:58.755 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T14:24:58.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:24:58.841 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T14:24:58.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:24:58.870 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T14:24:58.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:24:58.895 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T14:24:58.897 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-04-23T14:24:59.116 DEBUG:teuthology.orchestra.run.smithi173:> sudo /usr/sbin/setenforce permissive 2024-04-23T14:24:59.146 DEBUG:teuthology.orchestra.run.smithi203:> sudo /usr/sbin/setenforce permissive 2024-04-23T14:24:59.171 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T14:24:59.182 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T14:24:59.210 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T14:24:59.212 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi083.front.sepia.ceph.com,smithi173.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T14:30:13.997 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi083.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-04-23T14:30:13.999 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-23T14:30:14.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:30:14.114 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-23T14:30:14.266 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-23T14:30:14.266 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-23T14:30:14.267 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:30:14.341 DEBUG:teuthology.orchestra.run.smithi173:> true 2024-04-23T14:30:14.422 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2024-04-23T14:30:14.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-23T14:30:14.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T14:30:14.504 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-04-23T14:30:14.587 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-23T14:30:14.587 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T14:30:14.603 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T14:30:14.603 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T14:30:14.604 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T14:30:14.612 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T14:30:14.612 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T14:30:14.614 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T14:30:14.615 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T14:30:14.641 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T14:30:14.654 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T14:30:14.657 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T14:30:14.672 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T14:30:14.678 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: ntpd: command not found 2024-04-23T14:30:14.689 INFO:teuthology.orchestra.run.smithi173.stdout:506 Cannot talk to daemon 2024-04-23T14:30:14.695 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: ntpd: command not found 2024-04-23T14:30:14.698 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T14:30:14.702 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T14:30:14.706 INFO:teuthology.orchestra.run.smithi203.stdout:506 Cannot talk to daemon 2024-04-23T14:30:14.715 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T14:30:14.720 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T14:30:14.733 INFO:teuthology.orchestra.run.smithi203.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T14:30:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T14:30:14.768 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-04-23T14:30:14.772 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T14:30:14.772 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-04-23T14:30:14.772 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:30:14.772 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:30:14.772 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:30:14.772 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:30:14.787 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-04-23T14:30:14.791 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T14:30:14.791 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-04-23T14:30:14.791 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:30:14.791 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:30:14.791 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:30:14.791 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:30:14.815 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-04-23T14:30:14.844 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-04-23T14:30:14.886 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T14:30:14.929 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T14:30:15.076 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-23T14:30:15.084 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T14:30:15.084 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-23T14:30:15.084 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:30:15.084 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:30:15.085 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:30:15.085 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T14:30:15.086 INFO:teuthology.run_tasks:Running task install... 2024-04-23T14:30:15.095 DEBUG:teuthology.task.install:project ceph 2024-04-23T14:30:15.095 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'}} 2024-04-23T14:30:15.095 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'} 2024-04-23T14:30:15.095 INFO:teuthology.task.install:Using flavor: default 2024-04-23T14:30:15.101 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T14:30:15.102 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-23T14:30:15.102 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'tag': None, 'wait_for_package': False} 2024-04-23T14:30:15.103 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:30:15.105 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'tag': None, 'wait_for_package': False} 2024-04-23T14:30:15.105 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:30:15.106 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'tag': None, 'wait_for_package': False} 2024-04-23T14:30:15.106 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:30:15.269 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/ 2024-04-23T14:30:15.269 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3126.gd44fddd9 2024-04-23T14:30:15.276 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/ 2024-04-23T14:30:15.276 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3126.gd44fddd9 2024-04-23T14:30:15.381 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T14:30:15.382 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-23T14:30:15.382 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T14:30:15.398 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/ 2024-04-23T14:30:15.399 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3126.gd44fddd9 2024-04-23T14:30:15.409 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T14:30:15.409 DEBUG:teuthology.orchestra.run.smithi203:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d44fddd910b5e94da0f2357af4be37d7bd495ae0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T14:30:15.475 DEBUG:teuthology.orchestra.run.smithi203:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T14:30:15.509 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T14:30:15.509 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T14:30:15.509 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T14:30:15.536 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T14:30:15.536 DEBUG:teuthology.orchestra.run.smithi173:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d44fddd910b5e94da0f2357af4be37d7bd495ae0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T14:30:15.551 DEBUG:teuthology.orchestra.run.smithi203:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T14:30:15.602 DEBUG:teuthology.orchestra.run.smithi173:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T14:30:15.615 INFO:teuthology.orchestra.run.smithi203.stdout:check_obsoletes = 1 2024-04-23T14:30:15.616 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum clean all 2024-04-23T14:30:15.638 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T14:30:15.638 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:30:15.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T14:30:15.676 DEBUG:teuthology.orchestra.run.smithi173:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T14:30:15.722 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T14:30:15.723 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d44fddd910b5e94da0f2357af4be37d7bd495ae0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T14:30:15.740 INFO:teuthology.orchestra.run.smithi173.stdout:check_obsoletes = 1 2024-04-23T14:30:15.742 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum clean all 2024-04-23T14:30:15.845 INFO:teuthology.orchestra.run.smithi203.stdout:48 files removed 2024-04-23T14:30:15.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T14:30:15.872 DEBUG:teuthology.orchestra.run.smithi203:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T14:30:15.967 INFO:teuthology.orchestra.run.smithi173.stdout:48 files removed 2024-04-23T14:30:15.968 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T14:30:15.994 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T14:30:16.046 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-04-23T14:30:16.050 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-04-23T14:30:16.372 INFO:teuthology.orchestra.run.smithi083.stdout:48 files removed 2024-04-23T14:30:16.411 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T14:30:16.741 INFO:teuthology.orchestra.run.smithi203.stdout:ceph packages for x86_64 137 kB/s | 83 kB 00:00 2024-04-23T14:30:16.789 INFO:teuthology.orchestra.run.smithi173.stdout:ceph packages for x86_64 156 kB/s | 83 kB 00:00 2024-04-23T14:30:17.158 INFO:teuthology.orchestra.run.smithi203.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-23T14:30:17.198 INFO:teuthology.orchestra.run.smithi173.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-04-23T14:30:17.502 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 127 kB/s | 83 kB 00:00 2024-04-23T14:30:17.567 INFO:teuthology.orchestra.run.smithi203.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-23T14:30:17.682 INFO:teuthology.orchestra.run.smithi173.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-04-23T14:30:18.095 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-04-23T14:30:18.326 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-23T14:30:18.490 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-23T14:30:18.887 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-04-23T14:30:20.543 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-04-23T14:30:22.452 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - AppStream 6.2 MB/s | 19 MB 00:03 2024-04-23T14:30:22.466 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 19 MB 00:02 2024-04-23T14:30:24.511 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 6.9 MB/s | 19 MB 00:02 2024-04-23T14:30:26.594 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-23T14:30:27.142 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.2 MB 00:01 2024-04-23T14:30:28.228 INFO:teuthology.orchestra.run.smithi203.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-23T14:30:28.626 INFO:teuthology.orchestra.run.smithi173.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 16 kB 00:00 2024-04-23T14:30:29.404 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.2 MB 00:01 2024-04-23T14:30:29.974 INFO:teuthology.orchestra.run.smithi203.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-23T14:30:31.436 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 21 MB 00:02 2024-04-23T14:30:31.875 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-23T14:30:33.704 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-23T14:30:35.122 INFO:teuthology.orchestra.run.smithi203.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-23T14:30:36.486 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-04-23T14:30:37.240 INFO:teuthology.orchestra.run.smithi203.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T14:30:37.241 INFO:teuthology.orchestra.run.smithi203.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T14:30:37.252 INFO:teuthology.orchestra.run.smithi203.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T14:30:37.253 INFO:teuthology.orchestra.run.smithi203.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T14:30:37.254 INFO:teuthology.orchestra.run.smithi203.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T14:30:37.254 INFO:teuthology.orchestra.run.smithi203.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T14:30:37.256 INFO:teuthology.orchestra.run.smithi203.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T14:30:37.258 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T14:30:37.259 INFO:teuthology.orchestra.run.smithi203.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T14:30:37.321 INFO:teuthology.orchestra.run.smithi203.stdout:Dependencies resolved. 2024-04-23T14:30:37.347 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-04-23T14:30:37.348 INFO:teuthology.orchestra.run.smithi203.stdout: Package Arch Version Repository Size 2024-04-23T14:30:37.348 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-04-23T14:30:37.348 INFO:teuthology.orchestra.run.smithi203.stdout:Installing: 2024-04-23T14:30:37.348 INFO:teuthology.orchestra.run.smithi203.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T14:30:37.348 INFO:teuthology.orchestra.run.smithi203.stdout: ceph x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 6.6 k 2024-04-23T14:30:37.348 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 5.4 M 2024-04-23T14:30:37.348 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 891 k 2024-04-23T14:30:37.348 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 145 k 2024-04-23T14:30:37.348 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 1.5 M 2024-04-23T14:30:37.348 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 142 k 2024-04-23T14:30:37.348 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 3.5 M 2024-04-23T14:30:37.349 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 7.4 M 2024-04-23T14:30:37.349 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 49 k 2024-04-23T14:30:37.349 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 11 M 2024-04-23T14:30:37.349 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 49 M 2024-04-23T14:30:37.349 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 273 k 2024-04-23T14:30:37.349 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 762 k 2024-04-23T14:30:37.349 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 225 k 2024-04-23T14:30:37.349 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 27 k 2024-04-23T14:30:37.349 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 18 k 2024-04-23T14:30:37.349 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T14:30:37.349 INFO:teuthology.orchestra.run.smithi203.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T14:30:37.349 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 32 k 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 746 k 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 127 k 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 161 k 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 320 k 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 297 k 2024-04-23T14:30:37.350 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 100 k 2024-04-23T14:30:37.351 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 85 k 2024-04-23T14:30:37.351 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.1 M 2024-04-23T14:30:37.351 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 171 k 2024-04-23T14:30:37.351 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T14:30:37.351 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrading: 2024-04-23T14:30:37.351 INFO:teuthology.orchestra.run.smithi203.stdout: librados2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.4 M 2024-04-23T14:30:37.351 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.1 M 2024-04-23T14:30:37.351 INFO:teuthology.orchestra.run.smithi203.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T14:30:37.351 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T14:30:37.351 INFO:teuthology.orchestra.run.smithi203.stdout:Installing dependencies: 2024-04-23T14:30:37.351 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T14:30:37.351 INFO:teuthology.orchestra.run.smithi203.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 21 M 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 34 k 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 2.4 M 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 249 k 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 4.7 M 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 17 M 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 16 k 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 25 k 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T14:30:37.352 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T14:30:37.353 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T14:30:37.353 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T14:30:37.353 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T14:30:37.353 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 163 k 2024-04-23T14:30:37.353 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T14:30:37.353 INFO:teuthology.orchestra.run.smithi203.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T14:30:37.353 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T14:30:37.353 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T14:30:37.353 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T14:30:37.353 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 500 k 2024-04-23T14:30:37.353 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T14:30:37.353 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 5.4 M 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T14:30:37.354 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T14:30:37.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T14:30:37.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T14:30:37.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T14:30:37.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T14:30:37.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 45 k 2024-04-23T14:30:37.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 135 k 2024-04-23T14:30:37.355 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T14:30:37.356 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T14:30:37.357 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T14:30:37.358 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T14:30:37.359 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout:Installing weak dependencies: 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction Summary 2024-04-23T14:30:37.360 INFO:teuthology.orchestra.run.smithi203.stdout:======================================================================================= 2024-04-23T14:30:37.361 INFO:teuthology.orchestra.run.smithi203.stdout:Install 148 Packages 2024-04-23T14:30:37.361 INFO:teuthology.orchestra.run.smithi203.stdout:Upgrade 4 Packages 2024-04-23T14:30:37.361 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:30:37.361 INFO:teuthology.orchestra.run.smithi203.stdout:Total download size: 222 M 2024-04-23T14:30:37.361 INFO:teuthology.orchestra.run.smithi203.stdout:Downloading Packages: 2024-04-23T14:30:38.595 INFO:teuthology.orchestra.run.smithi173.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T14:30:38.596 INFO:teuthology.orchestra.run.smithi173.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T14:30:38.607 INFO:teuthology.orchestra.run.smithi173.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T14:30:38.608 INFO:teuthology.orchestra.run.smithi173.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T14:30:38.609 INFO:teuthology.orchestra.run.smithi173.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T14:30:38.610 INFO:teuthology.orchestra.run.smithi173.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T14:30:38.611 INFO:teuthology.orchestra.run.smithi173.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T14:30:38.613 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T14:30:38.614 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T14:30:38.674 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2024-04-23T14:30:38.692 INFO:teuthology.orchestra.run.smithi173.stdout:======================================================================================= 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout:======================================================================================= 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout: ceph x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 6.6 k 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 5.4 M 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 891 k 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 145 k 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 1.5 M 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 142 k 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 3.5 M 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 7.4 M 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 49 k 2024-04-23T14:30:38.693 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 11 M 2024-04-23T14:30:38.694 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 49 M 2024-04-23T14:30:38.694 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 273 k 2024-04-23T14:30:38.694 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 762 k 2024-04-23T14:30:38.694 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 225 k 2024-04-23T14:30:38.694 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-shell x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 27 k 2024-04-23T14:30:38.694 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 18 k 2024-04-23T14:30:38.694 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T14:30:38.694 INFO:teuthology.orchestra.run.smithi173.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T14:30:38.694 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T14:30:38.694 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 32 k 2024-04-23T14:30:38.694 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 746 k 2024-04-23T14:30:38.694 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 127 k 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 161 k 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 320 k 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 297 k 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 100 k 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 85 k 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.1 M 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 171 k 2024-04-23T14:30:38.695 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrading: 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout: librados2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.4 M 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.1 M 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 21 M 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 34 k 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 2.4 M 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 249 k 2024-04-23T14:30:38.696 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 4.7 M 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 17 M 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 16 k 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 25 k 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 163 k 2024-04-23T14:30:38.697 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 500 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 5.4 M 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T14:30:38.698 INFO:teuthology.orchestra.run.smithi173.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 45 k 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 135 k 2024-04-23T14:30:38.699 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T14:30:38.700 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T14:30:38.701 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T14:30:38.702 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T14:30:38.703 INFO:teuthology.orchestra.run.smithi173.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout:Installing weak dependencies: 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout:======================================================================================= 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout:Install 148 Packages 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrade 4 Packages 2024-04-23T14:30:38.704 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:30:38.705 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 222 M 2024-04-23T14:30:38.705 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2024-04-23T14:30:38.924 INFO:teuthology.orchestra.run.smithi203.stdout:(1/152): ceph-19.0.0-3126.gd44fddd9.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-23T14:30:39.639 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-04-23T14:30:39.887 INFO:teuthology.orchestra.run.smithi173.stdout:(1/152): ceph-19.0.0-3126.gd44fddd9.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-23T14:30:40.541 INFO:teuthology.orchestra.run.smithi203.stdout:(2/152): ceph-fuse-19.0.0-3126.gd44fddd9.el9.x8 551 kB/s | 891 kB 00:01 2024-04-23T14:30:40.882 INFO:teuthology.orchestra.run.smithi203.stdout:(3/152): ceph-immutable-object-cache-19.0.0-312 424 kB/s | 145 kB 00:00 2024-04-23T14:30:41.429 INFO:teuthology.orchestra.run.smithi173.stdout:(2/152): ceph-fuse-19.0.0-3126.gd44fddd9.el9.x8 578 kB/s | 891 kB 00:01 2024-04-23T14:30:41.662 INFO:teuthology.orchestra.run.smithi173.stdout:(3/152): ceph-immutable-object-cache-19.0.0-312 622 kB/s | 145 kB 00:00 2024-04-23T14:30:42.011 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T14:30:42.012 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T14:30:42.029 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T14:30:42.030 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T14:30:42.031 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T14:30:42.032 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T14:30:42.034 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T14:30:42.038 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T14:30:42.039 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T14:30:42.110 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-23T14:30:42.127 INFO:teuthology.orchestra.run.smithi173.stdout:(4/152): ceph-base-19.0.0-3126.gd44fddd9.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-04-23T14:30:42.135 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-23T14:30:42.135 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-04-23T14:30:42.135 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-23T14:30:42.135 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 6.6 k 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 5.4 M 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 891 k 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 145 k 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 1.5 M 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 142 k 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 3.5 M 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 7.4 M 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 49 k 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 11 M 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 49 M 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 273 k 2024-04-23T14:30:42.136 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 762 k 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 225 k 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 27 k 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 18 k 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 32 k 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 746 k 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 127 k 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T14:30:42.137 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 161 k 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 320 k 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 297 k 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 100 k 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 85 k 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.1 M 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 171 k 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.4 M 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 3.1 M 2024-04-23T14:30:42.138 INFO:teuthology.orchestra.run.smithi083.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 21 M 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 34 k 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 2.4 M 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 249 k 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 4.7 M 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 17 M 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3126.gd44fddd9.el9 ceph-noarch 16 k 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 25 k 2024-04-23T14:30:42.139 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 163 k 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T14:30:42.140 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 500 k 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 5.4 M 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T14:30:42.141 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 45 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.0.0-3126.gd44fddd9.el9 ceph 135 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T14:30:42.142 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T14:30:42.143 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T14:30:42.144 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T14:30:42.145 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T14:30:42.146 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-04-23T14:30:42.147 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T14:30:42.147 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:30:42.147 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-04-23T14:30:42.147 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-23T14:30:42.147 INFO:teuthology.orchestra.run.smithi083.stdout:Install 148 Packages 2024-04-23T14:30:42.147 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 4 Packages 2024-04-23T14:30:42.147 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:30:42.147 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 222 M 2024-04-23T14:30:42.148 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-04-23T14:30:42.405 INFO:teuthology.orchestra.run.smithi173.stdout:(5/152): ceph-mgr-19.0.0-3126.gd44fddd9.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-04-23T14:30:43.263 INFO:teuthology.orchestra.run.smithi173.stdout:(6/152): ceph-mds-19.0.0-3126.gd44fddd9.el9.x86 1.5 MB/s | 2.4 MB 00:01 2024-04-23T14:30:43.472 INFO:teuthology.orchestra.run.smithi173.stdout:(7/152): ceph-mon-19.0.0-3126.gd44fddd9.el9.x86 4.4 MB/s | 4.7 MB 00:01 2024-04-23T14:30:43.492 INFO:teuthology.orchestra.run.smithi083.stdout:(1/152): ceph-19.0.0-3126.gd44fddd9.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-23T14:30:43.708 INFO:teuthology.orchestra.run.smithi203.stdout:(4/152): ceph-common-19.0.0-3126.gd44fddd9.el9. 4.0 MB/s | 21 MB 00:05 2024-04-23T14:30:44.049 INFO:teuthology.orchestra.run.smithi203.stdout:(5/152): ceph-mds-19.0.0-3126.gd44fddd9.el9.x86 763 kB/s | 2.4 MB 00:03 2024-04-23T14:30:44.091 INFO:teuthology.orchestra.run.smithi203.stdout:(6/152): ceph-mgr-19.0.0-3126.gd44fddd9.el9.x86 3.9 MB/s | 1.5 MB 00:00 2024-04-23T14:30:44.125 INFO:teuthology.orchestra.run.smithi083.stdout:(2/152): ceph-fuse-19.0.0-3126.gd44fddd9.el9.x8 1.4 MB/s | 891 kB 00:00 2024-04-23T14:30:44.250 INFO:teuthology.orchestra.run.smithi083.stdout:(3/152): ceph-immutable-object-cache-19.0.0-312 1.1 MB/s | 145 kB 00:00 2024-04-23T14:30:44.675 INFO:teuthology.orchestra.run.smithi203.stdout:(7/152): ceph-base-19.0.0-3126.gd44fddd9.el9.x8 880 kB/s | 5.4 MB 00:06 2024-04-23T14:30:44.939 INFO:teuthology.orchestra.run.smithi173.stdout:(8/152): ceph-radosgw-19.0.0-3126.gd44fddd9.el9 7.2 MB/s | 11 MB 00:01 2024-04-23T14:30:45.014 INFO:teuthology.orchestra.run.smithi173.stdout:(9/152): ceph-selinux-19.0.0-3126.gd44fddd9.el9 338 kB/s | 25 kB 00:00 2024-04-23T14:30:45.101 INFO:teuthology.orchestra.run.smithi083.stdout:(4/152): ceph-mds-19.0.0-3126.gd44fddd9.el9.x86 2.8 MB/s | 2.4 MB 00:00 2024-04-23T14:30:45.569 INFO:teuthology.orchestra.run.smithi083.stdout:(5/152): ceph-mgr-19.0.0-3126.gd44fddd9.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-04-23T14:30:46.534 INFO:teuthology.orchestra.run.smithi083.stdout:(6/152): ceph-mon-19.0.0-3126.gd44fddd9.el9.x86 4.9 MB/s | 4.7 MB 00:00 2024-04-23T14:30:46.651 INFO:teuthology.orchestra.run.smithi083.stdout:(7/152): ceph-base-19.0.0-3126.gd44fddd9.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-04-23T14:30:47.742 INFO:teuthology.orchestra.run.smithi203.stdout:(8/152): ceph-osd-19.0.0-3126.gd44fddd9.el9.x86 4.6 MB/s | 17 MB 00:03 2024-04-23T14:30:47.817 INFO:teuthology.orchestra.run.smithi203.stdout:(9/152): ceph-selinux-19.0.0-3126.gd44fddd9.el9 337 kB/s | 25 kB 00:00 2024-04-23T14:30:48.867 INFO:teuthology.orchestra.run.smithi203.stdout:(10/152): ceph-mon-19.0.0-3126.gd44fddd9.el9.x8 998 kB/s | 4.7 MB 00:04 2024-04-23T14:30:49.150 INFO:teuthology.orchestra.run.smithi203.stdout:(11/152): cephfs-mirror-19.0.0-3126.gd44fddd9.e 794 kB/s | 225 kB 00:00 2024-04-23T14:30:49.242 INFO:teuthology.orchestra.run.smithi203.stdout:(12/152): cephfs-shell-19.0.0-3126.gd44fddd9.el 295 kB/s | 27 kB 00:00 2024-04-23T14:30:49.318 INFO:teuthology.orchestra.run.smithi083.stdout:(8/152): ceph-osd-19.0.0-3126.gd44fddd9.el9.x86 6.0 MB/s | 17 MB 00:02 2024-04-23T14:30:49.343 INFO:teuthology.orchestra.run.smithi203.stdout:(13/152): libcephfs-devel-19.0.0-3126.gd44fddd9 317 kB/s | 32 kB 00:00 2024-04-23T14:30:49.393 INFO:teuthology.orchestra.run.smithi083.stdout:(9/152): ceph-selinux-19.0.0-3126.gd44fddd9.el9 339 kB/s | 25 kB 00:00 2024-04-23T14:30:50.068 INFO:teuthology.orchestra.run.smithi203.stdout:(14/152): libcephfs2-19.0.0-3126.gd44fddd9.el9. 1.0 MB/s | 746 kB 00:00 2024-04-23T14:30:50.302 INFO:teuthology.orchestra.run.smithi203.stdout:(15/152): libcephsqlite-19.0.0-3126.gd44fddd9.e 700 kB/s | 163 kB 00:00 2024-04-23T14:30:50.427 INFO:teuthology.orchestra.run.smithi203.stdout:(16/152): librados-devel-19.0.0-3126.gd44fddd9. 1.0 MB/s | 127 kB 00:00 2024-04-23T14:30:50.589 INFO:teuthology.orchestra.run.smithi173.stdout:(10/152): ceph-common-19.0.0-3126.gd44fddd9.el9 1.9 MB/s | 21 MB 00:11 2024-04-23T14:30:50.764 INFO:teuthology.orchestra.run.smithi173.stdout:(11/152): cephfs-mirror-19.0.0-3126.gd44fddd9.e 1.3 MB/s | 225 kB 00:00 2024-04-23T14:30:50.847 INFO:teuthology.orchestra.run.smithi173.stdout:(12/152): cephfs-shell-19.0.0-3126.gd44fddd9.el 325 kB/s | 27 kB 00:00 2024-04-23T14:30:50.911 INFO:teuthology.orchestra.run.smithi203.stdout:(17/152): libradosstriper1-19.0.0-3126.gd44fddd 1.0 MB/s | 500 kB 00:00 2024-04-23T14:30:50.931 INFO:teuthology.orchestra.run.smithi173.stdout:(13/152): libcephfs-devel-19.0.0-3126.gd44fddd9 380 kB/s | 32 kB 00:00 2024-04-23T14:30:51.302 INFO:teuthology.orchestra.run.smithi083.stdout:(10/152): ceph-common-19.0.0-3126.gd44fddd9.el9 2.6 MB/s | 21 MB 00:08 2024-04-23T14:30:51.306 INFO:teuthology.orchestra.run.smithi173.stdout:(14/152): libcephfs2-19.0.0-3126.gd44fddd9.el9. 1.9 MB/s | 746 kB 00:00 2024-04-23T14:30:51.465 INFO:teuthology.orchestra.run.smithi173.stdout:(15/152): libcephsqlite-19.0.0-3126.gd44fddd9.e 1.0 MB/s | 163 kB 00:00 2024-04-23T14:30:51.494 INFO:teuthology.orchestra.run.smithi083.stdout:(11/152): cephfs-mirror-19.0.0-3126.gd44fddd9.e 1.2 MB/s | 225 kB 00:00 2024-04-23T14:30:51.548 INFO:teuthology.orchestra.run.smithi173.stdout:(16/152): librados-devel-19.0.0-3126.gd44fddd9. 1.5 MB/s | 127 kB 00:00 2024-04-23T14:30:51.761 INFO:teuthology.orchestra.run.smithi083.stdout:(12/152): ceph-radosgw-19.0.0-3126.gd44fddd9.el 2.1 MB/s | 11 MB 00:05 2024-04-23T14:30:51.767 INFO:teuthology.orchestra.run.smithi173.stdout:(17/152): libradosstriper1-19.0.0-3126.gd44fddd 2.2 MB/s | 500 kB 00:00 2024-04-23T14:30:51.777 INFO:teuthology.orchestra.run.smithi083.stdout:(13/152): cephfs-shell-19.0.0-3126.gd44fddd9.el 95 kB/s | 27 kB 00:00 2024-04-23T14:30:51.878 INFO:teuthology.orchestra.run.smithi083.stdout:(14/152): libcephfs-devel-19.0.0-3126.gd44fddd9 274 kB/s | 32 kB 00:00 2024-04-23T14:30:52.011 INFO:teuthology.orchestra.run.smithi083.stdout:(15/152): libcephsqlite-19.0.0-3126.gd44fddd9.e 1.2 MB/s | 163 kB 00:00 2024-04-23T14:30:52.095 INFO:teuthology.orchestra.run.smithi083.stdout:(16/152): libcephfs2-19.0.0-3126.gd44fddd9.el9. 2.3 MB/s | 746 kB 00:00 2024-04-23T14:30:52.128 INFO:teuthology.orchestra.run.smithi083.stdout:(17/152): librados-devel-19.0.0-3126.gd44fddd9. 1.1 MB/s | 127 kB 00:00 2024-04-23T14:30:52.329 INFO:teuthology.orchestra.run.smithi083.stdout:(18/152): libradosstriper1-19.0.0-3126.gd44fddd 2.1 MB/s | 500 kB 00:00 2024-04-23T14:30:52.454 INFO:teuthology.orchestra.run.smithi083.stdout:(19/152): python3-ceph-argparse-19.0.0-3126.gd4 363 kB/s | 45 kB 00:00 2024-04-23T14:30:52.571 INFO:teuthology.orchestra.run.smithi083.stdout:(20/152): python3-ceph-common-19.0.0-3126.gd44f 1.1 MB/s | 135 kB 00:00 2024-04-23T14:30:52.696 INFO:teuthology.orchestra.run.smithi083.stdout:(21/152): python3-cephfs-19.0.0-3126.gd44fddd9. 1.3 MB/s | 161 kB 00:00 2024-04-23T14:30:52.905 INFO:teuthology.orchestra.run.smithi083.stdout:(22/152): python3-rados-19.0.0-3126.gd44fddd9.e 1.5 MB/s | 320 kB 00:00 2024-04-23T14:30:53.047 INFO:teuthology.orchestra.run.smithi083.stdout:(23/152): python3-rbd-19.0.0-3126.gd44fddd9.el9 2.1 MB/s | 297 kB 00:00 2024-04-23T14:30:53.164 INFO:teuthology.orchestra.run.smithi083.stdout:(24/152): python3-rgw-19.0.0-3126.gd44fddd9.el9 857 kB/s | 100 kB 00:00 2024-04-23T14:30:53.281 INFO:teuthology.orchestra.run.smithi083.stdout:(25/152): rbd-fuse-19.0.0-3126.gd44fddd9.el9.x8 738 kB/s | 85 kB 00:00 2024-04-23T14:30:54.075 INFO:teuthology.orchestra.run.smithi173.stdout:(18/152): librgw2-19.0.0-3126.gd44fddd9.el9.x86 2.3 MB/s | 5.4 MB 00:02 2024-04-23T14:30:54.142 INFO:teuthology.orchestra.run.smithi173.stdout:(19/152): python3-ceph-argparse-19.0.0-3126.gd4 678 kB/s | 45 kB 00:00 2024-04-23T14:30:54.218 INFO:teuthology.orchestra.run.smithi173.stdout:(20/152): python3-ceph-common-19.0.0-3126.gd44f 1.8 MB/s | 135 kB 00:00 2024-04-23T14:30:54.293 INFO:teuthology.orchestra.run.smithi173.stdout:(21/152): python3-cephfs-19.0.0-3126.gd44fddd9. 2.1 MB/s | 161 kB 00:00 2024-04-23T14:30:54.352 INFO:teuthology.orchestra.run.smithi203.stdout:(18/152): ceph-radosgw-19.0.0-3126.gd44fddd9.el 1.1 MB/s | 11 MB 00:09 2024-04-23T14:30:54.418 INFO:teuthology.orchestra.run.smithi173.stdout:(22/152): python3-rados-19.0.0-3126.gd44fddd9.e 2.5 MB/s | 320 kB 00:00 2024-04-23T14:30:54.502 INFO:teuthology.orchestra.run.smithi203.stdout:(19/152): python3-ceph-argparse-19.0.0-3126.gd4 301 kB/s | 45 kB 00:00 2024-04-23T14:30:54.552 INFO:teuthology.orchestra.run.smithi173.stdout:(23/152): python3-rbd-19.0.0-3126.gd44fddd9.el9 2.2 MB/s | 297 kB 00:00 2024-04-23T14:30:54.635 INFO:teuthology.orchestra.run.smithi173.stdout:(24/152): python3-rgw-19.0.0-3126.gd44fddd9.el9 1.2 MB/s | 100 kB 00:00 2024-04-23T14:30:54.698 INFO:teuthology.orchestra.run.smithi083.stdout:(26/152): rbd-mirror-19.0.0-3126.gd44fddd9.el9. 2.2 MB/s | 3.1 MB 00:01 2024-04-23T14:30:54.711 INFO:teuthology.orchestra.run.smithi173.stdout:(25/152): rbd-fuse-19.0.0-3126.gd44fddd9.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-23T14:30:54.778 INFO:teuthology.orchestra.run.smithi203.stdout:(20/152): python3-ceph-common-19.0.0-3126.gd44f 490 kB/s | 135 kB 00:00 2024-04-23T14:30:54.873 INFO:teuthology.orchestra.run.smithi083.stdout:(27/152): rbd-nbd-19.0.0-3126.gd44fddd9.el9.x86 983 kB/s | 171 kB 00:00 2024-04-23T14:30:54.990 INFO:teuthology.orchestra.run.smithi083.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3126.g 294 kB/s | 34 kB 00:00 2024-04-23T14:30:55.078 INFO:teuthology.orchestra.run.smithi203.stdout:(21/152): python3-cephfs-19.0.0-3126.gd44fddd9. 538 kB/s | 161 kB 00:00 2024-04-23T14:30:55.115 INFO:teuthology.orchestra.run.smithi083.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3126.gd44fddd 1.1 MB/s | 142 kB 00:00 2024-04-23T14:30:55.670 INFO:teuthology.orchestra.run.smithi203.stdout:(22/152): librgw2-19.0.0-3126.gd44fddd9.el9.x86 1.1 MB/s | 5.4 MB 00:04 2024-04-23T14:30:55.695 INFO:teuthology.orchestra.run.smithi203.stdout:(23/152): python3-rados-19.0.0-3126.gd44fddd9.e 519 kB/s | 320 kB 00:00 2024-04-23T14:30:55.904 INFO:teuthology.orchestra.run.smithi203.stdout:(24/152): python3-rgw-19.0.0-3126.gd44fddd9.el9 479 kB/s | 100 kB 00:00 2024-04-23T14:30:55.971 INFO:teuthology.orchestra.run.smithi203.stdout:(25/152): python3-rbd-19.0.0-3126.gd44fddd9.el9 989 kB/s | 297 kB 00:00 2024-04-23T14:30:56.046 INFO:teuthology.orchestra.run.smithi203.stdout:(26/152): rbd-fuse-19.0.0-3126.gd44fddd9.el9.x8 602 kB/s | 85 kB 00:00 2024-04-23T14:30:56.244 INFO:teuthology.orchestra.run.smithi173.stdout:(26/152): ceph-test-19.0.0-3126.gd44fddd9.el9.x 4.4 MB/s | 49 MB 00:11 2024-04-23T14:30:56.272 INFO:teuthology.orchestra.run.smithi203.stdout:(27/152): rbd-nbd-19.0.0-3126.gd44fddd9.el9.x86 760 kB/s | 171 kB 00:00 2024-04-23T14:30:56.294 INFO:teuthology.orchestra.run.smithi173.stdout:(27/152): rbd-mirror-19.0.0-3126.gd44fddd9.el9. 2.0 MB/s | 3.1 MB 00:01 2024-04-23T14:30:56.344 INFO:teuthology.orchestra.run.smithi173.stdout:(28/152): rbd-nbd-19.0.0-3126.gd44fddd9.el9.x86 1.7 MB/s | 171 kB 00:00 2024-04-23T14:30:56.370 INFO:teuthology.orchestra.run.smithi173.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3126.g 455 kB/s | 34 kB 00:00 2024-04-23T14:30:56.403 INFO:teuthology.orchestra.run.smithi173.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3126.gd44fddd 2.4 MB/s | 142 kB 00:00 2024-04-23T14:30:56.439 INFO:teuthology.orchestra.run.smithi203.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3126.g 205 kB/s | 34 kB 00:00 2024-04-23T14:30:56.656 INFO:teuthology.orchestra.run.smithi203.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3126.gd44fddd 657 kB/s | 142 kB 00:00 2024-04-23T14:30:56.841 INFO:teuthology.orchestra.run.smithi083.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3126.gd44fd 2.1 MB/s | 3.5 MB 00:01 2024-04-23T14:30:57.112 INFO:teuthology.orchestra.run.smithi173.stdout:(31/152): ceph-osd-19.0.0-3126.gd44fddd9.el9.x8 1.2 MB/s | 17 MB 00:13 2024-04-23T14:30:57.116 INFO:teuthology.orchestra.run.smithi083.stdout:(31/152): librgw2-19.0.0-3126.gd44fddd9.el9.x86 1.1 MB/s | 5.4 MB 00:04 2024-04-23T14:30:57.387 INFO:teuthology.orchestra.run.smithi173.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3126.gd4 906 kB/s | 249 kB 00:00 2024-04-23T14:30:57.416 INFO:teuthology.orchestra.run.smithi083.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3126.gd4 832 kB/s | 249 kB 00:00 2024-04-23T14:30:57.479 INFO:teuthology.orchestra.run.smithi173.stdout:(33/152): ceph-mgr-rook-19.0.0-3126.gd44fddd9.e 539 kB/s | 49 kB 00:00 2024-04-23T14:30:57.516 INFO:teuthology.orchestra.run.smithi083.stdout:(33/152): ceph-mgr-rook-19.0.0-3126.gd44fddd9.e 496 kB/s | 49 kB 00:00 2024-04-23T14:30:57.571 INFO:teuthology.orchestra.run.smithi173.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3126.gd 179 kB/s | 16 kB 00:00 2024-04-23T14:30:57.617 INFO:teuthology.orchestra.run.smithi083.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3126.gd 166 kB/s | 16 kB 00:00 2024-04-23T14:30:57.680 INFO:teuthology.orchestra.run.smithi173.stdout:(35/152): ceph-mgr-dashboard-19.0.0-3126.gd44fd 2.7 MB/s | 3.5 MB 00:01 2024-04-23T14:30:57.963 INFO:teuthology.orchestra.run.smithi173.stdout:(36/152): ceph-volume-19.0.0-3126.gd44fddd9.el9 697 kB/s | 273 kB 00:00 2024-04-23T14:30:57.975 INFO:teuthology.orchestra.run.smithi083.stdout:(35/152): ceph-volume-19.0.0-3126.gd44fddd9.el9 763 kB/s | 273 kB 00:00 2024-04-23T14:30:58.063 INFO:teuthology.orchestra.run.smithi173.stdout:(37/152): cephadm-19.0.0-3126.gd44fddd9.el9.noa 1.9 MB/s | 762 kB 00:00 2024-04-23T14:30:58.088 INFO:teuthology.orchestra.run.smithi173.stdout:(38/152): cephfs-top-19.0.0-3126.gd44fddd9.el9. 146 kB/s | 18 kB 00:00 2024-04-23T14:30:58.205 INFO:teuthology.orchestra.run.smithi173.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-04-23T14:30:58.465 INFO:teuthology.orchestra.run.smithi173.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 755 kB/s | 195 kB 00:00 2024-04-23T14:30:58.498 INFO:teuthology.orchestra.run.smithi173.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-04-23T14:30:58.598 INFO:teuthology.orchestra.run.smithi173.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-04-23T14:30:58.640 INFO:teuthology.orchestra.run.smithi173.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.2 MB 00:00 2024-04-23T14:30:58.665 INFO:teuthology.orchestra.run.smithi173.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-23T14:30:58.690 INFO:teuthology.orchestra.run.smithi173.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-23T14:30:58.741 INFO:teuthology.orchestra.run.smithi173.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-04-23T14:30:58.841 INFO:teuthology.orchestra.run.smithi173.stdout:(47/152): ceph-mgr-diskprediction-local-19.0.0- 3.0 MB/s | 7.4 MB 00:02 2024-04-23T14:30:58.859 INFO:teuthology.orchestra.run.smithi083.stdout:(36/152): cephadm-19.0.0-3126.gd44fddd9.el9.noa 863 kB/s | 762 kB 00:00 2024-04-23T14:30:58.867 INFO:teuthology.orchestra.run.smithi173.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.2 MB/s | 215 kB 00:00 2024-04-23T14:30:58.900 INFO:teuthology.orchestra.run.smithi173.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-04-23T14:30:58.914 INFO:teuthology.orchestra.run.smithi203.stdout:(30/152): rbd-mirror-19.0.0-3126.gd44fddd9.el9. 1.0 MB/s | 3.1 MB 00:02 2024-04-23T14:30:58.925 INFO:teuthology.orchestra.run.smithi173.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-23T14:30:58.942 INFO:teuthology.orchestra.run.smithi173.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 513 kB/s | 21 kB 00:00 2024-04-23T14:30:58.967 INFO:teuthology.orchestra.run.smithi083.stdout:(37/152): cephfs-top-19.0.0-3126.gd44fddd9.el9. 170 kB/s | 18 kB 00:00 2024-04-23T14:30:58.976 INFO:teuthology.orchestra.run.smithi173.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-23T14:30:58.993 INFO:teuthology.orchestra.run.smithi173.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-04-23T14:30:59.043 INFO:teuthology.orchestra.run.smithi173.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-23T14:30:59.118 INFO:teuthology.orchestra.run.smithi173.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-04-23T14:30:59.122 INFO:teuthology.orchestra.run.smithi083.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 525 kB/s | 80 kB 00:00 2024-04-23T14:30:59.135 INFO:teuthology.orchestra.run.smithi173.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-04-23T14:30:59.160 INFO:teuthology.orchestra.run.smithi173.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-04-23T14:30:59.177 INFO:teuthology.orchestra.run.smithi173.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-23T14:30:59.219 INFO:teuthology.orchestra.run.smithi173.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-23T14:30:59.245 INFO:teuthology.orchestra.run.smithi173.stdout:(60/152): bison-3.7.4-5.el9.x86_64.rpm 2.3 MB/s | 943 kB 00:00 2024-04-23T14:30:59.260 INFO:teuthology.orchestra.run.smithi083.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.7 MB/s | 803 kB 00:00 2024-04-23T14:30:59.310 INFO:teuthology.orchestra.run.smithi083.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2024-04-23T14:30:59.312 INFO:teuthology.orchestra.run.smithi173.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 277 kB/s | 25 kB 00:00 2024-04-23T14:30:59.345 INFO:teuthology.orchestra.run.smithi173.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-23T14:30:59.360 INFO:teuthology.orchestra.run.smithi083.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-23T14:30:59.362 INFO:teuthology.orchestra.run.smithi173.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 420 kB/s | 47 kB 00:00 2024-04-23T14:30:59.396 INFO:teuthology.orchestra.run.smithi173.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-23T14:30:59.413 INFO:teuthology.orchestra.run.smithi173.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-04-23T14:30:59.438 INFO:teuthology.orchestra.run.smithi173.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-23T14:30:59.444 INFO:teuthology.orchestra.run.smithi083.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-23T14:30:59.494 INFO:teuthology.orchestra.run.smithi083.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-23T14:30:59.513 INFO:teuthology.orchestra.run.smithi173.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-23T14:30:59.545 INFO:teuthology.orchestra.run.smithi083.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-23T14:30:59.580 INFO:teuthology.orchestra.run.smithi173.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-04-23T14:30:59.595 INFO:teuthology.orchestra.run.smithi083.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-23T14:30:59.622 INFO:teuthology.orchestra.run.smithi173.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-23T14:30:59.645 INFO:teuthology.orchestra.run.smithi083.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.3 MB/s | 215 kB 00:00 2024-04-23T14:30:59.647 INFO:teuthology.orchestra.run.smithi173.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-23T14:30:59.706 INFO:teuthology.orchestra.run.smithi173.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-04-23T14:30:59.798 INFO:teuthology.orchestra.run.smithi173.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-04-23T14:30:59.823 INFO:teuthology.orchestra.run.smithi173.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 750 kB/s | 87 kB 00:00 2024-04-23T14:30:59.829 INFO:teuthology.orchestra.run.smithi083.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-04-23T14:30:59.890 INFO:teuthology.orchestra.run.smithi173.stdout:(74/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-04-23T14:30:59.913 INFO:teuthology.orchestra.run.smithi083.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-04-23T14:30:59.915 INFO:teuthology.orchestra.run.smithi173.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-23T14:30:59.940 INFO:teuthology.orchestra.run.smithi173.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-23T14:30:59.954 INFO:teuthology.orchestra.run.smithi083.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 107 kB 00:00 2024-04-23T14:30:59.966 INFO:teuthology.orchestra.run.smithi173.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-04-23T14:31:00.008 INFO:teuthology.orchestra.run.smithi173.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-23T14:31:00.024 INFO:teuthology.orchestra.run.smithi083.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 312 kB/s | 21 kB 00:00 2024-04-23T14:31:00.080 INFO:teuthology.orchestra.run.smithi083.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 5.6 MB/s | 313 kB 00:00 2024-04-23T14:31:00.083 INFO:teuthology.orchestra.run.smithi173.stdout:(79/152): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-04-23T14:31:00.108 INFO:teuthology.orchestra.run.smithi173.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-04-23T14:31:00.122 INFO:teuthology.orchestra.run.smithi083.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 792 kB/s | 33 kB 00:00 2024-04-23T14:31:00.159 INFO:teuthology.orchestra.run.smithi173.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-23T14:31:00.184 INFO:teuthology.orchestra.run.smithi173.stdout:(82/152): python3-jsonpointer-2.0-4.el9.noarch. 79 kB/s | 19 kB 00:00 2024-04-23T14:31:00.209 INFO:teuthology.orchestra.run.smithi173.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-23T14:31:00.226 INFO:teuthology.orchestra.run.smithi173.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-23T14:31:00.251 INFO:teuthology.orchestra.run.smithi173.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-23T14:31:00.256 INFO:teuthology.orchestra.run.smithi083.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 23 MB/s | 3.0 MB 00:00 2024-04-23T14:31:00.298 INFO:teuthology.orchestra.run.smithi083.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 447 kB/s | 18 kB 00:00 2024-04-23T14:31:00.318 INFO:teuthology.orchestra.run.smithi173.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 5.1 MB 00:00 2024-04-23T14:31:00.340 INFO:teuthology.orchestra.run.smithi083.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 271 kB/s | 11 kB 00:00 2024-04-23T14:31:00.343 INFO:teuthology.orchestra.run.smithi173.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-04-23T14:31:00.369 INFO:teuthology.orchestra.run.smithi173.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-23T14:31:00.390 INFO:teuthology.orchestra.run.smithi083.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 160 kB 00:00 2024-04-23T14:31:00.394 INFO:teuthology.orchestra.run.smithi173.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-23T14:31:00.419 INFO:teuthology.orchestra.run.smithi173.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-23T14:31:00.432 INFO:teuthology.orchestra.run.smithi083.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-23T14:31:00.444 INFO:teuthology.orchestra.run.smithi173.stdout:(91/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-04-23T14:31:00.470 INFO:teuthology.orchestra.run.smithi173.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-04-23T14:31:00.495 INFO:teuthology.orchestra.run.smithi173.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-04-23T14:31:00.499 INFO:teuthology.orchestra.run.smithi083.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 662 kB 00:00 2024-04-23T14:31:00.520 INFO:teuthology.orchestra.run.smithi173.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-23T14:31:00.537 INFO:teuthology.orchestra.run.smithi173.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 858 kB/s | 36 kB 00:00 2024-04-23T14:31:00.558 INFO:teuthology.orchestra.run.smithi083.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-23T14:31:00.562 INFO:teuthology.orchestra.run.smithi173.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-23T14:31:00.588 INFO:teuthology.orchestra.run.smithi173.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-23T14:31:00.600 INFO:teuthology.orchestra.run.smithi083.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 619 kB/s | 25 kB 00:00 2024-04-23T14:31:00.613 INFO:teuthology.orchestra.run.smithi173.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-23T14:31:00.638 INFO:teuthology.orchestra.run.smithi173.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-23T14:31:00.642 INFO:teuthology.orchestra.run.smithi083.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-23T14:31:00.664 INFO:teuthology.orchestra.run.smithi173.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-23T14:31:00.689 INFO:teuthology.orchestra.run.smithi173.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-04-23T14:31:00.689 INFO:teuthology.orchestra.run.smithi203.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3126.gd44fd 899 kB/s | 3.5 MB 00:04 2024-04-23T14:31:00.692 INFO:teuthology.orchestra.run.smithi083.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.8 MB/s | 292 kB 00:00 2024-04-23T14:31:00.706 INFO:teuthology.orchestra.run.smithi173.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 535 kB/s | 22 kB 00:00 2024-04-23T14:31:00.742 INFO:teuthology.orchestra.run.smithi083.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-23T14:31:00.756 INFO:teuthology.orchestra.run.smithi173.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-23T14:31:00.784 INFO:teuthology.orchestra.run.smithi083.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 858 kB/s | 35 kB 00:00 2024-04-23T14:31:00.952 INFO:teuthology.orchestra.run.smithi083.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 29 MB/s | 4.7 MB 00:00 2024-04-23T14:31:00.956 INFO:teuthology.orchestra.run.smithi203.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3126.gd4 933 kB/s | 249 kB 00:00 2024-04-23T14:31:01.040 INFO:teuthology.orchestra.run.smithi203.stdout:(33/152): ceph-mgr-rook-19.0.0-3126.gd44fddd9.e 592 kB/s | 49 kB 00:00 2024-04-23T14:31:01.093 INFO:teuthology.orchestra.run.smithi083.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-04-23T14:31:01.098 INFO:teuthology.orchestra.run.smithi203.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3126.gd 282 kB/s | 16 kB 00:00 2024-04-23T14:31:01.123 INFO:teuthology.orchestra.run.smithi173.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-23T14:31:01.177 INFO:teuthology.orchestra.run.smithi083.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-04-23T14:31:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 14 MB/s | 911 kB 00:00 2024-04-23T14:31:01.274 INFO:teuthology.orchestra.run.smithi203.stdout:(35/152): ceph-volume-19.0.0-3126.gd44fddd9.el9 1.5 MB/s | 273 kB 00:00 2024-04-23T14:31:01.286 INFO:teuthology.orchestra.run.smithi083.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 3.2 MB/s | 133 kB 00:00 2024-04-23T14:31:01.373 INFO:teuthology.orchestra.run.smithi173.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-04-23T14:31:01.399 INFO:teuthology.orchestra.run.smithi173.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 481 kB/s | 309 kB 00:00 2024-04-23T14:31:01.453 INFO:teuthology.orchestra.run.smithi083.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-04-23T14:31:01.499 INFO:teuthology.orchestra.run.smithi173.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-23T14:31:01.578 INFO:teuthology.orchestra.run.smithi083.stdout:(71/152): ceph-mgr-diskprediction-local-19.0.0- 1.6 MB/s | 7.4 MB 00:04 2024-04-23T14:31:01.599 INFO:teuthology.orchestra.run.smithi173.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-04-23T14:31:01.725 INFO:teuthology.orchestra.run.smithi173.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2024-04-23T14:31:01.850 INFO:teuthology.orchestra.run.smithi173.stdout:(110/152): abseil-cpp-20211102.0-3.el9.x86_64.r 479 kB/s | 548 kB 00:01 2024-04-23T14:31:01.875 INFO:teuthology.orchestra.run.smithi173.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-23T14:31:01.890 INFO:teuthology.orchestra.run.smithi203.stdout:(36/152): cephadm-19.0.0-3126.gd44fddd9.el9.noa 1.2 MB/s | 762 kB 00:00 2024-04-23T14:31:02.025 INFO:teuthology.orchestra.run.smithi203.stdout:(37/152): cephfs-top-19.0.0-3126.gd44fddd9.el9. 136 kB/s | 18 kB 00:00 2024-04-23T14:31:02.104 INFO:teuthology.orchestra.run.smithi083.stdout:(72/152): ceph-test-19.0.0-3126.gd44fddd9.el9.x 3.9 MB/s | 49 MB 00:12 2024-04-23T14:31:02.142 INFO:teuthology.orchestra.run.smithi173.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.0 MB/s | 548 kB 00:00 2024-04-23T14:31:02.154 INFO:teuthology.orchestra.run.smithi083.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 1.4 MB/s | 1.0 MB 00:00 2024-04-23T14:31:02.204 INFO:teuthology.orchestra.run.smithi083.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-04-23T14:31:02.209 INFO:teuthology.orchestra.run.smithi173.stdout:(113/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-04-23T14:31:02.229 INFO:teuthology.orchestra.run.smithi083.stdout:(75/152): python3-attrs-20.3.0-7.el9.noarch.rpm 134 kB/s | 87 kB 00:00 2024-04-23T14:31:02.259 INFO:teuthology.orchestra.run.smithi173.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2024-04-23T14:31:02.263 INFO:teuthology.orchestra.run.smithi083.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-04-23T14:31:02.288 INFO:teuthology.orchestra.run.smithi083.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 444 kB/s | 26 kB 00:00 2024-04-23T14:31:02.313 INFO:teuthology.orchestra.run.smithi083.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 388 kB/s | 19 kB 00:00 2024-04-23T14:31:02.326 INFO:teuthology.orchestra.run.smithi173.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2024-04-23T14:31:02.350 INFO:teuthology.orchestra.run.smithi203.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 247 kB/s | 80 kB 00:00 2024-04-23T14:31:02.355 INFO:teuthology.orchestra.run.smithi083.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-23T14:31:02.360 INFO:teuthology.orchestra.run.smithi173.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-04-23T14:31:02.381 INFO:teuthology.orchestra.run.smithi083.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 526 kB/s | 35 kB 00:00 2024-04-23T14:31:02.418 INFO:teuthology.orchestra.run.smithi173.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-23T14:31:02.431 INFO:teuthology.orchestra.run.smithi083.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-04-23T14:31:02.481 INFO:teuthology.orchestra.run.smithi083.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-23T14:31:02.502 INFO:teuthology.orchestra.run.smithi173.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-04-23T14:31:02.523 INFO:teuthology.orchestra.run.smithi083.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-23T14:31:02.527 INFO:teuthology.orchestra.run.smithi173.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-23T14:31:02.567 INFO:teuthology.orchestra.run.smithi083.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 983 kB/s | 42 kB 00:00 2024-04-23T14:31:02.602 INFO:teuthology.orchestra.run.smithi173.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-04-23T14:31:02.619 INFO:teuthology.orchestra.run.smithi173.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-23T14:31:02.728 INFO:teuthology.orchestra.run.smithi173.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-23T14:31:02.751 INFO:teuthology.orchestra.run.smithi083.stdout:(85/152): python3-babel-2.9.1-2.el9.noarch.rpm 9.2 MB/s | 6.0 MB 00:00 2024-04-23T14:31:02.784 INFO:teuthology.orchestra.run.smithi083.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 1.2 MB/s | 269 kB 00:00 2024-04-23T14:31:02.818 INFO:teuthology.orchestra.run.smithi083.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-04-23T14:31:02.836 INFO:teuthology.orchestra.run.smithi173.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-04-23T14:31:02.860 INFO:teuthology.orchestra.run.smithi083.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-23T14:31:02.870 INFO:teuthology.orchestra.run.smithi173.stdout:(124/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 822 kB/s | 838 kB 00:01 2024-04-23T14:31:02.885 INFO:teuthology.orchestra.run.smithi083.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 768 kB/s | 51 kB 00:00 2024-04-23T14:31:02.911 INFO:teuthology.orchestra.run.smithi083.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-23T14:31:02.912 INFO:teuthology.orchestra.run.smithi173.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-04-23T14:31:02.934 INFO:teuthology.orchestra.run.smithi203.stdout:(39/152): ceph-test-19.0.0-3126.gd44fddd9.el9.x 3.2 MB/s | 49 MB 00:15 2024-04-23T14:31:02.962 INFO:teuthology.orchestra.run.smithi173.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-04-23T14:31:02.976 INFO:teuthology.orchestra.run.smithi203.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2024-04-23T14:31:02.988 INFO:teuthology.orchestra.run.smithi173.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 197 kB/s | 23 kB 00:00 2024-04-23T14:31:03.021 INFO:teuthology.orchestra.run.smithi173.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 178 kB/s | 19 kB 00:00 2024-04-23T14:31:03.044 INFO:teuthology.orchestra.run.smithi083.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.4 MB/s | 5.1 MB 00:00 2024-04-23T14:31:03.070 INFO:teuthology.orchestra.run.smithi083.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 264 kB/s | 42 kB 00:00 2024-04-23T14:31:03.088 INFO:teuthology.orchestra.run.smithi173.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-04-23T14:31:03.095 INFO:teuthology.orchestra.run.smithi083.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 894 kB/s | 44 kB 00:00 2024-04-23T14:31:03.117 INFO:teuthology.orchestra.run.smithi203.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-23T14:31:03.122 INFO:teuthology.orchestra.run.smithi173.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-04-23T14:31:03.129 INFO:teuthology.orchestra.run.smithi083.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-23T14:31:03.151 INFO:teuthology.orchestra.run.smithi203.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 901 kB/s | 195 kB 00:00 2024-04-23T14:31:03.154 INFO:teuthology.orchestra.run.smithi083.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-04-23T14:31:03.155 INFO:teuthology.orchestra.run.smithi173.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-23T14:31:03.179 INFO:teuthology.orchestra.run.smithi083.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-04-23T14:31:03.210 INFO:teuthology.orchestra.run.smithi203.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-23T14:31:03.213 INFO:teuthology.orchestra.run.smithi083.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-23T14:31:03.222 INFO:teuthology.orchestra.run.smithi173.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 577 kB/s | 58 kB 00:00 2024-04-23T14:31:03.238 INFO:teuthology.orchestra.run.smithi083.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-23T14:31:03.264 INFO:teuthology.orchestra.run.smithi083.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-23T14:31:03.281 INFO:teuthology.orchestra.run.smithi173.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-23T14:31:03.285 INFO:teuthology.orchestra.run.smithi203.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-04-23T14:31:03.289 INFO:teuthology.orchestra.run.smithi083.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-23T14:31:03.314 INFO:teuthology.orchestra.run.smithi083.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-23T14:31:03.314 INFO:teuthology.orchestra.run.smithi173.stdout:(134/152): python3-more-itertools-8.12.0-2.el9. 349 kB/s | 79 kB 00:00 2024-04-23T14:31:03.340 INFO:teuthology.orchestra.run.smithi173.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-04-23T14:31:03.344 INFO:teuthology.orchestra.run.smithi203.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-23T14:31:03.423 INFO:teuthology.orchestra.run.smithi173.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-23T14:31:03.431 INFO:teuthology.orchestra.run.smithi083.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 158 kB/s | 22 kB 00:00 2024-04-23T14:31:03.436 INFO:teuthology.orchestra.run.smithi203.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-23T14:31:03.448 INFO:teuthology.orchestra.run.smithi173.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2024-04-23T14:31:03.482 INFO:teuthology.orchestra.run.smithi173.stdout:(138/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 448 kB/s | 90 kB 00:00 2024-04-23T14:31:03.511 INFO:teuthology.orchestra.run.smithi203.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-04-23T14:31:03.549 INFO:teuthology.orchestra.run.smithi173.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-23T14:31:03.574 INFO:teuthology.orchestra.run.smithi173.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 470 kB/s | 59 kB 00:00 2024-04-23T14:31:03.641 INFO:teuthology.orchestra.run.smithi173.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 225 kB/s | 36 kB 00:00 2024-04-23T14:31:03.665 INFO:teuthology.orchestra.run.smithi083.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-23T14:31:03.666 INFO:teuthology.orchestra.run.smithi173.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 434 kB/s | 51 kB 00:00 2024-04-23T14:31:03.692 INFO:teuthology.orchestra.run.smithi173.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-23T14:31:03.776 INFO:teuthology.orchestra.run.smithi173.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-23T14:31:03.799 INFO:teuthology.orchestra.run.smithi083.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-04-23T14:31:03.801 INFO:teuthology.orchestra.run.smithi173.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-23T14:31:03.826 INFO:teuthology.orchestra.run.smithi173.stdout:(146/152): python3-websocket-client-1.2.3-2.el9 486 kB/s | 90 kB 00:00 2024-04-23T14:31:03.882 INFO:teuthology.orchestra.run.smithi083.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.4 MB/s | 309 kB 00:00 2024-04-23T14:31:03.907 INFO:teuthology.orchestra.run.smithi083.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 181 kB/s | 19 kB 00:00 2024-04-23T14:31:03.927 INFO:teuthology.orchestra.run.smithi173.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-23T14:31:03.941 INFO:teuthology.orchestra.run.smithi083.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-23T14:31:03.966 INFO:teuthology.orchestra.run.smithi083.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 448 kB/s | 26 kB 00:00 2024-04-23T14:31:03.968 INFO:teuthology.orchestra.run.smithi173.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-23T14:31:04.000 INFO:teuthology.orchestra.run.smithi083.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 842 kB/s | 49 kB 00:00 2024-04-23T14:31:04.050 INFO:teuthology.orchestra.run.smithi083.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 814 kB/s | 67 kB 00:00 2024-04-23T14:31:04.101 INFO:teuthology.orchestra.run.smithi083.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-23T14:31:04.144 INFO:teuthology.orchestra.run.smithi173.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 1.3 MB 00:00 2024-04-23T14:31:04.227 INFO:teuthology.orchestra.run.smithi173.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-04-23T14:31:04.259 INFO:teuthology.orchestra.run.smithi083.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-04-23T14:31:04.310 INFO:teuthology.orchestra.run.smithi083.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 870 kB/s | 43 kB 00:00 2024-04-23T14:31:04.343 INFO:teuthology.orchestra.run.smithi083.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.8 MB/s | 838 kB 00:00 2024-04-23T14:31:04.368 INFO:teuthology.orchestra.run.smithi083.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 557 kB/s | 32 kB 00:00 2024-04-23T14:31:04.402 INFO:teuthology.orchestra.run.smithi083.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 259 kB/s | 15 kB 00:00 2024-04-23T14:31:04.469 INFO:teuthology.orchestra.run.smithi083.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-23T14:31:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 1.3 MB/s | 358 kB 00:00 2024-04-23T14:31:04.703 INFO:teuthology.orchestra.run.smithi083.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.3 MB/s | 303 kB 00:00 2024-04-23T14:31:04.753 INFO:teuthology.orchestra.run.smithi083.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 426 kB/s | 35 kB 00:00 2024-04-23T14:31:04.921 INFO:teuthology.orchestra.run.smithi083.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 947 kB/s | 206 kB 00:00 2024-04-23T14:31:04.978 INFO:teuthology.orchestra.run.smithi173.stdout:(151/152): librbd1-19.0.0-3126.gd44fddd9.el9.x8 3.0 MB/s | 3.1 MB 00:01 2024-04-23T14:31:05.121 INFO:teuthology.orchestra.run.smithi083.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 727 kB/s | 144 kB 00:00 2024-04-23T14:31:05.171 INFO:teuthology.orchestra.run.smithi083.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 217 kB/s | 11 kB 00:00 2024-04-23T14:31:05.236 INFO:teuthology.orchestra.run.smithi203.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 383 kB/s | 689 kB 00:01 2024-04-23T14:31:05.271 INFO:teuthology.orchestra.run.smithi083.stdout:(124/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-04-23T14:31:05.301 INFO:teuthology.orchestra.run.smithi083.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 137 kB/s | 18 kB 00:00 2024-04-23T14:31:05.405 INFO:teuthology.orchestra.run.smithi083.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 174 kB/s | 23 kB 00:00 2024-04-23T14:31:05.430 INFO:teuthology.orchestra.run.smithi083.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 151 kB/s | 19 kB 00:00 2024-04-23T14:31:05.472 INFO:teuthology.orchestra.run.smithi083.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 298 kB/s | 20 kB 00:00 2024-04-23T14:31:05.522 INFO:teuthology.orchestra.run.smithi083.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 935 kB/s | 46 kB 00:00 2024-04-23T14:31:05.590 INFO:teuthology.orchestra.run.smithi083.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-04-23T14:31:05.603 INFO:teuthology.orchestra.run.smithi203.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 451 kB/s | 943 kB 00:02 2024-04-23T14:31:05.615 INFO:teuthology.orchestra.run.smithi083.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 859 kB/s | 79 kB 00:00 2024-04-23T14:31:05.640 INFO:teuthology.orchestra.run.smithi083.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-23T14:31:05.699 INFO:teuthology.orchestra.run.smithi083.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 285 kB/s | 16 kB 00:00 2024-04-23T14:31:05.741 INFO:teuthology.orchestra.run.smithi083.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-23T14:31:05.758 INFO:teuthology.orchestra.run.smithi083.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-23T14:31:05.791 INFO:teuthology.orchestra.run.smithi083.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 579 kB/s | 28 kB 00:00 2024-04-23T14:31:05.816 INFO:teuthology.orchestra.run.smithi083.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 530 kB/s | 31 kB 00:00 2024-04-23T14:31:05.867 INFO:teuthology.orchestra.run.smithi083.stdout:(138/152): python3-kubernetes-26.1.0-2.el9.noar 2.3 MB/s | 1.0 MB 00:00 2024-04-23T14:31:05.892 INFO:teuthology.orchestra.run.smithi083.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 789 kB/s | 59 kB 00:00 2024-04-23T14:31:05.918 INFO:teuthology.orchestra.run.smithi083.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 719 kB/s | 36 kB 00:00 2024-04-23T14:31:05.943 INFO:teuthology.orchestra.run.smithi083.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-23T14:31:05.968 INFO:teuthology.orchestra.run.smithi083.stdout:(142/152): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-04-23T14:31:06.002 INFO:teuthology.orchestra.run.smithi083.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-23T14:31:06.003 INFO:teuthology.orchestra.run.smithi203.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 139 kB/s | 107 kB 00:00 2024-04-23T14:31:06.052 INFO:teuthology.orchestra.run.smithi083.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-23T14:31:06.077 INFO:teuthology.orchestra.run.smithi083.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 268 kB/s | 20 kB 00:00 2024-04-23T14:31:06.128 INFO:teuthology.orchestra.run.smithi083.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-23T14:31:06.161 INFO:teuthology.orchestra.run.smithi083.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.2 MB/s | 427 kB 00:00 2024-04-23T14:31:06.220 INFO:teuthology.orchestra.run.smithi203.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 35 kB/s | 21 kB 00:00 2024-04-23T14:31:06.329 INFO:teuthology.orchestra.run.smithi203.stdout:(52/152): ceph-mgr-diskprediction-local-19.0.0- 1.0 MB/s | 7.4 MB 00:07 2024-04-23T14:31:06.387 INFO:teuthology.orchestra.run.smithi083.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-04-23T14:31:06.578 INFO:teuthology.orchestra.run.smithi083.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.1 MB/s | 1.3 MB 00:00 2024-04-23T14:31:06.687 INFO:teuthology.orchestra.run.smithi083.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-23T14:31:06.946 INFO:teuthology.orchestra.run.smithi203.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 45 kB/s | 33 kB 00:00 2024-04-23T14:31:07.271 INFO:teuthology.orchestra.run.smithi203.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 247 kB/s | 313 kB 00:01 2024-04-23T14:31:07.788 INFO:teuthology.orchestra.run.smithi203.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 22 kB/s | 18 kB 00:00 2024-04-23T14:31:07.894 INFO:teuthology.orchestra.run.smithi173.stdout:(152/152): librados2-19.0.0-3126.gd44fddd9.el9. 854 kB/s | 3.4 MB 00:04 2024-04-23T14:31:07.896 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2024-04-23T14:31:07.897 INFO:teuthology.orchestra.run.smithi173.stdout:Total 7.6 MB/s | 222 MB 00:29 2024-04-23T14:31:08.239 INFO:teuthology.orchestra.run.smithi203.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 12 kB/s | 11 kB 00:00 2024-04-23T14:31:08.296 INFO:teuthology.orchestra.run.smithi083.stdout:(151/152): librados2-19.0.0-3126.gd44fddd9.el9. 1.6 MB/s | 3.4 MB 00:02 2024-04-23T14:31:08.847 INFO:teuthology.orchestra.run.smithi203.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 151 kB/s | 160 kB 00:01 2024-04-23T14:31:09.019 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2024-04-23T14:31:09.164 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2024-04-23T14:31:09.164 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2024-04-23T14:31:09.405 INFO:teuthology.orchestra.run.smithi203.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 39 kB/s | 45 kB 00:01 2024-04-23T14:31:10.287 INFO:teuthology.orchestra.run.smithi083.stdout:(152/152): librbd1-19.0.0-3126.gd44fddd9.el9.x8 780 kB/s | 3.1 MB 00:04 2024-04-23T14:31:10.293 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-23T14:31:10.294 INFO:teuthology.orchestra.run.smithi083.stdout:Total 7.9 MB/s | 222 MB 00:28 2024-04-23T14:31:10.539 INFO:teuthology.orchestra.run.smithi203.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 392 kB/s | 662 kB 00:01 2024-04-23T14:31:10.915 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2024-04-23T14:31:10.915 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2024-04-23T14:31:10.998 INFO:teuthology.orchestra.run.smithi203.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 366 kB/s | 582 kB 00:01 2024-04-23T14:31:11.232 INFO:teuthology.orchestra.run.smithi203.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 37 kB/s | 25 kB 00:00 2024-04-23T14:31:11.490 INFO:teuthology.orchestra.run.smithi203.stdout:(62/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 596 kB/s | 3.0 MB 00:05 2024-04-23T14:31:11.732 INFO:teuthology.orchestra.run.smithi203.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 65 kB/s | 47 kB 00:00 2024-04-23T14:31:12.282 INFO:teuthology.orchestra.run.smithi203.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 278 kB/s | 292 kB 00:01 2024-04-23T14:31:12.408 INFO:teuthology.orchestra.run.smithi203.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 52 kB/s | 35 kB 00:00 2024-04-23T14:31:12.441 INFO:teuthology.orchestra.run.smithi203.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 204 kB/s | 188 kB 00:00 2024-04-23T14:31:13.168 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-04-23T14:31:13.422 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-04-23T14:31:13.422 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-04-23T14:31:15.367 INFO:teuthology.orchestra.run.smithi203.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 556 kB/s | 1.6 MB 00:02 2024-04-23T14:31:16.165 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-04-23T14:31:16.167 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-04-23T14:31:16.868 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2024-04-23T14:31:16.947 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T14:31:17.002 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T14:31:17.056 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T14:31:17.090 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T14:31:17.115 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T14:31:17.158 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T14:31:17.233 INFO:teuthology.orchestra.run.smithi203.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 488 kB/s | 911 kB 00:01 2024-04-23T14:31:17.496 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T14:31:17.608 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:31:17.721 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:31:17.751 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:31:17.833 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:31:17.871 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 10/156 2024-04-23T14:31:18.026 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T14:31:18.076 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T14:31:18.105 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:31:18.158 INFO:teuthology.orchestra.run.smithi203.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 144 kB/s | 133 kB 00:00 2024-04-23T14:31:18.208 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:31:18.231 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 14/156 2024-04-23T14:31:18.293 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 14/156 2024-04-23T14:31:18.315 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T14:31:18.340 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T14:31:18.392 INFO:teuthology.orchestra.run.smithi203.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 794 kB/s | 4.7 MB 00:06 2024-04-23T14:31:18.531 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T14:31:18.564 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T14:31:18.671 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T14:31:18.768 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T14:31:18.797 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T14:31:18.832 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T14:31:18.897 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T14:31:18.930 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T14:31:18.974 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T14:31:19.007 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T14:31:19.035 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T14:31:19.104 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T14:31:19.157 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T14:31:19.187 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T14:31:19.217 INFO:teuthology.orchestra.run.smithi203.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 665 kB/s | 4.4 MB 00:06 2024-04-23T14:31:19.222 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T14:31:19.298 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T14:31:19.333 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9. 31/156 2024-04-23T14:31:19.377 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el 32/156 2024-04-23T14:31:19.405 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_6 33/156 2024-04-23T14:31:19.475 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T14:31:19.519 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T14:31:19.595 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T14:31:19.738 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T14:31:19.797 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T14:31:20.009 INFO:teuthology.orchestra.run.smithi203.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 110 kB/s | 87 kB 00:00 2024-04-23T14:31:20.330 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T14:31:20.418 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T14:31:20.447 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T14:31:20.490 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T14:31:20.526 INFO:teuthology.orchestra.run.smithi203.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 484 kB/s | 1.0 MB 00:02 2024-04-23T14:31:20.527 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_6 43/156 2024-04-23T14:31:20.569 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T14:31:20.723 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T14:31:20.780 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T14:31:20.835 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T14:31:20.908 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T14:31:20.939 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T14:31:20.984 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T14:31:21.031 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T14:31:21.075 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T14:31:21.119 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T14:31:21.160 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T14:31:21.298 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T14:31:21.341 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T14:31:21.371 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T14:31:21.412 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T14:31:21.507 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T14:31:21.546 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T14:31:21.586 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T14:31:21.611 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T14:31:21.636 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T14:31:21.712 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T14:31:21.760 INFO:teuthology.orchestra.run.smithi203.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 198 kB/s | 244 kB 00:01 2024-04-23T14:31:21.839 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T14:31:21.877 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T14:31:21.905 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T14:31:21.939 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T14:31:21.978 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T14:31:22.010 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T14:31:22.040 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T14:31:22.074 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T14:31:22.107 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T14:31:22.149 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T14:31:22.183 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T14:31:22.271 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T14:31:22.322 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T14:31:22.353 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T14:31:22.400 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T14:31:22.477 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-04-23T14:31:22.499 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T14:31:22.555 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T14:31:22.646 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T14:31:22.726 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T14:31:22.760 INFO:teuthology.orchestra.run.smithi203.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 248 kB/s | 248 kB 00:00 2024-04-23T14:31:22.834 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T14:31:22.897 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T14:31:22.930 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T14:31:23.061 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T14:31:23.300 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T14:31:23.343 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T14:31:23.377 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T14:31:23.385 INFO:teuthology.orchestra.run.smithi203.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 41 kB/s | 26 kB 00:00 2024-04-23T14:31:23.477 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T14:31:23.497 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T14:31:23.527 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T14:31:23.607 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T14:31:23.610 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:31:23.666 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T14:31:23.783 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:31:23.820 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:31:23.850 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T14:31:23.885 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T14:31:23.944 INFO:teuthology.orchestra.run.smithi203.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 34 kB/s | 19 kB 00:00 2024-04-23T14:31:23.960 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:31:23.963 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T14:31:23.995 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 10/156 2024-04-23T14:31:24.365 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T14:31:24.438 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T14:31:24.466 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:31:24.618 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:31:24.635 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T14:31:24.659 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 14/156 2024-04-23T14:31:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T14:31:24.761 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 14/156 2024-04-23T14:31:24.793 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T14:31:24.849 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T14:31:24.886 INFO:teuthology.orchestra.run.smithi203.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 183 kB/s | 172 kB 00:00 2024-04-23T14:31:25.163 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T14:31:25.217 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T14:31:25.414 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T14:31:25.531 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T14:31:25.561 INFO:teuthology.orchestra.run.smithi203.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 51 kB/s | 35 kB 00:00 2024-04-23T14:31:25.588 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T14:31:25.634 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T14:31:25.727 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T14:31:25.772 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T14:31:25.834 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T14:31:25.877 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T14:31:25.917 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T14:31:26.002 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T14:31:26.032 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T14:31:26.074 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T14:31:26.095 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T14:31:26.099 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T14:31:26.133 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T14:31:26.160 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T14:31:26.220 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T14:31:26.329 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T14:31:26.364 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9. 31/156 2024-04-23T14:31:26.420 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el 32/156 2024-04-23T14:31:26.445 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T14:31:26.455 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_6 33/156 2024-04-23T14:31:26.482 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 99/156 2024-04-23T14:31:26.553 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 99/156 2024-04-23T14:31:26.570 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T14:31:26.576 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 100/156 2024-04-23T14:31:26.611 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T14:31:26.633 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T14:31:26.732 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T14:31:26.915 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T14:31:26.961 INFO:teuthology.orchestra.run.smithi203.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 672 kB/s | 5.8 MB 00:08 2024-04-23T14:31:26.986 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T14:31:27.095 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T14:31:27.123 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 103/156 2024-04-23T14:31:27.203 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 103/156 2024-04-23T14:31:27.233 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 104/156 2024-04-23T14:31:27.657 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T14:31:27.765 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T14:31:27.803 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T14:31:27.852 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T14:31:27.897 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_6 43/156 2024-04-23T14:31:27.978 INFO:teuthology.orchestra.run.smithi203.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 247 kB/s | 250 kB 00:01 2024-04-23T14:31:27.991 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T14:31:28.289 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T14:31:28.357 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T14:31:28.440 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T14:31:28.545 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T14:31:28.589 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T14:31:28.629 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T14:31:28.696 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T14:31:28.737 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T14:31:28.800 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T14:31:28.873 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T14:31:28.911 INFO:teuthology.orchestra.run.smithi203.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 238 kB/s | 222 kB 00:00 2024-04-23T14:31:29.068 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T14:31:29.118 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T14:31:29.155 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T14:31:29.212 INFO:teuthology.orchestra.run.smithi203.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 664 kB/s | 6.0 MB 00:09 2024-04-23T14:31:29.212 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T14:31:29.338 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T14:31:29.381 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T14:31:29.427 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T14:31:29.462 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T14:31:29.495 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T14:31:29.513 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:31:29.555 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:31:29.693 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T14:31:29.695 INFO:teuthology.orchestra.run.smithi203.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 98 kB/s | 77 kB 00:00 2024-04-23T14:31:29.862 INFO:teuthology.orchestra.run.smithi203.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 68 kB/s | 42 kB 00:00 2024-04-23T14:31:29.872 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T14:31:29.919 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T14:31:29.969 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T14:31:30.013 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T14:31:30.064 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T14:31:30.106 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:31:30.111 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T14:31:30.156 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T14:31:30.200 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T14:31:30.250 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T14:31:30.274 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 106/156 2024-04-23T14:31:30.326 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T14:31:30.370 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 106/156 2024-04-23T14:31:30.371 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T14:31:30.371 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T14:31:30.371 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:31:30.375 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T14:31:30.400 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 107/156 2024-04-23T14:31:30.525 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T14:31:30.600 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T14:31:30.658 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T14:31:30.727 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T14:31:30.729 INFO:teuthology.orchestra.run.smithi203.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 261 kB/s | 269 kB 00:01 2024-04-23T14:31:30.796 INFO:teuthology.orchestra.run.smithi203.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 170 kB/s | 159 kB 00:00 2024-04-23T14:31:30.874 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T14:31:30.937 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T14:31:31.455 INFO:teuthology.orchestra.run.smithi203.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 77 kB/s | 51 kB 00:00 2024-04-23T14:31:31.755 INFO:teuthology.orchestra.run.smithi203.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 272 kB/s | 279 kB 00:01 2024-04-23T14:31:31.883 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T14:31:31.937 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T14:31:31.970 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T14:31:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T14:31:32.105 INFO:teuthology.orchestra.run.smithi203.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 83 kB/s | 54 kB 00:00 2024-04-23T14:31:32.164 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T14:31:32.259 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T14:31:32.305 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T14:31:32.534 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T14:31:32.569 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T14:31:32.680 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T14:31:32.764 INFO:teuthology.orchestra.run.smithi203.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 63 kB/s | 42 kB 00:00 2024-04-23T14:31:33.239 INFO:teuthology.orchestra.run.smithi203.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 680 kB/s | 5.1 MB 00:07 2024-04-23T14:31:33.473 INFO:teuthology.orchestra.run.smithi203.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 62 kB/s | 44 kB 00:00 2024-04-23T14:31:33.487 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T14:31:33.641 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T14:31:34.181 INFO:teuthology.orchestra.run.smithi203.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 51 kB/s | 36 kB 00:00 2024-04-23T14:31:34.223 INFO:teuthology.orchestra.run.smithi203.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 224 kB/s | 221 kB 00:00 2024-04-23T14:31:34.973 INFO:teuthology.orchestra.run.smithi203.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 60 kB/s | 47 kB 00:00 2024-04-23T14:31:35.236 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T14:31:35.360 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T14:31:35.398 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T14:31:35.444 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T14:31:35.632 INFO:teuthology.orchestra.run.smithi203.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 216 kB/s | 305 kB 00:01 2024-04-23T14:31:35.821 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T14:31:35.864 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 99/156 2024-04-23T14:31:35.874 INFO:teuthology.orchestra.run.smithi203.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 64 kB/s | 58 kB 00:00 2024-04-23T14:31:36.046 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 99/156 2024-04-23T14:31:36.079 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 100/156 2024-04-23T14:31:36.141 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T14:31:36.582 INFO:teuthology.orchestra.run.smithi203.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 99 kB/s | 94 kB 00:00 2024-04-23T14:31:36.687 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T14:31:36.724 INFO:teuthology.orchestra.run.smithi203.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 75 kB/s | 64 kB 00:00 2024-04-23T14:31:36.726 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 103/156 2024-04-23T14:31:36.851 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 103/156 2024-04-23T14:31:36.903 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 104/156 2024-04-23T14:31:37.066 INFO:teuthology.orchestra.run.smithi203.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 65 kB/s | 22 kB 00:00 2024-04-23T14:31:37.400 INFO:teuthology.orchestra.run.smithi203.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 55 kB/s | 45 kB 00:00 2024-04-23T14:31:37.775 INFO:teuthology.orchestra.run.smithi203.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-04-23T14:31:37.800 INFO:teuthology.orchestra.run.smithi203.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-04-23T14:31:37.868 INFO:teuthology.orchestra.run.smithi203.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 292 kB/s | 19 kB 00:00 2024-04-23T14:31:38.126 INFO:teuthology.orchestra.run.smithi203.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 881 kB/s | 309 kB 00:00 2024-04-23T14:31:38.193 INFO:teuthology.orchestra.run.smithi203.stdout:(107/152): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-23T14:31:38.218 INFO:teuthology.orchestra.run.smithi203.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-23T14:31:38.260 INFO:teuthology.orchestra.run.smithi203.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 727 kB/s | 49 kB 00:00 2024-04-23T14:31:38.294 INFO:teuthology.orchestra.run.smithi203.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 889 kB/s | 67 kB 00:00 2024-04-23T14:31:38.328 INFO:teuthology.orchestra.run.smithi203.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-04-23T14:31:38.437 INFO:teuthology.orchestra.run.smithi203.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-04-23T14:31:38.479 INFO:teuthology.orchestra.run.smithi203.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.4 MB/s | 838 kB 00:00 2024-04-23T14:31:38.504 INFO:teuthology.orchestra.run.smithi203.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 651 kB/s | 43 kB 00:00 2024-04-23T14:31:38.546 INFO:teuthology.orchestra.run.smithi203.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-04-23T14:31:38.571 INFO:teuthology.orchestra.run.smithi203.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 223 kB/s | 15 kB 00:00 2024-04-23T14:31:38.621 INFO:teuthology.orchestra.run.smithi203.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-23T14:31:38.647 INFO:teuthology.orchestra.run.smithi203.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-04-23T14:31:38.714 INFO:teuthology.orchestra.run.smithi203.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-23T14:31:38.739 INFO:teuthology.orchestra.run.smithi203.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-04-23T14:31:38.798 INFO:teuthology.orchestra.run.smithi203.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-23T14:31:38.865 INFO:teuthology.orchestra.run.smithi203.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-04-23T14:31:38.890 INFO:teuthology.orchestra.run.smithi203.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-23T14:31:38.924 INFO:teuthology.orchestra.run.smithi203.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-04-23T14:31:38.949 INFO:teuthology.orchestra.run.smithi203.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-04-23T14:31:38.983 INFO:teuthology.orchestra.run.smithi203.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 394 kB/s | 23 kB 00:00 2024-04-23T14:31:39.008 INFO:teuthology.orchestra.run.smithi203.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-04-23T14:31:39.033 INFO:teuthology.orchestra.run.smithi203.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-04-23T14:31:39.092 INFO:teuthology.orchestra.run.smithi203.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 795 kB/s | 46 kB 00:00 2024-04-23T14:31:39.134 INFO:teuthology.orchestra.run.smithi203.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-04-23T14:31:39.159 INFO:teuthology.orchestra.run.smithi203.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-04-23T14:31:39.184 INFO:teuthology.orchestra.run.smithi203.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-23T14:31:39.227 INFO:teuthology.orchestra.run.smithi203.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-23T14:31:39.243 INFO:teuthology.orchestra.run.smithi203.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 279 kB/s | 16 kB 00:00 2024-04-23T14:31:39.286 INFO:teuthology.orchestra.run.smithi203.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-23T14:31:39.303 INFO:teuthology.orchestra.run.smithi203.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 479 kB/s | 28 kB 00:00 2024-04-23T14:31:39.336 INFO:teuthology.orchestra.run.smithi203.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 609 kB/s | 31 kB 00:00 2024-04-23T14:31:39.362 INFO:teuthology.orchestra.run.smithi203.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-04-23T14:31:39.387 INFO:teuthology.orchestra.run.smithi203.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-04-23T14:31:39.413 INFO:teuthology.orchestra.run.smithi203.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-04-23T14:31:39.446 INFO:teuthology.orchestra.run.smithi203.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 872 kB/s | 51 kB 00:00 2024-04-23T14:31:39.471 INFO:teuthology.orchestra.run.smithi203.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-23T14:31:39.538 INFO:teuthology.orchestra.run.smithi203.stdout:(143/152): python3-werkzeug-2.0.3-3.el9.1.noarc 6.3 MB/s | 427 kB 00:00 2024-04-23T14:31:39.589 INFO:teuthology.orchestra.run.smithi203.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 629 kB/s | 90 kB 00:00 2024-04-23T14:31:39.614 INFO:teuthology.orchestra.run.smithi203.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-23T14:31:39.673 INFO:teuthology.orchestra.run.smithi203.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-23T14:31:39.723 INFO:teuthology.orchestra.run.smithi203.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-23T14:31:39.811 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:31:39.858 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:31:40.516 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:31:40.570 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 106/156 2024-04-23T14:31:40.782 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 106/156 2024-04-23T14:31:40.783 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T14:31:40.783 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T14:31:40.783 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:31:40.829 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 107/156 2024-04-23T14:31:41.795 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 107/156 2024-04-23T14:31:41.795 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /sys 2024-04-23T14:31:41.795 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /proc 2024-04-23T14:31:41.795 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /mnt 2024-04-23T14:31:41.795 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /var/tmp 2024-04-23T14:31:41.795 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /home 2024-04-23T14:31:41.795 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /root 2024-04-23T14:31:41.795 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /tmp 2024-04-23T14:31:41.795 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:31:41.971 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 108/156 2024-04-23T14:31:41.986 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 108/156 2024-04-23T14:31:41.986 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:41.986 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T14:31:41.986 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:31:43.457 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 109/156 2024-04-23T14:31:43.482 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 109/156 2024-04-23T14:31:43.482 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:43.483 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T14:31:43.483 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T14:31:43.483 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T14:31:43.483 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:31:43.590 INFO:teuthology.orchestra.run.smithi203.stdout:(148/152): librbd1-19.0.0-3126.gd44fddd9.el9.x8 833 kB/s | 3.1 MB 00:03 2024-04-23T14:31:43.716 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 110/156 2024-04-23T14:31:43.738 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 110/156 2024-04-23T14:31:43.739 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:43.739 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T14:31:43.739 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T14:31:43.739 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T14:31:43.739 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:31:44.192 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 111/156 2024-04-23T14:31:44.214 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 111/156 2024-04-23T14:31:44.215 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:44.215 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T14:31:44.215 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T14:31:44.215 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T14:31:44.215 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:31:44.231 INFO:teuthology.orchestra.run.smithi203.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 2.1 MB/s | 1.3 MB 00:00 2024-04-23T14:31:44.322 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T14:31:44.379 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T14:31:44.435 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T14:31:44.623 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T14:31:44.676 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T14:31:44.707 INFO:teuthology.orchestra.run.smithi203.stdout:(150/152): librados2-19.0.0-3126.gd44fddd9.el9. 690 kB/s | 3.4 MB 00:05 2024-04-23T14:31:44.754 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T14:31:44.844 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T14:31:44.907 INFO:teuthology.orchestra.run.smithi203.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-04-23T14:31:44.985 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T14:31:45.025 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T14:31:45.057 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T14:31:45.085 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T14:31:45.461 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 123/156 2024-04-23T14:31:45.500 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 123/156 2024-04-23T14:31:45.532 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.e 124/156 2024-04-23T14:31:45.602 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9. 125/156 2024-04-23T14:31:45.632 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 126/156 2024-04-23T14:31:46.680 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 126/156 2024-04-23T14:31:46.717 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 127/156 2024-04-23T14:31:47.686 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 127/156 2024-04-23T14:31:47.717 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 128/156 2024-04-23T14:31:47.820 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 128/156 2024-04-23T14:31:47.982 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el 129/156 2024-04-23T14:31:48.011 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 130/156 2024-04-23T14:31:48.034 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 130/156 2024-04-23T14:31:48.034 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:48.034 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T14:31:48.034 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T14:31:48.034 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T14:31:48.034 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:31:48.078 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 131/156 2024-04-23T14:31:48.090 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 131/156 2024-04-23T14:31:48.188 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 132/156 2024-04-23T14:31:48.244 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 133/156 2024-04-23T14:31:49.173 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T14:31:49.205 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 135/156 2024-04-23T14:31:49.228 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 135/156 2024-04-23T14:31:49.228 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:49.228 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T14:31:49.228 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:31:49.228 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:31:49.229 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:31:49.273 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 136/156 2024-04-23T14:31:49.295 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 136/156 2024-04-23T14:31:49.295 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:49.295 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T14:31:49.295 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:31:49.340 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 137/156 2024-04-23T14:31:49.362 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 137/156 2024-04-23T14:31:49.363 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:49.363 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T14:31:49.363 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:31:49.363 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:31:49.363 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:31:49.666 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 138/156 2024-04-23T14:31:49.689 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 138/156 2024-04-23T14:31:49.689 INFO:teuthology.orchestra.run.smithi173.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:49.689 INFO:teuthology.orchestra.run.smithi173.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T14:31:49.689 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T14:31:49.689 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T14:31:49.689 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:31:54.046 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 107/156 2024-04-23T14:31:54.047 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-04-23T14:31:54.047 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-04-23T14:31:54.047 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-04-23T14:31:54.047 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-04-23T14:31:54.047 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-04-23T14:31:54.047 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-04-23T14:31:54.047 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-04-23T14:31:54.047 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:31:54.287 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 108/156 2024-04-23T14:31:54.302 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 139/156 2024-04-23T14:31:54.333 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 108/156 2024-04-23T14:31:54.333 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:54.333 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T14:31:54.333 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:31:54.344 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 140/156 2024-04-23T14:31:54.385 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 141/156 2024-04-23T14:31:54.416 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T14:31:54.444 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T14:31:54.636 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 144/156 2024-04-23T14:31:54.682 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T14:31:54.711 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_ 146/156 2024-04-23T14:31:54.764 INFO:teuthology.orchestra.run.smithi203.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 723 kB/s | 16 MB 00:23 2024-04-23T14:31:54.769 INFO:teuthology.orchestra.run.smithi203.stdout:-------------------------------------------------------------------------------- 2024-04-23T14:31:54.769 INFO:teuthology.orchestra.run.smithi203.stdout:Total 2.9 MB/s | 222 MB 01:17 2024-04-23T14:31:55.337 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:31:55.376 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:31:55.396 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:31:55.882 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction check 2024-04-23T14:31:56.022 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 109/156 2024-04-23T14:31:56.027 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction check succeeded. 2024-04-23T14:31:56.027 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction test 2024-04-23T14:31:56.126 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 109/156 2024-04-23T14:31:56.127 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:56.127 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T14:31:56.127 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T14:31:56.127 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T14:31:56.127 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:31:56.237 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 148/156 2024-04-23T14:31:56.322 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T14:31:56.363 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T14:31:56.398 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 110/156 2024-04-23T14:31:56.446 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T14:31:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T14:31:56.476 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T14:31:56.491 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 110/156 2024-04-23T14:31:56.491 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:56.492 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T14:31:56.492 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T14:31:56.492 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T14:31:56.492 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:31:56.543 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T14:31:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T14:31:56.568 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T14:31:56.642 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T14:31:56.708 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:31:56.998 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 111/156 2024-04-23T14:31:57.059 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 111/156 2024-04-23T14:31:57.059 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:31:57.059 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T14:31:57.059 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T14:31:57.059 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T14:31:57.059 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:31:57.190 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T14:31:57.253 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T14:31:57.327 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T14:31:57.556 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T14:31:57.612 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T14:31:57.696 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T14:31:57.794 INFO:teuthology.orchestra.run.smithi203.stdout:Transaction test succeeded. 2024-04-23T14:31:57.794 INFO:teuthology.orchestra.run.smithi203.stdout:Running transaction 2024-04-23T14:31:57.805 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T14:31:57.981 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T14:31:58.044 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T14:31:58.085 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T14:31:58.127 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T14:31:58.784 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 123/156 2024-04-23T14:31:58.823 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 123/156 2024-04-23T14:31:58.877 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.e 124/156 2024-04-23T14:31:58.987 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9. 125/156 2024-04-23T14:31:59.024 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 126/156 2024-04-23T14:32:00.332 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 126/156 2024-04-23T14:32:00.409 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 127/156 2024-04-23T14:32:01.506 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 127/156 2024-04-23T14:32:01.534 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 128/156 2024-04-23T14:32:01.676 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 128/156 2024-04-23T14:32:01.849 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el 129/156 2024-04-23T14:32:01.878 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 130/156 2024-04-23T14:32:01.945 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 130/156 2024-04-23T14:32:01.945 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:01.945 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T14:32:01.945 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T14:32:01.945 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T14:32:01.945 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:32:02.004 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 131/156 2024-04-23T14:32:02.040 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 131/156 2024-04-23T14:32:02.178 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 132/156 2024-04-23T14:32:02.257 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 133/156 2024-04-23T14:32:02.729 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:32:02.729 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 1/156 2024-04-23T14:32:02.729 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 2/156 2024-04-23T14:32:02.729 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 3/156 2024-04-23T14:32:02.729 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 4/156 2024-04-23T14:32:02.729 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 5/156 2024-04-23T14:32:02.729 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 6/156 2024-04-23T14:32:02.729 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 7/156 2024-04-23T14:32:02.729 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:32:02.729 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 10/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 11/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 12/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 14/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_ 15/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 16/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 17/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_6 18/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 19/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 20/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el 21/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9. 22/156 2024-04-23T14:32:02.730 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_6 23/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 24/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 25/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 26/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 27/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 28/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 29/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9. 30/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 31/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 32/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 33/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el 34/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 35/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.e 36/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 37/156 2024-04-23T14:32:02.731 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 38/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 39/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T14:32:02.732 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T14:32:02.733 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T14:32:02.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T14:32:02.735 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T14:32:02.736 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T14:32:02.737 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T14:32:02.738 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 149/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 151/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T14:32:02.739 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T14:32:02.740 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T14:32:03.264 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T14:32:03.298 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 135/156 2024-04-23T14:32:03.369 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 135/156 2024-04-23T14:32:03.369 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:03.369 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T14:32:03.369 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:32:03.369 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:32:03.370 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:32:03.424 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 136/156 2024-04-23T14:32:03.492 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 136/156 2024-04-23T14:32:03.492 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:03.492 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T14:32:03.492 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:32:03.550 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 137/156 2024-04-23T14:32:03.620 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 137/156 2024-04-23T14:32:03.621 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:03.621 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T14:32:03.621 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:32:03.621 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:32:03.621 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:32:03.837 INFO:teuthology.orchestra.run.smithi203.stdout: Preparing : 1/1 2024-04-23T14:32:03.926 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T14:32:03.973 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T14:32:03.975 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 138/156 2024-04-23T14:32:04.026 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T14:32:04.048 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 138/156 2024-04-23T14:32:04.048 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:04.048 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T14:32:04.048 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T14:32:04.048 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T14:32:04.049 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:32:04.061 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T14:32:04.086 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T14:32:04.129 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T14:32:04.457 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T14:32:04.587 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:32:04.695 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:32:04.730 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:32:04.773 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:32:04.773 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:32:04.773 INFO:teuthology.orchestra.run.smithi173.stdout:Upgraded: 2024-04-23T14:32:04.773 INFO:teuthology.orchestra.run.smithi173.stdout: librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.773 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T14:32:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T14:32:04.774 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2024-04-23T14:32:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T14:32:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T14:32:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T14:32:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T14:32:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.774 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.775 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T14:32:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T14:32:04.776 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T14:32:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T14:32:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T14:32:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T14:32:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T14:32:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T14:32:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T14:32:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T14:32:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T14:32:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.777 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T14:32:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T14:32:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T14:32:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T14:32:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T14:32:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T14:32:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T14:32:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T14:32:04.778 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T14:32:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T14:32:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T14:32:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T14:32:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T14:32:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T14:32:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T14:32:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T14:32:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T14:32:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T14:32:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T14:32:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T14:32:04.779 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T14:32:04.780 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T14:32:04.780 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T14:32:04.780 INFO:teuthology.orchestra.run.smithi173.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T14:32:04.780 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T14:32:04.780 INFO:teuthology.orchestra.run.smithi173.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T14:32:04.780 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T14:32:04.780 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T14:32:04.780 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T14:32:04.780 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.780 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.780 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.781 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T14:32:04.781 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T14:32:04.781 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T14:32:04.781 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T14:32:04.781 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T14:32:04.781 INFO:teuthology.orchestra.run.smithi173.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T14:32:04.781 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T14:32:04.781 INFO:teuthology.orchestra.run.smithi173.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T14:32:04.781 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T14:32:04.781 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T14:32:04.781 INFO:teuthology.orchestra.run.smithi173.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T14:32:04.781 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T14:32:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T14:32:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T14:32:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T14:32:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T14:32:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T14:32:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T14:32:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T14:32:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T14:32:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T14:32:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T14:32:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T14:32:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T14:32:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T14:32:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T14:32:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T14:32:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T14:32:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T14:32:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T14:32:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T14:32:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T14:32:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T14:32:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T14:32:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T14:32:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T14:32:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T14:32:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T14:32:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T14:32:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T14:32:04.787 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T14:32:04.787 INFO:teuthology.orchestra.run.smithi173.stdout: 2024-04-23T14:32:04.787 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2024-04-23T14:32:04.804 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:32:04.843 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 10/156 2024-04-23T14:32:04.994 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T14:32:05.049 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T14:32:05.077 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:32:05.174 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:05.179 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:32:05.203 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 14/156 2024-04-23T14:32:05.265 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 14/156 2024-04-23T14:32:05.288 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T14:32:05.310 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T14:32:05.504 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T14:32:05.561 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T14:32:05.669 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T14:32:05.760 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T14:32:05.812 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T14:32:05.847 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T14:32:05.921 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T14:32:05.963 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T14:32:06.074 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T14:32:06.252 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T14:32:06.292 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T14:32:06.363 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T14:32:06.424 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T14:32:06.454 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T14:32:06.497 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T14:32:06.575 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T14:32:06.608 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9. 31/156 2024-04-23T14:32:06.652 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el 32/156 2024-04-23T14:32:06.688 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_6 33/156 2024-04-23T14:32:06.760 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T14:32:06.812 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T14:32:06.888 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T14:32:07.033 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T14:32:07.090 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T14:32:07.636 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T14:32:07.720 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T14:32:07.748 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T14:32:07.799 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T14:32:07.845 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_6 43/156 2024-04-23T14:32:07.888 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T14:32:08.038 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T14:32:08.090 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T14:32:08.146 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T14:32:08.220 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T14:32:08.250 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T14:32:08.286 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T14:32:08.342 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T14:32:08.377 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T14:32:08.421 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T14:32:08.463 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T14:32:08.604 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T14:32:08.660 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T14:32:08.691 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T14:32:08.732 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T14:32:08.836 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T14:32:08.874 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T14:32:08.914 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T14:32:08.940 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T14:32:08.964 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T14:32:09.033 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 139/156 2024-04-23T14:32:09.040 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T14:32:09.097 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 140/156 2024-04-23T14:32:09.168 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T14:32:09.185 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 141/156 2024-04-23T14:32:09.206 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T14:32:09.227 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T14:32:09.234 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T14:32:09.275 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T14:32:09.293 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T14:32:09.333 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T14:32:09.364 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T14:32:09.394 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T14:32:09.428 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T14:32:09.462 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T14:32:09.504 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T14:32:09.538 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T14:32:09.632 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 144/156 2024-04-23T14:32:09.635 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T14:32:09.686 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T14:32:09.703 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T14:32:09.717 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T14:32:09.731 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_ 146/156 2024-04-23T14:32:09.764 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T14:32:09.864 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T14:32:09.919 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T14:32:10.652 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T14:32:10.664 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:32:10.700 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T14:32:10.712 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:32:10.733 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T14:32:10.777 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:32:10.834 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T14:32:10.901 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T14:32:10.988 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T14:32:11.027 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T14:32:11.205 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T14:32:11.234 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T14:32:11.312 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T14:32:11.966 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T14:32:12.085 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T14:32:12.120 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 148/156 2024-04-23T14:32:12.295 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T14:32:12.387 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T14:32:12.579 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T14:32:12.627 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T14:32:12.627 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T14:32:12.787 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T14:32:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T14:32:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T14:32:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T14:32:13.139 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:32:13.326 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T14:32:13.423 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T14:32:13.441 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:32:13.441 INFO:teuthology.orchestra.run.smithi083.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-23T14:32:13.441 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:32:13.451 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T14:32:13.484 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T14:32:13.795 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T14:32:13.839 INFO:teuthology.orchestra.run.smithi203.stdout: Upgrading : librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 99/156 2024-04-23T14:32:13.929 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 99/156 2024-04-23T14:32:13.961 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 100/156 2024-04-23T14:32:14.005 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T14:32:14.486 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T14:32:14.518 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 103/156 2024-04-23T14:32:14.589 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 103/156 2024-04-23T14:32:14.611 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 104/156 2024-04-23T14:32:16.851 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:32:16.891 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:32:17.437 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 105/156 2024-04-23T14:32:17.479 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 106/156 2024-04-23T14:32:17.570 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 106/156 2024-04-23T14:32:17.570 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T14:32:17.570 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T14:32:17.570 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:17.597 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 107/156 2024-04-23T14:32:19.076 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 1/156 2024-04-23T14:32:19.076 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 2/156 2024-04-23T14:32:19.077 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 3/156 2024-04-23T14:32:19.077 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 4/156 2024-04-23T14:32:19.077 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 5/156 2024-04-23T14:32:19.077 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 6/156 2024-04-23T14:32:19.077 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 7/156 2024-04-23T14:32:19.077 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:32:19.077 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:32:19.077 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 10/156 2024-04-23T14:32:19.077 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 11/156 2024-04-23T14:32:19.077 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 12/156 2024-04-23T14:32:19.077 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:32:19.077 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 14/156 2024-04-23T14:32:19.078 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_ 15/156 2024-04-23T14:32:19.078 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 16/156 2024-04-23T14:32:19.078 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 17/156 2024-04-23T14:32:19.078 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_6 18/156 2024-04-23T14:32:19.078 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 19/156 2024-04-23T14:32:19.078 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 20/156 2024-04-23T14:32:19.078 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el 21/156 2024-04-23T14:32:19.078 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9. 22/156 2024-04-23T14:32:19.078 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_6 23/156 2024-04-23T14:32:19.078 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 24/156 2024-04-23T14:32:19.078 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 25/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 26/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 27/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 28/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 29/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9. 30/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 31/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 32/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 33/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el 34/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 35/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.e 36/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 37/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 38/156 2024-04-23T14:32:19.079 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 39/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T14:32:19.080 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T14:32:19.081 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T14:32:19.082 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T14:32:19.082 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T14:32:19.082 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T14:32:19.082 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T14:32:19.082 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T14:32:19.082 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T14:32:19.082 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T14:32:19.082 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T14:32:19.082 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T14:32:19.082 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T14:32:19.083 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T14:32:19.084 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T14:32:19.085 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T14:32:19.086 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T14:32:19.087 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T14:32:19.088 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T14:32:19.088 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 149/156 2024-04-23T14:32:19.088 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T14:32:19.088 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 151/156 2024-04-23T14:32:19.088 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T14:32:19.088 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T14:32:19.088 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T14:32:19.088 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T14:32:21.464 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:32:21.464 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.465 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:21.466 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T14:32:21.467 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.468 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T14:32:21.469 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.470 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T14:32:21.471 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T14:32:21.472 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.473 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T14:32:21.474 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T14:32:21.475 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T14:32:21.476 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T14:32:21.476 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:32:21.476 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-23T14:32:22.084 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:28.991 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 107/156 2024-04-23T14:32:28.992 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /sys 2024-04-23T14:32:28.992 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /proc 2024-04-23T14:32:28.992 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /mnt 2024-04-23T14:32:28.992 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /var/tmp 2024-04-23T14:32:28.992 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /home 2024-04-23T14:32:28.992 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /root 2024-04-23T14:32:28.992 INFO:teuthology.orchestra.run.smithi203.stdout:skipping the directory /tmp 2024-04-23T14:32:28.992 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:29.172 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 108/156 2024-04-23T14:32:29.192 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 108/156 2024-04-23T14:32:29.192 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:29.192 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T14:32:29.192 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:30.676 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 109/156 2024-04-23T14:32:30.702 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 109/156 2024-04-23T14:32:30.702 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:30.702 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T14:32:30.702 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T14:32:30.702 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T14:32:30.702 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:30.927 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 110/156 2024-04-23T14:32:30.949 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 110/156 2024-04-23T14:32:30.950 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:30.950 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T14:32:30.950 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T14:32:30.950 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T14:32:30.950 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:31.394 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 111/156 2024-04-23T14:32:31.416 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 111/156 2024-04-23T14:32:31.416 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:31.417 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T14:32:31.417 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T14:32:31.417 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T14:32:31.417 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:31.516 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T14:32:31.581 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T14:32:31.645 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T14:32:31.832 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T14:32:31.887 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T14:32:31.974 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T14:32:32.071 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T14:32:32.229 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T14:32:32.263 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T14:32:32.303 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T14:32:32.339 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T14:32:32.792 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 123/156 2024-04-23T14:32:32.821 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 123/156 2024-04-23T14:32:32.853 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.e 124/156 2024-04-23T14:32:32.924 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9. 125/156 2024-04-23T14:32:32.953 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 126/156 2024-04-23T14:32:33.992 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 126/156 2024-04-23T14:32:34.030 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 127/156 2024-04-23T14:32:34.991 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 127/156 2024-04-23T14:32:35.030 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 128/156 2024-04-23T14:32:35.134 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 128/156 2024-04-23T14:32:35.293 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el 129/156 2024-04-23T14:32:35.333 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 130/156 2024-04-23T14:32:35.356 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 130/156 2024-04-23T14:32:35.357 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:35.357 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T14:32:35.357 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T14:32:35.357 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T14:32:35.357 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:35.402 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 131/156 2024-04-23T14:32:35.414 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 131/156 2024-04-23T14:32:35.512 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 132/156 2024-04-23T14:32:35.568 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 133/156 2024-04-23T14:32:36.494 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T14:32:36.522 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 135/156 2024-04-23T14:32:36.545 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 135/156 2024-04-23T14:32:36.545 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:36.545 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T14:32:36.545 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:32:36.545 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T14:32:36.545 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:36.597 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 136/156 2024-04-23T14:32:36.619 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 136/156 2024-04-23T14:32:36.620 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:36.620 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T14:32:36.620 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:36.673 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 137/156 2024-04-23T14:32:36.695 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 137/156 2024-04-23T14:32:36.695 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:36.695 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T14:32:36.695 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:32:36.695 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T14:32:36.695 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:36.999 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 138/156 2024-04-23T14:32:37.021 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 138/156 2024-04-23T14:32:37.022 INFO:teuthology.orchestra.run.smithi203.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T14:32:37.022 INFO:teuthology.orchestra.run.smithi203.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T14:32:37.022 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T14:32:37.022 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T14:32:37.022 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:41.686 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 139/156 2024-04-23T14:32:41.727 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 140/156 2024-04-23T14:32:41.768 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 141/156 2024-04-23T14:32:41.800 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T14:32:41.836 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T14:32:42.033 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 144/156 2024-04-23T14:32:42.074 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T14:32:42.103 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_ 146/156 2024-04-23T14:32:42.726 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:32:42.767 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:32:42.788 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T14:32:43.627 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 148/156 2024-04-23T14:32:43.714 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T14:32:43.756 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T14:32:43.838 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T14:32:43.868 INFO:teuthology.orchestra.run.smithi203.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T14:32:43.868 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T14:32:43.961 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T14:32:44.002 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T14:32:44.002 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T14:32:44.092 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T14:32:44.158 INFO:teuthology.orchestra.run.smithi203.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:32:50.380 INFO:teuthology.orchestra.run.smithi203.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:32:50.381 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 1/156 2024-04-23T14:32:50.381 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 2/156 2024-04-23T14:32:50.381 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 3/156 2024-04-23T14:32:50.381 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 4/156 2024-04-23T14:32:50.381 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3126.gd44fd 5/156 2024-04-23T14:32:50.381 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 6/156 2024-04-23T14:32:50.381 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 7/156 2024-04-23T14:32:50.381 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 8/156 2024-04-23T14:32:50.381 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 9/156 2024-04-23T14:32:50.381 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 10/156 2024-04-23T14:32:50.381 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 11/156 2024-04-23T14:32:50.381 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 12/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 13/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 14/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_ 15/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 16/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 17/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_6 18/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86 19/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 20/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el 21/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9. 22/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_6 23/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 24/156 2024-04-23T14:32:50.382 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 25/156 2024-04-23T14:32:50.383 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 26/156 2024-04-23T14:32:50.383 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 27/156 2024-04-23T14:32:50.383 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 28/156 2024-04-23T14:32:50.383 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 29/156 2024-04-23T14:32:50.395 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9. 30/156 2024-04-23T14:32:50.395 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noa 31/156 2024-04-23T14:32:50.395 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.n 32/156 2024-04-23T14:32:50.395 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44 33/156 2024-04-23T14:32:50.395 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el 34/156 2024-04-23T14:32:50.395 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 35/156 2024-04-23T14:32:50.395 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.e 36/156 2024-04-23T14:32:50.396 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 37/156 2024-04-23T14:32:50.396 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 38/156 2024-04-23T14:32:50.396 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 39/156 2024-04-23T14:32:50.396 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T14:32:50.396 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T14:32:50.396 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T14:32:50.396 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T14:32:50.396 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T14:32:50.396 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T14:32:50.396 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T14:32:50.396 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T14:32:50.396 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T14:32:50.397 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T14:32:50.398 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T14:32:50.399 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T14:32:50.400 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T14:32:50.401 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T14:32:50.402 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T14:32:50.403 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T14:32:50.404 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T14:32:50.404 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T14:32:50.404 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 149/156 2024-04-23T14:32:50.404 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T14:32:50.404 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 151/156 2024-04-23T14:32:50.404 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T14:32:50.404 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T14:32:50.404 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T14:32:50.404 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T14:32:52.465 INFO:teuthology.orchestra.run.smithi203.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T14:32:52.465 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:52.465 INFO:teuthology.orchestra.run.smithi203.stdout:Upgraded: 2024-04-23T14:32:52.465 INFO:teuthology.orchestra.run.smithi203.stdout: librados2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.465 INFO:teuthology.orchestra.run.smithi203.stdout: librbd1-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.465 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T14:32:52.465 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T14:32:52.465 INFO:teuthology.orchestra.run.smithi203.stdout:Installed: 2024-04-23T14:32:52.465 INFO:teuthology.orchestra.run.smithi203.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T14:32:52.465 INFO:teuthology.orchestra.run.smithi203.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T14:32:52.465 INFO:teuthology.orchestra.run.smithi203.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T14:32:52.466 INFO:teuthology.orchestra.run.smithi203.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T14:32:52.466 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.466 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.466 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.466 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.466 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-grafana-dashboards-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:52.466 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-immutable-object-cache-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.466 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mds-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.466 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.466 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:52.466 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:52.466 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-rook-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mon-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-osd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-prometheus-alerts-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-radosgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-selinux-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-test-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-volume-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: cephadm-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-shell-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-top-2:19.0.0-3126.gd44fddd9.el9.noarch 2024-04-23T14:32:52.467 INFO:teuthology.orchestra.run.smithi203.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs-devel-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: libcephfs2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: libcephsqlite-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T14:32:52.468 INFO:teuthology.orchestra.run.smithi203.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: librados-devel-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: librgw2-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T14:32:52.469 INFO:teuthology.orchestra.run.smithi203.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T14:32:52.470 INFO:teuthology.orchestra.run.smithi203.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-common-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cephfs-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T14:32:52.471 INFO:teuthology.orchestra.run.smithi203.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T14:32:52.472 INFO:teuthology.orchestra.run.smithi203.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.473 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rgw-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T14:32:52.474 INFO:teuthology.orchestra.run.smithi203.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-fuse-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-mirror-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: rbd-nbd-2:19.0.0-3126.gd44fddd9.el9.x86_64 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T14:32:52.475 INFO:teuthology.orchestra.run.smithi203.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T14:32:52.476 INFO:teuthology.orchestra.run.smithi203.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T14:32:52.476 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:32:52.476 INFO:teuthology.orchestra.run.smithi203.stdout:Complete! 2024-04-23T14:32:52.861 DEBUG:teuthology.parallel:result is None 2024-04-23T14:32:52.862 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:32:53.005 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T14:32:53.076 INFO:teuthology.orchestra.run.smithi083.stdout:19.0.0-3126.gd44fddd9.el9 2024-04-23T14:32:53.076 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3126.gd44fddd9.el9 2024-04-23T14:32:53.076 INFO:teuthology.task.install:The correct ceph version 19.0.0-3126.gd44fddd9 is installed. 2024-04-23T14:32:53.078 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:32:53.217 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T14:32:53.242 INFO:teuthology.orchestra.run.smithi173.stdout:19.0.0-3126.gd44fddd9.el9 2024-04-23T14:32:53.242 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3126.gd44fddd9.el9 2024-04-23T14:32:53.242 INFO:teuthology.task.install:The correct ceph version 19.0.0-3126.gd44fddd9 is installed. 2024-04-23T14:32:53.244 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:32:53.480 DEBUG:teuthology.orchestra.run.smithi203:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T14:32:53.501 INFO:teuthology.orchestra.run.smithi203.stdout:19.0.0-3126.gd44fddd9.el9 2024-04-23T14:32:53.501 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3126.gd44fddd9.el9 2024-04-23T14:32:53.501 INFO:teuthology.task.install:The correct ceph version 19.0.0-3126.gd44fddd9 is installed. 2024-04-23T14:32:53.503 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T14:32:53.503 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:32:53.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T14:32:53.597 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T14:32:53.597 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T14:32:53.627 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-23T14:32:53.628 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T14:32:53.655 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T14:32:53.655 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:32:53.655 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T14:32:53.743 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T14:32:53.824 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T14:32:53.824 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T14:32:53.853 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T14:32:53.920 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-23T14:32:53.920 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T14:32:53.945 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T14:32:54.010 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T14:32:54.010 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:32:54.010 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T14:32:54.099 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T14:32:54.223 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T14:32:54.223 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T14:32:54.247 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T14:32:54.310 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-23T14:32:54.310 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T14:32:54.333 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T14:32:54.397 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T14:32:54.398 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:32:54.398 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T14:32:54.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T14:32:54.565 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T14:32:54.565 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T14:32:54.589 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T14:32:54.655 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-23T14:32:54.655 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T14:32:54.679 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T14:32:54.743 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-23T14:32:54.834 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd heartbeat grace': 60, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request', 'SLOW_OPS', 'slow request'], 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-23T14:32:54.835 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:32:54.835 INFO:tasks.cephadm:Cluster fsid is 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:32:54.835 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-23T14:32:54.835 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.83', 'mon.b': '172.21.15.173', 'mon.c': '172.21.15.203'} 2024-04-23T14:32:54.835 INFO:tasks.cephadm:First mon is mon.a on smithi083 2024-04-23T14:32:54.836 INFO:tasks.cephadm:First mgr is x 2024-04-23T14:32:54.836 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-23T14:32:54.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname $(hostname -s) 2024-04-23T14:32:54.923 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname $(hostname -s) 2024-04-23T14:32:54.947 DEBUG:teuthology.orchestra.run.smithi203:> sudo hostname $(hostname -s) 2024-04-23T14:32:54.971 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-23T14:32:54.971 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:32:55.203 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd44fddd910b5e94da0f2357af4be37d7bd495ae0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78800/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3126-gd44fddd9', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3126.gd44fddd9'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 14:41:17.415668', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240416.193735-3', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-23T14:32:55.433 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240416.193735-3, sha1 d44fddd910b5e94da0f2357af4be37d7bd495ae0 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:32:55.435 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/x86_64/flavors/default/cephadm 2024-04-23T14:32:55.435 DEBUG:teuthology.orchestra.run.smithi083:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T14:32:56.113 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r--. 1 ubuntu ubuntu 781447 Apr 23 14:32 /home/ubuntu/cephtest/cephadm 2024-04-23T14:32:56.113 DEBUG:teuthology.orchestra.run.smithi173:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T14:32:56.697 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r--. 1 ubuntu ubuntu 781447 Apr 23 14:32 /home/ubuntu/cephtest/cephadm 2024-04-23T14:32:56.697 DEBUG:teuthology.orchestra.run.smithi203:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240416.193735-3/d44fddd910b5e94da0f2357af4be37d7bd495ae0/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T14:32:57.385 INFO:teuthology.orchestra.run.smithi203.stdout:-rw-r--r--. 1 ubuntu ubuntu 781447 Apr 23 14:32 /home/ubuntu/cephtest/cephadm 2024-04-23T14:32:57.386 DEBUG:teuthology.orchestra.run.smithi083:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T14:32:57.446 DEBUG:teuthology.orchestra.run.smithi173:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T14:32:57.465 DEBUG:teuthology.orchestra.run.smithi203:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T14:32:57.499 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 on all hosts... 2024-04-23T14:32:57.499 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 pull 2024-04-23T14:32:57.506 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 pull 2024-04-23T14:32:57.509 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 pull 2024-04-23T14:32:58.988 INFO:teuthology.orchestra.run.smithi173.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0... 2024-04-23T14:32:59.232 INFO:teuthology.orchestra.run.smithi203.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0... 2024-04-23T14:32:59.512 INFO:teuthology.orchestra.run.smithi083.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0... 2024-04-23T14:35:50.272 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-23T14:35:50.272 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:35:50.272 INFO:teuthology.orchestra.run.smithi173.stdout: "image_id": "76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e", 2024-04-23T14:35:50.272 INFO:teuthology.orchestra.run.smithi173.stdout: "repo_digests": [ 2024-04-23T14:35:50.273 INFO:teuthology.orchestra.run.smithi173.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd" 2024-04-23T14:35:50.273 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2024-04-23T14:35:50.273 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-23T14:36:25.241 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:36:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:36:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "image_id": "76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e", 2024-04-23T14:36:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "repo_digests": [ 2024-04-23T14:36:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd" 2024-04-23T14:36:25.242 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-23T14:36:25.242 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:36:28.701 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-04-23T14:36:28.702 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev)", 2024-04-23T14:36:28.702 INFO:teuthology.orchestra.run.smithi203.stdout: "image_id": "76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e", 2024-04-23T14:36:28.702 INFO:teuthology.orchestra.run.smithi203.stdout: "repo_digests": [ 2024-04-23T14:36:28.702 INFO:teuthology.orchestra.run.smithi203.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd" 2024-04-23T14:36:28.702 INFO:teuthology.orchestra.run.smithi203.stdout: ] 2024-04-23T14:36:28.702 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-04-23T14:36:28.727 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph 2024-04-23T14:36:28.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph 2024-04-23T14:36:28.852 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph 2024-04-23T14:36:28.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /etc/ceph 2024-04-23T14:36:28.966 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /etc/ceph 2024-04-23T14:36:28.993 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /etc/ceph 2024-04-23T14:36:29.016 INFO:tasks.cephadm:Writing seed config... 2024-04-23T14:36:29.018 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-23T14:36:29.018 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-23T14:36:29.018 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-23T14:36:29.018 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-23T14:36:29.018 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-23T14:36:29.018 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-23T14:36:29.018 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-23T14:36:29.018 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-23T14:36:29.018 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-23T14:36:29.018 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-23T14:36:29.018 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-23T14:36:29.019 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [osd] filestore flush min = 0 2024-04-23T14:36:29.020 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-23T14:36:29.021 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-23T14:36:29.021 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-23T14:36:29.021 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-23T14:36:29.021 INFO:tasks.cephadm: override: [osd] osd heartbeat grace = 60 2024-04-23T14:36:29.021 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-23T14:36:29.021 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-23T14:36:29.021 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-23T14:36:29.021 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-23T14:36:29.022 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:36:29.022 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-23T14:36:29.079 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5f4fe708-017e-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 filestore flush min = 0 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd heartbeat grace = 60 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-23T14:36:29.079 DEBUG:teuthology.orchestra.run.smithi083:mon.a> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.a.service 2024-04-23T14:36:29.124 DEBUG:teuthology.orchestra.run.smithi083:mgr.x> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.x.service 2024-04-23T14:36:29.170 INFO:tasks.cephadm:Bootstrapping... 2024-04-23T14:36:29.170 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 -v bootstrap --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.83 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-23T14:36:29.439 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-23T14:36:29.439 INFO:teuthology.orchestra.run.smithi083.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0', '-v', 'bootstrap', '--fsid', '5f4fe708-017e-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.83', '--skip-admin-label'] 2024-04-23T14:36:29.440 INFO:teuthology.orchestra.run.smithi083.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-23T14:36:29.440 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying podman|docker is present... 2024-04-23T14:36:29.498 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T14:36:29.499 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying lvm2 is present... 2024-04-23T14:36:29.499 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying time synchronization is in place... 2024-04-23T14:36:29.516 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T14:36:29.516 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T14:36:29.530 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T14:36:29.530 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-23T14:36:29.552 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-04-23T14:36:29.575 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-04-23T14:36:29.575 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-04-23T14:36:29.575 INFO:teuthology.orchestra.run.smithi083.stdout:Repeating the final host check... 2024-04-23T14:36:29.634 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T14:36:29.634 INFO:teuthology.orchestra.run.smithi083.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-23T14:36:29.634 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl is present 2024-04-23T14:36:29.634 INFO:teuthology.orchestra.run.smithi083.stdout:lvcreate is present 2024-04-23T14:36:29.649 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T14:36:29.649 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T14:36:29.662 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T14:36:29.662 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-23T14:36:29.678 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout enabled 2024-04-23T14:36:29.695 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout active 2024-04-23T14:36:29.695 INFO:teuthology.orchestra.run.smithi083.stdout:Unit chronyd.service is enabled and running 2024-04-23T14:36:29.695 INFO:teuthology.orchestra.run.smithi083.stdout:Host looks OK 2024-04-23T14:36:29.695 INFO:teuthology.orchestra.run.smithi083.stdout:Cluster fsid: 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:36:29.695 INFO:teuthology.orchestra.run.smithi083.stdout:Acquiring lock 140542552927392 on /run/cephadm/5f4fe708-017e-11ef-bc93-c7b262605968.lock 2024-04-23T14:36:29.695 INFO:teuthology.orchestra.run.smithi083.stdout:Lock 140542552927392 acquired on /run/cephadm/5f4fe708-017e-11ef-bc93-c7b262605968.lock 2024-04-23T14:36:29.695 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 3300 ... 2024-04-23T14:36:29.696 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying IP 172.21.15.83 port 6789 ... 2024-04-23T14:36:29.696 INFO:teuthology.orchestra.run.smithi083.stdout:Base mon IP(s) is [172.21.15.83:3300, 172.21.15.83:6789], mon addrv is [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-04-23T14:36:29.705 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.83 metric 100 2024-04-23T14:36:29.705 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.83 metric 100 2024-04-23T14:36:29.712 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-23T14:36:29.713 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-23T14:36:29.713 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1767sec hoplimit 64 pref medium 2024-04-23T14:36:29.720 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-23T14:36:29.720 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-23T14:36:29.720 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T14:36:29.720 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-23T14:36:29.720 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-23T14:36:29.720 INFO:teuthology.orchestra.run.smithi083.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T14:36:29.721 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-04-23T14:36:29.721 INFO:teuthology.orchestra.run.smithi083.stdout:Mon IP `172.21.15.83` is in CIDR network `172.21.0.0/20` 2024-04-23T14:36:29.721 INFO:teuthology.orchestra.run.smithi083.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-23T14:36:29.722 INFO:teuthology.orchestra.run.smithi083.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-23T14:36:29.724 INFO:teuthology.orchestra.run.smithi083.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0... 2024-04-23T14:36:30.647 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stdout 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e 2024-04-23T14:36:30.648 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0... 2024-04-23T14:36:30.648 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Getting image source signatures 2024-04-23T14:36:30.648 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:871ee18b3c7f87d5ff3fbc84e47e46994673cad1275c5c818c1e7168af8e5da2 2024-04-23T14:36:30.648 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-23T14:36:30.648 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Copying config sha256:76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e 2024-04-23T14:36:30.648 INFO:teuthology.orchestra.run.smithi083.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-23T14:36:31.948 INFO:teuthology.orchestra.run.smithi083.stdout:ceph: stdout ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev) 2024-04-23T14:36:31.949 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph version: ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev) 2024-04-23T14:36:31.949 INFO:teuthology.orchestra.run.smithi083.stdout:Extracting ceph user uid/gid from container image... 2024-04-23T14:36:33.118 INFO:teuthology.orchestra.run.smithi083.stdout:stat: stdout 167 167 2024-04-23T14:36:33.118 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial keys... 2024-04-23T14:36:34.270 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQBxxydmenMeIxAAq3jmiCPQxJ7KGSuF0sVIiA== 2024-04-23T14:36:35.552 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQByxydm/WMIMBAA74pcChiFnY9mjKCUhCOA8g== 2024-04-23T14:36:36.689 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph-authtool: stdout AQB0xydmXgPcAxAA2xIwWo7L8MnpQDGWefQlrA== 2024-04-23T14:36:36.690 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial monmap... 2024-04-23T14:36:37.839 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T14:36:37.840 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-23T14:36:37.840 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:36:37.840 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T14:36:37.840 INFO:teuthology.orchestra.run.smithi083.stdout:monmaptool for a [v2:172.21.15.83:3300,v1:172.21.15.83:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T14:36:37.840 INFO:teuthology.orchestra.run.smithi083.stdout:setting min_mon_release = quincy 2024-04-23T14:36:37.840 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: set fsid to 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:36:37.840 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T14:36:37.840 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:36:37.841 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mon... 2024-04-23T14:36:39.153 INFO:teuthology.orchestra.run.smithi083.stdout:create mon.a on 2024-04-23T14:36:39.527 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-23T14:36:39.776 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-23T14:36:40.074 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5f4fe708-017e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-5f4fe708-017e-11ef-bc93-c7b262605968.target. 2024-04-23T14:36:40.074 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5f4fe708-017e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-5f4fe708-017e-11ef-bc93-c7b262605968.target. 2024-04-23T14:36:40.449 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.a 2024-04-23T14:36:40.449 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-23T14:36:40.760 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5f4fe708-017e-11ef-bc93-c7b262605968.target.wants/ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-5f4fe708-017e-11ef-bc93-c7b262605968@.service. 2024-04-23T14:36:41.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:41 smithi083 podman[27470]: 2024-04-23 14:36:41.065321986 +0000 UTC m=+0.033796455 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:36:45.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:45 smithi083 podman[27470]: 2024-04-23 14:36:45.434786392 +0000 UTC m=+4.403260882 container create 76140afe6206974c6b0e8c814136dc93a6a28bcfdefcb562f3c498f367c1146d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-23T14:36:49.960 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T14:36:49.961 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-04-23T14:36:49.975 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T14:36:49.975 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-23T14:36:49.975 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-04-23T14:36:49.975 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T14:36:49.975 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon to start... 2024-04-23T14:36:49.976 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mon... 2024-04-23T14:36:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:49 smithi083 podman[27470]: 2024-04-23 14:36:49.841566648 +0000 UTC m=+8.810041163 container init 76140afe6206974c6b0e8c814136dc93a6a28bcfdefcb562f3c498f367c1146d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd) 2024-04-23T14:36:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:49 smithi083 podman[27470]: 2024-04-23 14:36:49.852128119 +0000 UTC m=+8.820602609 container start 76140afe6206974c6b0e8c814136dc93a6a28bcfdefcb562f3c498f367c1146d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd) 2024-04-23T14:36:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:49 smithi083 bash[27470]: 76140afe6206974c6b0e8c814136dc93a6a28bcfdefcb562f3c498f367c1146d 2024-04-23T14:36:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:49 smithi083 ceph-mon[27489]: mkfs 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:36:50.123 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:49 smithi083 ceph-mon[27489]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T14:36:50.124 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:49 smithi083 systemd[1]: Started Ceph mon.a for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:36:51.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:50 smithi083 ceph-mon[27489]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T14:36:51.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:50 smithi083 ceph-mon[27489]: monmap epoch 1 2024-04-23T14:36:51.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:50 smithi083 ceph-mon[27489]: fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:36:51.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:50 smithi083 ceph-mon[27489]: last_changed 2024-04-23T14:36:37.193338+0000 2024-04-23T14:36:51.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:50 smithi083 ceph-mon[27489]: created 2024-04-23T14:36:37.193338+0000 2024-04-23T14:36:51.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:50 smithi083 ceph-mon[27489]: min_mon_release 19 (squid) 2024-04-23T14:36:51.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:50 smithi083 ceph-mon[27489]: election_strategy: 1 2024-04-23T14:36:51.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:50 smithi083 ceph-mon[27489]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-23T14:36:51.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:50 smithi083 ceph-mon[27489]: fsmap 2024-04-23T14:36:51.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:50 smithi083 ceph-mon[27489]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T14:36:51.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:50 smithi083 ceph-mon[27489]: mgrmap e1: no daemons active 2024-04-23T14:36:51.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:50 smithi083 ceph-mon[27489]: from='client.? 172.21.15.83:0/3243277071' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-23T14:36:51.352 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout cluster: 2024-04-23T14:36:51.352 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout id: 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:36:51.352 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-23T14:36:51.353 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:36:51.353 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout services: 2024-04-23T14:36:51.353 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.74993s) 2024-04-23T14:36:51.353 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-23T14:36:51.353 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-23T14:36:51.356 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:36:51.356 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout data: 2024-04-23T14:36:51.356 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-23T14:36:51.356 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-23T14:36:51.356 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-23T14:36:51.356 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout pgs: 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.584+0000 7fb938a00700 1 Processor -- start 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.584+0000 7fb938a00700 1 -- start start 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.585+0000 7fb938a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 0x7fb93407c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.585+0000 7fb938a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb93407c910 con 0x7fb93407de90 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.586+0000 7fb931e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 0x7fb93407c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.586+0000 7fb931e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 0x7fb93407c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38208/0 (socket says 172.21.15.83:38208) 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.586+0000 7fb931e00700 1 -- 172.21.15.83:0/1082749850 learned_addr learned my addr 172.21.15.83:0/1082749850 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.587+0000 7fb931e00700 1 -- 172.21.15.83:0/1082749850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93407ca90 con 0x7fb93407de90 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.587+0000 7fb931e00700 1 --2- 172.21.15.83:0/1082749850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 0x7fb93407c340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9240098a0 tx=0x7fb924009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88751f41e0dc2698 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.587+0000 7fb930a00700 1 -- 172.21.15.83:0/1082749850 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb924004160 con 0x7fb93407de90 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.587+0000 7fb930a00700 1 -- 172.21.15.83:0/1082749850 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb924004300 con 0x7fb93407de90 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.587+0000 7fb930a00700 1 -- 172.21.15.83:0/1082749850 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9240036a0 con 0x7fb93407de90 2024-04-23T14:36:51.357 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.588+0000 7fb938a00700 1 -- 172.21.15.83:0/1082749850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 msgr2=0x7fb93407c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:51.358 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.588+0000 7fb938a00700 1 --2- 172.21.15.83:0/1082749850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 0x7fb93407c340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9240098a0 tx=0x7fb924009bb0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.358 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.588+0000 7fb938a00700 1 -- 172.21.15.83:0/1082749850 shutdown_connections 2024-04-23T14:36:51.358 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.588+0000 7fb938a00700 1 --2- 172.21.15.83:0/1082749850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 0x7fb93407c340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.588+0000 7fb938a00700 1 -- 172.21.15.83:0/1082749850 >> 172.21.15.83:0/1082749850 conn(0x7fb934103d40 msgr2=0x7fb934106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:51.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.588+0000 7fb938a00700 1 -- 172.21.15.83:0/1082749850 shutdown_connections 2024-04-23T14:36:51.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.588+0000 7fb938a00700 1 -- 172.21.15.83:0/1082749850 wait complete. 2024-04-23T14:36:51.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.589+0000 7fb938a00700 1 Processor -- start 2024-04-23T14:36:51.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.589+0000 7fb938a00700 1 -- start start 2024-04-23T14:36:51.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.590+0000 7fb938a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 0x7fb93419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:51.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.590+0000 7fb938a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb93407cf60 con 0x7fb93407de90 2024-04-23T14:36:51.361 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.590+0000 7fb931e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 0x7fb93419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.590+0000 7fb931e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 0x7fb93419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38212/0 (socket says 172.21.15.83:38212) 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.590+0000 7fb931e00700 1 -- 172.21.15.83:0/3243277071 learned_addr learned my addr 172.21.15.83:0/3243277071 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.590+0000 7fb931e00700 1 -- 172.21.15.83:0/3243277071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93419e520 con 0x7fb93407de90 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.591+0000 7fb931e00700 1 --2- 172.21.15.83:0/3243277071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 0x7fb93419dfe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb924000c00 tx=0x7fb924004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.591+0000 7fb92aa00700 1 -- 172.21.15.83:0/3243277071 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb924003db0 con 0x7fb93407de90 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.591+0000 7fb92aa00700 1 -- 172.21.15.83:0/3243277071 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb924011630 con 0x7fb93407de90 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.591+0000 7fb938a00700 1 -- 172.21.15.83:0/3243277071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93419e7b0 con 0x7fb93407de90 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.591+0000 7fb92aa00700 1 -- 172.21.15.83:0/3243277071 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb924011950 con 0x7fb93407de90 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.591+0000 7fb938a00700 1 -- 172.21.15.83:0/3243277071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9341a14a0 con 0x7fb93407de90 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.591+0000 7fb92aa00700 1 -- 172.21.15.83:0/3243277071 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb92401a460 con 0x7fb93407de90 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.592+0000 7fb92aa00700 1 -- 172.21.15.83:0/3243277071 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb92401aa90 con 0x7fb93407de90 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.592+0000 7fb938a00700 1 -- 172.21.15.83:0/3243277071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb934064150 con 0x7fb93407de90 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.596+0000 7fb92aa00700 1 -- 172.21.15.83:0/3243277071 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb924042e50 con 0x7fb93407de90 2024-04-23T14:36:51.362 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.675+0000 7fb938a00700 1 -- 172.21.15.83:0/3243277071 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb93402e380 con 0x7fb93407de90 2024-04-23T14:36:51.363 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.675+0000 7fb92aa00700 1 -- 172.21.15.83:0/3243277071 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+316 (secure 0 0 0) 0x7fb924031070 con 0x7fb93407de90 2024-04-23T14:36:51.363 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.676+0000 7fb938a00700 1 -- 172.21.15.83:0/3243277071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 msgr2=0x7fb93419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:51.363 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.676+0000 7fb938a00700 1 --2- 172.21.15.83:0/3243277071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 0x7fb93419dfe0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb924000c00 tx=0x7fb924004750 comp rx=0 tx=0).stop 2024-04-23T14:36:51.363 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.677+0000 7fb938a00700 1 -- 172.21.15.83:0/3243277071 shutdown_connections 2024-04-23T14:36:51.363 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.677+0000 7fb938a00700 1 --2- 172.21.15.83:0/3243277071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb93407de90 0x7fb93419dfe0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:51.363 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.677+0000 7fb938a00700 1 -- 172.21.15.83:0/3243277071 >> 172.21.15.83:0/3243277071 conn(0x7fb934103d40 msgr2=0x7fb934104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:51.363 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.677+0000 7fb938a00700 1 -- 172.21.15.83:0/3243277071 shutdown_connections 2024-04-23T14:36:51.363 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:50.677+0000 7fb938a00700 1 -- 172.21.15.83:0/3243277071 wait complete. 2024-04-23T14:36:51.363 INFO:teuthology.orchestra.run.smithi083.stdout:mon is available 2024-04-23T14:36:51.363 INFO:teuthology.orchestra.run.smithi083.stdout:Assimilating anything we can from ceph.conf... 2024-04-23T14:36:52.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:52 smithi083 ceph-mon[27489]: from='client.? 172.21.15.83:0/4172374719' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T14:36:52.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:52 smithi083 ceph-mon[27489]: from='client.? 172.21.15.83:0/4172374719' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-23T14:36:52.721 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.83:3300,v1:172.21.15.83:6789] 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-04-23T14:36:52.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.964+0000 7fc36c600700 1 Processor -- start 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.964+0000 7fc36c600700 1 -- start start 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.965+0000 7fc36c600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 0x7fc36410ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.965+0000 7fc36c600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc36410b370 con 0x7fc36410a980 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.965+0000 7fc369c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 0x7fc36410ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.965+0000 7fc369c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 0x7fc36410ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38214/0 (socket says 172.21.15.83:38214) 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.965+0000 7fc369c00700 1 -- 172.21.15.83:0/1053968959 learned_addr learned my addr 172.21.15.83:0/1053968959 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.966+0000 7fc369c00700 1 -- 172.21.15.83:0/1053968959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36410bb30 con 0x7fc36410a980 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.966+0000 7fc369c00700 1 --2- 172.21.15.83:0/1053968959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 0x7fc36410ada0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc3580098a0 tx=0x7fc358009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bed47fafe6eab0a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.967+0000 7fc363e00700 1 -- 172.21.15.83:0/1053968959 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc358004160 con 0x7fc36410a980 2024-04-23T14:36:52.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.967+0000 7fc363e00700 1 -- 172.21.15.83:0/1053968959 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc358004300 con 0x7fc36410a980 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.967+0000 7fc363e00700 1 -- 172.21.15.83:0/1053968959 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3580036a0 con 0x7fc36410a980 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.967+0000 7fc36c600700 1 -- 172.21.15.83:0/1053968959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 msgr2=0x7fc36410ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.967+0000 7fc36c600700 1 --2- 172.21.15.83:0/1053968959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 0x7fc36410ada0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc3580098a0 tx=0x7fc358009bb0 comp rx=0 tx=0).stop 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.967+0000 7fc36c600700 1 -- 172.21.15.83:0/1053968959 shutdown_connections 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.967+0000 7fc36c600700 1 --2- 172.21.15.83:0/1053968959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 0x7fc36410ada0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.967+0000 7fc36c600700 1 -- 172.21.15.83:0/1053968959 >> 172.21.15.83:0/1053968959 conn(0x7fc36407def0 msgr2=0x7fc3641084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.968+0000 7fc36c600700 1 -- 172.21.15.83:0/1053968959 shutdown_connections 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.968+0000 7fc36c600700 1 -- 172.21.15.83:0/1053968959 wait complete. 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.969+0000 7fc36c600700 1 Processor -- start 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.969+0000 7fc36c600700 1 -- start start 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.970+0000 7fc36c600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 0x7fc36419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.970+0000 7fc36c600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc36410c060 con 0x7fc36410a980 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.970+0000 7fc369c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 0x7fc36419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.970+0000 7fc369c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 0x7fc36419dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38218/0 (socket says 172.21.15.83:38218) 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.970+0000 7fc369c00700 1 -- 172.21.15.83:0/4172374719 learned_addr learned my addr 172.21.15.83:0/4172374719 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:36:52.724 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.970+0000 7fc369c00700 1 -- 172.21.15.83:0/4172374719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36419e4e0 con 0x7fc36410a980 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.971+0000 7fc369c00700 1 --2- 172.21.15.83:0/4172374719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 0x7fc36419dfa0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc358000c00 tx=0x7fc358004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.971+0000 7fc362000700 1 -- 172.21.15.83:0/4172374719 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc358003db0 con 0x7fc36410a980 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.971+0000 7fc362000700 1 -- 172.21.15.83:0/4172374719 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc358011630 con 0x7fc36410a980 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.971+0000 7fc36c600700 1 -- 172.21.15.83:0/4172374719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36419e770 con 0x7fc36410a980 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.971+0000 7fc362000700 1 -- 172.21.15.83:0/4172374719 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc358011950 con 0x7fc36410a980 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.971+0000 7fc36c600700 1 -- 172.21.15.83:0/4172374719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3641a1460 con 0x7fc36410a980 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.971+0000 7fc362000700 1 -- 172.21.15.83:0/4172374719 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc358011af0 con 0x7fc36410a980 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.972+0000 7fc362000700 1 -- 172.21.15.83:0/4172374719 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc35801c450 con 0x7fc36410a980 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.972+0000 7fc36c600700 1 -- 172.21.15.83:0/4172374719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc364064150 con 0x7fc36410a980 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:51.975+0000 7fc362000700 1 -- 172.21.15.83:0/4172374719 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc358032020 con 0x7fc36410a980 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:52.051+0000 7fc36c600700 1 -- 172.21.15.83:0/4172374719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc36402e160 con 0x7fc36410a980 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:52.060+0000 7fc362000700 1 -- 172.21.15.83:0/4172374719 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+454 (secure 0 0 0) 0x7fc358004260 con 0x7fc36410a980 2024-04-23T14:36:52.725 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:52.062+0000 7fc362000700 1 -- 172.21.15.83:0/4172374719 <== mon.0 v2:172.21.15.83:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc35801b450 con 0x7fc36410a980 2024-04-23T14:36:52.726 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:52.063+0000 7fc36c600700 1 -- 172.21.15.83:0/4172374719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 msgr2=0x7fc36419dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:52.726 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:52.063+0000 7fc36c600700 1 --2- 172.21.15.83:0/4172374719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 0x7fc36419dfa0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc358000c00 tx=0x7fc358004750 comp rx=0 tx=0).stop 2024-04-23T14:36:52.726 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:52.063+0000 7fc36c600700 1 -- 172.21.15.83:0/4172374719 shutdown_connections 2024-04-23T14:36:52.726 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:52.063+0000 7fc36c600700 1 --2- 172.21.15.83:0/4172374719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc36410a980 0x7fc36419dfa0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:52.726 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:52.063+0000 7fc36c600700 1 -- 172.21.15.83:0/4172374719 >> 172.21.15.83:0/4172374719 conn(0x7fc36407def0 msgr2=0x7fc364194be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:52.726 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:52.064+0000 7fc36c600700 1 -- 172.21.15.83:0/4172374719 shutdown_connections 2024-04-23T14:36:52.726 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:52.064+0000 7fc36c600700 1 -- 172.21.15.83:0/4172374719 wait complete. 2024-04-23T14:36:52.726 INFO:teuthology.orchestra.run.smithi083.stdout:Generating new minimal ceph.conf... 2024-04-23T14:36:53.718 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:53 smithi083 ceph-mon[27489]: from='client.? 172.21.15.83:0/3216196282' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:36:54.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.303+0000 7fe487e00700 1 Processor -- start 2024-04-23T14:36:54.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.303+0000 7fe487e00700 1 -- start start 2024-04-23T14:36:54.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.303+0000 7fe487e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 0x7fe48810f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:54.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.304+0000 7fe487e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe48810f9b0 con 0x7fe48810efe0 2024-04-23T14:36:54.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.304+0000 7fe486a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 0x7fe48810f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:54.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.304+0000 7fe486a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 0x7fe48810f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38224/0 (socket says 172.21.15.83:38224) 2024-04-23T14:36:54.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.304+0000 7fe486a00700 1 -- 172.21.15.83:0/1827383636 learned_addr learned my addr 172.21.15.83:0/1827383636 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:36:54.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.304+0000 7fe486a00700 1 -- 172.21.15.83:0/1827383636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48810fb30 con 0x7fe48810efe0 2024-04-23T14:36:54.052 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.305+0000 7fe486a00700 1 --2- 172.21.15.83:0/1827383636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 0x7fe48810f3e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe4780098a0 tx=0x7fe478009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8dfce3317391247 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.305+0000 7fe485600700 1 -- 172.21.15.83:0/1827383636 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe478004160 con 0x7fe48810efe0 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.305+0000 7fe485600700 1 -- 172.21.15.83:0/1827383636 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe478004300 con 0x7fe48810efe0 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.305+0000 7fe485600700 1 -- 172.21.15.83:0/1827383636 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe478011b50 con 0x7fe48810efe0 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.306+0000 7fe487e00700 1 -- 172.21.15.83:0/1827383636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 msgr2=0x7fe48810f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.306+0000 7fe487e00700 1 --2- 172.21.15.83:0/1827383636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 0x7fe48810f3e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe4780098a0 tx=0x7fe478009bb0 comp rx=0 tx=0).stop 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.306+0000 7fe487e00700 1 -- 172.21.15.83:0/1827383636 shutdown_connections 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.306+0000 7fe487e00700 1 --2- 172.21.15.83:0/1827383636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 0x7fe48810f3e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.306+0000 7fe487e00700 1 -- 172.21.15.83:0/1827383636 >> 172.21.15.83:0/1827383636 conn(0x7fe48807ddf0 msgr2=0x7fe488080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.306+0000 7fe487e00700 1 -- 172.21.15.83:0/1827383636 shutdown_connections 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.306+0000 7fe487e00700 1 -- 172.21.15.83:0/1827383636 wait complete. 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.307+0000 7fe487e00700 1 Processor -- start 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.308+0000 7fe487e00700 1 -- start start 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.308+0000 7fe487e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 0x7fe4881a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:54.053 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.308+0000 7fe487e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe488112fd0 con 0x7fe48810efe0 2024-04-23T14:36:54.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.308+0000 7fe486a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 0x7fe4881a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:54.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.308+0000 7fe486a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 0x7fe4881a2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38236/0 (socket says 172.21.15.83:38236) 2024-04-23T14:36:54.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.308+0000 7fe486a00700 1 -- 172.21.15.83:0/3216196282 learned_addr learned my addr 172.21.15.83:0/3216196282 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:36:54.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.309+0000 7fe486a00700 1 -- 172.21.15.83:0/3216196282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4881a2fd0 con 0x7fe48810efe0 2024-04-23T14:36:54.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.309+0000 7fe486a00700 1 --2- 172.21.15.83:0/3216196282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 0x7fe4881a2a90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe478009fb0 tx=0x7fe478011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:54.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.309+0000 7fe47f400700 1 -- 172.21.15.83:0/3216196282 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4780038b0 con 0x7fe48810efe0 2024-04-23T14:36:54.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.309+0000 7fe47f400700 1 -- 172.21.15.83:0/3216196282 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe478011e50 con 0x7fe48810efe0 2024-04-23T14:36:54.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.309+0000 7fe487e00700 1 -- 172.21.15.83:0/3216196282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4881a3260 con 0x7fe48810efe0 2024-04-23T14:36:54.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.309+0000 7fe47f400700 1 -- 172.21.15.83:0/3216196282 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe478019420 con 0x7fe48810efe0 2024-04-23T14:36:54.054 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.309+0000 7fe487e00700 1 -- 172.21.15.83:0/3216196282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4881a5f50 con 0x7fe48810efe0 2024-04-23T14:36:54.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.310+0000 7fe47f400700 1 -- 172.21.15.83:0/3216196282 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe478019a80 con 0x7fe48810efe0 2024-04-23T14:36:54.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.310+0000 7fe47f400700 1 -- 172.21.15.83:0/3216196282 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe478024730 con 0x7fe48810efe0 2024-04-23T14:36:54.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.311+0000 7fe487e00700 1 -- 172.21.15.83:0/3216196282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe488113150 con 0x7fe48810efe0 2024-04-23T14:36:54.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.313+0000 7fe47f400700 1 -- 172.21.15.83:0/3216196282 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe478043e50 con 0x7fe48810efe0 2024-04-23T14:36:54.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.390+0000 7fe487e00700 1 -- 172.21.15.83:0/3216196282 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe4881a6490 con 0x7fe48810efe0 2024-04-23T14:36:54.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.390+0000 7fe47f400700 1 -- 172.21.15.83:0/3216196282 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fe478032030 con 0x7fe48810efe0 2024-04-23T14:36:54.056 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.392+0000 7fe487e00700 1 -- 172.21.15.83:0/3216196282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 msgr2=0x7fe4881a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:54.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.392+0000 7fe487e00700 1 --2- 172.21.15.83:0/3216196282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 0x7fe4881a2a90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe478009fb0 tx=0x7fe478011bf0 comp rx=0 tx=0).stop 2024-04-23T14:36:54.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.392+0000 7fe487e00700 1 -- 172.21.15.83:0/3216196282 shutdown_connections 2024-04-23T14:36:54.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.392+0000 7fe487e00700 1 --2- 172.21.15.83:0/3216196282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe48810efe0 0x7fe4881a2a90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:54.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.392+0000 7fe487e00700 1 -- 172.21.15.83:0/3216196282 >> 172.21.15.83:0/3216196282 conn(0x7fe48807ddf0 msgr2=0x7fe48807f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:54.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.392+0000 7fe487e00700 1 -- 172.21.15.83:0/3216196282 shutdown_connections 2024-04-23T14:36:54.057 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:53.392+0000 7fe487e00700 1 -- 172.21.15.83:0/3216196282 wait complete. 2024-04-23T14:36:54.057 INFO:teuthology.orchestra.run.smithi083.stdout:Restarting the monitor... 2024-04-23T14:36:54.458 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:54 smithi083 systemd[1]: Stopping Ceph mon.a for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:36:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:54 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a[27485]: 2024-04-23T14:36:54.665+0000 7f6cf8e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T14:36:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:54 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a[27485]: 2024-04-23T14:36:54.665+0000 7f6cf8e00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-23T14:36:55.105 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:54 smithi083 podman[27734]: 2024-04-23 14:36:54.912231023 +0000 UTC m=+0.527193074 container died 76140afe6206974c6b0e8c814136dc93a6a28bcfdefcb562f3c498f367c1146d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T14:36:55.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:55 smithi083 podman[27734]: 2024-04-23 14:36:55.612013695 +0000 UTC m=+1.226975779 container cleanup 76140afe6206974c6b0e8c814136dc93a6a28bcfdefcb562f3c498f367c1146d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-23T14:36:55.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:55 smithi083 bash[27734]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a 2024-04-23T14:36:56.100 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:55 smithi083 podman[27750]: 2024-04-23 14:36:55.787888096 +0000 UTC m=+0.866873401 container remove 76140afe6206974c6b0e8c814136dc93a6a28bcfdefcb562f3c498f367c1146d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-23T14:36:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 systemd[1]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-23T14:36:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 systemd[1]: Stopped Ceph mon.a for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:36:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 systemd[1]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.748s CPU time. 2024-04-23T14:36:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 systemd[1]: Starting Ceph mon.a for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:36:56.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 podman[27848]: 2024-04-23 14:36:56.445537093 +0000 UTC m=+0.038354249 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:36:56.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 podman[27848]: 2024-04-23 14:36:56.538596087 +0000 UTC m=+0.131413236 container create 4c2a91033a2a8cd38abaf6838701787456d7fa3872bce62a29bcbb48325afea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-23T14:36:56.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 podman[27848]: 2024-04-23 14:36:56.784323511 +0000 UTC m=+0.377140694 container init 4c2a91033a2a8cd38abaf6838701787456d7fa3872bce62a29bcbb48325afea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-23T14:36:56.914 INFO:teuthology.orchestra.run.smithi083.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-23T14:36:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 podman[27848]: 2024-04-23 14:36:56.794678173 +0000 UTC m=+0.387495355 container start 4c2a91033a2a8cd38abaf6838701787456d7fa3872bce62a29bcbb48325afea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T14:36:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T14:36:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev), process ceph-mon, pid 2 2024-04-23T14:36:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: pidfile_write: ignore empty --pid-file 2024-04-23T14:36:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: load: jerasure load: lrc 2024-04-23T14:36:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: RocksDB version: 7.9.2 2024-04-23T14:36:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Git sha 0 2024-04-23T14:36:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Compile date 2024-04-22 13:15:13 2024-04-23T14:36:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: DB SUMMARY 2024-04-23T14:36:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: DB Session ID: PSS4AMD7O2UJN0KN8Z84 2024-04-23T14:36:57.069 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: CURRENT file: CURRENT 2024-04-23T14:36:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T14:36:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-23T14:36:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-23T14:36:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106126 ; 2024-04-23T14:36:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.error_if_exists: 0 2024-04-23T14:36:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.create_if_missing: 0 2024-04-23T14:36:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.paranoid_checks: 1 2024-04-23T14:36:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T14:36:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T14:36:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T14:36:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.env: 0x562c2d154cc0 2024-04-23T14:36:57.070 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T14:36:57.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.info_log: 0x562c6053bc60 2024-04-23T14:36:57.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T14:36:57.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.statistics: (nil) 2024-04-23T14:36:57.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.use_fsync: 0 2024-04-23T14:36:57.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_log_file_size: 0 2024-04-23T14:36:57.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T14:36:57.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T14:36:57.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T14:36:57.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T14:36:57.071 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.allow_fallocate: 1 2024-04-23T14:36:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T14:36:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T14:36:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.use_direct_reads: 0 2024-04-23T14:36:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T14:36:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T14:36:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.db_log_dir: 2024-04-23T14:36:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.wal_dir: 2024-04-23T14:36:57.072 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.write_buffer_manager: 0x562c6050a1e0 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T14:36:57.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.unordered_write: 0 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.row_cache: None 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.wal_filter: None 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T14:36:57.074 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.two_write_queues: 0 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.wal_compression: 0 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.atomic_flush: 0 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.log_readahead_size: 0 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T14:36:57.075 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_background_jobs: 2 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_background_compactions: -1 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_subcompactions: 1 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_open_files: -1 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T14:36:57.076 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_background_flushes: -1 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Compression algorithms supported: 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: kZSTD supported: 0 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: kXpressCompression supported: 0 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: kZlibCompression supported: 1 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: kSnappyCompression supported: 1 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: kLZ4Compression supported: 1 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: kBZip2Compression supported: 0 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T14:36:57.077 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.merge_operator: 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_filter: None 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562c6053be20) 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks: 1 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T14:36:57.078 INFO:journalctl@ceph.mon.a.smithi083.stdout: pin_top_level_index_and_filter: 1 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_type: 0 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_index_type: 0 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_shortening: 1 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: checksum: 4 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: no_block_cache: 0 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache: 0x562c6053d090 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_name: BinnedLRUCache 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_options: 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: capacity : 536870912 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_shard_bits : 4 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: strict_capacity_limit : 0 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: high_pri_pool_ratio: 0.000 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_cache_compressed: (nil) 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: persistent_cache: (nil) 2024-04-23T14:36:57.079 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size: 4096 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_size_deviation: 10 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_restart_interval: 16 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: index_block_restart_interval: 1 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: metadata_block_size: 4096 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: partition_filters: 0 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: use_delta_encoding: 1 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: filter_policy: bloomfilter 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: whole_key_filtering: 1 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: verify_compression: 0 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: read_amp_bytes_per_bit: 0 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: format_version: 5 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_index_compression: 1 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: block_align: 0 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: max_auto_readahead_size: 262144 2024-04-23T14:36:57.080 INFO:journalctl@ceph.mon.a.smithi083.stdout: prepopulate_block_cache: 0 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout: initial_auto_readahead_size: 8192 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compression: NoCompression 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.num_levels: 7 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T14:36:57.081 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T14:36:57.082 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T14:36:57.083 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T14:36:57.084 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T14:36:57.084 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T14:36:57.084 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T14:36:57.084 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T14:36:57.084 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T14:36:57.084 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T14:36:57.085 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.inplace_update_support: 0 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.bloom_locality: 0 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.max_successive_merges: 0 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.ttl: 2592000 2024-04-23T14:36:57.086 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.enable_blob_files: false 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.min_blob_size: 0 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-23T14:36:57.087 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b4fea8c8-ce71-42b5-947d-b1b045849456 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713883016852792, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713883016854654, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101126, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 99025, "index_size": 293, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14423, "raw_average_key_size": 49, "raw_value_size": 91228, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713883016, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b4fea8c8-ce71-42b5-947d-b1b045849456", "db_session_id": "PSS4AMD7O2UJN0KN8Z84", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713883016854920, "job": 1, "event": "recovery_finished"} 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562c6062a000 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: DB pointer 0x562c60614000 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** DB Stats ** 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T14:36:57.088 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: L0 2/0 100.62 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: Sum 2/0 100.62 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** Compaction Stats [default] ** 2024-04-23T14:36:57.089 INFO:journalctl@ceph.mon.a.smithi083.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 78.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: Cumulative compaction: 0.00 GB write, 13.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: Interval compaction: 0.00 GB write, 13.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T14:36:57.090 INFO:journalctl@ceph.mon.a.smithi083.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache BinnedLRUCache@0x562c6053d090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: starting mon.a rank 0 at public addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] at bind addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: mon.a@-1(???) e1 preinit fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: mon.a@-1(???).mds e1 new map 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: mon.a@-1(???).mds e1 print_map 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout: e1 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout: legacy client fscid: -1 2024-04-23T14:36:57.091 INFO:journalctl@ceph.mon.a.smithi083.stdout: 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout: No filesystems configured 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 bash[27848]: 4c2a91033a2a8cd38abaf6838701787456d7fa3872bce62a29bcbb48325afea2 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 systemd[1]: Started Ceph mon.a for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: monmap epoch 1 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: last_changed 2024-04-23T14:36:37.193338+0000 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: created 2024-04-23T14:36:37.193338+0000 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: min_mon_release 19 (squid) 2024-04-23T14:36:57.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: election_strategy: 1 2024-04-23T14:36:57.093 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-23T14:36:57.093 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: fsmap 2024-04-23T14:36:57.093 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T14:36:57.093 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:56 smithi083 ceph-mon[27867]: mgrmap e1: no daemons active 2024-04-23T14:36:58.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.591+0000 7f9b4ca00700 1 Processor -- start 2024-04-23T14:36:58.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.592+0000 7f9b4ca00700 1 -- start start 2024-04-23T14:36:58.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.592+0000 7f9b4ca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 0x7f9b481089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:58.258 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.592+0000 7f9b4ca00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9b48108f90 con 0x7f9b481085a0 2024-04-23T14:36:58.258 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.593+0000 7f9b46800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 0x7f9b481089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:58.258 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.593+0000 7f9b46800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 0x7f9b481089c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48858/0 (socket says 172.21.15.83:48858) 2024-04-23T14:36:58.258 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.593+0000 7f9b46800700 1 -- 172.21.15.83:0/1466309142 learned_addr learned my addr 172.21.15.83:0/1466309142 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:36:58.258 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.593+0000 7f9b46800700 1 -- 172.21.15.83:0/1466309142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b48109700 con 0x7f9b481085a0 2024-04-23T14:36:58.258 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.594+0000 7f9b46800700 1 --2- 172.21.15.83:0/1466309142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 0x7f9b481089c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009e80 tx=0x7f9b3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf34c86fe7de56e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:58.258 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.594+0000 7f9b45e00700 1 -- 172.21.15.83:0/1466309142 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b38017070 con 0x7f9b481085a0 2024-04-23T14:36:58.258 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.594+0000 7f9b45e00700 1 -- 172.21.15.83:0/1466309142 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9b3800b8c0 con 0x7f9b481085a0 2024-04-23T14:36:58.258 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.594+0000 7f9b45e00700 1 -- 172.21.15.83:0/1466309142 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b380045c0 con 0x7f9b481085a0 2024-04-23T14:36:58.258 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.594+0000 7f9b4ca00700 1 -- 172.21.15.83:0/1466309142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 msgr2=0x7f9b481089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:58.258 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.594+0000 7f9b4ca00700 1 --2- 172.21.15.83:0/1466309142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 0x7f9b481089c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009e80 tx=0x7f9b3800b330 comp rx=0 tx=0).stop 2024-04-23T14:36:58.258 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.595+0000 7f9b4ca00700 1 -- 172.21.15.83:0/1466309142 shutdown_connections 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.595+0000 7f9b4ca00700 1 --2- 172.21.15.83:0/1466309142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 0x7f9b481089c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.595+0000 7f9b4ca00700 1 -- 172.21.15.83:0/1466309142 >> 172.21.15.83:0/1466309142 conn(0x7f9b48103be0 msgr2=0x7f9b48106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.595+0000 7f9b4ca00700 1 -- 172.21.15.83:0/1466309142 shutdown_connections 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.595+0000 7f9b4ca00700 1 -- 172.21.15.83:0/1466309142 wait complete. 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.596+0000 7f9b4ca00700 1 Processor -- start 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.596+0000 7f9b4ca00700 1 -- start start 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.596+0000 7f9b4ca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 0x7f9b4819c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.596+0000 7f9b4ca00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4810cab0 con 0x7f9b481085a0 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.597+0000 7f9b46800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 0x7f9b4819c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.597+0000 7f9b46800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 0x7f9b4819c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48864/0 (socket says 172.21.15.83:48864) 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.597+0000 7f9b46800700 1 -- 172.21.15.83:0/835790927 learned_addr learned my addr 172.21.15.83:0/835790927 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.597+0000 7f9b46800700 1 -- 172.21.15.83:0/835790927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4819c980 con 0x7f9b481085a0 2024-04-23T14:36:58.259 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.597+0000 7f9b46800700 1 --2- 172.21.15.83:0/835790927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 0x7f9b4819c440 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009530 tx=0x7f9b3800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.598+0000 7f9b3fe00700 1 -- 172.21.15.83:0/835790927 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b38020020 con 0x7f9b481085a0 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.598+0000 7f9b3fe00700 1 -- 172.21.15.83:0/835790927 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9b38003b50 con 0x7f9b481085a0 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.598+0000 7f9b3fe00700 1 -- 172.21.15.83:0/835790927 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b38017040 con 0x7f9b481085a0 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.598+0000 7f9b4ca00700 1 -- 172.21.15.83:0/835790927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4819cc10 con 0x7f9b481085a0 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.598+0000 7f9b4ca00700 1 -- 172.21.15.83:0/835790927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4819f900 con 0x7f9b481085a0 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.598+0000 7f9b3fe00700 1 -- 172.21.15.83:0/835790927 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9b38023020 con 0x7f9b481085a0 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.598+0000 7f9b3fe00700 1 -- 172.21.15.83:0/835790927 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9b3800f050 con 0x7f9b481085a0 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.599+0000 7f9b4ca00700 1 -- 172.21.15.83:0/835790927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b48064150 con 0x7f9b481085a0 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.603+0000 7f9b3fe00700 1 -- 172.21.15.83:0/835790927 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9b3802a020 con 0x7f9b481085a0 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.678+0000 7f9b4ca00700 1 -- 172.21.15.83:0/835790927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f9b4819fad0 con 0x7f9b481085a0 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.682+0000 7f9b3fe00700 1 -- 172.21.15.83:0/835790927 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f9b3800b860 con 0x7f9b481085a0 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.683+0000 7f9b4ca00700 1 -- 172.21.15.83:0/835790927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 msgr2=0x7f9b4819c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.683+0000 7f9b4ca00700 1 --2- 172.21.15.83:0/835790927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 0x7f9b4819c440 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009530 tx=0x7f9b3800bd50 comp rx=0 tx=0).stop 2024-04-23T14:36:58.260 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.683+0000 7f9b4ca00700 1 -- 172.21.15.83:0/835790927 shutdown_connections 2024-04-23T14:36:58.261 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.683+0000 7f9b4ca00700 1 --2- 172.21.15.83:0/835790927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b481085a0 0x7f9b4819c440 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:36:58.261 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.683+0000 7f9b4ca00700 1 -- 172.21.15.83:0/835790927 >> 172.21.15.83:0/835790927 conn(0x7f9b48103be0 msgr2=0x7f9b481046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:36:58.261 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.684+0000 7f9b4ca00700 1 -- 172.21.15.83:0/835790927 shutdown_connections 2024-04-23T14:36:58.261 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:36:57.684+0000 7f9b4ca00700 1 -- 172.21.15.83:0/835790927 wait complete. 2024-04-23T14:36:58.261 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-23T14:36:58.280 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-23T14:36:58.281 INFO:teuthology.orchestra.run.smithi083.stdout:Creating mgr... 2024-04-23T14:36:58.282 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-23T14:36:58.282 INFO:teuthology.orchestra.run.smithi083.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-23T14:36:58.727 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.x 2024-04-23T14:36:58.728 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-23T14:36:58.994 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5f4fe708-017e-11ef-bc93-c7b262605968.target.wants/ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-5f4fe708-017e-11ef-bc93-c7b262605968@.service. 2024-04-23T14:36:59.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:36:58 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/835790927' entity='client.admin' 2024-04-23T14:36:59.399 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:36:59 smithi083 systemd[1]: Starting Ceph mgr.x for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:36:59.656 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:36:59 smithi083 podman[28082]: 2024-04-23 14:36:59.301193352 +0000 UTC m=+0.033644334 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:36:59.656 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:36:59 smithi083 podman[28082]: 2024-04-23 14:36:59.399798716 +0000 UTC m=+0.132249721 container create 00a122261f10fc638711ef69ea53cf03db57490734f4692e215ace2d337b161e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-23T14:36:59.656 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:36:59 smithi083 podman[28082]: 2024-04-23 14:36:59.654869431 +0000 UTC m=+0.387320498 container init 00a122261f10fc638711ef69ea53cf03db57490734f4692e215ace2d337b161e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T14:36:59.755 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T14:36:59.755 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-04-23T14:36:59.771 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T14:36:59.771 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-23T14:36:59.771 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-04-23T14:36:59.771 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T14:36:59.789 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T14:36:59.789 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-04-23T14:36:59.806 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T14:36:59.806 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-23T14:36:59.807 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-04-23T14:36:59.807 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-23T14:36:59.807 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr to start... 2024-04-23T14:36:59.807 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr... 2024-04-23T14:36:59.954 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:36:59 smithi083 podman[28082]: 2024-04-23 14:36:59.665864318 +0000 UTC m=+0.398315328 container start 00a122261f10fc638711ef69ea53cf03db57490734f4692e215ace2d337b161e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-04-23T14:36:59.954 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:36:59 smithi083 bash[28082]: 00a122261f10fc638711ef69ea53cf03db57490734f4692e215ace2d337b161e 2024-04-23T14:36:59.954 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:36:59 smithi083 systemd[1]: Started Ceph mgr.x for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:36:59.954 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:36:59 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:36:59.904+0000 7f2a39834200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T14:37:00.249 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:00 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:00.169+0000 7f2a39834200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:37:00.500 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:00 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:00.307+0000 7f2a39834200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T14:37:00.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:00 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1721664656' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T14:37:00.756 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:00 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:00.741+0000 7f2a39834200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T14:37:01.226 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:00 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:00.858+0000 7f2a39834200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T14:37:01.239 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:37:01.243 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-23T14:37:01.243 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "5f4fe708-017e-11ef-bc93-c7b262605968", 2024-04-23T14:37:01.243 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T14:37:01.243 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T14:37:01.243 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T14:37:01.244 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T14:37:01.245 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T14:37:01.246 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T14:37:01.246 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T14:37:01.246 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T14:37:01.246 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T14:37:01.246 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T14:37:01.253 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T14:36:49.929071+0000", 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.403+0000 7efc40e00700 1 Processor -- start 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.404+0000 7efc40e00700 1 -- start start 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.404+0000 7efc40e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 0x7efc3c10b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.254 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.404+0000 7efc40e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c07be50 con 0x7efc3c108fe0 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.405+0000 7efc3b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 0x7efc3c10b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.405+0000 7efc3b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 0x7efc3c10b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48878/0 (socket says 172.21.15.83:48878) 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.405+0000 7efc3b400700 1 -- 172.21.15.83:0/234411286 learned_addr learned my addr 172.21.15.83:0/234411286 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.405+0000 7efc3b400700 1 -- 172.21.15.83:0/234411286 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c10b910 con 0x7efc3c108fe0 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.406+0000 7efc3b400700 1 --2- 172.21.15.83:0/234411286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 0x7efc3c10b3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0098a0 tx=0x7efc2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6116cc1abb0bc4f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.406+0000 7efc3aa00700 1 -- 172.21.15.83:0/234411286 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc2c004160 con 0x7efc3c108fe0 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.406+0000 7efc3aa00700 1 -- 172.21.15.83:0/234411286 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc2c004300 con 0x7efc3c108fe0 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.406+0000 7efc3aa00700 1 -- 172.21.15.83:0/234411286 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc2c011b50 con 0x7efc3c108fe0 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.407+0000 7efc40e00700 1 -- 172.21.15.83:0/234411286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 msgr2=0x7efc3c10b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.407+0000 7efc40e00700 1 --2- 172.21.15.83:0/234411286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 0x7efc3c10b3d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0098a0 tx=0x7efc2c009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.407+0000 7efc40e00700 1 -- 172.21.15.83:0/234411286 shutdown_connections 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.407+0000 7efc40e00700 1 --2- 172.21.15.83:0/234411286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 0x7efc3c10b3d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.407+0000 7efc40e00700 1 -- 172.21.15.83:0/234411286 >> 172.21.15.83:0/234411286 conn(0x7efc3c102a10 msgr2=0x7efc3c104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:01.255 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.407+0000 7efc40e00700 1 -- 172.21.15.83:0/234411286 shutdown_connections 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.407+0000 7efc40e00700 1 -- 172.21.15.83:0/234411286 wait complete. 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.408+0000 7efc40e00700 1 Processor -- start 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.408+0000 7efc40e00700 1 -- start start 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.409+0000 7efc40e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 0x7efc3c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.409+0000 7efc40e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c10c8d0 con 0x7efc3c108fe0 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.409+0000 7efc3b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 0x7efc3c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.409+0000 7efc3b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 0x7efc3c19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48892/0 (socket says 172.21.15.83:48892) 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.409+0000 7efc3b400700 1 -- 172.21.15.83:0/1721664656 learned_addr learned my addr 172.21.15.83:0/1721664656 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.409+0000 7efc3b400700 1 -- 172.21.15.83:0/1721664656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c19ec70 con 0x7efc3c108fe0 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.410+0000 7efc3b400700 1 --2- 172.21.15.83:0/1721664656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 0x7efc3c19e730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efc2c011b60 tx=0x7efc2c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.410+0000 7efc38c00700 1 -- 172.21.15.83:0/1721664656 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc2c003970 con 0x7efc3c108fe0 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.410+0000 7efc38c00700 1 -- 172.21.15.83:0/1721664656 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc2c003b10 con 0x7efc3c108fe0 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.410+0000 7efc38c00700 1 -- 172.21.15.83:0/1721664656 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc2c024610 con 0x7efc3c108fe0 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.410+0000 7efc40e00700 1 -- 172.21.15.83:0/1721664656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c19ef00 con 0x7efc3c108fe0 2024-04-23T14:37:01.256 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.410+0000 7efc40e00700 1 -- 172.21.15.83:0/1721664656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c1a1bf0 con 0x7efc3c108fe0 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.410+0000 7efc38c00700 1 -- 172.21.15.83:0/1721664656 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efc2c024c50 con 0x7efc3c108fe0 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.410+0000 7efc38c00700 1 -- 172.21.15.83:0/1721664656 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efc2c011e00 con 0x7efc3c108fe0 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.412+0000 7efc40e00700 1 -- 172.21.15.83:0/1721664656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3c064150 con 0x7efc3c108fe0 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.414+0000 7efc38c00700 1 -- 172.21.15.83:0/1721664656 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7efc2c01a9f0 con 0x7efc3c108fe0 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.498+0000 7efc40e00700 1 -- 172.21.15.83:0/1721664656 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efc3c1a1dc0 con 0x7efc3c108fe0 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.499+0000 7efc38c00700 1 -- 172.21.15.83:0/1721664656 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7efc2c01a430 con 0x7efc3c108fe0 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.500+0000 7efc40e00700 1 -- 172.21.15.83:0/1721664656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 msgr2=0x7efc3c19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.500+0000 7efc40e00700 1 --2- 172.21.15.83:0/1721664656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 0x7efc3c19e730 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efc2c011b60 tx=0x7efc2c011c40 comp rx=0 tx=0).stop 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.500+0000 7efc40e00700 1 -- 172.21.15.83:0/1721664656 shutdown_connections 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.500+0000 7efc40e00700 1 --2- 172.21.15.83:0/1721664656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc3c108fe0 0x7efc3c19e730 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.500+0000 7efc40e00700 1 -- 172.21.15.83:0/1721664656 >> 172.21.15.83:0/1721664656 conn(0x7efc3c102a10 msgr2=0x7efc3c103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.501+0000 7efc40e00700 1 -- 172.21.15.83:0/1721664656 shutdown_connections 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:00.501+0000 7efc40e00700 1 -- 172.21.15.83:0/1721664656 wait complete. 2024-04-23T14:37:01.257 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (1/15)... 2024-04-23T14:37:02.757 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:02 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:02.439+0000 7f2a39834200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T14:37:02.757 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:02 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:02.663+0000 7f2a39834200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T14:37:03.253 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:02 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:02.877+0000 7f2a39834200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T14:37:03.253 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:03 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:03.252+0000 7f2a39834200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T14:37:03.506 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:03 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:03.377+0000 7f2a39834200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T14:37:03.812 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:03 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:03.635+0000 7f2a39834200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T14:37:04.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:04 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/4212585307' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T14:37:04.257 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:03 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:03.902+0000 7f2a39834200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:37:04.661 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:37:04.661 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-23T14:37:04.661 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "5f4fe708-017e-11ef-bc93-c7b262605968", 2024-04-23T14:37:04.661 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T14:37:04.661 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T14:37:04.661 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T14:37:04.661 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T14:37:04.661 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:04.661 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:04.662 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T14:37:04.663 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-23T14:37:04.664 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T14:37:04.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:04.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T14:37:04.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:04.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T14:36:49.929071+0000", 2024-04-23T14:37:04.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T14:37:04.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:04.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T14:37:04.665 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.860+0000 7f9590800700 1 Processor -- start 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.860+0000 7f9590800700 1 -- start start 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.860+0000 7f9590800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 0x7f9588108b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.860+0000 7f9590800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9588109100 con 0x7f9588108710 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.861+0000 7f958de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 0x7f9588108b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.861+0000 7f958de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 0x7f9588108b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48894/0 (socket says 172.21.15.83:48894) 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.861+0000 7f958de00700 1 -- 172.21.15.83:0/2705361740 learned_addr learned my addr 172.21.15.83:0/2705361740 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.861+0000 7f958de00700 1 -- 172.21.15.83:0/2705361740 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95881098c0 con 0x7f9588108710 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.861+0000 7f958de00700 1 --2- 172.21.15.83:0/2705361740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 0x7f9588108b30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f957c0098a0 tx=0x7f957c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5dfe27a03bb96e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.862+0000 7f958ca00700 1 -- 172.21.15.83:0/2705361740 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f957c004160 con 0x7f9588108710 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.862+0000 7f958ca00700 1 -- 172.21.15.83:0/2705361740 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f957c004300 con 0x7f9588108710 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.862+0000 7f958ca00700 1 -- 172.21.15.83:0/2705361740 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f957c011b50 con 0x7f9588108710 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.862+0000 7f9590800700 1 -- 172.21.15.83:0/2705361740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 msgr2=0x7f9588108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.666 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.862+0000 7f9590800700 1 --2- 172.21.15.83:0/2705361740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 0x7f9588108b30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f957c0098a0 tx=0x7f957c009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.863+0000 7f9590800700 1 -- 172.21.15.83:0/2705361740 shutdown_connections 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.863+0000 7f9590800700 1 --2- 172.21.15.83:0/2705361740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 0x7f9588108b30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.863+0000 7f9590800700 1 -- 172.21.15.83:0/2705361740 >> 172.21.15.83:0/2705361740 conn(0x7f9588103d70 msgr2=0x7f9588106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.863+0000 7f9590800700 1 -- 172.21.15.83:0/2705361740 shutdown_connections 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.863+0000 7f9590800700 1 -- 172.21.15.83:0/2705361740 wait complete. 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.864+0000 7f9590800700 1 Processor -- start 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.864+0000 7f9590800700 1 -- start start 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.864+0000 7f9590800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 0x7f958819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.864+0000 7f9590800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f958810cc70 con 0x7f9588108710 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.865+0000 7f958de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 0x7f958819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.865+0000 7f958de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 0x7f958819e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48902/0 (socket says 172.21.15.83:48902) 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.865+0000 7f958de00700 1 -- 172.21.15.83:0/4212585307 learned_addr learned my addr 172.21.15.83:0/4212585307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.865+0000 7f958de00700 1 -- 172.21.15.83:0/4212585307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958819ee50 con 0x7f9588108710 2024-04-23T14:37:04.667 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.865+0000 7f958de00700 1 --2- 172.21.15.83:0/4212585307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 0x7f958819e910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f957c011850 tx=0x7f957c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.865+0000 7f9582a00700 1 -- 172.21.15.83:0/4212585307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f957c003af0 con 0x7f9588108710 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.866+0000 7f9582a00700 1 -- 172.21.15.83:0/4212585307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f957c020070 con 0x7f9588108710 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.866+0000 7f9590800700 1 -- 172.21.15.83:0/4212585307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958819f0e0 con 0x7f9588108710 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.866+0000 7f9582a00700 1 -- 172.21.15.83:0/4212585307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f957c0195a0 con 0x7f9588108710 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.866+0000 7f9590800700 1 -- 172.21.15.83:0/4212585307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95881a1dd0 con 0x7f9588108710 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.866+0000 7f9582a00700 1 -- 172.21.15.83:0/4212585307 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f957c019be0 con 0x7f9588108710 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.866+0000 7f9582a00700 1 -- 172.21.15.83:0/4212585307 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f957c011e00 con 0x7f9588108710 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.867+0000 7f9590800700 1 -- 172.21.15.83:0/4212585307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9588064150 con 0x7f9588108710 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.869+0000 7f9582a00700 1 -- 172.21.15.83:0/4212585307 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f957c003c90 con 0x7f9588108710 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.949+0000 7f9590800700 1 -- 172.21.15.83:0/4212585307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f95881a1fa0 con 0x7f9588108710 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.950+0000 7f9582a00700 1 -- 172.21.15.83:0/4212585307 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f957c025460 con 0x7f9588108710 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.951+0000 7f9590800700 1 -- 172.21.15.83:0/4212585307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 msgr2=0x7f958819e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.951+0000 7f9590800700 1 --2- 172.21.15.83:0/4212585307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 0x7f958819e910 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f957c011850 tx=0x7f957c011c10 comp rx=0 tx=0).stop 2024-04-23T14:37:04.668 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.951+0000 7f9590800700 1 -- 172.21.15.83:0/4212585307 shutdown_connections 2024-04-23T14:37:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.951+0000 7f9590800700 1 --2- 172.21.15.83:0/4212585307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9588108710 0x7f958819e910 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.951+0000 7f9590800700 1 -- 172.21.15.83:0/4212585307 >> 172.21.15.83:0/4212585307 conn(0x7f9588103d70 msgr2=0x7f9588104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.952+0000 7f9590800700 1 -- 172.21.15.83:0/4212585307 shutdown_connections 2024-04-23T14:37:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:03.952+0000 7f9590800700 1 -- 172.21.15.83:0/4212585307 wait complete. 2024-04-23T14:37:04.669 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (2/15)... 2024-04-23T14:37:05.007 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:04 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:04.709+0000 7f2a39834200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T14:37:05.504 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:05 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:05.251+0000 7f2a39834200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T14:37:05.504 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:05 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:05.374+0000 7f2a39834200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T14:37:05.756 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:05 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:05.638+0000 7f2a39834200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T14:37:06.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:06 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:06.277+0000 7f2a39834200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T14:37:06.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:06 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:06.398+0000 7f2a39834200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T14:37:07.006 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:06 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:06.726+0000 7f2a39834200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T14:37:07.006 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:06 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:06.911+0000 7f2a39834200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T14:37:07.304 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:07 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:07.039+0000 7f2a39834200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T14:37:07.304 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:07 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:07.190+0000 7f2a39834200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T14:37:07.567 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:07 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1527758384' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T14:37:08.006 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:07 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:07.566+0000 7f2a39834200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "5f4fe708-017e-11ef-bc93-c7b262605968", 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T14:37:08.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T14:37:08.069 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T14:37:08.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:08.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T14:37:08.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T14:37:08.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T14:37:08.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T14:37:08.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T14:37:08.070 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T14:37:08.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T14:37:08.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T14:37:08.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T14:37:08.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:08.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T14:37:08.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:08.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T14:37:08.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T14:37:08.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:08.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T14:37:08.078 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T14:36:49.929071+0000", 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-23T14:37:08.079 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.303+0000 7f75e2600700 1 Processor -- start 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.304+0000 7f75e2600700 1 -- start start 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.304+0000 7f75e2600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 0x7f75dc07c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.304+0000 7f75e2600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc07c840 con 0x7f75dc07de50 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.304+0000 7f75e1200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 0x7f75dc07c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.305+0000 7f75e1200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 0x7f75dc07c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56790/0 (socket says 172.21.15.83:56790) 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.305+0000 7f75e1200700 1 -- 172.21.15.83:0/3324195181 learned_addr learned my addr 172.21.15.83:0/3324195181 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.305+0000 7f75e1200700 1 -- 172.21.15.83:0/3324195181 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc07c9c0 con 0x7f75dc07de50 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.305+0000 7f75e1200700 1 --2- 172.21.15.83:0/3324195181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 0x7f75dc07c300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f75d00098a0 tx=0x7f75d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dbea71188bef97 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.306+0000 7f75db400700 1 -- 172.21.15.83:0/3324195181 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75d0004160 con 0x7f75dc07de50 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.306+0000 7f75db400700 1 -- 172.21.15.83:0/3324195181 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f75d0004300 con 0x7f75dc07de50 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.306+0000 7f75db400700 1 -- 172.21.15.83:0/3324195181 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75d0011b50 con 0x7f75dc07de50 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.306+0000 7f75e2600700 1 -- 172.21.15.83:0/3324195181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 msgr2=0x7f75dc07c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:08.080 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.306+0000 7f75e2600700 1 --2- 172.21.15.83:0/3324195181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 0x7f75dc07c300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f75d00098a0 tx=0x7f75d0009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.306+0000 7f75e2600700 1 -- 172.21.15.83:0/3324195181 shutdown_connections 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.306+0000 7f75e2600700 1 --2- 172.21.15.83:0/3324195181 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 0x7f75dc07c300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.306+0000 7f75e2600700 1 -- 172.21.15.83:0/3324195181 >> 172.21.15.83:0/3324195181 conn(0x7f75dc103ac0 msgr2=0x7f75dc105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.306+0000 7f75e2600700 1 -- 172.21.15.83:0/3324195181 shutdown_connections 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.307+0000 7f75e2600700 1 -- 172.21.15.83:0/3324195181 wait complete. 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.307+0000 7f75e2600700 1 Processor -- start 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.308+0000 7f75e2600700 1 -- start start 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.308+0000 7f75e2600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 0x7f75dc19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.308+0000 7f75e2600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc10eaf0 con 0x7f75dc07de50 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.308+0000 7f75e1200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 0x7f75dc19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.308+0000 7f75e1200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 0x7f75dc19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56794/0 (socket says 172.21.15.83:56794) 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.308+0000 7f75e1200700 1 -- 172.21.15.83:0/1527758384 learned_addr learned my addr 172.21.15.83:0/1527758384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.309+0000 7f75e1200700 1 -- 172.21.15.83:0/1527758384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc19ebc0 con 0x7f75dc07de50 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.309+0000 7f75e1200700 1 --2- 172.21.15.83:0/1527758384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 0x7f75dc19e680 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f75d0011b60 tx=0x7f75d0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.309+0000 7f75d9600700 1 -- 172.21.15.83:0/1527758384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75d0003b20 con 0x7f75dc07de50 2024-04-23T14:37:08.081 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.309+0000 7f75d9600700 1 -- 172.21.15.83:0/1527758384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f75d0003cc0 con 0x7f75dc07de50 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.309+0000 7f75e2600700 1 -- 172.21.15.83:0/1527758384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75dc19ee50 con 0x7f75dc07de50 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.309+0000 7f75d9600700 1 -- 172.21.15.83:0/1527758384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75d0024840 con 0x7f75dc07de50 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.309+0000 7f75e2600700 1 -- 172.21.15.83:0/1527758384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75dc1a1b40 con 0x7f75dc07de50 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.310+0000 7f75d9600700 1 -- 172.21.15.83:0/1527758384 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f75d0022020 con 0x7f75dc07de50 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.310+0000 7f75d9600700 1 -- 172.21.15.83:0/1527758384 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f75d0011e00 con 0x7f75dc07de50 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.311+0000 7f75e2600700 1 -- 172.21.15.83:0/1527758384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75dc064150 con 0x7f75dc07de50 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.313+0000 7f75d9600700 1 -- 172.21.15.83:0/1527758384 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f75d0043e50 con 0x7f75dc07de50 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.393+0000 7f75e2600700 1 -- 172.21.15.83:0/1527758384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f75dc1a2160 con 0x7f75dc07de50 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.394+0000 7f75d9600700 1 -- 172.21.15.83:0/1527758384 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f75d0032030 con 0x7f75dc07de50 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.395+0000 7f75e2600700 1 -- 172.21.15.83:0/1527758384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 msgr2=0x7f75dc19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.395+0000 7f75e2600700 1 --2- 172.21.15.83:0/1527758384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 0x7f75dc19e680 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f75d0011b60 tx=0x7f75d0011c40 comp rx=0 tx=0).stop 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.395+0000 7f75e2600700 1 -- 172.21.15.83:0/1527758384 shutdown_connections 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.395+0000 7f75e2600700 1 --2- 172.21.15.83:0/1527758384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75dc07de50 0x7f75dc19e680 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:08.082 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.395+0000 7f75e2600700 1 -- 172.21.15.83:0/1527758384 >> 172.21.15.83:0/1527758384 conn(0x7f75dc103ac0 msgr2=0x7f75dc104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:08.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.395+0000 7f75e2600700 1 -- 172.21.15.83:0/1527758384 shutdown_connections 2024-04-23T14:37:08.083 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:07.395+0000 7f75e2600700 1 -- 172.21.15.83:0/1527758384 wait complete. 2024-04-23T14:37:08.083 INFO:teuthology.orchestra.run.smithi083.stdout:mgr not available, waiting (3/15)... 2024-04-23T14:37:08.610 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:08.329+0000 7f2a39834200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T14:37:09.007 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:08.609+0000 7f2a39834200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T14:37:09.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: Activating manager daemon x 2024-04-23T14:37:09.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: mgrmap e2: x(active, starting, since 0.00974995s) 2024-04-23T14:37:09.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: from='mgr.14100 172.21.15.83:0/2126704879' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T14:37:09.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: from='mgr.14100 172.21.15.83:0/2126704879' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T14:37:09.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: from='mgr.14100 172.21.15.83:0/2126704879' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T14:37:09.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: from='mgr.14100 172.21.15.83:0/2126704879' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:37:09.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: from='mgr.14100 172.21.15.83:0/2126704879' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T14:37:09.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: Manager daemon x is now available 2024-04-23T14:37:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: from='mgr.14100 172.21.15.83:0/2126704879' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T14:37:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: from='mgr.14100 172.21.15.83:0/2126704879' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T14:37:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: from='mgr.14100 172.21.15.83:0/2126704879' entity='mgr.x' 2024-04-23T14:37:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: from='mgr.14100 172.21.15.83:0/2126704879' entity='mgr.x' 2024-04-23T14:37:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:08 smithi083 ceph-mon[27867]: from='mgr.14100 172.21.15.83:0/2126704879' entity='mgr.x' 2024-04-23T14:37:11.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:10 smithi083 ceph-mon[27867]: mgrmap e3: x(active, since 1.01943s) 2024-04-23T14:37:11.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:11 smithi083 ceph-mon[27867]: mgrmap e4: x(active, since 2s) 2024-04-23T14:37:11.703 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:11 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/4233284889' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T14:37:11.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:37:11.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-23T14:37:11.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsid": "5f4fe708-017e-11ef-bc93-c7b262605968", 2024-04-23T14:37:11.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T14:37:11.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T14:37:11.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T14:37:11.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T14:37:11.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:11.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T14:37:11.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T14:37:11.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 0 2024-04-23T14:37:11.721 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-23T14:37:11.721 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T14:37:11.722 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T14:37:11.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T14:37:11.723 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T14:37:11.727 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T14:37:11.727 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:11.727 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T14:37:11.727 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T14:37:11.727 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T14:37:11.727 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T14:37:11.727 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T14:37:11.728 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ], 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T14:36:49.929071+0000", 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout }, 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-23T14:37:11.729 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.711+0000 7f03d4a00700 1 Processor -- start 2024-04-23T14:37:11.730 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.711+0000 7f03d4a00700 1 -- start start 2024-04-23T14:37:11.730 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.712+0000 7f03d4a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 0x7f03d007eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.730 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.712+0000 7f03d4a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03d007f270 con 0x7f03d007e880 2024-04-23T14:37:11.730 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.712+0000 7f03cde00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 0x7f03d007eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.730 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.712+0000 7f03cde00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 0x7f03d007eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56868/0 (socket says 172.21.15.83:56868) 2024-04-23T14:37:11.730 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.712+0000 7f03cde00700 1 -- 172.21.15.83:0/4169679762 learned_addr learned my addr 172.21.15.83:0/4169679762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:11.735 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.713+0000 7f03cde00700 1 -- 172.21.15.83:0/4169679762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d007fa80 con 0x7f03d007e880 2024-04-23T14:37:11.735 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.713+0000 7f03cde00700 1 --2- 172.21.15.83:0/4169679762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 0x7f03d007eca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f03c00098a0 tx=0x7f03c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c4274136cc50f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:11.735 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.713+0000 7f03cd400700 1 -- 172.21.15.83:0/4169679762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03c0004160 con 0x7f03d007e880 2024-04-23T14:37:11.735 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.713+0000 7f03cd400700 1 -- 172.21.15.83:0/4169679762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03c0004300 con 0x7f03d007e880 2024-04-23T14:37:11.735 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.713+0000 7f03cd400700 1 -- 172.21.15.83:0/4169679762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03c0011b50 con 0x7f03d007e880 2024-04-23T14:37:11.735 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.714+0000 7f03d4a00700 1 -- 172.21.15.83:0/4169679762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 msgr2=0x7f03d007eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.735 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.714+0000 7f03d4a00700 1 --2- 172.21.15.83:0/4169679762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 0x7f03d007eca0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f03c00098a0 tx=0x7f03c0009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.735 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.714+0000 7f03d4a00700 1 -- 172.21.15.83:0/4169679762 shutdown_connections 2024-04-23T14:37:11.735 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.714+0000 7f03d4a00700 1 --2- 172.21.15.83:0/4169679762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 0x7f03d007eca0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.735 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.714+0000 7f03d4a00700 1 -- 172.21.15.83:0/4169679762 >> 172.21.15.83:0/4169679762 conn(0x7f03d007def0 msgr2=0x7f03d01084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:11.735 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.714+0000 7f03d4a00700 1 -- 172.21.15.83:0/4169679762 shutdown_connections 2024-04-23T14:37:11.735 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.714+0000 7f03d4a00700 1 -- 172.21.15.83:0/4169679762 wait complete. 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.715+0000 7f03d4a00700 1 Processor -- start 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.716+0000 7f03d4a00700 1 -- start start 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.716+0000 7f03d4a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 0x7f03d01a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.716+0000 7f03d4a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03d0080e40 con 0x7f03d007e880 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.716+0000 7f03cde00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 0x7f03d01a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.716+0000 7f03cde00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 0x7f03d01a2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56884/0 (socket says 172.21.15.83:56884) 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.716+0000 7f03cde00700 1 -- 172.21.15.83:0/4233284889 learned_addr learned my addr 172.21.15.83:0/4233284889 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.716+0000 7f03cde00700 1 -- 172.21.15.83:0/4233284889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d01a3200 con 0x7f03d007e880 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.717+0000 7f03cde00700 1 --2- 172.21.15.83:0/4233284889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 0x7f03d01a2cc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f03c0011b60 tx=0x7f03c0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.717+0000 7f03c6a00700 1 -- 172.21.15.83:0/4233284889 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03c0003970 con 0x7f03d007e880 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.717+0000 7f03c6a00700 1 -- 172.21.15.83:0/4233284889 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03c0003b10 con 0x7f03d007e880 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.717+0000 7f03c6a00700 1 -- 172.21.15.83:0/4233284889 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03c0024610 con 0x7f03d007e880 2024-04-23T14:37:11.736 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.717+0000 7f03d4a00700 1 -- 172.21.15.83:0/4233284889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d01a3490 con 0x7f03d007e880 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.717+0000 7f03d4a00700 1 -- 172.21.15.83:0/4233284889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d01a6180 con 0x7f03d007e880 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.718+0000 7f03c6a00700 1 -- 172.21.15.83:0/4233284889 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f03c001a9f0 con 0x7f03d007e880 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.718+0000 7f03d4a00700 1 -- 172.21.15.83:0/4233284889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d0064150 con 0x7f03d007e880 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.719+0000 7f03c6a00700 1 --2- 172.21.15.83:0/4233284889 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7f03b4039560 0x7f03b403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.719+0000 7f03c6a00700 1 -- 172.21.15.83:0/4233284889 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f03c0011e00 con 0x7f03d007e880 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.719+0000 7f03c7e00700 1 --2- 172.21.15.83:0/4233284889 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7f03b4039560 0x7f03b403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.720+0000 7f03c7e00700 1 --2- 172.21.15.83:0/4233284889 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7f03b4039560 0x7f03b403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f03b8006fd0 tx=0x7f03b8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:10.725+0000 7f03c6a00700 1 -- 172.21.15.83:0/4233284889 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03c000b3e0 con 0x7f03d007e880 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:11.032+0000 7f03d4a00700 1 -- 172.21.15.83:0/4233284889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f03d002e2a0 con 0x7f03d007e880 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:11.033+0000 7f03c6a00700 1 -- 172.21.15.83:0/4233284889 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f03c004d250 con 0x7f03d007e880 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:11.036+0000 7f03d4a00700 1 -- 172.21.15.83:0/4233284889 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7f03b4039560 msgr2=0x7f03b403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:11.036+0000 7f03d4a00700 1 --2- 172.21.15.83:0/4233284889 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7f03b4039560 0x7f03b403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f03b8006fd0 tx=0x7f03b8006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:11.036+0000 7f03d4a00700 1 -- 172.21.15.83:0/4233284889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 msgr2=0x7f03d01a2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:11.036+0000 7f03d4a00700 1 --2- 172.21.15.83:0/4233284889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 0x7f03d01a2cc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f03c0011b60 tx=0x7f03c0011c40 comp rx=0 tx=0).stop 2024-04-23T14:37:11.737 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:11.036+0000 7f03d4a00700 1 -- 172.21.15.83:0/4233284889 shutdown_connections 2024-04-23T14:37:11.738 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:11.036+0000 7f03d4a00700 1 --2- 172.21.15.83:0/4233284889 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7f03b4039560 0x7f03b403ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.738 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:11.036+0000 7f03d4a00700 1 --2- 172.21.15.83:0/4233284889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03d007e880 0x7f03d01a2cc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:11.738 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:11.036+0000 7f03d4a00700 1 -- 172.21.15.83:0/4233284889 >> 172.21.15.83:0/4233284889 conn(0x7f03d007def0 msgr2=0x7f03d01999e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:11.738 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:11.037+0000 7f03d4a00700 1 -- 172.21.15.83:0/4233284889 shutdown_connections 2024-04-23T14:37:11.738 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:11.037+0000 7f03d4a00700 1 -- 172.21.15.83:0/4233284889 wait complete. 2024-04-23T14:37:11.738 INFO:teuthology.orchestra.run.smithi083.stdout:mgr is available 2024-04-23T14:37:12.966 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:12 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2000016279' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T14:37:13.306 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:37:13.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T14:37:13.307 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout fsid = 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:37:13.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T14:37:13.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T14:37:13.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T14:37:13.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T14:37:13.308 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:37:13.309 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T14:37:13.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T14:37:13.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 2024-04-23T14:37:13.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T14:37:13.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout filestore_flush_min = 0 2024-04-23T14:37:13.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T14:37:13.310 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T14:37:13.311 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T14:37:13.311 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T14:37:13.311 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T14:37:13.311 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.350+0000 7fbd0b200700 1 Processor -- start 2024-04-23T14:37:13.311 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.350+0000 7fbd0b200700 1 -- start start 2024-04-23T14:37:13.311 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.350+0000 7fbd0b200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 0x7fbd0c10f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:13.312 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.350+0000 7fbd0b200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c10f9b0 con 0x7fbd0c10efe0 2024-04-23T14:37:13.313 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.351+0000 7fbd09e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 0x7fbd0c10f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:13.313 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.351+0000 7fbd09e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 0x7fbd0c10f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56892/0 (socket says 172.21.15.83:56892) 2024-04-23T14:37:13.313 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.351+0000 7fbd09e00700 1 -- 172.21.15.83:0/502424046 learned_addr learned my addr 172.21.15.83:0/502424046 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:13.313 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.351+0000 7fbd09e00700 1 -- 172.21.15.83:0/502424046 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c10fb30 con 0x7fbd0c10efe0 2024-04-23T14:37:13.313 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.352+0000 7fbd09e00700 1 --2- 172.21.15.83:0/502424046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 0x7fbd0c10f3e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0098a0 tx=0x7fbcfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b766223e48f18b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:13.313 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.352+0000 7fbd09400700 1 -- 172.21.15.83:0/502424046 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcfc004160 con 0x7fbd0c10efe0 2024-04-23T14:37:13.313 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.352+0000 7fbd09400700 1 -- 172.21.15.83:0/502424046 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbcfc004300 con 0x7fbd0c10efe0 2024-04-23T14:37:13.313 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.352+0000 7fbd09400700 1 -- 172.21.15.83:0/502424046 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcfc011b50 con 0x7fbd0c10efe0 2024-04-23T14:37:13.314 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.353+0000 7fbd0b200700 1 -- 172.21.15.83:0/502424046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 msgr2=0x7fbd0c10f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:13.314 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.353+0000 7fbd0b200700 1 --2- 172.21.15.83:0/502424046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 0x7fbd0c10f3e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0098a0 tx=0x7fbcfc009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:13.315 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.353+0000 7fbd0b200700 1 -- 172.21.15.83:0/502424046 shutdown_connections 2024-04-23T14:37:13.315 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.353+0000 7fbd0b200700 1 --2- 172.21.15.83:0/502424046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 0x7fbd0c10f3e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:13.315 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.353+0000 7fbd0b200700 1 -- 172.21.15.83:0/502424046 >> 172.21.15.83:0/502424046 conn(0x7fbd0c07ddf0 msgr2=0x7fbd0c080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:13.315 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.353+0000 7fbd0b200700 1 -- 172.21.15.83:0/502424046 shutdown_connections 2024-04-23T14:37:13.315 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.353+0000 7fbd0b200700 1 -- 172.21.15.83:0/502424046 wait complete. 2024-04-23T14:37:13.315 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.354+0000 7fbd0b200700 1 Processor -- start 2024-04-23T14:37:13.316 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.355+0000 7fbd0b200700 1 -- start start 2024-04-23T14:37:13.316 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.355+0000 7fbd0b200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 0x7fbd0c1a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:13.316 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.355+0000 7fbd0b200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c112fd0 con 0x7fbd0c10efe0 2024-04-23T14:37:13.316 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.355+0000 7fbd09e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 0x7fbd0c1a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:13.316 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.355+0000 7fbd09e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 0x7fbd0c1a2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56906/0 (socket says 172.21.15.83:56906) 2024-04-23T14:37:13.316 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.355+0000 7fbd09e00700 1 -- 172.21.15.83:0/2000016279 learned_addr learned my addr 172.21.15.83:0/2000016279 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:13.317 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.356+0000 7fbd09e00700 1 -- 172.21.15.83:0/2000016279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c1a2fd0 con 0x7fbd0c10efe0 2024-04-23T14:37:13.317 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.356+0000 7fbd09e00700 1 --2- 172.21.15.83:0/2000016279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 0x7fbd0c1a2a90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc011b60 tx=0x7fbcfc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:13.317 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.356+0000 7fbd02a00700 1 -- 172.21.15.83:0/2000016279 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcfc003970 con 0x7fbd0c10efe0 2024-04-23T14:37:13.318 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.356+0000 7fbd02a00700 1 -- 172.21.15.83:0/2000016279 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbcfc003b10 con 0x7fbd0c10efe0 2024-04-23T14:37:13.318 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.356+0000 7fbd02a00700 1 -- 172.21.15.83:0/2000016279 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcfc024610 con 0x7fbd0c10efe0 2024-04-23T14:37:13.318 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.356+0000 7fbd0b200700 1 -- 172.21.15.83:0/2000016279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0c1a3260 con 0x7fbd0c10efe0 2024-04-23T14:37:13.318 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.357+0000 7fbd0b200700 1 -- 172.21.15.83:0/2000016279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0c1a5f50 con 0x7fbd0c10efe0 2024-04-23T14:37:13.318 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.358+0000 7fbd02a00700 1 -- 172.21.15.83:0/2000016279 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fbcfc01a9f0 con 0x7fbd0c10efe0 2024-04-23T14:37:13.318 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.358+0000 7fbd0b200700 1 -- 172.21.15.83:0/2000016279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd0c113150 con 0x7fbd0c10efe0 2024-04-23T14:37:13.319 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.358+0000 7fbd02a00700 1 --2- 172.21.15.83:0/2000016279 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fbcf00395b0 0x7fbcf003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:13.319 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.358+0000 7fbd02a00700 1 -- 172.21.15.83:0/2000016279 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbcfc011e00 con 0x7fbd0c10efe0 2024-04-23T14:37:13.319 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.359+0000 7fbd03e00700 1 --2- 172.21.15.83:0/2000016279 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fbcf00395b0 0x7fbcf003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:13.319 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.359+0000 7fbd03e00700 1 --2- 172.21.15.83:0/2000016279 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fbcf00395b0 0x7fbcf003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4006fd0 tx=0x7fbcf4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:13.319 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.364+0000 7fbd02a00700 1 -- 172.21.15.83:0/2000016279 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcfc00b5a0 con 0x7fbd0c10efe0 2024-04-23T14:37:13.319 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.584+0000 7fbd0b200700 1 -- 172.21.15.83:0/2000016279 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbd0c064150 con 0x7fbd0c10efe0 2024-04-23T14:37:13.320 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.586+0000 7fbd02a00700 1 -- 172.21.15.83:0/2000016279 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+398 (secure 0 0 0) 0x7fbcfc04d210 con 0x7fbd0c10efe0 2024-04-23T14:37:13.320 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.589+0000 7fbd0b200700 1 -- 172.21.15.83:0/2000016279 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fbcf00395b0 msgr2=0x7fbcf003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:13.321 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.589+0000 7fbd0b200700 1 --2- 172.21.15.83:0/2000016279 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fbcf00395b0 0x7fbcf003ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4006fd0 tx=0x7fbcf4006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:13.321 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.589+0000 7fbd0b200700 1 -- 172.21.15.83:0/2000016279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 msgr2=0x7fbd0c1a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:13.321 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.589+0000 7fbd0b200700 1 --2- 172.21.15.83:0/2000016279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 0x7fbd0c1a2a90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc011b60 tx=0x7fbcfc011c40 comp rx=0 tx=0).stop 2024-04-23T14:37:13.321 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.589+0000 7fbd0b200700 1 -- 172.21.15.83:0/2000016279 shutdown_connections 2024-04-23T14:37:13.321 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.589+0000 7fbd0b200700 1 --2- 172.21.15.83:0/2000016279 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fbcf00395b0 0x7fbcf003ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:13.321 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.589+0000 7fbd0b200700 1 --2- 172.21.15.83:0/2000016279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd0c10efe0 0x7fbd0c1a2a90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:13.322 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.589+0000 7fbd0b200700 1 -- 172.21.15.83:0/2000016279 >> 172.21.15.83:0/2000016279 conn(0x7fbd0c07ddf0 msgr2=0x7fbd0c07f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:13.322 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.589+0000 7fbd0b200700 1 -- 172.21.15.83:0/2000016279 shutdown_connections 2024-04-23T14:37:13.322 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:12.589+0000 7fbd0b200700 1 -- 172.21.15.83:0/2000016279 wait complete. 2024-04-23T14:37:13.322 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling cephadm module... 2024-04-23T14:37:14.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:14 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1351369763' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-23T14:37:14.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:14 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: ignoring --setuser ceph since I am not root 2024-04-23T14:37:14.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:14 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: ignoring --setgroup ceph since I am not root 2024-04-23T14:37:14.885 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:14 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:14.557+0000 7f7e93a8f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T14:37:14.886 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:14 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:14.794+0000 7f7e93a8f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:37:15.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.926+0000 7f8d76400700 1 Processor -- start 2024-04-23T14:37:15.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.926+0000 7f8d76400700 1 -- start start 2024-04-23T14:37:15.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.926+0000 7f8d76400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 0x7f8d7010f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:15.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.926+0000 7f8d76400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7010f9b0 con 0x7f8d7010efe0 2024-04-23T14:37:15.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.927+0000 7f8d75000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 0x7f8d7010f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:15.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.927+0000 7f8d75000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 0x7f8d7010f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56914/0 (socket says 172.21.15.83:56914) 2024-04-23T14:37:15.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.927+0000 7f8d75000700 1 -- 172.21.15.83:0/2479841207 learned_addr learned my addr 172.21.15.83:0/2479841207 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:15.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.927+0000 7f8d75000700 1 -- 172.21.15.83:0/2479841207 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7010fb30 con 0x7f8d7010efe0 2024-04-23T14:37:15.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.928+0000 7f8d75000700 1 --2- 172.21.15.83:0/2479841207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 0x7f8d7010f3e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8d640098a0 tx=0x7f8d64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550038e12f2d3379 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:15.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.928+0000 7f8d6f400700 1 -- 172.21.15.83:0/2479841207 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d64004160 con 0x7f8d7010efe0 2024-04-23T14:37:15.064 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.928+0000 7f8d6f400700 1 -- 172.21.15.83:0/2479841207 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d64004300 con 0x7f8d7010efe0 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.928+0000 7f8d6f400700 1 -- 172.21.15.83:0/2479841207 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d64011b50 con 0x7f8d7010efe0 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.929+0000 7f8d76400700 1 -- 172.21.15.83:0/2479841207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 msgr2=0x7f8d7010f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.929+0000 7f8d76400700 1 --2- 172.21.15.83:0/2479841207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 0x7f8d7010f3e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8d640098a0 tx=0x7f8d64009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.929+0000 7f8d76400700 1 -- 172.21.15.83:0/2479841207 shutdown_connections 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.929+0000 7f8d76400700 1 --2- 172.21.15.83:0/2479841207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 0x7f8d7010f3e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.929+0000 7f8d76400700 1 -- 172.21.15.83:0/2479841207 >> 172.21.15.83:0/2479841207 conn(0x7f8d7007ddf0 msgr2=0x7f8d70080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.929+0000 7f8d76400700 1 -- 172.21.15.83:0/2479841207 shutdown_connections 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.929+0000 7f8d76400700 1 -- 172.21.15.83:0/2479841207 wait complete. 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.930+0000 7f8d76400700 1 Processor -- start 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.930+0000 7f8d76400700 1 -- start start 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.931+0000 7f8d76400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 0x7f8d701a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.931+0000 7f8d76400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70112fd0 con 0x7f8d7010efe0 2024-04-23T14:37:15.065 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.931+0000 7f8d75000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 0x7f8d701a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.931+0000 7f8d75000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 0x7f8d701a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56926/0 (socket says 172.21.15.83:56926) 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.931+0000 7f8d75000700 1 -- 172.21.15.83:0/1351369763 learned_addr learned my addr 172.21.15.83:0/1351369763 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.931+0000 7f8d75000700 1 -- 172.21.15.83:0/1351369763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d701a3010 con 0x7f8d7010efe0 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.932+0000 7f8d75000700 1 --2- 172.21.15.83:0/1351369763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 0x7f8d701a2ad0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8d64011b60 tx=0x7f8d64011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.932+0000 7f8d6d600700 1 -- 172.21.15.83:0/1351369763 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d64003970 con 0x7f8d7010efe0 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.932+0000 7f8d6d600700 1 -- 172.21.15.83:0/1351369763 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d64003b10 con 0x7f8d7010efe0 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.932+0000 7f8d6d600700 1 -- 172.21.15.83:0/1351369763 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d64024610 con 0x7f8d7010efe0 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.932+0000 7f8d76400700 1 -- 172.21.15.83:0/1351369763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d701a32a0 con 0x7f8d7010efe0 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.932+0000 7f8d76400700 1 -- 172.21.15.83:0/1351369763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d701a5f90 con 0x7f8d7010efe0 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.933+0000 7f8d6d600700 1 -- 172.21.15.83:0/1351369763 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f8d6401a9f0 con 0x7f8d7010efe0 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.933+0000 7f8d76400700 1 -- 172.21.15.83:0/1351369763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d70113150 con 0x7f8d7010efe0 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.934+0000 7f8d6d600700 1 --2- 172.21.15.83:0/1351369763 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7f8d580395b0 0x7f8d5803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:15.066 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.934+0000 7f8d6d600700 1 -- 172.21.15.83:0/1351369763 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8d64011e00 con 0x7f8d7010efe0 2024-04-23T14:37:15.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.934+0000 7f8d6fe00700 1 --2- 172.21.15.83:0/1351369763 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7f8d580395b0 0x7f8d5803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:15.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.935+0000 7f8d6fe00700 1 --2- 172.21.15.83:0/1351369763 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7f8d580395b0 0x7f8d5803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c006fd0 tx=0x7f8d5c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:15.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:13.939+0000 7f8d6d600700 1 -- 172.21.15.83:0/1351369763 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d64016260 con 0x7f8d7010efe0 2024-04-23T14:37:15.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.217+0000 7f8d76400700 1 -- 172.21.15.83:0/1351369763 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f8d701a65e0 con 0x7f8d7010efe0 2024-04-23T14:37:15.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.271+0000 7f8d6d600700 1 -- 172.21.15.83:0/1351369763 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f8d64019e10 con 0x7f8d7010efe0 2024-04-23T14:37:15.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.274+0000 7f8d6d600700 1 -- 172.21.15.83:0/1351369763 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f8d64024c40 con 0x7f8d7010efe0 2024-04-23T14:37:15.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.276+0000 7f8d76400700 1 -- 172.21.15.83:0/1351369763 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7f8d580395b0 msgr2=0x7f8d5803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:15.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.276+0000 7f8d76400700 1 --2- 172.21.15.83:0/1351369763 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7f8d580395b0 0x7f8d5803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c006fd0 tx=0x7f8d5c006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:15.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.276+0000 7f8d76400700 1 -- 172.21.15.83:0/1351369763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 msgr2=0x7f8d701a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:15.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.276+0000 7f8d76400700 1 --2- 172.21.15.83:0/1351369763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 0x7f8d701a2ad0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8d64011b60 tx=0x7f8d64011c40 comp rx=0 tx=0).stop 2024-04-23T14:37:15.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.277+0000 7f8d76400700 1 -- 172.21.15.83:0/1351369763 shutdown_connections 2024-04-23T14:37:15.067 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.277+0000 7f8d76400700 1 --2- 172.21.15.83:0/1351369763 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7f8d580395b0 0x7f8d5803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:15.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.277+0000 7f8d76400700 1 --2- 172.21.15.83:0/1351369763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d7010efe0 0x7f8d701a2ad0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:15.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.277+0000 7f8d76400700 1 -- 172.21.15.83:0/1351369763 >> 172.21.15.83:0/1351369763 conn(0x7f8d7007ddf0 msgr2=0x7f8d7007f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:15.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.277+0000 7f8d76400700 1 -- 172.21.15.83:0/1351369763 shutdown_connections 2024-04-23T14:37:15.068 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:14.277+0000 7f8d76400700 1 -- 172.21.15.83:0/1351369763 wait complete. 2024-04-23T14:37:15.206 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:14 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:14.925+0000 7f7e93a8f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T14:37:15.503 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:15 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1351369763' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-23T14:37:15.503 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:15 smithi083 ceph-mon[27867]: mgrmap e5: x(active, since 5s) 2024-04-23T14:37:15.503 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:15 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:15.331+0000 7f7e93a8f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T14:37:15.504 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:15 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:15.451+0000 7f7e93a8f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T14:37:16.606 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:16 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1413376798' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T14:37:16.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-23T14:37:16.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-23T14:37:16.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T14:37:16.620 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.655+0000 7ff748400700 1 Processor -- start 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.655+0000 7ff748400700 1 -- start start 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.655+0000 7ff748400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 0x7ff740108b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.655+0000 7ff748400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7401090e0 con 0x7ff7401086f0 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.656+0000 7ff745a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 0x7ff740108b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.656+0000 7ff745a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 0x7ff740108b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52238/0 (socket says 172.21.15.83:52238) 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.656+0000 7ff745a00700 1 -- 172.21.15.83:0/1069976311 learned_addr learned my addr 172.21.15.83:0/1069976311 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.656+0000 7ff745a00700 1 -- 172.21.15.83:0/1069976311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7401098a0 con 0x7ff7401086f0 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.656+0000 7ff745a00700 1 --2- 172.21.15.83:0/1069976311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 0x7ff740108b10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff7340098a0 tx=0x7ff734009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25f9afadbf4b63f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.657+0000 7ff73fe00700 1 -- 172.21.15.83:0/1069976311 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff734004160 con 0x7ff7401086f0 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.657+0000 7ff73fe00700 1 -- 172.21.15.83:0/1069976311 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff734004300 con 0x7ff7401086f0 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.657+0000 7ff73fe00700 1 -- 172.21.15.83:0/1069976311 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff734011b50 con 0x7ff7401086f0 2024-04-23T14:37:16.621 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.658+0000 7ff748400700 1 -- 172.21.15.83:0/1069976311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 msgr2=0x7ff740108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.658+0000 7ff748400700 1 --2- 172.21.15.83:0/1069976311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 0x7ff740108b10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff7340098a0 tx=0x7ff734009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.658+0000 7ff748400700 1 -- 172.21.15.83:0/1069976311 shutdown_connections 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.658+0000 7ff748400700 1 --2- 172.21.15.83:0/1069976311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 0x7ff740108b10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.658+0000 7ff748400700 1 -- 172.21.15.83:0/1069976311 >> 172.21.15.83:0/1069976311 conn(0x7ff740103d10 msgr2=0x7ff740106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.658+0000 7ff748400700 1 -- 172.21.15.83:0/1069976311 shutdown_connections 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.658+0000 7ff748400700 1 -- 172.21.15.83:0/1069976311 wait complete. 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.659+0000 7ff748400700 1 Processor -- start 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.659+0000 7ff748400700 1 -- start start 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.659+0000 7ff748400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 0x7ff74019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.659+0000 7ff748400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff74007c100 con 0x7ff7401086f0 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.660+0000 7ff745a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 0x7ff74019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.660+0000 7ff745a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 0x7ff74019e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52252/0 (socket says 172.21.15.83:52252) 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.660+0000 7ff745a00700 1 -- 172.21.15.83:0/1413376798 learned_addr learned my addr 172.21.15.83:0/1413376798 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.660+0000 7ff745a00700 1 -- 172.21.15.83:0/1413376798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74019ee30 con 0x7ff7401086f0 2024-04-23T14:37:16.622 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.660+0000 7ff745a00700 1 --2- 172.21.15.83:0/1413376798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 0x7ff74019e8f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff734011850 tx=0x7ff734011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.661+0000 7ff73e000700 1 -- 172.21.15.83:0/1413376798 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff734003af0 con 0x7ff7401086f0 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.661+0000 7ff73e000700 1 -- 172.21.15.83:0/1413376798 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff734020070 con 0x7ff7401086f0 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.661+0000 7ff748400700 1 -- 172.21.15.83:0/1413376798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74019f0c0 con 0x7ff7401086f0 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.661+0000 7ff73e000700 1 -- 172.21.15.83:0/1413376798 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7340195a0 con 0x7ff7401086f0 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.661+0000 7ff748400700 1 -- 172.21.15.83:0/1413376798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7401a1db0 con 0x7ff7401086f0 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.662+0000 7ff748400700 1 -- 172.21.15.83:0/1413376798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff740064150 con 0x7ff7401086f0 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.662+0000 7ff73e000700 1 -- 172.21.15.83:0/1413376798 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff734019850 con 0x7ff7401086f0 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.663+0000 7ff73e000700 1 --2- 172.21.15.83:0/1413376798 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7ff728039600 0x7ff72803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.663+0000 7ff73e000700 1 -- 172.21.15.83:0/1413376798 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff734011e00 con 0x7ff7401086f0 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.663+0000 7ff745000700 1 -- 172.21.15.83:0/1413376798 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7ff728039600 msgr2=0x7ff72803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1316070089 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.663+0000 7ff745000700 1 --2- 172.21.15.83:0/1413376798 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7ff728039600 0x7ff72803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.668+0000 7ff73e000700 1 -- 172.21.15.83:0/1413376798 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff734003c90 con 0x7ff7401086f0 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.864+0000 7ff745000700 1 -- 172.21.15.83:0/1413376798 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7ff728039600 msgr2=0x7ff72803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1316070089 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.864+0000 7ff745000700 1 --2- 172.21.15.83:0/1413376798 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7ff728039600 0x7ff72803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T14:37:16.623 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.951+0000 7ff748400700 1 -- 172.21.15.83:0/1413376798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff7400008d0 con 0x7ff7401086f0 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.951+0000 7ff73e000700 1 -- 172.21.15.83:0/1413376798 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7ff734023530 con 0x7ff7401086f0 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.954+0000 7ff748400700 1 -- 172.21.15.83:0/1413376798 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7ff728039600 msgr2=0x7ff72803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.954+0000 7ff748400700 1 --2- 172.21.15.83:0/1413376798 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7ff728039600 0x7ff72803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.954+0000 7ff748400700 1 -- 172.21.15.83:0/1413376798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 msgr2=0x7ff74019e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.954+0000 7ff748400700 1 --2- 172.21.15.83:0/1413376798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 0x7ff74019e8f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff734011850 tx=0x7ff734011c10 comp rx=0 tx=0).stop 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.954+0000 7ff748400700 1 -- 172.21.15.83:0/1413376798 shutdown_connections 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.954+0000 7ff748400700 1 --2- 172.21.15.83:0/1413376798 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7ff728039600 0x7ff72803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.954+0000 7ff748400700 1 --2- 172.21.15.83:0/1413376798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7401086f0 0x7ff74019e8f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.954+0000 7ff748400700 1 -- 172.21.15.83:0/1413376798 >> 172.21.15.83:0/1413376798 conn(0x7ff740103d10 msgr2=0x7ff740195bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.954+0000 7ff748400700 1 -- 172.21.15.83:0/1413376798 shutdown_connections 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:15.954+0000 7ff748400700 1 -- 172.21.15.83:0/1413376798 wait complete. 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-04-23T14:37:16.624 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 5... 2024-04-23T14:37:17.133 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:17 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:17.034+0000 7f7e93a8f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T14:37:17.482 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:17 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:17.264+0000 7f7e93a8f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T14:37:17.482 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:17 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:17.480+0000 7f7e93a8f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T14:37:18.250 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:17 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:17.860+0000 7f7e93a8f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T14:37:18.250 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:17 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:17.982+0000 7f7e93a8f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T14:37:18.504 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:18 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:18.246+0000 7f7e93a8f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T14:37:18.504 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:18 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:18.501+0000 7f7e93a8f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:37:19.757 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:19 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:19.270+0000 7f7e93a8f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T14:37:20.202 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:19 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:19.814+0000 7f7e93a8f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T14:37:20.203 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:19 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:19.939+0000 7f7e93a8f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T14:37:20.203 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:20 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:20.201+0000 7f7e93a8f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T14:37:21.256 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:20 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:20.835+0000 7f7e93a8f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T14:37:21.257 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:20 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:20.957+0000 7f7e93a8f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T14:37:21.576 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:21 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:21.274+0000 7f7e93a8f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T14:37:21.577 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:21 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:21.449+0000 7f7e93a8f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T14:37:21.577 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:21 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:21.574+0000 7f7e93a8f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T14:37:22.006 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:21 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:21.722+0000 7f7e93a8f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T14:37:22.506 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:22 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:22.087+0000 7f7e93a8f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T14:37:23.109 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:22 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:22.826+0000 7f7e93a8f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T14:37:23.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:23 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:23.107+0000 7f7e93a8f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T14:37:23.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:23 smithi083 ceph-mon[27867]: Active manager daemon x restarted 2024-04-23T14:37:23.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:23 smithi083 ceph-mon[27867]: Activating manager daemon x 2024-04-23T14:37:23.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:23 smithi083 ceph-mon[27867]: osdmap e2: 0 total, 0 up, 0 in 2024-04-23T14:37:23.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:23 smithi083 ceph-mon[27867]: mgrmap e6: x(active, starting, since 0.00938273s) 2024-04-23T14:37:23.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:23 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:37:23.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:23 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T14:37:23.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:23 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T14:37:23.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:23 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T14:37:23.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:23 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T14:37:23.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:23 smithi083 ceph-mon[27867]: Manager daemon x is now available 2024-04-23T14:37:24.485 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:24 smithi083 ceph-mon[27867]: Found migration_current of "None". Setting to last migration. 2024-04-23T14:37:24.485 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:24 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:24.485 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:24 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:24.485 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:24 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:37:24.485 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:24 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:37:24.485 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:24 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T14:37:24.485 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:24 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T14:37:24.485 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:24 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:24.485 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:24 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:24.485 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:24 smithi083 ceph-mon[27867]: mgrmap e7: x(active, since 1.01635s) 2024-04-23T14:37:24.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-23T14:37:24.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-23T14:37:24.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T14:37:24.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-23T14:37:24.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.276+0000 7fc931400700 1 Processor -- start 2024-04-23T14:37:24.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.276+0000 7fc931400700 1 -- start start 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.277+0000 7fc931400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 0x7fc92c108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.277+0000 7fc931400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c108f10 con 0x7fc92c108520 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.277+0000 7fc92be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 0x7fc92c108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.277+0000 7fc92be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 0x7fc92c108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52266/0 (socket says 172.21.15.83:52266) 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.277+0000 7fc92be00700 1 -- 172.21.15.83:0/3451704198 learned_addr learned my addr 172.21.15.83:0/3451704198 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.278+0000 7fc92be00700 1 -- 172.21.15.83:0/3451704198 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92c1096d0 con 0x7fc92c108520 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.278+0000 7fc92be00700 1 --2- 172.21.15.83:0/3451704198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 0x7fc92c108940 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0098a0 tx=0x7fc91c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8880fb2575f5983 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.278+0000 7fc92aa00700 1 -- 172.21.15.83:0/3451704198 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc91c004160 con 0x7fc92c108520 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.278+0000 7fc92aa00700 1 -- 172.21.15.83:0/3451704198 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc91c004300 con 0x7fc92c108520 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.278+0000 7fc92aa00700 1 -- 172.21.15.83:0/3451704198 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc91c011b50 con 0x7fc92c108520 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.279+0000 7fc931400700 1 -- 172.21.15.83:0/3451704198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 msgr2=0x7fc92c108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.279+0000 7fc931400700 1 --2- 172.21.15.83:0/3451704198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 0x7fc92c108940 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0098a0 tx=0x7fc91c009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.279+0000 7fc931400700 1 -- 172.21.15.83:0/3451704198 shutdown_connections 2024-04-23T14:37:24.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.279+0000 7fc931400700 1 --2- 172.21.15.83:0/3451704198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 0x7fc92c108940 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.279+0000 7fc931400700 1 -- 172.21.15.83:0/3451704198 >> 172.21.15.83:0/3451704198 conn(0x7fc92c103ac0 msgr2=0x7fc92c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.279+0000 7fc931400700 1 -- 172.21.15.83:0/3451704198 shutdown_connections 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.280+0000 7fc931400700 1 -- 172.21.15.83:0/3451704198 wait complete. 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.281+0000 7fc931400700 1 Processor -- start 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.281+0000 7fc931400700 1 -- start start 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.282+0000 7fc931400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 0x7fc92c07db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.282+0000 7fc931400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc92c10ca80 con 0x7fc92c108520 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.282+0000 7fc92be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 0x7fc92c07db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.282+0000 7fc92be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 0x7fc92c07db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52268/0 (socket says 172.21.15.83:52268) 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.282+0000 7fc92be00700 1 -- 172.21.15.83:0/2877800367 learned_addr learned my addr 172.21.15.83:0/2877800367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.282+0000 7fc92be00700 1 -- 172.21.15.83:0/2877800367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92c07e0d0 con 0x7fc92c108520 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.283+0000 7fc92be00700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 0x7fc92c07db90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc91c000c00 tx=0x7fc91c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.283+0000 7fc928c00700 1 -- 172.21.15.83:0/2877800367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc91c003b20 con 0x7fc92c108520 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.283+0000 7fc928c00700 1 -- 172.21.15.83:0/2877800367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc91c003cc0 con 0x7fc92c108520 2024-04-23T14:37:24.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.283+0000 7fc931400700 1 -- 172.21.15.83:0/2877800367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc92c07c2c0 con 0x7fc92c108520 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.283+0000 7fc928c00700 1 -- 172.21.15.83:0/2877800367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc91c024910 con 0x7fc92c108520 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.283+0000 7fc931400700 1 -- 172.21.15.83:0/2877800367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92c07c800 con 0x7fc92c108520 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.284+0000 7fc928c00700 1 -- 172.21.15.83:0/2877800367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fc91c022020 con 0x7fc92c108520 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.285+0000 7fc928c00700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 0x7fc91003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.285+0000 7fc928c00700 1 -- 172.21.15.83:0/2877800367 --> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc91003c1d0 con 0x7fc910039600 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.285+0000 7fc928c00700 1 -- 172.21.15.83:0/2877800367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc91c011e00 con 0x7fc92c108520 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.285+0000 7fc92b400700 1 -- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 msgr2=0x7fc91003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1316070089 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.285+0000 7fc92b400700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 0x7fc91003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.486+0000 7fc92b400700 1 -- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 msgr2=0x7fc91003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1316070089 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.486+0000 7fc92b400700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 0x7fc91003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.887+0000 7fc92b400700 1 -- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 msgr2=0x7fc91003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1316070089 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:17.887+0000 7fc92b400700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 0x7fc91003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T14:37:24.822 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:18.688+0000 7fc92b400700 1 -- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 msgr2=0x7fc91003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1316070089 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:18.688+0000 7fc92b400700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 0x7fc91003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:20.290+0000 7fc92b400700 1 -- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 msgr2=0x7fc91003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1316070089 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:20.290+0000 7fc92b400700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 0x7fc91003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:23.117+0000 7fc928c00700 1 -- 172.21.15.83:0/2877800367 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fc91c01a430 con 0x7fc92c108520 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:23.117+0000 7fc928c00700 1 -- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 msgr2=0x7fc91003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:23.117+0000 7fc928c00700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 0x7fc91003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.120+0000 7fc928c00700 1 -- 172.21.15.83:0/2877800367 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fc91c017ca0 con 0x7fc92c108520 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.121+0000 7fc928c00700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fc91003cfc0 0x7fc91003f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.121+0000 7fc928c00700 1 -- 172.21.15.83:0/2877800367 --> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc91c001d30 con 0x7fc91003cfc0 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.121+0000 7fc92b400700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fc91003cfc0 0x7fc91003f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.123+0000 7fc92b400700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fc91003cfc0 0x7fc91003f3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc918003a10 tx=0x7fc9180092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.125+0000 7fc928c00700 1 -- 172.21.15.83:0/2877800367 <== mgr.14120 v2:172.21.15.83:6800/3974620988 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fc91c001d30 con 0x7fc91003cfc0 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.135+0000 7fc931400700 1 -- 172.21.15.83:0/2877800367 --> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fc92c064150 con 0x7fc91003cfc0 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.135+0000 7fc928c00700 1 -- 172.21.15.83:0/2877800367 <== mgr.14120 v2:172.21.15.83:6800/3974620988 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fc92c064150 con 0x7fc91003cfc0 2024-04-23T14:37:24.823 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.136+0000 7fc931400700 1 -- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fc91003cfc0 msgr2=0x7fc91003f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.136+0000 7fc931400700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fc91003cfc0 0x7fc91003f3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc918003a10 tx=0x7fc9180092b0 comp rx=0 tx=0).stop 2024-04-23T14:37:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.136+0000 7fc931400700 1 -- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 msgr2=0x7fc92c07db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.136+0000 7fc931400700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 0x7fc92c07db90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc91c000c00 tx=0x7fc91c011c10 comp rx=0 tx=0).stop 2024-04-23T14:37:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.136+0000 7fc931400700 1 -- 172.21.15.83:0/2877800367 shutdown_connections 2024-04-23T14:37:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.136+0000 7fc931400700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fc91003cfc0 0x7fc91003f3b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.136+0000 7fc931400700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc92c108520 0x7fc92c07db90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.136+0000 7fc931400700 1 --2- 172.21.15.83:0/2877800367 >> [v2:172.21.15.83:6800/1316070089,v1:172.21.15.83:6801/1316070089] conn(0x7fc910039600 0x7fc91003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.136+0000 7fc931400700 1 -- 172.21.15.83:0/2877800367 >> 172.21.15.83:0/2877800367 conn(0x7fc92c103ac0 msgr2=0x7fc92c104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.136+0000 7fc931400700 1 -- 172.21.15.83:0/2877800367 shutdown_connections 2024-04-23T14:37:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:24.136+0000 7fc931400700 1 -- 172.21.15.83:0/2877800367 wait complete. 2024-04-23T14:37:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 5 is available 2024-04-23T14:37:24.824 INFO:teuthology.orchestra.run.smithi083.stdout:Setting orchestrator backend to cephadm... 2024-04-23T14:37:25.427 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:25 smithi083 ceph-mon[27867]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T14:37:25.427 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:25 smithi083 ceph-mon[27867]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T14:37:25.428 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:25 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:37:26.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:26 smithi083 ceph-mon[27867]: [23/Apr/2024:14:37:24] ENGINE Bus STARTING 2024-04-23T14:37:26.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:26 smithi083 ceph-mon[27867]: [23/Apr/2024:14:37:24] ENGINE Serving on https://172.21.15.83:7150 2024-04-23T14:37:26.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:26 smithi083 ceph-mon[27867]: [23/Apr/2024:14:37:24] ENGINE Serving on http://172.21.15.83:8765 2024-04-23T14:37:26.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:26 smithi083 ceph-mon[27867]: [23/Apr/2024:14:37:24] ENGINE Bus STARTED 2024-04-23T14:37:26.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:26 smithi083 ceph-mon[27867]: mgrmap e8: x(active, since 2s) 2024-04-23T14:37:26.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:26 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:26.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:26 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:37:26.295 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.425+0000 7f482b000700 1 Processor -- start 2024-04-23T14:37:26.295 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.426+0000 7f482b000700 1 -- start start 2024-04-23T14:37:26.296 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.426+0000 7f482b000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 0x7f482410f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:26.296 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.426+0000 7f482b000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f482410f990 con 0x7f482410f030 2024-04-23T14:37:26.296 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.426+0000 7f4823e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 0x7f482410f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:26.296 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.426+0000 7f4823e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 0x7f482410f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40624/0 (socket says 172.21.15.83:40624) 2024-04-23T14:37:26.296 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.426+0000 7f4823e00700 1 -- 172.21.15.83:0/3087176716 learned_addr learned my addr 172.21.15.83:0/3087176716 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:26.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.427+0000 7f4823e00700 1 -- 172.21.15.83:0/3087176716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482410fb10 con 0x7f482410f030 2024-04-23T14:37:26.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.427+0000 7f4823e00700 1 --2- 172.21.15.83:0/3087176716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 0x7f482410f450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f48140098a0 tx=0x7f4814009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2ff9bd68aa86d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:26.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.427+0000 7f4823400700 1 -- 172.21.15.83:0/3087176716 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4814004160 con 0x7f482410f030 2024-04-23T14:37:26.297 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.428+0000 7f4823400700 1 -- 172.21.15.83:0/3087176716 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4814004300 con 0x7f482410f030 2024-04-23T14:37:26.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.428+0000 7f4823400700 1 -- 172.21.15.83:0/3087176716 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4814011b50 con 0x7f482410f030 2024-04-23T14:37:26.298 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.428+0000 7f482b000700 1 -- 172.21.15.83:0/3087176716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 msgr2=0x7f482410f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:26.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.428+0000 7f482b000700 1 --2- 172.21.15.83:0/3087176716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 0x7f482410f450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f48140098a0 tx=0x7f4814009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:26.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.428+0000 7f482b000700 1 -- 172.21.15.83:0/3087176716 shutdown_connections 2024-04-23T14:37:26.299 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.428+0000 7f482b000700 1 --2- 172.21.15.83:0/3087176716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 0x7f482410f450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:26.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.428+0000 7f482b000700 1 -- 172.21.15.83:0/3087176716 >> 172.21.15.83:0/3087176716 conn(0x7f482407def0 msgr2=0x7f4824080310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:26.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.428+0000 7f482b000700 1 -- 172.21.15.83:0/3087176716 shutdown_connections 2024-04-23T14:37:26.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.428+0000 7f482b000700 1 -- 172.21.15.83:0/3087176716 wait complete. 2024-04-23T14:37:26.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.429+0000 7f482b000700 1 Processor -- start 2024-04-23T14:37:26.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.430+0000 7f482b000700 1 -- start start 2024-04-23T14:37:26.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.430+0000 7f482b000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 0x7f48241a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:26.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.430+0000 7f482b000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4824110d30 con 0x7f482410f030 2024-04-23T14:37:26.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.430+0000 7f4823e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 0x7f48241a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:26.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.430+0000 7f4823e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 0x7f48241a2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40626/0 (socket says 172.21.15.83:40626) 2024-04-23T14:37:26.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.430+0000 7f4823e00700 1 -- 172.21.15.83:0/3982564360 learned_addr learned my addr 172.21.15.83:0/3982564360 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:26.300 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.430+0000 7f4823e00700 1 -- 172.21.15.83:0/3982564360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48241a3170 con 0x7f482410f030 2024-04-23T14:37:26.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.431+0000 7f4823e00700 1 --2- 172.21.15.83:0/3982564360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 0x7f48241a2c30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4814011b60 tx=0x7f4814011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:26.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.431+0000 7f4821600700 1 -- 172.21.15.83:0/3982564360 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4814003b20 con 0x7f482410f030 2024-04-23T14:37:26.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.431+0000 7f4821600700 1 -- 172.21.15.83:0/3982564360 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4814003cc0 con 0x7f482410f030 2024-04-23T14:37:26.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.431+0000 7f4821600700 1 -- 172.21.15.83:0/3982564360 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4814024840 con 0x7f482410f030 2024-04-23T14:37:26.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.431+0000 7f482b000700 1 -- 172.21.15.83:0/3982564360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48241a3400 con 0x7f482410f030 2024-04-23T14:37:26.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.431+0000 7f482b000700 1 -- 172.21.15.83:0/3982564360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48241a60f0 con 0x7f482410f030 2024-04-23T14:37:26.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.432+0000 7f4821600700 1 -- 172.21.15.83:0/3982564360 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4814022020 con 0x7f482410f030 2024-04-23T14:37:26.303 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.432+0000 7f482b000700 1 -- 172.21.15.83:0/3982564360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4824064150 con 0x7f482410f030 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.433+0000 7f4821600700 1 --2- 172.21.15.83:0/3982564360 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f48080395b0 0x7f480803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.433+0000 7f4821600700 1 -- 172.21.15.83:0/3982564360 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4814011e00 con 0x7f482410f030 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.433+0000 7f481be00700 1 --2- 172.21.15.83:0/3982564360 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f48080395b0 0x7f480803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.434+0000 7f481be00700 1 --2- 172.21.15.83:0/3982564360 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f48080395b0 0x7f480803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f480c006fd0 tx=0x7f480c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.439+0000 7f4821600700 1 -- 172.21.15.83:0/3982564360 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f481404d240 con 0x7f482410f030 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.680+0000 7f482b000700 1 -- 172.21.15.83:0/3982564360 --> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f482407f0d0 con 0x7f48080395b0 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.691+0000 7f4821600700 1 -- 172.21.15.83:0/3982564360 <== mgr.14120 v2:172.21.15.83:6800/3974620988 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f482407f0d0 con 0x7f48080395b0 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.694+0000 7f482b000700 1 -- 172.21.15.83:0/3982564360 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f48080395b0 msgr2=0x7f480803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.694+0000 7f482b000700 1 --2- 172.21.15.83:0/3982564360 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f48080395b0 0x7f480803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f480c006fd0 tx=0x7f480c006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.694+0000 7f482b000700 1 -- 172.21.15.83:0/3982564360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 msgr2=0x7f48241a2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.694+0000 7f482b000700 1 --2- 172.21.15.83:0/3982564360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 0x7f48241a2c30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4814011b60 tx=0x7f4814011c40 comp rx=0 tx=0).stop 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.694+0000 7f482b000700 1 -- 172.21.15.83:0/3982564360 shutdown_connections 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.694+0000 7f482b000700 1 --2- 172.21.15.83:0/3982564360 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f48080395b0 0x7f480803ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.694+0000 7f482b000700 1 --2- 172.21.15.83:0/3982564360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f482410f030 0x7f48241a2c30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:26.304 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.694+0000 7f482b000700 1 -- 172.21.15.83:0/3982564360 >> 172.21.15.83:0/3982564360 conn(0x7f482407def0 msgr2=0x7f482407e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:26.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.695+0000 7f482b000700 1 -- 172.21.15.83:0/3982564360 shutdown_connections 2024-04-23T14:37:26.305 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:25.695+0000 7f482b000700 1 -- 172.21.15.83:0/3982564360 wait complete. 2024-04-23T14:37:27.208 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:27 smithi083 ceph-mon[27867]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:37:27.939 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.942+0000 7f63a8400700 1 Processor -- start 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.942+0000 7f63a8400700 1 -- start start 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.943+0000 7f63a8400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 0x7f63a0108b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.943+0000 7f63a8400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63a01090e0 con 0x7f63a01086f0 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.943+0000 7f63a5a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 0x7f63a0108b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.943+0000 7f63a5a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 0x7f63a0108b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40628/0 (socket says 172.21.15.83:40628) 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.943+0000 7f63a5a00700 1 -- 172.21.15.83:0/1133283757 learned_addr learned my addr 172.21.15.83:0/1133283757 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.944+0000 7f63a5a00700 1 -- 172.21.15.83:0/1133283757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a01098a0 con 0x7f63a01086f0 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.944+0000 7f63a5a00700 1 --2- 172.21.15.83:0/1133283757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 0x7f63a0108b10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63940098a0 tx=0x7f6394009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198a4a49a5fdb33e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.944+0000 7f639fe00700 1 -- 172.21.15.83:0/1133283757 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6394004160 con 0x7f63a01086f0 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.945+0000 7f639fe00700 1 -- 172.21.15.83:0/1133283757 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6394004300 con 0x7f63a01086f0 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.945+0000 7f639fe00700 1 -- 172.21.15.83:0/1133283757 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6394011b50 con 0x7f63a01086f0 2024-04-23T14:37:27.940 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.945+0000 7f63a8400700 1 -- 172.21.15.83:0/1133283757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 msgr2=0x7f63a0108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.945+0000 7f63a8400700 1 --2- 172.21.15.83:0/1133283757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 0x7f63a0108b10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63940098a0 tx=0x7f6394009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.945+0000 7f63a8400700 1 -- 172.21.15.83:0/1133283757 shutdown_connections 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.945+0000 7f63a8400700 1 --2- 172.21.15.83:0/1133283757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 0x7f63a0108b10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.945+0000 7f63a8400700 1 -- 172.21.15.83:0/1133283757 >> 172.21.15.83:0/1133283757 conn(0x7f63a0103d10 msgr2=0x7f63a0106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.945+0000 7f63a8400700 1 -- 172.21.15.83:0/1133283757 shutdown_connections 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.946+0000 7f63a8400700 1 -- 172.21.15.83:0/1133283757 wait complete. 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.946+0000 7f63a8400700 1 Processor -- start 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.947+0000 7f63a8400700 1 -- start start 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.947+0000 7f63a8400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 0x7f63a019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.947+0000 7f63a8400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63a007c100 con 0x7f63a01086f0 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.947+0000 7f63a5a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 0x7f63a019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.947+0000 7f63a5a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 0x7f63a019e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40636/0 (socket says 172.21.15.83:40636) 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.947+0000 7f63a5a00700 1 -- 172.21.15.83:0/2222053302 learned_addr learned my addr 172.21.15.83:0/2222053302 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.948+0000 7f63a5a00700 1 -- 172.21.15.83:0/2222053302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a019ee50 con 0x7f63a01086f0 2024-04-23T14:37:27.941 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.948+0000 7f63a5a00700 1 --2- 172.21.15.83:0/2222053302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 0x7f63a019e910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6394011850 tx=0x7f6394011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:27.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.948+0000 7f639e000700 1 -- 172.21.15.83:0/2222053302 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6394003af0 con 0x7f63a01086f0 2024-04-23T14:37:27.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.948+0000 7f63a8400700 1 -- 172.21.15.83:0/2222053302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a019f0e0 con 0x7f63a01086f0 2024-04-23T14:37:27.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.948+0000 7f63a8400700 1 -- 172.21.15.83:0/2222053302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a01a1dd0 con 0x7f63a01086f0 2024-04-23T14:37:27.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.948+0000 7f639e000700 1 -- 172.21.15.83:0/2222053302 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6394020070 con 0x7f63a01086f0 2024-04-23T14:37:27.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.948+0000 7f639e000700 1 -- 172.21.15.83:0/2222053302 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63940193c0 con 0x7f63a01086f0 2024-04-23T14:37:27.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.949+0000 7f63a8400700 1 -- 172.21.15.83:0/2222053302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6380005330 con 0x7f63a01086f0 2024-04-23T14:37:27.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.950+0000 7f639e000700 1 -- 172.21.15.83:0/2222053302 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f639401a430 con 0x7f63a01086f0 2024-04-23T14:37:27.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.950+0000 7f639e000700 1 --2- 172.21.15.83:0/2222053302 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f6388039600 0x7f638803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:27.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.950+0000 7f639e000700 1 -- 172.21.15.83:0/2222053302 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6394011e00 con 0x7f63a01086f0 2024-04-23T14:37:27.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.951+0000 7f63a5000700 1 --2- 172.21.15.83:0/2222053302 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f6388039600 0x7f638803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:27.942 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.951+0000 7f63a5000700 1 --2- 172.21.15.83:0/2222053302 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f6388039600 0x7f638803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f638c006fd0 tx=0x7f638c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:27.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:26.957+0000 7f639e000700 1 -- 172.21.15.83:0/2222053302 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f639400e260 con 0x7f63a01086f0 2024-04-23T14:37:27.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:27.184+0000 7f63a8400700 1 -- 172.21.15.83:0/2222053302 --> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f6380000bc0 con 0x7f6388039600 2024-04-23T14:37:27.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:27.186+0000 7f639e000700 1 -- 172.21.15.83:0/2222053302 <== mgr.14120 v2:172.21.15.83:6800/3974620988 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f6380000bc0 con 0x7f6388039600 2024-04-23T14:37:27.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:27.189+0000 7f63a8400700 1 -- 172.21.15.83:0/2222053302 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f6388039600 msgr2=0x7f638803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:27.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:27.189+0000 7f63a8400700 1 --2- 172.21.15.83:0/2222053302 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f6388039600 0x7f638803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f638c006fd0 tx=0x7f638c006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:27.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:27.189+0000 7f63a8400700 1 -- 172.21.15.83:0/2222053302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 msgr2=0x7f63a019e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:27.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:27.189+0000 7f63a8400700 1 --2- 172.21.15.83:0/2222053302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 0x7f63a019e910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6394011850 tx=0x7f6394011c10 comp rx=0 tx=0).stop 2024-04-23T14:37:27.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:27.189+0000 7f63a8400700 1 -- 172.21.15.83:0/2222053302 shutdown_connections 2024-04-23T14:37:27.944 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:27.189+0000 7f63a8400700 1 --2- 172.21.15.83:0/2222053302 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f6388039600 0x7f638803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:27.945 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:27.189+0000 7f63a8400700 1 --2- 172.21.15.83:0/2222053302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63a01086f0 0x7f63a019e910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:27.945 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:27.189+0000 7f63a8400700 1 -- 172.21.15.83:0/2222053302 >> 172.21.15.83:0/2222053302 conn(0x7f63a0103d10 msgr2=0x7f63a0195bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:27.945 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:27.190+0000 7f63a8400700 1 -- 172.21.15.83:0/2222053302 shutdown_connections 2024-04-23T14:37:27.945 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:27.190+0000 7f63a8400700 1 -- 172.21.15.83:0/2222053302 wait complete. 2024-04-23T14:37:27.945 INFO:teuthology.orchestra.run.smithi083.stdout:Generating ssh key... 2024-04-23T14:37:28.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-mon[27867]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:37:29.237 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: Generating public/private rsa key pair. 2024-04-23T14:37:29.237 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: Your identification has been saved in /tmp/tmpyn8ne5tx/key. 2024-04-23T14:37:29.237 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: Your public key has been saved in /tmp/tmpyn8ne5tx/key.pub. 2024-04-23T14:37:29.237 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: The key fingerprint is: 2024-04-23T14:37:29.237 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: SHA256:fwHzDzVqh+qDZDXTfmMSjmGlj90OyGj36WG8mY+KE24 ceph-5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:37:29.237 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: The key's randomart image is: 2024-04-23T14:37:29.237 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: +---[RSA 3072]----+ 2024-04-23T14:37:29.238 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: | | 2024-04-23T14:37:29.238 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: | . | 2024-04-23T14:37:29.238 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: | o+ o | 2024-04-23T14:37:29.238 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: | B+o+ . | 2024-04-23T14:37:29.238 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: | S= %Bo. | 2024-04-23T14:37:29.238 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: | *.*+B== | 2024-04-23T14:37:29.238 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: | = +o.=B.. | 2024-04-23T14:37:29.238 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: | Eo.oo*. | 2024-04-23T14:37:29.238 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: | ...o+*.. | 2024-04-23T14:37:29.238 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: +----[SHA256]-----+ 2024-04-23T14:37:29.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.584+0000 7f1c3a400700 1 Processor -- start 2024-04-23T14:37:29.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.584+0000 7f1c3a400700 1 -- start start 2024-04-23T14:37:29.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.585+0000 7f1c3a400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 0x7f1c3410ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:29.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.585+0000 7f1c3a400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3410b120 con 0x7f1c3410a730 2024-04-23T14:37:29.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.585+0000 7f1c39000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 0x7f1c3410ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:29.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.585+0000 7f1c39000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 0x7f1c3410ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40652/0 (socket says 172.21.15.83:40652) 2024-04-23T14:37:29.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.585+0000 7f1c39000700 1 -- 172.21.15.83:0/4020318317 learned_addr learned my addr 172.21.15.83:0/4020318317 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:29.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.586+0000 7f1c39000700 1 -- 172.21.15.83:0/4020318317 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3410b8e0 con 0x7f1c3410a730 2024-04-23T14:37:29.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.586+0000 7f1c39000700 1 --2- 172.21.15.83:0/4020318317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 0x7f1c3410ab50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1c280098a0 tx=0x7f1c28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71609de4b6bb4e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:29.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.586+0000 7f1c33400700 1 -- 172.21.15.83:0/4020318317 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c28004160 con 0x7f1c3410a730 2024-04-23T14:37:29.565 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.586+0000 7f1c33400700 1 -- 172.21.15.83:0/4020318317 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c28004300 con 0x7f1c3410a730 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.586+0000 7f1c33400700 1 -- 172.21.15.83:0/4020318317 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c28011b50 con 0x7f1c3410a730 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.587+0000 7f1c3a400700 1 -- 172.21.15.83:0/4020318317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 msgr2=0x7f1c3410ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.587+0000 7f1c3a400700 1 --2- 172.21.15.83:0/4020318317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 0x7f1c3410ab50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1c280098a0 tx=0x7f1c28009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.587+0000 7f1c3a400700 1 -- 172.21.15.83:0/4020318317 shutdown_connections 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.587+0000 7f1c3a400700 1 --2- 172.21.15.83:0/4020318317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 0x7f1c3410ab50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.587+0000 7f1c3a400700 1 -- 172.21.15.83:0/4020318317 >> 172.21.15.83:0/4020318317 conn(0x7f1c3407ddf0 msgr2=0x7f1c341084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.587+0000 7f1c3a400700 1 -- 172.21.15.83:0/4020318317 shutdown_connections 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.587+0000 7f1c3a400700 1 -- 172.21.15.83:0/4020318317 wait complete. 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.588+0000 7f1c3a400700 1 Processor -- start 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.589+0000 7f1c3a400700 1 -- start start 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.589+0000 7f1c3a400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 0x7f1c3419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.589+0000 7f1c3a400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3410ec90 con 0x7f1c3410a730 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.589+0000 7f1c39000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 0x7f1c3419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:29.566 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.589+0000 7f1c39000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 0x7f1c3419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40664/0 (socket says 172.21.15.83:40664) 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.589+0000 7f1c39000700 1 -- 172.21.15.83:0/2525687451 learned_addr learned my addr 172.21.15.83:0/2525687451 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.590+0000 7f1c39000700 1 -- 172.21.15.83:0/2525687451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3419ec40 con 0x7f1c3410a730 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.590+0000 7f1c39000700 1 --2- 172.21.15.83:0/2525687451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 0x7f1c3419e700 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1c28011850 tx=0x7f1c28011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.590+0000 7f1c31600700 1 -- 172.21.15.83:0/2525687451 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c28003af0 con 0x7f1c3410a730 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.590+0000 7f1c31600700 1 -- 172.21.15.83:0/2525687451 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c28020070 con 0x7f1c3410a730 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.590+0000 7f1c31600700 1 -- 172.21.15.83:0/2525687451 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c280195a0 con 0x7f1c3410a730 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.590+0000 7f1c3a400700 1 -- 172.21.15.83:0/2525687451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3419eed0 con 0x7f1c3410a730 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.590+0000 7f1c3a400700 1 -- 172.21.15.83:0/2525687451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c341a1bc0 con 0x7f1c3410a730 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.592+0000 7f1c3a400700 1 -- 172.21.15.83:0/2525687451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c34064150 con 0x7f1c3410a730 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.592+0000 7f1c31600700 1 -- 172.21.15.83:0/2525687451 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1c280198a0 con 0x7f1c3410a730 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.592+0000 7f1c31600700 1 --2- 172.21.15.83:0/2525687451 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f1c1c0395b0 0x7f1c1c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.592+0000 7f1c31600700 1 -- 172.21.15.83:0/2525687451 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1c28011e00 con 0x7f1c3410a730 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.592+0000 7f1c33e00700 1 --2- 172.21.15.83:0/2525687451 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f1c1c0395b0 0x7f1c1c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.593+0000 7f1c33e00700 1 --2- 172.21.15.83:0/2525687451 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f1c1c0395b0 0x7f1c1c03ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c20006fd0 tx=0x7f1c20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:29.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.598+0000 7f1c31600700 1 -- 172.21.15.83:0/2525687451 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c2800e260 con 0x7f1c3410a730 2024-04-23T14:37:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.825+0000 7f1c3a400700 1 -- 172.21.15.83:0/2525687451 --> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f1c34108410 con 0x7f1c1c0395b0 2024-04-23T14:37:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.919+0000 7f1c31600700 1 -- 172.21.15.83:0/2525687451 <== mgr.14120 v2:172.21.15.83:6800/3974620988 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1c34108410 con 0x7f1c1c0395b0 2024-04-23T14:37:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.922+0000 7f1c3a400700 1 -- 172.21.15.83:0/2525687451 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f1c1c0395b0 msgr2=0x7f1c1c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.922+0000 7f1c3a400700 1 --2- 172.21.15.83:0/2525687451 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f1c1c0395b0 0x7f1c1c03ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c20006fd0 tx=0x7f1c20006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.922+0000 7f1c3a400700 1 -- 172.21.15.83:0/2525687451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 msgr2=0x7f1c3419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.922+0000 7f1c3a400700 1 --2- 172.21.15.83:0/2525687451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 0x7f1c3419e700 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1c28011850 tx=0x7f1c28011c10 comp rx=0 tx=0).stop 2024-04-23T14:37:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.922+0000 7f1c3a400700 1 -- 172.21.15.83:0/2525687451 shutdown_connections 2024-04-23T14:37:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.922+0000 7f1c3a400700 1 --2- 172.21.15.83:0/2525687451 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f1c1c0395b0 0x7f1c1c03ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.922+0000 7f1c3a400700 1 --2- 172.21.15.83:0/2525687451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c3410a730 0x7f1c3419e700 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.922+0000 7f1c3a400700 1 -- 172.21.15.83:0/2525687451 >> 172.21.15.83:0/2525687451 conn(0x7f1c3407ddf0 msgr2=0x7f1c34107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.923+0000 7f1c3a400700 1 -- 172.21.15.83:0/2525687451 shutdown_connections 2024-04-23T14:37:29.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:28.923+0000 7f1c3a400700 1 -- 172.21.15.83:0/2525687451 wait complete. 2024-04-23T14:37:30.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:29 smithi083 ceph-mon[27867]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:37:30.127 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:29 smithi083 ceph-mon[27867]: Generating ssh key... 2024-04-23T14:37:30.127 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:29 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:30.127 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:29 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:31.167 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJGG75I/Qdtsgz+a1mxtiBgJ0YdzfyhCpNowEP92NvPT3oCpKRWmLlreR/0csmAcAlAdUgrsGpYgT3BLs87dmifRI1fxyO1i5KHZP94L46BjSNDg0dcxyaM8yOaJf6BYZfoICSTLAvc2XjGnYfsGwUClahL1VSX4wKDW92l+Lohm26NtaY6yGbO7gvKhtiR95LxiQEbRe7+J2mAvYc67t8uLPuZBuwoSnoFtv+bKAk29bDkzDtgjZ37KXwaTNG0XbACg3Hq0tlQjU0gl1ZJcsseOmT4N0t8/0PT1IHGf0HrDO+dXb2+lodItpntCLF9aPlYvBvOw8WbNPwJQjy6yi8jKfNxZKIdLmk0aH8xa09FpXZpLbYwGvAXX3a36tC58nO+QTqxp1091Ym5j7fw8hdWhLv4ludT3UxTMaZ67vaeH3kPr9KbXslRvdGZI3qZ3WAf6bXIZHEHjab7IrAwppa3vIyFOEAf3zOZ6+4btDQY0sx8TTH2zeNO2BToneKOeM= ceph-5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:37:31.167 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.209+0000 7f4d4e800700 1 Processor -- start 2024-04-23T14:37:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.209+0000 7f4d4e800700 1 -- start start 2024-04-23T14:37:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.210+0000 7f4d4e800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 0x7f4d48108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:31.168 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.210+0000 7f4d4e800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48108f10 con 0x7f4d48108520 2024-04-23T14:37:31.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.210+0000 7f4d4d400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 0x7f4d48108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:31.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.210+0000 7f4d4d400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 0x7f4d48108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40678/0 (socket says 172.21.15.83:40678) 2024-04-23T14:37:31.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.210+0000 7f4d4d400700 1 -- 172.21.15.83:0/4207794027 learned_addr learned my addr 172.21.15.83:0/4207794027 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:31.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.211+0000 7f4d4d400700 1 -- 172.21.15.83:0/4207794027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d481096d0 con 0x7f4d48108520 2024-04-23T14:37:31.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.211+0000 7f4d4d400700 1 --2- 172.21.15.83:0/4207794027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 0x7f4d48108940 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0098a0 tx=0x7f4d3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119f1d0984171b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:31.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.211+0000 7f4d47e00700 1 -- 172.21.15.83:0/4207794027 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3c004160 con 0x7f4d48108520 2024-04-23T14:37:31.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.211+0000 7f4d47e00700 1 -- 172.21.15.83:0/4207794027 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4d3c004300 con 0x7f4d48108520 2024-04-23T14:37:31.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.211+0000 7f4d47e00700 1 -- 172.21.15.83:0/4207794027 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3c011b50 con 0x7f4d48108520 2024-04-23T14:37:31.172 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.212+0000 7f4d4e800700 1 -- 172.21.15.83:0/4207794027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 msgr2=0x7f4d48108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.212+0000 7f4d4e800700 1 --2- 172.21.15.83:0/4207794027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 0x7f4d48108940 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0098a0 tx=0x7f4d3c009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.212+0000 7f4d4e800700 1 -- 172.21.15.83:0/4207794027 shutdown_connections 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.212+0000 7f4d4e800700 1 --2- 172.21.15.83:0/4207794027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 0x7f4d48108940 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.212+0000 7f4d4e800700 1 -- 172.21.15.83:0/4207794027 >> 172.21.15.83:0/4207794027 conn(0x7f4d48103ac0 msgr2=0x7f4d48105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.212+0000 7f4d4e800700 1 -- 172.21.15.83:0/4207794027 shutdown_connections 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.212+0000 7f4d4e800700 1 -- 172.21.15.83:0/4207794027 wait complete. 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.213+0000 7f4d4e800700 1 Processor -- start 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.213+0000 7f4d4e800700 1 -- start start 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.214+0000 7f4d4e800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 0x7f4d4819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.214+0000 7f4d4e800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4810ca80 con 0x7f4d48108520 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.214+0000 7f4d4d400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 0x7f4d4819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.214+0000 7f4d4d400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 0x7f4d4819e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40688/0 (socket says 172.21.15.83:40688) 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.214+0000 7f4d4d400700 1 -- 172.21.15.83:0/1803759115 learned_addr learned my addr 172.21.15.83:0/1803759115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:31.173 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.214+0000 7f4d4d400700 1 -- 172.21.15.83:0/1803759115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4819ebd0 con 0x7f4d48108520 2024-04-23T14:37:31.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.215+0000 7f4d4d400700 1 --2- 172.21.15.83:0/1803759115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 0x7f4d4819e690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c000c00 tx=0x7f4d3c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:31.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.215+0000 7f4d46000700 1 -- 172.21.15.83:0/1803759115 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3c003b20 con 0x7f4d48108520 2024-04-23T14:37:31.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.215+0000 7f4d46000700 1 -- 172.21.15.83:0/1803759115 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4d3c003cc0 con 0x7f4d48108520 2024-04-23T14:37:31.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.215+0000 7f4d46000700 1 -- 172.21.15.83:0/1803759115 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3c024910 con 0x7f4d48108520 2024-04-23T14:37:31.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.215+0000 7f4d4e800700 1 -- 172.21.15.83:0/1803759115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4819ee60 con 0x7f4d48108520 2024-04-23T14:37:31.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.215+0000 7f4d4e800700 1 -- 172.21.15.83:0/1803759115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d481a1b50 con 0x7f4d48108520 2024-04-23T14:37:31.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.216+0000 7f4d4e800700 1 -- 172.21.15.83:0/1803759115 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d48064150 con 0x7f4d48108520 2024-04-23T14:37:31.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.216+0000 7f4d46000700 1 -- 172.21.15.83:0/1803759115 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4d3c022020 con 0x7f4d48108520 2024-04-23T14:37:31.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.217+0000 7f4d46000700 1 --2- 172.21.15.83:0/1803759115 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f4d30039600 0x7f4d3003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:31.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.217+0000 7f4d46000700 1 -- 172.21.15.83:0/1803759115 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4d3c011e00 con 0x7f4d48108520 2024-04-23T14:37:31.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.217+0000 7f4d4ca00700 1 --2- 172.21.15.83:0/1803759115 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f4d30039600 0x7f4d3003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:31.174 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.218+0000 7f4d4ca00700 1 --2- 172.21.15.83:0/1803759115 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f4d30039600 0x7f4d3003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4d34006fd0 tx=0x7f4d34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:31.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.222+0000 7f4d46000700 1 -- 172.21.15.83:0/1803759115 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d3c01ad80 con 0x7f4d48108520 2024-04-23T14:37:31.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.444+0000 7f4d4e800700 1 -- 172.21.15.83:0/1803759115 --> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f4d48104c90 con 0x7f4d30039600 2024-04-23T14:37:31.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.446+0000 7f4d46000700 1 -- 172.21.15.83:0/1803759115 <== mgr.14120 v2:172.21.15.83:6800/3974620988 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f4d48104c90 con 0x7f4d30039600 2024-04-23T14:37:31.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.449+0000 7f4d4e800700 1 -- 172.21.15.83:0/1803759115 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f4d30039600 msgr2=0x7f4d3003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:31.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.449+0000 7f4d4e800700 1 --2- 172.21.15.83:0/1803759115 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f4d30039600 0x7f4d3003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4d34006fd0 tx=0x7f4d34006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:31.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.449+0000 7f4d4e800700 1 -- 172.21.15.83:0/1803759115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 msgr2=0x7f4d4819e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:31.176 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.449+0000 7f4d4e800700 1 --2- 172.21.15.83:0/1803759115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 0x7f4d4819e690 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c000c00 tx=0x7f4d3c011c10 comp rx=0 tx=0).stop 2024-04-23T14:37:31.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.449+0000 7f4d4e800700 1 -- 172.21.15.83:0/1803759115 shutdown_connections 2024-04-23T14:37:31.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.449+0000 7f4d4e800700 1 --2- 172.21.15.83:0/1803759115 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f4d30039600 0x7f4d3003bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:31.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.449+0000 7f4d4e800700 1 --2- 172.21.15.83:0/1803759115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d48108520 0x7f4d4819e690 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:31.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.449+0000 7f4d4e800700 1 -- 172.21.15.83:0/1803759115 >> 172.21.15.83:0/1803759115 conn(0x7f4d48103ac0 msgr2=0x7f4d48104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:31.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.449+0000 7f4d4e800700 1 -- 172.21.15.83:0/1803759115 shutdown_connections 2024-04-23T14:37:31.177 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:30.449+0000 7f4d4e800700 1 -- 172.21.15.83:0/1803759115 wait complete. 2024-04-23T14:37:31.177 INFO:teuthology.orchestra.run.smithi083.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-23T14:37:31.177 INFO:teuthology.orchestra.run.smithi083.stdout:Adding key to root@localhost authorized_keys... 2024-04-23T14:37:31.183 INFO:teuthology.orchestra.run.smithi083.stdout:Adding host smithi083... 2024-04-23T14:37:31.699 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:31 smithi083 ceph-mon[27867]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:37:32.701 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:32 smithi083 ceph-mon[27867]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi083", "addr": "172.21.15.83", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:37:33.544 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:33 smithi083 ceph-mon[27867]: Deploying cephadm binary to smithi083 2024-04-23T14:37:38.676 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Added host 'smithi083' with addr '172.21.15.83' 2024-04-23T14:37:38.676 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.782+0000 7fd96ee00700 1 Processor -- start 2024-04-23T14:37:38.676 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.782+0000 7fd96ee00700 1 -- start start 2024-04-23T14:37:38.676 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.783+0000 7fd96ee00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 0x7fd96807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:38.676 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.783+0000 7fd96ee00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd96807c820 con 0x7fd96807de30 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.783+0000 7fd967e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 0x7fd96807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.783+0000 7fd967e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 0x7fd96807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40700/0 (socket says 172.21.15.83:40700) 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.783+0000 7fd967e00700 1 -- 172.21.15.83:0/3776180121 learned_addr learned my addr 172.21.15.83:0/3776180121 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.784+0000 7fd967e00700 1 -- 172.21.15.83:0/3776180121 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96807c9a0 con 0x7fd96807de30 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.784+0000 7fd967e00700 1 --2- 172.21.15.83:0/3776180121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 0x7fd96807c2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd9580098a0 tx=0x7fd958009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33593d18020fa8e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.784+0000 7fd966a00700 1 -- 172.21.15.83:0/3776180121 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd958004160 con 0x7fd96807de30 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.784+0000 7fd966a00700 1 -- 172.21.15.83:0/3776180121 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd958004300 con 0x7fd96807de30 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.784+0000 7fd966a00700 1 -- 172.21.15.83:0/3776180121 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd958011b50 con 0x7fd96807de30 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.785+0000 7fd96ee00700 1 -- 172.21.15.83:0/3776180121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 msgr2=0x7fd96807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.785+0000 7fd96ee00700 1 --2- 172.21.15.83:0/3776180121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 0x7fd96807c2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd9580098a0 tx=0x7fd958009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.785+0000 7fd96ee00700 1 -- 172.21.15.83:0/3776180121 shutdown_connections 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.785+0000 7fd96ee00700 1 --2- 172.21.15.83:0/3776180121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 0x7fd96807c2e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:38.677 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.785+0000 7fd96ee00700 1 -- 172.21.15.83:0/3776180121 >> 172.21.15.83:0/3776180121 conn(0x7fd968103d10 msgr2=0x7fd968106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:38.678 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.786+0000 7fd96ee00700 1 -- 172.21.15.83:0/3776180121 shutdown_connections 2024-04-23T14:37:38.678 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.786+0000 7fd96ee00700 1 -- 172.21.15.83:0/3776180121 wait complete. 2024-04-23T14:37:38.678 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.786+0000 7fd96ee00700 1 Processor -- start 2024-04-23T14:37:38.678 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.787+0000 7fd96ee00700 1 -- start start 2024-04-23T14:37:38.678 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.787+0000 7fd96ee00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 0x7fd96819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:38.678 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.787+0000 7fd96ee00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd96810ece0 con 0x7fd96807de30 2024-04-23T14:37:38.678 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.787+0000 7fd967e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 0x7fd96819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:38.678 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.787+0000 7fd967e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 0x7fd96819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40716/0 (socket says 172.21.15.83:40716) 2024-04-23T14:37:38.678 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.787+0000 7fd967e00700 1 -- 172.21.15.83:0/3710569547 learned_addr learned my addr 172.21.15.83:0/3710569547 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:38.678 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.788+0000 7fd967e00700 1 -- 172.21.15.83:0/3710569547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96819edf0 con 0x7fd96807de30 2024-04-23T14:37:38.678 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.788+0000 7fd967e00700 1 --2- 172.21.15.83:0/3710569547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 0x7fd96819e8b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd958009fb0 tx=0x7fd958011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:38.678 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.788+0000 7fd964c00700 1 -- 172.21.15.83:0/3710569547 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9580038b0 con 0x7fd96807de30 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.788+0000 7fd964c00700 1 -- 172.21.15.83:0/3710569547 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd958011e50 con 0x7fd96807de30 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.788+0000 7fd96ee00700 1 -- 172.21.15.83:0/3710569547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96819f080 con 0x7fd96807de30 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.788+0000 7fd964c00700 1 -- 172.21.15.83:0/3710569547 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd958019420 con 0x7fd96807de30 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.788+0000 7fd96ee00700 1 -- 172.21.15.83:0/3710569547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9681a1d70 con 0x7fd96807de30 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.790+0000 7fd96ee00700 1 -- 172.21.15.83:0/3710569547 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd968064150 con 0x7fd96807de30 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.790+0000 7fd964c00700 1 -- 172.21.15.83:0/3710569547 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd958003a50 con 0x7fd96807de30 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.790+0000 7fd964c00700 1 --2- 172.21.15.83:0/3710569547 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fd94c039560 0x7fd94c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.790+0000 7fd964c00700 1 -- 172.21.15.83:0/3710569547 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd95804da50 con 0x7fd96807de30 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.791+0000 7fd967400700 1 --2- 172.21.15.83:0/3710569547 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fd94c039560 0x7fd94c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.791+0000 7fd967400700 1 --2- 172.21.15.83:0/3710569547 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fd94c039560 0x7fd94c03ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd954006fd0 tx=0x7fd954006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:31.796+0000 7fd964c00700 1 -- 172.21.15.83:0/3710569547 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd95804d3a0 con 0x7fd96807de30 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:32.026+0000 7fd96ee00700 1 -- 172.21.15.83:0/3710569547 --> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi083", "addr": "172.21.15.83", "target": ["mon-mgr", ""]}) -- 0x7fd968104fa0 con 0x7fd94c039560 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:37.971+0000 7fd964c00700 1 -- 172.21.15.83:0/3710569547 <== mgr.14120 v2:172.21.15.83:6800/3974620988 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fd968104fa0 con 0x7fd94c039560 2024-04-23T14:37:38.679 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:37.975+0000 7fd96ee00700 1 -- 172.21.15.83:0/3710569547 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fd94c039560 msgr2=0x7fd94c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:38.680 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:37.975+0000 7fd96ee00700 1 --2- 172.21.15.83:0/3710569547 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fd94c039560 0x7fd94c03ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd954006fd0 tx=0x7fd954006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:38.680 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:37.975+0000 7fd96ee00700 1 -- 172.21.15.83:0/3710569547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 msgr2=0x7fd96819e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:38.680 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:37.975+0000 7fd96ee00700 1 --2- 172.21.15.83:0/3710569547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 0x7fd96819e8b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd958009fb0 tx=0x7fd958011bf0 comp rx=0 tx=0).stop 2024-04-23T14:37:38.680 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:37.976+0000 7fd96ee00700 1 -- 172.21.15.83:0/3710569547 shutdown_connections 2024-04-23T14:37:38.680 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:37.976+0000 7fd96ee00700 1 --2- 172.21.15.83:0/3710569547 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fd94c039560 0x7fd94c03ba20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:38.680 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:37.976+0000 7fd96ee00700 1 --2- 172.21.15.83:0/3710569547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd96807de30 0x7fd96819e8b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:38.680 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:37.976+0000 7fd96ee00700 1 -- 172.21.15.83:0/3710569547 >> 172.21.15.83:0/3710569547 conn(0x7fd968103d10 msgr2=0x7fd968104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:38.680 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:37.976+0000 7fd96ee00700 1 -- 172.21.15.83:0/3710569547 shutdown_connections 2024-04-23T14:37:38.680 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:37.976+0000 7fd96ee00700 1 -- 172.21.15.83:0/3710569547 wait complete. 2024-04-23T14:37:38.680 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mon service... 2024-04-23T14:37:39.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:38 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:39.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:38 smithi083 ceph-mon[27867]: Added host smithi083 2024-04-23T14:37:39.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:38 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:37:40.918 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:40 smithi083 ceph-mon[27867]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:37:40.919 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:40 smithi083 ceph-mon[27867]: Saving service mon spec with placement count:5 2024-04-23T14:37:40.919 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:40 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:41.274 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-23T14:37:41.274 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.368+0000 7f0e17200700 1 Processor -- start 2024-04-23T14:37:41.274 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.368+0000 7f0e17200700 1 -- start start 2024-04-23T14:37:41.274 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.369+0000 7f0e17200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 0x7f0e1010f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:41.274 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.369+0000 7f0e17200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1010f940 con 0x7f0e1010efe0 2024-04-23T14:37:41.274 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.369+0000 7f0e0fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 0x7f0e1010f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:41.274 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.369+0000 7f0e0fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 0x7f0e1010f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40686/0 (socket says 172.21.15.83:40686) 2024-04-23T14:37:41.274 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.369+0000 7f0e0fe00700 1 -- 172.21.15.83:0/1500495169 learned_addr learned my addr 172.21.15.83:0/1500495169 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:41.275 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.370+0000 7f0e0fe00700 1 -- 172.21.15.83:0/1500495169 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1010fac0 con 0x7f0e1010efe0 2024-04-23T14:37:41.275 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.370+0000 7f0e0fe00700 1 --2- 172.21.15.83:0/1500495169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 0x7f0e1010f400 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e000098a0 tx=0x7f0e00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbe826a35cfa974 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:41.275 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.370+0000 7f0e0ea00700 1 -- 172.21.15.83:0/1500495169 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e00004160 con 0x7f0e1010efe0 2024-04-23T14:37:41.275 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.370+0000 7f0e0ea00700 1 -- 172.21.15.83:0/1500495169 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0e00004300 con 0x7f0e1010efe0 2024-04-23T14:37:41.279 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.370+0000 7f0e0ea00700 1 -- 172.21.15.83:0/1500495169 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e00011b50 con 0x7f0e1010efe0 2024-04-23T14:37:41.279 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.371+0000 7f0e17200700 1 -- 172.21.15.83:0/1500495169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 msgr2=0x7f0e1010f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:41.279 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.371+0000 7f0e17200700 1 --2- 172.21.15.83:0/1500495169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 0x7f0e1010f400 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e000098a0 tx=0x7f0e00009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:41.279 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.371+0000 7f0e17200700 1 -- 172.21.15.83:0/1500495169 shutdown_connections 2024-04-23T14:37:41.279 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.371+0000 7f0e17200700 1 --2- 172.21.15.83:0/1500495169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 0x7f0e1010f400 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:41.279 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.371+0000 7f0e17200700 1 -- 172.21.15.83:0/1500495169 >> 172.21.15.83:0/1500495169 conn(0x7f0e1007def0 msgr2=0x7f0e10080310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:41.279 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.371+0000 7f0e17200700 1 -- 172.21.15.83:0/1500495169 shutdown_connections 2024-04-23T14:37:41.279 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.371+0000 7f0e17200700 1 -- 172.21.15.83:0/1500495169 wait complete. 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.372+0000 7f0e17200700 1 Processor -- start 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.372+0000 7f0e17200700 1 -- start start 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.373+0000 7f0e17200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 0x7f0e101a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.373+0000 7f0e17200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e10110d30 con 0x7f0e1010efe0 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.373+0000 7f0e0fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 0x7f0e101a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.373+0000 7f0e0fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 0x7f0e101a2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40702/0 (socket says 172.21.15.83:40702) 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.373+0000 7f0e0fe00700 1 -- 172.21.15.83:0/3916769522 learned_addr learned my addr 172.21.15.83:0/3916769522 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.373+0000 7f0e0fe00700 1 -- 172.21.15.83:0/3916769522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e101a31b0 con 0x7f0e1010efe0 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.374+0000 7f0e0fe00700 1 --2- 172.21.15.83:0/3916769522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 0x7f0e101a2c70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e00011850 tx=0x7f0e00011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.374+0000 7f0e0cc00700 1 -- 172.21.15.83:0/3916769522 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e00003af0 con 0x7f0e1010efe0 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.374+0000 7f0e0cc00700 1 -- 172.21.15.83:0/3916769522 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0e00020070 con 0x7f0e1010efe0 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.374+0000 7f0e17200700 1 -- 172.21.15.83:0/3916769522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e101a34a0 con 0x7f0e1010efe0 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.374+0000 7f0e0cc00700 1 -- 172.21.15.83:0/3916769522 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e00019550 con 0x7f0e1010efe0 2024-04-23T14:37:41.280 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.374+0000 7f0e17200700 1 -- 172.21.15.83:0/3916769522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e101a6130 con 0x7f0e1010efe0 2024-04-23T14:37:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.375+0000 7f0e17200700 1 -- 172.21.15.83:0/3916769522 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e10064150 con 0x7f0e1010efe0 2024-04-23T14:37:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.376+0000 7f0e0cc00700 1 -- 172.21.15.83:0/3916769522 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0e00019850 con 0x7f0e1010efe0 2024-04-23T14:37:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.376+0000 7f0e0cc00700 1 --2- 172.21.15.83:0/3916769522 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f0df40395b0 0x7f0df403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.376+0000 7f0e0cc00700 1 -- 172.21.15.83:0/3916769522 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0e00011e00 con 0x7f0e1010efe0 2024-04-23T14:37:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.377+0000 7f0e0f400700 1 --2- 172.21.15.83:0/3916769522 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f0df40395b0 0x7f0df403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.377+0000 7f0e0f400700 1 --2- 172.21.15.83:0/3916769522 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f0df40395b0 0x7f0df403ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc006fd0 tx=0x7f0dfc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.382+0000 7f0e0cc00700 1 -- 172.21.15.83:0/3916769522 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e00023370 con 0x7f0e1010efe0 2024-04-23T14:37:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.646+0000 7f0e17200700 1 -- 172.21.15.83:0/3916769522 --> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f0e101a6300 con 0x7f0df40395b0 2024-04-23T14:37:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.656+0000 7f0e0cc00700 1 -- 172.21.15.83:0/3916769522 <== mgr.14120 v2:172.21.15.83:6800/3974620988 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0e101a6300 con 0x7f0df40395b0 2024-04-23T14:37:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.659+0000 7f0e17200700 1 -- 172.21.15.83:0/3916769522 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f0df40395b0 msgr2=0x7f0df403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.659+0000 7f0e17200700 1 --2- 172.21.15.83:0/3916769522 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f0df40395b0 0x7f0df403ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc006fd0 tx=0x7f0dfc006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:41.281 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.659+0000 7f0e17200700 1 -- 172.21.15.83:0/3916769522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 msgr2=0x7f0e101a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:41.282 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.659+0000 7f0e17200700 1 --2- 172.21.15.83:0/3916769522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 0x7f0e101a2c70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e00011850 tx=0x7f0e00011c10 comp rx=0 tx=0).stop 2024-04-23T14:37:41.282 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.659+0000 7f0e17200700 1 -- 172.21.15.83:0/3916769522 shutdown_connections 2024-04-23T14:37:41.282 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.659+0000 7f0e17200700 1 --2- 172.21.15.83:0/3916769522 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f0df40395b0 0x7f0df403ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:41.282 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.659+0000 7f0e17200700 1 --2- 172.21.15.83:0/3916769522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e1010efe0 0x7f0e101a2c70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:41.282 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.659+0000 7f0e17200700 1 -- 172.21.15.83:0/3916769522 >> 172.21.15.83:0/3916769522 conn(0x7f0e1007def0 msgr2=0x7f0e1007ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:41.282 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.659+0000 7f0e17200700 1 -- 172.21.15.83:0/3916769522 shutdown_connections 2024-04-23T14:37:41.282 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:39.660+0000 7f0e17200700 1 -- 172.21.15.83:0/3916769522 wait complete. 2024-04-23T14:37:41.282 INFO:teuthology.orchestra.run.smithi083.stdout:Deploying unmanaged mgr service... 2024-04-23T14:37:42.092 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:41 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:42.093 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:41 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:42.717 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-23T14:37:42.717 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.837+0000 7f1ac0000700 1 Processor -- start 2024-04-23T14:37:42.717 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.837+0000 7f1ac0000700 1 -- start start 2024-04-23T14:37:42.717 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.837+0000 7f1ac0000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 0x7f1ab8108b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.717 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.837+0000 7f1ac0000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81090e0 con 0x7f1ab81086f0 2024-04-23T14:37:42.717 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.838+0000 7f1abd600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 0x7f1ab8108b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.717 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.838+0000 7f1abd600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 0x7f1ab8108b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40712/0 (socket says 172.21.15.83:40712) 2024-04-23T14:37:42.717 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.838+0000 7f1abd600700 1 -- 172.21.15.83:0/2477188733 learned_addr learned my addr 172.21.15.83:0/2477188733 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:42.717 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.838+0000 7f1abd600700 1 -- 172.21.15.83:0/2477188733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab81098a0 con 0x7f1ab81086f0 2024-04-23T14:37:42.717 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.839+0000 7f1abd600700 1 --2- 172.21.15.83:0/2477188733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 0x7f1ab8108b10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0098a0 tx=0x7f1aac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a900e24a4e8f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.839+0000 7f1ab7e00700 1 -- 172.21.15.83:0/2477188733 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aac004160 con 0x7f1ab81086f0 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.839+0000 7f1ab7e00700 1 -- 172.21.15.83:0/2477188733 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aac004300 con 0x7f1ab81086f0 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.839+0000 7f1ab7e00700 1 -- 172.21.15.83:0/2477188733 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aac011b20 con 0x7f1ab81086f0 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.840+0000 7f1ac0000700 1 -- 172.21.15.83:0/2477188733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 msgr2=0x7f1ab8108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.840+0000 7f1ac0000700 1 --2- 172.21.15.83:0/2477188733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 0x7f1ab8108b10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0098a0 tx=0x7f1aac009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.840+0000 7f1ac0000700 1 -- 172.21.15.83:0/2477188733 shutdown_connections 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.840+0000 7f1ac0000700 1 --2- 172.21.15.83:0/2477188733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 0x7f1ab8108b10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.840+0000 7f1ac0000700 1 -- 172.21.15.83:0/2477188733 >> 172.21.15.83:0/2477188733 conn(0x7f1ab8103d10 msgr2=0x7f1ab8106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.840+0000 7f1ac0000700 1 -- 172.21.15.83:0/2477188733 shutdown_connections 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.840+0000 7f1ac0000700 1 -- 172.21.15.83:0/2477188733 wait complete. 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.841+0000 7f1ac0000700 1 Processor -- start 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.841+0000 7f1ac0000700 1 -- start start 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.842+0000 7f1ac0000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 0x7f1ab819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.718 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.842+0000 7f1ac0000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab810cc50 con 0x7f1ab81086f0 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.842+0000 7f1abd600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 0x7f1ab819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.842+0000 7f1abd600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 0x7f1ab819e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40718/0 (socket says 172.21.15.83:40718) 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.842+0000 7f1abd600700 1 -- 172.21.15.83:0/213870075 learned_addr learned my addr 172.21.15.83:0/213870075 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.843+0000 7f1abd600700 1 -- 172.21.15.83:0/213870075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab819edd0 con 0x7f1ab81086f0 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.843+0000 7f1abd600700 1 --2- 172.21.15.83:0/213870075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 0x7f1ab819e890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1aac000c00 tx=0x7f1aac011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.843+0000 7f1ab6000700 1 -- 172.21.15.83:0/213870075 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aac003b60 con 0x7f1ab81086f0 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.843+0000 7f1ab6000700 1 -- 172.21.15.83:0/213870075 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aac01a910 con 0x7f1ab81086f0 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.843+0000 7f1ac0000700 1 -- 172.21.15.83:0/213870075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab819f060 con 0x7f1ab81086f0 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.843+0000 7f1ab6000700 1 -- 172.21.15.83:0/213870075 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aac024930 con 0x7f1ab81086f0 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.843+0000 7f1ac0000700 1 -- 172.21.15.83:0/213870075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab819f4c0 con 0x7f1ab81086f0 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.845+0000 7f1ab6000700 1 -- 172.21.15.83:0/213870075 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1aac022020 con 0x7f1ab81086f0 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.845+0000 7f1ac0000700 1 -- 172.21.15.83:0/213870075 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab8064150 con 0x7f1ab81086f0 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.845+0000 7f1ab6000700 1 --2- 172.21.15.83:0/213870075 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f1aa0039600 0x7f1aa003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.845+0000 7f1ab6000700 1 -- 172.21.15.83:0/213870075 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1aac04d830 con 0x7f1ab81086f0 2024-04-23T14:37:42.719 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.846+0000 7f1abcc00700 1 --2- 172.21.15.83:0/213870075 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f1aa0039600 0x7f1aa003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.847+0000 7f1abcc00700 1 --2- 172.21.15.83:0/213870075 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f1aa0039600 0x7f1aa003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4006fd0 tx=0x7f1aa4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:41.851+0000 7f1ab6000700 1 -- 172.21.15.83:0/213870075 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1aac02c510 con 0x7f1ab81086f0 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:42.092+0000 7f1ac0000700 1 -- 172.21.15.83:0/213870075 --> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1ab8104eb0 con 0x7f1aa0039600 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:42.100+0000 7f1ab6000700 1 -- 172.21.15.83:0/213870075 <== mgr.14120 v2:172.21.15.83:6800/3974620988 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1ab8104eb0 con 0x7f1aa0039600 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:42.103+0000 7f1ac0000700 1 -- 172.21.15.83:0/213870075 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f1aa0039600 msgr2=0x7f1aa003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:42.103+0000 7f1ac0000700 1 --2- 172.21.15.83:0/213870075 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f1aa0039600 0x7f1aa003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4006fd0 tx=0x7f1aa4006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:42.103+0000 7f1ac0000700 1 -- 172.21.15.83:0/213870075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 msgr2=0x7f1ab819e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:42.103+0000 7f1ac0000700 1 --2- 172.21.15.83:0/213870075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 0x7f1ab819e890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1aac000c00 tx=0x7f1aac011c10 comp rx=0 tx=0).stop 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:42.103+0000 7f1ac0000700 1 -- 172.21.15.83:0/213870075 shutdown_connections 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:42.103+0000 7f1ac0000700 1 --2- 172.21.15.83:0/213870075 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f1aa0039600 0x7f1aa003bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:42.103+0000 7f1ac0000700 1 --2- 172.21.15.83:0/213870075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ab81086f0 0x7f1ab819e890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:42.103+0000 7f1ac0000700 1 -- 172.21.15.83:0/213870075 >> 172.21.15.83:0/213870075 conn(0x7f1ab8103d10 msgr2=0x7f1ab81047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:42.103+0000 7f1ac0000700 1 -- 172.21.15.83:0/213870075 shutdown_connections 2024-04-23T14:37:42.720 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:42.103+0000 7f1ac0000700 1 -- 172.21.15.83:0/213870075 wait complete. 2024-04-23T14:37:43.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:43 smithi083 ceph-mon[27867]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:37:43.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:43 smithi083 ceph-mon[27867]: Saving service mgr spec with placement count:2 2024-04-23T14:37:43.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:43 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:44.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:44 smithi083 ceph-mon[27867]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:37:44.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:44 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3056574227' entity='client.admin' 2024-04-23T14:37:44.818 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.298+0000 7f808c600700 1 Processor -- start 2024-04-23T14:37:44.818 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.299+0000 7f808c600700 1 -- start start 2024-04-23T14:37:44.818 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.299+0000 7f808c600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 0x7f808410f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.818 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.299+0000 7f808c600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f808410f940 con 0x7f808410efe0 2024-04-23T14:37:44.818 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.299+0000 7f8089c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 0x7f808410f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.818 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.300+0000 7f8089c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 0x7f808410f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40726/0 (socket says 172.21.15.83:40726) 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.300+0000 7f8089c00700 1 -- 172.21.15.83:0/3633859581 learned_addr learned my addr 172.21.15.83:0/3633859581 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.300+0000 7f8089c00700 1 -- 172.21.15.83:0/3633859581 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808410fac0 con 0x7f808410efe0 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.300+0000 7f8089c00700 1 --2- 172.21.15.83:0/3633859581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 0x7f808410f400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f80780098a0 tx=0x7f8078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915b92a1608367d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.301+0000 7f8083e00700 1 -- 172.21.15.83:0/3633859581 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8078004160 con 0x7f808410efe0 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.301+0000 7f8083e00700 1 -- 172.21.15.83:0/3633859581 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8078004300 con 0x7f808410efe0 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.301+0000 7f8083e00700 1 -- 172.21.15.83:0/3633859581 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8078011b20 con 0x7f808410efe0 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.301+0000 7f808c600700 1 -- 172.21.15.83:0/3633859581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 msgr2=0x7f808410f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.301+0000 7f808c600700 1 --2- 172.21.15.83:0/3633859581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 0x7f808410f400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f80780098a0 tx=0x7f8078009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.302+0000 7f808c600700 1 -- 172.21.15.83:0/3633859581 shutdown_connections 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.302+0000 7f808c600700 1 --2- 172.21.15.83:0/3633859581 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 0x7f808410f400 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.302+0000 7f808c600700 1 -- 172.21.15.83:0/3633859581 >> 172.21.15.83:0/3633859581 conn(0x7f808407def0 msgr2=0x7f8084080310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.302+0000 7f808c600700 1 -- 172.21.15.83:0/3633859581 shutdown_connections 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.302+0000 7f808c600700 1 -- 172.21.15.83:0/3633859581 wait complete. 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.303+0000 7f808c600700 1 Processor -- start 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.303+0000 7f808c600700 1 -- start start 2024-04-23T14:37:44.819 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.303+0000 7f808c600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 0x7f80841a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.303+0000 7f808c600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8084113220 con 0x7f808410efe0 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.304+0000 7f8089c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 0x7f80841a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.304+0000 7f8089c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 0x7f80841a2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40732/0 (socket says 172.21.15.83:40732) 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.304+0000 7f8089c00700 1 -- 172.21.15.83:0/3056574227 learned_addr learned my addr 172.21.15.83:0/3056574227 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.304+0000 7f8089c00700 1 -- 172.21.15.83:0/3056574227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80841a3160 con 0x7f808410efe0 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.304+0000 7f8089c00700 1 --2- 172.21.15.83:0/3056574227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 0x7f80841a2c20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f80780116a0 tx=0x7f8078011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.305+0000 7f8082000700 1 -- 172.21.15.83:0/3056574227 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8078003a40 con 0x7f808410efe0 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.305+0000 7f8082000700 1 -- 172.21.15.83:0/3056574227 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8078003be0 con 0x7f808410efe0 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.305+0000 7f808c600700 1 -- 172.21.15.83:0/3056574227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80841a3450 con 0x7f808410efe0 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.305+0000 7f8082000700 1 -- 172.21.15.83:0/3056574227 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f807801a720 con 0x7f808410efe0 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.305+0000 7f808c600700 1 -- 172.21.15.83:0/3056574227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80841a60e0 con 0x7f808410efe0 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.306+0000 7f808c600700 1 -- 172.21.15.83:0/3056574227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8084064150 con 0x7f808410efe0 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.306+0000 7f8082000700 1 -- 172.21.15.83:0/3056574227 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f807801aa20 con 0x7f808410efe0 2024-04-23T14:37:44.820 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.307+0000 7f8082000700 1 --2- 172.21.15.83:0/3056574227 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f806c039560 0x7f806c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.307+0000 7f8082000700 1 -- 172.21.15.83:0/3056574227 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f807804dce0 con 0x7f808410efe0 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.307+0000 7f8089200700 1 --2- 172.21.15.83:0/3056574227 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f806c039560 0x7f806c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.308+0000 7f8089200700 1 --2- 172.21.15.83:0/3056574227 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f806c039560 0x7f806c03ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8070006fd0 tx=0x7f8070006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.313+0000 7f8082000700 1 -- 172.21.15.83:0/3056574227 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8078011e20 con 0x7f808410efe0 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.553+0000 7f808c600700 1 -- 172.21.15.83:0/3056574227 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f80841a67c0 con 0x7f808410efe0 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.559+0000 7f8082000700 1 -- 172.21.15.83:0/3056574227 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f807801adc0 con 0x7f808410efe0 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.561+0000 7f808c600700 1 -- 172.21.15.83:0/3056574227 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f806c039560 msgr2=0x7f806c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.561+0000 7f808c600700 1 --2- 172.21.15.83:0/3056574227 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f806c039560 0x7f806c03ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8070006fd0 tx=0x7f8070006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.562+0000 7f808c600700 1 -- 172.21.15.83:0/3056574227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 msgr2=0x7f80841a2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.562+0000 7f808c600700 1 --2- 172.21.15.83:0/3056574227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 0x7f80841a2c20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f80780116a0 tx=0x7f8078011ba0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.562+0000 7f808c600700 1 -- 172.21.15.83:0/3056574227 shutdown_connections 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.562+0000 7f808c600700 1 --2- 172.21.15.83:0/3056574227 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f806c039560 0x7f806c03ba20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.562+0000 7f808c600700 1 --2- 172.21.15.83:0/3056574227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808410efe0 0x7f80841a2c20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.562+0000 7f808c600700 1 -- 172.21.15.83:0/3056574227 >> 172.21.15.83:0/3056574227 conn(0x7f808407def0 msgr2=0x7f808407ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.562+0000 7f808c600700 1 -- 172.21.15.83:0/3056574227 shutdown_connections 2024-04-23T14:37:44.821 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:43.562+0000 7f808c600700 1 -- 172.21.15.83:0/3056574227 wait complete. 2024-04-23T14:37:46.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:45 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:46.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:45 smithi083 ceph-mon[27867]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:37:46.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:45 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/921723959' entity='client.admin' 2024-04-23T14:37:46.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.464+0000 7f376aa00700 1 Processor -- start 2024-04-23T14:37:46.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.464+0000 7f376aa00700 1 -- start start 2024-04-23T14:37:46.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.465+0000 7f376aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 0x7f3764107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:46.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.465+0000 7f376aa00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3764107ff0 con 0x7f3764107600 2024-04-23T14:37:46.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.465+0000 7f3763e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 0x7f3764107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:46.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.465+0000 7f3763e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 0x7f3764107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56632/0 (socket says 172.21.15.83:56632) 2024-04-23T14:37:46.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.465+0000 7f3763e00700 1 -- 172.21.15.83:0/3276359167 learned_addr learned my addr 172.21.15.83:0/3276359167 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:46.449 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.466+0000 7f3763e00700 1 -- 172.21.15.83:0/3276359167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3764108800 con 0x7f3764107600 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.466+0000 7f3763e00700 1 --2- 172.21.15.83:0/3276359167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 0x7f3764107a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f375400bb40 tx=0x7f375400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1557b5190d067bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.466+0000 7f3762a00700 1 -- 172.21.15.83:0/3276359167 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f375400dbe0 con 0x7f3764107600 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.466+0000 7f3762a00700 1 -- 172.21.15.83:0/3276359167 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3754004510 con 0x7f3764107600 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.466+0000 7f3762a00700 1 -- 172.21.15.83:0/3276359167 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3754010470 con 0x7f3764107600 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.467+0000 7f376aa00700 1 -- 172.21.15.83:0/3276359167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 msgr2=0x7f3764107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.467+0000 7f376aa00700 1 --2- 172.21.15.83:0/3276359167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 0x7f3764107a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f375400bb40 tx=0x7f375400be50 comp rx=0 tx=0).stop 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.467+0000 7f376aa00700 1 -- 172.21.15.83:0/3276359167 shutdown_connections 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.467+0000 7f376aa00700 1 --2- 172.21.15.83:0/3276359167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 0x7f3764107a20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.467+0000 7f376aa00700 1 -- 172.21.15.83:0/3276359167 >> 172.21.15.83:0/3276359167 conn(0x7f3764102c00 msgr2=0x7f3764105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.467+0000 7f376aa00700 1 -- 172.21.15.83:0/3276359167 shutdown_connections 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.468+0000 7f376aa00700 1 -- 172.21.15.83:0/3276359167 wait complete. 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.468+0000 7f376aa00700 1 Processor -- start 2024-04-23T14:37:46.450 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.469+0000 7f376aa00700 1 -- start start 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.469+0000 7f376aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 0x7f376419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.469+0000 7f376aa00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f376407aff0 con 0x7f3764107600 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.469+0000 7f3763e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 0x7f376419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.469+0000 7f3763e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 0x7f376419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56642/0 (socket says 172.21.15.83:56642) 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.469+0000 7f3763e00700 1 -- 172.21.15.83:0/921723959 learned_addr learned my addr 172.21.15.83:0/921723959 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.470+0000 7f3763e00700 1 -- 172.21.15.83:0/921723959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376419eeb0 con 0x7f3764107600 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.470+0000 7f3763e00700 1 --2- 172.21.15.83:0/921723959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 0x7f376419e970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f37540046e0 tx=0x7f37540040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.470+0000 7f3760c00700 1 -- 172.21.15.83:0/921723959 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f375400de20 con 0x7f3764107600 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.470+0000 7f3760c00700 1 -- 172.21.15.83:0/921723959 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375401d070 con 0x7f3764107600 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.470+0000 7f376aa00700 1 -- 172.21.15.83:0/921723959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376419f140 con 0x7f3764107600 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.470+0000 7f3760c00700 1 -- 172.21.15.83:0/921723959 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3754021a80 con 0x7f3764107600 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.471+0000 7f376aa00700 1 -- 172.21.15.83:0/921723959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37641a1e30 con 0x7f3764107600 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.472+0000 7f376aa00700 1 -- 172.21.15.83:0/921723959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3764064150 con 0x7f3764107600 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.472+0000 7f3760c00700 1 -- 172.21.15.83:0/921723959 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f375401f030 con 0x7f3764107600 2024-04-23T14:37:46.451 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.472+0000 7f3760c00700 1 --2- 172.21.15.83:0/921723959 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f3748039750 0x7f374803bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.473+0000 7f3760c00700 1 -- 172.21.15.83:0/921723959 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f375404d850 con 0x7f3764107600 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.473+0000 7f3763400700 1 --2- 172.21.15.83:0/921723959 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f3748039750 0x7f374803bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.474+0000 7f3763400700 1 --2- 172.21.15.83:0/921723959 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f3748039750 0x7f374803bc10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3750006fd0 tx=0x7f3750006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.478+0000 7f3760c00700 1 -- 172.21.15.83:0/921723959 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3754015070 con 0x7f3764107600 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.708+0000 7f376aa00700 1 -- 172.21.15.83:0/921723959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f3764197ed0 con 0x7f3764107600 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.716+0000 7f3760c00700 1 -- 172.21.15.83:0/921723959 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f3764197ed0 con 0x7f3764107600 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.719+0000 7f376aa00700 1 -- 172.21.15.83:0/921723959 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f3748039750 msgr2=0x7f374803bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.719+0000 7f376aa00700 1 --2- 172.21.15.83:0/921723959 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f3748039750 0x7f374803bc10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3750006fd0 tx=0x7f3750006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.719+0000 7f376aa00700 1 -- 172.21.15.83:0/921723959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 msgr2=0x7f376419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.719+0000 7f376aa00700 1 --2- 172.21.15.83:0/921723959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 0x7f376419e970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f37540046e0 tx=0x7f37540040b0 comp rx=0 tx=0).stop 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.719+0000 7f376aa00700 1 -- 172.21.15.83:0/921723959 shutdown_connections 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.719+0000 7f376aa00700 1 --2- 172.21.15.83:0/921723959 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7f3748039750 0x7f374803bc10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.719+0000 7f376aa00700 1 --2- 172.21.15.83:0/921723959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764107600 0x7f376419e970 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:46.452 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.719+0000 7f376aa00700 1 -- 172.21.15.83:0/921723959 >> 172.21.15.83:0/921723959 conn(0x7f3764102c00 msgr2=0x7f3764194f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:46.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.720+0000 7f376aa00700 1 -- 172.21.15.83:0/921723959 shutdown_connections 2024-04-23T14:37:46.453 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:45.720+0000 7f376aa00700 1 -- 172.21.15.83:0/921723959 wait complete. 2024-04-23T14:37:46.453 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling the dashboard module... 2024-04-23T14:37:48.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:48 smithi083 ceph-mon[27867]: from='mgr.14120 172.21.15.83:0/1203422052' entity='mgr.x' 2024-04-23T14:37:48.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:48 smithi083 ceph-mon[27867]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:37:48.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:48 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2467177065' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-23T14:37:48.438 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:48 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: ignoring --setuser ceph since I am not root 2024-04-23T14:37:48.438 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:48 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: ignoring --setgroup ceph since I am not root 2024-04-23T14:37:48.438 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:48 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:48.436+0000 7f951f31b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T14:37:48.757 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:48 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:48.685+0000 7f951f31b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:37:49.109 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:48 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:48.816+0000 7f951f31b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T14:37:49.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:49 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2467177065' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-23T14:37:49.481 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:49 smithi083 ceph-mon[27867]: mgrmap e9: x(active, since 25s) 2024-04-23T14:37:49.481 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:49 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:49.222+0000 7f951f31b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T14:37:49.481 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:49 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:49.333+0000 7f951f31b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T14:37:49.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.031+0000 7ff4e8a00700 1 Processor -- start 2024-04-23T14:37:49.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.031+0000 7ff4e8a00700 1 -- start start 2024-04-23T14:37:49.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.031+0000 7ff4e8a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 0x7ff4e407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:49.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.031+0000 7ff4e8a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e407c840 con 0x7ff4e407de50 2024-04-23T14:37:49.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.032+0000 7ff4e1e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 0x7ff4e407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:49.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.032+0000 7ff4e1e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 0x7ff4e407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56654/0 (socket says 172.21.15.83:56654) 2024-04-23T14:37:49.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.032+0000 7ff4e1e00700 1 -- 172.21.15.83:0/1145644164 learned_addr learned my addr 172.21.15.83:0/1145644164 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:49.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.032+0000 7ff4e1e00700 1 -- 172.21.15.83:0/1145644164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e407c9c0 con 0x7ff4e407de50 2024-04-23T14:37:49.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.032+0000 7ff4e1e00700 1 --2- 172.21.15.83:0/1145644164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 0x7ff4e407c300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0098a0 tx=0x7ff4cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225321c4f64ff6ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:49.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.033+0000 7ff4e1400700 1 -- 172.21.15.83:0/1145644164 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4cc004160 con 0x7ff4e407de50 2024-04-23T14:37:49.861 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.033+0000 7ff4e1400700 1 -- 172.21.15.83:0/1145644164 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc004300 con 0x7ff4e407de50 2024-04-23T14:37:49.862 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.033+0000 7ff4e1400700 1 -- 172.21.15.83:0/1145644164 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4cc011b20 con 0x7ff4e407de50 2024-04-23T14:37:49.862 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.034+0000 7ff4e8a00700 1 -- 172.21.15.83:0/1145644164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 msgr2=0x7ff4e407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:49.862 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.034+0000 7ff4e8a00700 1 --2- 172.21.15.83:0/1145644164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 0x7ff4e407c300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0098a0 tx=0x7ff4cc009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:49.863 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.034+0000 7ff4e8a00700 1 -- 172.21.15.83:0/1145644164 shutdown_connections 2024-04-23T14:37:49.863 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.034+0000 7ff4e8a00700 1 --2- 172.21.15.83:0/1145644164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 0x7ff4e407c300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:49.863 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.034+0000 7ff4e8a00700 1 -- 172.21.15.83:0/1145644164 >> 172.21.15.83:0/1145644164 conn(0x7ff4e4103d10 msgr2=0x7ff4e4106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:49.863 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.034+0000 7ff4e8a00700 1 -- 172.21.15.83:0/1145644164 shutdown_connections 2024-04-23T14:37:49.863 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.034+0000 7ff4e8a00700 1 -- 172.21.15.83:0/1145644164 wait complete. 2024-04-23T14:37:49.863 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.035+0000 7ff4e8a00700 1 Processor -- start 2024-04-23T14:37:49.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.035+0000 7ff4e8a00700 1 -- start start 2024-04-23T14:37:49.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.035+0000 7ff4e8a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 0x7ff4e419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:49.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.035+0000 7ff4e8a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e410ecb0 con 0x7ff4e407de50 2024-04-23T14:37:49.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.036+0000 7ff4e1e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 0x7ff4e419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:49.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.036+0000 7ff4e1e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 0x7ff4e419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56662/0 (socket says 172.21.15.83:56662) 2024-04-23T14:37:49.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.036+0000 7ff4e1e00700 1 -- 172.21.15.83:0/2467177065 learned_addr learned my addr 172.21.15.83:0/2467177065 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:49.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.036+0000 7ff4e1e00700 1 -- 172.21.15.83:0/2467177065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e419ed90 con 0x7ff4e407de50 2024-04-23T14:37:49.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.036+0000 7ff4e1e00700 1 --2- 172.21.15.83:0/2467177065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 0x7ff4e419e850 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0116a0 tx=0x7ff4cc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:49.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.037+0000 7ff4daa00700 1 -- 172.21.15.83:0/2467177065 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4cc003ac0 con 0x7ff4e407de50 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.037+0000 7ff4daa00700 1 -- 172.21.15.83:0/2467177065 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc003c60 con 0x7ff4e407de50 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.037+0000 7ff4e8a00700 1 -- 172.21.15.83:0/2467177065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e419f020 con 0x7ff4e407de50 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.037+0000 7ff4daa00700 1 -- 172.21.15.83:0/2467177065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4cc01a7d0 con 0x7ff4e407de50 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.037+0000 7ff4e8a00700 1 -- 172.21.15.83:0/2467177065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e419f480 con 0x7ff4e407de50 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.038+0000 7ff4daa00700 1 -- 172.21.15.83:0/2467177065 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff4cc01aa80 con 0x7ff4e407de50 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.038+0000 7ff4e8a00700 1 -- 172.21.15.83:0/2467177065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e4064150 con 0x7ff4e407de50 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.039+0000 7ff4daa00700 1 --2- 172.21.15.83:0/2467177065 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff4c80395b0 0x7ff4c803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.039+0000 7ff4daa00700 1 -- 172.21.15.83:0/2467177065 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff4cc0248f0 con 0x7ff4e407de50 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.039+0000 7ff4dbe00700 1 --2- 172.21.15.83:0/2467177065 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff4c80395b0 0x7ff4c803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.040+0000 7ff4dbe00700 1 --2- 172.21.15.83:0/2467177065 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff4c80395b0 0x7ff4c803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4006fd0 tx=0x7ff4d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.045+0000 7ff4daa00700 1 -- 172.21.15.83:0/2467177065 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4cc011e20 con 0x7ff4e407de50 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:47.345+0000 7ff4e8a00700 1 -- 172.21.15.83:0/2467177065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ff4e41a2250 con 0x7ff4e407de50 2024-04-23T14:37:49.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:48.113+0000 7ff4daa00700 1 -- 172.21.15.83:0/2467177065 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ff4cc01ae20 con 0x7ff4e407de50 2024-04-23T14:37:49.867 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:48.117+0000 7ff4daa00700 1 -- 172.21.15.83:0/2467177065 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7ff4cc02d910 con 0x7ff4e407de50 2024-04-23T14:37:49.867 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:48.117+0000 7ff4e8a00700 1 -- 172.21.15.83:0/2467177065 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff4c80395b0 msgr2=0x7ff4c803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:49.867 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:48.117+0000 7ff4e8a00700 1 --2- 172.21.15.83:0/2467177065 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff4c80395b0 0x7ff4c803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4006fd0 tx=0x7ff4d4006e40 comp rx=0 tx=0).stop 2024-04-23T14:37:49.867 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:48.117+0000 7ff4e8a00700 1 -- 172.21.15.83:0/2467177065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 msgr2=0x7ff4e419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:49.867 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:48.117+0000 7ff4e8a00700 1 --2- 172.21.15.83:0/2467177065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 0x7ff4e419e850 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0116a0 tx=0x7ff4cc011c10 comp rx=0 tx=0).stop 2024-04-23T14:37:49.867 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:48.118+0000 7ff4e8a00700 1 -- 172.21.15.83:0/2467177065 shutdown_connections 2024-04-23T14:37:49.867 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:48.118+0000 7ff4e8a00700 1 --2- 172.21.15.83:0/2467177065 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff4c80395b0 0x7ff4c803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:49.867 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:48.118+0000 7ff4e8a00700 1 --2- 172.21.15.83:0/2467177065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4e407de50 0x7ff4e419e850 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:49.867 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:48.118+0000 7ff4e8a00700 1 -- 172.21.15.83:0/2467177065 >> 172.21.15.83:0/2467177065 conn(0x7ff4e4103d10 msgr2=0x7ff4e4104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:49.867 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:48.118+0000 7ff4e8a00700 1 -- 172.21.15.83:0/2467177065 shutdown_connections 2024-04-23T14:37:49.867 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:48.118+0000 7ff4e8a00700 1 -- 172.21.15.83:0/2467177065 wait complete. 2024-04-23T14:37:51.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:50 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2465879790' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T14:37:51.007 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:50 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:50.912+0000 7f951f31b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T14:37:51.359 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:51 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:51.144+0000 7f951f31b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T14:37:51.359 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:51 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:51.358+0000 7f951f31b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T14:37:51.406 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-23T14:37:51.406 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.440+0000 7fbd59a00700 1 Processor -- start 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.440+0000 7fbd59a00700 1 -- start start 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.440+0000 7fbd59a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 0x7fbd5410b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.440+0000 7fbd59a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5407bdf0 con 0x7fbd54109250 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.441+0000 7fbd52a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 0x7fbd5410b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.441+0000 7fbd52a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 0x7fbd5410b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56702/0 (socket says 172.21.15.83:56702) 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.441+0000 7fbd52a00700 1 -- 172.21.15.83:0/3313593541 learned_addr learned my addr 172.21.15.83:0/3313593541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.441+0000 7fbd52a00700 1 -- 172.21.15.83:0/3313593541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5410bb80 con 0x7fbd54109250 2024-04-23T14:37:51.407 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.442+0000 7fbd52a00700 1 --2- 172.21.15.83:0/3313593541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 0x7fbd5410b640 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd440098a0 tx=0x7fbd44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586c4c76fa7060b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.442+0000 7fbd52000700 1 -- 172.21.15.83:0/3313593541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd44004160 con 0x7fbd54109250 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.442+0000 7fbd52000700 1 -- 172.21.15.83:0/3313593541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd44004300 con 0x7fbd54109250 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.442+0000 7fbd52000700 1 -- 172.21.15.83:0/3313593541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd44011b20 con 0x7fbd54109250 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.443+0000 7fbd59a00700 1 -- 172.21.15.83:0/3313593541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 msgr2=0x7fbd5410b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.443+0000 7fbd59a00700 1 --2- 172.21.15.83:0/3313593541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 0x7fbd5410b640 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd440098a0 tx=0x7fbd44009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.443+0000 7fbd59a00700 1 -- 172.21.15.83:0/3313593541 shutdown_connections 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.443+0000 7fbd59a00700 1 --2- 172.21.15.83:0/3313593541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 0x7fbd5410b640 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.443+0000 7fbd59a00700 1 -- 172.21.15.83:0/3313593541 >> 172.21.15.83:0/3313593541 conn(0x7fbd54102c00 msgr2=0x7fbd54105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.443+0000 7fbd59a00700 1 -- 172.21.15.83:0/3313593541 shutdown_connections 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.443+0000 7fbd59a00700 1 -- 172.21.15.83:0/3313593541 wait complete. 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.444+0000 7fbd59a00700 1 Processor -- start 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.444+0000 7fbd59a00700 1 -- start start 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.444+0000 7fbd59a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 0x7fbd5419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:51.408 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.444+0000 7fbd59a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5410cb20 con 0x7fbd54109250 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.445+0000 7fbd52a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 0x7fbd5419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.445+0000 7fbd52a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 0x7fbd5419e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56706/0 (socket says 172.21.15.83:56706) 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.445+0000 7fbd52a00700 1 -- 172.21.15.83:0/2465879790 learned_addr learned my addr 172.21.15.83:0/2465879790 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.445+0000 7fbd52a00700 1 -- 172.21.15.83:0/2465879790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5419ee50 con 0x7fbd54109250 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.445+0000 7fbd52a00700 1 --2- 172.21.15.83:0/2465879790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 0x7fbd5419e910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd44011b60 tx=0x7fbd44011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.446+0000 7fbd4b400700 1 -- 172.21.15.83:0/2465879790 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd44003b60 con 0x7fbd54109250 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.446+0000 7fbd4b400700 1 -- 172.21.15.83:0/2465879790 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd4401a910 con 0x7fbd54109250 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.446+0000 7fbd59a00700 1 -- 172.21.15.83:0/2465879790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5419f0e0 con 0x7fbd54109250 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.446+0000 7fbd4b400700 1 -- 172.21.15.83:0/2465879790 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd44024930 con 0x7fbd54109250 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.446+0000 7fbd59a00700 1 -- 172.21.15.83:0/2465879790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5419f540 con 0x7fbd54109250 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.447+0000 7fbd59a00700 1 -- 172.21.15.83:0/2465879790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd54064150 con 0x7fbd54109250 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.447+0000 7fbd4b400700 1 -- 172.21.15.83:0/2465879790 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fbd44022020 con 0x7fbd54109250 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.448+0000 7fbd4b400700 1 --2- 172.21.15.83:0/2465879790 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fbd38039600 0x7fbd3803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.448+0000 7fbd4b400700 1 -- 172.21.15.83:0/2465879790 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbd4404d950 con 0x7fbd54109250 2024-04-23T14:37:51.409 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.448+0000 7fbd4be00700 1 -- 172.21.15.83:0/2465879790 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fbd38039600 msgr2=0x7fbd3803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3974620988 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.448+0000 7fbd4be00700 1 --2- 172.21.15.83:0/2465879790 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fbd38039600 0x7fbd3803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.453+0000 7fbd4b400700 1 -- 172.21.15.83:0/2465879790 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd4400b3e0 con 0x7fbd54109250 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.649+0000 7fbd4be00700 1 -- 172.21.15.83:0/2465879790 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fbd38039600 msgr2=0x7fbd3803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3974620988 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.649+0000 7fbd4be00700 1 --2- 172.21.15.83:0/2465879790 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fbd38039600 0x7fbd3803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.736+0000 7fbd59a00700 1 -- 172.21.15.83:0/2465879790 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fbd541a1f60 con 0x7fbd54109250 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.737+0000 7fbd4b400700 1 -- 172.21.15.83:0/2465879790 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fbd4402ddb0 con 0x7fbd54109250 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.740+0000 7fbd59a00700 1 -- 172.21.15.83:0/2465879790 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fbd38039600 msgr2=0x7fbd3803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.740+0000 7fbd59a00700 1 --2- 172.21.15.83:0/2465879790 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fbd38039600 0x7fbd3803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.740+0000 7fbd59a00700 1 -- 172.21.15.83:0/2465879790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 msgr2=0x7fbd5419e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.740+0000 7fbd59a00700 1 --2- 172.21.15.83:0/2465879790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 0x7fbd5419e910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbd44011b60 tx=0x7fbd44011c40 comp rx=0 tx=0).stop 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.740+0000 7fbd59a00700 1 -- 172.21.15.83:0/2465879790 shutdown_connections 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.740+0000 7fbd59a00700 1 --2- 172.21.15.83:0/2465879790 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7fbd38039600 0x7fbd3803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.740+0000 7fbd59a00700 1 --2- 172.21.15.83:0/2465879790 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54109250 0x7fbd5419e910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.740+0000 7fbd59a00700 1 -- 172.21.15.83:0/2465879790 >> 172.21.15.83:0/2465879790 conn(0x7fbd54102c00 msgr2=0x7fbd541036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:51.410 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.740+0000 7fbd59a00700 1 -- 172.21.15.83:0/2465879790 shutdown_connections 2024-04-23T14:37:51.411 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:50.741+0000 7fbd59a00700 1 -- 172.21.15.83:0/2465879790 wait complete. 2024-04-23T14:37:51.411 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for the mgr to restart... 2024-04-23T14:37:51.411 INFO:teuthology.orchestra.run.smithi083.stdout:Waiting for mgr epoch 9... 2024-04-23T14:37:51.972 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:51 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:51.758+0000 7f951f31b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T14:37:51.973 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:51 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:51.881+0000 7f951f31b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T14:37:52.257 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:52 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:52.135+0000 7f951f31b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T14:37:52.756 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:52 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:52.387+0000 7f951f31b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:37:53.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:53 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:53.152+0000 7f951f31b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T14:37:53.819 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:53 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:53.693+0000 7f951f31b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T14:37:54.081 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:53 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:53.818+0000 7f951f31b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T14:37:54.081 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:54 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:54.080+0000 7f951f31b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T14:37:54.856 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:54 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:54.725+0000 7f951f31b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T14:37:54.856 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:54 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:54.855+0000 7f951f31b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T14:37:55.506 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:55 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:55.198+0000 7f951f31b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T14:37:55.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:55 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:55.383+0000 7f951f31b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T14:37:56.007 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:55 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:55.515+0000 7f951f31b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T14:37:56.007 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:55 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:55.667+0000 7f951f31b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T14:37:56.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:56 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:56.036+0000 7f951f31b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T14:37:57.063 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:56 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:56.781+0000 7f951f31b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T14:37:57.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:37:57 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:37:57.061+0000 7f951f31b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T14:37:57.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:57 smithi083 ceph-mon[27867]: Active manager daemon x restarted 2024-04-23T14:37:57.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:57 smithi083 ceph-mon[27867]: Activating manager daemon x 2024-04-23T14:37:57.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:57 smithi083 ceph-mon[27867]: osdmap e3: 0 total, 0 up, 0 in 2024-04-23T14:37:57.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:57 smithi083 ceph-mon[27867]: mgrmap e10: x(active, starting, since 0.00892077s) 2024-04-23T14:37:57.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:37:57.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T14:37:57.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T14:37:57.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T14:37:57.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T14:37:57.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:57 smithi083 ceph-mon[27867]: Manager daemon x is now available 2024-04-23T14:37:58.436 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:37:58.455 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T14:37:58.455 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T14:37:58.455 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:58 smithi083 ceph-mon[27867]: mgrmap e11: x(active, since 1.00892s) 2024-04-23T14:37:58.786 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout { 2024-04-23T14:37:58.786 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-23T14:37:58.786 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T14:37:58.786 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout } 2024-04-23T14:37:58.786 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.972+0000 7ff307e00700 1 Processor -- start 2024-04-23T14:37:58.786 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.972+0000 7ff307e00700 1 -- start start 2024-04-23T14:37:58.786 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.972+0000 7ff307e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 0x7ff30807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:58.786 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.972+0000 7ff307e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff30807c840 con 0x7ff30807de50 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.973+0000 7ff306a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 0x7ff30807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.973+0000 7ff306a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 0x7ff30807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56710/0 (socket says 172.21.15.83:56710) 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.973+0000 7ff306a00700 1 -- 172.21.15.83:0/3747544676 learned_addr learned my addr 172.21.15.83:0/3747544676 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.973+0000 7ff306a00700 1 -- 172.21.15.83:0/3747544676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30807c9c0 con 0x7ff30807de50 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.973+0000 7ff306a00700 1 --2- 172.21.15.83:0/3747544676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 0x7ff30807c300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80098a0 tx=0x7ff2f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf745d6b6373d526 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.974+0000 7ff305600700 1 -- 172.21.15.83:0/3747544676 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2f8004160 con 0x7ff30807de50 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.974+0000 7ff305600700 1 -- 172.21.15.83:0/3747544676 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f8004300 con 0x7ff30807de50 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.974+0000 7ff305600700 1 -- 172.21.15.83:0/3747544676 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2f8011b20 con 0x7ff30807de50 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.975+0000 7ff307e00700 1 -- 172.21.15.83:0/3747544676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 msgr2=0x7ff30807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.975+0000 7ff307e00700 1 --2- 172.21.15.83:0/3747544676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 0x7ff30807c300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80098a0 tx=0x7ff2f8009bb0 comp rx=0 tx=0).stop 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.975+0000 7ff307e00700 1 -- 172.21.15.83:0/3747544676 shutdown_connections 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.975+0000 7ff307e00700 1 --2- 172.21.15.83:0/3747544676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 0x7ff30807c300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.975+0000 7ff307e00700 1 -- 172.21.15.83:0/3747544676 >> 172.21.15.83:0/3747544676 conn(0x7ff308103ac0 msgr2=0x7ff308105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:58.787 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.975+0000 7ff307e00700 1 -- 172.21.15.83:0/3747544676 shutdown_connections 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.975+0000 7ff307e00700 1 -- 172.21.15.83:0/3747544676 wait complete. 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.976+0000 7ff307e00700 1 Processor -- start 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.977+0000 7ff307e00700 1 -- start start 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.977+0000 7ff307e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 0x7ff30819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.977+0000 7ff307e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff30810eae0 con 0x7ff30807de50 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.977+0000 7ff306a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 0x7ff30819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.977+0000 7ff306a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 0x7ff30819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56722/0 (socket says 172.21.15.83:56722) 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.977+0000 7ff306a00700 1 -- 172.21.15.83:0/2324533878 learned_addr learned my addr 172.21.15.83:0/2324533878 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.978+0000 7ff306a00700 1 -- 172.21.15.83:0/2324533878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30819eb70 con 0x7ff30807de50 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.978+0000 7ff306a00700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 0x7ff30819e630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80117e0 tx=0x7ff2f8011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.978+0000 7ff2ff400700 1 -- 172.21.15.83:0/2324533878 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2f801a430 con 0x7ff30807de50 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.978+0000 7ff2ff400700 1 -- 172.21.15.83:0/2324533878 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f801aab0 con 0x7ff30807de50 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.978+0000 7ff307e00700 1 -- 172.21.15.83:0/2324533878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30819ee00 con 0x7ff30807de50 2024-04-23T14:37:58.788 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.978+0000 7ff2ff400700 1 -- 172.21.15.83:0/2324533878 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2f8024a60 con 0x7ff30807de50 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.978+0000 7ff307e00700 1 -- 172.21.15.83:0/2324533878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30819f260 con 0x7ff30807de50 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.980+0000 7ff2ff400700 1 -- 172.21.15.83:0/2324533878 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7ff2f8024d10 con 0x7ff30807de50 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.980+0000 7ff2ff400700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 0x7ff2ec03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.980+0000 7ff2ff400700 1 -- 172.21.15.83:0/2324533878 --> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff2ec03c1d0 con 0x7ff2ec039600 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.980+0000 7ff2ff400700 1 -- 172.21.15.83:0/2324533878 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff2f804dea0 con 0x7ff30807de50 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.980+0000 7ff306000700 1 -- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 msgr2=0x7ff2ec03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3974620988 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:51.980+0000 7ff306000700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 0x7ff2ec03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:52.181+0000 7ff306000700 1 -- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 msgr2=0x7ff2ec03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3974620988 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:52.181+0000 7ff306000700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 0x7ff2ec03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:52.582+0000 7ff306000700 1 -- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 msgr2=0x7ff2ec03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3974620988 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:52.582+0000 7ff306000700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 0x7ff2ec03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:53.383+0000 7ff306000700 1 -- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 msgr2=0x7ff2ec03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3974620988 2024-04-23T14:37:58.789 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:53.383+0000 7ff306000700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 0x7ff2ec03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:54.984+0000 7ff306000700 1 -- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 msgr2=0x7ff2ec03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/3974620988 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:54.984+0000 7ff306000700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 0x7ff2ec03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:57.068+0000 7ff2ff400700 1 -- 172.21.15.83:0/2324533878 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7ff2f8022020 con 0x7ff30807de50 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:57.068+0000 7ff2ff400700 1 -- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 msgr2=0x7ff2ec03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:57.068+0000 7ff2ff400700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 0x7ff2ec03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.070+0000 7ff2ff400700 1 -- 172.21.15.83:0/2324533878 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7ff2f804faa0 con 0x7ff30807de50 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.070+0000 7ff2ff400700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff2ec03cfe0 0x7ff2ec03f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.070+0000 7ff2ff400700 1 -- 172.21.15.83:0/2324533878 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff2f802be50 con 0x7ff2ec03cfe0 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.071+0000 7ff306000700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff2ec03cfe0 0x7ff2ec03f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.072+0000 7ff306000700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff2ec03cfe0 0x7ff2ec03f3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4003a10 tx=0x7ff2f40092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.073+0000 7ff2ff400700 1 -- 172.21.15.83:0/2324533878 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ff2f802be50 con 0x7ff2ec03cfe0 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.081+0000 7ff307e00700 1 -- 172.21.15.83:0/2324533878 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff308064150 con 0x7ff2ec03cfe0 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.081+0000 7ff2ff400700 1 -- 172.21.15.83:0/2324533878 <== mgr.14152 v2:172.21.15.83:6800/1964818199 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7ff308064150 con 0x7ff2ec03cfe0 2024-04-23T14:37:58.790 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.082+0000 7ff307e00700 1 -- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff2ec03cfe0 msgr2=0x7ff2ec03f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:58.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.082+0000 7ff307e00700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff2ec03cfe0 0x7ff2ec03f3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4003a10 tx=0x7ff2f40092b0 comp rx=0 tx=0).stop 2024-04-23T14:37:58.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.082+0000 7ff307e00700 1 -- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 msgr2=0x7ff30819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:37:58.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.082+0000 7ff307e00700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 0x7ff30819e630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80117e0 tx=0x7ff2f8011ce0 comp rx=0 tx=0).stop 2024-04-23T14:37:58.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.082+0000 7ff307e00700 1 -- 172.21.15.83:0/2324533878 shutdown_connections 2024-04-23T14:37:58.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.082+0000 7ff307e00700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff2ec03cfe0 0x7ff2ec03f3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:58.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.082+0000 7ff307e00700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30807de50 0x7ff30819e630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:58.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.082+0000 7ff307e00700 1 --2- 172.21.15.83:0/2324533878 >> [v2:172.21.15.83:6800/3974620988,v1:172.21.15.83:6801/3974620988] conn(0x7ff2ec039600 0x7ff2ec03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:37:58.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.082+0000 7ff307e00700 1 -- 172.21.15.83:0/2324533878 >> 172.21.15.83:0/2324533878 conn(0x7ff308103ac0 msgr2=0x7ff308104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:37:58.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.082+0000 7ff307e00700 1 -- 172.21.15.83:0/2324533878 shutdown_connections 2024-04-23T14:37:58.791 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:58.082+0000 7ff307e00700 1 -- 172.21.15.83:0/2324533878 wait complete. 2024-04-23T14:37:58.791 INFO:teuthology.orchestra.run.smithi083.stdout:mgr epoch 9 is available 2024-04-23T14:37:58.791 INFO:teuthology.orchestra.run.smithi083.stdout:Generating a dashboard self-signed certificate... 2024-04-23T14:37:59.414 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:59 smithi083 ceph-mon[27867]: [23/Apr/2024:14:37:57] ENGINE Bus STARTING 2024-04-23T14:37:59.414 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:59 smithi083 ceph-mon[27867]: [23/Apr/2024:14:37:57] ENGINE Serving on http://172.21.15.83:8765 2024-04-23T14:37:59.414 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:59 smithi083 ceph-mon[27867]: [23/Apr/2024:14:37:57] ENGINE Serving on https://172.21.15.83:7150 2024-04-23T14:37:59.414 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:59 smithi083 ceph-mon[27867]: [23/Apr/2024:14:37:57] ENGINE Bus STARTED 2024-04-23T14:37:59.414 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:59 smithi083 ceph-mon[27867]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T14:37:59.414 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:37:59 smithi083 ceph-mon[27867]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T14:38:00.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:00 smithi083 ceph-mon[27867]: mgrmap e12: x(active, since 2s) 2024-04-23T14:38:00.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:00.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:00.567 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.462+0000 7fd924800700 1 Processor -- start 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.462+0000 7fd924800700 1 -- start start 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.462+0000 7fd924800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 0x7fd91c07c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.462+0000 7fd924800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c07c820 con 0x7fd91c07de30 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.463+0000 7fd921e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 0x7fd91c07c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.463+0000 7fd921e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 0x7fd91c07c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44190/0 (socket says 172.21.15.83:44190) 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.463+0000 7fd921e00700 1 -- 172.21.15.83:0/1493653913 learned_addr learned my addr 172.21.15.83:0/1493653913 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.463+0000 7fd921e00700 1 -- 172.21.15.83:0/1493653913 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c07c9a0 con 0x7fd91c07de30 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.464+0000 7fd921e00700 1 --2- 172.21.15.83:0/1493653913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 0x7fd91c07c2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd9100098a0 tx=0x7fd910009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c9db61f59ef898 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.464+0000 7fd920a00700 1 -- 172.21.15.83:0/1493653913 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd910004160 con 0x7fd91c07de30 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.464+0000 7fd920a00700 1 -- 172.21.15.83:0/1493653913 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd910004300 con 0x7fd91c07de30 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.464+0000 7fd920a00700 1 -- 172.21.15.83:0/1493653913 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd910011b20 con 0x7fd91c07de30 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.465+0000 7fd924800700 1 -- 172.21.15.83:0/1493653913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 msgr2=0x7fd91c07c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:00.568 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.465+0000 7fd924800700 1 --2- 172.21.15.83:0/1493653913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 0x7fd91c07c2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd9100098a0 tx=0x7fd910009bb0 comp rx=0 tx=0).stop 2024-04-23T14:38:00.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.465+0000 7fd924800700 1 -- 172.21.15.83:0/1493653913 shutdown_connections 2024-04-23T14:38:00.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.465+0000 7fd924800700 1 --2- 172.21.15.83:0/1493653913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 0x7fd91c07c2e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:00.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.465+0000 7fd924800700 1 -- 172.21.15.83:0/1493653913 >> 172.21.15.83:0/1493653913 conn(0x7fd91c103da0 msgr2=0x7fd91c1061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:00.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.465+0000 7fd924800700 1 -- 172.21.15.83:0/1493653913 shutdown_connections 2024-04-23T14:38:00.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.465+0000 7fd924800700 1 -- 172.21.15.83:0/1493653913 wait complete. 2024-04-23T14:38:00.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.466+0000 7fd924800700 1 Processor -- start 2024-04-23T14:38:00.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.466+0000 7fd924800700 1 -- start start 2024-04-23T14:38:00.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.466+0000 7fd924800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 0x7fd91c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:00.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.466+0000 7fd924800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c10ed60 con 0x7fd91c07de30 2024-04-23T14:38:00.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.467+0000 7fd921e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 0x7fd91c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:00.569 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.467+0000 7fd921e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 0x7fd91c19e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44204/0 (socket says 172.21.15.83:44204) 2024-04-23T14:38:00.572 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.467+0000 7fd921e00700 1 -- 172.21.15.83:0/4060891308 learned_addr learned my addr 172.21.15.83:0/4060891308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:00.572 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.467+0000 7fd921e00700 1 -- 172.21.15.83:0/4060891308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c19ed50 con 0x7fd91c07de30 2024-04-23T14:38:00.572 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.467+0000 7fd921e00700 1 --2- 172.21.15.83:0/4060891308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 0x7fd91c19e810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd910011b60 tx=0x7fd910011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:00.572 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.468+0000 7fd916a00700 1 -- 172.21.15.83:0/4060891308 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd910003b60 con 0x7fd91c07de30 2024-04-23T14:38:00.572 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.468+0000 7fd916a00700 1 -- 172.21.15.83:0/4060891308 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd91001a910 con 0x7fd91c07de30 2024-04-23T14:38:00.572 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.468+0000 7fd924800700 1 -- 172.21.15.83:0/4060891308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91c19efe0 con 0x7fd91c07de30 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.468+0000 7fd916a00700 1 -- 172.21.15.83:0/4060891308 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd910024930 con 0x7fd91c07de30 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.468+0000 7fd924800700 1 -- 172.21.15.83:0/4060891308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91c19f440 con 0x7fd91c07de30 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.469+0000 7fd916a00700 1 -- 172.21.15.83:0/4060891308 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fd910022020 con 0x7fd91c07de30 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.469+0000 7fd924800700 1 -- 172.21.15.83:0/4060891308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd91c064150 con 0x7fd91c07de30 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.470+0000 7fd916a00700 1 --2- 172.21.15.83:0/4060891308 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd904039660 0x7fd90403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.470+0000 7fd916a00700 1 -- 172.21.15.83:0/4060891308 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd91004db40 con 0x7fd91c07de30 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.470+0000 7fd921400700 1 --2- 172.21.15.83:0/4060891308 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd904039660 0x7fd90403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.471+0000 7fd921400700 1 --2- 172.21.15.83:0/4060891308 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd904039660 0x7fd90403bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd90c006fd0 tx=0x7fd90c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.475+0000 7fd916a00700 1 -- 172.21.15.83:0/4060891308 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd91001a430 con 0x7fd91c07de30 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.709+0000 7fd924800700 1 -- 172.21.15.83:0/4060891308 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fd91c104fa0 con 0x7fd904039660 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.859+0000 7fd916a00700 1 -- 172.21.15.83:0/4060891308 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fd91c104fa0 con 0x7fd904039660 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.862+0000 7fd924800700 1 -- 172.21.15.83:0/4060891308 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd904039660 msgr2=0x7fd90403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.862+0000 7fd924800700 1 --2- 172.21.15.83:0/4060891308 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd904039660 0x7fd90403bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd90c006fd0 tx=0x7fd90c006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.862+0000 7fd924800700 1 -- 172.21.15.83:0/4060891308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 msgr2=0x7fd91c19e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:00.573 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.862+0000 7fd924800700 1 --2- 172.21.15.83:0/4060891308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 0x7fd91c19e810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd910011b60 tx=0x7fd910011c40 comp rx=0 tx=0).stop 2024-04-23T14:38:00.574 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.862+0000 7fd924800700 1 -- 172.21.15.83:0/4060891308 shutdown_connections 2024-04-23T14:38:00.574 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.862+0000 7fd924800700 1 --2- 172.21.15.83:0/4060891308 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd904039660 0x7fd90403bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:00.574 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.862+0000 7fd924800700 1 --2- 172.21.15.83:0/4060891308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd91c07de30 0x7fd91c19e810 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:00.574 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.862+0000 7fd924800700 1 -- 172.21.15.83:0/4060891308 >> 172.21.15.83:0/4060891308 conn(0x7fd91c103da0 msgr2=0x7fd91c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:00.574 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.863+0000 7fd924800700 1 -- 172.21.15.83:0/4060891308 shutdown_connections 2024-04-23T14:38:00.574 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:37:59.863+0000 7fd924800700 1 -- 172.21.15.83:0/4060891308 wait complete. 2024-04-23T14:38:00.574 INFO:teuthology.orchestra.run.smithi083.stdout:Creating initial admin user... 2024-04-23T14:38:01.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:01 smithi083 ceph-mon[27867]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:38:01.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:01 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:02.411 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$L90aUvP0JX3r8D09Lsc0EOj.vY5AP0ViYG0zxIPPQQXrMPR1JkVLi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713883081, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-23T14:38:02.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.138+0000 7f4985600700 1 Processor -- start 2024-04-23T14:38:02.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.138+0000 7f4985600700 1 -- start start 2024-04-23T14:38:02.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.139+0000 7f4985600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 0x7f498010ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:02.412 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.139+0000 7f4985600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f498010b300 con 0x7f498010a910 2024-04-23T14:38:02.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.139+0000 7f497ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 0x7f498010ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:02.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.139+0000 7f497ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 0x7f498010ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44208/0 (socket says 172.21.15.83:44208) 2024-04-23T14:38:02.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.139+0000 7f497ea00700 1 -- 172.21.15.83:0/2660003500 learned_addr learned my addr 172.21.15.83:0/2660003500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:02.413 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.140+0000 7f497ea00700 1 -- 172.21.15.83:0/2660003500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498010bac0 con 0x7f498010a910 2024-04-23T14:38:02.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.140+0000 7f497ea00700 1 --2- 172.21.15.83:0/2660003500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 0x7f498010ad30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f49700098a0 tx=0x7f4970009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b6f5dde4b76d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:02.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.140+0000 7f497d600700 1 -- 172.21.15.83:0/2660003500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4970004160 con 0x7f498010a910 2024-04-23T14:38:02.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.140+0000 7f497d600700 1 -- 172.21.15.83:0/2660003500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4970004300 con 0x7f498010a910 2024-04-23T14:38:02.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.140+0000 7f497d600700 1 -- 172.21.15.83:0/2660003500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4970011b20 con 0x7f498010a910 2024-04-23T14:38:02.415 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.141+0000 7f4985600700 1 -- 172.21.15.83:0/2660003500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 msgr2=0x7f498010ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.141+0000 7f4985600700 1 --2- 172.21.15.83:0/2660003500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 0x7f498010ad30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f49700098a0 tx=0x7f4970009bb0 comp rx=0 tx=0).stop 2024-04-23T14:38:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.141+0000 7f4985600700 1 -- 172.21.15.83:0/2660003500 shutdown_connections 2024-04-23T14:38:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.141+0000 7f4985600700 1 --2- 172.21.15.83:0/2660003500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 0x7f498010ad30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.141+0000 7f4985600700 1 -- 172.21.15.83:0/2660003500 >> 172.21.15.83:0/2660003500 conn(0x7f498007de90 msgr2=0x7f49801084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.141+0000 7f4985600700 1 -- 172.21.15.83:0/2660003500 shutdown_connections 2024-04-23T14:38:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.141+0000 7f4985600700 1 -- 172.21.15.83:0/2660003500 wait complete. 2024-04-23T14:38:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.142+0000 7f4985600700 1 Processor -- start 2024-04-23T14:38:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.142+0000 7f4985600700 1 -- start start 2024-04-23T14:38:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.143+0000 7f4985600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 0x7f498019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.143+0000 7f4985600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f498010ee70 con 0x7f498010a910 2024-04-23T14:38:02.416 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.143+0000 7f497ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 0x7f498019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:02.417 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.143+0000 7f497ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 0x7f498019e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44210/0 (socket says 172.21.15.83:44210) 2024-04-23T14:38:02.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.143+0000 7f497ea00700 1 -- 172.21.15.83:0/2598868627 learned_addr learned my addr 172.21.15.83:0/2598868627 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:02.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.143+0000 7f497ea00700 1 -- 172.21.15.83:0/2598868627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498019ed10 con 0x7f498010a910 2024-04-23T14:38:02.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.144+0000 7f497ea00700 1 --2- 172.21.15.83:0/2598868627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 0x7f498019e7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f49700117e0 tx=0x7f4970011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:02.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.144+0000 7f4977400700 1 -- 172.21.15.83:0/2598868627 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4970003bf0 con 0x7f498010a910 2024-04-23T14:38:02.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.144+0000 7f4977400700 1 -- 172.21.15.83:0/2598868627 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f497001a910 con 0x7f498010a910 2024-04-23T14:38:02.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.144+0000 7f4985600700 1 -- 172.21.15.83:0/2598868627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498019efa0 con 0x7f498010a910 2024-04-23T14:38:02.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.144+0000 7f4977400700 1 -- 172.21.15.83:0/2598868627 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4970024970 con 0x7f498010a910 2024-04-23T14:38:02.419 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.144+0000 7f4985600700 1 -- 172.21.15.83:0/2598868627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498019f400 con 0x7f498010a910 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.145+0000 7f4977400700 1 -- 172.21.15.83:0/2598868627 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f4970022020 con 0x7f498010a910 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.145+0000 7f4985600700 1 -- 172.21.15.83:0/2598868627 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4980064150 con 0x7f498010a910 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.146+0000 7f4977400700 1 --2- 172.21.15.83:0/2598868627 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4964039660 0x7f496403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.146+0000 7f4977400700 1 -- 172.21.15.83:0/2598868627 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f497004dd60 con 0x7f498010a910 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.146+0000 7f497e000700 1 --2- 172.21.15.83:0/2598868627 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4964039660 0x7f496403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.147+0000 7f497e000700 1 --2- 172.21.15.83:0/2598868627 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4964039660 0x7f496403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f496c006fd0 tx=0x7f496c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.151+0000 7f4977400700 1 -- 172.21.15.83:0/2598868627 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f497002d910 con 0x7f498010a910 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.395+0000 7f4985600700 1 -- 172.21.15.83:0/2598868627 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f49801a1e30 con 0x7f4964039660 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.662+0000 7f4977400700 1 -- 172.21.15.83:0/2598868627 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f49801a1e30 con 0x7f4964039660 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.665+0000 7f4985600700 1 -- 172.21.15.83:0/2598868627 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4964039660 msgr2=0x7f496403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.665+0000 7f4985600700 1 --2- 172.21.15.83:0/2598868627 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4964039660 0x7f496403bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f496c006fd0 tx=0x7f496c006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.665+0000 7f4985600700 1 -- 172.21.15.83:0/2598868627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 msgr2=0x7f498019e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.665+0000 7f4985600700 1 --2- 172.21.15.83:0/2598868627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 0x7f498019e7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f49700117e0 tx=0x7f4970011ba0 comp rx=0 tx=0).stop 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.665+0000 7f4985600700 1 -- 172.21.15.83:0/2598868627 shutdown_connections 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.665+0000 7f4985600700 1 --2- 172.21.15.83:0/2598868627 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4964039660 0x7f496403bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:02.420 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.665+0000 7f4985600700 1 --2- 172.21.15.83:0/2598868627 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f498010a910 0x7f498019e7d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:02.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.665+0000 7f4985600700 1 -- 172.21.15.83:0/2598868627 >> 172.21.15.83:0/2598868627 conn(0x7f498007de90 msgr2=0x7f4980106b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:02.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.666+0000 7f4985600700 1 -- 172.21.15.83:0/2598868627 shutdown_connections 2024-04-23T14:38:02.421 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:01.666+0000 7f4985600700 1 -- 172.21.15.83:0/2598868627 wait complete. 2024-04-23T14:38:02.421 INFO:teuthology.orchestra.run.smithi083.stdout:Fetching dashboard port number... 2024-04-23T14:38:02.964 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:02 smithi083 ceph-mon[27867]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:38:02.964 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:03.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:03.911 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:03 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1496896314' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-23T14:38:03.921 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stdout 8443 2024-04-23T14:38:03.921 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.033+0000 7f6f9f400700 1 Processor -- start 2024-04-23T14:38:03.921 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.033+0000 7f6f9f400700 1 -- start start 2024-04-23T14:38:03.922 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.033+0000 7f6f9f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 0x7f6f9807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:03.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.033+0000 7f6f9f400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9807c8f0 con 0x7f6f9807de70 2024-04-23T14:38:03.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.034+0000 7f6f9ca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 0x7f6f9807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:03.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.034+0000 7f6f9ca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 0x7f6f9807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44214/0 (socket says 172.21.15.83:44214) 2024-04-23T14:38:03.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.034+0000 7f6f9ca00700 1 -- 172.21.15.83:0/2414169830 learned_addr learned my addr 172.21.15.83:0/2414169830 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:03.923 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.035+0000 7f6f9ca00700 1 -- 172.21.15.83:0/2414169830 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9807ca70 con 0x7f6f9807de70 2024-04-23T14:38:03.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.035+0000 7f6f9ca00700 1 --2- 172.21.15.83:0/2414169830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 0x7f6f9807c320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0098a0 tx=0x7f6f8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67be500c602cfba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:03.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.035+0000 7f6f97400700 1 -- 172.21.15.83:0/2414169830 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f8c004160 con 0x7f6f9807de70 2024-04-23T14:38:03.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.035+0000 7f6f97400700 1 -- 172.21.15.83:0/2414169830 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8c004300 con 0x7f6f9807de70 2024-04-23T14:38:03.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.035+0000 7f6f97400700 1 -- 172.21.15.83:0/2414169830 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f8c011b20 con 0x7f6f9807de70 2024-04-23T14:38:03.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.036+0000 7f6f9f400700 1 -- 172.21.15.83:0/2414169830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 msgr2=0x7f6f9807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:03.924 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.036+0000 7f6f9f400700 1 --2- 172.21.15.83:0/2414169830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 0x7f6f9807c320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0098a0 tx=0x7f6f8c009bb0 comp rx=0 tx=0).stop 2024-04-23T14:38:03.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.036+0000 7f6f9f400700 1 -- 172.21.15.83:0/2414169830 shutdown_connections 2024-04-23T14:38:03.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.036+0000 7f6f9f400700 1 --2- 172.21.15.83:0/2414169830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 0x7f6f9807c320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:03.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.036+0000 7f6f9f400700 1 -- 172.21.15.83:0/2414169830 >> 172.21.15.83:0/2414169830 conn(0x7f6f98103d40 msgr2=0x7f6f98106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:03.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.037+0000 7f6f9f400700 1 -- 172.21.15.83:0/2414169830 shutdown_connections 2024-04-23T14:38:03.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.037+0000 7f6f9f400700 1 -- 172.21.15.83:0/2414169830 wait complete. 2024-04-23T14:38:03.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.038+0000 7f6f9f400700 1 Processor -- start 2024-04-23T14:38:03.925 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.038+0000 7f6f9f400700 1 -- start start 2024-04-23T14:38:03.926 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.038+0000 7f6f9f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 0x7f6f9819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:03.926 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.038+0000 7f6f9f400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9810ed00 con 0x7f6f9807de70 2024-04-23T14:38:03.927 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.039+0000 7f6f9ca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 0x7f6f9819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:03.927 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.039+0000 7f6f9ca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 0x7f6f9819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44220/0 (socket says 172.21.15.83:44220) 2024-04-23T14:38:03.927 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.039+0000 7f6f9ca00700 1 -- 172.21.15.83:0/1496896314 learned_addr learned my addr 172.21.15.83:0/1496896314 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:03.927 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.039+0000 7f6f9ca00700 1 -- 172.21.15.83:0/1496896314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9819edb0 con 0x7f6f9807de70 2024-04-23T14:38:03.927 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.039+0000 7f6f9ca00700 1 --2- 172.21.15.83:0/1496896314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 0x7f6f9819e870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c011b60 tx=0x7f6f8c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:03.927 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.040+0000 7f6f95600700 1 -- 172.21.15.83:0/1496896314 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f8c003b60 con 0x7f6f9807de70 2024-04-23T14:38:03.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.040+0000 7f6f95600700 1 -- 172.21.15.83:0/1496896314 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8c01a910 con 0x7f6f9807de70 2024-04-23T14:38:03.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.040+0000 7f6f95600700 1 -- 172.21.15.83:0/1496896314 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f8c024930 con 0x7f6f9807de70 2024-04-23T14:38:03.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.040+0000 7f6f9f400700 1 -- 172.21.15.83:0/1496896314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9819f040 con 0x7f6f9807de70 2024-04-23T14:38:03.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.040+0000 7f6f9f400700 1 -- 172.21.15.83:0/1496896314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f9819f4a0 con 0x7f6f9807de70 2024-04-23T14:38:03.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.041+0000 7f6f95600700 1 -- 172.21.15.83:0/1496896314 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f6f8c022020 con 0x7f6f9807de70 2024-04-23T14:38:03.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.041+0000 7f6f9f400700 1 -- 172.21.15.83:0/1496896314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f98064150 con 0x7f6f9807de70 2024-04-23T14:38:03.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.041+0000 7f6f95600700 1 --2- 172.21.15.83:0/1496896314 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6f80039660 0x7f6f8003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:03.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.041+0000 7f6f95600700 1 -- 172.21.15.83:0/1496896314 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f8c04db30 con 0x7f6f9807de70 2024-04-23T14:38:03.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.042+0000 7f6f97e00700 1 --2- 172.21.15.83:0/1496896314 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6f80039660 0x7f6f8003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:03.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.043+0000 7f6f97e00700 1 --2- 172.21.15.83:0/1496896314 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6f80039660 0x7f6f8003bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6f84006fd0 tx=0x7f6f84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:03.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.052+0000 7f6f95600700 1 -- 172.21.15.83:0/1496896314 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f8c01a4e0 con 0x7f6f9807de70 2024-04-23T14:38:03.930 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.284+0000 7f6f9f400700 1 -- 172.21.15.83:0/1496896314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f6f981a1e70 con 0x7f6f9807de70 2024-04-23T14:38:03.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.285+0000 7f6f95600700 1 -- 172.21.15.83:0/1496896314 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f6f8c02d3f0 con 0x7f6f9807de70 2024-04-23T14:38:03.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.289+0000 7f6f9f400700 1 -- 172.21.15.83:0/1496896314 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6f80039660 msgr2=0x7f6f8003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:03.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.289+0000 7f6f9f400700 1 --2- 172.21.15.83:0/1496896314 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6f80039660 0x7f6f8003bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6f84006fd0 tx=0x7f6f84006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:03.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.289+0000 7f6f9f400700 1 -- 172.21.15.83:0/1496896314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 msgr2=0x7f6f9819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:03.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.289+0000 7f6f9f400700 1 --2- 172.21.15.83:0/1496896314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 0x7f6f9819e870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c011b60 tx=0x7f6f8c011c40 comp rx=0 tx=0).stop 2024-04-23T14:38:03.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.290+0000 7f6f9f400700 1 -- 172.21.15.83:0/1496896314 shutdown_connections 2024-04-23T14:38:03.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.290+0000 7f6f9f400700 1 --2- 172.21.15.83:0/1496896314 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6f80039660 0x7f6f8003bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:03.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.290+0000 7f6f9f400700 1 --2- 172.21.15.83:0/1496896314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f9807de70 0x7f6f9819e870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:03.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.290+0000 7f6f9f400700 1 -- 172.21.15.83:0/1496896314 >> 172.21.15.83:0/1496896314 conn(0x7f6f98103d40 msgr2=0x7f6f98104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:03.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.290+0000 7f6f9f400700 1 -- 172.21.15.83:0/1496896314 shutdown_connections 2024-04-23T14:38:03.931 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:03.290+0000 7f6f9f400700 1 -- 172.21.15.83:0/1496896314 wait complete. 2024-04-23T14:38:03.963 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T14:38:03.964 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout disabled 2024-04-23T14:38:03.981 INFO:teuthology.orchestra.run.smithi083.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T14:38:03.982 INFO:teuthology.orchestra.run.smithi083.stdout:systemctl: stdout inactive 2024-04-23T14:38:03.982 INFO:teuthology.orchestra.run.smithi083.stdout:firewalld.service is not enabled 2024-04-23T14:38:03.982 INFO:teuthology.orchestra.run.smithi083.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-23T14:38:03.985 INFO:teuthology.orchestra.run.smithi083.stdout:Ceph Dashboard is now available at: 2024-04-23T14:38:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:38:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: URL: https://smithi083.front.sepia.ceph.com:8443/ 2024-04-23T14:38:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: User: admin 2024-04-23T14:38:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: Password: yku0ap2rn9 2024-04-23T14:38:03.985 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:38:03.986 INFO:teuthology.orchestra.run.smithi083.stdout:Saving cluster configuration to /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config directory 2024-04-23T14:38:03.987 INFO:teuthology.orchestra.run.smithi083.stdout:Enabling autotune for osd_memory_target 2024-04-23T14:38:04.792 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:04 smithi083 ceph-mon[27867]: mgrmap e13: x(active, since 6s) 2024-04-23T14:38:05.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.740+0000 7f1d61600700 1 Processor -- start 2024-04-23T14:38:05.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.740+0000 7f1d61600700 1 -- start start 2024-04-23T14:38:05.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.740+0000 7f1d61600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 0x7f1d5c10b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:05.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.740+0000 7f1d61600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c07be50 con 0x7f1d5c1091f0 2024-04-23T14:38:05.911 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.741+0000 7f1d5aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 0x7f1d5c10b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.741+0000 7f1d5aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 0x7f1d5c10b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44228/0 (socket says 172.21.15.83:44228) 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.741+0000 7f1d5aa00700 1 -- 172.21.15.83:0/1748689845 learned_addr learned my addr 172.21.15.83:0/1748689845 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.741+0000 7f1d5aa00700 1 -- 172.21.15.83:0/1748689845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c10bb20 con 0x7f1d5c1091f0 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.742+0000 7f1d5aa00700 1 --2- 172.21.15.83:0/1748689845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 0x7f1d5c10b5e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0098a0 tx=0x7f1d4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9e71c790075ebe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.742+0000 7f1d59600700 1 -- 172.21.15.83:0/1748689845 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d4c004160 con 0x7f1d5c1091f0 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.742+0000 7f1d59600700 1 -- 172.21.15.83:0/1748689845 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d4c004300 con 0x7f1d5c1091f0 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.742+0000 7f1d59600700 1 -- 172.21.15.83:0/1748689845 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d4c011b20 con 0x7f1d5c1091f0 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.743+0000 7f1d61600700 1 -- 172.21.15.83:0/1748689845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 msgr2=0x7f1d5c10b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.743+0000 7f1d61600700 1 --2- 172.21.15.83:0/1748689845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 0x7f1d5c10b5e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0098a0 tx=0x7f1d4c009bb0 comp rx=0 tx=0).stop 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.743+0000 7f1d61600700 1 -- 172.21.15.83:0/1748689845 shutdown_connections 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.743+0000 7f1d61600700 1 --2- 172.21.15.83:0/1748689845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 0x7f1d5c10b5e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.743+0000 7f1d61600700 1 -- 172.21.15.83:0/1748689845 >> 172.21.15.83:0/1748689845 conn(0x7f1d5c102c00 msgr2=0x7f1d5c105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.743+0000 7f1d61600700 1 -- 172.21.15.83:0/1748689845 shutdown_connections 2024-04-23T14:38:05.912 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.743+0000 7f1d61600700 1 -- 172.21.15.83:0/1748689845 wait complete. 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.744+0000 7f1d61600700 1 Processor -- start 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.744+0000 7f1d61600700 1 -- start start 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.745+0000 7f1d61600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 0x7f1d5c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.745+0000 7f1d61600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c10cb20 con 0x7f1d5c1091f0 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.745+0000 7f1d5aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 0x7f1d5c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.745+0000 7f1d5aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 0x7f1d5c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44240/0 (socket says 172.21.15.83:44240) 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.745+0000 7f1d5aa00700 1 -- 172.21.15.83:0/417373106 learned_addr learned my addr 172.21.15.83:0/417373106 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.745+0000 7f1d5aa00700 1 -- 172.21.15.83:0/417373106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c19ee10 con 0x7f1d5c1091f0 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.745+0000 7f1d5aa00700 1 --2- 172.21.15.83:0/417373106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 0x7f1d5c19e8d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c011b60 tx=0x7f1d4c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.746+0000 7f1d53400700 1 -- 172.21.15.83:0/417373106 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d4c003b60 con 0x7f1d5c1091f0 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.746+0000 7f1d53400700 1 -- 172.21.15.83:0/417373106 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d4c01a910 con 0x7f1d5c1091f0 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.746+0000 7f1d61600700 1 -- 172.21.15.83:0/417373106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5c19f0a0 con 0x7f1d5c1091f0 2024-04-23T14:38:05.913 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.746+0000 7f1d53400700 1 -- 172.21.15.83:0/417373106 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d4c024930 con 0x7f1d5c1091f0 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.746+0000 7f1d61600700 1 -- 172.21.15.83:0/417373106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d5c19f500 con 0x7f1d5c1091f0 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.747+0000 7f1d53400700 1 -- 172.21.15.83:0/417373106 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1d4c022020 con 0x7f1d5c1091f0 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.747+0000 7f1d61600700 1 -- 172.21.15.83:0/417373106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d5c064150 con 0x7f1d5c1091f0 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.747+0000 7f1d53400700 1 --2- 172.21.15.83:0/417373106 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1d400396a0 0x7f1d4003bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.747+0000 7f1d53400700 1 -- 172.21.15.83:0/417373106 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1d4c04db00 con 0x7f1d5c1091f0 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.748+0000 7f1d5a000700 1 --2- 172.21.15.83:0/417373106 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1d400396a0 0x7f1d4003bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.749+0000 7f1d5a000700 1 --2- 172.21.15.83:0/417373106 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1d400396a0 0x7f1d4003bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d48006fd0 tx=0x7f1d48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.753+0000 7f1d53400700 1 -- 172.21.15.83:0/417373106 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d4c00b3e0 con 0x7f1d5c1091f0 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.983+0000 7f1d61600700 1 -- 172.21.15.83:0/417373106 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f1d5c1a1ed0 con 0x7f1d5c1091f0 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.984+0000 7f1d53400700 1 -- 172.21.15.83:0/417373106 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f1d4c02ddb0 con 0x7f1d5c1091f0 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.987+0000 7f1d61600700 1 -- 172.21.15.83:0/417373106 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1d400396a0 msgr2=0x7f1d4003bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.987+0000 7f1d61600700 1 --2- 172.21.15.83:0/417373106 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1d400396a0 0x7f1d4003bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1d48006fd0 tx=0x7f1d48006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.987+0000 7f1d61600700 1 -- 172.21.15.83:0/417373106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 msgr2=0x7f1d5c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:05.914 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.987+0000 7f1d61600700 1 --2- 172.21.15.83:0/417373106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 0x7f1d5c19e8d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c011b60 tx=0x7f1d4c011c40 comp rx=0 tx=0).stop 2024-04-23T14:38:05.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.987+0000 7f1d61600700 1 -- 172.21.15.83:0/417373106 shutdown_connections 2024-04-23T14:38:05.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.987+0000 7f1d61600700 1 --2- 172.21.15.83:0/417373106 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1d400396a0 0x7f1d4003bb60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:05.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.987+0000 7f1d61600700 1 --2- 172.21.15.83:0/417373106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d5c1091f0 0x7f1d5c19e8d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:05.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.987+0000 7f1d61600700 1 -- 172.21.15.83:0/417373106 >> 172.21.15.83:0/417373106 conn(0x7f1d5c102c00 msgr2=0x7f1d5c103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:05.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.987+0000 7f1d61600700 1 -- 172.21.15.83:0/417373106 shutdown_connections 2024-04-23T14:38:05.915 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:04.987+0000 7f1d61600700 1 -- 172.21.15.83:0/417373106 wait complete. 2024-04-23T14:38:07.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.829+0000 7fe720c00700 1 Processor -- start 2024-04-23T14:38:07.865 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.829+0000 7fe720c00700 1 -- start start 2024-04-23T14:38:07.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.830+0000 7fe720c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 0x7fe71c108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:07.866 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.830+0000 7fe720c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c1090f0 con 0x7fe71c108700 2024-04-23T14:38:07.871 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.830+0000 7fe719e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 0x7fe71c108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:07.871 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.830+0000 7fe719e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 0x7fe71c108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44390/0 (socket says 172.21.15.83:44390) 2024-04-23T14:38:07.871 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.830+0000 7fe719e00700 1 -- 172.21.15.83:0/515950519 learned_addr learned my addr 172.21.15.83:0/515950519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:07.871 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.831+0000 7fe719e00700 1 -- 172.21.15.83:0/515950519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c1098b0 con 0x7fe71c108700 2024-04-23T14:38:07.871 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.831+0000 7fe719e00700 1 --2- 172.21.15.83:0/515950519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 0x7fe71c108b20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0098a0 tx=0x7fe70c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cdc393389178c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.831+0000 7fe718a00700 1 -- 172.21.15.83:0/515950519 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe70c004160 con 0x7fe71c108700 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.831+0000 7fe718a00700 1 -- 172.21.15.83:0/515950519 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70c004300 con 0x7fe71c108700 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.831+0000 7fe718a00700 1 -- 172.21.15.83:0/515950519 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe70c011b20 con 0x7fe71c108700 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.832+0000 7fe720c00700 1 -- 172.21.15.83:0/515950519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 msgr2=0x7fe71c108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.832+0000 7fe720c00700 1 --2- 172.21.15.83:0/515950519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 0x7fe71c108b20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0098a0 tx=0x7fe70c009bb0 comp rx=0 tx=0).stop 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.832+0000 7fe720c00700 1 -- 172.21.15.83:0/515950519 shutdown_connections 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.832+0000 7fe720c00700 1 --2- 172.21.15.83:0/515950519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 0x7fe71c108b20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.832+0000 7fe720c00700 1 -- 172.21.15.83:0/515950519 >> 172.21.15.83:0/515950519 conn(0x7fe71c103d40 msgr2=0x7fe71c106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.832+0000 7fe720c00700 1 -- 172.21.15.83:0/515950519 shutdown_connections 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.832+0000 7fe720c00700 1 -- 172.21.15.83:0/515950519 wait complete. 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.833+0000 7fe720c00700 1 Processor -- start 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.834+0000 7fe720c00700 1 -- start start 2024-04-23T14:38:07.872 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.834+0000 7fe720c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 0x7fe71c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.834+0000 7fe720c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c10cc60 con 0x7fe71c108700 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.834+0000 7fe719e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 0x7fe71c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.834+0000 7fe719e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 0x7fe71c19c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44404/0 (socket says 172.21.15.83:44404) 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.834+0000 7fe719e00700 1 -- 172.21.15.83:0/1266234513 learned_addr learned my addr 172.21.15.83:0/1266234513 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.835+0000 7fe719e00700 1 -- 172.21.15.83:0/1266234513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c19cba0 con 0x7fe71c108700 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.835+0000 7fe719e00700 1 --2- 172.21.15.83:0/1266234513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 0x7fe71c19c660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0116a0 tx=0x7fe70c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.835+0000 7fe712a00700 1 -- 172.21.15.83:0/1266234513 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe70c003a40 con 0x7fe71c108700 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.835+0000 7fe712a00700 1 -- 172.21.15.83:0/1266234513 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70c003be0 con 0x7fe71c108700 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.835+0000 7fe720c00700 1 -- 172.21.15.83:0/1266234513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c19ce30 con 0x7fe71c108700 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.835+0000 7fe712a00700 1 -- 172.21.15.83:0/1266234513 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe70c01a720 con 0x7fe71c108700 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.835+0000 7fe720c00700 1 -- 172.21.15.83:0/1266234513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c19d290 con 0x7fe71c108700 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.836+0000 7fe712a00700 1 -- 172.21.15.83:0/1266234513 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe70c024aa0 con 0x7fe71c108700 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.836+0000 7fe720c00700 1 -- 172.21.15.83:0/1266234513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe71c064150 con 0x7fe71c108700 2024-04-23T14:38:07.873 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.837+0000 7fe712a00700 1 --2- 172.21.15.83:0/1266234513 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe700039650 0x7fe70003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:07.874 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.837+0000 7fe712a00700 1 -- 172.21.15.83:0/1266234513 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe70c04dec0 con 0x7fe71c108700 2024-04-23T14:38:07.874 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.837+0000 7fe719400700 1 --2- 172.21.15.83:0/1266234513 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe700039650 0x7fe70003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:07.874 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.838+0000 7fe719400700 1 --2- 172.21.15.83:0/1266234513 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe700039650 0x7fe70003bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe708006fd0 tx=0x7fe708006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:07.874 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:06.843+0000 7fe712a00700 1 -- 172.21.15.83:0/1266234513 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe70c011e20 con 0x7fe71c108700 2024-04-23T14:38:07.874 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:07.182+0000 7fe720c00700 1 -- 172.21.15.83:0/1266234513 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe71c19fc60 con 0x7fe71c108700 2024-04-23T14:38:07.874 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:07.185+0000 7fe712a00700 1 -- 172.21.15.83:0/1266234513 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fe70c022330 con 0x7fe71c108700 2024-04-23T14:38:07.874 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:07.187+0000 7fe720c00700 1 -- 172.21.15.83:0/1266234513 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe700039650 msgr2=0x7fe70003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:07.874 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:07.187+0000 7fe720c00700 1 --2- 172.21.15.83:0/1266234513 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe700039650 0x7fe70003bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe708006fd0 tx=0x7fe708006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:07.874 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:07.187+0000 7fe720c00700 1 -- 172.21.15.83:0/1266234513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 msgr2=0x7fe71c19c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:07.874 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:07.187+0000 7fe720c00700 1 --2- 172.21.15.83:0/1266234513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 0x7fe71c19c660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0116a0 tx=0x7fe70c011ba0 comp rx=0 tx=0).stop 2024-04-23T14:38:07.874 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:07.187+0000 7fe720c00700 1 -- 172.21.15.83:0/1266234513 shutdown_connections 2024-04-23T14:38:07.874 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:07.187+0000 7fe720c00700 1 --2- 172.21.15.83:0/1266234513 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe700039650 0x7fe70003bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:07.875 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:07.187+0000 7fe720c00700 1 --2- 172.21.15.83:0/1266234513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe71c108700 0x7fe71c19c660 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:07.187+0000 7fe720c00700 1 -- 172.21.15.83:0/1266234513 >> 172.21.15.83:0/1266234513 conn(0x7fe71c103d40 msgr2=0x7fe71c104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:07.188+0000 7fe720c00700 1 -- 172.21.15.83:0/1266234513 shutdown_connections 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr 2024-04-23T14:38:07.188+0000 7fe720c00700 1 -- 172.21.15.83:0/1266234513 wait complete. 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout:Or, if you are only running a single cluster on this host: 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-23T14:38:07.876 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:38:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: ceph telemetry on 2024-04-23T14:38:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:38:07.877 INFO:teuthology.orchestra.run.smithi083.stdout:For more information see: 2024-04-23T14:38:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:38:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-23T14:38:07.877 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:38:07.877 INFO:teuthology.orchestra.run.smithi083.stdout:Bootstrap complete. 2024-04-23T14:38:07.983 INFO:tasks.cephadm:Fetching config... 2024-04-23T14:38:07.983 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:38:07.983 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-23T14:38:08.042 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-23T14:38:08.042 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:38:08.042 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-23T14:38:08.085 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-23T14:38:08.085 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:38:08.085 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-23T14:38:08.198 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-23T14:38:08.198 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:38:08.198 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-23T14:38:08.289 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-23T14:38:08.290 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJGG75I/Qdtsgz+a1mxtiBgJ0YdzfyhCpNowEP92NvPT3oCpKRWmLlreR/0csmAcAlAdUgrsGpYgT3BLs87dmifRI1fxyO1i5KHZP94L46BjSNDg0dcxyaM8yOaJf6BYZfoICSTLAvc2XjGnYfsGwUClahL1VSX4wKDW92l+Lohm26NtaY6yGbO7gvKhtiR95LxiQEbRe7+J2mAvYc67t8uLPuZBuwoSnoFtv+bKAk29bDkzDtgjZ37KXwaTNG0XbACg3Hq0tlQjU0gl1ZJcsseOmT4N0t8/0PT1IHGf0HrDO+dXb2+lodItpntCLF9aPlYvBvOw8WbNPwJQjy6yi8jKfNxZKIdLmk0aH8xa09FpXZpLbYwGvAXX3a36tC58nO+QTqxp1091Ym5j7fw8hdWhLv4ludT3UxTMaZ67vaeH3kPr9KbXslRvdGZI3qZ3WAf6bXIZHEHjab7IrAwppa3vIyFOEAf3zOZ6+4btDQY0sx8TTH2zeNO2BToneKOeM= ceph-5f4fe708-017e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T14:38:08.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:08 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1266234513' entity='client.admin' 2024-04-23T14:38:08.444 INFO:teuthology.orchestra.run.smithi083.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJGG75I/Qdtsgz+a1mxtiBgJ0YdzfyhCpNowEP92NvPT3oCpKRWmLlreR/0csmAcAlAdUgrsGpYgT3BLs87dmifRI1fxyO1i5KHZP94L46BjSNDg0dcxyaM8yOaJf6BYZfoICSTLAvc2XjGnYfsGwUClahL1VSX4wKDW92l+Lohm26NtaY6yGbO7gvKhtiR95LxiQEbRe7+J2mAvYc67t8uLPuZBuwoSnoFtv+bKAk29bDkzDtgjZ37KXwaTNG0XbACg3Hq0tlQjU0gl1ZJcsseOmT4N0t8/0PT1IHGf0HrDO+dXb2+lodItpntCLF9aPlYvBvOw8WbNPwJQjy6yi8jKfNxZKIdLmk0aH8xa09FpXZpLbYwGvAXX3a36tC58nO+QTqxp1091Ym5j7fw8hdWhLv4ludT3UxTMaZ67vaeH3kPr9KbXslRvdGZI3qZ3WAf6bXIZHEHjab7IrAwppa3vIyFOEAf3zOZ6+4btDQY0sx8TTH2zeNO2BToneKOeM= ceph-5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:38:08.474 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJGG75I/Qdtsgz+a1mxtiBgJ0YdzfyhCpNowEP92NvPT3oCpKRWmLlreR/0csmAcAlAdUgrsGpYgT3BLs87dmifRI1fxyO1i5KHZP94L46BjSNDg0dcxyaM8yOaJf6BYZfoICSTLAvc2XjGnYfsGwUClahL1VSX4wKDW92l+Lohm26NtaY6yGbO7gvKhtiR95LxiQEbRe7+J2mAvYc67t8uLPuZBuwoSnoFtv+bKAk29bDkzDtgjZ37KXwaTNG0XbACg3Hq0tlQjU0gl1ZJcsseOmT4N0t8/0PT1IHGf0HrDO+dXb2+lodItpntCLF9aPlYvBvOw8WbNPwJQjy6yi8jKfNxZKIdLmk0aH8xa09FpXZpLbYwGvAXX3a36tC58nO+QTqxp1091Ym5j7fw8hdWhLv4ludT3UxTMaZ67vaeH3kPr9KbXslRvdGZI3qZ3WAf6bXIZHEHjab7IrAwppa3vIyFOEAf3zOZ6+4btDQY0sx8TTH2zeNO2BToneKOeM= ceph-5f4fe708-017e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T14:38:08.514 INFO:teuthology.orchestra.run.smithi173.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJGG75I/Qdtsgz+a1mxtiBgJ0YdzfyhCpNowEP92NvPT3oCpKRWmLlreR/0csmAcAlAdUgrsGpYgT3BLs87dmifRI1fxyO1i5KHZP94L46BjSNDg0dcxyaM8yOaJf6BYZfoICSTLAvc2XjGnYfsGwUClahL1VSX4wKDW92l+Lohm26NtaY6yGbO7gvKhtiR95LxiQEbRe7+J2mAvYc67t8uLPuZBuwoSnoFtv+bKAk29bDkzDtgjZ37KXwaTNG0XbACg3Hq0tlQjU0gl1ZJcsseOmT4N0t8/0PT1IHGf0HrDO+dXb2+lodItpntCLF9aPlYvBvOw8WbNPwJQjy6yi8jKfNxZKIdLmk0aH8xa09FpXZpLbYwGvAXX3a36tC58nO+QTqxp1091Ym5j7fw8hdWhLv4ludT3UxTMaZ67vaeH3kPr9KbXslRvdGZI3qZ3WAf6bXIZHEHjab7IrAwppa3vIyFOEAf3zOZ6+4btDQY0sx8TTH2zeNO2BToneKOeM= ceph-5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:38:08.526 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJGG75I/Qdtsgz+a1mxtiBgJ0YdzfyhCpNowEP92NvPT3oCpKRWmLlreR/0csmAcAlAdUgrsGpYgT3BLs87dmifRI1fxyO1i5KHZP94L46BjSNDg0dcxyaM8yOaJf6BYZfoICSTLAvc2XjGnYfsGwUClahL1VSX4wKDW92l+Lohm26NtaY6yGbO7gvKhtiR95LxiQEbRe7+J2mAvYc67t8uLPuZBuwoSnoFtv+bKAk29bDkzDtgjZ37KXwaTNG0XbACg3Hq0tlQjU0gl1ZJcsseOmT4N0t8/0PT1IHGf0HrDO+dXb2+lodItpntCLF9aPlYvBvOw8WbNPwJQjy6yi8jKfNxZKIdLmk0aH8xa09FpXZpLbYwGvAXX3a36tC58nO+QTqxp1091Ym5j7fw8hdWhLv4ludT3UxTMaZ67vaeH3kPr9KbXslRvdGZI3qZ3WAf6bXIZHEHjab7IrAwppa3vIyFOEAf3zOZ6+4btDQY0sx8TTH2zeNO2BToneKOeM= ceph-5f4fe708-017e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T14:38:08.568 INFO:teuthology.orchestra.run.smithi203.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJGG75I/Qdtsgz+a1mxtiBgJ0YdzfyhCpNowEP92NvPT3oCpKRWmLlreR/0csmAcAlAdUgrsGpYgT3BLs87dmifRI1fxyO1i5KHZP94L46BjSNDg0dcxyaM8yOaJf6BYZfoICSTLAvc2XjGnYfsGwUClahL1VSX4wKDW92l+Lohm26NtaY6yGbO7gvKhtiR95LxiQEbRe7+J2mAvYc67t8uLPuZBuwoSnoFtv+bKAk29bDkzDtgjZ37KXwaTNG0XbACg3Hq0tlQjU0gl1ZJcsseOmT4N0t8/0PT1IHGf0HrDO+dXb2+lodItpntCLF9aPlYvBvOw8WbNPwJQjy6yi8jKfNxZKIdLmk0aH8xa09FpXZpLbYwGvAXX3a36tC58nO+QTqxp1091Ym5j7fw8hdWhLv4ludT3UxTMaZ67vaeH3kPr9KbXslRvdGZI3qZ3WAf6bXIZHEHjab7IrAwppa3vIyFOEAf3zOZ6+4btDQY0sx8TTH2zeNO2BToneKOeM= ceph-5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:38:08.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-23T14:38:08.998 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:38:11.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.170+0000 7f0e2ec00700 1 -- 172.21.15.83:0/2701858132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e28108cb0 msgr2=0x7f0e281090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:11.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.170+0000 7f0e2ec00700 1 --2- 172.21.15.83:0/2701858132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e28108cb0 0x7f0e281090d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009910 tx=0x7f0e1c009c20 comp rx=0 tx=0).stop 2024-04-23T14:38:11.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.170+0000 7f0e2ec00700 1 -- 172.21.15.83:0/2701858132 shutdown_connections 2024-04-23T14:38:11.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.170+0000 7f0e2ec00700 1 --2- 172.21.15.83:0/2701858132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e28108cb0 0x7f0e281090d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:11.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.170+0000 7f0e2ec00700 1 -- 172.21.15.83:0/2701858132 >> 172.21.15.83:0/2701858132 conn(0x7f0e28077b10 msgr2=0x7f0e28079f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:11.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.170+0000 7f0e2ec00700 1 -- 172.21.15.83:0/2701858132 shutdown_connections 2024-04-23T14:38:11.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.170+0000 7f0e2ec00700 1 -- 172.21.15.83:0/2701858132 wait complete. 2024-04-23T14:38:11.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.171+0000 7f0e2ec00700 1 Processor -- start 2024-04-23T14:38:11.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.172+0000 7f0e2ec00700 1 -- start start 2024-04-23T14:38:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.172+0000 7f0e2ec00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e28108cb0 0x7f0e2819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.172+0000 7f0e2ec00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2810e9d0 con 0x7f0e28108cb0 2024-04-23T14:38:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.172+0000 7f0e2d800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e28108cb0 0x7f0e2819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.172+0000 7f0e2d800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e28108cb0 0x7f0e2819e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44422/0 (socket says 172.21.15.83:44422) 2024-04-23T14:38:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.172+0000 7f0e2d800700 1 -- 172.21.15.83:0/570039786 learned_addr learned my addr 172.21.15.83:0/570039786 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.173+0000 7f0e2d800700 1 -- 172.21.15.83:0/570039786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2819e5a0 con 0x7f0e28108cb0 2024-04-23T14:38:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.173+0000 7f0e2d800700 1 --2- 172.21.15.83:0/570039786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e28108cb0 0x7f0e2819e060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c005950 tx=0x7f0e1c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.173+0000 7f0e26000700 1 -- 172.21.15.83:0/570039786 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e1c026070 con 0x7f0e28108cb0 2024-04-23T14:38:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.173+0000 7f0e26000700 1 -- 172.21.15.83:0/570039786 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e1c00bc30 con 0x7f0e28108cb0 2024-04-23T14:38:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.173+0000 7f0e26000700 1 -- 172.21.15.83:0/570039786 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e1c003b30 con 0x7f0e28108cb0 2024-04-23T14:38:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.173+0000 7f0e2ec00700 1 -- 172.21.15.83:0/570039786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2819e830 con 0x7f0e28108cb0 2024-04-23T14:38:11.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.173+0000 7f0e2ec00700 1 -- 172.21.15.83:0/570039786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e281a1520 con 0x7f0e28108cb0 2024-04-23T14:38:11.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.174+0000 7f0e26000700 1 -- 172.21.15.83:0/570039786 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0e1c029070 con 0x7f0e28108cb0 2024-04-23T14:38:11.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.174+0000 7f0e2ec00700 1 -- 172.21.15.83:0/570039786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e2810f150 con 0x7f0e28108cb0 2024-04-23T14:38:11.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.175+0000 7f0e26000700 1 --2- 172.21.15.83:0/570039786 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0e100396b0 0x7f0e1003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:11.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.175+0000 7f0e26000700 1 -- 172.21.15.83:0/570039786 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0e1c01e070 con 0x7f0e28108cb0 2024-04-23T14:38:11.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.175+0000 7f0e2ce00700 1 --2- 172.21.15.83:0/570039786 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0e100396b0 0x7f0e1003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:11.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.176+0000 7f0e2ce00700 1 --2- 172.21.15.83:0/570039786 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0e100396b0 0x7f0e1003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0e14006fd0 tx=0x7f0e14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:11.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.181+0000 7f0e26000700 1 -- 172.21.15.83:0/570039786 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e1c013070 con 0x7f0e28108cb0 2024-04-23T14:38:11.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.409+0000 7f0e2ec00700 1 -- 172.21.15.83:0/570039786 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f0e2802e0f0 con 0x7f0e28108cb0 2024-04-23T14:38:11.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.418+0000 7f0e26000700 1 -- 172.21.15.83:0/570039786 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f0e1c0300c0 con 0x7f0e28108cb0 2024-04-23T14:38:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.421+0000 7f0e2ec00700 1 -- 172.21.15.83:0/570039786 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0e100396b0 msgr2=0x7f0e1003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.421+0000 7f0e2ec00700 1 --2- 172.21.15.83:0/570039786 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0e100396b0 0x7f0e1003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0e14006fd0 tx=0x7f0e14006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.421+0000 7f0e2ec00700 1 -- 172.21.15.83:0/570039786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e28108cb0 msgr2=0x7f0e2819e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.421+0000 7f0e2ec00700 1 --2- 172.21.15.83:0/570039786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e28108cb0 0x7f0e2819e060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c005950 tx=0x7f0e1c005030 comp rx=0 tx=0).stop 2024-04-23T14:38:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.421+0000 7f0e2ec00700 1 -- 172.21.15.83:0/570039786 shutdown_connections 2024-04-23T14:38:11.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.421+0000 7f0e2ec00700 1 --2- 172.21.15.83:0/570039786 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0e100396b0 0x7f0e1003bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.421+0000 7f0e2ec00700 1 --2- 172.21.15.83:0/570039786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e28108cb0 0x7f0e2819e060 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.421+0000 7f0e2ec00700 1 -- 172.21.15.83:0/570039786 >> 172.21.15.83:0/570039786 conn(0x7f0e28077b10 msgr2=0x7f0e28078520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.422+0000 7f0e2ec00700 1 -- 172.21.15.83:0/570039786 shutdown_connections 2024-04-23T14:38:11.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:11.422+0000 7f0e2ec00700 1 -- 172.21.15.83:0/570039786 wait complete. 2024-04-23T14:38:12.508 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-23T14:38:12.509 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-23T14:38:12.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:12.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:12 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/570039786' entity='client.admin' 2024-04-23T14:38:12.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:12.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:38:12.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:12.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:38:12.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:12.919 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:38:15.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.399+0000 7f27d5600700 1 -- 172.21.15.83:0/2848879170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27d01047a0 msgr2=0x7f27d0104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.399+0000 7f27d5600700 1 --2- 172.21.15.83:0/2848879170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27d01047a0 0x7f27d0104bc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009910 tx=0x7f27c0009c20 comp rx=0 tx=0).stop 2024-04-23T14:38:15.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.399+0000 7f27d5600700 1 -- 172.21.15.83:0/2848879170 shutdown_connections 2024-04-23T14:38:15.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.399+0000 7f27d5600700 1 --2- 172.21.15.83:0/2848879170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27d01047a0 0x7f27d0104bc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.399+0000 7f27d5600700 1 -- 172.21.15.83:0/2848879170 >> 172.21.15.83:0/2848879170 conn(0x7f27d00ffd20 msgr2=0x7f27d0102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:15.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.399+0000 7f27d5600700 1 -- 172.21.15.83:0/2848879170 shutdown_connections 2024-04-23T14:38:15.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.399+0000 7f27d5600700 1 -- 172.21.15.83:0/2848879170 wait complete. 2024-04-23T14:38:15.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.400+0000 7f27d5600700 1 Processor -- start 2024-04-23T14:38:15.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.401+0000 7f27d5600700 1 -- start start 2024-04-23T14:38:15.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.401+0000 7f27d5600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27d01047a0 0x7f27d0199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.401+0000 7f27d5600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f27d010a830 con 0x7f27d01047a0 2024-04-23T14:38:15.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.401+0000 7f27cea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27d01047a0 0x7f27d0199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.401+0000 7f27cea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27d01047a0 0x7f27d0199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56810/0 (socket says 172.21.15.83:56810) 2024-04-23T14:38:15.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.401+0000 7f27cea00700 1 -- 172.21.15.83:0/2362241781 learned_addr learned my addr 172.21.15.83:0/2362241781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:15.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.402+0000 7f27cea00700 1 -- 172.21.15.83:0/2362241781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d019a3b0 con 0x7f27d01047a0 2024-04-23T14:38:15.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.402+0000 7f27cea00700 1 --2- 172.21.15.83:0/2362241781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27d01047a0 0x7f27d0199e70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005850 tx=0x7f27c0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:15.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.402+0000 7f27c7400700 1 -- 172.21.15.83:0/2362241781 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27c0026070 con 0x7f27d01047a0 2024-04-23T14:38:15.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.402+0000 7f27c7400700 1 -- 172.21.15.83:0/2362241781 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c00044a0 con 0x7f27d01047a0 2024-04-23T14:38:15.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.402+0000 7f27d5600700 1 -- 172.21.15.83:0/2362241781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d019a640 con 0x7f27d01047a0 2024-04-23T14:38:15.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.402+0000 7f27d5600700 1 -- 172.21.15.83:0/2362241781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d019d330 con 0x7f27d01047a0 2024-04-23T14:38:15.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.403+0000 7f27c7400700 1 -- 172.21.15.83:0/2362241781 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27c0004830 con 0x7f27d01047a0 2024-04-23T14:38:15.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.404+0000 7f27d5600700 1 -- 172.21.15.83:0/2362241781 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d010afb0 con 0x7f27d01047a0 2024-04-23T14:38:15.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.404+0000 7f27c7400700 1 -- 172.21.15.83:0/2362241781 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f27c0003920 con 0x7f27d01047a0 2024-04-23T14:38:15.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.405+0000 7f27c7400700 1 --2- 172.21.15.83:0/2362241781 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f27b40397f0 0x7f27b403bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:15.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.405+0000 7f27c7400700 1 -- 172.21.15.83:0/2362241781 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27c001e070 con 0x7f27d01047a0 2024-04-23T14:38:15.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.405+0000 7f27ce000700 1 --2- 172.21.15.83:0/2362241781 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f27b40397f0 0x7f27b403bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:15.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.406+0000 7f27ce000700 1 --2- 172.21.15.83:0/2362241781 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f27b40397f0 0x7f27b403bcb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f27bc006fd0 tx=0x7f27bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:15.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.410+0000 7f27c7400700 1 -- 172.21.15.83:0/2362241781 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27c0013070 con 0x7f27d01047a0 2024-04-23T14:38:15.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.637+0000 7f27d5600700 1 -- 172.21.15.83:0/2362241781 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f27d002e0e0 con 0x7f27b40397f0 2024-04-23T14:38:15.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.646+0000 7f27c7400700 1 -- 172.21.15.83:0/2362241781 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f27d002e0e0 con 0x7f27b40397f0 2024-04-23T14:38:15.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.649+0000 7f27d5600700 1 -- 172.21.15.83:0/2362241781 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f27b40397f0 msgr2=0x7f27b403bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.649+0000 7f27d5600700 1 --2- 172.21.15.83:0/2362241781 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f27b40397f0 0x7f27b403bcb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f27bc006fd0 tx=0x7f27bc006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.649+0000 7f27d5600700 1 -- 172.21.15.83:0/2362241781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27d01047a0 msgr2=0x7f27d0199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.649+0000 7f27d5600700 1 --2- 172.21.15.83:0/2362241781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27d01047a0 0x7f27d0199e70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005850 tx=0x7f27c0005030 comp rx=0 tx=0).stop 2024-04-23T14:38:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.649+0000 7f27d5600700 1 -- 172.21.15.83:0/2362241781 shutdown_connections 2024-04-23T14:38:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.649+0000 7f27d5600700 1 --2- 172.21.15.83:0/2362241781 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f27b40397f0 0x7f27b403bcb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.649+0000 7f27d5600700 1 --2- 172.21.15.83:0/2362241781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27d01047a0 0x7f27d0199e70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.649+0000 7f27d5600700 1 -- 172.21.15.83:0/2362241781 >> 172.21.15.83:0/2362241781 conn(0x7f27d00ffd20 msgr2=0x7f27d0100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.649+0000 7f27d5600700 1 -- 172.21.15.83:0/2362241781 shutdown_connections 2024-04-23T14:38:15.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:15.649+0000 7f27d5600700 1 -- 172.21.15.83:0/2362241781 wait complete. 2024-04-23T14:38:16.422 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi173 2024-04-23T14:38:16.422 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T14:38:16.422 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.conf 2024-04-23T14:38:16.440 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T14:38:16.440 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:38:16.497 INFO:tasks.cephadm:Adding host smithi173 to orchestrator... 2024-04-23T14:38:16.498 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch host add smithi173 2024-04-23T14:38:16.900 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:38:17.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:16 smithi083 ceph-mon[27867]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:38:17.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:16 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:17.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:16 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:38:17.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:16 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:38:17.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:16 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:38:17.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:16 smithi083 ceph-mon[27867]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-23T14:38:17.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:16 smithi083 ceph-mon[27867]: Updating smithi083:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:38:18.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:17 smithi083 ceph-mon[27867]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:38:18.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:17 smithi083 ceph-mon[27867]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:19.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.013+0000 7f4343e00700 1 -- 172.21.15.83:0/1776845394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434410a620 msgr2=0x7f434410aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:19.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.013+0000 7f4343e00700 1 --2- 172.21.15.83:0/1776845394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434410a620 0x7f434410aa00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4334009910 tx=0x7f4334009c20 comp rx=0 tx=0).stop 2024-04-23T14:38:19.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.013+0000 7f4343e00700 1 -- 172.21.15.83:0/1776845394 shutdown_connections 2024-04-23T14:38:19.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.013+0000 7f4343e00700 1 --2- 172.21.15.83:0/1776845394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434410a620 0x7f434410aa00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.013+0000 7f4343e00700 1 -- 172.21.15.83:0/1776845394 >> 172.21.15.83:0/1776845394 conn(0x7f4344100020 msgr2=0x7f4344102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.013+0000 7f4343e00700 1 -- 172.21.15.83:0/1776845394 shutdown_connections 2024-04-23T14:38:19.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.014+0000 7f4343e00700 1 -- 172.21.15.83:0/1776845394 wait complete. 2024-04-23T14:38:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.014+0000 7f4343e00700 1 Processor -- start 2024-04-23T14:38:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.015+0000 7f4343e00700 1 -- start start 2024-04-23T14:38:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.015+0000 7f4343e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434410a620 0x7f434419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.015+0000 7f4343e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f434410ebe0 con 0x7f434410a620 2024-04-23T14:38:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.015+0000 7f4342a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434410a620 0x7f434419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.015+0000 7f4342a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434410a620 0x7f434419a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56820/0 (socket says 172.21.15.83:56820) 2024-04-23T14:38:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.015+0000 7f4342a00700 1 -- 172.21.15.83:0/4107964672 learned_addr learned my addr 172.21.15.83:0/4107964672 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:19.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.016+0000 7f4342a00700 1 -- 172.21.15.83:0/4107964672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434419a540 con 0x7f434410a620 2024-04-23T14:38:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.016+0000 7f4342a00700 1 --2- 172.21.15.83:0/4107964672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434410a620 0x7f434419a000 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4334005850 tx=0x7f4334005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.016+0000 7f433b400700 1 -- 172.21.15.83:0/4107964672 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4334026070 con 0x7f434410a620 2024-04-23T14:38:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.016+0000 7f433b400700 1 -- 172.21.15.83:0/4107964672 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f433400bc30 con 0x7f434410a620 2024-04-23T14:38:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.016+0000 7f433b400700 1 -- 172.21.15.83:0/4107964672 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4334003a20 con 0x7f434410a620 2024-04-23T14:38:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.016+0000 7f4343e00700 1 -- 172.21.15.83:0/4107964672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434419a7d0 con 0x7f434410a620 2024-04-23T14:38:19.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.017+0000 7f4343e00700 1 -- 172.21.15.83:0/4107964672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434419e8d0 con 0x7f434410a620 2024-04-23T14:38:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.018+0000 7f4343e00700 1 -- 172.21.15.83:0/4107964672 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4344105c90 con 0x7f434410a620 2024-04-23T14:38:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.018+0000 7f433b400700 1 -- 172.21.15.83:0/4107964672 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4334029070 con 0x7f434410a620 2024-04-23T14:38:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.018+0000 7f433b400700 1 --2- 172.21.15.83:0/4107964672 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f43280396b0 0x7f432803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:19.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.018+0000 7f433b400700 1 -- 172.21.15.83:0/4107964672 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f433401e070 con 0x7f434410a620 2024-04-23T14:38:19.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.019+0000 7f4342000700 1 --2- 172.21.15.83:0/4107964672 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f43280396b0 0x7f432803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:19.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.019+0000 7f4342000700 1 --2- 172.21.15.83:0/4107964672 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f43280396b0 0x7f432803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4330006fd0 tx=0x7f4330006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:19.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.025+0000 7f433b400700 1 -- 172.21.15.83:0/4107964672 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4334013070 con 0x7f434410a620 2024-04-23T14:38:19.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:19 smithi083 ceph-mon[27867]: Updating smithi083:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:38:19.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:19.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:19.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:19.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:19.256+0000 7f4343e00700 1 -- 172.21.15.83:0/4107964672 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}) -- 0x7f434419b360 con 0x7f43280396b0 2024-04-23T14:38:20.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:20 smithi083 ceph-mon[27867]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:20.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:20 smithi083 ceph-mon[27867]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:38:21.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:21 smithi083 ceph-mon[27867]: Deploying cephadm binary to smithi173 2024-04-23T14:38:22.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:22 smithi083 ceph-mon[27867]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:23.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:23.995+0000 7f433b400700 1 -- 172.21.15.83:0/4107964672 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f434419b360 con 0x7f43280396b0 2024-04-23T14:38:23.996 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi173' with addr '172.21.15.173' 2024-04-23T14:38:23.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:23.999+0000 7f4343e00700 1 -- 172.21.15.83:0/4107964672 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f43280396b0 msgr2=0x7f432803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:23.999+0000 7f4343e00700 1 --2- 172.21.15.83:0/4107964672 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f43280396b0 0x7f432803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4330006fd0 tx=0x7f4330006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:24.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:23.999+0000 7f4343e00700 1 -- 172.21.15.83:0/4107964672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434410a620 msgr2=0x7f434419a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:24.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:23.999+0000 7f4343e00700 1 --2- 172.21.15.83:0/4107964672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434410a620 0x7f434419a000 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4334005850 tx=0x7f4334005030 comp rx=0 tx=0).stop 2024-04-23T14:38:24.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:23.999+0000 7f4343e00700 1 -- 172.21.15.83:0/4107964672 shutdown_connections 2024-04-23T14:38:24.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:23.999+0000 7f4343e00700 1 --2- 172.21.15.83:0/4107964672 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f43280396b0 0x7f432803bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:23.999+0000 7f4343e00700 1 --2- 172.21.15.83:0/4107964672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434410a620 0x7f434419a000 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:24.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:23.999+0000 7f4343e00700 1 -- 172.21.15.83:0/4107964672 >> 172.21.15.83:0/4107964672 conn(0x7f4344100020 msgr2=0x7f4344100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:24.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:24.000+0000 7f4343e00700 1 -- 172.21.15.83:0/4107964672 shutdown_connections 2024-04-23T14:38:24.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:24.000+0000 7f4343e00700 1 -- 172.21.15.83:0/4107964672 wait complete. 2024-04-23T14:38:24.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:24 smithi083 ceph-mon[27867]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:24.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:24.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:38:24.729 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T14:38:25.160 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:38:25.442 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:25 smithi083 ceph-mon[27867]: Added host smithi173 2024-04-23T14:38:25.442 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:25 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:25.442 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:25 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:26.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:26 smithi083 ceph-mon[27867]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:27.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.143+0000 7f7f9d800700 1 -- 172.21.15.83:0/732176230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f981085d0 msgr2=0x7f7f981089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.143+0000 7f7f9d800700 1 --2- 172.21.15.83:0/732176230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f981085d0 0x7f7f981089b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009910 tx=0x7f7f88009c20 comp rx=0 tx=0).stop 2024-04-23T14:38:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.143+0000 7f7f9d800700 1 -- 172.21.15.83:0/732176230 shutdown_connections 2024-04-23T14:38:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.143+0000 7f7f9d800700 1 --2- 172.21.15.83:0/732176230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f981085d0 0x7f7f981089b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.143+0000 7f7f9d800700 1 -- 172.21.15.83:0/732176230 >> 172.21.15.83:0/732176230 conn(0x7f7f980fe030 msgr2=0x7f7f98100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.143+0000 7f7f9d800700 1 -- 172.21.15.83:0/732176230 shutdown_connections 2024-04-23T14:38:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.143+0000 7f7f9d800700 1 -- 172.21.15.83:0/732176230 wait complete. 2024-04-23T14:38:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.144+0000 7f7f9d800700 1 Processor -- start 2024-04-23T14:38:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.144+0000 7f7f9d800700 1 -- start start 2024-04-23T14:38:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.145+0000 7f7f9d800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f981085d0 0x7f7f98074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:27.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.145+0000 7f7f9d800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9810cb90 con 0x7f7f981085d0 2024-04-23T14:38:27.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.145+0000 7f7f96a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f981085d0 0x7f7f98074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.145+0000 7f7f96a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f981085d0 0x7f7f98074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52532/0 (socket says 172.21.15.83:52532) 2024-04-23T14:38:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.145+0000 7f7f96a00700 1 -- 172.21.15.83:0/2995027030 learned_addr learned my addr 172.21.15.83:0/2995027030 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.145+0000 7f7f96a00700 1 -- 172.21.15.83:0/2995027030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f98076ce0 con 0x7f7f981085d0 2024-04-23T14:38:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.146+0000 7f7f96a00700 1 --2- 172.21.15.83:0/2995027030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f981085d0 0x7f7f98074df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7f88005850 tx=0x7f7f88005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.146+0000 7f7f8f400700 1 -- 172.21.15.83:0/2995027030 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f88026070 con 0x7f7f981085d0 2024-04-23T14:38:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.146+0000 7f7f8f400700 1 -- 172.21.15.83:0/2995027030 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f8800bc30 con 0x7f7f981085d0 2024-04-23T14:38:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.146+0000 7f7f8f400700 1 -- 172.21.15.83:0/2995027030 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f88003a20 con 0x7f7f981085d0 2024-04-23T14:38:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.146+0000 7f7f9d800700 1 -- 172.21.15.83:0/2995027030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f98075330 con 0x7f7f981085d0 2024-04-23T14:38:27.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.146+0000 7f7f9d800700 1 -- 172.21.15.83:0/2995027030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f980757f0 con 0x7f7f981085d0 2024-04-23T14:38:27.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.147+0000 7f7f8f400700 1 -- 172.21.15.83:0/2995027030 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7f88029070 con 0x7f7f981085d0 2024-04-23T14:38:27.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.147+0000 7f7f9d800700 1 -- 172.21.15.83:0/2995027030 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f98103c40 con 0x7f7f981085d0 2024-04-23T14:38:27.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.148+0000 7f7f8f400700 1 --2- 172.21.15.83:0/2995027030 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f7c0396b0 0x7f7f7c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:27.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.148+0000 7f7f8f400700 1 -- 172.21.15.83:0/2995027030 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7f8801e070 con 0x7f7f981085d0 2024-04-23T14:38:27.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.148+0000 7f7f96000700 1 --2- 172.21.15.83:0/2995027030 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f7c0396b0 0x7f7f7c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:27.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.149+0000 7f7f96000700 1 --2- 172.21.15.83:0/2995027030 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f7c0396b0 0x7f7f7c03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7f84006fd0 tx=0x7f7f84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:27.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.155+0000 7f7f8f400700 1 -- 172.21.15.83:0/2995027030 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f88013070 con 0x7f7f981085d0 2024-04-23T14:38:27.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.392+0000 7f7f9d800700 1 -- 172.21.15.83:0/2995027030 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f98076390 con 0x7f7f7c0396b0 2024-04-23T14:38:27.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.394+0000 7f7f8f400700 1 -- 172.21.15.83:0/2995027030 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f7f98076390 con 0x7f7f7c0396b0 2024-04-23T14:38:27.395 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:38:27.395 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.173", "hostname": "smithi173", "labels": [], "status": ""}] 2024-04-23T14:38:27.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.397+0000 7f7f9d800700 1 -- 172.21.15.83:0/2995027030 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f7c0396b0 msgr2=0x7f7f7c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:27.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.397+0000 7f7f9d800700 1 --2- 172.21.15.83:0/2995027030 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f7c0396b0 0x7f7f7c03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7f84006fd0 tx=0x7f7f84006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:27.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.397+0000 7f7f9d800700 1 -- 172.21.15.83:0/2995027030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f981085d0 msgr2=0x7f7f98074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:27.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.397+0000 7f7f9d800700 1 --2- 172.21.15.83:0/2995027030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f981085d0 0x7f7f98074df0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7f88005850 tx=0x7f7f88005030 comp rx=0 tx=0).stop 2024-04-23T14:38:27.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.397+0000 7f7f9d800700 1 -- 172.21.15.83:0/2995027030 shutdown_connections 2024-04-23T14:38:27.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.397+0000 7f7f9d800700 1 --2- 172.21.15.83:0/2995027030 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f7c0396b0 0x7f7f7c03bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.397+0000 7f7f9d800700 1 --2- 172.21.15.83:0/2995027030 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f981085d0 0x7f7f98074df0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:27.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.397+0000 7f7f9d800700 1 -- 172.21.15.83:0/2995027030 >> 172.21.15.83:0/2995027030 conn(0x7f7f980fe030 msgr2=0x7f7f980ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:27.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.398+0000 7f7f9d800700 1 -- 172.21.15.83:0/2995027030 shutdown_connections 2024-04-23T14:38:27.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:27.398+0000 7f7f9d800700 1 -- 172.21.15.83:0/2995027030 wait complete. 2024-04-23T14:38:28.091 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:27 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:28.091 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:27 smithi083 ceph-mon[27867]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:28.139 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi203 2024-04-23T14:38:28.139 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-23T14:38:28.139 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.conf 2024-04-23T14:38:28.157 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-23T14:38:28.157 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:38:28.214 INFO:tasks.cephadm:Adding host smithi203 to orchestrator... 2024-04-23T14:38:28.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch host add smithi203 2024-04-23T14:38:28.622 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:38:29.013 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:28 smithi083 ceph-mon[27867]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:38:29.870 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:29 smithi083 ceph-mon[27867]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:30.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.678+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1188694774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d68074e90 msgr2=0x7f6d68075270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:30.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.678+0000 7f6d6fe00700 1 --2- 172.21.15.83:0/1188694774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d68074e90 0x7f6d68075270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00b0a0 tx=0x7f6d5c00b3b0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.678+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1188694774 shutdown_connections 2024-04-23T14:38:30.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.678+0000 7f6d6fe00700 1 --2- 172.21.15.83:0/1188694774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d68074e90 0x7f6d68075270 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:30.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.678+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1188694774 >> 172.21.15.83:0/1188694774 conn(0x7f6d680fe030 msgr2=0x7f6d68100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:30.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.678+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1188694774 shutdown_connections 2024-04-23T14:38:30.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.679+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1188694774 wait complete. 2024-04-23T14:38:30.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.680+0000 7f6d6fe00700 1 Processor -- start 2024-04-23T14:38:30.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.680+0000 7f6d6fe00700 1 -- start start 2024-04-23T14:38:30.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.680+0000 7f6d6fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d68074e90 0x7f6d6819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.680+0000 7f6d6fe00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d681131a0 con 0x7f6d68074e90 2024-04-23T14:38:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.680+0000 7f6d6d400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d68074e90 0x7f6d6819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.680+0000 7f6d6d400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d68074e90 0x7f6d6819e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52546/0 (socket says 172.21.15.83:52546) 2024-04-23T14:38:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.680+0000 7f6d6d400700 1 -- 172.21.15.83:0/1728140999 learned_addr learned my addr 172.21.15.83:0/1728140999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.681+0000 7f6d6d400700 1 -- 172.21.15.83:0/1728140999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6819eb10 con 0x7f6d68074e90 2024-04-23T14:38:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.681+0000 7f6d6d400700 1 --2- 172.21.15.83:0/1728140999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d68074e90 0x7f6d6819e5d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c000f80 tx=0x7f6d5c00bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.681+0000 7f6d66000700 1 -- 172.21.15.83:0/1728140999 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d5c020020 con 0x7f6d68074e90 2024-04-23T14:38:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.682+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1728140999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6819eda0 con 0x7f6d68074e90 2024-04-23T14:38:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.682+0000 7f6d66000700 1 -- 172.21.15.83:0/1728140999 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d5c018040 con 0x7f6d68074e90 2024-04-23T14:38:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.682+0000 7f6d66000700 1 -- 172.21.15.83:0/1728140999 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d5c017070 con 0x7f6d68074e90 2024-04-23T14:38:30.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.682+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1728140999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d681a2ea0 con 0x7f6d68074e90 2024-04-23T14:38:30.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.683+0000 7f6d66000700 1 -- 172.21.15.83:0/1728140999 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6d5c023020 con 0x7f6d68074e90 2024-04-23T14:38:30.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.683+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1728140999 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d680765d0 con 0x7f6d68074e90 2024-04-23T14:38:30.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.683+0000 7f6d66000700 1 --2- 172.21.15.83:0/1728140999 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6d500396f0 0x7f6d5003bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:30.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.684+0000 7f6d66000700 1 -- 172.21.15.83:0/1728140999 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6d5c00f050 con 0x7f6d68074e90 2024-04-23T14:38:30.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.684+0000 7f6d6ca00700 1 --2- 172.21.15.83:0/1728140999 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6d500396f0 0x7f6d5003bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:30.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.685+0000 7f6d6ca00700 1 --2- 172.21.15.83:0/1728140999 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6d500396f0 0x7f6d5003bbb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006fd0 tx=0x7f6d54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:30.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.689+0000 7f6d66000700 1 -- 172.21.15.83:0/1728140999 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d5c02a020 con 0x7f6d68074e90 2024-04-23T14:38:30.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:30.942+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1728140999 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}) -- 0x7f6d6819f930 con 0x7f6d500396f0 2024-04-23T14:38:31.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:38:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:38:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:38:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:30 smithi083 ceph-mon[27867]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T14:38:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:30 smithi083 ceph-mon[27867]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:38:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:32 smithi083 ceph-mon[27867]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:38:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:32 smithi083 ceph-mon[27867]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:38:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:32 smithi083 ceph-mon[27867]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:38:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:32 smithi083 ceph-mon[27867]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:32 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:32 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:32 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:33.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:33 smithi083 ceph-mon[27867]: Deploying cephadm binary to smithi203 2024-04-23T14:38:34.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:34 smithi083 ceph-mon[27867]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:35.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:35.711+0000 7f6d66000700 1 -- 172.21.15.83:0/1728140999 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6d6819f930 con 0x7f6d500396f0 2024-04-23T14:38:35.712 INFO:teuthology.orchestra.run.smithi083.stdout:Added host 'smithi203' with addr '172.21.15.203' 2024-04-23T14:38:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:35.715+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1728140999 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6d500396f0 msgr2=0x7f6d5003bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:35.715+0000 7f6d6fe00700 1 --2- 172.21.15.83:0/1728140999 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6d500396f0 0x7f6d5003bbb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006fd0 tx=0x7f6d54006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:35.715+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1728140999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d68074e90 msgr2=0x7f6d6819e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:35.715+0000 7f6d6fe00700 1 --2- 172.21.15.83:0/1728140999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d68074e90 0x7f6d6819e5d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c000f80 tx=0x7f6d5c00bd20 comp rx=0 tx=0).stop 2024-04-23T14:38:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:35.715+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1728140999 shutdown_connections 2024-04-23T14:38:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:35.715+0000 7f6d6fe00700 1 --2- 172.21.15.83:0/1728140999 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6d500396f0 0x7f6d5003bbb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:35.715+0000 7f6d6fe00700 1 --2- 172.21.15.83:0/1728140999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d68074e90 0x7f6d6819e5d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:35.715+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1728140999 >> 172.21.15.83:0/1728140999 conn(0x7f6d680fe030 msgr2=0x7f6d680ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:35.715+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1728140999 shutdown_connections 2024-04-23T14:38:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:35.715+0000 7f6d6fe00700 1 -- 172.21.15.83:0/1728140999 wait complete. 2024-04-23T14:38:36.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:36 smithi083 ceph-mon[27867]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:36.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:36 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:36.363 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:36 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:38:36.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T14:38:36.862 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:38:37.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:37 smithi083 ceph-mon[27867]: Added host smithi203 2024-04-23T14:38:37.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:37 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:37.300 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:37 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:38.344 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:38 smithi083 ceph-mon[27867]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:38.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.794+0000 7f0cae000700 1 -- 172.21.15.83:0/3545457370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca810a630 msgr2=0x7f0ca810aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:38.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.794+0000 7f0cae000700 1 --2- 172.21.15.83:0/3545457370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca810a630 0x7f0ca810aa10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009910 tx=0x7f0c9c009c20 comp rx=0 tx=0).stop 2024-04-23T14:38:38.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.794+0000 7f0cae000700 1 -- 172.21.15.83:0/3545457370 shutdown_connections 2024-04-23T14:38:38.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.794+0000 7f0cae000700 1 --2- 172.21.15.83:0/3545457370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca810a630 0x7f0ca810aa10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:38.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.794+0000 7f0cae000700 1 -- 172.21.15.83:0/3545457370 >> 172.21.15.83:0/3545457370 conn(0x7f0ca8100070 msgr2=0x7f0ca8102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:38.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.795+0000 7f0cae000700 1 -- 172.21.15.83:0/3545457370 shutdown_connections 2024-04-23T14:38:38.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.795+0000 7f0cae000700 1 -- 172.21.15.83:0/3545457370 wait complete. 2024-04-23T14:38:38.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.796+0000 7f0cae000700 1 Processor -- start 2024-04-23T14:38:38.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.796+0000 7f0cae000700 1 -- start start 2024-04-23T14:38:38.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.796+0000 7f0cae000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca810a630 0x7f0ca819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:38.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.796+0000 7f0cae000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca810ebf0 con 0x7f0ca810a630 2024-04-23T14:38:38.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.796+0000 7f0cacc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca810a630 0x7f0ca819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:38.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.797+0000 7f0cacc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca810a630 0x7f0ca819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40380/0 (socket says 172.21.15.83:40380) 2024-04-23T14:38:38.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.797+0000 7f0cacc00700 1 -- 172.21.15.83:0/2262428360 learned_addr learned my addr 172.21.15.83:0/2262428360 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.797+0000 7f0cacc00700 1 -- 172.21.15.83:0/2262428360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca819a570 con 0x7f0ca810a630 2024-04-23T14:38:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.797+0000 7f0cacc00700 1 --2- 172.21.15.83:0/2262428360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca810a630 0x7f0ca819a030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c006010 tx=0x7f0c9c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.797+0000 7f0ca5600700 1 -- 172.21.15.83:0/2262428360 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c9c026070 con 0x7f0ca810a630 2024-04-23T14:38:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.798+0000 7f0ca5600700 1 -- 172.21.15.83:0/2262428360 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c9c00bc30 con 0x7f0ca810a630 2024-04-23T14:38:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.798+0000 7f0ca5600700 1 -- 172.21.15.83:0/2262428360 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c9c0039d0 con 0x7f0ca810a630 2024-04-23T14:38:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.798+0000 7f0cae000700 1 -- 172.21.15.83:0/2262428360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca819a800 con 0x7f0ca810a630 2024-04-23T14:38:38.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.798+0000 7f0cae000700 1 -- 172.21.15.83:0/2262428360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca819e900 con 0x7f0ca810a630 2024-04-23T14:38:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.799+0000 7f0ca5600700 1 -- 172.21.15.83:0/2262428360 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0c9c029070 con 0x7f0ca810a630 2024-04-23T14:38:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.799+0000 7f0cae000700 1 -- 172.21.15.83:0/2262428360 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca8105ca0 con 0x7f0ca810a630 2024-04-23T14:38:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.799+0000 7f0ca5600700 1 --2- 172.21.15.83:0/2262428360 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c90039700 0x7f0c9003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.799+0000 7f0ca5600700 1 -- 172.21.15.83:0/2262428360 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0c9c01e070 con 0x7f0ca810a630 2024-04-23T14:38:38.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.800+0000 7f0ca7e00700 1 --2- 172.21.15.83:0/2262428360 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c90039700 0x7f0c9003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:38.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.800+0000 7f0ca7e00700 1 --2- 172.21.15.83:0/2262428360 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c90039700 0x7f0c9003bbc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c94006fd0 tx=0x7f0c94006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:38.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:38.805+0000 7f0ca5600700 1 -- 172.21.15.83:0/2262428360 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c9c013070 con 0x7f0ca810a630 2024-04-23T14:38:39.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:39.036+0000 7f0cae000700 1 -- 172.21.15.83:0/2262428360 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ca819b390 con 0x7f0c90039700 2024-04-23T14:38:39.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:39.039+0000 7f0ca5600700 1 -- 172.21.15.83:0/2262428360 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f0ca819b390 con 0x7f0c90039700 2024-04-23T14:38:39.040 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:38:39.040 INFO:teuthology.orchestra.run.smithi083.stdout:[{"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}, {"addr": "172.21.15.173", "hostname": "smithi173", "labels": [], "status": ""}, {"addr": "172.21.15.203", "hostname": "smithi203", "labels": [], "status": ""}] 2024-04-23T14:38:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:39.041+0000 7f0cae000700 1 -- 172.21.15.83:0/2262428360 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c90039700 msgr2=0x7f0c9003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:39.041+0000 7f0cae000700 1 --2- 172.21.15.83:0/2262428360 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c90039700 0x7f0c9003bbc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c94006fd0 tx=0x7f0c94006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:39.041+0000 7f0cae000700 1 -- 172.21.15.83:0/2262428360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca810a630 msgr2=0x7f0ca819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:39.041+0000 7f0cae000700 1 --2- 172.21.15.83:0/2262428360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca810a630 0x7f0ca819a030 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c006010 tx=0x7f0c9c005030 comp rx=0 tx=0).stop 2024-04-23T14:38:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:39.042+0000 7f0cae000700 1 -- 172.21.15.83:0/2262428360 shutdown_connections 2024-04-23T14:38:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:39.042+0000 7f0cae000700 1 --2- 172.21.15.83:0/2262428360 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c90039700 0x7f0c9003bbc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:39.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:39.042+0000 7f0cae000700 1 --2- 172.21.15.83:0/2262428360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ca810a630 0x7f0ca819a030 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:39.042+0000 7f0cae000700 1 -- 172.21.15.83:0/2262428360 >> 172.21.15.83:0/2262428360 conn(0x7f0ca8100070 msgr2=0x7f0ca81018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:39.042+0000 7f0cae000700 1 -- 172.21.15.83:0/2262428360 shutdown_connections 2024-04-23T14:38:39.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:39.042+0000 7f0cae000700 1 -- 172.21.15.83:0/2262428360 wait complete. 2024-04-23T14:38:39.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:39.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:39 smithi083 ceph-mon[27867]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:38:39.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:39 smithi083 ceph-mon[27867]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:39.753 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-23T14:38:39.753 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-23T14:38:40.178 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:38:42.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.198+0000 7f6be6000700 1 -- 172.21.15.83:0/2426465460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be0107430 msgr2=0x7f6be0069f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:42.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.198+0000 7f6be6000700 1 --2- 172.21.15.83:0/2426465460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be0107430 0x7f6be0069f10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009910 tx=0x7f6bd0009c20 comp rx=0 tx=0).stop 2024-04-23T14:38:42.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.198+0000 7f6be6000700 1 -- 172.21.15.83:0/2426465460 shutdown_connections 2024-04-23T14:38:42.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.198+0000 7f6be6000700 1 --2- 172.21.15.83:0/2426465460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be0107430 0x7f6be0069f10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:42.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.198+0000 7f6be6000700 1 -- 172.21.15.83:0/2426465460 >> 172.21.15.83:0/2426465460 conn(0x7f6be0076f50 msgr2=0x7f6be0077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:42.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.198+0000 7f6be6000700 1 -- 172.21.15.83:0/2426465460 shutdown_connections 2024-04-23T14:38:42.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.198+0000 7f6be6000700 1 -- 172.21.15.83:0/2426465460 wait complete. 2024-04-23T14:38:42.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.199+0000 7f6be6000700 1 Processor -- start 2024-04-23T14:38:42.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.200+0000 7f6be6000700 1 -- start start 2024-04-23T14:38:42.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.200+0000 7f6be6000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be0107430 0x7f6be019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:42.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.200+0000 7f6be6000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6be01131b0 con 0x7f6be0107430 2024-04-23T14:38:42.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.200+0000 7f6bdf400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be0107430 0x7f6be019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:42.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.200+0000 7f6bdf400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be0107430 0x7f6be019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:40386/0 (socket says 172.21.15.83:40386) 2024-04-23T14:38:42.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.200+0000 7f6bdf400700 1 -- 172.21.15.83:0/3443801362 learned_addr learned my addr 172.21.15.83:0/3443801362 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:38:42.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.201+0000 7f6bdf400700 1 -- 172.21.15.83:0/3443801362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be019ebb0 con 0x7f6be0107430 2024-04-23T14:38:42.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.201+0000 7f6bdf400700 1 --2- 172.21.15.83:0/3443801362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be0107430 0x7f6be019e670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0006010 tx=0x7f6bd0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:42.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.201+0000 7f6bdcc00700 1 -- 172.21.15.83:0/3443801362 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bd0026070 con 0x7f6be0107430 2024-04-23T14:38:42.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.201+0000 7f6bdcc00700 1 -- 172.21.15.83:0/3443801362 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bd000bc30 con 0x7f6be0107430 2024-04-23T14:38:42.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.201+0000 7f6bdcc00700 1 -- 172.21.15.83:0/3443801362 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bd00039d0 con 0x7f6be0107430 2024-04-23T14:38:42.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.201+0000 7f6be6000700 1 -- 172.21.15.83:0/3443801362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be019ee40 con 0x7f6be0107430 2024-04-23T14:38:42.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.201+0000 7f6be6000700 1 -- 172.21.15.83:0/3443801362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be019f2a0 con 0x7f6be0107430 2024-04-23T14:38:42.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.202+0000 7f6bdcc00700 1 -- 172.21.15.83:0/3443801362 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6bd0029070 con 0x7f6be0107430 2024-04-23T14:38:42.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.202+0000 7f6be6000700 1 -- 172.21.15.83:0/3443801362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be0101a20 con 0x7f6be0107430 2024-04-23T14:38:42.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.203+0000 7f6bdcc00700 1 --2- 172.21.15.83:0/3443801362 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6bc40396b0 0x7f6bc403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:42.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.203+0000 7f6bdcc00700 1 -- 172.21.15.83:0/3443801362 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6bd001e070 con 0x7f6be0107430 2024-04-23T14:38:42.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.203+0000 7f6bd7e00700 1 --2- 172.21.15.83:0/3443801362 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6bc40396b0 0x7f6bc403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:42.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.204+0000 7f6bd7e00700 1 --2- 172.21.15.83:0/3443801362 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6bc40396b0 0x7f6bc403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8006fd0 tx=0x7f6bc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:42.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.209+0000 7f6bdcc00700 1 -- 172.21.15.83:0/3443801362 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bd0013070 con 0x7f6be0107430 2024-04-23T14:38:42.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:42.443+0000 7f6be6000700 1 -- 172.21.15.83:0/3443801362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f6be019f9d0 con 0x7f6be0107430 2024-04-23T14:38:42.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:42 smithi083 ceph-mon[27867]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:42.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:42.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:42.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:42.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:42.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:38:42.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:38:42.445 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:38:43.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:43.142+0000 7f6bdcc00700 1 -- 172.21.15.83:0/3443801362 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f6bd000de00 con 0x7f6be0107430 2024-04-23T14:38:43.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:43.146+0000 7f6be6000700 1 -- 172.21.15.83:0/3443801362 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6bc40396b0 msgr2=0x7f6bc403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:43.146+0000 7f6be6000700 1 --2- 172.21.15.83:0/3443801362 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6bc40396b0 0x7f6bc403bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8006fd0 tx=0x7f6bc8006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:43.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:43.146+0000 7f6be6000700 1 -- 172.21.15.83:0/3443801362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be0107430 msgr2=0x7f6be019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:43.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:43.146+0000 7f6be6000700 1 --2- 172.21.15.83:0/3443801362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be0107430 0x7f6be019e670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0006010 tx=0x7f6bd0005030 comp rx=0 tx=0).stop 2024-04-23T14:38:43.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:43.146+0000 7f6be6000700 1 -- 172.21.15.83:0/3443801362 shutdown_connections 2024-04-23T14:38:43.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:43.147+0000 7f6be6000700 1 --2- 172.21.15.83:0/3443801362 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6bc40396b0 0x7f6bc403bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:43.147+0000 7f6be6000700 1 --2- 172.21.15.83:0/3443801362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6be0107430 0x7f6be019e670 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:43.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:43.147+0000 7f6be6000700 1 -- 172.21.15.83:0/3443801362 >> 172.21.15.83:0/3443801362 conn(0x7f6be0076f50 msgr2=0x7f6be00fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:43.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:43.147+0000 7f6be6000700 1 -- 172.21.15.83:0/3443801362 shutdown_connections 2024-04-23T14:38:43.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:38:43.147+0000 7f6be6000700 1 -- 172.21.15.83:0/3443801362 wait complete. 2024-04-23T14:38:43.150 INFO:teuthology.orchestra.run.smithi083.stderr:adjusted tunables profile to default 2024-04-23T14:38:43.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:43 smithi083 ceph-mon[27867]: Updating smithi203:/etc/ceph/ceph.conf 2024-04-23T14:38:43.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:43 smithi083 ceph-mon[27867]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:38:43.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:43 smithi083 ceph-mon[27867]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:38:43.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:43 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3443801362' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T14:38:43.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:43.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:43.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:43.918 INFO:tasks.cephadm:Adding mon.a on smithi083 2024-04-23T14:38:43.918 INFO:tasks.cephadm:Adding mon.b on smithi173 2024-04-23T14:38:43.918 INFO:tasks.cephadm:Adding mon.c on smithi203 2024-04-23T14:38:43.919 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi083:172.21.15.83=a;smithi173:172.21.15.173=b;smithi203:172.21.15.203=c' 2024-04-23T14:38:44.234 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:38:44.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:44 smithi083 ceph-mon[27867]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:38:44.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:44 smithi083 ceph-mon[27867]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:44.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:44 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3443801362' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T14:38:44.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:44 smithi083 ceph-mon[27867]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T14:38:45.590 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.589+0000 7f26a89ab700 1 -- 172.21.15.203:0/919157930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a41085d0 msgr2=0x7f26a41089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.590 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.589+0000 7f26a89ab700 1 --2- 172.21.15.203:0/919157930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a41085d0 0x7f26a41089b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2698009910 tx=0x7f2698009c20 comp rx=0 tx=0).stop 2024-04-23T14:38:45.590 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.589+0000 7f26a89ab700 1 -- 172.21.15.203:0/919157930 shutdown_connections 2024-04-23T14:38:45.591 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.589+0000 7f26a89ab700 1 --2- 172.21.15.203:0/919157930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a41085d0 0x7f26a41089b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.591 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.589+0000 7f26a89ab700 1 -- 172.21.15.203:0/919157930 >> 172.21.15.203:0/919157930 conn(0x7f26a40fe030 msgr2=0x7f26a4100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:45.591 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.589+0000 7f26a89ab700 1 -- 172.21.15.203:0/919157930 shutdown_connections 2024-04-23T14:38:45.591 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.589+0000 7f26a89ab700 1 -- 172.21.15.203:0/919157930 wait complete. 2024-04-23T14:38:45.591 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.590+0000 7f26a89ab700 1 Processor -- start 2024-04-23T14:38:45.591 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.590+0000 7f26a89ab700 1 -- start start 2024-04-23T14:38:45.591 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.590+0000 7f26a89ab700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a41085d0 0x7f26a4074df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.591 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.590+0000 7f26a89ab700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26a410cb90 con 0x7f26a41085d0 2024-04-23T14:38:45.592 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.590+0000 7f26a259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a41085d0 0x7f26a4074df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.592 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.591+0000 7f26a259c700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a41085d0 0x7f26a4074df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.203:34046/0 (socket says 172.21.15.203:34046) 2024-04-23T14:38:45.592 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.591+0000 7f26a259c700 1 -- 172.21.15.203:0/1683446032 learned_addr learned my addr 172.21.15.203:0/1683446032 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-23T14:38:45.592 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.591+0000 7f26a259c700 1 -- 172.21.15.203:0/1683446032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a4076ce0 con 0x7f26a41085d0 2024-04-23T14:38:45.592 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.592+0000 7f26a259c700 1 --2- 172.21.15.203:0/1683446032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a41085d0 0x7f26a4074df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2698005850 tx=0x7f2698005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:45.593 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.592+0000 7f26937fe700 1 -- 172.21.15.203:0/1683446032 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2698026070 con 0x7f26a41085d0 2024-04-23T14:38:45.593 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.592+0000 7f26a89ab700 1 -- 172.21.15.203:0/1683446032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a4075330 con 0x7f26a41085d0 2024-04-23T14:38:45.593 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.592+0000 7f26937fe700 1 -- 172.21.15.203:0/1683446032 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f269800bc30 con 0x7f26a41085d0 2024-04-23T14:38:45.593 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.592+0000 7f26a89ab700 1 -- 172.21.15.203:0/1683446032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a40757f0 con 0x7f26a41085d0 2024-04-23T14:38:45.593 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.592+0000 7f26937fe700 1 -- 172.21.15.203:0/1683446032 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2698003a10 con 0x7f26a41085d0 2024-04-23T14:38:45.594 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.593+0000 7f26a89ab700 1 -- 172.21.15.203:0/1683446032 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a4103c40 con 0x7f26a41085d0 2024-04-23T14:38:45.594 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.593+0000 7f26937fe700 1 -- 172.21.15.203:0/1683446032 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2698029070 con 0x7f26a41085d0 2024-04-23T14:38:45.594 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.593+0000 7f26937fe700 1 --2- 172.21.15.203:0/1683446032 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f268c039650 0x7f268c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:45.594 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.594+0000 7f26937fe700 1 -- 172.21.15.203:0/1683446032 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f269801e070 con 0x7f26a41085d0 2024-04-23T14:38:45.594 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.594+0000 7f26a1d9b700 1 --2- 172.21.15.203:0/1683446032 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f268c039650 0x7f268c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:45.595 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.595+0000 7f26a1d9b700 1 --2- 172.21.15.203:0/1683446032 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f268c039650 0x7f268c03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2694006fd0 tx=0x7f2694006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:45.609 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.608+0000 7f26937fe700 1 -- 172.21.15.203:0/1683446032 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2698013070 con 0x7f26a41085d0 2024-04-23T14:38:45.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.819+0000 7f26a89ab700 1 -- 172.21.15.203:0/1683446032 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi173:172.21.15.173=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}) -- 0x7f26a4075a20 con 0x7f268c039650 2024-04-23T14:38:45.834 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.834+0000 7f26937fe700 1 -- 172.21.15.203:0/1683446032 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f26a4075a20 con 0x7f268c039650 2024-04-23T14:38:45.834 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mon update... 2024-04-23T14:38:45.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.836+0000 7f26a89ab700 1 -- 172.21.15.203:0/1683446032 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f268c039650 msgr2=0x7f268c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.836+0000 7f26a89ab700 1 --2- 172.21.15.203:0/1683446032 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f268c039650 0x7f268c03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2694006fd0 tx=0x7f2694006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:45.836 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.836+0000 7f26a89ab700 1 -- 172.21.15.203:0/1683446032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a41085d0 msgr2=0x7f26a4074df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:45.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.836+0000 7f26a89ab700 1 --2- 172.21.15.203:0/1683446032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a41085d0 0x7f26a4074df0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2698005850 tx=0x7f2698005030 comp rx=0 tx=0).stop 2024-04-23T14:38:45.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.836+0000 7f26a89ab700 1 -- 172.21.15.203:0/1683446032 shutdown_connections 2024-04-23T14:38:45.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.836+0000 7f26a89ab700 1 --2- 172.21.15.203:0/1683446032 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f268c039650 0x7f268c03bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.836+0000 7f26a89ab700 1 --2- 172.21.15.203:0/1683446032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26a41085d0 0x7f26a4074df0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:45.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.836+0000 7f26a89ab700 1 -- 172.21.15.203:0/1683446032 >> 172.21.15.203:0/1683446032 conn(0x7f26a40fe030 msgr2=0x7f26a40ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:45.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.836+0000 7f26a89ab700 1 -- 172.21.15.203:0/1683446032 shutdown_connections 2024-04-23T14:38:45.837 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:45.836+0000 7f26a89ab700 1 -- 172.21.15.203:0/1683446032 wait complete. 2024-04-23T14:38:46.458 DEBUG:teuthology.orchestra.run.smithi173:mon.b> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.b.service 2024-04-23T14:38:46.461 DEBUG:teuthology.orchestra.run.smithi203:mon.c> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.c.service 2024-04-23T14:38:46.463 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T14:38:46.463 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T14:38:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:46 smithi083 ceph-mon[27867]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:38:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:38:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:38:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:38:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:38:47.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:47 smithi083 ceph-mon[27867]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi173:172.21.15.173=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:38:47.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:47 smithi083 ceph-mon[27867]: Saving service mon spec with placement smithi083:172.21.15.83=a;smithi173:172.21.15.173=b;smithi203:172.21.15.203=c;count:3 2024-04-23T14:38:47.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:47 smithi083 ceph-mon[27867]: Deploying daemon mon.c on smithi203 2024-04-23T14:38:47.645 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:38:48.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:48 smithi083 ceph-mon[27867]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:49.515 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.513+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/3297611914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef01085d0 msgr2=0x7f9ef01089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.515 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.513+0000 7f9ef5b3d700 1 --2- 172.21.15.203:0/3297611914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef01085d0 0x7f9ef01089b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009910 tx=0x7f9ee0009c20 comp rx=0 tx=0).stop 2024-04-23T14:38:49.515 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.513+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/3297611914 shutdown_connections 2024-04-23T14:38:49.515 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.513+0000 7f9ef5b3d700 1 --2- 172.21.15.203:0/3297611914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef01085d0 0x7f9ef01089b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.515 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.513+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/3297611914 >> 172.21.15.203:0/3297611914 conn(0x7f9ef00fe030 msgr2=0x7f9ef0100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:49.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.513+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/3297611914 shutdown_connections 2024-04-23T14:38:49.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.513+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/3297611914 wait complete. 2024-04-23T14:38:49.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.514+0000 7f9ef5b3d700 1 Processor -- start 2024-04-23T14:38:49.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.514+0000 7f9ef5b3d700 1 -- start start 2024-04-23T14:38:49.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.514+0000 7f9ef5b3d700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef01085d0 0x7f9ef0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.514+0000 7f9ef5b3d700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef010cb90 con 0x7f9ef01085d0 2024-04-23T14:38:49.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.515+0000 7f9eef7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef01085d0 0x7f9ef0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.515+0000 7f9eef7fe700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef01085d0 0x7f9ef0197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.203:35082/0 (socket says 172.21.15.203:35082) 2024-04-23T14:38:49.516 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.515+0000 7f9eef7fe700 1 -- 172.21.15.203:0/2604420463 learned_addr learned my addr 172.21.15.203:0/2604420463 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-23T14:38:49.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.515+0000 7f9eef7fe700 1 -- 172.21.15.203:0/2604420463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef0198510 con 0x7f9ef01085d0 2024-04-23T14:38:49.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.516+0000 7f9eef7fe700 1 --2- 172.21.15.203:0/2604420463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef01085d0 0x7f9ef0197fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005850 tx=0x7f9ee0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:49.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.516+0000 7f9eecff9700 1 -- 172.21.15.203:0/2604420463 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee0026070 con 0x7f9ef01085d0 2024-04-23T14:38:49.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.516+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/2604420463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef01987a0 con 0x7f9ef01085d0 2024-04-23T14:38:49.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.516+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/2604420463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef019c8a0 con 0x7f9ef01085d0 2024-04-23T14:38:49.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.516+0000 7f9eecff9700 1 -- 172.21.15.203:0/2604420463 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee000bc30 con 0x7f9ef01085d0 2024-04-23T14:38:49.517 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.516+0000 7f9eecff9700 1 -- 172.21.15.203:0/2604420463 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ee0004800 con 0x7f9ef01085d0 2024-04-23T14:38:49.518 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.517+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/2604420463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9edc005330 con 0x7f9ef01085d0 2024-04-23T14:38:49.518 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.517+0000 7f9eecff9700 1 -- 172.21.15.203:0/2604420463 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9ee0029070 con 0x7f9ef01085d0 2024-04-23T14:38:49.518 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.518+0000 7f9eecff9700 1 --2- 172.21.15.203:0/2604420463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9ed8039650 0x7f9ed803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:49.518 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.518+0000 7f9eecff9700 1 -- 172.21.15.203:0/2604420463 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9ee001e070 con 0x7f9ef01085d0 2024-04-23T14:38:49.519 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.518+0000 7f9eeeffd700 1 --2- 172.21.15.203:0/2604420463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9ed8039650 0x7f9ed803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:49.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.519+0000 7f9eeeffd700 1 --2- 172.21.15.203:0/2604420463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9ed8039650 0x7f9ed803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4006fd0 tx=0x7f9ee4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:49.534 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.534+0000 7f9eecff9700 1 -- 172.21.15.203:0/2604420463 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ee0013070 con 0x7f9ef01085d0 2024-04-23T14:38:49.816 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.816+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/2604420463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9edc0058f0 con 0x7f9ef01085d0 2024-04-23T14:38:49.817 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.817+0000 7f9eecff9700 1 -- 172.21.15.203:0/2604420463 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f9ee0034090 con 0x7f9ef01085d0 2024-04-23T14:38:49.818 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:38:49.818 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":1,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","modified":"2024-04-23T14:36:37.193338Z","created":"2024-04-23T14:36:37.193338Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T14:38:49.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.819+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/2604420463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9ed8039650 msgr2=0x7f9ed803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.819+0000 7f9ef5b3d700 1 --2- 172.21.15.203:0/2604420463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9ed8039650 0x7f9ed803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4006fd0 tx=0x7f9ee4006e40 comp rx=0 tx=0).stop 2024-04-23T14:38:49.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.819+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/2604420463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef01085d0 msgr2=0x7f9ef0197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:49.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.819+0000 7f9ef5b3d700 1 --2- 172.21.15.203:0/2604420463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef01085d0 0x7f9ef0197fd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0005850 tx=0x7f9ee0005030 comp rx=0 tx=0).stop 2024-04-23T14:38:49.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.819+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/2604420463 shutdown_connections 2024-04-23T14:38:49.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.819+0000 7f9ef5b3d700 1 --2- 172.21.15.203:0/2604420463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9ed8039650 0x7f9ed803bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.819+0000 7f9ef5b3d700 1 --2- 172.21.15.203:0/2604420463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ef01085d0 0x7f9ef0197fd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:49.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.819+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/2604420463 >> 172.21.15.203:0/2604420463 conn(0x7f9ef00fe030 msgr2=0x7f9ef00ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:49.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.819+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/2604420463 shutdown_connections 2024-04-23T14:38:49.820 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:49.819+0000 7f9ef5b3d700 1 -- 172.21.15.203:0/2604420463 wait complete. 2024-04-23T14:38:49.821 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 1 2024-04-23T14:38:50.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:50 smithi083 ceph-mon[27867]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:50 smithi083 ceph-mon[27867]: from='client.? 172.21.15.203:0/2604420463' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T14:38:51.508 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 podman[32780]: 2024-04-23 14:38:51.41088941 +0000 UTC m=+0.028424812 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T14:38:51.772 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 podman[32780]: 2024-04-23 14:38:51.746258717 +0000 UTC m=+0.363794091 container init 06fa1ec7928659783b41e449b5525419181bc1b0f99e0c41f71e32d17c3e413c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T14:38:51.772 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 podman[32780]: 2024-04-23 14:38:51.749239113 +0000 UTC m=+0.366774490 container start 06fa1ec7928659783b41e449b5525419181bc1b0f99e0c41f71e32d17c3e413c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-23T14:38:51.922 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T14:38:51.922 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T14:38:52.153 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:38:52.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T14:38:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: ceph version 19.0.0-3126-gd44fddd9 (d44fddd910b5e94da0f2357af4be37d7bd495ae0) squid (dev), process ceph-mon, pid 2 2024-04-23T14:38:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pidfile_write: ignore empty --pid-file 2024-04-23T14:38:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: load: jerasure load: lrc 2024-04-23T14:38:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: RocksDB version: 7.9.2 2024-04-23T14:38:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Git sha 0 2024-04-23T14:38:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Compile date 2024-04-22 13:15:13 2024-04-23T14:38:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: DB SUMMARY 2024-04-23T14:38:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: DB Session ID: 1U4SU1QS351D2N6BB2P1 2024-04-23T14:38:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: CURRENT file: CURRENT 2024-04-23T14:38:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.error_if_exists: 0 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.create_if_missing: 0 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.paranoid_checks: 1 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.env: 0x55f0227afcc0 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.info_log: 0x55f024dc1c20 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T14:38:52.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.statistics: (nil) 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.use_fsync: 0 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_log_file_size: 0 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.allow_fallocate: 1 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.use_direct_reads: 0 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.db_log_dir: 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.wal_dir: 2024-04-23T14:38:52.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T14:38:52.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T14:38:52.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T14:38:52.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.write_buffer_manager: 0x55f024d901e0 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T14:38:52.227 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.unordered_write: 0 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.row_cache: None 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.wal_filter: None 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.two_write_queues: 0 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.wal_compression: 0 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.atomic_flush: 0 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T14:38:52.228 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.log_readahead_size: 0 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_background_jobs: 2 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_background_compactions: -1 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_subcompactions: 1 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T14:38:52.229 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_open_files: -1 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_background_flushes: -1 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Compression algorithms supported: 2024-04-23T14:38:52.230 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: kZSTD supported: 0 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: kXpressCompression supported: 0 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: kZlibCompression supported: 1 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: kSnappyCompression supported: 1 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: kLZ4Compression supported: 1 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: kBZip2Compression supported: 0 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.merge_operator: 2024-04-23T14:38:52.231 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_filter: None 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f024dc1de0) 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout: cache_index_and_filter_blocks: 1 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout: pin_top_level_index_and_filter: 1 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_type: 0 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout: data_block_index_type: 0 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_shortening: 1 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T14:38:52.232 INFO:journalctl@ceph.mon.c.smithi203.stdout: checksum: 4 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: no_block_cache: 0 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache: 0x55f024dc3090 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_name: BinnedLRUCache 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_options: 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: capacity : 536870912 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: num_shard_bits : 4 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: strict_capacity_limit : 0 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: high_pri_pool_ratio: 0.000 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_cache_compressed: (nil) 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: persistent_cache: (nil) 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_size: 4096 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_size_deviation: 10 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_restart_interval: 16 2024-04-23T14:38:52.233 INFO:journalctl@ceph.mon.c.smithi203.stdout: index_block_restart_interval: 1 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: metadata_block_size: 4096 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: partition_filters: 0 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: use_delta_encoding: 1 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: filter_policy: bloomfilter 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: whole_key_filtering: 1 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: verify_compression: 0 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: read_amp_bytes_per_bit: 0 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: format_version: 5 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_index_compression: 1 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: block_align: 0 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: max_auto_readahead_size: 262144 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: prepopulate_block_cache: 0 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: initial_auto_readahead_size: 8192 2024-04-23T14:38:52.234 INFO:journalctl@ceph.mon.c.smithi203.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compression: NoCompression 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.num_levels: 7 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T14:38:52.235 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T14:38:52.236 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T14:38:52.237 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T14:38:52.238 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.inplace_update_support: 0 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.bloom_locality: 0 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.max_successive_merges: 0 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T14:38:52.239 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.ttl: 2592000 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.enable_blob_files: false 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.min_blob_size: 0 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T14:38:52.240 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 907d94a3-f36f-4523-9389-4e79cc3a9751 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713883131778659, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713883131779085, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713883131, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "907d94a3-f36f-4523-9389-4e79cc3a9751", "db_session_id": "1U4SU1QS351D2N6BB2P1", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713883131779164, "job": 1, "event": "recovery_finished"} 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f024eb0000 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: DB pointer 0x55f024e9a000 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T14:38:52.241 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** DB Stats ** 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** Compaction Stats [default] ** 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T14:38:52.242 INFO:journalctl@ceph.mon.c.smithi203.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** Compaction Stats [default] ** 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T14:38:52.243 INFO:journalctl@ceph.mon.c.smithi203.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout: Block cache BinnedLRUCache@0x55f024dc3090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: using public_addr v2:172.21.15.203:0/0 -> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: starting mon.c rank -1 at public addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] at bind addrs [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(???) e0 preinit fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).mds e1 new map 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout: e1 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T14:38:52.244 INFO:journalctl@ceph.mon.c.smithi203.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout: legacy client fscid: -1 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout: 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout: No filesystems configured 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.245 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Added host smithi173 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.246 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi203", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Deploying cephadm binary to smithi203 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.247 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Added host smithi203 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi203", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:38:52.248 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Updating smithi203:/etc/ceph/ceph.conf 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3443801362' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3443801362' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.249 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi083:172.21.15.83=a;smithi173:172.21.15.173=b;smithi203:172.21.15.203=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Saving service mon spec with placement smithi083:172.21.15.83=a;smithi173:172.21.15.173=b;smithi203:172.21.15.203=c;count:3 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: Deploying daemon mon.c on smithi203 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: from='client.? 172.21.15.203:0/2604420463' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 ceph-mon[32800]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 bash[32780]: 06fa1ec7928659783b41e449b5525419181bc1b0f99e0c41f71e32d17c3e413c 2024-04-23T14:38:52.250 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:51 smithi203 systemd[1]: Started Ceph mon.c for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:38:57.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.176+0000 7fb4215af700 1 -- 172.21.15.203:0/920443356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4080035c0 msgr2=0x7fb408005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:57.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.176+0000 7fb4215af700 1 --2- 172.21.15.203:0/920443356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4080035c0 0x7fb408005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb41c102550 tx=0x7fb414014ec0 comp rx=0 tx=0).stop 2024-04-23T14:38:57.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.176+0000 7fb4215af700 1 -- 172.21.15.203:0/920443356 shutdown_connections 2024-04-23T14:38:57.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.176+0000 7fb4215af700 1 --2- 172.21.15.203:0/920443356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4080035c0 0x7fb408005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:57.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.176+0000 7fb4215af700 1 -- 172.21.15.203:0/920443356 >> 172.21.15.203:0/920443356 conn(0x7fb41c0fe0f0 msgr2=0x7fb41c100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:57.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.176+0000 7fb4215af700 1 -- 172.21.15.203:0/920443356 shutdown_connections 2024-04-23T14:38:57.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.176+0000 7fb4215af700 1 -- 172.21.15.203:0/920443356 wait complete. 2024-04-23T14:38:57.179 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: mon.a calling monitor election 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: mon.c calling monitor election 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: monmap epoch 2 2024-04-23T14:38:57.180 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:38:57.181 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: last_changed 2024-04-23T14:38:51.812045+0000 2024-04-23T14:38:57.181 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: created 2024-04-23T14:36:37.193338+0000 2024-04-23T14:38:57.181 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: min_mon_release 19 (squid) 2024-04-23T14:38:57.181 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: election_strategy: 1 2024-04-23T14:38:57.181 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-23T14:38:57.181 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-04-23T14:38:57.181 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: fsmap 2024-04-23T14:38:57.181 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T14:38:57.181 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: mgrmap e13: x(active, since 59s) 2024-04-23T14:38:57.181 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: overall HEALTH_OK 2024-04-23T14:38:57.181 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:57.181 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:57.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.177+0000 7fb4215af700 1 Processor -- start 2024-04-23T14:38:57.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.178+0000 7fb4215af700 1 -- start start 2024-04-23T14:38:57.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.178+0000 7fb4215af700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4080035c0 0x7fb41c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:57.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.178+0000 7fb4215af700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb41c198650 0x7fb41c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:57.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.178+0000 7fb4215af700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c10a6c0 con 0x7fb4080035c0 2024-04-23T14:38:57.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.178+0000 7fb4215af700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c10a540 con 0x7fb41c198650 2024-04-23T14:38:57.183 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.178+0000 7fb41a7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb41c198650 0x7fb41c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.178+0000 7fb41a7fc700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb41c198650 0x7fb41c19c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57826/0 (socket says 172.21.15.203:57826) 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.178+0000 7fb41a7fc700 1 -- 172.21.15.203:0/2279440144 learned_addr learned my addr 172.21.15.203:0/2279440144 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.179+0000 7fb41a7fc700 1 -- 172.21.15.203:0/2279440144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb41c198650 msgr2=0x7fb41c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.179+0000 7fb41a7fc700 1 -- 172.21.15.203:0/2279440144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb41c198650 msgr2=0x7fb41c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.179+0000 7fb41a7fc700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb41c198650 0x7fb41c19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.179+0000 7fb41a7fc700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb41c198650 0x7fb41c19c9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.179+0000 7fb41affd700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4080035c0 0x7fb41c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.179+0000 7fb41affd700 1 -- 172.21.15.203:0/2279440144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb41c198650 msgr2=0x7fb41c19c9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.179+0000 7fb41affd700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb41c198650 0x7fb41c19c9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.179+0000 7fb41affd700 1 -- 172.21.15.203:0/2279440144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c198c90 con 0x7fb4080035c0 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.180+0000 7fb41affd700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4080035c0 0x7fb41c198110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb4140051b0 tx=0x7fb414005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.180+0000 7fb413fff700 1 -- 172.21.15.203:0/2279440144 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb414004aa0 con 0x7fb4080035c0 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.180+0000 7fb4215af700 1 -- 172.21.15.203:0/2279440144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41c19cf20 con 0x7fb4080035c0 2024-04-23T14:38:57.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.180+0000 7fb4215af700 1 -- 172.21.15.203:0/2279440144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41c19d480 con 0x7fb4080035c0 2024-04-23T14:38:57.185 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.180+0000 7fb413fff700 1 -- 172.21.15.203:0/2279440144 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb414004c40 con 0x7fb4080035c0 2024-04-23T14:38:57.185 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.180+0000 7fb413fff700 1 -- 172.21.15.203:0/2279440144 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb414012650 con 0x7fb4080035c0 2024-04-23T14:38:57.185 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.182+0000 7fb4215af700 1 -- 172.21.15.203:0/2279440144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb408002140 con 0x7fb4080035c0 2024-04-23T14:38:57.185 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.182+0000 7fb413fff700 1 -- 172.21.15.203:0/2279440144 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb41401e470 con 0x7fb4080035c0 2024-04-23T14:38:57.185 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.182+0000 7fb413fff700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb3fc0393f0 0x7fb3fc03b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:38:57.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.182+0000 7fb413fff700 1 -- 172.21.15.203:0/2279440144 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb414025070 con 0x7fb4080035c0 2024-04-23T14:38:57.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.183+0000 7fb41a7fc700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb3fc0393f0 0x7fb3fc03b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:38:57.186 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.184+0000 7fb41a7fc700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb3fc0393f0 0x7fb3fc03b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb404009180 tx=0x7fb404008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:38:57.201 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.200+0000 7fb413fff700 1 -- 172.21.15.203:0/2279440144 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb414053070 con 0x7fb4080035c0 2024-04-23T14:38:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:38:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: mon.a calling monitor election 2024-04-23T14:38:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: mon.c calling monitor election 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: monmap epoch 2 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: last_changed 2024-04-23T14:38:51.812045+0000 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: created 2024-04-23T14:36:37.193338+0000 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: min_mon_release 19 (squid) 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: election_strategy: 1 2024-04-23T14:38:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-23T14:38:57.260 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-04-23T14:38:57.260 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: fsmap 2024-04-23T14:38:57.260 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T14:38:57.260 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: mgrmap e13: x(active, since 59s) 2024-04-23T14:38:57.260 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: overall HEALTH_OK 2024-04-23T14:38:57.260 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:57.260 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:57.487 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.485+0000 7fb4215af700 1 -- 172.21.15.203:0/2279440144 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb408002c30 con 0x7fb4080035c0 2024-04-23T14:38:57.487 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.486+0000 7fb413fff700 1 -- 172.21.15.203:0/2279440144 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fb414027bf0 con 0x7fb4080035c0 2024-04-23T14:38:57.488 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:38:57.488 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":2,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","modified":"2024-04-23T14:38:51.812045Z","created":"2024-04-23T14:36:37.193338Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T14:38:57.489 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.488+0000 7fb4215af700 1 -- 172.21.15.203:0/2279440144 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb3fc0393f0 msgr2=0x7fb3fc03b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:57.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.488+0000 7fb4215af700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb3fc0393f0 0x7fb3fc03b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb404009180 tx=0x7fb404008040 comp rx=0 tx=0).stop 2024-04-23T14:38:57.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.488+0000 7fb4215af700 1 -- 172.21.15.203:0/2279440144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4080035c0 msgr2=0x7fb41c198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:38:57.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.488+0000 7fb4215af700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4080035c0 0x7fb41c198110 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb4140051b0 tx=0x7fb414005500 comp rx=0 tx=0).stop 2024-04-23T14:38:57.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.488+0000 7fb4215af700 1 -- 172.21.15.203:0/2279440144 shutdown_connections 2024-04-23T14:38:57.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.488+0000 7fb4215af700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb3fc0393f0 0x7fb3fc03b8b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:57.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.488+0000 7fb4215af700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4080035c0 0x7fb41c198110 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:57.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.488+0000 7fb4215af700 1 --2- 172.21.15.203:0/2279440144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb41c198650 0x7fb41c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:38:57.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.488+0000 7fb4215af700 1 -- 172.21.15.203:0/2279440144 >> 172.21.15.203:0/2279440144 conn(0x7fb41c0fe0f0 msgr2=0x7fb41c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:38:57.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.488+0000 7fb4215af700 1 -- 172.21.15.203:0/2279440144 shutdown_connections 2024-04-23T14:38:57.490 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:38:57.488+0000 7fb4215af700 1 -- 172.21.15.203:0/2279440144 wait complete. 2024-04-23T14:38:57.491 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 2 2024-04-23T14:38:58.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:58.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:38:58.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:38:58.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:57 smithi203 ceph-mon[32800]: Deploying daemon mon.b on smithi173 2024-04-23T14:38:58.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:57 smithi203 ceph-mon[32800]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:58.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:57 smithi203 ceph-mon[32800]: from='client.? 172.21.15.203:0/2279440144' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T14:38:58.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:38:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:58.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:38:58.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:38:58.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:38:58.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:57 smithi083 ceph-mon[27867]: Deploying daemon mon.b on smithi173 2024-04-23T14:38:58.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:57 smithi083 ceph-mon[27867]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:38:58.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:57 smithi083 ceph-mon[27867]: from='client.? 172.21.15.203:0/2279440144' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T14:38:58.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:38:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:38:58.946 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T14:38:58.947 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T14:38:59.197 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:39:00.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:00 smithi203 ceph-mon[32800]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:00.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:00 smithi083 ceph-mon[27867]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:00.674 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.671+0000 7f8726605700 1 -- 172.21.15.203:0/387733054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8720100f60 msgr2=0x7f8720101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:00.674 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.671+0000 7f8726605700 1 --2- 172.21.15.203:0/387733054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8720100f60 0x7f8720101340 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8710009910 tx=0x7f8710009c20 comp rx=0 tx=0).stop 2024-04-23T14:39:00.674 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.671+0000 7f8726605700 1 -- 172.21.15.203:0/387733054 shutdown_connections 2024-04-23T14:39:00.674 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.671+0000 7f8726605700 1 --2- 172.21.15.203:0/387733054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8720100f60 0x7f8720101340 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:00.674 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.671+0000 7f8726605700 1 -- 172.21.15.203:0/387733054 >> 172.21.15.203:0/387733054 conn(0x7f8720077910 msgr2=0x7f8720077d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:00.674 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.671+0000 7f8726605700 1 -- 172.21.15.203:0/387733054 shutdown_connections 2024-04-23T14:39:00.674 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.671+0000 7f8726605700 1 -- 172.21.15.203:0/387733054 wait complete. 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f8726605700 1 Processor -- start 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f8726605700 1 -- start start 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f8726605700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8720100f60 0x7f872019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f8726605700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872019aa10 0x7f872019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f8726605700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f872010cc60 con 0x7f8720100f60 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f8726605700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f872010cae0 con 0x7f872019aa10 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f871f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872019aa10 0x7f872019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f871f7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872019aa10 0x7f872019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:38836/0 (socket says 172.21.15.203:38836) 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f871f7fe700 1 -- 172.21.15.203:0/795567600 learned_addr learned my addr 172.21.15.203:0/795567600 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f871f7fe700 1 -- 172.21.15.203:0/795567600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872019aa10 msgr2=0x7f872019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f871f7fe700 1 -- 172.21.15.203:0/795567600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872019aa10 msgr2=0x7f872019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f871f7fe700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872019aa10 0x7f872019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f871f7fe700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872019aa10 0x7f872019eda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:39:00.675 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.672+0000 7f871ffff700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8720100f60 0x7f872019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:00.677 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.673+0000 7f871ffff700 1 -- 172.21.15.203:0/795567600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872019aa10 msgr2=0x7f872019eda0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:39:00.677 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.673+0000 7f871ffff700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872019aa10 0x7f872019eda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:00.677 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.673+0000 7f871ffff700 1 -- 172.21.15.203:0/795567600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872019b050 con 0x7f8720100f60 2024-04-23T14:39:00.677 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.673+0000 7f871ffff700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8720100f60 0x7f872019a4d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8710014000 tx=0x7f8710005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:00.677 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.674+0000 7f871d7fa700 1 -- 172.21.15.203:0/795567600 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8710005dc0 con 0x7f8720100f60 2024-04-23T14:39:00.677 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.674+0000 7f8726605700 1 -- 172.21.15.203:0/795567600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872019f2e0 con 0x7f8720100f60 2024-04-23T14:39:00.678 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.674+0000 7f8726605700 1 -- 172.21.15.203:0/795567600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872019f840 con 0x7f8720100f60 2024-04-23T14:39:00.678 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.674+0000 7f871d7fa700 1 -- 172.21.15.203:0/795567600 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f871000be40 con 0x7f8720100f60 2024-04-23T14:39:00.678 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.674+0000 7f871d7fa700 1 -- 172.21.15.203:0/795567600 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8710022ba0 con 0x7f8720100f60 2024-04-23T14:39:00.678 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.675+0000 7f8726605700 1 -- 172.21.15.203:0/795567600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f870c005330 con 0x7f8720100f60 2024-04-23T14:39:00.678 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.675+0000 7f871d7fa700 1 -- 172.21.15.203:0/795567600 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f871002a070 con 0x7f8720100f60 2024-04-23T14:39:00.678 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.675+0000 7f871d7fa700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f87080393f0 0x7f870803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:00.678 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.675+0000 7f871d7fa700 1 -- 172.21.15.203:0/795567600 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f871001e030 con 0x7f8720100f60 2024-04-23T14:39:00.678 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.676+0000 7f871f7fe700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f87080393f0 0x7f870803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:00.678 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.677+0000 7f871f7fe700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f87080393f0 0x7f870803b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8714009180 tx=0x7f8714008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:00.691 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.690+0000 7f871d7fa700 1 -- 172.21.15.203:0/795567600 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8710013070 con 0x7f8720100f60 2024-04-23T14:39:00.746 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.745+0000 7f871d7fa700 1 -- 172.21.15.203:0/795567600 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871000b960 con 0x7f8720100f60 2024-04-23T14:39:00.974 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:00.972+0000 7f8726605700 1 -- 172.21.15.203:0/795567600 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f870c0058f0 con 0x7f8720100f60 2024-04-23T14:39:01.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:00 smithi173 systemd[1]: Started Ceph mon.b for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:39:05.769 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-23T14:39:05.770 INFO:teuthology.orchestra.run.smithi203.stdout:{"epoch":3,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","modified":"2024-04-23T14:39:00.743834Z","created":"2024-04-23T14:36:37.193338Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:3300","nonce":0},{"type":"v1","addr":"172.21.15.203:6789","nonce":0}]},"addr":"172.21.15.203:6789/0","public_addr":"172.21.15.203:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T14:39:05.770 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:05.766+0000 7f871d7fa700 1 -- 172.21.15.203:0/795567600 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f8710022470 con 0x7f8720100f60 2024-04-23T14:39:05.770 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:05.769+0000 7f8726605700 1 -- 172.21.15.203:0/795567600 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f87080393f0 msgr2=0x7f870803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:05.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:05.769+0000 7f8726605700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f87080393f0 0x7f870803b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8714009180 tx=0x7f8714008040 comp rx=0 tx=0).stop 2024-04-23T14:39:05.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:05.769+0000 7f8726605700 1 -- 172.21.15.203:0/795567600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8720100f60 msgr2=0x7f872019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:05.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:05.769+0000 7f8726605700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8720100f60 0x7f872019a4d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8710014000 tx=0x7f8710005030 comp rx=0 tx=0).stop 2024-04-23T14:39:05.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:05.770+0000 7f8726605700 1 -- 172.21.15.203:0/795567600 shutdown_connections 2024-04-23T14:39:05.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:05.770+0000 7f8726605700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f87080393f0 0x7f870803b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:05.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:05.770+0000 7f8726605700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8720100f60 0x7f872019a4d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:05.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:05.770+0000 7f8726605700 1 --2- 172.21.15.203:0/795567600 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f872019aa10 0x7f872019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:05.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:05.770+0000 7f8726605700 1 -- 172.21.15.203:0/795567600 >> 172.21.15.203:0/795567600 conn(0x7f8720077910 msgr2=0x7f87200ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:05.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:05.770+0000 7f8726605700 1 -- 172.21.15.203:0/795567600 shutdown_connections 2024-04-23T14:39:05.771 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:05.770+0000 7f8726605700 1 -- 172.21.15.203:0/795567600 wait complete. 2024-04-23T14:39:05.772 INFO:teuthology.orchestra.run.smithi203.stderr:dumped monmap epoch 3 2024-04-23T14:39:06.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: mon.c calling monitor election 2024-04-23T14:39:06.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:39:06.057 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: mon.a calling monitor election 2024-04-23T14:39:06.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:06.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:39:06.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: from='client.? 172.21.15.203:0/795567600' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T14:39:06.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:06.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:06.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:06.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:06.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:06.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:06.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:06.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:06.058 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T14:39:06.059 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: monmap epoch 3 2024-04-23T14:39:06.059 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:39:06.059 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: last_changed 2024-04-23T14:39:00.743834+0000 2024-04-23T14:39:06.059 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: created 2024-04-23T14:36:37.193338+0000 2024-04-23T14:39:06.059 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: min_mon_release 19 (squid) 2024-04-23T14:39:06.059 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: election_strategy: 1 2024-04-23T14:39:06.059 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-23T14:39:06.059 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-04-23T14:39:06.059 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: 2: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-04-23T14:39:06.059 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: fsmap 2024-04-23T14:39:06.059 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T14:39:06.060 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: mgrmap e13: x(active, since 68s) 2024-04-23T14:39:06.060 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: overall HEALTH_OK 2024-04-23T14:39:06.060 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:06.060 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:06.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: mon.c calling monitor election 2024-04-23T14:39:06.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:39:06.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: mon.a calling monitor election 2024-04-23T14:39:06.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:06.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:39:06.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: from='client.? 172.21.15.203:0/795567600' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T14:39:06.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:06.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:06.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:06.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:06.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:06.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: monmap epoch 3 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: last_changed 2024-04-23T14:39:00.743834+0000 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: created 2024-04-23T14:36:37.193338+0000 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: min_mon_release 19 (squid) 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: election_strategy: 1 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: 2: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: fsmap 2024-04-23T14:39:06.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T14:39:06.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: mgrmap e13: x(active, since 68s) 2024-04-23T14:39:06.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: overall HEALTH_OK 2024-04-23T14:39:06.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:06.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:06.386 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-23T14:39:06.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-23T14:39:06.812 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:39:07.110 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:06 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:07.110 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:06 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:07.110 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:06 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:07.111 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:06 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:07.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:06 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:07.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:06 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:07.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:06 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:07.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:06 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:08.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: mon.c calling monitor election 2024-04-23T14:39:08.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:39:08.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: mon.a calling monitor election 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='client.? 172.21.15.203:0/795567600' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: monmap epoch 3 2024-04-23T14:39:08.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: last_changed 2024-04-23T14:39:00.743834+0000 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: created 2024-04-23T14:36:37.193338+0000 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: min_mon_release 19 (squid) 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: election_strategy: 1 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: 2: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: fsmap 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: mgrmap e13: x(active, since 68s) 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: overall HEALTH_OK 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:08.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:08.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: mon.b calling monitor election 2024-04-23T14:39:08.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-23T14:39:08.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T14:39:08.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: Updating smithi203:/etc/ceph/ceph.conf 2024-04-23T14:39:08.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:39:08.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:39:08.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: mon.b calling monitor election 2024-04-23T14:39:08.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: mon.c calling monitor election 2024-04-23T14:39:08.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: mon.a calling monitor election 2024-04-23T14:39:08.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T14:39:08.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: monmap epoch 3 2024-04-23T14:39:08.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:39:08.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: last_changed 2024-04-23T14:39:00.743834+0000 2024-04-23T14:39:08.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: created 2024-04-23T14:36:37.193338+0000 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: min_mon_release 19 (squid) 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: election_strategy: 1 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: 2: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: fsmap 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: mgrmap e13: x(active, since 70s) 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: overall HEALTH_OK 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.386 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:08 smithi083 ceph-mon[27867]: Updating smithi083:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:39:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: mon.b calling monitor election 2024-04-23T14:39:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-23T14:39:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T14:39:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: Updating smithi203:/etc/ceph/ceph.conf 2024-04-23T14:39:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:39:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:39:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: mon.b calling monitor election 2024-04-23T14:39:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: mon.c calling monitor election 2024-04-23T14:39:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: mon.a calling monitor election 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: monmap epoch 3 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: last_changed 2024-04-23T14:39:00.743834+0000 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: created 2024-04-23T14:36:37.193338+0000 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: min_mon_release 19 (squid) 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: election_strategy: 1 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: 2: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: fsmap 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: mgrmap e13: x(active, since 70s) 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: overall HEALTH_OK 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:08 smithi173 ceph-mon[32443]: Updating smithi083:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:39:08.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: mon.b calling monitor election 2024-04-23T14:39:08.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: Updating smithi203:/etc/ceph/ceph.conf 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: mon.b calling monitor election 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: mon.c calling monitor election 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: mon.a calling monitor election 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: monmap epoch 3 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: last_changed 2024-04-23T14:39:00.743834+0000 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: created 2024-04-23T14:36:37.193338+0000 2024-04-23T14:39:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: min_mon_release 19 (squid) 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: election_strategy: 1 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: 0: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.a 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: 1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.c 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: 2: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: fsmap 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: mgrmap e13: x(active, since 70s) 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: overall HEALTH_OK 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:08.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:08 smithi203 ceph-mon[32800]: Updating smithi083:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:39:08.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.935+0000 7f1ddb800700 1 -- 172.21.15.83:0/2621402993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd4074ec0 msgr2=0x7f1dd40752a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:08.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.935+0000 7f1ddb800700 1 --2- 172.21.15.83:0/2621402993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd4074ec0 0x7f1dd40752a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009910 tx=0x7f1dc8009c20 comp rx=0 tx=0).stop 2024-04-23T14:39:08.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.935+0000 7f1ddb800700 1 -- 172.21.15.83:0/2621402993 shutdown_connections 2024-04-23T14:39:08.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.935+0000 7f1ddb800700 1 --2- 172.21.15.83:0/2621402993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd4074ec0 0x7f1dd40752a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:08.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.935+0000 7f1ddb800700 1 -- 172.21.15.83:0/2621402993 >> 172.21.15.83:0/2621402993 conn(0x7f1dd40fe030 msgr2=0x7f1dd4100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:08.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.936+0000 7f1ddb800700 1 -- 172.21.15.83:0/2621402993 shutdown_connections 2024-04-23T14:39:08.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.936+0000 7f1ddb800700 1 -- 172.21.15.83:0/2621402993 wait complete. 2024-04-23T14:39:08.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.937+0000 7f1ddb800700 1 Processor -- start 2024-04-23T14:39:08.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.937+0000 7f1ddb800700 1 -- start start 2024-04-23T14:39:08.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.938+0000 7f1ddb800700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dd4074ec0 0x7f1dd419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:08.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.938+0000 7f1ddb800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd419aa10 0x7f1dd419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:08.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.938+0000 7f1ddb800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dd419f2e0 0x7f1dd419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:08.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.938+0000 7f1ddb800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd410eda0 con 0x7f1dd419aa10 2024-04-23T14:39:08.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.938+0000 7f1ddb800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd410ec20 con 0x7f1dd419f2e0 2024-04-23T14:39:08.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.938+0000 7f1ddb800700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd410ef20 con 0x7f1dd4074ec0 2024-04-23T14:39:08.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.938+0000 7f1dd3e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd419aa10 0x7f1dd419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:08.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.938+0000 7f1dd3e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd419aa10 0x7f1dd419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43216/0 (socket says 172.21.15.83:43216) 2024-04-23T14:39:08.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.938+0000 7f1dd3e00700 1 -- 172.21.15.83:0/1415336673 learned_addr learned my addr 172.21.15.83:0/1415336673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:39:08.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.938+0000 7f1dd9800700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dd419f2e0 0x7f1dd419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:08.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.938+0000 7f1dd8e00700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dd4074ec0 0x7f1dd419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:08.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.939+0000 7f1dd3e00700 1 -- 172.21.15.83:0/1415336673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dd4074ec0 msgr2=0x7f1dd419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:08.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.939+0000 7f1dd3e00700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dd4074ec0 0x7f1dd419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:08.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.939+0000 7f1dd3e00700 1 -- 172.21.15.83:0/1415336673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dd419f2e0 msgr2=0x7f1dd419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:08.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.939+0000 7f1dd3e00700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dd419f2e0 0x7f1dd419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:08.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.939+0000 7f1dd3e00700 1 -- 172.21.15.83:0/1415336673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd41a7050 con 0x7f1dd419aa10 2024-04-23T14:39:08.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.939+0000 7f1dd3e00700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd419aa10 0x7f1dd419eda0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000b780 tx=0x7f1dc000ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:08.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.939+0000 7f1dd1600700 1 -- 172.21.15.83:0/1415336673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0016070 con 0x7f1dd419aa10 2024-04-23T14:39:08.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.939+0000 7f1dd1600700 1 -- 172.21.15.83:0/1415336673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dc00036a0 con 0x7f1dd419aa10 2024-04-23T14:39:08.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.939+0000 7f1ddb800700 1 -- 172.21.15.83:0/1415336673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd41a7280 con 0x7f1dd419aa10 2024-04-23T14:39:08.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.939+0000 7f1dd1600700 1 -- 172.21.15.83:0/1415336673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0011af0 con 0x7f1dd419aa10 2024-04-23T14:39:08.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.940+0000 7f1ddb800700 1 -- 172.21.15.83:0/1415336673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd41a7810 con 0x7f1dd419aa10 2024-04-23T14:39:08.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.941+0000 7f1dd1600700 1 -- 172.21.15.83:0/1415336673 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1dc0019070 con 0x7f1dd419aa10 2024-04-23T14:39:08.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.941+0000 7f1ddb800700 1 -- 172.21.15.83:0/1415336673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db4005330 con 0x7f1dd419aa10 2024-04-23T14:39:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.942+0000 7f1dd1600700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1dbc039ae0 0x7f1dbc03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.942+0000 7f1dd1600700 1 -- 172.21.15.83:0/1415336673 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1dc004b2d0 con 0x7f1dd419aa10 2024-04-23T14:39:08.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.942+0000 7f1dd8e00700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1dbc039ae0 0x7f1dbc03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:08.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.943+0000 7f1dd8e00700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1dbc039ae0 0x7f1dbc03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8006010 tx=0x7f1dc80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:08.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:08.948+0000 7f1dd1600700 1 -- 172.21.15.83:0/1415336673 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1dc0015170 con 0x7f1dd419aa10 2024-04-23T14:39:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.174+0000 7f1ddb800700 1 -- 172.21.15.83:0/1415336673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f1db4005620 con 0x7f1dd419aa10 2024-04-23T14:39:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.175+0000 7f1dd1600700 1 -- 172.21.15.83:0/1415336673 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f1dc0019bb0 con 0x7f1dd419aa10 2024-04-23T14:39:09.175 INFO:teuthology.orchestra.run.smithi083.stdout:# minimal ceph.conf for 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:39:09.175 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-04-23T14:39:09.175 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:39:09.176 INFO:teuthology.orchestra.run.smithi083.stdout: mon_host = [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] 2024-04-23T14:39:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.179+0000 7f1ddb800700 1 -- 172.21.15.83:0/1415336673 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1dbc039ae0 msgr2=0x7f1dbc03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.179+0000 7f1ddb800700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1dbc039ae0 0x7f1dbc03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8006010 tx=0x7f1dc80058e0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.179+0000 7f1ddb800700 1 -- 172.21.15.83:0/1415336673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd419aa10 msgr2=0x7f1dd419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.179+0000 7f1ddb800700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd419aa10 0x7f1dd419eda0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000b780 tx=0x7f1dc000ba90 comp rx=0 tx=0).stop 2024-04-23T14:39:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.179+0000 7f1ddb800700 1 -- 172.21.15.83:0/1415336673 shutdown_connections 2024-04-23T14:39:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.179+0000 7f1ddb800700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1dbc039ae0 0x7f1dbc03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.179+0000 7f1ddb800700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dd419f2e0 0x7f1dd419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.179+0000 7f1ddb800700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1dd4074ec0 0x7f1dd419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.179+0000 7f1ddb800700 1 --2- 172.21.15.83:0/1415336673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dd419aa10 0x7f1dd419eda0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.179+0000 7f1ddb800700 1 -- 172.21.15.83:0/1415336673 >> 172.21.15.83:0/1415336673 conn(0x7f1dd40fe030 msgr2=0x7f1dd4106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.180+0000 7f1ddb800700 1 -- 172.21.15.83:0/1415336673 shutdown_connections 2024-04-23T14:39:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:09.180+0000 7f1ddb800700 1 -- 172.21.15.83:0/1415336673 wait complete. 2024-04-23T14:39:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T14:39:09.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:39:09.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T14:39:09.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:09.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: Reconfiguring daemon mon.a on smithi083 2024-04-23T14:39:09.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:09.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:09 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1415336673' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:09.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:09.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T14:39:09.954 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:39:09.954 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T14:39:09.954 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:09.954 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: Reconfiguring daemon mon.a on smithi083 2024-04-23T14:39:09.954 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:09.954 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:09 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1415336673' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:09.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:39:09.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:09.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T14:39:09.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:39:09.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T14:39:09.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:09.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: Reconfiguring daemon mon.a on smithi083 2024-04-23T14:39:09.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:09.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:09 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1415336673' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:10.010 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-23T14:39:10.010 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:39:10.011 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T14:39:10.098 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:39:10.098 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:39:10.221 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T14:39:10.221 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T14:39:10.250 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T14:39:10.250 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:39:10.316 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-23T14:39:10.316 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T14:39:10.347 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-23T14:39:10.347 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:39:10.414 INFO:tasks.cephadm:Adding mgr.x on smithi083 2024-04-23T14:39:10.415 INFO:tasks.cephadm:Adding mgr.y on smithi173 2024-04-23T14:39:10.415 INFO:tasks.cephadm:Adding mgr.z on smithi203 2024-04-23T14:39:10.415 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi083=x;smithi173=y;smithi203=z' 2024-04-23T14:39:10.686 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:39:12.238 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.234+0000 7f5705522700 1 -- 172.21.15.203:0/3616149101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700102680 msgr2=0x7f5700102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:12.238 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.234+0000 7f5705522700 1 --2- 172.21.15.203:0/3616149101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700102680 0x7f5700102aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009910 tx=0x7f56f0009c20 comp rx=0 tx=0).stop 2024-04-23T14:39:12.238 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.234+0000 7f5705522700 1 -- 172.21.15.203:0/3616149101 shutdown_connections 2024-04-23T14:39:12.238 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.234+0000 7f5705522700 1 --2- 172.21.15.203:0/3616149101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700102680 0x7f5700102aa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:12.238 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.234+0000 7f5705522700 1 -- 172.21.15.203:0/3616149101 >> 172.21.15.203:0/3616149101 conn(0x7f57000fdc20 msgr2=0x7f5700100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:12.238 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.234+0000 7f5705522700 1 -- 172.21.15.203:0/3616149101 shutdown_connections 2024-04-23T14:39:12.238 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.234+0000 7f5705522700 1 -- 172.21.15.203:0/3616149101 wait complete. 2024-04-23T14:39:12.238 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.235+0000 7f5705522700 1 Processor -- start 2024-04-23T14:39:12.238 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.235+0000 7f5705522700 1 -- start start 2024-04-23T14:39:12.238 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.235+0000 7f5705522700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700102680 0x7f5700197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:12.238 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.235+0000 7f5705522700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57001983d0 0x7f570019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.235+0000 7f5705522700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5700198850 0x7f570019f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.235+0000 7f5705522700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5700108710 con 0x7f5700102680 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.235+0000 7f5705522700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5700108590 con 0x7f57001983d0 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.235+0000 7f5705522700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5700108890 con 0x7f5700198850 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.235+0000 7f56ff7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5700198850 0x7f570019f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.235+0000 7f56ff7fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5700198850 0x7f570019f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51808/0 (socket says 172.21.15.203:51808) 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.235+0000 7f56ff7fe700 1 -- 172.21.15.203:0/2134720008 learned_addr learned my addr 172.21.15.203:0/2134720008 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56feffd700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700102680 0x7f5700197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56f67fc700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57001983d0 0x7f570019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56ff7fe700 1 -- 172.21.15.203:0/2134720008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57001983d0 msgr2=0x7f570019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56ff7fe700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57001983d0 0x7f570019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56ff7fe700 1 -- 172.21.15.203:0/2134720008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700102680 msgr2=0x7f5700197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56ff7fe700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700102680 0x7f5700197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56ff7fe700 1 -- 172.21.15.203:0/2134720008 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57001a02a0 con 0x7f5700198850 2024-04-23T14:39:12.239 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56feffd700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700102680 0x7f5700197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:12.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56f67fc700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57001983d0 0x7f570019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:39:12.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56ff7fe700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5700198850 0x7f570019f9d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f56f800ba50 tx=0x7f56f800be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:12.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56fcff9700 1 -- 172.21.15.203:0/2134720008 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8019070 con 0x7f5700198850 2024-04-23T14:39:12.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56fcff9700 1 -- 172.21.15.203:0/2134720008 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f8014440 con 0x7f5700198850 2024-04-23T14:39:12.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f5705522700 1 -- 172.21.15.203:0/2134720008 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57001a0590 con 0x7f5700198850 2024-04-23T14:39:12.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f56fcff9700 1 -- 172.21.15.203:0/2134720008 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8007590 con 0x7f5700198850 2024-04-23T14:39:12.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.236+0000 7f5705522700 1 -- 172.21.15.203:0/2134720008 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57001a0b00 con 0x7f5700198850 2024-04-23T14:39:12.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.237+0000 7f56fcff9700 1 -- 172.21.15.203:0/2134720008 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f56f801c070 con 0x7f5700198850 2024-04-23T14:39:12.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.237+0000 7f5705522700 1 -- 172.21.15.203:0/2134720008 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5700064150 con 0x7f5700198850 2024-04-23T14:39:12.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.237+0000 7f56fcff9700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f56ec039ae0 0x7f56ec03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:12.240 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.237+0000 7f56fcff9700 1 -- 172.21.15.203:0/2134720008 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f56f804c070 con 0x7f5700198850 2024-04-23T14:39:12.241 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.238+0000 7f56feffd700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f56ec039ae0 0x7f56ec03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:12.241 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.239+0000 7f56feffd700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f56ec039ae0 0x7f56ec03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f56f0005330 tx=0x7f56f0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:12.243 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.242+0000 7f56fcff9700 1 -- 172.21.15.203:0/2134720008 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56f8014910 con 0x7f5700198850 2024-04-23T14:39:12.275 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:12 smithi173 ceph-mon[32443]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:12.276 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:12.276 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:12.276 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:39:12.276 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T14:39:12.276 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:12.450 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.449+0000 7f5705522700 1 -- 172.21.15.203:0/2134720008 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi173=y;smithi203=z", "target": ["mon-mgr", ""]}) -- 0x7f5700062fc0 con 0x7f56ec039ae0 2024-04-23T14:39:12.450 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:12 smithi203 ceph-mon[32800]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:12.450 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:12.450 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:12.450 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:39:12.451 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T14:39:12.451 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:12.463 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.463+0000 7f56fcff9700 1 -- 172.21.15.203:0/2134720008 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5700062fc0 con 0x7f56ec039ae0 2024-04-23T14:39:12.463 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mgr update... 2024-04-23T14:39:12.466 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.465+0000 7f5705522700 1 -- 172.21.15.203:0/2134720008 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f56ec039ae0 msgr2=0x7f56ec03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:12.466 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.465+0000 7f5705522700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f56ec039ae0 0x7f56ec03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f56f0005330 tx=0x7f56f0005fd0 comp rx=0 tx=0).stop 2024-04-23T14:39:12.466 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.465+0000 7f5705522700 1 -- 172.21.15.203:0/2134720008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5700198850 msgr2=0x7f570019f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:12.466 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.465+0000 7f5705522700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5700198850 0x7f570019f9d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f56f800ba50 tx=0x7f56f800be10 comp rx=0 tx=0).stop 2024-04-23T14:39:12.466 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.465+0000 7f5705522700 1 -- 172.21.15.203:0/2134720008 shutdown_connections 2024-04-23T14:39:12.466 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.465+0000 7f5705522700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f56ec039ae0 0x7f56ec03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:12.466 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.465+0000 7f5705522700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5700198850 0x7f570019f9d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:12.466 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.465+0000 7f5705522700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5700102680 0x7f5700197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:12.466 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.465+0000 7f5705522700 1 --2- 172.21.15.203:0/2134720008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57001983d0 0x7f570019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:12.467 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.465+0000 7f5705522700 1 -- 172.21.15.203:0/2134720008 >> 172.21.15.203:0/2134720008 conn(0x7f57000fdc20 msgr2=0x7f57000fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:12.467 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.465+0000 7f5705522700 1 -- 172.21.15.203:0/2134720008 shutdown_connections 2024-04-23T14:39:12.467 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:39:12.465+0000 7f5705522700 1 -- 172.21.15.203:0/2134720008 wait complete. 2024-04-23T14:39:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:12 smithi083 ceph-mon[27867]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:39:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T14:39:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:13.063 DEBUG:teuthology.orchestra.run.smithi173:mgr.y> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.y.service 2024-04-23T14:39:13.066 DEBUG:teuthology.orchestra.run.smithi203:mgr.z> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.z.service 2024-04-23T14:39:13.068 INFO:tasks.cephadm:Deploying OSDs... 2024-04-23T14:39:13.068 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:39:13.068 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T14:39:13.125 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T14:39:13.125 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-04-23T14:39:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T14:39:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:39:13.174 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-23T14:39:13.174 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:39:13.174 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:39:13.174 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-23 14:38:13.141572763 +0000 2024-04-23T14:39:13.174 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-23 14:29:45.617705758 +0000 2024-04-23T14:39:13.174 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-23 14:29:45.617705758 +0000 2024-04-23T14:39:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-23 14:29:45.617705758 +0000 2024-04-23T14:39:13.174 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T14:39:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-23T14:39:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-23T14:39:13.245 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000332723 s, 1.5 MB/s 2024-04-23T14:39:13.249 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T14:39:13.301 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-04-23T14:39:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T14:39:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:39:13.350 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-23T14:39:13.350 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:39:13.350 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:39:13.350 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-23 14:38:13.203571255 +0000 2024-04-23T14:39:13.350 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-23 14:29:46.241690987 +0000 2024-04-23T14:39:13.350 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-23 14:29:46.241690987 +0000 2024-04-23T14:39:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-23 14:29:46.241690987 +0000 2024-04-23T14:39:13.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T14:39:13.416 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:13 smithi083 ceph-mon[27867]: Reconfiguring mon.b (monmap changed)... 2024-04-23T14:39:13.416 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:13 smithi083 ceph-mon[27867]: Reconfiguring daemon mon.b on smithi173 2024-04-23T14:39:13.416 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:13.417 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:13.417 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:13.417 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:39:13.417 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T14:39:13.417 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:13.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-23T14:39:13.424 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-23T14:39:13.425 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000348092 s, 1.5 MB/s 2024-04-23T14:39:13.428 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T14:39:13.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:13 smithi203 ceph-mon[32800]: Reconfiguring mon.b (monmap changed)... 2024-04-23T14:39:13.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:13 smithi203 ceph-mon[32800]: Reconfiguring daemon mon.b on smithi173 2024-04-23T14:39:13.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:13.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:13.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:13.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:39:13.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T14:39:13.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:13.485 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-04-23T14:39:13.534 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T14:39:13.534 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:39:13.534 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-23T14:39:13.534 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:39:13.534 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:39:13.534 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-23 14:38:13.260569869 +0000 2024-04-23T14:39:13.534 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-23 14:29:46.862676287 +0000 2024-04-23T14:39:13.534 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-23 14:29:46.862676287 +0000 2024-04-23T14:39:13.535 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-23 14:29:46.862676287 +0000 2024-04-23T14:39:13.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T14:39:13.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:13 smithi173 ceph-mon[32443]: Reconfiguring mon.b (monmap changed)... 2024-04-23T14:39:13.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:13 smithi173 ceph-mon[32443]: Reconfiguring daemon mon.b on smithi173 2024-04-23T14:39:13.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:13.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:13.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:13.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T14:39:13.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T14:39:13.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:13.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-23T14:39:13.605 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-23T14:39:13.605 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000355891 s, 1.4 MB/s 2024-04-23T14:39:13.608 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T14:39:13.660 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-04-23T14:39:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T14:39:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:39:13.708 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-23T14:39:13.708 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:39:13.708 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:39:13.709 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-23 14:38:13.307568725 +0000 2024-04-23T14:39:13.709 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-23 14:29:47.462662084 +0000 2024-04-23T14:39:13.709 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-23 14:29:47.462662084 +0000 2024-04-23T14:39:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-23 14:29:47.462662084 +0000 2024-04-23T14:39:13.709 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T14:39:13.784 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-23T14:39:13.785 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-23T14:39:13.785 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000469181 s, 1.1 MB/s 2024-04-23T14:39:13.788 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T14:39:13.843 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2024-04-23T14:39:13.843 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T14:39:13.860 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T14:39:13.860 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2024-04-23T14:39:13.916 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T14:39:13.916 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:39:13.916 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-23T14:39:13.916 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:39:13.916 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:39:13.916 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-23 14:39:00.192891684 +0000 2024-04-23T14:39:13.916 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-23 14:28:45.371925222 +0000 2024-04-23T14:39:13.917 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-23 14:28:45.371925222 +0000 2024-04-23T14:39:13.917 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-23 14:28:45.371925222 +0000 2024-04-23T14:39:13.917 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T14:39:13.980 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-23T14:39:13.980 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-23T14:39:13.980 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.00022189 s, 2.3 MB/s 2024-04-23T14:39:13.981 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T14:39:14.038 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2024-04-23T14:39:14.093 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T14:39:14.093 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:39:14.093 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-23T14:39:14.093 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:39:14.093 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:39:14.093 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-23 14:39:00.192891684 +0000 2024-04-23T14:39:14.093 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-23 14:28:45.728917033 +0000 2024-04-23T14:39:14.093 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-23 14:28:45.728917033 +0000 2024-04-23T14:39:14.094 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-23 14:28:45.728917033 +0000 2024-04-23T14:39:14.094 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T14:39:14.155 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-23T14:39:14.155 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-23T14:39:14.155 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000209825 s, 2.4 MB/s 2024-04-23T14:39:14.157 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T14:39:14.212 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2024-04-23T14:39:14.267 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T14:39:14.268 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:39:14.268 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-23T14:39:14.268 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:39:14.268 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:39:14.268 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-23 14:39:00.193891662 +0000 2024-04-23T14:39:14.268 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-23 14:28:46.065909325 +0000 2024-04-23T14:39:14.268 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-23 14:28:46.065909325 +0000 2024-04-23T14:39:14.268 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-23 14:28:46.065909325 +0000 2024-04-23T14:39:14.268 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T14:39:14.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:14 smithi173 ceph-mon[32443]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi173=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:39:14.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:14 smithi173 ceph-mon[32443]: Saving service mgr spec with placement smithi083=x;smithi173=y;smithi203=z;count:3 2024-04-23T14:39:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:14 smithi173 ceph-mon[32443]: Reconfiguring mon.c (monmap changed)... 2024-04-23T14:39:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:14 smithi173 ceph-mon[32443]: Reconfiguring daemon mon.c on smithi203 2024-04-23T14:39:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:14 smithi173 ceph-mon[32443]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:14.330 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-23T14:39:14.330 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-23T14:39:14.330 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000181835 s, 2.8 MB/s 2024-04-23T14:39:14.331 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T14:39:14.386 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2024-04-23T14:39:14.440 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T14:39:14.440 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:39:14.440 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-23T14:39:14.440 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:39:14.441 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:39:14.441 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2024-04-23 14:39:00.193891662 +0000 2024-04-23T14:39:14.441 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2024-04-23 14:28:46.403901593 +0000 2024-04-23T14:39:14.441 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2024-04-23 14:28:46.403901593 +0000 2024-04-23T14:39:14.441 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: 2024-04-23 14:28:46.403901593 +0000 2024-04-23T14:39:14.441 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T14:39:14.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:14 smithi203 ceph-mon[32800]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi173=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:39:14.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:14 smithi203 ceph-mon[32800]: Saving service mgr spec with placement smithi083=x;smithi173=y;smithi203=z;count:3 2024-04-23T14:39:14.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:14 smithi203 ceph-mon[32800]: Reconfiguring mon.c (monmap changed)... 2024-04-23T14:39:14.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:14 smithi203 ceph-mon[32800]: Reconfiguring daemon mon.c on smithi203 2024-04-23T14:39:14.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:14 smithi203 ceph-mon[32800]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:14.501 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2024-04-23T14:39:14.501 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2024-04-23T14:39:14.501 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000195923 s, 2.6 MB/s 2024-04-23T14:39:14.503 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T14:39:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:14 smithi083 ceph-mon[27867]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi083=x;smithi173=y;smithi203=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:39:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:14 smithi083 ceph-mon[27867]: Saving service mgr spec with placement smithi083=x;smithi173=y;smithi203=z;count:3 2024-04-23T14:39:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:14 smithi083 ceph-mon[27867]: Reconfiguring mon.c (monmap changed)... 2024-04-23T14:39:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:14 smithi083 ceph-mon[27867]: Reconfiguring daemon mon.c on smithi203 2024-04-23T14:39:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:14 smithi083 ceph-mon[27867]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:14.559 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-23T14:39:14.559 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T14:39:14.574 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T14:39:14.575 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-04-23T14:39:14.629 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T14:39:14.629 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:39:14.629 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-23T14:39:14.629 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:39:14.629 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:39:14.629 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-23 14:38:51.184044440 +0000 2024-04-23T14:39:14.630 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-23 14:29:03.212206021 +0000 2024-04-23T14:39:14.630 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-23 14:29:03.212206021 +0000 2024-04-23T14:39:14.630 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-23 14:29:03.212206021 +0000 2024-04-23T14:39:14.630 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T14:39:14.692 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-23T14:39:14.692 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-23T14:39:14.692 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000246045 s, 2.1 MB/s 2024-04-23T14:39:14.693 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T14:39:14.749 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-04-23T14:39:14.804 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T14:39:14.804 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:39:14.804 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-23T14:39:14.804 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:39:14.804 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:39:14.804 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-23 14:38:51.185044414 +0000 2024-04-23T14:39:14.805 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-23 14:29:03.548197320 +0000 2024-04-23T14:39:14.805 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-23 14:29:03.548197320 +0000 2024-04-23T14:39:14.805 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-23 14:29:03.548197320 +0000 2024-04-23T14:39:14.805 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T14:39:14.866 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-23T14:39:14.867 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-23T14:39:14.867 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000198946 s, 2.6 MB/s 2024-04-23T14:39:14.868 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T14:39:14.923 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-04-23T14:39:14.978 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T14:39:14.978 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:39:14.978 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-23T14:39:14.978 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:39:14.978 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:39:14.978 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-23 14:38:51.185044414 +0000 2024-04-23T14:39:14.979 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-23 14:29:03.915187817 +0000 2024-04-23T14:39:14.979 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-23 14:29:03.915187817 +0000 2024-04-23T14:39:14.979 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-23 14:29:03.915187817 +0000 2024-04-23T14:39:14.979 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T14:39:15.040 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-23T14:39:15.040 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-23T14:39:15.040 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000176509 s, 2.9 MB/s 2024-04-23T14:39:15.041 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T14:39:15.097 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-04-23T14:39:15.153 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T14:39:15.153 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T14:39:15.153 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-23T14:39:15.153 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:39:15.153 INFO:teuthology.orchestra.run.smithi203.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T14:39:15.153 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-23 14:38:51.185044414 +0000 2024-04-23T14:39:15.153 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-23 14:29:04.268178676 +0000 2024-04-23T14:39:15.154 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-23 14:29:04.268178676 +0000 2024-04-23T14:39:15.154 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: 2024-04-23 14:29:04.268178676 +0000 2024-04-23T14:39:15.154 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T14:39:15.215 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-23T14:39:15.215 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-23T14:39:15.215 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000189997 s, 2.7 MB/s 2024-04-23T14:39:15.217 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T14:39:15.274 INFO:tasks.cephadm:Deploying osd.0 on smithi083 with /dev/vg_nvme/lv_4... 2024-04-23T14:39:15.274 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T14:39:15.693 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:39:15.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:15.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T14:39:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T14:39:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T14:39:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:15 smithi083 ceph-mon[27867]: Deploying daemon mgr.z on smithi203 2024-04-23T14:39:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:15 smithi083 ceph-mon[27867]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:15.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:15.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:15.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:15.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:15.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:15.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:15.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T14:39:15.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T14:39:15.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T14:39:15.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:15.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:15 smithi173 ceph-mon[32443]: Deploying daemon mgr.z on smithi203 2024-04-23T14:39:15.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:15 smithi173 ceph-mon[32443]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:15.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:15.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:15.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:15.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:15.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:15.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:15.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T14:39:15.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T14:39:15.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T14:39:15.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:15.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:15 smithi203 ceph-mon[32800]: Deploying daemon mgr.z on smithi203 2024-04-23T14:39:15.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:15 smithi203 ceph-mon[32800]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:16.877 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:16 smithi203 systemd[1]: Starting Ceph mgr.z for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:39:17.171 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:16 smithi203 podman[34643]: 2024-04-23 14:39:16.780254827 +0000 UTC m=+0.018224930 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T14:39:17.171 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:16 smithi203 podman[34643]: 2024-04-23 14:39:16.901587419 +0000 UTC m=+0.139557523 container create 96b8fc56554b16714cd7d4d69c92541cd57fbd01194d34884786150cc1b40ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-23T14:39:17.171 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:17 smithi203 podman[34643]: 2024-04-23 14:39:17.095669963 +0000 UTC m=+0.333640066 container init 96b8fc56554b16714cd7d4d69c92541cd57fbd01194d34884786150cc1b40ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-23T14:39:17.171 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:17 smithi203 podman[34643]: 2024-04-23 14:39:17.098110913 +0000 UTC m=+0.336081028 container start 96b8fc56554b16714cd7d4d69c92541cd57fbd01194d34884786150cc1b40ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, CEPH_POINT_RELEASE=) 2024-04-23T14:39:17.171 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:17 smithi203 ceph-mgr[34662]: -- 172.21.15.203:0/3278878233 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b67b025a0 con 0x557b67ad7000 2024-04-23T14:39:17.171 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:17 smithi203 bash[34643]: 96b8fc56554b16714cd7d4d69c92541cd57fbd01194d34884786150cc1b40ba9 2024-04-23T14:39:17.435 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:17 smithi203 systemd[1]: Started Ceph mgr.z for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:39:17.435 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:17 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:17.251+0000 7ff16d6ba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T14:39:17.435 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:17 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:17.434+0000 7ff16d6ba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:39:17.721 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:17 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:17.533+0000 7ff16d6ba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T14:39:18.221 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:17 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:17.838+0000 7ff16d6ba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T14:39:18.221 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:17 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:17.920+0000 7ff16d6ba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T14:39:18.458 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T14:39:18.458 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T14:39:18.458 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-23T14:39:18.458 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-23T14:39:18.458 INFO:teuthology.orchestra.run.smithi083.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0337556 s, 311 MB/s 2024-04-23T14:39:18.458 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-23T14:39:18.505 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:18 smithi083 ceph-mon[27867]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:18.505 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:18.505 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:18.505 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:18.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T14:39:18.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T14:39:18.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T14:39:18.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:18.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:18 smithi203 ceph-mon[32800]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:18.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:18.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:18.537 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:18.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T14:39:18.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T14:39:18.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T14:39:18.538 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:18 smithi173 ceph-mon[32443]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T14:39:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T14:39:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T14:39:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:19.203 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_4 2024-04-23T14:39:19.254 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:19 smithi173 systemd[1]: Starting Ceph mgr.y for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:39:19.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:19 smithi203 ceph-mon[32800]: Deploying daemon mgr.y on smithi173 2024-04-23T14:39:19.471 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:19 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:19.137+0000 7ff16d6ba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T14:39:19.471 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:19 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:19.310+0000 7ff16d6ba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T14:39:19.506 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:19 smithi173 ceph-mon[32443]: Deploying daemon mgr.y on smithi173 2024-04-23T14:39:19.507 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:19 smithi173 podman[33802]: 2024-04-23 14:39:19.280166762 +0000 UTC m=+0.017770400 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T14:39:19.507 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:19 smithi173 podman[33802]: 2024-04-23 14:39:19.381672186 +0000 UTC m=+0.119275815 container create 0c1434fbabb195a7562e692c0158e201be8b9f5a340e3734edf3a29a79ed5ef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-23T14:39:19.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:19 smithi083 ceph-mon[27867]: Deploying daemon mgr.y on smithi173 2024-04-23T14:39:19.641 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:39:19.772 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:19 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:19.484+0000 7ff16d6ba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T14:39:19.773 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:19 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:19.771+0000 7ff16d6ba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T14:39:19.919 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:19 smithi173 podman[33802]: 2024-04-23 14:39:19.576858457 +0000 UTC m=+0.314462086 container init 0c1434fbabb195a7562e692c0158e201be8b9f5a340e3734edf3a29a79ed5ef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5) 2024-04-23T14:39:19.919 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:19 smithi173 podman[33802]: 2024-04-23 14:39:19.579474906 +0000 UTC m=+0.317078546 container start 0c1434fbabb195a7562e692c0158e201be8b9f5a340e3734edf3a29a79ed5ef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-23T14:39:19.919 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:19 smithi173 ceph-mgr[33821]: -- 172.21.15.173:0/3422982418 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d3fc405a0 con 0x563d3fc14c00 2024-04-23T14:39:19.919 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:19 smithi173 bash[33802]: 0c1434fbabb195a7562e692c0158e201be8b9f5a340e3734edf3a29a79ed5ef7 2024-04-23T14:39:19.919 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:19 smithi173 systemd[1]: Started Ceph mgr.y for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:39:19.919 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:19 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:19.732+0000 7fcbf8cec200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T14:39:19.919 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:19 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:19.918+0000 7fcbf8cec200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:39:20.080 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:19 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:19.864+0000 7ff16d6ba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T14:39:20.256 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:20 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:20.021+0000 7fcbf8cec200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T14:39:20.480 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:20 smithi203 ceph-mon[32800]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:20.480 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:20 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:20.480 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:20 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:20.480 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:20 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:20.480 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:20 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:20.480 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:20 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:20.481 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:20 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:20.079+0000 7ff16d6ba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T14:39:20.481 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:20 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:20.282+0000 7ff16d6ba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:39:20.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:20 smithi083 ceph-mon[27867]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:20.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:20 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:20.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:20 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:20.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:20 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:20.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:20 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:20.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:20 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:20.576 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:20 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:20.354+0000 7fcbf8cec200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T14:39:20.576 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:20 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:20.445+0000 7fcbf8cec200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T14:39:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:20 smithi173 ceph-mon[32443]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:20 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:20 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:20 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:20 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:20.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:20 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:21.221 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:20 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:20.924+0000 7ff16d6ba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T14:39:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.614+0000 7f8177e00700 1 -- 172.21.15.83:0/1195255256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8170108840 msgr2=0x7f817010acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.614+0000 7f8177e00700 1 --2- 172.21.15.83:0/1195255256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8170108840 0x7f817010acb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f816800b0a0 tx=0x7f816800b3b0 comp rx=0 tx=0).stop 2024-04-23T14:39:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.614+0000 7f8177e00700 1 -- 172.21.15.83:0/1195255256 shutdown_connections 2024-04-23T14:39:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.614+0000 7f8177e00700 1 --2- 172.21.15.83:0/1195255256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8170108840 0x7f817010acb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.614+0000 7f8177e00700 1 --2- 172.21.15.83:0/1195255256 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81701035e0 0x7f81701059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.614+0000 7f8177e00700 1 --2- 172.21.15.83:0/1195255256 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8170105f10 0x7f8170108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.614+0000 7f8177e00700 1 -- 172.21.15.83:0/1195255256 >> 172.21.15.83:0/1195255256 conn(0x7f81700fcff0 msgr2=0x7f81700ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.615+0000 7f8177e00700 1 -- 172.21.15.83:0/1195255256 shutdown_connections 2024-04-23T14:39:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.615+0000 7f8177e00700 1 -- 172.21.15.83:0/1195255256 wait complete. 2024-04-23T14:39:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8177e00700 1 Processor -- start 2024-04-23T14:39:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8177e00700 1 -- start start 2024-04-23T14:39:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8177e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81701035e0 0x7f8170198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8177e00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8170105f10 0x7f8170199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8177e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8170108840 0x7f81701a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8177e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f817010e770 con 0x7f8170108840 2024-04-23T14:39:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8177e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f817010e5f0 con 0x7f81701035e0 2024-04-23T14:39:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8177e00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f817010e8f0 con 0x7f8170105f10 2024-04-23T14:39:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8175400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81701035e0 0x7f8170198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8175400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81701035e0 0x7f8170198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:35778/0 (socket says 172.21.15.83:35778) 2024-04-23T14:39:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8175400700 1 -- 172.21.15.83:0/2752118383 learned_addr learned my addr 172.21.15.83:0/2752118383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8174a00700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8170105f10 0x7f8170199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.616+0000 7f8175e00700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8170108840 0x7f81701a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.617+0000 7f8175400700 1 -- 172.21.15.83:0/2752118383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8170105f10 msgr2=0x7f8170199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.617+0000 7f8175400700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8170105f10 0x7f8170199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.617+0000 7f8175400700 1 -- 172.21.15.83:0/2752118383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8170108840 msgr2=0x7f81701a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.617+0000 7f8175400700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8170108840 0x7f81701a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.617+0000 7f8175400700 1 -- 172.21.15.83:0/2752118383 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81701a0a10 con 0x7f81701035e0 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.617+0000 7f8175400700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81701035e0 0x7f8170198e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f816400e9c0 tx=0x7f816400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.617+0000 7f816e000700 1 -- 172.21.15.83:0/2752118383 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400cbd0 con 0x7f81701035e0 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.617+0000 7f816e000700 1 -- 172.21.15.83:0/2752118383 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816400ee60 con 0x7f81701035e0 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.617+0000 7f8177e00700 1 -- 172.21.15.83:0/2752118383 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81701a0d00 con 0x7f81701035e0 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.617+0000 7f816e000700 1 -- 172.21.15.83:0/2752118383 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164005680 con 0x7f81701035e0 2024-04-23T14:39:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.618+0000 7f8177e00700 1 -- 172.21.15.83:0/2752118383 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81701a1190 con 0x7f81701035e0 2024-04-23T14:39:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.619+0000 7f816e000700 1 -- 172.21.15.83:0/2752118383 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8164021440 con 0x7f81701035e0 2024-04-23T14:39:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.619+0000 7f8177e00700 1 -- 172.21.15.83:0/2752118383 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f817002dfe0 con 0x7f81701035e0 2024-04-23T14:39:21.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.619+0000 7f816e000700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8158039890 0x7f815803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:21.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.619+0000 7f816e000700 1 -- 172.21.15.83:0/2752118383 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f816404cdf0 con 0x7f81701035e0 2024-04-23T14:39:21.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.619+0000 7f8174a00700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8158039890 0x7f815803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:21.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.620+0000 7f8174a00700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8158039890 0x7f815803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f815c009c60 tx=0x7f815c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:21.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.629+0000 7f816e000700 1 -- 172.21.15.83:0/2752118383 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8164020c70 con 0x7f81701035e0 2024-04-23T14:39:21.664 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:21 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:21.353+0000 7ff16d6ba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T14:39:21.664 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:21 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:21.460+0000 7ff16d6ba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T14:39:21.664 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:21 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:21.663+0000 7ff16d6ba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T14:39:21.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:21.869+0000 7f8177e00700 1 -- 172.21.15.83:0/2752118383 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8170062f30 con 0x7f8158039890 2024-04-23T14:39:22.069 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:21 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:21.727+0000 7fcbf8cec200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T14:39:22.070 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:21 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:21.897+0000 7fcbf8cec200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T14:39:22.070 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:22.069+0000 7fcbf8cec200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T14:39:22.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:22.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:22.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:22.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T14:39:22.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T14:39:22.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:22.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:39:22.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:39:22.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:22.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.471 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:22.190+0000 7ff16d6ba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T14:39:22.471 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:22.282+0000 7ff16d6ba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T14:39:22.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:22.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.573 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:39:22.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:22.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:22 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:22.654 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:22.357+0000 7fcbf8cec200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T14:39:22.654 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:22.448+0000 7fcbf8cec200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T14:39:22.655 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:22.654+0000 7fcbf8cec200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T14:39:22.867 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:22.522+0000 7ff16d6ba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T14:39:22.867 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:22.654+0000 7ff16d6ba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T14:39:22.867 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:22.749+0000 7ff16d6ba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T14:39:22.867 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:22 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:22.866+0000 7ff16d6ba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T14:39:23.075 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:22 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:22.846+0000 7fcbf8cec200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:39:23.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:23 smithi203 ceph-mon[32800]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T14:39:23.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:23 smithi203 ceph-mon[32800]: Reconfiguring daemon mgr.x on smithi083 2024-04-23T14:39:23.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:23 smithi203 ceph-mon[32800]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:39:23.471 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:23 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:23.170+0000 7ff16d6ba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T14:39:23.576 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:23 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:23.464+0000 7fcbf8cec200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T14:39:23.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:23 smithi173 ceph-mon[32443]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T14:39:23.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:23 smithi173 ceph-mon[32443]: Reconfiguring daemon mgr.x on smithi083 2024-04-23T14:39:23.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:23 smithi173 ceph-mon[32443]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:39:23.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:23 smithi083 ceph-mon[27867]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T14:39:23.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:23 smithi083 ceph-mon[27867]: Reconfiguring daemon mgr.x on smithi083 2024-04-23T14:39:23.594 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:23 smithi083 ceph-mon[27867]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:39:24.183 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:23 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:23.887+0000 7fcbf8cec200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T14:39:24.183 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:23 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:23.985+0000 7fcbf8cec200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T14:39:24.184 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:24.183+0000 7fcbf8cec200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T14:39:24.221 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:23 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:23.770+0000 7ff16d6ba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T14:39:24.221 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:39:23 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:39:23.989+0000 7ff16d6ba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T14:39:24.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:24 smithi083 ceph-mon[27867]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:24.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:24.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:24.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:24.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:24.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:24.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:24.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:24 smithi083 ceph-mon[27867]: Standby manager daemon z started 2024-04-23T14:39:24.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:24 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.203:0/371591090' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T14:39:24.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:24 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.203:0/371591090' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T14:39:24.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:24 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.203:0/371591090' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T14:39:24.532 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:24 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.203:0/371591090' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T14:39:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-mon[32443]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-mon[32443]: Standby manager daemon z started 2024-04-23T14:39:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.203:0/371591090' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T14:39:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.203:0/371591090' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T14:39:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.203:0/371591090' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T14:39:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.203:0/371591090' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T14:39:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:24 smithi203 ceph-mon[32800]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:24.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:24.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:24 smithi203 ceph-mon[32800]: Standby manager daemon z started 2024-04-23T14:39:24.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:24 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.203:0/371591090' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T14:39:24.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:24 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.203:0/371591090' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T14:39:24.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:24 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.203:0/371591090' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T14:39:24.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:24 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.203:0/371591090' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T14:39:24.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:24.785+0000 7f816e000700 1 -- 172.21.15.83:0/2752118383 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f81640107b0 con 0x7f81701035e0 2024-04-23T14:39:25.061 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:24.728+0000 7fcbf8cec200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T14:39:25.062 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:24 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:24.820+0000 7fcbf8cec200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T14:39:25.326 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:25 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:25.061+0000 7fcbf8cec200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T14:39:25.326 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:25 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:25.193+0000 7fcbf8cec200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T14:39:25.326 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:25 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:25.287+0000 7fcbf8cec200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T14:39:25.727 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:25 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:25.403+0000 7fcbf8cec200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T14:39:25.727 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:25 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:25.723+0000 7fcbf8cec200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T14:39:26.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:25 smithi173 ceph-mon[32443]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-23T14:39:26.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:25 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T14:39:26.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:25 smithi173 ceph-mon[32443]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:26.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:25 smithi203 ceph-mon[32800]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-23T14:39:26.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:25 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T14:39:26.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:25 smithi203 ceph-mon[32800]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:26.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:25 smithi083 ceph-mon[27867]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-23T14:39:26.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:25 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T14:39:26.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:25 smithi083 ceph-mon[27867]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:26.576 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:26 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:26.320+0000 7fcbf8cec200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T14:39:26.576 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:39:26 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:39:26.536+0000 7fcbf8cec200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T14:39:26.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:26.811+0000 7f816e000700 1 -- 172.21.15.83:0/2752118383 <== mon.2 v2:172.21.15.173:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f8164016330 con 0x7f81701035e0 2024-04-23T14:39:27.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:26 smithi173 ceph-mon[32443]: Standby manager daemon y started 2024-04-23T14:39:27.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:26 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.173:0/1189722990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T14:39:27.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:26 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.173:0/1189722990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T14:39:27.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:26 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.173:0/1189722990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T14:39:27.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:26 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.173:0/1189722990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T14:39:27.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:26 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2485276917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f0dcc33-b313-4e7b-8cbf-d541c92624bb"}]: dispatch 2024-04-23T14:39:27.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:26 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f0dcc33-b313-4e7b-8cbf-d541c92624bb"}]: dispatch 2024-04-23T14:39:27.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:26 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f0dcc33-b313-4e7b-8cbf-d541c92624bb"}]': finished 2024-04-23T14:39:27.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:26 smithi173 ceph-mon[32443]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T14:39:27.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:26 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:27.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:26 smithi083 ceph-mon[27867]: Standby manager daemon y started 2024-04-23T14:39:27.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:26 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.173:0/1189722990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T14:39:27.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:26 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.173:0/1189722990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T14:39:27.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:26 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.173:0/1189722990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T14:39:27.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:26 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.173:0/1189722990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T14:39:27.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:26 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2485276917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f0dcc33-b313-4e7b-8cbf-d541c92624bb"}]: dispatch 2024-04-23T14:39:27.158 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:26 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f0dcc33-b313-4e7b-8cbf-d541c92624bb"}]: dispatch 2024-04-23T14:39:27.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:26 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f0dcc33-b313-4e7b-8cbf-d541c92624bb"}]': finished 2024-04-23T14:39:27.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:26 smithi083 ceph-mon[27867]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T14:39:27.159 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:26 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:27.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:26 smithi203 ceph-mon[32800]: Standby manager daemon y started 2024-04-23T14:39:27.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:26 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.173:0/1189722990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T14:39:27.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:26 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.173:0/1189722990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T14:39:27.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:26 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.173:0/1189722990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T14:39:27.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:26 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.173:0/1189722990' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T14:39:27.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:26 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2485276917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f0dcc33-b313-4e7b-8cbf-d541c92624bb"}]: dispatch 2024-04-23T14:39:27.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:26 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f0dcc33-b313-4e7b-8cbf-d541c92624bb"}]: dispatch 2024-04-23T14:39:27.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:26 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f0dcc33-b313-4e7b-8cbf-d541c92624bb"}]': finished 2024-04-23T14:39:27.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:26 smithi203 ceph-mon[32800]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T14:39:27.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:26 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:28.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:27 smithi173 ceph-mon[32443]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-04-23T14:39:28.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:27 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T14:39:28.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:27 smithi173 ceph-mon[32443]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:28.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:27 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3560465110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:39:28.189 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:27 smithi083 ceph-mon[27867]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-04-23T14:39:28.189 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:27 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T14:39:28.189 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:27 smithi083 ceph-mon[27867]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:28.189 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:27 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3560465110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:39:28.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:27 smithi203 ceph-mon[32800]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-04-23T14:39:28.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:27 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T14:39:28.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:27 smithi203 ceph-mon[32800]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:28.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:27 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3560465110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:39:30.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:30 smithi203 ceph-mon[32800]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:30.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:30 smithi083 ceph-mon[27867]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:30.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:30 smithi173 ceph-mon[32443]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:32.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:32 smithi203 ceph-mon[32800]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:32.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:32 smithi083 ceph-mon[27867]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:32.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:32 smithi173 ceph-mon[32443]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:34.462 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:34 smithi083 ceph-mon[27867]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:34.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:34 smithi203 ceph-mon[32800]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:34 smithi173 ceph-mon[32443]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:36.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:36 smithi203 ceph-mon[32800]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:36.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:36 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T14:39:36.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:36 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:36.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:36 smithi083 ceph-mon[27867]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:36.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:36 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T14:39:36.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:36 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:36.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:36 smithi173 ceph-mon[32443]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:36 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T14:39:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:36 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:37.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:37 smithi083 ceph-mon[27867]: Deploying daemon osd.0 on smithi083 2024-04-23T14:39:37.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:37 smithi203 ceph-mon[32800]: Deploying daemon osd.0 on smithi083 2024-04-23T14:39:37.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:37 smithi173 ceph-mon[32443]: Deploying daemon osd.0 on smithi083 2024-04-23T14:39:38.430 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:38 smithi083 ceph-mon[27867]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:38 smithi203 ceph-mon[32800]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:38 smithi173 ceph-mon[32443]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:40.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:40 smithi083 ceph-mon[27867]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:40.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:40 smithi203 ceph-mon[32800]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:40.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:40 smithi173 ceph-mon[32443]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:42.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:42 smithi083 ceph-mon[27867]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:42.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:42 smithi203 ceph-mon[32800]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:42.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:42 smithi173 ceph-mon[32443]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:44.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:44 smithi203 ceph-mon[32800]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:44.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:44 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:44.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:44 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:44.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:44 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:44.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:44 smithi083 ceph-mon[27867]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:44.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:44 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:44.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:44 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:44.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:44 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:44.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:44 smithi173 ceph-mon[32443]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:44.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:44 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:44.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:44 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:44.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:44 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:46.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:46 smithi203 ceph-mon[32800]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:46.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:46 smithi083 ceph-mon[27867]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:46.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:46 smithi173 ceph-mon[32443]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:48.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:48 smithi083 ceph-mon[27867]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:48.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:48 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:48.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:48 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:48.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:48 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:48.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:48 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:48.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:48 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:48.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:48 smithi083 ceph-mon[27867]: from='osd.0 [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T14:39:48.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:48 smithi083 ceph-mon[27867]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T14:39:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:48 smithi173 ceph-mon[32443]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:48 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:48 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:48 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:48 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:48.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:48 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:48.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:48 smithi173 ceph-mon[32443]: from='osd.0 [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T14:39:48.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:48 smithi173 ceph-mon[32443]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T14:39:48.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:48 smithi203 ceph-mon[32800]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:48.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:48 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:48.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:48 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:48.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:48 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:48.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:48 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:48.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:48 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:48.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:48 smithi203 ceph-mon[32800]: from='osd.0 [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T14:39:48.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:48 smithi203 ceph-mon[32800]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T14:39:48.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.760+0000 7f816e000700 1 -- 172.21.15.83:0/2752118383 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8170062f30 con 0x7f8158039890 2024-04-23T14:39:48.760 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 0 on host 'smithi083' 2024-04-23T14:39:48.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.763+0000 7f8177e00700 1 -- 172.21.15.83:0/2752118383 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8158039890 msgr2=0x7f815803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:48.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.763+0000 7f8177e00700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8158039890 0x7f815803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f815c009c60 tx=0x7f815c009260 comp rx=0 tx=0).stop 2024-04-23T14:39:48.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.763+0000 7f8177e00700 1 -- 172.21.15.83:0/2752118383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81701035e0 msgr2=0x7f8170198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:48.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.763+0000 7f8177e00700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81701035e0 0x7f8170198e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f816400e9c0 tx=0x7f816400ecd0 comp rx=0 tx=0).stop 2024-04-23T14:39:48.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.764+0000 7f8177e00700 1 -- 172.21.15.83:0/2752118383 shutdown_connections 2024-04-23T14:39:48.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.764+0000 7f8177e00700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8158039890 0x7f815803bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.764+0000 7f8177e00700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8170108840 0x7f81701a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.764+0000 7f8177e00700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81701035e0 0x7f8170198e50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.764+0000 7f8177e00700 1 --2- 172.21.15.83:0/2752118383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8170105f10 0x7f8170199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.764+0000 7f8177e00700 1 -- 172.21.15.83:0/2752118383 >> 172.21.15.83:0/2752118383 conn(0x7f81700fcff0 msgr2=0x7f8170103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.764+0000 7f8177e00700 1 -- 172.21.15.83:0/2752118383 shutdown_connections 2024-04-23T14:39:48.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:39:48.764+0000 7f8177e00700 1 -- 172.21.15.83:0/2752118383 wait complete. 2024-04-23T14:39:49.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:49 smithi083 ceph-mon[27867]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T14:39:49.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:49 smithi083 ceph-mon[27867]: from='osd.0 [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:39:49.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:49 smithi083 ceph-mon[27867]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T14:39:49.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:49 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:49.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:49 smithi083 ceph-mon[27867]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:39:49.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:49 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:49.500 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:49 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:49.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:49 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:49.548 DEBUG:teuthology.orchestra.run.smithi083:osd.0> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.0.service 2024-04-23T14:39:49.553 INFO:tasks.cephadm:Deploying osd.1 on smithi173 with /dev/vg_nvme/lv_4... 2024-04-23T14:39:49.553 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T14:39:49.567 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:49 smithi173 ceph-mon[32443]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T14:39:49.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:49 smithi173 ceph-mon[32443]: from='osd.0 [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:39:49.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:49 smithi173 ceph-mon[32443]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T14:39:49.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:49 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:49.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:49 smithi173 ceph-mon[32443]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:39:49.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:49 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:49.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:49 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:49.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:49 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:49.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:49 smithi203 ceph-mon[32800]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T14:39:49.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:49 smithi203 ceph-mon[32800]: from='osd.0 [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:39:49.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:49 smithi203 ceph-mon[32800]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T14:39:49.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:49 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:49.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:49 smithi203 ceph-mon[32800]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:39:49.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:49 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:49.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:49 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:49.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:49 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:39:49.782 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T14:39:50.163 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 14:39:49 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0[42503]: 2024-04-23T14:39:49.886+0000 7f02fb400700 -1 osd.0 0 waiting for initial osdmap 2024-04-23T14:39:50.164 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 14:39:49 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0[42503]: 2024-04-23T14:39:49.912+0000 7f02f1200700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:39:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:50 smithi083 ceph-mon[27867]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:50 smithi083 ceph-mon[27867]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-23T14:39:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:50 smithi083 ceph-mon[27867]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T14:39:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:50 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:50 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:50 smithi083 ceph-mon[27867]: from='osd.0 ' entity='osd.0' 2024-04-23T14:39:50.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:50 smithi173 ceph-mon[32443]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:50.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:50 smithi173 ceph-mon[32443]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-23T14:39:50.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:50 smithi173 ceph-mon[32443]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T14:39:50.543 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:50 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:50.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:50 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:50.544 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:50 smithi173 ceph-mon[32443]: from='osd.0 ' entity='osd.0' 2024-04-23T14:39:50.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:50 smithi203 ceph-mon[32800]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:50.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:50 smithi203 ceph-mon[32800]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-23T14:39:50.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:50 smithi203 ceph-mon[32800]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T14:39:50.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:50 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:50.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:50 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:50.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:50 smithi203 ceph-mon[32800]: from='osd.0 ' entity='osd.0' 2024-04-23T14:39:51.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:51 smithi083 ceph-mon[27867]: purged_snaps scrub starts 2024-04-23T14:39:51.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:51 smithi083 ceph-mon[27867]: purged_snaps scrub ok 2024-04-23T14:39:51.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:51 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:51.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:51 smithi083 ceph-mon[27867]: osd.0 [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722] boot 2024-04-23T14:39:51.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:51 smithi083 ceph-mon[27867]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T14:39:51.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:51 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:51.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:51 smithi173 ceph-mon[32443]: purged_snaps scrub starts 2024-04-23T14:39:51.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:51 smithi173 ceph-mon[32443]: purged_snaps scrub ok 2024-04-23T14:39:51.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:51 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:51.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:51 smithi173 ceph-mon[32443]: osd.0 [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722] boot 2024-04-23T14:39:51.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:51 smithi173 ceph-mon[32443]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T14:39:51.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:51 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:51.650 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T14:39:51.650 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T14:39:51.650 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-23T14:39:51.650 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-23T14:39:51.650 INFO:teuthology.orchestra.run.smithi173.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011169 s, 939 MB/s 2024-04-23T14:39:51.650 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-23T14:39:51.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:51 smithi203 ceph-mon[32800]: purged_snaps scrub starts 2024-04-23T14:39:51.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:51 smithi203 ceph-mon[32800]: purged_snaps scrub ok 2024-04-23T14:39:51.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:51.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:51 smithi203 ceph-mon[32800]: osd.0 [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722] boot 2024-04-23T14:39:51.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:51 smithi203 ceph-mon[32800]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T14:39:51.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:39:52.278 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_4 2024-04-23T14:39:52.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:52 smithi083 ceph-mon[27867]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:52.562 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T14:39:52.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:52 smithi173 ceph-mon[32443]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:52.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:52 smithi203 ceph-mon[32800]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T14:39:53.572 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:53 smithi173 ceph-mon[32443]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T14:39:53.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:53 smithi203 ceph-mon[32800]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T14:39:53.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:53 smithi083 ceph-mon[27867]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T14:39:54.074 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.072+0000 7f0804f32700 1 -- 172.21.15.173:0/24498974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08001035d0 msgr2=0x7f08001059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:54.074 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.072+0000 7f0804f32700 1 --2- 172.21.15.173:0/24498974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08001035d0 0x7f08001059c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009990 tx=0x7f07f4009ca0 comp rx=0 tx=0).stop 2024-04-23T14:39:54.074 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.072+0000 7f0804f32700 1 -- 172.21.15.173:0/24498974 shutdown_connections 2024-04-23T14:39:54.074 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.072+0000 7f0804f32700 1 --2- 172.21.15.173:0/24498974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0800108830 0x7f080010aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:54.074 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.072+0000 7f0804f32700 1 --2- 172.21.15.173:0/24498974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08001035d0 0x7f08001059c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:54.074 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.072+0000 7f0804f32700 1 --2- 172.21.15.173:0/24498974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0800105f00 0x7f08001082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:54.074 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.072+0000 7f0804f32700 1 -- 172.21.15.173:0/24498974 >> 172.21.15.173:0/24498974 conn(0x7f08000fcfe0 msgr2=0x7f08000ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:39:54.074 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.072+0000 7f0804f32700 1 -- 172.21.15.173:0/24498974 shutdown_connections 2024-04-23T14:39:54.074 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.072+0000 7f0804f32700 1 -- 172.21.15.173:0/24498974 wait complete. 2024-04-23T14:39:54.076 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.073+0000 7f0804f32700 1 Processor -- start 2024-04-23T14:39:54.076 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.073+0000 7f0804f32700 1 -- start start 2024-04-23T14:39:54.076 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.073+0000 7f0804f32700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08001035d0 0x7f0800198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:54.076 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.073+0000 7f0804f32700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0800105f00 0x7f08001992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.073+0000 7f0804f32700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0800108830 0x7f08001a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.073+0000 7f0804f32700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f080010e790 con 0x7f0800105f00 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.073+0000 7f0804f32700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f080010e610 con 0x7f0800108830 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.073+0000 7f0804f32700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f080010e910 con 0x7f08001035d0 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.073+0000 7f07fed9d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0800108830 0x7f08001a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07fed9d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0800108830 0x7f08001a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:58778/0 (socket says 172.21.15.173:58778) 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07fed9d700 1 -- 172.21.15.173:0/735018894 learned_addr learned my addr 172.21.15.173:0/735018894 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07fe59c700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08001035d0 0x7f0800198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07fdd9b700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0800105f00 0x7f08001992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07fed9d700 1 -- 172.21.15.173:0/735018894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08001035d0 msgr2=0x7f0800198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07fed9d700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08001035d0 0x7f0800198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07fed9d700 1 -- 172.21.15.173:0/735018894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0800105f00 msgr2=0x7f08001992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07fed9d700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0800105f00 0x7f08001992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:39:54.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07fed9d700 1 -- 172.21.15.173:0/735018894 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08001a0a00 con 0x7f0800108830 2024-04-23T14:39:54.078 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07fed9d700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0800108830 0x7f08001a0360 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f07f800b9e0 tx=0x7f07f800bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:54.078 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07ef7fe700 1 -- 172.21.15.173:0/735018894 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8019070 con 0x7f0800108830 2024-04-23T14:39:54.078 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07ef7fe700 1 -- 172.21.15.173:0/735018894 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07f800ce10 con 0x7f0800108830 2024-04-23T14:39:54.078 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f0804f32700 1 -- 172.21.15.173:0/735018894 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08001a0cf0 con 0x7f0800108830 2024-04-23T14:39:54.078 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f07ef7fe700 1 -- 172.21.15.173:0/735018894 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f80123a0 con 0x7f0800108830 2024-04-23T14:39:54.078 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.074+0000 7f0804f32700 1 -- 172.21.15.173:0/735018894 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08001a12a0 con 0x7f0800108830 2024-04-23T14:39:54.079 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.075+0000 7f0804f32700 1 -- 172.21.15.173:0/735018894 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08000697e0 con 0x7f0800108830 2024-04-23T14:39:54.079 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.076+0000 7f07ef7fe700 1 -- 172.21.15.173:0/735018894 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f07f801c070 con 0x7f0800108830 2024-04-23T14:39:54.079 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.077+0000 7f07ef7fe700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f07e80a3c30 0x7f07e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:39:54.079 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.077+0000 7f07ef7fe700 1 -- 172.21.15.173:0/735018894 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f07f80cdb70 con 0x7f0800108830 2024-04-23T14:39:54.079 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.077+0000 7f07fe59c700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f07e80a3c30 0x7f07e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:39:54.079 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.078+0000 7f07fe59c700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f07e80a3c30 0x7f07e80a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f07f400b5c0 tx=0x7f07f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:39:54.083 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.082+0000 7f07ef7fe700 1 -- 172.21.15.173:0/735018894 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07f809bfc0 con 0x7f0800108830 2024-04-23T14:39:54.293 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:39:54.292+0000 7f0804f32700 1 -- 172.21.15.173:0/735018894 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0800062f30 con 0x7f07e80a3c30 2024-04-23T14:39:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:54 smithi173 ceph-mon[32443]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:39:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:39:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:54.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:54 smithi203 ceph-mon[32800]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:39:54.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:54.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:54.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:39:54.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:54.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:54.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:54.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:54 smithi083 ceph-mon[27867]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:39:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:39:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:39:54.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:39:55.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:55 smithi173 ceph-mon[32443]: Detected new or changed devices on smithi083 2024-04-23T14:39:55.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:55 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi083 to 17089M 2024-04-23T14:39:55.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:55 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:39:55.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:55 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:39:55.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:55 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:55 smithi203 ceph-mon[32800]: Detected new or changed devices on smithi083 2024-04-23T14:39:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:55 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi083 to 17089M 2024-04-23T14:39:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:55 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:39:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:55 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:39:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:55 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:55.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:55 smithi083 ceph-mon[27867]: Detected new or changed devices on smithi083 2024-04-23T14:39:55.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:55 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi083 to 17089M 2024-04-23T14:39:55.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:55 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:39:55.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:55 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:39:55.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:55 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:39:56.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:56 smithi173 ceph-mon[32443]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:39:56.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:56 smithi173 ceph-mon[32443]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:39:56.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:56 smithi203 ceph-mon[32800]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:39:56.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:56 smithi203 ceph-mon[32800]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:39:56.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:56 smithi083 ceph-mon[27867]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:39:56.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:56 smithi083 ceph-mon[27867]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:39:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:57 smithi173 ceph-mon[32443]: from='client.? 172.21.15.173:0/3570583235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d1440d-148d-4828-8607-63ebd9076ccb"}]: dispatch 2024-04-23T14:39:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:57 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d1440d-148d-4828-8607-63ebd9076ccb"}]: dispatch 2024-04-23T14:39:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:57 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50d1440d-148d-4828-8607-63ebd9076ccb"}]': finished 2024-04-23T14:39:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:57 smithi173 ceph-mon[32443]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T14:39:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:39:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:57 smithi173 ceph-mon[32443]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:39:57.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:57 smithi083 ceph-mon[27867]: from='client.? 172.21.15.173:0/3570583235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d1440d-148d-4828-8607-63ebd9076ccb"}]: dispatch 2024-04-23T14:39:57.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:57 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d1440d-148d-4828-8607-63ebd9076ccb"}]: dispatch 2024-04-23T14:39:57.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:57 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50d1440d-148d-4828-8607-63ebd9076ccb"}]': finished 2024-04-23T14:39:57.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:57 smithi083 ceph-mon[27867]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T14:39:57.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:39:57.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:57 smithi083 ceph-mon[27867]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:39:57.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:57 smithi203 ceph-mon[32800]: from='client.? 172.21.15.173:0/3570583235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d1440d-148d-4828-8607-63ebd9076ccb"}]: dispatch 2024-04-23T14:39:57.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:57 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50d1440d-148d-4828-8607-63ebd9076ccb"}]: dispatch 2024-04-23T14:39:57.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:57 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50d1440d-148d-4828-8607-63ebd9076ccb"}]': finished 2024-04-23T14:39:57.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:57 smithi203 ceph-mon[32800]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T14:39:57.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:39:57.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:57 smithi203 ceph-mon[32800]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:39:58.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:58 smithi173 ceph-mon[32443]: from='client.? 172.21.15.173:0/4091385968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:39:58.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:58 smithi083 ceph-mon[27867]: from='client.? 172.21.15.173:0/4091385968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:39:58.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:58 smithi203 ceph-mon[32800]: from='client.? 172.21.15.173:0/4091385968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:39:59.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:39:59 smithi173 ceph-mon[32443]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:39:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:39:59 smithi203 ceph-mon[32800]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:39:59.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:39:59 smithi083 ceph-mon[27867]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:00.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:00 smithi083 ceph-mon[27867]: overall HEALTH_OK 2024-04-23T14:40:00.524 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:00 smithi173 ceph-mon[32443]: overall HEALTH_OK 2024-04-23T14:40:00.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:00 smithi203 ceph-mon[32800]: overall HEALTH_OK 2024-04-23T14:40:01.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:01 smithi173 ceph-mon[32443]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:01.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:01 smithi203 ceph-mon[32800]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:01.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:01 smithi083 ceph-mon[27867]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:04.424 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:04 smithi203 ceph-mon[32800]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:04.424 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:04 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T14:40:04.425 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:04 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:04.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:04 smithi083 ceph-mon[27867]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:04.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:04 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T14:40:04.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:04 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:04 smithi173 ceph-mon[32443]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:04 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T14:40:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:04 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:05.281 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:05 smithi173 ceph-mon[32443]: Deploying daemon osd.1 on smithi173 2024-04-23T14:40:05.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:05 smithi203 ceph-mon[32800]: Deploying daemon osd.1 on smithi173 2024-04-23T14:40:05.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:05 smithi083 ceph-mon[27867]: Deploying daemon osd.1 on smithi173 2024-04-23T14:40:06.262 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:06 smithi173 ceph-mon[32443]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:06.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:06 smithi203 ceph-mon[32800]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:06.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:06 smithi083 ceph-mon[27867]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:08.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:08 smithi203 ceph-mon[32800]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:08.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:08 smithi083 ceph-mon[27867]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:08 smithi173 ceph-mon[32443]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:10.234 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:10 smithi173 ceph-mon[32443]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:10.234 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:10 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:10.235 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:10 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:10.235 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:10 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:40:10.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:10 smithi203 ceph-mon[32800]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:10.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:10 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:10.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:10 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:10.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:10 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:40:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:10 smithi083 ceph-mon[27867]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:10 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:10 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:10 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:40:12.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:12 smithi173 ceph-mon[32443]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:12.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:12.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:12.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:12.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:40:12.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:12.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:12 smithi203 ceph-mon[32800]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:12.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:12.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:12.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:12.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:40:12.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:12 smithi083 ceph-mon[27867]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:40:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:13.075 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.074+0000 7f07ef7fe700 1 -- 172.21.15.173:0/735018894 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0800062f30 con 0x7f07e80a3c30 2024-04-23T14:40:13.075 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 1 on host 'smithi173' 2024-04-23T14:40:13.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.076+0000 7f0804f32700 1 -- 172.21.15.173:0/735018894 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f07e80a3c30 msgr2=0x7f07e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.076+0000 7f0804f32700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f07e80a3c30 0x7f07e80a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f07f400b5c0 tx=0x7f07f4005fd0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.076+0000 7f0804f32700 1 -- 172.21.15.173:0/735018894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0800108830 msgr2=0x7f08001a0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:13.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.076+0000 7f0804f32700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0800108830 0x7f08001a0360 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f07f800b9e0 tx=0x7f07f800bcf0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.076+0000 7f0804f32700 1 -- 172.21.15.173:0/735018894 shutdown_connections 2024-04-23T14:40:13.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.076+0000 7f0804f32700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f07e80a3c30 0x7f07e80a60f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.076+0000 7f0804f32700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0800108830 0x7f08001a0360 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.076+0000 7f0804f32700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08001035d0 0x7f0800198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.076+0000 7f0804f32700 1 --2- 172.21.15.173:0/735018894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0800105f00 0x7f08001992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:13.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.076+0000 7f0804f32700 1 -- 172.21.15.173:0/735018894 >> 172.21.15.173:0/735018894 conn(0x7f08000fcfe0 msgr2=0x7f0800103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:13.077 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.076+0000 7f0804f32700 1 -- 172.21.15.173:0/735018894 shutdown_connections 2024-04-23T14:40:13.078 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:40:13.076+0000 7f0804f32700 1 -- 172.21.15.173:0/735018894 wait complete. 2024-04-23T14:40:13.668 DEBUG:teuthology.orchestra.run.smithi173:osd.1> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.1.service 2024-04-23T14:40:13.670 INFO:tasks.cephadm:Deploying osd.2 on smithi203 with /dev/vg_nvme/lv_4... 2024-04-23T14:40:13.670 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T14:40:13.923 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:40:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:14 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:14 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:40:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:14 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:14 smithi173 ceph-mon[32443]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:14 smithi173 ceph-mon[32443]: from='osd.1 [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T14:40:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:14 smithi173 ceph-mon[32443]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T14:40:14.349 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:14 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:14.349 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:14 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:40:14.349 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:14 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:14.349 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:14 smithi203 ceph-mon[32800]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:14.349 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:14 smithi203 ceph-mon[32800]: from='osd.1 [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T14:40:14.349 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:14 smithi203 ceph-mon[32800]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T14:40:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:14 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:14 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:40:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:14 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:14 smithi083 ceph-mon[27867]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:14 smithi083 ceph-mon[27867]: from='osd.1 [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T14:40:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:14 smithi083 ceph-mon[27867]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T14:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:15 smithi203 ceph-mon[32800]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T14:40:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:15 smithi203 ceph-mon[32800]: from='osd.1 [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:15 smithi203 ceph-mon[32800]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T14:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:40:15.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:15 smithi203 ceph-mon[32800]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:40:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:15 smithi173 ceph-mon[32443]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T14:40:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:15 smithi173 ceph-mon[32443]: from='osd.1 [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:40:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:15 smithi173 ceph-mon[32443]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T14:40:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:40:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:15 smithi173 ceph-mon[32443]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:40:15.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:15 smithi083 ceph-mon[27867]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T14:40:15.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:15 smithi083 ceph-mon[27867]: from='osd.1 [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:40:15.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:15 smithi083 ceph-mon[27867]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T14:40:15.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:40:15.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:15 smithi083 ceph-mon[27867]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:40:15.786 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T14:40:15.786 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T14:40:15.786 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-04-23T14:40:15.786 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-04-23T14:40:15.786 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0142829 s, 734 MB/s 2024-04-23T14:40:15.786 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-04-23T14:40:15.826 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 14:40:15 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1[37018]: 2024-04-23T14:40:15.506+0000 7f8992afd700 -1 osd.1 0 waiting for initial osdmap 2024-04-23T14:40:15.826 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 14:40:15 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1[37018]: 2024-04-23T14:40:15.511+0000 7f898a8e8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:40:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:16 smithi173 ceph-mon[32443]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T14:40:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:16 smithi173 ceph-mon[32443]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T14:40:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:16 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:40:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:16 smithi173 ceph-mon[32443]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:16 smithi173 ceph-mon[32443]: from='osd.1 ' entity='osd.1' 2024-04-23T14:40:16.376 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:16 smithi203 ceph-mon[32800]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T14:40:16.376 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:16 smithi203 ceph-mon[32800]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T14:40:16.377 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:16 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:40:16.377 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:16 smithi203 ceph-mon[32800]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:16.377 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:16 smithi203 ceph-mon[32800]: from='osd.1 ' entity='osd.1' 2024-04-23T14:40:16.403 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_4 2024-04-23T14:40:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:16 smithi083 ceph-mon[27867]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T14:40:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:16 smithi083 ceph-mon[27867]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T14:40:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:16 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:40:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:16 smithi083 ceph-mon[27867]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T14:40:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:16 smithi083 ceph-mon[27867]: from='osd.1 ' entity='osd.1' 2024-04-23T14:40:16.640 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:40:17.433 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: purged_snaps scrub starts 2024-04-23T14:40:17.433 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: purged_snaps scrub ok 2024-04-23T14:40:17.433 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:40:17.433 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: osd.1 [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483] boot 2024-04-23T14:40:17.433 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T14:40:17.433 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:40:17.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:17.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:17.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:40:17.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:17.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:17.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:40:17.434 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:17 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:17.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: purged_snaps scrub starts 2024-04-23T14:40:17.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: purged_snaps scrub ok 2024-04-23T14:40:17.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:40:17.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: osd.1 [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483] boot 2024-04-23T14:40:17.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T14:40:17.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:40:17.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:17.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:17.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:40:17.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:17.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:17.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:40:17.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:17 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:17.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: purged_snaps scrub starts 2024-04-23T14:40:17.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: purged_snaps scrub ok 2024-04-23T14:40:17.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:40:17.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: osd.1 [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483] boot 2024-04-23T14:40:17.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T14:40:17.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:40:17.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:17.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:17.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:40:17.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:17.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:17.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:40:17.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:17 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:18.282 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.278+0000 7f59b36ef700 1 -- 172.21.15.203:0/2102653304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ac102b50 msgr2=0x7f59ac102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:18.282 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.278+0000 7f59b36ef700 1 --2- 172.21.15.203:0/2102653304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ac102b50 0x7f59ac102f70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009990 tx=0x7f59a8009ca0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.282 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.278+0000 7f59b36ef700 1 -- 172.21.15.203:0/2102653304 shutdown_connections 2024-04-23T14:40:18.282 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.278+0000 7f59b36ef700 1 --2- 172.21.15.203:0/2102653304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ac1046f0 0x7f59ac10afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.282 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.278+0000 7f59b36ef700 1 --2- 172.21.15.203:0/2102653304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ac102b50 0x7f59ac102f70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.282 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.278+0000 7f59b36ef700 1 --2- 172.21.15.203:0/2102653304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59ac103d50 0x7f59ac1041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.278+0000 7f59b36ef700 1 -- 172.21.15.203:0/2102653304 >> 172.21.15.203:0/2102653304 conn(0x7f59ac0fe0f0 msgr2=0x7f59ac100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.278+0000 7f59b36ef700 1 -- 172.21.15.203:0/2102653304 shutdown_connections 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.278+0000 7f59b36ef700 1 -- 172.21.15.203:0/2102653304 wait complete. 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.279+0000 7f59b36ef700 1 Processor -- start 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.279+0000 7f59b36ef700 1 -- start start 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.279+0000 7f59b36ef700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ac102b50 0x7f59ac198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.279+0000 7f59b36ef700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ac103d50 0x7f59ac199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.279+0000 7f59b36ef700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59ac1046f0 0x7f59ac1a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.279+0000 7f59b36ef700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac10e810 con 0x7f59ac103d50 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.279+0000 7f59b36ef700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac10e690 con 0x7f59ac1046f0 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.279+0000 7f59b36ef700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac10e990 con 0x7f59ac102b50 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.279+0000 7f59b148b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ac102b50 0x7f59ac198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.279+0000 7f59b148b700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ac102b50 0x7f59ac198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:44790/0 (socket says 172.21.15.203:44790) 2024-04-23T14:40:18.283 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.279+0000 7f59b148b700 1 -- 172.21.15.203:0/2075661455 learned_addr learned my addr 172.21.15.203:0/2075661455 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-23T14:40:18.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.280+0000 7f59b1c8c700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59ac1046f0 0x7f59ac1a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:18.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.280+0000 7f59b148b700 1 -- 172.21.15.203:0/2075661455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59ac1046f0 msgr2=0x7f59ac1a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:18.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.280+0000 7f59b148b700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59ac1046f0 0x7f59ac1a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.280+0000 7f59b148b700 1 -- 172.21.15.203:0/2075661455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ac103d50 msgr2=0x7f59ac199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:18.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.280+0000 7f59b148b700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ac103d50 0x7f59ac199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:18.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.280+0000 7f59b148b700 1 -- 172.21.15.203:0/2075661455 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac1a0b50 con 0x7f59ac102b50 2024-04-23T14:40:18.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.280+0000 7f59b148b700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ac102b50 0x7f59ac198f40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f59a80047b0 tx=0x7f59a8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:18.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.280+0000 7f599e7fc700 1 -- 172.21.15.203:0/2075661455 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8027070 con 0x7f59ac102b50 2024-04-23T14:40:18.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.280+0000 7f599e7fc700 1 -- 172.21.15.203:0/2075661455 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a8022470 con 0x7f59ac102b50 2024-04-23T14:40:18.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.280+0000 7f59b36ef700 1 -- 172.21.15.203:0/2075661455 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ac1a0de0 con 0x7f59ac102b50 2024-04-23T14:40:18.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.280+0000 7f599e7fc700 1 -- 172.21.15.203:0/2075661455 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a800f6c0 con 0x7f59ac102b50 2024-04-23T14:40:18.284 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.280+0000 7f59b36ef700 1 -- 172.21.15.203:0/2075661455 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ac1a1200 con 0x7f59ac102b50 2024-04-23T14:40:18.285 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.281+0000 7f59b36ef700 1 -- 172.21.15.203:0/2075661455 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ac0697e0 con 0x7f59ac102b50 2024-04-23T14:40:18.285 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.282+0000 7f599e7fc700 1 -- 172.21.15.203:0/2075661455 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f59a802a070 con 0x7f59ac102b50 2024-04-23T14:40:18.286 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.283+0000 7f599e7fc700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f59980a3bb0 0x7f59980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:18.286 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.283+0000 7f599e7fc700 1 -- 172.21.15.203:0/2075661455 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f59a80cfc70 con 0x7f59ac102b50 2024-04-23T14:40:18.286 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.283+0000 7f59b0c8a700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f59980a3bb0 0x7f59980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:18.286 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.284+0000 7f59b0c8a700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f59980a3bb0 0x7f59980a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59a00097b0 tx=0x7f59a0006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:18.289 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.288+0000 7f599e7fc700 1 -- 172.21.15.203:0/2075661455 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59a805eff0 con 0x7f59ac102b50 2024-04-23T14:40:18.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:18 smithi203 ceph-mon[32800]: Detected new or changed devices on smithi173 2024-04-23T14:40:18.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:18 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi173 to 17088M 2024-04-23T14:40:18.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:18 smithi203 ceph-mon[32800]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:18.478 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:18 smithi173 ceph-mon[32443]: Detected new or changed devices on smithi173 2024-04-23T14:40:18.478 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:18 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi173 to 17088M 2024-04-23T14:40:18.478 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:18 smithi173 ceph-mon[32443]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:18.495 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:18.493+0000 7f59b36ef700 1 -- 172.21.15.203:0/2075661455 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f59ac062f30 con 0x7f59980a3bb0 2024-04-23T14:40:18.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:18 smithi083 ceph-mon[27867]: Detected new or changed devices on smithi173 2024-04-23T14:40:18.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:18 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi173 to 17088M 2024-04-23T14:40:18.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:18 smithi083 ceph-mon[27867]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:19.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:19 smithi203 ceph-mon[32800]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T14:40:19.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:19 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:40:19.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:19 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:40:19.432 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:19 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:19.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:19 smithi083 ceph-mon[27867]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T14:40:19.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:40:19.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:40:19.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:19.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:19 smithi173 ceph-mon[32443]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T14:40:19.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:19 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:40:19.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:19 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:40:19.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:19 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:20.307 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:20 smithi203 ceph-mon[32800]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:40:20.308 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:20 smithi203 ceph-mon[32800]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:20.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:20 smithi083 ceph-mon[27867]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:40:20.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:20 smithi083 ceph-mon[27867]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:20 smithi173 ceph-mon[32443]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:40:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:20 smithi173 ceph-mon[32443]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:21.460 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:21 smithi203 ceph-mon[32800]: from='client.? 172.21.15.203:0/2404584306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7fd9255-ac0b-47bb-813f-d29f1ffa131b"}]: dispatch 2024-04-23T14:40:21.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:21 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7fd9255-ac0b-47bb-813f-d29f1ffa131b"}]: dispatch 2024-04-23T14:40:21.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:21 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7fd9255-ac0b-47bb-813f-d29f1ffa131b"}]': finished 2024-04-23T14:40:21.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:21 smithi203 ceph-mon[32800]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T14:40:21.461 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:21 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:21.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:21 smithi083 ceph-mon[27867]: from='client.? 172.21.15.203:0/2404584306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7fd9255-ac0b-47bb-813f-d29f1ffa131b"}]: dispatch 2024-04-23T14:40:21.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:21 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7fd9255-ac0b-47bb-813f-d29f1ffa131b"}]: dispatch 2024-04-23T14:40:21.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:21 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7fd9255-ac0b-47bb-813f-d29f1ffa131b"}]': finished 2024-04-23T14:40:21.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:21 smithi083 ceph-mon[27867]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T14:40:21.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:21 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:21 smithi173 ceph-mon[32443]: from='client.? 172.21.15.203:0/2404584306' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7fd9255-ac0b-47bb-813f-d29f1ffa131b"}]: dispatch 2024-04-23T14:40:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:21 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7fd9255-ac0b-47bb-813f-d29f1ffa131b"}]: dispatch 2024-04-23T14:40:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:21 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7fd9255-ac0b-47bb-813f-d29f1ffa131b"}]': finished 2024-04-23T14:40:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:21 smithi173 ceph-mon[32443]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T14:40:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:21 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:22.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:22 smithi203 ceph-mon[32800]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:22.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:22 smithi203 ceph-mon[32800]: from='client.? 172.21.15.203:0/2696487515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:40:22.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:22 smithi083 ceph-mon[27867]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:22.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:22 smithi083 ceph-mon[27867]: from='client.? 172.21.15.203:0/2696487515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:40:22.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:22 smithi173 ceph-mon[32443]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:22.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:22 smithi173 ceph-mon[32443]: from='client.? 172.21.15.203:0/2696487515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:40:24.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:24 smithi203 ceph-mon[32800]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:24.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:24 smithi083 ceph-mon[27867]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:24.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:24 smithi173 ceph-mon[32443]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:26.332 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:26 smithi203 ceph-mon[32800]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:26.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:26 smithi083 ceph-mon[27867]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:26 smithi173 ceph-mon[32443]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:28.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:28 smithi203 ceph-mon[32800]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:28.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:28 smithi083 ceph-mon[27867]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:28.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:28 smithi173 ceph-mon[32443]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:29.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:29 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T14:40:29.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:29 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:29.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:29 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T14:40:29.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:29 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:29.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:29 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T14:40:29.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:29 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:30.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:30 smithi203 ceph-mon[32800]: Deploying daemon osd.2 on smithi203 2024-04-23T14:40:30.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:30 smithi203 ceph-mon[32800]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:30.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:30 smithi083 ceph-mon[27867]: Deploying daemon osd.2 on smithi203 2024-04-23T14:40:30.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:30 smithi083 ceph-mon[27867]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:30 smithi173 ceph-mon[32443]: Deploying daemon osd.2 on smithi203 2024-04-23T14:40:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:30 smithi173 ceph-mon[32443]: pgmap v81: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:32.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:32 smithi203 ceph-mon[32800]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:32.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:32 smithi083 ceph-mon[27867]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:32.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:32 smithi173 ceph-mon[32443]: pgmap v82: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:34.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:34 smithi203 ceph-mon[32800]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:34.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:34.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:40:34.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:34 smithi083 ceph-mon[27867]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:40:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:34 smithi173 ceph-mon[32443]: pgmap v83: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:40:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:36.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:36 smithi203 ceph-mon[32800]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:36.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:36 smithi083 ceph-mon[27867]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:36 smithi173 ceph-mon[32443]: pgmap v84: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:37.780 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 2 on host 'smithi203' 2024-04-23T14:40:37.781 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.780+0000 7f599e7fc700 1 -- 172.21.15.203:0/2075661455 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f59ac062f30 con 0x7f59980a3bb0 2024-04-23T14:40:37.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.782+0000 7f59b36ef700 1 -- 172.21.15.203:0/2075661455 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f59980a3bb0 msgr2=0x7f59980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:37.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.782+0000 7f59b36ef700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f59980a3bb0 0x7f59980a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59a00097b0 tx=0x7f59a0006d20 comp rx=0 tx=0).stop 2024-04-23T14:40:37.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.782+0000 7f59b36ef700 1 -- 172.21.15.203:0/2075661455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ac102b50 msgr2=0x7f59ac198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:37.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.782+0000 7f59b36ef700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ac102b50 0x7f59ac198f40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f59a80047b0 tx=0x7f59a8009f90 comp rx=0 tx=0).stop 2024-04-23T14:40:37.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.782+0000 7f59b36ef700 1 -- 172.21.15.203:0/2075661455 shutdown_connections 2024-04-23T14:40:37.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.782+0000 7f59b36ef700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f59980a3bb0 0x7f59980a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:37.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.782+0000 7f59b36ef700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59ac1046f0 0x7f59ac1a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:37.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.782+0000 7f59b36ef700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59ac102b50 0x7f59ac198f40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:37.784 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.782+0000 7f59b36ef700 1 --2- 172.21.15.203:0/2075661455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59ac103d50 0x7f59ac199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:37.784 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.782+0000 7f59b36ef700 1 -- 172.21.15.203:0/2075661455 >> 172.21.15.203:0/2075661455 conn(0x7f59ac0fe0f0 msgr2=0x7f59ac1071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:37.784 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.782+0000 7f59b36ef700 1 -- 172.21.15.203:0/2075661455 shutdown_connections 2024-04-23T14:40:37.784 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:40:37.782+0000 7f59b36ef700 1 -- 172.21.15.203:0/2075661455 wait complete. 2024-04-23T14:40:37.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:37 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:37.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:37 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:37.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:37 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:37.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:37 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:40:37.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:37 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:37.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:37 smithi203 ceph-mon[32800]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:37.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:37 smithi203 ceph-mon[32800]: from='osd.2 [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T14:40:37.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:37 smithi203 ceph-mon[32800]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T14:40:37.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:37 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:37.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:37 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:37.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:37 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:40:38.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:37 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:38.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:37 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:38.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:37 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:38.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:37 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:40:38.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:37 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:38.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:37 smithi173 ceph-mon[32443]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:38.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:37 smithi173 ceph-mon[32443]: from='osd.2 [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T14:40:38.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:37 smithi173 ceph-mon[32443]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T14:40:38.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:37 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:38.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:37 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:38.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:37 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:40:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:37 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:37 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:37 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:37 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:40:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:37 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:37 smithi083 ceph-mon[27867]: pgmap v85: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:37 smithi083 ceph-mon[27867]: from='osd.2 [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T14:40:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:37 smithi083 ceph-mon[27867]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T14:40:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:37 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:37 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:37 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:40:38.447 DEBUG:teuthology.orchestra.run.smithi203:osd.2> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.2.service 2024-04-23T14:40:38.450 INFO:tasks.cephadm:Deploying osd.3 on smithi083 with /dev/vg_nvme/lv_3... 2024-04-23T14:40:38.450 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T14:40:38.897 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:40:38.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:38 smithi203 ceph-mon[32800]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T14:40:38.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:38 smithi203 ceph-mon[32800]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T14:40:38.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:38.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:38 smithi203 ceph-mon[32800]: from='osd.2 [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:40:38.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:38 smithi203 ceph-mon[32800]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:40:39.109 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:38 smithi083 ceph-mon[27867]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T14:40:39.110 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:38 smithi083 ceph-mon[27867]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T14:40:39.110 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:39.110 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:38 smithi083 ceph-mon[27867]: from='osd.2 [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:40:39.110 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:38 smithi083 ceph-mon[27867]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:40:39.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:38 smithi173 ceph-mon[32443]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T14:40:39.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:38 smithi173 ceph-mon[32443]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T14:40:39.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:39.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:38 smithi173 ceph-mon[32443]: from='osd.2 [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:40:39.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:38 smithi173 ceph-mon[32443]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:40:39.722 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 14:40:39 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2[37959]: 2024-04-23T14:40:39.325+0000 7f687f9a2700 -1 osd.2 0 waiting for initial osdmap 2024-04-23T14:40:39.722 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 14:40:39 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2[37959]: 2024-04-23T14:40:39.329+0000 7f687778d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:40:40.121 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:39 smithi083 ceph-mon[27867]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-04-23T14:40:40.121 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:39 smithi083 ceph-mon[27867]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T14:40:40.121 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:40.121 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:40.121 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:39 smithi083 ceph-mon[27867]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:40.121 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:39 smithi083 ceph-mon[27867]: from='osd.2 ' entity='osd.2' 2024-04-23T14:40:40.134 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:39 smithi203 ceph-mon[32800]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-04-23T14:40:40.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:39 smithi203 ceph-mon[32800]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T14:40:40.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:39 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:40.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:39 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:40.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:39 smithi203 ceph-mon[32800]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:40.135 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:39 smithi203 ceph-mon[32800]: from='osd.2 ' entity='osd.2' 2024-04-23T14:40:40.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:39 smithi173 ceph-mon[32443]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-04-23T14:40:40.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:39 smithi173 ceph-mon[32443]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T14:40:40.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:39 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:40.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:39 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:40.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:39 smithi173 ceph-mon[32443]: pgmap v88: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T14:40:40.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:39 smithi173 ceph-mon[32443]: from='osd.2 ' entity='osd.2' 2024-04-23T14:40:41.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:40 smithi203 ceph-mon[32800]: purged_snaps scrub starts 2024-04-23T14:40:41.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:40 smithi203 ceph-mon[32800]: purged_snaps scrub ok 2024-04-23T14:40:41.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:40 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:41.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:40 smithi203 ceph-mon[32800]: osd.2 [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379] boot 2024-04-23T14:40:41.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:40 smithi203 ceph-mon[32800]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T14:40:41.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:40 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:41.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:40 smithi083 ceph-mon[27867]: purged_snaps scrub starts 2024-04-23T14:40:41.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:40 smithi083 ceph-mon[27867]: purged_snaps scrub ok 2024-04-23T14:40:41.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:40 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:41.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:40 smithi083 ceph-mon[27867]: osd.2 [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379] boot 2024-04-23T14:40:41.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:40 smithi083 ceph-mon[27867]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T14:40:41.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:40 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:41.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:40 smithi173 ceph-mon[32443]: purged_snaps scrub starts 2024-04-23T14:40:41.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:40 smithi173 ceph-mon[32443]: purged_snaps scrub ok 2024-04-23T14:40:41.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:40 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:41.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:40 smithi173 ceph-mon[32443]: osd.2 [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379] boot 2024-04-23T14:40:41.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:40 smithi173 ceph-mon[32443]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T14:40:41.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:40 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:40:41.805 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T14:40:41.805 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T14:40:41.805 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-23T14:40:41.805 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-23T14:40:41.805 INFO:teuthology.orchestra.run.smithi083.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.058474 s, 179 MB/s 2024-04-23T14:40:41.805 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 2024-04-23T14:40:41.805 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-23T14:40:41.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:41 smithi203 ceph-mon[32800]: pgmap v90: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:41.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T14:40:41.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T14:40:41.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:41 smithi203 ceph-mon[32800]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T14:40:41.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T14:40:41.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:41.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:41.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:40:41.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:41.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:41.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:40:41.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:42.114 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:41 smithi083 ceph-mon[27867]: pgmap v90: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:42.114 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T14:40:42.114 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T14:40:42.114 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:41 smithi083 ceph-mon[27867]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T14:40:42.115 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T14:40:42.115 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:42.115 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:42.115 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:40:42.115 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:42.115 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:42.115 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:40:42.115 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:42.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:41 smithi173 ceph-mon[32443]: pgmap v90: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:42.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T14:40:42.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T14:40:42.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:41 smithi173 ceph-mon[32443]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T14:40:42.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T14:40:42.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:42.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:42.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:40:42.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:42.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:42.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:40:42.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:42.658 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_3 2024-04-23T14:40:42.705 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46576]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T14:40:42.705 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46576]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T14:40:42.705 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46576]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T14:40:42.705 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46576]: pam_unix(sudo:session): session closed for user root 2024-04-23T14:40:42.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 ceph-mon[27867]: Detected new or changed devices on smithi203 2024-04-23T14:40:42.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi203 to 17088M 2024-04-23T14:40:42.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T14:40:42.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 ceph-mon[27867]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T14:40:42.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:42.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:42.962 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46604]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T14:40:42.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46604]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T14:40:42.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46604]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T14:40:42.963 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46604]: pam_unix(sudo:session): session closed for user root 2024-04-23T14:40:42.963 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46595]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T14:40:42.963 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46595]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T14:40:42.963 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46595]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T14:40:42.963 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46595]: pam_unix(sudo:session): session closed for user root 2024-04-23T14:40:43.049 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 14:40:42 smithi173 sudo[41154]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T14:40:43.050 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 14:40:42 smithi173 sudo[41154]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T14:40:43.050 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 14:40:42 smithi173 sudo[41154]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T14:40:43.050 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 14:40:42 smithi173 sudo[41154]: pam_unix(sudo:session): session closed for user root 2024-04-23T14:40:43.050 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 14:40:42 smithi173 sudo[41157]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T14:40:43.050 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 14:40:42 smithi173 sudo[41157]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T14:40:43.050 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 14:40:42 smithi173 sudo[41157]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T14:40:43.050 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 14:40:42 smithi173 sudo[41157]: pam_unix(sudo:session): session closed for user root 2024-04-23T14:40:43.050 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:42 smithi173 ceph-mon[32443]: Detected new or changed devices on smithi203 2024-04-23T14:40:43.050 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:42 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi203 to 17088M 2024-04-23T14:40:43.051 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T14:40:43.051 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:42 smithi173 ceph-mon[32443]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T14:40:43.051 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:43.051 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:43.091 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:40:43.222 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 14:40:42 smithi203 sudo[41589]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T14:40:43.222 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 14:40:42 smithi203 sudo[41589]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T14:40:43.222 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 14:40:42 smithi203 sudo[41589]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T14:40:43.222 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 14:40:42 smithi203 sudo[41589]: pam_unix(sudo:session): session closed for user root 2024-04-23T14:40:43.222 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 14:40:42 smithi203 sudo[41592]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T14:40:43.222 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 14:40:42 smithi203 sudo[41592]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T14:40:43.222 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 14:40:42 smithi203 sudo[41592]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T14:40:43.223 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 14:40:42 smithi203 sudo[41592]: pam_unix(sudo:session): session closed for user root 2024-04-23T14:40:43.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:42 smithi203 ceph-mon[32800]: Detected new or changed devices on smithi203 2024-04-23T14:40:43.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:42 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi203 to 17088M 2024-04-23T14:40:43.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T14:40:43.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:42 smithi203 ceph-mon[32800]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T14:40:43.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:43.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:43.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 sudo[41595]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T14:40:43.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 sudo[41595]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T14:40:43.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 sudo[41595]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T14:40:43.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 sudo[41595]: pam_unix(sudo:session): session closed for user root 2024-04-23T14:40:43.224 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 sudo[41598]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T14:40:43.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 sudo[41598]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T14:40:43.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 sudo[41598]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T14:40:43.225 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 sudo[41598]: pam_unix(sudo:session): session closed for user root 2024-04-23T14:40:43.238 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46618]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T14:40:43.239 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46618]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T14:40:43.239 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46618]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T14:40:43.239 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:42 smithi083 sudo[46618]: pam_unix(sudo:session): session closed for user root 2024-04-23T14:40:43.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 sudo[41160]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T14:40:43.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 sudo[41160]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T14:40:43.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 sudo[41160]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T14:40:43.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 sudo[41160]: pam_unix(sudo:session): session closed for user root 2024-04-23T14:40:43.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 sudo[41163]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T14:40:43.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 sudo[41163]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T14:40:43.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 sudo[41163]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T14:40:43.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 sudo[41163]: pam_unix(sudo:session): session closed for user root 2024-04-23T14:40:44.180 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:44.180 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T14:40:44.180 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T14:40:44.180 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:40:44.180 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:40:44.180 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:40:44.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T14:40:44.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:40:44.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:40:44.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:40:44.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T14:40:44.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T14:40:44.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:40:44.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:40:44.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:40:44.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T14:40:44.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:44.181 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:43 smithi083 ceph-mon[27867]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T14:40:44.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:40:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:40:44.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:40:44.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T14:40:44.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:44.223 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:43 smithi203 ceph-mon[32800]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T14:40:44.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:40:44.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:40:44.329 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:40:44.329 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T14:40:44.329 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:44.329 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:43 smithi173 ceph-mon[32443]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T14:40:45.194 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:44 smithi083 ceph-mon[27867]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T14:40:45.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:44 smithi203 ceph-mon[32800]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T14:40:45.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:44 smithi173 ceph-mon[32443]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T14:40:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.362+0000 7fbb99a00700 1 -- 172.21.15.83:0/361616423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb94104c70 msgr2=0x7fbb94105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.362+0000 7fbb99a00700 1 --2- 172.21.15.83:0/361616423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb94104c70 0x7fbb94105090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbb840099c0 tx=0x7fbb84009cd0 comp rx=0 tx=0).stop 2024-04-23T14:40:45.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.362+0000 7fbb99a00700 1 -- 172.21.15.83:0/361616423 shutdown_connections 2024-04-23T14:40:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.362+0000 7fbb99a00700 1 --2- 172.21.15.83:0/361616423 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb94106810 0x7fbb9410d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.362+0000 7fbb99a00700 1 --2- 172.21.15.83:0/361616423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb94104c70 0x7fbb94105090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.362+0000 7fbb99a00700 1 --2- 172.21.15.83:0/361616423 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb94105e70 0x7fbb941062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.362+0000 7fbb99a00700 1 -- 172.21.15.83:0/361616423 >> 172.21.15.83:0/361616423 conn(0x7fbb94100230 msgr2=0x7fbb94102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:40:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.363+0000 7fbb99a00700 1 -- 172.21.15.83:0/361616423 shutdown_connections 2024-04-23T14:40:45.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.363+0000 7fbb99a00700 1 -- 172.21.15.83:0/361616423 wait complete. 2024-04-23T14:40:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.364+0000 7fbb99a00700 1 Processor -- start 2024-04-23T14:40:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.364+0000 7fbb99a00700 1 -- start start 2024-04-23T14:40:45.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.364+0000 7fbb99a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb94104c70 0x7fbb9419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.364+0000 7fbb99a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb94105e70 0x7fbb9419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.364+0000 7fbb99a00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb94106810 0x7fbb941a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.364+0000 7fbb99a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94110850 con 0x7fbb94105e70 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.364+0000 7fbb99a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941106d0 con 0x7fbb94104c70 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.364+0000 7fbb99a00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941109d0 con 0x7fbb94106810 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.364+0000 7fbb93400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb94105e70 0x7fbb9419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.365+0000 7fbb93400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb94105e70 0x7fbb9419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50190/0 (socket says 172.21.15.83:50190) 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.365+0000 7fbb93400700 1 -- 172.21.15.83:0/501342769 learned_addr learned my addr 172.21.15.83:0/501342769 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.365+0000 7fbb99000700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb94106810 0x7fbb941a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.365+0000 7fbb93e00700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb94104c70 0x7fbb9419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.365+0000 7fbb93400700 1 -- 172.21.15.83:0/501342769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb94106810 msgr2=0x7fbb941a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.365+0000 7fbb93400700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb94106810 0x7fbb941a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.365+0000 7fbb93400700 1 -- 172.21.15.83:0/501342769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb94104c70 msgr2=0x7fbb9419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:40:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.365+0000 7fbb93400700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb94104c70 0x7fbb9419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:40:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.365+0000 7fbb93400700 1 -- 172.21.15.83:0/501342769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb941a2a90 con 0x7fbb94105e70 2024-04-23T14:40:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.365+0000 7fbb99000700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb94106810 0x7fbb941a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:40:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.365+0000 7fbb93400700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb94105e70 0x7fbb9419b370 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000e9d0 tx=0x7fbb8000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.366+0000 7fbb90c00700 1 -- 172.21.15.83:0/501342769 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000cbd0 con 0x7fbb94105e70 2024-04-23T14:40:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.366+0000 7fbb90c00700 1 -- 172.21.15.83:0/501342769 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb8000ee70 con 0x7fbb94105e70 2024-04-23T14:40:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.366+0000 7fbb99a00700 1 -- 172.21.15.83:0/501342769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb941a2d80 con 0x7fbb94105e70 2024-04-23T14:40:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.366+0000 7fbb90c00700 1 -- 172.21.15.83:0/501342769 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80018660 con 0x7fbb94105e70 2024-04-23T14:40:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.366+0000 7fbb99a00700 1 -- 172.21.15.83:0/501342769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941a3260 con 0x7fbb94105e70 2024-04-23T14:40:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.367+0000 7fbb99a00700 1 -- 172.21.15.83:0/501342769 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb940697e0 con 0x7fbb94105e70 2024-04-23T14:40:45.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.369+0000 7fbb90c00700 1 -- 172.21.15.83:0/501342769 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbb80010430 con 0x7fbb94105e70 2024-04-23T14:40:45.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.370+0000 7fbb90c00700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb780a3c40 0x7fbb780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:40:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.370+0000 7fbb90c00700 1 -- 172.21.15.83:0/501342769 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7fbb800cf310 con 0x7fbb94105e70 2024-04-23T14:40:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.370+0000 7fbb93e00700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb780a3c40 0x7fbb780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:40:45.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.371+0000 7fbb93e00700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb780a3c40 0x7fbb780a6100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbb84003790 tx=0x7fbb84005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:40:45.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.375+0000 7fbb90c00700 1 -- 172.21.15.83:0/501342769 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb8009d020 con 0x7fbb94105e70 2024-04-23T14:40:45.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:40:45.612+0000 7fbb99a00700 1 -- 172.21.15.83:0/501342769 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbb94062f30 con 0x7fbb780a3c40 2024-04-23T14:40:46.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:45 smithi203 ceph-mon[32800]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:46.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:45 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:40:46.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:45 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:40:46.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:45 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:46.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:45 smithi083 ceph-mon[27867]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:46.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:45 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:40:46.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:45 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:40:46.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:45 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:46.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:45 smithi173 ceph-mon[32443]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:46.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:45 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:40:46.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:45 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:40:46.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:45 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:47.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:46 smithi203 ceph-mon[32800]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:40:47.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:46 smithi083 ceph-mon[27867]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:40:47.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:46 smithi173 ceph-mon[32443]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:40:48.066 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:47 smithi083 ceph-mon[27867]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:48.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:47 smithi203 ceph-mon[32800]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:48.227 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:47 smithi173 ceph-mon[32443]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:50.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:50 smithi203 ceph-mon[32800]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:50.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:50 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2843311131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8caf9f27-bac9-4ff3-b700-bcca3c250e69"}]: dispatch 2024-04-23T14:40:50.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:50 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8caf9f27-bac9-4ff3-b700-bcca3c250e69"}]: dispatch 2024-04-23T14:40:50.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:50 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8caf9f27-bac9-4ff3-b700-bcca3c250e69"}]': finished 2024-04-23T14:40:50.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:50 smithi203 ceph-mon[32800]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T14:40:50.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:50 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:40:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:50 smithi083 ceph-mon[27867]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:50 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2843311131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8caf9f27-bac9-4ff3-b700-bcca3c250e69"}]: dispatch 2024-04-23T14:40:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:50 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8caf9f27-bac9-4ff3-b700-bcca3c250e69"}]: dispatch 2024-04-23T14:40:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:50 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8caf9f27-bac9-4ff3-b700-bcca3c250e69"}]': finished 2024-04-23T14:40:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:50 smithi083 ceph-mon[27867]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T14:40:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:50 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:40:50.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:50 smithi173 ceph-mon[32443]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:50.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:50 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2843311131' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8caf9f27-bac9-4ff3-b700-bcca3c250e69"}]: dispatch 2024-04-23T14:40:50.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:50 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8caf9f27-bac9-4ff3-b700-bcca3c250e69"}]: dispatch 2024-04-23T14:40:50.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:50 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8caf9f27-bac9-4ff3-b700-bcca3c250e69"}]': finished 2024-04-23T14:40:50.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:50 smithi173 ceph-mon[32443]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T14:40:50.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:50 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:40:51.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:51 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1702520090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:40:51.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:51 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1702520090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:40:51.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:51 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1702520090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:40:52.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:52 smithi203 ceph-mon[32800]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:52.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:52 smithi083 ceph-mon[27867]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:52.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:52 smithi173 ceph-mon[32443]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:54.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:54 smithi203 ceph-mon[32800]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:54.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:54 smithi083 ceph-mon[27867]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:54 smithi173 ceph-mon[32443]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:56.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:56 smithi203 ceph-mon[32800]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:56 smithi083 ceph-mon[27867]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:56.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:56 smithi173 ceph-mon[32443]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:58.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:58 smithi083 ceph-mon[27867]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:58.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:58 smithi203 ceph-mon[32800]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:58.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:58 smithi173 ceph-mon[32443]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:40:59.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:59 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T14:40:59.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:40:59 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:59.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:59 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T14:40:59.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:40:59 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:40:59.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:59 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T14:40:59.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:40:59 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:00.460 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:00 smithi083 ceph-mon[27867]: Deploying daemon osd.3 on smithi083 2024-04-23T14:41:00.460 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:00 smithi083 ceph-mon[27867]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:00.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:00 smithi203 ceph-mon[32800]: Deploying daemon osd.3 on smithi083 2024-04-23T14:41:00.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:00 smithi203 ceph-mon[32800]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:00.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:00 smithi173 ceph-mon[32443]: Deploying daemon osd.3 on smithi083 2024-04-23T14:41:00.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:00 smithi173 ceph-mon[32443]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:02.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:02 smithi203 ceph-mon[32800]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:02.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:02 smithi083 ceph-mon[27867]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:02 smithi173 ceph-mon[32443]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:04.396 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:04 smithi083 ceph-mon[27867]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:04.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:04 smithi173 ceph-mon[32443]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:04.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:04 smithi203 ceph-mon[32800]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:06.483 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:06 smithi203 ceph-mon[32800]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:06.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:06 smithi083 ceph-mon[27867]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:06 smithi173 ceph-mon[32443]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:07.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:07 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:41:07.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:07 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:07.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:07 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:41:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:07 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:41:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:07 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:07 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:08.323 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:08 smithi083 ceph-mon[27867]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:08 smithi173 ceph-mon[32443]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:08.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:08 smithi203 ceph-mon[32800]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:10 smithi083 ceph-mon[27867]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:10 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:10 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:10 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:10 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:41:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:10 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:10 smithi173 ceph-mon[32443]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:10 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:10 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:10.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:10 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:10.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:10 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:41:10.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:10 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:10 smithi203 ceph-mon[32800]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:10 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:10.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:10 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:10.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:10 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:10.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:10 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:41:10.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:10 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:11.322 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:11 smithi083 ceph-mon[27867]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T14:41:11.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:11 smithi173 ceph-mon[32443]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T14:41:11.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:11 smithi203 ceph-mon[32800]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T14:41:12.322 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 3 on host 'smithi083' 2024-04-23T14:41:12.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.321+0000 7fbb90c00700 1 -- 172.21.15.83:0/501342769 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbb94062f30 con 0x7fbb780a3c40 2024-04-23T14:41:12.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.324+0000 7fbb99a00700 1 -- 172.21.15.83:0/501342769 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb780a3c40 msgr2=0x7fbb780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.324+0000 7fbb99a00700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb780a3c40 0x7fbb780a6100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbb84003790 tx=0x7fbb84005fb0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.324+0000 7fbb99a00700 1 -- 172.21.15.83:0/501342769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb94105e70 msgr2=0x7fbb9419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.324+0000 7fbb99a00700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb94105e70 0x7fbb9419b370 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000e9d0 tx=0x7fbb8000ece0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.325+0000 7fbb99a00700 1 -- 172.21.15.83:0/501342769 shutdown_connections 2024-04-23T14:41:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.325+0000 7fbb99a00700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb780a3c40 0x7fbb780a6100 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.325+0000 7fbb99a00700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb94106810 0x7fbb941a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.325+0000 7fbb99a00700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb94104c70 0x7fbb9419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.325+0000 7fbb99a00700 1 --2- 172.21.15.83:0/501342769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb94105e70 0x7fbb9419b370 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.325+0000 7fbb99a00700 1 -- 172.21.15.83:0/501342769 >> 172.21.15.83:0/501342769 conn(0x7fbb94100230 msgr2=0x7fbb94109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.325+0000 7fbb99a00700 1 -- 172.21.15.83:0/501342769 shutdown_connections 2024-04-23T14:41:12.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:41:12.325+0000 7fbb99a00700 1 -- 172.21.15.83:0/501342769 wait complete. 2024-04-23T14:41:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:12 smithi083 ceph-mon[27867]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:12 smithi083 ceph-mon[27867]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T14:41:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:12 smithi083 ceph-mon[27867]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T14:41:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:12 smithi083 ceph-mon[27867]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:41:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:41:12.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:12 smithi173 ceph-mon[32443]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:12.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:12 smithi173 ceph-mon[32443]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T14:41:12.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:12 smithi173 ceph-mon[32443]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T14:41:12.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:12 smithi173 ceph-mon[32443]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:41:12.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:41:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:12 smithi203 ceph-mon[32800]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:12 smithi203 ceph-mon[32800]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T14:41:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:12 smithi203 ceph-mon[32800]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T14:41:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:12 smithi203 ceph-mon[32800]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:41:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:41:13.131 DEBUG:teuthology.orchestra.run.smithi083:osd.3> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.3.service 2024-04-23T14:41:13.135 INFO:tasks.cephadm:Deploying osd.4 on smithi173 with /dev/vg_nvme/lv_3... 2024-04-23T14:41:13.135 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T14:41:13.381 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T14:41:13.497 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:13.497 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:13 smithi083 ceph-mon[27867]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-23T14:41:13.498 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:13 smithi083 ceph-mon[27867]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T14:41:13.498 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:41:13.498 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:13.498 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:41:13.498 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:41:13.498 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:13 smithi083 ceph-mon[27867]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' 2024-04-23T14:41:13.498 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:13 smithi083 ceph-mon[27867]: osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161] boot 2024-04-23T14:41:13.498 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:13 smithi083 ceph-mon[27867]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T14:41:13.502 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:13.502 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:13 smithi173 ceph-mon[32443]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-23T14:41:13.502 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:13 smithi173 ceph-mon[32443]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T14:41:13.502 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:41:13.502 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:13.502 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:41:13.502 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:41:13.502 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:13 smithi173 ceph-mon[32443]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' 2024-04-23T14:41:13.502 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:13 smithi173 ceph-mon[32443]: osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161] boot 2024-04-23T14:41:13.502 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:13 smithi173 ceph-mon[32443]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T14:41:13.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:13.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:13 smithi203 ceph-mon[32800]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-23T14:41:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:13 smithi203 ceph-mon[32800]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T14:41:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:41:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:41:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:41:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:13 smithi203 ceph-mon[32800]: from='osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161]' entity='osd.3' 2024-04-23T14:41:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:13 smithi203 ceph-mon[32800]: osd.3 [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161] boot 2024-04-23T14:41:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:13 smithi203 ceph-mon[32800]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T14:41:14.567 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:14 smithi173 ceph-mon[32443]: purged_snaps scrub starts 2024-04-23T14:41:14.567 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:14 smithi173 ceph-mon[32443]: purged_snaps scrub ok 2024-04-23T14:41:14.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:14 smithi173 ceph-mon[32443]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:14.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:14 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:41:14.568 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:14 smithi173 ceph-mon[32443]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T14:41:14.645 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:14 smithi083 ceph-mon[27867]: purged_snaps scrub starts 2024-04-23T14:41:14.645 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:14 smithi083 ceph-mon[27867]: purged_snaps scrub ok 2024-04-23T14:41:14.645 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:14 smithi083 ceph-mon[27867]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:14.645 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:14 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:41:14.645 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:14 smithi083 ceph-mon[27867]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T14:41:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:14 smithi203 ceph-mon[32800]: purged_snaps scrub starts 2024-04-23T14:41:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:14 smithi203 ceph-mon[32800]: purged_snaps scrub ok 2024-04-23T14:41:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:14 smithi203 ceph-mon[32800]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-23T14:41:14.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:14 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:41:14.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:14 smithi203 ceph-mon[32800]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T14:41:15.416 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T14:41:15.417 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T14:41:15.417 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-23T14:41:15.417 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-23T14:41:15.417 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113439 s, 924 MB/s 2024-04-23T14:41:15.417 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-23T14:41:16.109 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_3 2024-04-23T14:41:16.351 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T14:41:16.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:16 smithi173 ceph-mon[32443]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:16.473 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:16 smithi173 ceph-mon[32443]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T14:41:16.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:16 smithi083 ceph-mon[27867]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:16.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:16 smithi083 ceph-mon[27867]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T14:41:16.543 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:16 smithi203 ceph-mon[32800]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:16.543 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:16 smithi203 ceph-mon[32800]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T14:41:17.363 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:17 smithi173 ceph-mon[32443]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T14:41:17.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:17 smithi203 ceph-mon[32800]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T14:41:17.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:17 smithi083 ceph-mon[27867]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T14:41:17.877 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.876+0000 7ff0910ab700 1 -- 172.21.15.173:0/1871970623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c1035f0 msgr2=0x7ff08c109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:17.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.876+0000 7ff0910ab700 1 --2- 172.21.15.173:0/1871970623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c1035f0 0x7ff08c109ea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff08400b0d0 tx=0x7ff08400b3e0 comp rx=0 tx=0).stop 2024-04-23T14:41:17.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.876+0000 7ff0910ab700 1 -- 172.21.15.173:0/1871970623 shutdown_connections 2024-04-23T14:41:17.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.876+0000 7ff0910ab700 1 --2- 172.21.15.173:0/1871970623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c1035f0 0x7ff08c109ea0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:17.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.876+0000 7ff0910ab700 1 --2- 172.21.15.173:0/1871970623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff08c101a50 0x7ff08c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:17.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.876+0000 7ff0910ab700 1 --2- 172.21.15.173:0/1871970623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c102c50 0x7ff08c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:17.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.876+0000 7ff0910ab700 1 -- 172.21.15.173:0/1871970623 >> 172.21.15.173:0/1871970623 conn(0x7ff08c0fcff0 msgr2=0x7ff08c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:17.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.876+0000 7ff0910ab700 1 -- 172.21.15.173:0/1871970623 shutdown_connections 2024-04-23T14:41:17.878 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.876+0000 7ff0910ab700 1 -- 172.21.15.173:0/1871970623 wait complete. 2024-04-23T14:41:17.880 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff0910ab700 1 Processor -- start 2024-04-23T14:41:17.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff0910ab700 1 -- start start 2024-04-23T14:41:17.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff0910ab700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c101a50 0x7ff08c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:17.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff0910ab700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c102c50 0x7ff08c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:17.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff0910ab700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff08c1035f0 0x7ff08c1a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:17.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff0910ab700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c10e7d0 con 0x7ff08c1035f0 2024-04-23T14:41:17.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff0910ab700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c10e650 con 0x7ff08c102c50 2024-04-23T14:41:17.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff0910ab700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c10e950 con 0x7ff08c101a50 2024-04-23T14:41:17.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff08a59c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c102c50 0x7ff08c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:17.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff08a59c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c102c50 0x7ff08c1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:57010/0 (socket says 172.21.15.173:57010) 2024-04-23T14:41:17.881 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff08ad9d700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c101a50 0x7ff08c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff08a59c700 1 -- 172.21.15.173:0/3001048796 learned_addr learned my addr 172.21.15.173:0/3001048796 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff08b59e700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff08c1035f0 0x7ff08c1a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff08a59c700 1 -- 172.21.15.173:0/3001048796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c101a50 msgr2=0x7ff08c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.877+0000 7ff08a59c700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c101a50 0x7ff08c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.878+0000 7ff08a59c700 1 -- 172.21.15.173:0/3001048796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff08c1035f0 msgr2=0x7ff08c1a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.878+0000 7ff08a59c700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff08c1035f0 0x7ff08c1a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.878+0000 7ff08a59c700 1 -- 172.21.15.173:0/3001048796 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c1a0a10 con 0x7ff08c102c50 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.878+0000 7ff08ad9d700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c101a50 0x7ff08c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.878+0000 7ff08a59c700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c102c50 0x7ff08c1992f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff07400c970 tx=0x7ff07400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.878+0000 7ff083fff700 1 -- 172.21.15.173:0/3001048796 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074007aa0 con 0x7ff08c102c50 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.878+0000 7ff0910ab700 1 -- 172.21.15.173:0/3001048796 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c1a0d00 con 0x7ff08c102c50 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.878+0000 7ff083fff700 1 -- 172.21.15.173:0/3001048796 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff074004d10 con 0x7ff08c102c50 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.878+0000 7ff083fff700 1 -- 172.21.15.173:0/3001048796 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074005690 con 0x7ff08c102c50 2024-04-23T14:41:17.882 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.878+0000 7ff0910ab700 1 -- 172.21.15.173:0/3001048796 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c1a1210 con 0x7ff08c102c50 2024-04-23T14:41:17.883 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.879+0000 7ff0910ab700 1 -- 172.21.15.173:0/3001048796 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c0697e0 con 0x7ff08c102c50 2024-04-23T14:41:17.883 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.880+0000 7ff083fff700 1 -- 172.21.15.173:0/3001048796 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff074004750 con 0x7ff08c102c50 2024-04-23T14:41:17.883 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.880+0000 7ff083fff700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff0780a3c40 0x7ff0780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:17.884 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.881+0000 7ff083fff700 1 -- 172.21.15.173:0/3001048796 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7ff0740cef40 con 0x7ff08c102c50 2024-04-23T14:41:17.884 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.881+0000 7ff08ad9d700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff0780a3c40 0x7ff0780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:17.884 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.881+0000 7ff08ad9d700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff0780a3c40 0x7ff0780a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff08c102ab0 tx=0x7ff07c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:17.886 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:17.885+0000 7ff083fff700 1 -- 172.21.15.173:0/3001048796 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff07409cbd0 con 0x7ff08c102c50 2024-04-23T14:41:18.095 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:18.094+0000 7ff0910ab700 1 -- 172.21.15.173:0/3001048796 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff08c062f30 con 0x7ff0780a3c40 2024-04-23T14:41:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:18 smithi173 ceph-mon[32443]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T14:41:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:18 smithi173 ceph-mon[32443]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T14:41:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:41:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:41:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:18.635 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:18 smithi083 ceph-mon[27867]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T14:41:18.636 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:18 smithi083 ceph-mon[27867]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T14:41:18.636 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:41:18.636 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:41:18.636 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:18.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:18 smithi203 ceph-mon[32800]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T14:41:18.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:18 smithi203 ceph-mon[32800]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T14:41:18.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:41:18.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:41:18.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:19.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:19 smithi173 ceph-mon[32443]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:41:19.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:19 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:19.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:19 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:19.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:19 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:41:19.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:19 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:41:19.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:19 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:19.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:19 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:19.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:19 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:41:19.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:19 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:19.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:19 smithi203 ceph-mon[32800]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:41:19.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:19 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:19.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:19 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:19.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:19 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:41:19.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:19 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:41:19.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:19 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:19.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:19 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:19.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:19 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:41:19.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:19 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:19 smithi083 ceph-mon[27867]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:41:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:41:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:41:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:41:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:19 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:20 smithi173 ceph-mon[32443]: Detected new or changed devices on smithi083 2024-04-23T14:41:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:20 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-23T14:41:20.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:20 smithi173 ceph-mon[32443]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-04-23T14:41:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:20 smithi203 ceph-mon[32800]: Detected new or changed devices on smithi083 2024-04-23T14:41:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:20 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-23T14:41:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:20 smithi203 ceph-mon[32800]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-04-23T14:41:20.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:20 smithi083 ceph-mon[27867]: Detected new or changed devices on smithi083 2024-04-23T14:41:20.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:20 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi083 to 8544M 2024-04-23T14:41:20.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:20 smithi083 ceph-mon[27867]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-04-23T14:41:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:21 smithi173 ceph-mon[32443]: from='client.? 172.21.15.173:0/2585174878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07401fed-df44-4869-92ff-72c233457a2d"}]: dispatch 2024-04-23T14:41:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:21 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07401fed-df44-4869-92ff-72c233457a2d"}]: dispatch 2024-04-23T14:41:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:21 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07401fed-df44-4869-92ff-72c233457a2d"}]': finished 2024-04-23T14:41:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:21 smithi173 ceph-mon[32443]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T14:41:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:21 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:21.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:21 smithi203 ceph-mon[32800]: from='client.? 172.21.15.173:0/2585174878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07401fed-df44-4869-92ff-72c233457a2d"}]: dispatch 2024-04-23T14:41:21.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:21 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07401fed-df44-4869-92ff-72c233457a2d"}]: dispatch 2024-04-23T14:41:21.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:21 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07401fed-df44-4869-92ff-72c233457a2d"}]': finished 2024-04-23T14:41:21.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:21 smithi203 ceph-mon[32800]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T14:41:21.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:21 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:21.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:21 smithi083 ceph-mon[27867]: from='client.? 172.21.15.173:0/2585174878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07401fed-df44-4869-92ff-72c233457a2d"}]: dispatch 2024-04-23T14:41:21.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:21 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07401fed-df44-4869-92ff-72c233457a2d"}]: dispatch 2024-04-23T14:41:21.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:21 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07401fed-df44-4869-92ff-72c233457a2d"}]': finished 2024-04-23T14:41:21.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:21 smithi083 ceph-mon[27867]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T14:41:21.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:21 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:22.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:22 smithi173 ceph-mon[32443]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-23T14:41:22.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:22 smithi173 ceph-mon[32443]: from='client.? 172.21.15.173:0/3698272690' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:41:22.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:22 smithi203 ceph-mon[32800]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-23T14:41:22.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:22 smithi203 ceph-mon[32800]: from='client.? 172.21.15.173:0/3698272690' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:41:22.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:22 smithi083 ceph-mon[27867]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-23T14:41:22.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:22 smithi083 ceph-mon[27867]: from='client.? 172.21.15.173:0/3698272690' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:41:24.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:24 smithi083 ceph-mon[27867]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-23T14:41:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:24 smithi173 ceph-mon[32443]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-23T14:41:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:24 smithi203 ceph-mon[32800]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-23T14:41:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:26 smithi173 ceph-mon[32443]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:26 smithi203 ceph-mon[32800]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:26.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:26 smithi083 ceph-mon[27867]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:27.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:27 smithi203 ceph-mon[32800]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:27.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:27 smithi083 ceph-mon[27867]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:27.794 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:27 smithi173 ceph-mon[32443]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:28.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:28 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T14:41:28.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:28 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:28.659 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:28 smithi083 ceph-mon[27867]: Deploying daemon osd.4 on smithi173 2024-04-23T14:41:28.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:28 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T14:41:28.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:28 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:28.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:28 smithi203 ceph-mon[32800]: Deploying daemon osd.4 on smithi173 2024-04-23T14:41:28.773 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:28 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T14:41:28.773 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:28 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:28.773 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:28 smithi173 ceph-mon[32443]: Deploying daemon osd.4 on smithi173 2024-04-23T14:41:29.690 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:29 smithi083 ceph-mon[27867]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:29.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:29 smithi203 ceph-mon[32800]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:29.752 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:29 smithi173 ceph-mon[32443]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:32.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:32 smithi203 ceph-mon[32800]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:32.489 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:32 smithi173 ceph-mon[32443]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:32 smithi083 ceph-mon[27867]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:34.385 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:34 smithi173 ceph-mon[32443]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:34.385 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:34.385 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:34.385 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:41:34.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:34 smithi203 ceph-mon[32800]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:34.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:34.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:34.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:41:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:34 smithi083 ceph-mon[27867]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:41:36.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:36 smithi173 ceph-mon[32443]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:36.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:36 smithi203 ceph-mon[32800]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:36.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:36 smithi083 ceph-mon[27867]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:37.225 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.224+0000 7ff083fff700 1 -- 172.21.15.173:0/3001048796 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff08c062f30 con 0x7ff0780a3c40 2024-04-23T14:41:37.225 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 4 on host 'smithi173' 2024-04-23T14:41:37.227 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.226+0000 7ff0910ab700 1 -- 172.21.15.173:0/3001048796 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff0780a3c40 msgr2=0x7ff0780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:37.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.226+0000 7ff0910ab700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff0780a3c40 0x7ff0780a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff08c102ab0 tx=0x7ff07c009190 comp rx=0 tx=0).stop 2024-04-23T14:41:37.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.226+0000 7ff0910ab700 1 -- 172.21.15.173:0/3001048796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c102c50 msgr2=0x7ff08c1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:37.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.226+0000 7ff0910ab700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c102c50 0x7ff08c1992f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff07400c970 tx=0x7ff07400cc80 comp rx=0 tx=0).stop 2024-04-23T14:41:37.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.227+0000 7ff0910ab700 1 -- 172.21.15.173:0/3001048796 shutdown_connections 2024-04-23T14:41:37.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.227+0000 7ff0910ab700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff0780a3c40 0x7ff0780a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.227+0000 7ff0910ab700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff08c1035f0 0x7ff08c1a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.227+0000 7ff0910ab700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff08c101a50 0x7ff08c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.227+0000 7ff0910ab700 1 --2- 172.21.15.173:0/3001048796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff08c102c50 0x7ff08c1992f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:37.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.227+0000 7ff0910ab700 1 -- 172.21.15.173:0/3001048796 >> 172.21.15.173:0/3001048796 conn(0x7ff08c0fcff0 msgr2=0x7ff08c105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:37.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.227+0000 7ff0910ab700 1 -- 172.21.15.173:0/3001048796 shutdown_connections 2024-04-23T14:41:37.228 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:41:37.227+0000 7ff0910ab700 1 -- 172.21.15.173:0/3001048796 wait complete. 2024-04-23T14:41:37.828 DEBUG:teuthology.orchestra.run.smithi173:osd.4> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.4.service 2024-04-23T14:41:37.830 INFO:tasks.cephadm:Deploying osd.5 on smithi203 with /dev/vg_nvme/lv_3... 2024-04-23T14:41:37.830 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T14:41:38.082 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:41:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:38 smithi203 ceph-mon[32800]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:38 smithi203 ceph-mon[32800]: from='osd.4 [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T14:41:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:38 smithi203 ceph-mon[32800]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T14:41:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:38.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:38 smithi083 ceph-mon[27867]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:38.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:38 smithi083 ceph-mon[27867]: from='osd.4 [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T14:41:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:38 smithi083 ceph-mon[27867]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T14:41:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:38.526 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:38 smithi173 ceph-mon[32443]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:38.527 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:38.527 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:38.527 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:38 smithi173 ceph-mon[32443]: from='osd.4 [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T14:41:38.527 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:38 smithi173 ceph-mon[32443]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T14:41:38.527 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:38.527 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:39.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:39 smithi203 ceph-mon[32800]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T14:41:39.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:39 smithi203 ceph-mon[32800]: from='osd.4 [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:41:39.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:39 smithi203 ceph-mon[32800]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T14:41:39.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:39 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:39.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:39 smithi203 ceph-mon[32800]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:41:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:39 smithi083 ceph-mon[27867]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T14:41:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:39 smithi083 ceph-mon[27867]: from='osd.4 [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:41:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:39 smithi083 ceph-mon[27867]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T14:41:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:39 smithi083 ceph-mon[27867]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:41:39.534 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:39 smithi173 ceph-mon[32443]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T14:41:39.534 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:39 smithi173 ceph-mon[32443]: from='osd.4 [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:41:39.534 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:39 smithi173 ceph-mon[32443]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T14:41:39.534 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:39 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:39.534 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:39 smithi173 ceph-mon[32443]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:41:39.534 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 14:41:39 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4[43975]: 2024-04-23T14:41:39.226+0000 7fa11f26f700 -1 osd.4 0 waiting for initial osdmap 2024-04-23T14:41:39.535 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 14:41:39 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4[43975]: 2024-04-23T14:41:39.232+0000 7fa11b062700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:41:40.114 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T14:41:40.115 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T14:41:40.115 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-04-23T14:41:40.115 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-04-23T14:41:40.115 INFO:teuthology.orchestra.run.smithi203.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123871 s, 847 MB/s 2024-04-23T14:41:40.115 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-04-23T14:41:40.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:40 smithi203 ceph-mon[32800]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:40.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:40 smithi203 ceph-mon[32800]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T14:41:40.495 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:40 smithi203 ceph-mon[32800]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T14:41:40.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:40 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:40.496 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:40 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:40 smithi083 ceph-mon[27867]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:40 smithi083 ceph-mon[27867]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T14:41:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:40 smithi083 ceph-mon[27867]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T14:41:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:40 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:40 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:40 smithi173 ceph-mon[32443]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:40 smithi173 ceph-mon[32443]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T14:41:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:40 smithi173 ceph-mon[32443]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T14:41:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:40 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:40 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:40.823 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_3 2024-04-23T14:41:41.060 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:41:41.433 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:41 smithi203 ceph-mon[32800]: purged_snaps scrub starts 2024-04-23T14:41:41.433 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:41 smithi203 ceph-mon[32800]: purged_snaps scrub ok 2024-04-23T14:41:41.433 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:41 smithi203 ceph-mon[32800]: osd.4 [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908] boot 2024-04-23T14:41:41.433 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:41 smithi203 ceph-mon[32800]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T14:41:41.433 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:41.451 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:41 smithi173 ceph-mon[32443]: purged_snaps scrub starts 2024-04-23T14:41:41.451 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:41 smithi173 ceph-mon[32443]: purged_snaps scrub ok 2024-04-23T14:41:41.451 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:41 smithi173 ceph-mon[32443]: osd.4 [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908] boot 2024-04-23T14:41:41.451 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:41 smithi173 ceph-mon[32443]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T14:41:41.451 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:41.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:41 smithi083 ceph-mon[27867]: purged_snaps scrub starts 2024-04-23T14:41:41.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:41 smithi083 ceph-mon[27867]: purged_snaps scrub ok 2024-04-23T14:41:41.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:41 smithi083 ceph-mon[27867]: osd.4 [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908] boot 2024-04-23T14:41:41.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:41 smithi083 ceph-mon[27867]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T14:41:41.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:41:42.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:42.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T14:41:42.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:41:42.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:41:42.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:42.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:41:42.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:41:42.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:42.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:41:42.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:42.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T14:41:42.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:41:42.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:41:42.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:42.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:41:42.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:41:42.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:42.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:41:42.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-04-23T14:41:42.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T14:41:42.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:41:42.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:41:42.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:42.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:41:42.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:41:42.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:42.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:41:42.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:41:42.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.792+0000 7f807d314700 1 -- 172.21.15.203:0/1420379628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8078105fa0 msgr2=0x7f8078108390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.792+0000 7f807d314700 1 --2- 172.21.15.203:0/1420379628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8078105fa0 0x7f8078108390 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8060009990 tx=0x7f8060009ca0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.792+0000 7f807d314700 1 -- 172.21.15.203:0/1420379628 shutdown_connections 2024-04-23T14:41:42.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.792+0000 7f807d314700 1 --2- 172.21.15.203:0/1420379628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80781088d0 0x7f807810ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.795 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.792+0000 7f807d314700 1 --2- 172.21.15.203:0/1420379628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8078103670 0x7f8078105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.792+0000 7f807d314700 1 --2- 172.21.15.203:0/1420379628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8078105fa0 0x7f8078108390 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.792+0000 7f807d314700 1 -- 172.21.15.203:0/1420379628 >> 172.21.15.203:0/1420379628 conn(0x7f80780fd040 msgr2=0x7f80780ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:41:42.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.792+0000 7f807d314700 1 -- 172.21.15.203:0/1420379628 shutdown_connections 2024-04-23T14:41:42.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.792+0000 7f807d314700 1 -- 172.21.15.203:0/1420379628 wait complete. 2024-04-23T14:41:42.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f807d314700 1 Processor -- start 2024-04-23T14:41:42.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f807d314700 1 -- start start 2024-04-23T14:41:42.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f807d314700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8078103670 0x7f8078198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f807d314700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8078105fa0 0x7f80781992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.796 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f807d314700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80781088d0 0x7f80781a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.797 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f807d314700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f807810e7d0 con 0x7f8078103670 2024-04-23T14:41:42.797 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f807d314700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f807810e650 con 0x7f8078105fa0 2024-04-23T14:41:42.797 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f807d314700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f807810e950 con 0x7f80781088d0 2024-04-23T14:41:42.797 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f80777fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80781088d0 0x7f80781a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.797 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f80777fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80781088d0 0x7f80781a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:45964/0 (socket says 172.21.15.203:45964) 2024-04-23T14:41:42.797 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f80777fe700 1 -- 172.21.15.203:0/4076242258 learned_addr learned my addr 172.21.15.203:0/4076242258 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-23T14:41:42.797 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f80767fc700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8078105fa0 0x7f80781992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.797 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f80777fe700 1 -- 172.21.15.203:0/4076242258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8078105fa0 msgr2=0x7f80781992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.797 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f80777fe700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8078105fa0 0x7f80781992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.797 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f80777fe700 1 -- 172.21.15.203:0/4076242258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8078103670 msgr2=0x7f8078198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:41:42.798 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f80777fe700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8078103670 0x7f8078198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:41:42.798 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.793+0000 7f80777fe700 1 -- 172.21.15.203:0/4076242258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80781a09f0 con 0x7f80781088d0 2024-04-23T14:41:42.798 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.794+0000 7f80777fe700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80781088d0 0x7f80781a0320 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f807000b9e0 tx=0x7f807000bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:42.798 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.794+0000 7f806ffff700 1 -- 172.21.15.203:0/4076242258 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070019070 con 0x7f80781088d0 2024-04-23T14:41:42.798 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.794+0000 7f807d314700 1 -- 172.21.15.203:0/4076242258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80781a0ce0 con 0x7f80781088d0 2024-04-23T14:41:42.798 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.794+0000 7f806ffff700 1 -- 172.21.15.203:0/4076242258 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8070014440 con 0x7f80781088d0 2024-04-23T14:41:42.798 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.794+0000 7f806ffff700 1 -- 172.21.15.203:0/4076242258 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80700125a0 con 0x7f80781088d0 2024-04-23T14:41:42.798 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.794+0000 7f807d314700 1 -- 172.21.15.203:0/4076242258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80781a1240 con 0x7f80781088d0 2024-04-23T14:41:42.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.795+0000 7f807d314700 1 -- 172.21.15.203:0/4076242258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80780697e0 con 0x7f80781088d0 2024-04-23T14:41:42.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.796+0000 7f806ffff700 1 -- 172.21.15.203:0/4076242258 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f807001c070 con 0x7f80781088d0 2024-04-23T14:41:42.799 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.797+0000 7f806ffff700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80640a3c40 0x7f80640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:41:42.800 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.797+0000 7f806ffff700 1 -- 172.21.15.203:0/4076242258 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f80700ce770 con 0x7f80781088d0 2024-04-23T14:41:42.800 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.797+0000 7f8076ffd700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80640a3c40 0x7f80640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:41:42.800 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.798+0000 7f8076ffd700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80640a3c40 0x7f80640a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f807819a280 tx=0x7f8068005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:41:42.802 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:42.801+0000 7f806ffff700 1 -- 172.21.15.203:0/4076242258 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f807009c1c0 con 0x7f80781088d0 2024-04-23T14:41:43.008 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:41:43.007+0000 7f807d314700 1 -- 172.21.15.203:0/4076242258 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8078062f30 con 0x7f80640a3c40 2024-04-23T14:41:43.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:43 smithi173 ceph-mon[32443]: Detected new or changed devices on smithi173 2024-04-23T14:41:43.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:43 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi173 to 8544M 2024-04-23T14:41:43.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:41:43.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:41:43.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:43.646 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:43 smithi203 ceph-mon[32800]: Detected new or changed devices on smithi173 2024-04-23T14:41:43.646 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:43 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi173 to 8544M 2024-04-23T14:41:43.646 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:41:43.646 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:41:43.646 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:43.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:43 smithi083 ceph-mon[27867]: Detected new or changed devices on smithi173 2024-04-23T14:41:43.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:43 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi173 to 8544M 2024-04-23T14:41:43.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:41:43.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:41:43.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:44.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:44 smithi173 ceph-mon[32443]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:41:44.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:44 smithi173 ceph-mon[32443]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:44.613 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:44 smithi203 ceph-mon[32800]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:41:44.613 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:44 smithi203 ceph-mon[32800]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:44.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:44 smithi083 ceph-mon[27867]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:41:44.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:44 smithi083 ceph-mon[27867]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:46.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:46 smithi203 ceph-mon[32800]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:46 smithi203 ceph-mon[32800]: from='client.? 172.21.15.203:0/383064794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04c3ad4f-1bd5-45fa-a333-26b242b2ba1e"}]: dispatch 2024-04-23T14:41:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:46 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04c3ad4f-1bd5-45fa-a333-26b242b2ba1e"}]: dispatch 2024-04-23T14:41:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:46 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04c3ad4f-1bd5-45fa-a333-26b242b2ba1e"}]': finished 2024-04-23T14:41:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:46 smithi203 ceph-mon[32800]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T14:41:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:46 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:41:46.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:46 smithi173 ceph-mon[32443]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:46.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:46 smithi173 ceph-mon[32443]: from='client.? 172.21.15.203:0/383064794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04c3ad4f-1bd5-45fa-a333-26b242b2ba1e"}]: dispatch 2024-04-23T14:41:46.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:46 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04c3ad4f-1bd5-45fa-a333-26b242b2ba1e"}]: dispatch 2024-04-23T14:41:46.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:46 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04c3ad4f-1bd5-45fa-a333-26b242b2ba1e"}]': finished 2024-04-23T14:41:46.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:46 smithi173 ceph-mon[32443]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T14:41:46.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:46 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:41:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:46 smithi083 ceph-mon[27867]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:46 smithi083 ceph-mon[27867]: from='client.? 172.21.15.203:0/383064794' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04c3ad4f-1bd5-45fa-a333-26b242b2ba1e"}]: dispatch 2024-04-23T14:41:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:46 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04c3ad4f-1bd5-45fa-a333-26b242b2ba1e"}]: dispatch 2024-04-23T14:41:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:46 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04c3ad4f-1bd5-45fa-a333-26b242b2ba1e"}]': finished 2024-04-23T14:41:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:46 smithi083 ceph-mon[27867]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T14:41:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:41:47.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:47 smithi173 ceph-mon[32443]: from='client.? 172.21.15.203:0/2003466751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:41:47.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:47 smithi203 ceph-mon[32800]: from='client.? 172.21.15.203:0/2003466751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:41:47.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:47 smithi083 ceph-mon[27867]: from='client.? 172.21.15.203:0/2003466751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:41:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:48 smithi173 ceph-mon[32443]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:48.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:48 smithi203 ceph-mon[32800]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:48.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:48 smithi083 ceph-mon[27867]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:50.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:50 smithi173 ceph-mon[32443]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:50.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:50 smithi203 ceph-mon[32800]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:50.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:50 smithi083 ceph-mon[27867]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:52.449 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:52 smithi203 ceph-mon[32800]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:52.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:52 smithi173 ceph-mon[32443]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:52.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:52 smithi083 ceph-mon[27867]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:54.399 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:54 smithi203 ceph-mon[32800]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:54.399 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T14:41:54.400 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:54 smithi173 ceph-mon[32443]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T14:41:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:54 smithi083 ceph-mon[27867]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T14:41:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:41:55.466 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:55 smithi203 ceph-mon[32800]: Deploying daemon osd.5 on smithi203 2024-04-23T14:41:55.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:55 smithi173 ceph-mon[32443]: Deploying daemon osd.5 on smithi203 2024-04-23T14:41:55.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:55 smithi083 ceph-mon[27867]: Deploying daemon osd.5 on smithi203 2024-04-23T14:41:56.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:56 smithi173 ceph-mon[32443]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:56.598 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:56 smithi203 ceph-mon[32800]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:56.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:56 smithi083 ceph-mon[27867]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:58.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:41:58 smithi173 ceph-mon[32443]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:58.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:41:58 smithi203 ceph-mon[32800]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:41:58.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:41:58 smithi083 ceph-mon[27867]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:00.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:00 smithi203 ceph-mon[32800]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:00.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:00 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:00.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:00 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:00.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:00 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:42:00.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:00 smithi083 ceph-mon[27867]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:00.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:00.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:00.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:42:00.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:00 smithi173 ceph-mon[32443]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:00.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:00 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:00.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:00 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:00.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:00 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:42:02.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:02 smithi203 ceph-mon[32800]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:02.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:02.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:02.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:02.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:42:02.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:02.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:02 smithi083 ceph-mon[27867]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:02.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:02.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:02.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:02.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:42:02.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:02 smithi173 ceph-mon[32443]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:42:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:03.248 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.247+0000 7f806ffff700 1 -- 172.21.15.203:0/4076242258 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8078062f30 con 0x7f80640a3c40 2024-04-23T14:42:03.249 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 5 on host 'smithi203' 2024-04-23T14:42:03.250 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.250+0000 7f807d314700 1 -- 172.21.15.203:0/4076242258 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80640a3c40 msgr2=0x7f80640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:03.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.250+0000 7f807d314700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80640a3c40 0x7f80640a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f807819a280 tx=0x7f8068005c80 comp rx=0 tx=0).stop 2024-04-23T14:42:03.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.250+0000 7f807d314700 1 -- 172.21.15.203:0/4076242258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80781088d0 msgr2=0x7f80781a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:03.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.250+0000 7f807d314700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80781088d0 0x7f80781a0320 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f807000b9e0 tx=0x7f807000bda0 comp rx=0 tx=0).stop 2024-04-23T14:42:03.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.250+0000 7f807d314700 1 -- 172.21.15.203:0/4076242258 shutdown_connections 2024-04-23T14:42:03.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.250+0000 7f807d314700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80640a3c40 0x7f80640a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:03.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.250+0000 7f807d314700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80781088d0 0x7f80781a0320 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:03.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.250+0000 7f807d314700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8078103670 0x7f8078198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:03.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.250+0000 7f807d314700 1 --2- 172.21.15.203:0/4076242258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8078105fa0 0x7f80781992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:03.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.250+0000 7f807d314700 1 -- 172.21.15.203:0/4076242258 >> 172.21.15.203:0/4076242258 conn(0x7f80780fd040 msgr2=0x7f8078103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:03.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.250+0000 7f807d314700 1 -- 172.21.15.203:0/4076242258 shutdown_connections 2024-04-23T14:42:03.251 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:42:03.250+0000 7f807d314700 1 -- 172.21.15.203:0/4076242258 wait complete. 2024-04-23T14:42:03.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:03 smithi203 ceph-mon[32800]: from='osd.5 [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T14:42:03.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:03 smithi203 ceph-mon[32800]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T14:42:03.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:03 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:03.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:03 smithi083 ceph-mon[27867]: from='osd.5 [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T14:42:03.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:03 smithi083 ceph-mon[27867]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T14:42:03.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:03.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:03 smithi173 ceph-mon[32443]: from='osd.5 [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T14:42:03.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:03 smithi173 ceph-mon[32443]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T14:42:03.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:03 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:03.892 DEBUG:teuthology.orchestra.run.smithi203:osd.5> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.5.service 2024-04-23T14:42:03.895 INFO:tasks.cephadm:Deploying osd.6 on smithi083 with /dev/vg_nvme/lv_2... 2024-04-23T14:42:03.895 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T14:42:04.341 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:42:04.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:04 smithi083 ceph-mon[27867]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:04.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:04 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:42:04.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:04 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:04.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:04 smithi083 ceph-mon[27867]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T14:42:04.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:04 smithi083 ceph-mon[27867]: from='osd.5 [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:42:04.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:04 smithi083 ceph-mon[27867]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T14:42:04.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:04 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:42:04.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:04 smithi083 ceph-mon[27867]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:42:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:04 smithi173 ceph-mon[32443]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:04 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:42:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:04 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:04 smithi173 ceph-mon[32443]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T14:42:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:04 smithi173 ceph-mon[32443]: from='osd.5 [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:42:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:04 smithi173 ceph-mon[32443]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T14:42:04.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:04 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:42:04.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:04 smithi173 ceph-mon[32443]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:42:04.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:04 smithi203 ceph-mon[32800]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:04.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:04 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:42:04.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:04 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:04.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:04 smithi203 ceph-mon[32800]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T14:42:04.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:04 smithi203 ceph-mon[32800]: from='osd.5 [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:42:04.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:04 smithi203 ceph-mon[32800]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T14:42:04.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:04 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:42:04.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:04 smithi203 ceph-mon[32800]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:42:04.723 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 14:42:04 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5[44480]: 2024-04-23T14:42:04.230+0000 7f83edda0700 -1 osd.5 0 waiting for initial osdmap 2024-04-23T14:42:04.723 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 14:42:04 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5[44480]: 2024-04-23T14:42:04.236+0000 7f83e9b93700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:42:05.465 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:05 smithi203 ceph-mon[32800]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-04-23T14:42:05.465 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:05 smithi203 ceph-mon[32800]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T14:42:05.465 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:42:05.465 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:42:05.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:05 smithi083 ceph-mon[27867]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-04-23T14:42:05.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:05 smithi083 ceph-mon[27867]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T14:42:05.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:42:05.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:42:05.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:05 smithi173 ceph-mon[32443]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-04-23T14:42:05.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:05 smithi173 ceph-mon[32443]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T14:42:05.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:05 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:42:05.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:05 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:42:06.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:06 smithi083 ceph-mon[27867]: purged_snaps scrub starts 2024-04-23T14:42:06.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:06 smithi083 ceph-mon[27867]: purged_snaps scrub ok 2024-04-23T14:42:06.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:06 smithi083 ceph-mon[27867]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:06.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:06 smithi083 ceph-mon[27867]: osd.5 [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636] boot 2024-04-23T14:42:06.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:06 smithi083 ceph-mon[27867]: osdmap e38: 6 total, 6 up, 6 in 2024-04-23T14:42:06.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:06 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:42:06.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:06 smithi083 ceph-mon[27867]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T14:42:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:06 smithi173 ceph-mon[32443]: purged_snaps scrub starts 2024-04-23T14:42:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:06 smithi173 ceph-mon[32443]: purged_snaps scrub ok 2024-04-23T14:42:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:06 smithi173 ceph-mon[32443]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:06 smithi173 ceph-mon[32443]: osd.5 [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636] boot 2024-04-23T14:42:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:06 smithi173 ceph-mon[32443]: osdmap e38: 6 total, 6 up, 6 in 2024-04-23T14:42:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:06 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:42:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:06 smithi173 ceph-mon[32443]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T14:42:06.603 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:06 smithi203 ceph-mon[32800]: purged_snaps scrub starts 2024-04-23T14:42:06.603 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:06 smithi203 ceph-mon[32800]: purged_snaps scrub ok 2024-04-23T14:42:06.603 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:06 smithi203 ceph-mon[32800]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-04-23T14:42:06.603 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:06 smithi203 ceph-mon[32800]: osd.5 [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636] boot 2024-04-23T14:42:06.603 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:06 smithi203 ceph-mon[32800]: osdmap e38: 6 total, 6 up, 6 in 2024-04-23T14:42:06.603 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:06 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:42:06.603 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:06 smithi203 ceph-mon[32800]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T14:42:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T14:42:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T14:42:07.279 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-23T14:42:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-23T14:42:07.279 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.034537 s, 304 MB/s 2024-04-23T14:42:07.279 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-23T14:42:08.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_2 2024-04-23T14:42:08.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:08 smithi083 ceph-mon[27867]: Detected new or changed devices on smithi203 2024-04-23T14:42:08.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:08.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:08.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:08.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:08.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:08 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi203 to 8544M 2024-04-23T14:42:08.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:08.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:08.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:42:08.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:08 smithi083 ceph-mon[27867]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:08.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:08.371 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:08 smithi083 ceph-mon[27867]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T14:42:08.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:08 smithi203 ceph-mon[32800]: Detected new or changed devices on smithi203 2024-04-23T14:42:08.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:08.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:08.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:08.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:08.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:08 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi203 to 8544M 2024-04-23T14:42:08.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:08.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:08.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:42:08.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:08 smithi203 ceph-mon[32800]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:08.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:08.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:08 smithi203 ceph-mon[32800]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T14:42:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:08 smithi173 ceph-mon[32443]: Detected new or changed devices on smithi203 2024-04-23T14:42:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:08 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi203 to 8544M 2024-04-23T14:42:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:42:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:08 smithi173 ceph-mon[32443]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:08 smithi173 ceph-mon[32443]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T14:42:08.722 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:42:09.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:09 smithi083 ceph-mon[27867]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T14:42:09.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:09 smithi173 ceph-mon[32443]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T14:42:09.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:09 smithi203 ceph-mon[32800]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T14:42:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:10 smithi083 ceph-mon[27867]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:10 smithi083 ceph-mon[27867]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T14:42:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:10 smithi173 ceph-mon[32443]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:10 smithi173 ceph-mon[32443]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T14:42:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:10 smithi203 ceph-mon[32800]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:10 smithi203 ceph-mon[32800]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T14:42:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.023+0000 7f040ac00700 1 -- 172.21.15.83:0/3508264698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04041035e0 msgr2=0x7f04041059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.023+0000 7f040ac00700 1 --2- 172.21.15.83:0/3508264698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04041035e0 0x7f04041059d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009990 tx=0x7f03f4009ca0 comp rx=0 tx=0).stop 2024-04-23T14:42:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.023+0000 7f040ac00700 1 -- 172.21.15.83:0/3508264698 shutdown_connections 2024-04-23T14:42:11.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.023+0000 7f040ac00700 1 --2- 172.21.15.83:0/3508264698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0404108840 0x7f040410acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.023+0000 7f040ac00700 1 --2- 172.21.15.83:0/3508264698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04041035e0 0x7f04041059d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.023+0000 7f040ac00700 1 --2- 172.21.15.83:0/3508264698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0404105f10 0x7f0404108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.023+0000 7f040ac00700 1 -- 172.21.15.83:0/3508264698 >> 172.21.15.83:0/3508264698 conn(0x7f04040fd010 msgr2=0x7f04040ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.023+0000 7f040ac00700 1 -- 172.21.15.83:0/3508264698 shutdown_connections 2024-04-23T14:42:11.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.023+0000 7f040ac00700 1 -- 172.21.15.83:0/3508264698 wait complete. 2024-04-23T14:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.024+0000 7f040ac00700 1 Processor -- start 2024-04-23T14:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.024+0000 7f040ac00700 1 -- start start 2024-04-23T14:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f040ac00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04041035e0 0x7f040419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f040ac00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0404105f10 0x7f040419b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f040ac00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0404108840 0x7f04041a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f040ac00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f040410e750 con 0x7f0404108840 2024-04-23T14:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f040ac00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f040410e5d0 con 0x7f04041035e0 2024-04-23T14:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f040ac00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f040410e8d0 con 0x7f0404105f10 2024-04-23T14:42:11.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f0403400700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0404105f10 0x7f040419b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f0403400700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0404105f10 0x7f040419b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:37800/0 (socket says 172.21.15.83:37800) 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f0403400700 1 -- 172.21.15.83:0/3736240069 learned_addr learned my addr 172.21.15.83:0/3736240069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f0403e00700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04041035e0 0x7f040419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f0408c00700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0404108840 0x7f04041a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f0403400700 1 -- 172.21.15.83:0/3736240069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04041035e0 msgr2=0x7f040419af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f0403400700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04041035e0 0x7f040419af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.025+0000 7f0403400700 1 -- 172.21.15.83:0/3736240069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0404108840 msgr2=0x7f04041a2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.026+0000 7f0403400700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0404108840 0x7f04041a2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.026+0000 7f0403400700 1 -- 172.21.15.83:0/3736240069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04041a2bd0 con 0x7f0404105f10 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.026+0000 7f0403400700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0404105f10 0x7f040419b4b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f03f000eb60 tx=0x7f03f000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.026+0000 7f0400c00700 1 -- 172.21.15.83:0/3736240069 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f000cbd0 con 0x7f0404105f10 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.026+0000 7f0400c00700 1 -- 172.21.15.83:0/3736240069 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f0004500 con 0x7f0404105f10 2024-04-23T14:42:11.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.026+0000 7f040ac00700 1 -- 172.21.15.83:0/3736240069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04041a2ec0 con 0x7f0404105f10 2024-04-23T14:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.026+0000 7f0400c00700 1 -- 172.21.15.83:0/3736240069 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0010430 con 0x7f0404105f10 2024-04-23T14:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.026+0000 7f040ac00700 1 -- 172.21.15.83:0/3736240069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04041a3370 con 0x7f0404105f10 2024-04-23T14:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.027+0000 7f040ac00700 1 -- 172.21.15.83:0/3736240069 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04040697e0 con 0x7f0404105f10 2024-04-23T14:42:11.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.029+0000 7f0400c00700 1 -- 172.21.15.83:0/3736240069 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f03f0003680 con 0x7f0404105f10 2024-04-23T14:42:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.031+0000 7f0400c00700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f03e80a3c40 0x7f03e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.031+0000 7f0400c00700 1 -- 172.21.15.83:0/3736240069 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f03f00cf790 con 0x7f0404105f10 2024-04-23T14:42:11.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.031+0000 7f0403e00700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f03e80a3c40 0x7f03e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:11.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.032+0000 7f0403e00700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f03e80a3c40 0x7f03e80a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f03f400b2e0 tx=0x7f03f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:11.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.035+0000 7f0400c00700 1 -- 172.21.15.83:0/3736240069 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03f000cd70 con 0x7f0404105f10 2024-04-23T14:42:11.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:11.272+0000 7f040ac00700 1 -- 172.21.15.83:0/3736240069 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0404062f30 con 0x7f03e80a3c40 2024-04-23T14:42:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:12 smithi083 ceph-mon[27867]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:12 smithi083 ceph-mon[27867]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:42:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:42:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:42:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:12 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:12.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:12 smithi173 ceph-mon[32443]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:12.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:12 smithi173 ceph-mon[32443]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:42:12.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:42:12.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:42:12.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:12 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:12 smithi203 ceph-mon[32800]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:12 smithi203 ceph-mon[32800]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:42:12.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:42:12.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:42:12.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:12 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:14.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:14 smithi083 ceph-mon[27867]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T14:42:14.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:14 smithi173 ceph-mon[32443]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T14:42:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:14 smithi203 ceph-mon[32800]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T14:42:16.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:16 smithi203 ceph-mon[32800]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-23T14:42:16.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:16 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2245186776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "835bb798-4cbd-4689-af88-4da9744dd601"}]: dispatch 2024-04-23T14:42:16.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:16 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2245186776' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "835bb798-4cbd-4689-af88-4da9744dd601"}]': finished 2024-04-23T14:42:16.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:16 smithi203 ceph-mon[32800]: osdmap e43: 7 total, 6 up, 7 in 2024-04-23T14:42:16.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:16 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:16.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:16 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1373599482' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:42:16.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:16 smithi173 ceph-mon[32443]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-23T14:42:16.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:16 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2245186776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "835bb798-4cbd-4689-af88-4da9744dd601"}]: dispatch 2024-04-23T14:42:16.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:16 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2245186776' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "835bb798-4cbd-4689-af88-4da9744dd601"}]': finished 2024-04-23T14:42:16.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:16 smithi173 ceph-mon[32443]: osdmap e43: 7 total, 6 up, 7 in 2024-04-23T14:42:16.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:16 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:16.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:16 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1373599482' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:42:16.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:16 smithi083 ceph-mon[27867]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-23T14:42:16.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:16 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2245186776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "835bb798-4cbd-4689-af88-4da9744dd601"}]: dispatch 2024-04-23T14:42:16.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:16 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2245186776' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "835bb798-4cbd-4689-af88-4da9744dd601"}]': finished 2024-04-23T14:42:16.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:16 smithi083 ceph-mon[27867]: osdmap e43: 7 total, 6 up, 7 in 2024-04-23T14:42:16.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:16 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:16.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:16 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1373599482' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:42:18.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:18 smithi083 ceph-mon[27867]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T14:42:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:18 smithi173 ceph-mon[32443]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T14:42:18.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:18 smithi203 ceph-mon[32800]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T14:42:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:20 smithi173 ceph-mon[32443]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T14:42:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:20 smithi203 ceph-mon[32800]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T14:42:20.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:20 smithi083 ceph-mon[27867]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T14:42:22.531 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:22 smithi083 ceph-mon[27867]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T14:42:22.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:22 smithi173 ceph-mon[32443]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T14:42:22.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:22 smithi203 ceph-mon[32800]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T14:42:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:24 smithi173 ceph-mon[32443]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:24 smithi203 ceph-mon[32800]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:24.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:24 smithi083 ceph-mon[27867]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:25 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T14:42:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:25 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:25 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T14:42:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:25 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:25.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:25 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T14:42:25.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:25 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:26 smithi173 ceph-mon[32443]: Deploying daemon osd.6 on smithi083 2024-04-23T14:42:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:26 smithi173 ceph-mon[32443]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:26 smithi203 ceph-mon[32800]: Deploying daemon osd.6 on smithi083 2024-04-23T14:42:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:26 smithi203 ceph-mon[32800]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:26.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:26 smithi083 ceph-mon[27867]: Deploying daemon osd.6 on smithi083 2024-04-23T14:42:26.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:26 smithi083 ceph-mon[27867]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:28.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:28 smithi173 ceph-mon[32443]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:28.692 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:28 smithi083 ceph-mon[27867]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:28.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:28 smithi203 ceph-mon[32800]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:30.406 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:30 smithi083 ceph-mon[27867]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:30 smithi173 ceph-mon[32443]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:30.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:30 smithi203 ceph-mon[32800]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:32.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:32 smithi173 ceph-mon[32443]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:32.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:32 smithi203 ceph-mon[32800]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:32.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:32 smithi083 ceph-mon[27867]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:34.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:34 smithi083 ceph-mon[27867]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:42:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:34 smithi173 ceph-mon[32443]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:42:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:34.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:34 smithi203 ceph-mon[32800]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:34.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:34.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:42:34.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:36 smithi173 ceph-mon[32443]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:36.621 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:36 smithi083 ceph-mon[27867]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:36.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:36 smithi203 ceph-mon[32800]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:38 smithi173 ceph-mon[32443]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:38 smithi173 ceph-mon[32443]: from='osd.6 [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T14:42:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:38 smithi173 ceph-mon[32443]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T14:42:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:38.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:38 smithi203 ceph-mon[32800]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:38.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:38 smithi203 ceph-mon[32800]: from='osd.6 [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T14:42:38.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:38 smithi203 ceph-mon[32800]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T14:42:38.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:38.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:38.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:38 smithi083 ceph-mon[27867]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:38.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:38 smithi083 ceph-mon[27867]: from='osd.6 [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T14:42:38.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:38 smithi083 ceph-mon[27867]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T14:42:38.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:38.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:39.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.031+0000 7f0400c00700 1 -- 172.21.15.83:0/3736240069 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0404062f30 con 0x7f03e80a3c40 2024-04-23T14:42:39.033 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 6 on host 'smithi083' 2024-04-23T14:42:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.035+0000 7f040ac00700 1 -- 172.21.15.83:0/3736240069 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f03e80a3c40 msgr2=0x7f03e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:39.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.035+0000 7f040ac00700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f03e80a3c40 0x7f03e80a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f03f400b2e0 tx=0x7f03f4005fd0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.035+0000 7f040ac00700 1 -- 172.21.15.83:0/3736240069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0404105f10 msgr2=0x7f040419b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.035+0000 7f040ac00700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0404105f10 0x7f040419b4b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f03f000eb60 tx=0x7f03f000ee70 comp rx=0 tx=0).stop 2024-04-23T14:42:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.035+0000 7f040ac00700 1 -- 172.21.15.83:0/3736240069 shutdown_connections 2024-04-23T14:42:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.035+0000 7f040ac00700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f03e80a3c40 0x7f03e80a6100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.035+0000 7f040ac00700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0404108840 0x7f04041a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.035+0000 7f040ac00700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04041035e0 0x7f040419af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.035+0000 7f040ac00700 1 --2- 172.21.15.83:0/3736240069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0404105f10 0x7f040419b4b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.035+0000 7f040ac00700 1 -- 172.21.15.83:0/3736240069 >> 172.21.15.83:0/3736240069 conn(0x7f04040fd010 msgr2=0x7f0404102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.035+0000 7f040ac00700 1 -- 172.21.15.83:0/3736240069 shutdown_connections 2024-04-23T14:42:39.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:42:39.035+0000 7f040ac00700 1 -- 172.21.15.83:0/3736240069 wait complete. 2024-04-23T14:42:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:39 smithi203 ceph-mon[32800]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T14:42:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:39 smithi203 ceph-mon[32800]: from='osd.6 [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:42:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:39 smithi203 ceph-mon[32800]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T14:42:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:39 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:39.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:39 smithi203 ceph-mon[32800]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:42:39.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:39 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:39.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:39 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:39.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:39 smithi203 ceph-mon[32800]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:39.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:39 smithi083 ceph-mon[27867]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T14:42:39.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:39 smithi083 ceph-mon[27867]: from='osd.6 [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:42:39.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:39 smithi083 ceph-mon[27867]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T14:42:39.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:39.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:39 smithi083 ceph-mon[27867]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:42:39.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:39.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:39.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:39 smithi083 ceph-mon[27867]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:39 smithi173 ceph-mon[32443]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T14:42:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:39 smithi173 ceph-mon[32443]: from='osd.6 [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:42:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:39 smithi173 ceph-mon[32443]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T14:42:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:39 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:39 smithi173 ceph-mon[32443]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:42:39.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:39 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:39.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:39 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:39.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:39 smithi173 ceph-mon[32443]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:40.024 DEBUG:teuthology.orchestra.run.smithi083:osd.6> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.6.service 2024-04-23T14:42:40.031 INFO:tasks.cephadm:Deploying osd.7 on smithi173 with /dev/vg_nvme/lv_2... 2024-04-23T14:42:40.031 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T14:42:40.274 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T14:42:40.599 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:40 smithi083 ceph-mon[27867]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-23T14:42:40.599 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:40 smithi083 ceph-mon[27867]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T14:42:40.599 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:40 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:40.600 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:40 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:40.626 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:40 smithi173 ceph-mon[32443]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-23T14:42:40.626 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:40 smithi173 ceph-mon[32443]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T14:42:40.626 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:40 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:40.626 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:40 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:40.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:40 smithi203 ceph-mon[32800]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-23T14:42:40.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:40 smithi203 ceph-mon[32800]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T14:42:40.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:40 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:40.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:40 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:41.615 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:41 smithi173 ceph-mon[32443]: purged_snaps scrub starts 2024-04-23T14:42:41.615 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:41 smithi173 ceph-mon[32443]: purged_snaps scrub ok 2024-04-23T14:42:41.615 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:41.615 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:41 smithi173 ceph-mon[32443]: osd.6 [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631] boot 2024-04-23T14:42:41.615 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:41 smithi173 ceph-mon[32443]: osdmap e46: 7 total, 7 up, 7 in 2024-04-23T14:42:41.615 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:41.615 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:41 smithi173 ceph-mon[32443]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:41.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:41 smithi203 ceph-mon[32800]: purged_snaps scrub starts 2024-04-23T14:42:41.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:41 smithi203 ceph-mon[32800]: purged_snaps scrub ok 2024-04-23T14:42:41.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:41.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:41 smithi203 ceph-mon[32800]: osd.6 [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631] boot 2024-04-23T14:42:41.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:41 smithi203 ceph-mon[32800]: osdmap e46: 7 total, 7 up, 7 in 2024-04-23T14:42:41.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:41.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:41 smithi203 ceph-mon[32800]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:41.737 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:41 smithi083 ceph-mon[27867]: purged_snaps scrub starts 2024-04-23T14:42:41.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:41 smithi083 ceph-mon[27867]: purged_snaps scrub ok 2024-04-23T14:42:41.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:41.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:41 smithi083 ceph-mon[27867]: osd.6 [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631] boot 2024-04-23T14:42:41.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:41 smithi083 ceph-mon[27867]: osdmap e46: 7 total, 7 up, 7 in 2024-04-23T14:42:41.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:42:41.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:41 smithi083 ceph-mon[27867]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T14:42:42.298 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T14:42:42.298 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T14:42:42.298 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-23T14:42:42.299 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-23T14:42:42.299 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112407 s, 933 MB/s 2024-04-23T14:42:42.299 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-23T14:42:42.575 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:42 smithi083 ceph-mon[27867]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T14:42:42.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:42 smithi173 ceph-mon[32443]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T14:42:42.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:42 smithi203 ceph-mon[32800]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T14:42:43.003 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_2 2024-04-23T14:42:43.240 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T14:42:43.685 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:43 smithi173 ceph-mon[32443]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:43.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:43 smithi203 ceph-mon[32800]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:43.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:43 smithi083 ceph-mon[27867]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:44.900 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.896+0000 7f8f9ceae700 1 -- 172.21.15.173:0/4248466722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f98103d50 msgr2=0x7f8f981041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:44.900 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.896+0000 7f8f9ceae700 1 --2- 172.21.15.173:0/4248466722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f98103d50 0x7f8f981041b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0099c0 tx=0x7f8f8c009cd0 comp rx=0 tx=0).stop 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.897+0000 7f8f9ceae700 1 -- 172.21.15.173:0/4248466722 shutdown_connections 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.897+0000 7f8f9ceae700 1 --2- 172.21.15.173:0/4248466722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f981046f0 0x7f8f9810afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.897+0000 7f8f9ceae700 1 --2- 172.21.15.173:0/4248466722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f98102b50 0x7f8f98102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.897+0000 7f8f9ceae700 1 --2- 172.21.15.173:0/4248466722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f98103d50 0x7f8f981041b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.897+0000 7f8f9ceae700 1 -- 172.21.15.173:0/4248466722 >> 172.21.15.173:0/4248466722 conn(0x7f8f980fe0f0 msgr2=0x7f8f98100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.897+0000 7f8f9ceae700 1 -- 172.21.15.173:0/4248466722 shutdown_connections 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.897+0000 7f8f9ceae700 1 -- 172.21.15.173:0/4248466722 wait complete. 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.897+0000 7f8f9ceae700 1 Processor -- start 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.897+0000 7f8f9ceae700 1 -- start start 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9ceae700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f98102b50 0x7f8f98198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9ceae700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f98103d50 0x7f8f981992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9ceae700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f981046f0 0x7f8f981a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:44.901 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9ceae700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9810e790 con 0x7f8f981046f0 2024-04-23T14:42:44.902 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9ceae700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9810e610 con 0x7f8f98102b50 2024-04-23T14:42:44.902 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9ceae700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9810e910 con 0x7f8f98103d50 2024-04-23T14:42:44.902 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9659c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f98102b50 0x7f8f98198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:44.902 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9659c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f98102b50 0x7f8f98198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:34590/0 (socket says 172.21.15.173:34590) 2024-04-23T14:42:44.902 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9659c700 1 -- 172.21.15.173:0/2195693378 learned_addr learned my addr 172.21.15.173:0/2195693378 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T14:42:44.902 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f95d9b700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f98103d50 0x7f8f981992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:44.902 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9659c700 1 -- 172.21.15.173:0/2195693378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f98103d50 msgr2=0x7f8f981992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:44.902 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9659c700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f98103d50 0x7f8f981992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:44.902 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9659c700 1 -- 172.21.15.173:0/2195693378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f981046f0 msgr2=0x7f8f981a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:42:44.902 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9659c700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f981046f0 0x7f8f981a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:42:44.902 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9659c700 1 -- 172.21.15.173:0/2195693378 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f981a0a10 con 0x7f8f98102b50 2024-04-23T14:42:44.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f95d9b700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f98103d50 0x7f8f981992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:42:44.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9659c700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f98102b50 0x7f8f98198db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c970 tx=0x7f8f8800cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:44.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f877fe700 1 -- 172.21.15.173:0/2195693378 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88004220 con 0x7f8f98102b50 2024-04-23T14:42:44.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9ceae700 1 -- 172.21.15.173:0/2195693378 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f981a0d00 con 0x7f8f98102b50 2024-04-23T14:42:44.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f877fe700 1 -- 172.21.15.173:0/2195693378 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f88007d80 con 0x7f8f98102b50 2024-04-23T14:42:44.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f877fe700 1 -- 172.21.15.173:0/2195693378 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8800f450 con 0x7f8f98102b50 2024-04-23T14:42:44.903 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.898+0000 7f8f9ceae700 1 -- 172.21.15.173:0/2195693378 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f981a1210 con 0x7f8f98102b50 2024-04-23T14:42:44.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.899+0000 7f8f9ceae700 1 -- 172.21.15.173:0/2195693378 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f9802dfe0 con 0x7f8f98102b50 2024-04-23T14:42:44.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.900+0000 7f8f877fe700 1 -- 172.21.15.173:0/2195693378 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8f880078a0 con 0x7f8f98102b50 2024-04-23T14:42:44.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.901+0000 7f8f877fe700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8f800a3c40 0x7f8f800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:42:44.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.901+0000 7f8f877fe700 1 -- 172.21.15.173:0/2195693378 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f8f880cf350 con 0x7f8f98102b50 2024-04-23T14:42:44.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.902+0000 7f8f95d9b700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8f800a3c40 0x7f8f800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:42:44.904 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.903+0000 7f8f95d9b700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8f800a3c40 0x7f8f800a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c000c00 tx=0x7f8f8c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:42:44.907 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:44.906+0000 7f8f877fe700 1 -- 172.21.15.173:0/2195693378 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f88014030 con 0x7f8f98102b50 2024-04-23T14:42:45.114 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:42:45.112+0000 7f8f9ceae700 1 -- 172.21.15.173:0/2195693378 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8f98063930 con 0x7f8f800a3c40 2024-04-23T14:42:45.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:45 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:42:45.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:45 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:42:45.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:45 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:45.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:45 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:45.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:45 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:45.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:45 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:45.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:45 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:45.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:45 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:45.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:45 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:42:45.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:45 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:42:45.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:45 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:45.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:45 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:45.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:45 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:45.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:45 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:45.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:45 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:45.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:45 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:45.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:45 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:42:45.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:45 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:42:45.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:45 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:45.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:45 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:45.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:45 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:45.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:45 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:45.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:45 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:45.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:45 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:42:46.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:46 smithi173 ceph-mon[32443]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:42:46.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:46 smithi173 ceph-mon[32443]: Detected new or changed devices on smithi083 2024-04-23T14:42:46.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:46 smithi173 ceph-mon[32443]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:46.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:46 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-23T14:42:46.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:46 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:46.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:46 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:46.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:46 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:42:46.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:46 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:46.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:46 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:42:46.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:46 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:46.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:46 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:42:46.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:46 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:46.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:46 smithi203 ceph-mon[32800]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:42:46.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:46 smithi203 ceph-mon[32800]: Detected new or changed devices on smithi083 2024-04-23T14:42:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:46 smithi203 ceph-mon[32800]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:46 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-23T14:42:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:46 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:46 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:46 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:42:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:46 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:46 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:42:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:46 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:46 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:42:46.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:46 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:46 smithi083 ceph-mon[27867]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:42:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:46 smithi083 ceph-mon[27867]: Detected new or changed devices on smithi083 2024-04-23T14:42:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:46 smithi083 ceph-mon[27867]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:46 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi083 to 5696M 2024-04-23T14:42:46.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:46.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:46.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:42:46.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:46.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:42:46.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:46.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:42:46.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:46 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:42:48.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:48 smithi203 ceph-mon[32800]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:48.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:48 smithi203 ceph-mon[32800]: from='client.? 172.21.15.173:0/3872342282' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0ac95e1-5d76-4f47-9483-1420f8c834c1"}]: dispatch 2024-04-23T14:42:48.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:48 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0ac95e1-5d76-4f47-9483-1420f8c834c1"}]: dispatch 2024-04-23T14:42:48.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:48 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0ac95e1-5d76-4f47-9483-1420f8c834c1"}]': finished 2024-04-23T14:42:48.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:48 smithi203 ceph-mon[32800]: osdmap e48: 8 total, 7 up, 8 in 2024-04-23T14:42:48.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:48 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:42:48.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:48 smithi083 ceph-mon[27867]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:48.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:48 smithi083 ceph-mon[27867]: from='client.? 172.21.15.173:0/3872342282' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0ac95e1-5d76-4f47-9483-1420f8c834c1"}]: dispatch 2024-04-23T14:42:48.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:48 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0ac95e1-5d76-4f47-9483-1420f8c834c1"}]: dispatch 2024-04-23T14:42:48.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:48 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0ac95e1-5d76-4f47-9483-1420f8c834c1"}]': finished 2024-04-23T14:42:48.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:48 smithi083 ceph-mon[27867]: osdmap e48: 8 total, 7 up, 8 in 2024-04-23T14:42:48.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:48 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:42:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:48 smithi173 ceph-mon[32443]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:48 smithi173 ceph-mon[32443]: from='client.? 172.21.15.173:0/3872342282' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0ac95e1-5d76-4f47-9483-1420f8c834c1"}]: dispatch 2024-04-23T14:42:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:48 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0ac95e1-5d76-4f47-9483-1420f8c834c1"}]: dispatch 2024-04-23T14:42:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:48 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0ac95e1-5d76-4f47-9483-1420f8c834c1"}]': finished 2024-04-23T14:42:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:48 smithi173 ceph-mon[32443]: osdmap e48: 8 total, 7 up, 8 in 2024-04-23T14:42:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:48 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:42:49.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:49 smithi203 ceph-mon[32800]: from='client.? 172.21.15.173:0/402570408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:42:49.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:49 smithi083 ceph-mon[27867]: from='client.? 172.21.15.173:0/402570408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:42:49.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:49 smithi173 ceph-mon[32443]: from='client.? 172.21.15.173:0/402570408' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:42:50.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:50 smithi203 ceph-mon[32800]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:50.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:50 smithi083 ceph-mon[27867]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:50.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:50 smithi173 ceph-mon[32443]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:52.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:52 smithi203 ceph-mon[32800]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:52.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:52 smithi083 ceph-mon[27867]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:52.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:52 smithi173 ceph-mon[32443]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:54.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:54 smithi173 ceph-mon[32443]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:54.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:54 smithi203 ceph-mon[32800]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:54.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:54 smithi083 ceph-mon[27867]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:55.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:55 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T14:42:55.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:55 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:55.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:55 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T14:42:55.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:55 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:55 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T14:42:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:55 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:42:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:56 smithi083 ceph-mon[27867]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:56 smithi083 ceph-mon[27867]: Deploying daemon osd.7 on smithi173 2024-04-23T14:42:56.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:56 smithi173 ceph-mon[32443]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:56.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:56 smithi173 ceph-mon[32443]: Deploying daemon osd.7 on smithi173 2024-04-23T14:42:56.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:56 smithi203 ceph-mon[32800]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:56.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:56 smithi203 ceph-mon[32800]: Deploying daemon osd.7 on smithi173 2024-04-23T14:42:58.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:42:58 smithi083 ceph-mon[27867]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:58.514 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:42:58 smithi173 ceph-mon[32443]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:42:58.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:42:58 smithi203 ceph-mon[32800]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:00.287 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:00 smithi173 ceph-mon[32443]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:00.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:00 smithi083 ceph-mon[27867]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:00.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:00 smithi203 ceph-mon[32800]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:02.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:01 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:02.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:01 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:02.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:01 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:43:02.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:01 smithi203 ceph-mon[32800]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:02.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:01 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:02.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:01 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:02.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:01 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:43:02.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:01 smithi083 ceph-mon[27867]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:02.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:01 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:02.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:01 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:02.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:01 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:43:02.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:01 smithi173 ceph-mon[32443]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:04.453 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:04 smithi173 ceph-mon[32443]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:04.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:04 smithi203 ceph-mon[32800]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:04.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:04 smithi083 ceph-mon[27867]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:04.548 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.547+0000 7f8f877fe700 1 -- 172.21.15.173:0/2195693378 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8f98063930 con 0x7f8f800a3c40 2024-04-23T14:43:04.548 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 7 on host 'smithi173' 2024-04-23T14:43:04.550 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.550+0000 7f8f9ceae700 1 -- 172.21.15.173:0/2195693378 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8f800a3c40 msgr2=0x7f8f800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:04.551 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.550+0000 7f8f9ceae700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8f800a3c40 0x7f8f800a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c000c00 tx=0x7f8f8c005fd0 comp rx=0 tx=0).stop 2024-04-23T14:43:04.551 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.550+0000 7f8f9ceae700 1 -- 172.21.15.173:0/2195693378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f98102b50 msgr2=0x7f8f98198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:04.551 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.550+0000 7f8f9ceae700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f98102b50 0x7f8f98198db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c970 tx=0x7f8f8800cd30 comp rx=0 tx=0).stop 2024-04-23T14:43:04.551 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.550+0000 7f8f9ceae700 1 -- 172.21.15.173:0/2195693378 shutdown_connections 2024-04-23T14:43:04.551 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.550+0000 7f8f9ceae700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8f800a3c40 0x7f8f800a6100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:04.551 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.550+0000 7f8f9ceae700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f981046f0 0x7f8f981a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:04.551 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.550+0000 7f8f9ceae700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8f98102b50 0x7f8f98198db0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:04.551 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.550+0000 7f8f9ceae700 1 --2- 172.21.15.173:0/2195693378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f98103d50 0x7f8f981992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:04.551 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.550+0000 7f8f9ceae700 1 -- 172.21.15.173:0/2195693378 >> 172.21.15.173:0/2195693378 conn(0x7f8f980fe0f0 msgr2=0x7f8f981070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:04.551 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.550+0000 7f8f9ceae700 1 -- 172.21.15.173:0/2195693378 shutdown_connections 2024-04-23T14:43:04.551 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:43:04.550+0000 7f8f9ceae700 1 -- 172.21.15.173:0/2195693378 wait complete. 2024-04-23T14:43:05.261 DEBUG:teuthology.orchestra.run.smithi173:osd.7> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.7.service 2024-04-23T14:43:05.264 INFO:tasks.cephadm:Deploying osd.8 on smithi203 with /dev/vg_nvme/lv_2... 2024-04-23T14:43:05.264 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T14:43:05.513 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:43:05.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:05.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:05.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:05 smithi203 ceph-mon[32800]: from='osd.7 [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T14:43:05.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:05 smithi203 ceph-mon[32800]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T14:43:05.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:05.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:05.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:05 smithi203 ceph-mon[32800]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:05 smithi083 ceph-mon[27867]: from='osd.7 [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T14:43:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:05 smithi083 ceph-mon[27867]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T14:43:05.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:05.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:05.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:05 smithi083 ceph-mon[27867]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:05.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:05 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:05.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:05 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:05.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:05 smithi173 ceph-mon[32443]: from='osd.7 [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T14:43:05.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:05 smithi173 ceph-mon[32443]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T14:43:05.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:05 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:05.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:05 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:05.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:05 smithi173 ceph-mon[32443]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:06.823 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:06 smithi173 ceph-mon[32443]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T14:43:06.823 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:06 smithi173 ceph-mon[32443]: from='osd.7 [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:43:06.823 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:06 smithi173 ceph-mon[32443]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T14:43:06.823 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:06 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:06.823 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:06 smithi173 ceph-mon[32443]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:43:06.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:06 smithi203 ceph-mon[32800]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T14:43:06.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:06 smithi203 ceph-mon[32800]: from='osd.7 [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:43:06.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:06 smithi203 ceph-mon[32800]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T14:43:06.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:06 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:06.839 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:06 smithi203 ceph-mon[32800]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:43:07.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:06 smithi083 ceph-mon[27867]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T14:43:07.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:06 smithi083 ceph-mon[27867]: from='osd.7 [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:43:07.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:06 smithi083 ceph-mon[27867]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T14:43:07.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:06 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:07.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:06 smithi083 ceph-mon[27867]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:43:07.076 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 14:43:07 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7[50825]: 2024-04-23T14:43:07.050+0000 7faee6957700 -1 osd.7 0 waiting for initial osdmap 2024-04-23T14:43:07.076 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 14:43:07 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7[50825]: 2024-04-23T14:43:07.059+0000 7faede742700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:43:07.550 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T14:43:07.550 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T14:43:07.550 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-04-23T14:43:07.550 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-04-23T14:43:07.550 INFO:teuthology.orchestra.run.smithi203.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124963 s, 839 MB/s 2024-04-23T14:43:07.550 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-04-23T14:43:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:07 smithi203 ceph-mon[32800]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T14:43:07.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:07 smithi203 ceph-mon[32800]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T14:43:07.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:07 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:07.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:07 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:07.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:07 smithi203 ceph-mon[32800]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T14:43:07.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:07 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:07.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:07 smithi203 ceph-mon[32800]: from='osd.7 ' entity='osd.7' 2024-04-23T14:43:07.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:07 smithi203 ceph-mon[32800]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:07.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:07 smithi173 ceph-mon[32443]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T14:43:07.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:07 smithi173 ceph-mon[32443]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T14:43:07.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:07.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:07.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:07 smithi173 ceph-mon[32443]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T14:43:07.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:07.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:07 smithi173 ceph-mon[32443]: from='osd.7 ' entity='osd.7' 2024-04-23T14:43:07.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:07 smithi173 ceph-mon[32443]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:08.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:07 smithi083 ceph-mon[27867]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T14:43:08.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:07 smithi083 ceph-mon[27867]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T14:43:08.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:07 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:08.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:07 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:08.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:07 smithi083 ceph-mon[27867]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T14:43:08.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:07 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:08.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:07 smithi083 ceph-mon[27867]: from='osd.7 ' entity='osd.7' 2024-04-23T14:43:08.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:07 smithi083 ceph-mon[27867]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T14:43:08.254 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_2 2024-04-23T14:43:08.495 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:43:08.780 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: purged_snaps scrub starts 2024-04-23T14:43:08.780 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: purged_snaps scrub ok 2024-04-23T14:43:08.780 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:08.781 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: osd.7 [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936] boot 2024-04-23T14:43:08.781 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T14:43:08.781 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:08.781 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: Detected new or changed devices on smithi173 2024-04-23T14:43:08.781 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:08.781 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:08.782 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:08.782 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:08.782 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:08.782 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:08.782 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:08.782 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:43:08.783 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:08.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: purged_snaps scrub starts 2024-04-23T14:43:08.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: purged_snaps scrub ok 2024-04-23T14:43:08.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:08.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: osd.7 [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936] boot 2024-04-23T14:43:08.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T14:43:08.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:08.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: Detected new or changed devices on smithi173 2024-04-23T14:43:08.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:08.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:08.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:08.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:08.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:08.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:08.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:08.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:43:08.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:09.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: purged_snaps scrub starts 2024-04-23T14:43:09.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: purged_snaps scrub ok 2024-04-23T14:43:09.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:09.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: osd.7 [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936] boot 2024-04-23T14:43:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T14:43:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:43:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: Detected new or changed devices on smithi173 2024-04-23T14:43:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:43:09.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:09.651 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:09 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi173 to 5696M 2024-04-23T14:43:09.652 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:43:09.652 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:09.652 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:43:09.652 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:09 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:09.652 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:09 smithi203 ceph-mon[32800]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T14:43:09.652 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:09 smithi203 ceph-mon[32800]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:09 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi173 to 5696M 2024-04-23T14:43:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:43:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:43:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:09 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:09 smithi083 ceph-mon[27867]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T14:43:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:09 smithi083 ceph-mon[27867]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:09 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi173 to 5696M 2024-04-23T14:43:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:43:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:43:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:09 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:09 smithi173 ceph-mon[32443]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T14:43:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:09 smithi173 ceph-mon[32443]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:10.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.175+0000 7f2409e97700 1 -- 172.21.15.203:0/1115680031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2404103610 msgr2=0x7f2404105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:10.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.175+0000 7f2409e97700 1 --2- 172.21.15.203:0/1115680031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2404103610 0x7f2404105a00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009990 tx=0x7f23fc009ca0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.175+0000 7f2409e97700 1 -- 172.21.15.203:0/1115680031 shutdown_connections 2024-04-23T14:43:10.176 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.175+0000 7f2409e97700 1 --2- 172.21.15.203:0/1115680031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2404108870 0x7f240410ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.175+0000 7f2409e97700 1 --2- 172.21.15.203:0/1115680031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2404103610 0x7f2404105a00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.175+0000 7f2409e97700 1 --2- 172.21.15.203:0/1115680031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2404105f40 0x7f2404108330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.175+0000 7f2409e97700 1 -- 172.21.15.203:0/1115680031 >> 172.21.15.203:0/1115680031 conn(0x7f24040fd040 msgr2=0x7f24040ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:10.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.175+0000 7f2409e97700 1 -- 172.21.15.203:0/1115680031 shutdown_connections 2024-04-23T14:43:10.177 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.175+0000 7f2409e97700 1 -- 172.21.15.203:0/1115680031 wait complete. 2024-04-23T14:43:10.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.176+0000 7f2409e97700 1 Processor -- start 2024-04-23T14:43:10.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.176+0000 7f2409e97700 1 -- start start 2024-04-23T14:43:10.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.176+0000 7f2409e97700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2404103610 0x7f2404102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.176+0000 7f2409e97700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2404105f40 0x7f24041015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.176+0000 7f2409e97700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2404108870 0x7f2404101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.176+0000 7f2409e97700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f240410e850 con 0x7f2404108870 2024-04-23T14:43:10.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.176+0000 7f2409e97700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f240410e6d0 con 0x7f2404105f40 2024-04-23T14:43:10.179 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.176+0000 7f2409e97700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f240410e9d0 con 0x7f2404103610 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f24037fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2404103610 0x7f2404102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f24037fe700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2404103610 0x7f2404102f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:57030/0 (socket says 172.21.15.203:57030) 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f24037fe700 1 -- 172.21.15.203:0/2197957449 learned_addr learned my addr 172.21.15.203:0/2197957449 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f2402ffd700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2404105f40 0x7f24041015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f24037fe700 1 -- 172.21.15.203:0/2197957449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2404105f40 msgr2=0x7f24041015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f24037fe700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2404105f40 0x7f24041015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f24037fe700 1 -- 172.21.15.203:0/2197957449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2404108870 msgr2=0x7f2404101b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f24037fe700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2404108870 0x7f2404101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f24037fe700 1 -- 172.21.15.203:0/2197957449 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24041023d0 con 0x7f2404103610 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f24037fe700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2404103610 0x7f2404102f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f23fc005910 tx=0x7f23fc0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f2400ff9700 1 -- 172.21.15.203:0/2197957449 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc027070 con 0x7f2404103610 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f2400ff9700 1 -- 172.21.15.203:0/2197957449 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23fc022470 con 0x7f2404103610 2024-04-23T14:43:10.180 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f2409e97700 1 -- 172.21.15.203:0/2197957449 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24041a2f90 con 0x7f2404103610 2024-04-23T14:43:10.181 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f2400ff9700 1 -- 172.21.15.203:0/2197957449 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc00f6c0 con 0x7f2404103610 2024-04-23T14:43:10.181 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.177+0000 7f2409e97700 1 -- 172.21.15.203:0/2197957449 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24041a33b0 con 0x7f2404103610 2024-04-23T14:43:10.181 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.178+0000 7f2409e97700 1 -- 172.21.15.203:0/2197957449 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24040697e0 con 0x7f2404103610 2024-04-23T14:43:10.181 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.179+0000 7f2400ff9700 1 -- 172.21.15.203:0/2197957449 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f23fc02a070 con 0x7f2404103610 2024-04-23T14:43:10.181 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.180+0000 7f2400ff9700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f23f00a3cc0 0x7f23f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:10.182 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.180+0000 7f2400ff9700 1 -- 172.21.15.203:0/2197957449 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f23fc0d0d70 con 0x7f2404103610 2024-04-23T14:43:10.182 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.180+0000 7f2402ffd700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f23f00a3cc0 0x7f23f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:10.182 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.181+0000 7f2402ffd700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f23f00a3cc0 0x7f23f00a6180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23ec006fd0 tx=0x7f23ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:10.184 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.184+0000 7f2400ff9700 1 -- 172.21.15.203:0/2197957449 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23fc031070 con 0x7f2404103610 2024-04-23T14:43:10.389 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:10.388+0000 7f2409e97700 1 -- 172.21.15.203:0/2197957449 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2404062f30 con 0x7f23f00a3cc0 2024-04-23T14:43:11.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:11 smithi173 ceph-mon[32443]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T14:43:11.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:11 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:43:11.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:11 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:43:11.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:11 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:11.359 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:11 smithi203 ceph-mon[32800]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T14:43:11.359 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:11 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:43:11.359 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:11 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:43:11.359 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:11 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:11.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:11 smithi083 ceph-mon[27867]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T14:43:11.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:11 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:43:11.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:11 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:43:11.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:11 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:12.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:12 smithi173 ceph-mon[32443]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:43:12.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:12 smithi173 ceph-mon[32443]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T14:43:12.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:12 smithi173 ceph-mon[32443]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:12.348 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:12 smithi203 ceph-mon[32800]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:43:12.349 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:12 smithi203 ceph-mon[32800]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T14:43:12.349 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:12 smithi203 ceph-mon[32800]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:12 smithi083 ceph-mon[27867]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:43:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:12 smithi083 ceph-mon[27867]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T14:43:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:12 smithi083 ceph-mon[27867]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:13.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:13 smithi173 ceph-mon[32443]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T14:43:13.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:13 smithi203 ceph-mon[32800]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T14:43:13.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:13 smithi083 ceph-mon[27867]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T14:43:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:14 smithi173 ceph-mon[32443]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:14 smithi173 ceph-mon[32443]: from='client.? 172.21.15.203:0/3020873502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75266f49-b8fb-46c3-aa62-bef7957954dc"}]: dispatch 2024-04-23T14:43:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:14 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75266f49-b8fb-46c3-aa62-bef7957954dc"}]: dispatch 2024-04-23T14:43:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:14 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75266f49-b8fb-46c3-aa62-bef7957954dc"}]': finished 2024-04-23T14:43:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:14 smithi173 ceph-mon[32443]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T14:43:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:14 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:43:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:14 smithi173 ceph-mon[32443]: from='client.? 172.21.15.203:0/1798663675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:43:14.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:14 smithi203 ceph-mon[32800]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:14.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:14 smithi203 ceph-mon[32800]: from='client.? 172.21.15.203:0/3020873502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75266f49-b8fb-46c3-aa62-bef7957954dc"}]: dispatch 2024-04-23T14:43:14.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:14 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75266f49-b8fb-46c3-aa62-bef7957954dc"}]: dispatch 2024-04-23T14:43:14.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:14 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75266f49-b8fb-46c3-aa62-bef7957954dc"}]': finished 2024-04-23T14:43:14.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:14 smithi203 ceph-mon[32800]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T14:43:14.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:14 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:43:14.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:14 smithi203 ceph-mon[32800]: from='client.? 172.21.15.203:0/1798663675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:43:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:14 smithi083 ceph-mon[27867]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:14 smithi083 ceph-mon[27867]: from='client.? 172.21.15.203:0/3020873502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75266f49-b8fb-46c3-aa62-bef7957954dc"}]: dispatch 2024-04-23T14:43:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:14 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75266f49-b8fb-46c3-aa62-bef7957954dc"}]: dispatch 2024-04-23T14:43:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:14 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75266f49-b8fb-46c3-aa62-bef7957954dc"}]': finished 2024-04-23T14:43:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:14 smithi083 ceph-mon[27867]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T14:43:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:14 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:43:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:14 smithi083 ceph-mon[27867]: from='client.? 172.21.15.203:0/1798663675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:43:16.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:16 smithi203 ceph-mon[32800]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:16 smithi083 ceph-mon[27867]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:16.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:16 smithi173 ceph-mon[32443]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:18.421 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:18 smithi203 ceph-mon[32800]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:18.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:18 smithi083 ceph-mon[27867]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:18 smithi173 ceph-mon[32443]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:20.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:20 smithi203 ceph-mon[32800]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:20.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:20 smithi083 ceph-mon[27867]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:20 smithi173 ceph-mon[32443]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:21.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:21 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T14:43:21.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:21 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:21.507 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:21 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T14:43:21.507 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:21 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:21 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T14:43:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:21 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:22.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:22 smithi203 ceph-mon[32800]: Deploying daemon osd.8 on smithi203 2024-04-23T14:43:22.489 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:22 smithi203 ceph-mon[32800]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:22.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:22 smithi083 ceph-mon[27867]: Deploying daemon osd.8 on smithi203 2024-04-23T14:43:22.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:22 smithi083 ceph-mon[27867]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:22.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:22 smithi173 ceph-mon[32443]: Deploying daemon osd.8 on smithi203 2024-04-23T14:43:22.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:22 smithi173 ceph-mon[32443]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:24.404 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:24 smithi203 ceph-mon[32800]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:24.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:24 smithi083 ceph-mon[27867]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:24 smithi173 ceph-mon[32443]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:26.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:26 smithi203 ceph-mon[32800]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:26.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:26 smithi083 ceph-mon[27867]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:26 smithi173 ceph-mon[32443]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:27.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:27 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:27.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:27 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:27.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:27 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:43:27.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:27 smithi203 ceph-mon[32800]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:27.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:27 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:27.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:27 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:27.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:27 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:43:27.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:27 smithi083 ceph-mon[27867]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:27.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:27 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:27.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:27 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:27.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:27 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:43:27.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:27 smithi173 ceph-mon[32443]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:30.467 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:30 smithi203 ceph-mon[32800]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:30.468 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:30 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:30.468 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:30 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:30.468 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:30 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:30.468 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:30 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:43:30.468 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:30 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:30.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:30 smithi083 ceph-mon[27867]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:30.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:30.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:30.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:30.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:43:30.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:30.529 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.525+0000 7f2400ff9700 1 -- 172.21.15.203:0/2197957449 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2404062f30 con 0x7f23f00a3cc0 2024-04-23T14:43:30.530 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 8 on host 'smithi203' 2024-04-23T14:43:30.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.528+0000 7f2409e97700 1 -- 172.21.15.203:0/2197957449 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f23f00a3cc0 msgr2=0x7f23f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:30.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.528+0000 7f2409e97700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f23f00a3cc0 0x7f23f00a6180 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23ec006fd0 tx=0x7f23ec008040 comp rx=0 tx=0).stop 2024-04-23T14:43:30.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.528+0000 7f2409e97700 1 -- 172.21.15.203:0/2197957449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2404103610 msgr2=0x7f2404102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:30.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.528+0000 7f2409e97700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2404103610 0x7f2404102f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f23fc005910 tx=0x7f23fc0047b0 comp rx=0 tx=0).stop 2024-04-23T14:43:30.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.528+0000 7f2409e97700 1 -- 172.21.15.203:0/2197957449 shutdown_connections 2024-04-23T14:43:30.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.528+0000 7f2409e97700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f23f00a3cc0 0x7f23f00a6180 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:30.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.528+0000 7f2409e97700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2404108870 0x7f2404101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:30.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.528+0000 7f2409e97700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2404103610 0x7f2404102f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:30.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.528+0000 7f2409e97700 1 --2- 172.21.15.203:0/2197957449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2404105f40 0x7f24041015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:30.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.528+0000 7f2409e97700 1 -- 172.21.15.203:0/2197957449 >> 172.21.15.203:0/2197957449 conn(0x7f24040fd040 msgr2=0x7f2404103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:30.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.528+0000 7f2409e97700 1 -- 172.21.15.203:0/2197957449 shutdown_connections 2024-04-23T14:43:30.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:43:30.528+0000 7f2409e97700 1 -- 172.21.15.203:0/2197957449 wait complete. 2024-04-23T14:43:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:30 smithi173 ceph-mon[32443]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:30 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:30 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:30 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:30 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:43:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:30 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:31.297 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:31 smithi203 ceph-mon[32800]: from='osd.8 [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T14:43:31.297 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:31 smithi203 ceph-mon[32800]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T14:43:31.297 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:31 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:43:31.297 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:31 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:31.297 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:31 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:31.329 DEBUG:teuthology.orchestra.run.smithi203:osd.8> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.8.service 2024-04-23T14:43:31.331 INFO:tasks.cephadm:Deploying osd.9 on smithi083 with /dev/vg_nvme/lv_1... 2024-04-23T14:43:31.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T14:43:31.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:31 smithi083 ceph-mon[27867]: from='osd.8 [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T14:43:31.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:31 smithi083 ceph-mon[27867]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T14:43:31.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:31 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:43:31.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:31 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:31.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:31 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:31.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:31 smithi173 ceph-mon[32443]: from='osd.8 [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T14:43:31.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:31 smithi173 ceph-mon[32443]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T14:43:31.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:31 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:43:31.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:31 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:31.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:31 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:31.779 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:43:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:32 smithi083 ceph-mon[27867]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:32 smithi083 ceph-mon[27867]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T14:43:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:32 smithi083 ceph-mon[27867]: from='osd.8 [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:43:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:32 smithi083 ceph-mon[27867]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T14:43:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:32 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:43:32.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:32 smithi083 ceph-mon[27867]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:43:32.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:32 smithi173 ceph-mon[32443]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:32.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:32 smithi173 ceph-mon[32443]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T14:43:32.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:32 smithi173 ceph-mon[32443]: from='osd.8 [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:43:32.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:32 smithi173 ceph-mon[32443]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T14:43:32.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:32 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:43:32.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:32 smithi173 ceph-mon[32443]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:43:32.636 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:32 smithi203 ceph-mon[32800]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:32.637 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:32 smithi203 ceph-mon[32800]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T14:43:32.637 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:32 smithi203 ceph-mon[32800]: from='osd.8 [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:43:32.637 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:32 smithi203 ceph-mon[32800]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T14:43:32.637 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:32 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:43:32.637 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:32 smithi203 ceph-mon[32800]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:43:32.894 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 14:43:32 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8[51492]: 2024-04-23T14:43:32.664+0000 7f79e98d2700 -1 osd.8 0 waiting for initial osdmap 2024-04-23T14:43:32.894 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 14:43:32 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8[51492]: 2024-04-23T14:43:32.676+0000 7f79e16bd700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:43:33.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:33 smithi083 ceph-mon[27867]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-04-23T14:43:33.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:33 smithi083 ceph-mon[27867]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T14:43:33.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:33 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:43:33.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:33 smithi083 ceph-mon[27867]: from='osd.8 ' entity='osd.8' 2024-04-23T14:43:33.560 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:33 smithi203 ceph-mon[32800]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-04-23T14:43:33.561 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:33 smithi203 ceph-mon[32800]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T14:43:33.561 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:33 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:43:33.561 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:33 smithi203 ceph-mon[32800]: from='osd.8 ' entity='osd.8' 2024-04-23T14:43:33.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:33 smithi173 ceph-mon[32443]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-04-23T14:43:33.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:33 smithi173 ceph-mon[32443]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T14:43:33.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:33 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:43:33.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:33 smithi173 ceph-mon[32443]: from='osd.8 ' entity='osd.8' 2024-04-23T14:43:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:34 smithi173 ceph-mon[32443]: purged_snaps scrub starts 2024-04-23T14:43:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:34 smithi173 ceph-mon[32443]: purged_snaps scrub ok 2024-04-23T14:43:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:34 smithi173 ceph-mon[32443]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:34.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:34 smithi173 ceph-mon[32443]: osd.8 [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524] boot 2024-04-23T14:43:34.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:34 smithi173 ceph-mon[32443]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T14:43:34.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:43:34.637 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:34 smithi203 ceph-mon[32800]: purged_snaps scrub starts 2024-04-23T14:43:34.637 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:34 smithi203 ceph-mon[32800]: purged_snaps scrub ok 2024-04-23T14:43:34.637 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:34 smithi203 ceph-mon[32800]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:34.637 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:34 smithi203 ceph-mon[32800]: osd.8 [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524] boot 2024-04-23T14:43:34.638 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:34 smithi203 ceph-mon[32800]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T14:43:34.638 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:43:34.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:34 smithi083 ceph-mon[27867]: purged_snaps scrub starts 2024-04-23T14:43:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:34 smithi083 ceph-mon[27867]: purged_snaps scrub ok 2024-04-23T14:43:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:34 smithi083 ceph-mon[27867]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T14:43:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:34 smithi083 ceph-mon[27867]: osd.8 [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524] boot 2024-04-23T14:43:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:34 smithi083 ceph-mon[27867]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T14:43:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:43:34.854 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T14:43:34.854 INFO:teuthology.orchestra.run.smithi083.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T14:43:34.854 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10+0 records in 2024-04-23T14:43:34.854 INFO:teuthology.orchestra.run.smithi083.stderr:10+0 records out 2024-04-23T14:43:34.854 INFO:teuthology.orchestra.run.smithi083.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0336697 s, 311 MB/s 2024-04-23T14:43:34.855 INFO:teuthology.orchestra.run.smithi083.stderr:--> Zapping successful for: 2024-04-23T14:43:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:35 smithi173 ceph-mon[32443]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T14:43:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:35 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:35 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:35 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:35 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:35 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:35 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:35 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:35 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:43:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:35 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:35.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:35 smithi203 ceph-mon[32800]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T14:43:35.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:35 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:35.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:35 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:35.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:35 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:35.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:35 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:35.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:35 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:35.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:35 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:35.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:35 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:35.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:35 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:43:35.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:35 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:35.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:35 smithi083 ceph-mon[27867]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T14:43:35.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:35 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:35.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:35 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:35.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:35 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:35.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:35 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:35.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:35 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:43:35.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:35 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:35.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:35 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:35.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:35 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:43:35.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:35 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:43:35.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_1 2024-04-23T14:43:36.296 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:43:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:36 smithi173 ceph-mon[32443]: Detected new or changed devices on smithi203 2024-04-23T14:43:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:36 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi203 to 5696M 2024-04-23T14:43:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:36 smithi173 ceph-mon[32443]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 291 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:36 smithi173 ceph-mon[32443]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T14:43:36.614 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:36 smithi083 ceph-mon[27867]: Detected new or changed devices on smithi203 2024-04-23T14:43:36.614 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:36 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi203 to 5696M 2024-04-23T14:43:36.614 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:36 smithi083 ceph-mon[27867]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 291 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:36.615 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:36 smithi083 ceph-mon[27867]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T14:43:36.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:36 smithi203 ceph-mon[32800]: Detected new or changed devices on smithi203 2024-04-23T14:43:36.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:36 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi203 to 5696M 2024-04-23T14:43:36.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:36 smithi203 ceph-mon[32800]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 291 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:36.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:36 smithi203 ceph-mon[32800]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T14:43:37.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:37 smithi203 ceph-mon[32800]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T14:43:37.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:37 smithi173 ceph-mon[32443]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T14:43:37.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:37 smithi083 ceph-mon[27867]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T14:43:38.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:38 smithi083 ceph-mon[27867]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:38.408 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:38 smithi083 ceph-mon[27867]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T14:43:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:38 smithi173 ceph-mon[32443]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:38 smithi173 ceph-mon[32443]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T14:43:38.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.577+0000 7f4507200700 1 -- 172.21.15.83:0/3126432800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4500075550 msgr2=0x7f45000759d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:38.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.577+0000 7f4507200700 1 --2- 172.21.15.83:0/3126432800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4500075550 0x7f45000759d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009990 tx=0x7f44f0009ca0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.577+0000 7f4507200700 1 -- 172.21.15.83:0/3126432800 shutdown_connections 2024-04-23T14:43:38.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.577+0000 7f4507200700 1 --2- 172.21.15.83:0/3126432800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f450010a950 0x7f450010cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.577+0000 7f4507200700 1 --2- 172.21.15.83:0/3126432800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4500076b20 0x7f4500074f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.577+0000 7f4507200700 1 --2- 172.21.15.83:0/3126432800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4500075550 0x7f45000759d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.577+0000 7f4507200700 1 -- 172.21.15.83:0/3126432800 >> 172.21.15.83:0/3126432800 conn(0x7f45000fe020 msgr2=0x7f4500100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:43:38.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.577+0000 7f4507200700 1 -- 172.21.15.83:0/3126432800 shutdown_connections 2024-04-23T14:43:38.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.577+0000 7f4507200700 1 -- 172.21.15.83:0/3126432800 wait complete. 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.578+0000 7f4507200700 1 Processor -- start 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.578+0000 7f4507200700 1 -- start start 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4507200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4500075550 0x7f450019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4507200700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4500076b20 0x7f450019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4507200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f450010a950 0x7f45001a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4507200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4500110880 con 0x7f4500075550 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4507200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4500110700 con 0x7f450010a950 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4507200700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4500110a00 con 0x7f4500076b20 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4505e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4500075550 0x7f450019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4505e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4500075550 0x7f450019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50842/0 (socket says 172.21.15.83:50842) 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4505e00700 1 -- 172.21.15.83:0/1356110038 learned_addr learned my addr 172.21.15.83:0/1356110038 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4506800700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f450010a950 0x7f45001a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:38.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4505400700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4500076b20 0x7f450019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4505e00700 1 -- 172.21.15.83:0/1356110038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4500076b20 msgr2=0x7f450019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4505e00700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4500076b20 0x7f450019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4505e00700 1 -- 172.21.15.83:0/1356110038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f450010a950 msgr2=0x7f45001a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4505e00700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f450010a950 0x7f45001a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4505e00700 1 -- 172.21.15.83:0/1356110038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45001a2ac0 con 0x7f4500075550 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.579+0000 7f4506800700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f450010a950 0x7f45001a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.580+0000 7f4505e00700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4500075550 0x7f450019ae60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009f40 tx=0x7f44f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.580+0000 7f44f6a00700 1 -- 172.21.15.83:0/1356110038 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f800c9c0 con 0x7f4500075550 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.580+0000 7f44f6a00700 1 -- 172.21.15.83:0/1356110038 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f8019070 con 0x7f4500075550 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.580+0000 7f4507200700 1 -- 172.21.15.83:0/1356110038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45001a2d50 con 0x7f4500075550 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.580+0000 7f44f6a00700 1 -- 172.21.15.83:0/1356110038 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f8014480 con 0x7f4500075550 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.580+0000 7f4507200700 1 -- 172.21.15.83:0/1356110038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45001a31d0 con 0x7f4500075550 2024-04-23T14:43:38.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.581+0000 7f4507200700 1 -- 172.21.15.83:0/1356110038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45000697e0 con 0x7f4500075550 2024-04-23T14:43:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.583+0000 7f44f6a00700 1 -- 172.21.15.83:0/1356110038 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f44f8012050 con 0x7f4500075550 2024-04-23T14:43:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.585+0000 7f44f6a00700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f44ec0a3c40 0x7f44ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:43:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.585+0000 7f44f6a00700 1 -- 172.21.15.83:0/1356110038 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f44f80cf8e0 con 0x7f4500075550 2024-04-23T14:43:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.585+0000 7f4505400700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f44ec0a3c40 0x7f44ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:43:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.586+0000 7f4505400700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f44ec0a3c40 0x7f44ec0a6100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f44f000b7b0 tx=0x7f44f000b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:43:38.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.593+0000 7f44f6a00700 1 -- 172.21.15.83:0/1356110038 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44f8010040 con 0x7f4500075550 2024-04-23T14:43:38.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:38 smithi203 ceph-mon[32800]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:38.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:38 smithi203 ceph-mon[32800]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T14:43:38.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:43:38.830+0000 7f4507200700 1 -- 172.21.15.83:0/1356110038 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4500062f30 con 0x7f44ec0a3c40 2024-04-23T14:43:39.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:39 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:43:39.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:39 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:43:39.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:39 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:39 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:43:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:39 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:43:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:39 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:39.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:43:39.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:43:39.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:40 smithi083 ceph-mon[27867]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:43:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:40 smithi083 ceph-mon[27867]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-23T14:43:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:40 smithi173 ceph-mon[32443]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:43:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:40 smithi173 ceph-mon[32443]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-23T14:43:40.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:40 smithi203 ceph-mon[32800]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:43:40.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:40 smithi203 ceph-mon[32800]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-23T14:43:42.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:42 smithi173 ceph-mon[32443]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T14:43:42.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:42 smithi203 ceph-mon[32800]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T14:43:42.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:42 smithi083 ceph-mon[27867]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T14:43:43.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:43 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3572131480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf897b76-ece1-4df5-8531-ed3aab716d5f"}]: dispatch 2024-04-23T14:43:43.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:43 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3572131480' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf897b76-ece1-4df5-8531-ed3aab716d5f"}]': finished 2024-04-23T14:43:43.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:43 smithi173 ceph-mon[32443]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T14:43:43.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:43:43.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:43 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3572131480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf897b76-ece1-4df5-8531-ed3aab716d5f"}]: dispatch 2024-04-23T14:43:43.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:43 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3572131480' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf897b76-ece1-4df5-8531-ed3aab716d5f"}]': finished 2024-04-23T14:43:43.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:43 smithi203 ceph-mon[32800]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T14:43:43.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:43:43.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:43 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3572131480' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf897b76-ece1-4df5-8531-ed3aab716d5f"}]: dispatch 2024-04-23T14:43:43.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:43 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3572131480' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf897b76-ece1-4df5-8531-ed3aab716d5f"}]': finished 2024-04-23T14:43:43.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:43 smithi083 ceph-mon[27867]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T14:43:43.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:43:44.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:44 smithi173 ceph-mon[32443]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-23T14:43:44.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:44 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3156223539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:43:44.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:44 smithi203 ceph-mon[32800]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-23T14:43:44.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:44 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3156223539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:43:44.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:44 smithi083 ceph-mon[27867]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-23T14:43:44.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:44 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3156223539' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:43:46.664 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:46 smithi203 ceph-mon[32800]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T14:43:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:46 smithi083 ceph-mon[27867]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T14:43:46.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:46 smithi173 ceph-mon[32443]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T14:43:48.627 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:48 smithi173 ceph-mon[32443]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T14:43:48.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:48 smithi203 ceph-mon[32800]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T14:43:48.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:48 smithi083 ceph-mon[27867]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T14:43:50.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:50 smithi203 ceph-mon[32800]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:50.734 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:50 smithi083 ceph-mon[27867]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:50.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:50 smithi173 ceph-mon[32443]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:51.656 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:51 smithi083 ceph-mon[27867]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:51.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:51 smithi203 ceph-mon[32800]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:51.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:51 smithi173 ceph-mon[32443]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:52.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:52 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T14:43:52.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:52 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:52.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:52 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T14:43:52.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:52 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:53.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:52 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T14:43:53.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:52 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:43:53.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:53 smithi083 ceph-mon[27867]: Deploying daemon osd.9 on smithi083 2024-04-23T14:43:53.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:53 smithi083 ceph-mon[27867]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:53 smithi203 ceph-mon[32800]: Deploying daemon osd.9 on smithi083 2024-04-23T14:43:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:53 smithi203 ceph-mon[32800]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:54.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:53 smithi173 ceph-mon[32443]: Deploying daemon osd.9 on smithi083 2024-04-23T14:43:54.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:53 smithi173 ceph-mon[32443]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:56 smithi083 ceph-mon[27867]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:56.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:56 smithi173 ceph-mon[32443]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:56.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:56 smithi203 ceph-mon[32800]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:58.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:43:58 smithi083 ceph-mon[27867]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:58.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:43:58 smithi173 ceph-mon[32443]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:43:58.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:43:58 smithi203 ceph-mon[32800]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:00.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:00 smithi083 ceph-mon[27867]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:00.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:00 smithi173 ceph-mon[32443]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:00.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:00 smithi203 ceph-mon[32800]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:02.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:02 smithi203 ceph-mon[32800]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:02.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:02.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:02.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:02.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:02 smithi083 ceph-mon[27867]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:02.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:02.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:02.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:02 smithi173 ceph-mon[32443]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:04.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:04 smithi203 ceph-mon[32800]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:04.477 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:04 smithi083 ceph-mon[27867]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:04 smithi173 ceph-mon[32443]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:05.411 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:05 smithi083 ceph-mon[27867]: from='osd.9 [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T14:44:05.411 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:05 smithi083 ceph-mon[27867]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T14:44:05.411 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:05.411 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:05 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:05.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:05 smithi203 ceph-mon[32800]: from='osd.9 [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T14:44:05.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:05 smithi203 ceph-mon[32800]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T14:44:05.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:05.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:05 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:05.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:05 smithi173 ceph-mon[32443]: from='osd.9 [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T14:44:05.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:05 smithi173 ceph-mon[32443]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T14:44:05.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:05 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:05.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:05 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:06.491 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 9 on host 'smithi083' 2024-04-23T14:44:06.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.489+0000 7f44f6a00700 1 -- 172.21.15.83:0/1356110038 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4500062f30 con 0x7f44ec0a3c40 2024-04-23T14:44:06.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:06 smithi083 ceph-mon[27867]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:06.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:06 smithi083 ceph-mon[27867]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T14:44:06.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:06 smithi083 ceph-mon[27867]: from='osd.9 [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:44:06.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:06 smithi083 ceph-mon[27867]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T14:44:06.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:06 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:06.492 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:06 smithi083 ceph-mon[27867]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:44:06.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.493+0000 7f4507200700 1 -- 172.21.15.83:0/1356110038 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f44ec0a3c40 msgr2=0x7f44ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.493+0000 7f4507200700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f44ec0a3c40 0x7f44ec0a6100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f44f000b7b0 tx=0x7f44f000b880 comp rx=0 tx=0).stop 2024-04-23T14:44:06.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.493+0000 7f4507200700 1 -- 172.21.15.83:0/1356110038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4500075550 msgr2=0x7f450019ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:06.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.493+0000 7f4507200700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4500075550 0x7f450019ae60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009f40 tx=0x7f44f800c3a0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.494+0000 7f4507200700 1 -- 172.21.15.83:0/1356110038 shutdown_connections 2024-04-23T14:44:06.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.494+0000 7f4507200700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f44ec0a3c40 0x7f44ec0a6100 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.494+0000 7f4507200700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f450010a950 0x7f45001a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.494+0000 7f4507200700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4500075550 0x7f450019ae60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.494+0000 7f4507200700 1 --2- 172.21.15.83:0/1356110038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4500076b20 0x7f450019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:06.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.494+0000 7f4507200700 1 -- 172.21.15.83:0/1356110038 >> 172.21.15.83:0/1356110038 conn(0x7f45000fe020 msgr2=0x7f4500106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:06.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.494+0000 7f4507200700 1 -- 172.21.15.83:0/1356110038 shutdown_connections 2024-04-23T14:44:06.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:44:06.494+0000 7f4507200700 1 -- 172.21.15.83:0/1356110038 wait complete. 2024-04-23T14:44:06.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:06 smithi173 ceph-mon[32443]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:06.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:06 smithi173 ceph-mon[32443]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T14:44:06.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:06 smithi173 ceph-mon[32443]: from='osd.9 [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:44:06.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:06 smithi173 ceph-mon[32443]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T14:44:06.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:06 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:06.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:06 smithi173 ceph-mon[32443]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:44:06.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:06 smithi203 ceph-mon[32800]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:06.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:06 smithi203 ceph-mon[32800]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T14:44:06.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:06 smithi203 ceph-mon[32800]: from='osd.9 [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:44:06.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:06 smithi203 ceph-mon[32800]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T14:44:06.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:06 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:06.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:06 smithi203 ceph-mon[32800]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-04-23T14:44:07.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:07 smithi083 ceph-mon[27867]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-23T14:44:07.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:07 smithi083 ceph-mon[27867]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T14:44:07.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:07 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:07.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:07 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:07.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:07 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:07.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:07 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:07.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:07 smithi083 ceph-mon[27867]: from='osd.9 ' entity='osd.9' 2024-04-23T14:44:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:07 smithi173 ceph-mon[32443]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-23T14:44:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:07 smithi173 ceph-mon[32443]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T14:44:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:07 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:07 smithi173 ceph-mon[32443]: from='osd.9 ' entity='osd.9' 2024-04-23T14:44:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:07 smithi203 ceph-mon[32800]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-04-23T14:44:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:07 smithi203 ceph-mon[32800]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T14:44:07.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:07 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:07.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:07 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:07.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:07 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:07.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:07 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:07.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:07 smithi203 ceph-mon[32800]: from='osd.9 ' entity='osd.9' 2024-04-23T14:44:07.736 DEBUG:teuthology.orchestra.run.smithi083:osd.9> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.9.service 2024-04-23T14:44:07.740 INFO:tasks.cephadm:Deploying osd.10 on smithi173 with /dev/vg_nvme/lv_1... 2024-04-23T14:44:07.740 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T14:44:07.987 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T14:44:08.451 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:08 smithi173 ceph-mon[32443]: purged_snaps scrub starts 2024-04-23T14:44:08.451 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:08 smithi173 ceph-mon[32443]: purged_snaps scrub ok 2024-04-23T14:44:08.451 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:08 smithi173 ceph-mon[32443]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:08.451 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:08.451 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:08 smithi173 ceph-mon[32443]: osd.9 [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956] boot 2024-04-23T14:44:08.451 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:08 smithi173 ceph-mon[32443]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T14:44:08.451 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:08 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:08.486 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:08 smithi083 ceph-mon[27867]: purged_snaps scrub starts 2024-04-23T14:44:08.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:08 smithi083 ceph-mon[27867]: purged_snaps scrub ok 2024-04-23T14:44:08.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:08 smithi083 ceph-mon[27867]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:08.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:08.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:08 smithi083 ceph-mon[27867]: osd.9 [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956] boot 2024-04-23T14:44:08.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:08 smithi083 ceph-mon[27867]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T14:44:08.487 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:08 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:08.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:08 smithi203 ceph-mon[32800]: purged_snaps scrub starts 2024-04-23T14:44:08.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:08 smithi203 ceph-mon[32800]: purged_snaps scrub ok 2024-04-23T14:44:08.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:08 smithi203 ceph-mon[32800]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T14:44:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:08 smithi203 ceph-mon[32800]: osd.9 [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956] boot 2024-04-23T14:44:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:08 smithi203 ceph-mon[32800]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T14:44:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:08 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:44:10.045 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T14:44:10.045 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T14:44:10.045 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2024-04-23T14:44:10.045 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2024-04-23T14:44:10.045 INFO:teuthology.orchestra.run.smithi173.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112867 s, 929 MB/s 2024-04-23T14:44:10.045 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 2024-04-23T14:44:10.045 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2024-04-23T14:44:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:10 smithi083 ceph-mon[27867]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:10 smithi083 ceph-mon[27867]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T14:44:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:10 smithi173 ceph-mon[32443]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:10 smithi173 ceph-mon[32443]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T14:44:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:10 smithi203 ceph-mon[32800]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:10 smithi203 ceph-mon[32800]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T14:44:10.760 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi173:vg_nvme/lv_1 2024-04-23T14:44:10.995 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T14:44:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:12 smithi083 ceph-mon[27867]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:12.530 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:12 smithi173 ceph-mon[32443]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:12 smithi203 ceph-mon[32800]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:12.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.738+0000 7f5bdba10700 1 -- 172.21.15.173:0/1864335048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd4103d50 msgr2=0x7f5bd41041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:12.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.738+0000 7f5bdba10700 1 --2- 172.21.15.173:0/1864335048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd4103d50 0x7f5bd41041b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00099c0 tx=0x7f5bd0009cd0 comp rx=0 tx=0).stop 2024-04-23T14:44:12.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.738+0000 7f5bdba10700 1 -- 172.21.15.173:0/1864335048 shutdown_connections 2024-04-23T14:44:12.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.738+0000 7f5bdba10700 1 --2- 172.21.15.173:0/1864335048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bd41046f0 0x7f5bd410afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:12.742 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.738+0000 7f5bdba10700 1 --2- 172.21.15.173:0/1864335048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bd4102b50 0x7f5bd4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.738+0000 7f5bdba10700 1 --2- 172.21.15.173:0/1864335048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd4103d50 0x7f5bd41041b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.738+0000 7f5bdba10700 1 -- 172.21.15.173:0/1864335048 >> 172.21.15.173:0/1864335048 conn(0x7f5bd40fe0f0 msgr2=0x7f5bd4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.738+0000 7f5bdba10700 1 -- 172.21.15.173:0/1864335048 shutdown_connections 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.738+0000 7f5bdba10700 1 -- 172.21.15.173:0/1864335048 wait complete. 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.739+0000 7f5bdba10700 1 Processor -- start 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.739+0000 7f5bdba10700 1 -- start start 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.739+0000 7f5bdba10700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bd4102b50 0x7f5bd4198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.739+0000 7f5bdba10700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bd4103d50 0x7f5bd4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.739+0000 7f5bdba10700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd41046f0 0x7f5bd41a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.739+0000 7f5bdba10700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd410e7b0 con 0x7f5bd4102b50 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.739+0000 7f5bdba10700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd410e630 con 0x7f5bd41046f0 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.739+0000 7f5bdba10700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd410e930 con 0x7f5bd4103d50 2024-04-23T14:44:12.743 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.739+0000 7f5bd9fad700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd41046f0 0x7f5bd41a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.739+0000 7f5bd9fad700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd41046f0 0x7f5bd41a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:59510/0 (socket says 172.21.15.173:59510) 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.739+0000 7f5bd9fad700 1 -- 172.21.15.173:0/1576867913 learned_addr learned my addr 172.21.15.173:0/1576867913 (peer_addr_for_me v2:172.21.15.173:0/0) 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.739+0000 7f5bd8fab700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bd4103d50 0x7f5bd4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.740+0000 7f5bd9fad700 1 -- 172.21.15.173:0/1576867913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bd4103d50 msgr2=0x7f5bd4199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.740+0000 7f5bd9fad700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bd4103d50 0x7f5bd4199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.740+0000 7f5bd9fad700 1 -- 172.21.15.173:0/1576867913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bd4102b50 msgr2=0x7f5bd4198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.740+0000 7f5bd9fad700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bd4102b50 0x7f5bd4198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.740+0000 7f5bd9fad700 1 -- 172.21.15.173:0/1576867913 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd41a0a50 con 0x7f5bd41046f0 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.740+0000 7f5bd9fad700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd41046f0 0x7f5bd41a03b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00b970 tx=0x7f5bcc00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.740+0000 7f5bca7fc700 1 -- 172.21.15.173:0/1576867913 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc019070 con 0x7f5bd41046f0 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.740+0000 7f5bca7fc700 1 -- 172.21.15.173:0/1576867913 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bcc014440 con 0x7f5bd41046f0 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.740+0000 7f5bca7fc700 1 -- 172.21.15.173:0/1576867913 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0075d0 con 0x7f5bd41046f0 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.740+0000 7f5bdba10700 1 -- 172.21.15.173:0/1576867913 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd41a0d40 con 0x7f5bd41046f0 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.740+0000 7f5bdba10700 1 -- 172.21.15.173:0/1576867913 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd41a12a0 con 0x7f5bd41046f0 2024-04-23T14:44:12.744 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.741+0000 7f5bdba10700 1 -- 172.21.15.173:0/1576867913 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd40697e0 con 0x7f5bd41046f0 2024-04-23T14:44:12.745 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.742+0000 7f5bca7fc700 1 -- 172.21.15.173:0/1576867913 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5bcc01c070 con 0x7f5bd41046f0 2024-04-23T14:44:12.745 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.743+0000 7f5bca7fc700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5bc00a3cc0 0x7f5bc00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:12.745 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.743+0000 7f5bca7fc700 1 -- 172.21.15.173:0/1576867913 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f5bcc0cf4d0 con 0x7f5bd41046f0 2024-04-23T14:44:12.745 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.743+0000 7f5bd97ac700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5bc00a3cc0 0x7f5bc00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:12.746 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.745+0000 7f5bd97ac700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5bc00a3cc0 0x7f5bc00a6180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4103bb0 tx=0x7f5bc4005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:12.748 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.747+0000 7f5bca7fc700 1 -- 172.21.15.173:0/1576867913 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5bcc09c610 con 0x7f5bd41046f0 2024-04-23T14:44:12.962 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:12.960+0000 7f5bdba10700 1 -- 172.21.15.173:0/1576867913 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5bd4062f30 con 0x7f5bc00a3cc0 2024-04-23T14:44:13.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: Detected new or changed devices on smithi083 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:13.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:44:13.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:44:13.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:44:13.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:44:13.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:13.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:13 smithi203 ceph-mon[32800]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: Detected new or changed devices on smithi083 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.744 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:13.745 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:13.745 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:44:13.745 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.745 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:44:13.745 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:44:13.745 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:44:13.745 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:13.745 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:13 smithi173 ceph-mon[32443]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:13.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: Detected new or changed devices on smithi083 2024-04-23T14:44:13.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:13.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:13.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:13.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:13.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi083 to 4272M 2024-04-23T14:44:13.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:13.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:44:13.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:13.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:13.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:44:13.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:13.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='client.24368 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:44:13.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:44:13.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:44:13.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:13.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:13 smithi083 ceph-mon[27867]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:16 smithi173 ceph-mon[32443]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:16 smithi173 ceph-mon[32443]: from='client.? 172.21.15.173:0/3922246337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b95d0e9-c1de-4607-b765-2eb61bb91faf"}]: dispatch 2024-04-23T14:44:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:16 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b95d0e9-c1de-4607-b765-2eb61bb91faf"}]: dispatch 2024-04-23T14:44:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:16 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b95d0e9-c1de-4607-b765-2eb61bb91faf"}]': finished 2024-04-23T14:44:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:16 smithi173 ceph-mon[32443]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T14:44:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:16 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:44:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:16 smithi083 ceph-mon[27867]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:16 smithi083 ceph-mon[27867]: from='client.? 172.21.15.173:0/3922246337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b95d0e9-c1de-4607-b765-2eb61bb91faf"}]: dispatch 2024-04-23T14:44:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:16 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b95d0e9-c1de-4607-b765-2eb61bb91faf"}]: dispatch 2024-04-23T14:44:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:16 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b95d0e9-c1de-4607-b765-2eb61bb91faf"}]': finished 2024-04-23T14:44:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:16 smithi083 ceph-mon[27867]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T14:44:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:16 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:44:16.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:16 smithi203 ceph-mon[32800]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:16.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:16 smithi203 ceph-mon[32800]: from='client.? 172.21.15.173:0/3922246337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b95d0e9-c1de-4607-b765-2eb61bb91faf"}]: dispatch 2024-04-23T14:44:16.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:16 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b95d0e9-c1de-4607-b765-2eb61bb91faf"}]: dispatch 2024-04-23T14:44:16.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:16 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b95d0e9-c1de-4607-b765-2eb61bb91faf"}]': finished 2024-04-23T14:44:16.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:16 smithi203 ceph-mon[32800]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T14:44:16.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:16 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:44:17.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:17 smithi083 ceph-mon[27867]: from='client.? 172.21.15.173:0/3522311371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:44:17.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:17 smithi173 ceph-mon[32443]: from='client.? 172.21.15.173:0/3522311371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:44:17.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:17 smithi203 ceph-mon[32800]: from='client.? 172.21.15.173:0/3522311371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:44:18.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:18 smithi083 ceph-mon[27867]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:18.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:18 smithi173 ceph-mon[32443]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:18.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:18 smithi203 ceph-mon[32800]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:20.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:20 smithi083 ceph-mon[27867]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:20.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:20 smithi173 ceph-mon[32443]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:20 smithi203 ceph-mon[32800]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:22.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:22 smithi083 ceph-mon[27867]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:22.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:22 smithi173 ceph-mon[32443]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:22.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:22 smithi203 ceph-mon[32800]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:24 smithi173 ceph-mon[32443]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T14:44:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:24 smithi173 ceph-mon[32443]: Deploying daemon osd.10 on smithi173 2024-04-23T14:44:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:24 smithi203 ceph-mon[32800]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T14:44:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:24 smithi203 ceph-mon[32800]: Deploying daemon osd.10 on smithi173 2024-04-23T14:44:24.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:24 smithi083 ceph-mon[27867]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:24.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T14:44:24.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:24.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:24 smithi083 ceph-mon[27867]: Deploying daemon osd.10 on smithi173 2024-04-23T14:44:26.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:26 smithi173 ceph-mon[32443]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:26 smithi203 ceph-mon[32800]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:26.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:26 smithi083 ceph-mon[27867]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:28.535 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:28 smithi173 ceph-mon[32443]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:28.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:28 smithi203 ceph-mon[32800]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:28.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:28 smithi083 ceph-mon[27867]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:30.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:30 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:30.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:30 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:30.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:30 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:30.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:30 smithi203 ceph-mon[32800]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:30.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:30.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:30.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:30 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:30.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:30 smithi083 ceph-mon[27867]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:30 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:30 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:30 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:30 smithi173 ceph-mon[32443]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:32.280 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:32 smithi173 ceph-mon[32443]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:32 smithi083 ceph-mon[27867]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:32.696 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 10 on host 'smithi173' 2024-04-23T14:44:32.696 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.695+0000 7f5bca7fc700 1 -- 172.21.15.173:0/1576867913 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5bd4062f30 con 0x7f5bc00a3cc0 2024-04-23T14:44:32.698 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.697+0000 7f5bdba10700 1 -- 172.21.15.173:0/1576867913 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5bc00a3cc0 msgr2=0x7f5bc00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:32.698 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.697+0000 7f5bdba10700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5bc00a3cc0 0x7f5bc00a6180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4103bb0 tx=0x7f5bc4005ca0 comp rx=0 tx=0).stop 2024-04-23T14:44:32.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.697+0000 7f5bdba10700 1 -- 172.21.15.173:0/1576867913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd41046f0 msgr2=0x7f5bd41a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:32.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.697+0000 7f5bdba10700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd41046f0 0x7f5bd41a03b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00b970 tx=0x7f5bcc00bd30 comp rx=0 tx=0).stop 2024-04-23T14:44:32.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.697+0000 7f5bdba10700 1 -- 172.21.15.173:0/1576867913 shutdown_connections 2024-04-23T14:44:32.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.697+0000 7f5bdba10700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5bc00a3cc0 0x7f5bc00a6180 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:32.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.697+0000 7f5bdba10700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd41046f0 0x7f5bd41a03b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:32.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.697+0000 7f5bdba10700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bd4102b50 0x7f5bd4198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:32.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.697+0000 7f5bdba10700 1 --2- 172.21.15.173:0/1576867913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5bd4103d50 0x7f5bd4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:32.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.697+0000 7f5bdba10700 1 -- 172.21.15.173:0/1576867913 >> 172.21.15.173:0/1576867913 conn(0x7f5bd40fe0f0 msgr2=0x7f5bd4107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:32.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.698+0000 7f5bdba10700 1 -- 172.21.15.173:0/1576867913 shutdown_connections 2024-04-23T14:44:32.699 INFO:teuthology.orchestra.run.smithi173.stderr:2024-04-23T14:44:32.698+0000 7f5bdba10700 1 -- 172.21.15.173:0/1576867913 wait complete. 2024-04-23T14:44:32.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:32 smithi203 ceph-mon[32800]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:33.403 DEBUG:teuthology.orchestra.run.smithi173:osd.10> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.10.service 2024-04-23T14:44:33.405 INFO:tasks.cephadm:Deploying osd.11 on smithi203 with /dev/vg_nvme/lv_1... 2024-04-23T14:44:33.405 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T14:44:33.654 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:44:33.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:33 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:33.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:33 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:33.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:33 smithi203 ceph-mon[32800]: from='osd.10 [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T14:44:33.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:33 smithi203 ceph-mon[32800]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T14:44:33.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:33 smithi203 ceph-mon[32800]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:33.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:33 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:33.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:33 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:34.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:33 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:34.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:33 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:34.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:33 smithi083 ceph-mon[27867]: from='osd.10 [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T14:44:34.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:33 smithi083 ceph-mon[27867]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T14:44:34.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:33 smithi083 ceph-mon[27867]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:34.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:33 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:34.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:33 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:34.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:33 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:34.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:33 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:34.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:33 smithi173 ceph-mon[32443]: from='osd.10 [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T14:44:34.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:33 smithi173 ceph-mon[32443]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T14:44:34.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:33 smithi173 ceph-mon[32443]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:34.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:33 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:34.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:33 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:34.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:34 smithi173 ceph-mon[32443]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T14:44:34.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:34 smithi173 ceph-mon[32443]: from='osd.10 [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:44:34.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:34 smithi173 ceph-mon[32443]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T14:44:34.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:44:34.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:34 smithi173 ceph-mon[32443]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:44:34.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:34 smithi203 ceph-mon[32800]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T14:44:34.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:34 smithi203 ceph-mon[32800]: from='osd.10 [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:44:34.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:34 smithi203 ceph-mon[32800]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T14:44:34.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:44:34.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:34 smithi203 ceph-mon[32800]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:44:35.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:34 smithi083 ceph-mon[27867]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T14:44:35.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:34 smithi083 ceph-mon[27867]: from='osd.10 [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:44:35.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:34 smithi083 ceph-mon[27867]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T14:44:35.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:44:35.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:34 smithi083 ceph-mon[27867]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2024-04-23T14:44:35.414 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 14:44:35 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T14:44:35.107+0000 7f2c60f08700 -1 osd.10 0 waiting for initial osdmap 2024-04-23T14:44:35.415 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 14:44:35 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T14:44:35.119+0000 7f2c584f2700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:44:35.877 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T14:44:35.877 INFO:teuthology.orchestra.run.smithi203.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T14:44:35.877 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10+0 records in 2024-04-23T14:44:35.877 INFO:teuthology.orchestra.run.smithi203.stderr:10+0 records out 2024-04-23T14:44:35.877 INFO:teuthology.orchestra.run.smithi203.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0144883 s, 724 MB/s 2024-04-23T14:44:35.878 INFO:teuthology.orchestra.run.smithi203.stderr:--> Zapping successful for: 2024-04-23T14:44:35.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:35 smithi203 ceph-mon[32800]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T14:44:35.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:35 smithi203 ceph-mon[32800]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T14:44:35.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:35 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:44:35.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:35 smithi203 ceph-mon[32800]: from='osd.10 ' entity='osd.10' 2024-04-23T14:44:35.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:35 smithi203 ceph-mon[32800]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:36.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:35 smithi083 ceph-mon[27867]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T14:44:36.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:35 smithi083 ceph-mon[27867]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T14:44:36.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:35 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:44:36.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:35 smithi083 ceph-mon[27867]: from='osd.10 ' entity='osd.10' 2024-04-23T14:44:36.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:35 smithi083 ceph-mon[27867]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:36.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:35 smithi173 ceph-mon[32443]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2024-04-23T14:44:36.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:35 smithi173 ceph-mon[32443]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T14:44:36.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:35 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:44:36.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:35 smithi173 ceph-mon[32443]: from='osd.10 ' entity='osd.10' 2024-04-23T14:44:36.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:35 smithi173 ceph-mon[32443]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T14:44:36.596 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi203:vg_nvme/lv_1 2024-04-23T14:44:36.840 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:44:36.878 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:36 smithi203 ceph-mon[32800]: purged_snaps scrub starts 2024-04-23T14:44:36.878 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:36 smithi203 ceph-mon[32800]: purged_snaps scrub ok 2024-04-23T14:44:36.878 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:36 smithi203 ceph-mon[32800]: osd.10 [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520] boot 2024-04-23T14:44:36.879 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:36 smithi203 ceph-mon[32800]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T14:44:36.879 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:36 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:44:36.879 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:36 smithi203 ceph-mon[32800]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T14:44:36.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:36 smithi173 ceph-mon[32443]: purged_snaps scrub starts 2024-04-23T14:44:36.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:36 smithi173 ceph-mon[32443]: purged_snaps scrub ok 2024-04-23T14:44:36.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:36 smithi173 ceph-mon[32443]: osd.10 [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520] boot 2024-04-23T14:44:36.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:36 smithi173 ceph-mon[32443]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T14:44:36.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:36 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:44:36.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:36 smithi173 ceph-mon[32443]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T14:44:37.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:36 smithi083 ceph-mon[27867]: purged_snaps scrub starts 2024-04-23T14:44:37.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:36 smithi083 ceph-mon[27867]: purged_snaps scrub ok 2024-04-23T14:44:37.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:36 smithi083 ceph-mon[27867]: osd.10 [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520] boot 2024-04-23T14:44:37.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:36 smithi083 ceph-mon[27867]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T14:44:37.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:36 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:44:37.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:36 smithi083 ceph-mon[27867]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T14:44:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: Detected new or changed devices on smithi173 2024-04-23T14:44:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi173 to 4272M 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:38 smithi173 ceph-mon[32443]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T14:44:38.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: Detected new or changed devices on smithi173 2024-04-23T14:44:38.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:38.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:38.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:38.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:38.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi173 to 4272M 2024-04-23T14:44:38.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:38.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:44:38.355 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:38.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:38.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:38.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:44:38.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.356 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:38 smithi203 ceph-mon[32800]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T14:44:38.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: Detected new or changed devices on smithi173 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi173 to 4272M 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:44:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:38.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:38 smithi083 ceph-mon[27867]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T14:44:38.602 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 -- 172.21.15.203:0/2639968421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd80108840 msgr2=0x7fbd8010acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.602 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 --2- 172.21.15.203:0/2639968421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd80108840 0x7fbd8010acb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800b0a0 tx=0x7fbd7800b3b0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.603 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 -- 172.21.15.203:0/2639968421 shutdown_connections 2024-04-23T14:44:38.603 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 --2- 172.21.15.203:0/2639968421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd80108840 0x7fbd8010acb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.603 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 --2- 172.21.15.203:0/2639968421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd801035e0 0x7fbd801059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.603 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 --2- 172.21.15.203:0/2639968421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd80105f10 0x7fbd80108300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.603 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 -- 172.21.15.203:0/2639968421 >> 172.21.15.203:0/2639968421 conn(0x7fbd800fcff0 msgr2=0x7fbd800ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:38.603 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 -- 172.21.15.203:0/2639968421 shutdown_connections 2024-04-23T14:44:38.603 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 -- 172.21.15.203:0/2639968421 wait complete. 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 Processor -- start 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 -- start start 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd801035e0 0x7fbd80102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd80105f10 0x7fbd80101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.602+0000 7fbd87ada700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd80108840 0x7fbd80101ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd87ada700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8010e800 con 0x7fbd801035e0 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd87ada700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8010e680 con 0x7fbd80105f10 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd87ada700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbd8010e980 con 0x7fbd80108840 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd86077700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd80108840 0x7fbd80101ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.605 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd86077700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd80108840 0x7fbd80101ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:53672/0 (socket says 172.21.15.203:53672) 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd86077700 1 -- 172.21.15.203:0/3264235155 learned_addr learned my addr 172.21.15.203:0/3264235155 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd85075700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd80105f10 0x7fbd80101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd86077700 1 -- 172.21.15.203:0/3264235155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd80105f10 msgr2=0x7fbd80101590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd86077700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd80105f10 0x7fbd80101590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd86077700 1 -- 172.21.15.203:0/3264235155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd801035e0 msgr2=0x7fbd80102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd86077700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd801035e0 0x7fbd80102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd86077700 1 -- 172.21.15.203:0/3264235155 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd80102380 con 0x7fbd80108840 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd86077700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd80108840 0x7fbd80101ad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd78007f00 tx=0x7fbd78004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd76ffd700 1 -- 172.21.15.203:0/3264235155 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78021070 con 0x7fbd80108840 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd76ffd700 1 -- 172.21.15.203:0/3264235155 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd7801d930 con 0x7fbd80108840 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd87ada700 1 -- 172.21.15.203:0/3264235155 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd801a2ef0 con 0x7fbd80108840 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd76ffd700 1 -- 172.21.15.203:0/3264235155 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd780055a0 con 0x7fbd80108840 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.603+0000 7fbd87ada700 1 -- 172.21.15.203:0/3264235155 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd801a3310 con 0x7fbd80108840 2024-04-23T14:44:38.606 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.604+0000 7fbd87ada700 1 -- 172.21.15.203:0/3264235155 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd800697e0 con 0x7fbd80108840 2024-04-23T14:44:38.607 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.606+0000 7fbd76ffd700 1 -- 172.21.15.203:0/3264235155 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbd78024070 con 0x7fbd80108840 2024-04-23T14:44:38.607 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.606+0000 7fbd76ffd700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbd6c0a3c40 0x7fbd6c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:44:38.607 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.606+0000 7fbd76ffd700 1 -- 172.21.15.203:0/3264235155 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7fbd7800f020 con 0x7fbd80108840 2024-04-23T14:44:38.607 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.607+0000 7fbd85876700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbd6c0a3c40 0x7fbd6c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:44:38.608 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.608+0000 7fbd85876700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbd6c0a3c40 0x7fbd6c0a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0097b0 tx=0x7fbd7c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:44:38.610 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.610+0000 7fbd76ffd700 1 -- 172.21.15.203:0/3264235155 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd7802b020 con 0x7fbd80108840 2024-04-23T14:44:38.821 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:38.820+0000 7fbd87ada700 1 -- 172.21.15.203:0/3264235155 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbd80062f30 con 0x7fbd6c0a3c40 2024-04-23T14:44:39.715 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:39 smithi203 ceph-mon[32800]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T14:44:39.715 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:39 smithi203 ceph-mon[32800]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:44:39.715 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:39 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:44:39.715 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:39 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:44:39.715 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:39 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:39.715 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:39 smithi203 ceph-mon[32800]: pgmap v267: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 371 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T14:44:40.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:39 smithi083 ceph-mon[27867]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T14:44:40.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:39 smithi083 ceph-mon[27867]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:44:40.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:44:40.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:44:40.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:39 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:40.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:39 smithi083 ceph-mon[27867]: pgmap v267: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 371 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T14:44:40.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:39 smithi173 ceph-mon[32443]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T14:44:40.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:39 smithi173 ceph-mon[32443]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi203:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:44:40.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:39 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T14:44:40.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:39 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T14:44:40.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:39 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:40.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:39 smithi173 ceph-mon[32443]: pgmap v267: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 371 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T14:44:40.768 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:40 smithi203 ceph-mon[32800]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T14:44:41.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:40 smithi083 ceph-mon[27867]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T14:44:41.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:40 smithi173 ceph-mon[32443]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T14:44:41.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:41 smithi203 ceph-mon[32800]: pgmap v269: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-23T14:44:42.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:41 smithi083 ceph-mon[27867]: pgmap v269: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-23T14:44:42.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:41 smithi173 ceph-mon[32443]: pgmap v269: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-23T14:44:42.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:42 smithi203 ceph-mon[32800]: from='client.? 172.21.15.203:0/1301535096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "636b2be4-2656-4ec1-89c4-739c158b6221"}]: dispatch 2024-04-23T14:44:42.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:42 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "636b2be4-2656-4ec1-89c4-739c158b6221"}]: dispatch 2024-04-23T14:44:42.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:42 smithi203 ceph-mon[32800]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "636b2be4-2656-4ec1-89c4-739c158b6221"}]': finished 2024-04-23T14:44:42.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:42 smithi203 ceph-mon[32800]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T14:44:42.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:42 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:44:42.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:42 smithi203 ceph-mon[32800]: from='client.? 172.21.15.203:0/41872271' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:44:43.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:42 smithi083 ceph-mon[27867]: from='client.? 172.21.15.203:0/1301535096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "636b2be4-2656-4ec1-89c4-739c158b6221"}]: dispatch 2024-04-23T14:44:43.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:42 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "636b2be4-2656-4ec1-89c4-739c158b6221"}]: dispatch 2024-04-23T14:44:43.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:42 smithi083 ceph-mon[27867]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "636b2be4-2656-4ec1-89c4-739c158b6221"}]': finished 2024-04-23T14:44:43.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:42 smithi083 ceph-mon[27867]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T14:44:43.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:42 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:44:43.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:42 smithi083 ceph-mon[27867]: from='client.? 172.21.15.203:0/41872271' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:44:43.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:42 smithi173 ceph-mon[32443]: from='client.? 172.21.15.203:0/1301535096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "636b2be4-2656-4ec1-89c4-739c158b6221"}]: dispatch 2024-04-23T14:44:43.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:42 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "636b2be4-2656-4ec1-89c4-739c158b6221"}]: dispatch 2024-04-23T14:44:43.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:42 smithi173 ceph-mon[32443]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "636b2be4-2656-4ec1-89c4-739c158b6221"}]': finished 2024-04-23T14:44:43.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:42 smithi173 ceph-mon[32443]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T14:44:43.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:42 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:44:43.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:42 smithi173 ceph-mon[32443]: from='client.? 172.21.15.203:0/41872271' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T14:44:43.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:43 smithi203 ceph-mon[32800]: pgmap v271: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-23T14:44:44.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:43 smithi083 ceph-mon[27867]: pgmap v271: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-23T14:44:44.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:43 smithi173 ceph-mon[32443]: pgmap v271: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-04-23T14:44:46.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:46 smithi203 ceph-mon[32800]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:46 smithi083 ceph-mon[27867]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:46.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:46 smithi173 ceph-mon[32443]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:48.357 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:48 smithi203 ceph-mon[32800]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:48.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:48 smithi083 ceph-mon[27867]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:48 smithi173 ceph-mon[32443]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:49.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:49 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T14:44:49.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:49 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:49.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:49 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T14:44:49.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:49 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:49.570 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:49 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T14:44:49.570 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:49 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:44:50.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:50 smithi203 ceph-mon[32800]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:50.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:50 smithi203 ceph-mon[32800]: Deploying daemon osd.11 on smithi203 2024-04-23T14:44:50.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:50 smithi173 ceph-mon[32443]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:50.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:50 smithi173 ceph-mon[32443]: Deploying daemon osd.11 on smithi203 2024-04-23T14:44:50.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:50 smithi083 ceph-mon[27867]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:50.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:50 smithi083 ceph-mon[27867]: Deploying daemon osd.11 on smithi203 2024-04-23T14:44:52.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:52 smithi173 ceph-mon[32443]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:52.598 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:52 smithi203 ceph-mon[32800]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:52.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:52 smithi083 ceph-mon[27867]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:54.425 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:54 smithi203 ceph-mon[32800]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:54 smithi173 ceph-mon[32443]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:54 smithi083 ceph-mon[27867]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:56.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:56 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:56.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:56 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:56.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:56 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:56.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:56 smithi173 ceph-mon[32443]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:56.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:56.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:56.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:56.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:56 smithi203 ceph-mon[32800]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:44:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:56 smithi083 ceph-mon[27867]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:58.404 INFO:teuthology.orchestra.run.smithi203.stdout:Created osd(s) 11 on host 'smithi203' 2024-04-23T14:44:58.404 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.403+0000 7fbd76ffd700 1 -- 172.21.15.203:0/3264235155 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbd80062f30 con 0x7fbd6c0a3c40 2024-04-23T14:44:58.406 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.405+0000 7fbd87ada700 1 -- 172.21.15.203:0/3264235155 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbd6c0a3c40 msgr2=0x7fbd6c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.407 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.405+0000 7fbd87ada700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbd6c0a3c40 0x7fbd6c0a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0097b0 tx=0x7fbd7c006d20 comp rx=0 tx=0).stop 2024-04-23T14:44:58.407 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.405+0000 7fbd87ada700 1 -- 172.21.15.203:0/3264235155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd80108840 msgr2=0x7fbd80101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:44:58.407 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.405+0000 7fbd87ada700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd80108840 0x7fbd80101ad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd78007f00 tx=0x7fbd78004e40 comp rx=0 tx=0).stop 2024-04-23T14:44:58.407 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.405+0000 7fbd87ada700 1 -- 172.21.15.203:0/3264235155 shutdown_connections 2024-04-23T14:44:58.407 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.405+0000 7fbd87ada700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbd6c0a3c40 0x7fbd6c0a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.407 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.405+0000 7fbd87ada700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbd80108840 0x7fbd80101ad0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.407 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.405+0000 7fbd87ada700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd801035e0 0x7fbd80102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.407 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.405+0000 7fbd87ada700 1 --2- 172.21.15.203:0/3264235155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbd80105f10 0x7fbd80101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:44:58.407 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.405+0000 7fbd87ada700 1 -- 172.21.15.203:0/3264235155 >> 172.21.15.203:0/3264235155 conn(0x7fbd800fcff0 msgr2=0x7fbd80103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:44:58.407 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.405+0000 7fbd87ada700 1 -- 172.21.15.203:0/3264235155 shutdown_connections 2024-04-23T14:44:58.407 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:44:58.405+0000 7fbd87ada700 1 -- 172.21.15.203:0/3264235155 wait complete. 2024-04-23T14:44:58.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:58 smithi083 ceph-mon[27867]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:58.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:58 smithi173 ceph-mon[32443]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:58.597 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:58 smithi203 ceph-mon[32800]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:44:59.366 DEBUG:teuthology.orchestra.run.smithi203:osd.11> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.11.service 2024-04-23T14:44:59.368 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-23T14:44:59.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-23T14:44:59.641 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:59 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:59.641 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:59 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:59.641 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:59 smithi173 ceph-mon[32443]: from='osd.11 [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T14:44:59.641 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:44:59 smithi173 ceph-mon[32443]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T14:44:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:59 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:59 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:59 smithi203 ceph-mon[32800]: from='osd.11 [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T14:44:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:44:59 smithi203 ceph-mon[32800]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T14:44:59.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:59 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:59.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:59 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:44:59.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:59 smithi083 ceph-mon[27867]: from='osd.11 [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T14:44:59.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:44:59 smithi083 ceph-mon[27867]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T14:44:59.817 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:45:00.453 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:00 smithi203 ceph-mon[32800]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:45:00.454 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:00 smithi203 ceph-mon[32800]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T14:45:00.454 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:00 smithi203 ceph-mon[32800]: from='osd.11 [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:45:00.454 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:00 smithi203 ceph-mon[32800]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T14:45:00.454 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:00 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:00.454 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:00 smithi203 ceph-mon[32800]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:45:00.454 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:00 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:00.454 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:00 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:00.718 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:00 smithi083 ceph-mon[27867]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:45:00.719 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:00 smithi083 ceph-mon[27867]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T14:45:00.719 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:00 smithi083 ceph-mon[27867]: from='osd.11 [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:45:00.719 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:00 smithi083 ceph-mon[27867]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T14:45:00.719 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:00.719 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:00 smithi083 ceph-mon[27867]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:45:00.719 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:00.719 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:00 smithi173 ceph-mon[32443]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:45:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:00 smithi173 ceph-mon[32443]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T14:45:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:00 smithi173 ceph-mon[32443]: from='osd.11 [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:45:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:00 smithi173 ceph-mon[32443]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T14:45:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:00 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:00 smithi173 ceph-mon[32443]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]: dispatch 2024-04-23T14:45:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:00 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:00 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:01.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 14:45:00 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T14:45:00.831+0000 7f84c02de700 -1 osd.11 0 waiting for initial osdmap 2024-04-23T14:45:01.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 14:45:00 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T14:45:00.843+0000 7f84b80c9700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T14:45:01.554 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:01 smithi203 ceph-mon[32800]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-04-23T14:45:01.554 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:01 smithi203 ceph-mon[32800]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T14:45:01.554 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:01 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:01.554 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:01 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:01.554 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:01 smithi203 ceph-mon[32800]: from='osd.11 ' entity='osd.11' 2024-04-23T14:45:01.554 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:01 smithi203 ceph-mon[32800]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:45:01.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:01 smithi083 ceph-mon[27867]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-04-23T14:45:01.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:01 smithi083 ceph-mon[27867]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T14:45:01.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:01 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:01.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:01 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:01.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:01 smithi083 ceph-mon[27867]: from='osd.11 ' entity='osd.11' 2024-04-23T14:45:01.624 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:01 smithi083 ceph-mon[27867]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:45:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:01 smithi173 ceph-mon[32443]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi203", "root=default"]}]': finished 2024-04-23T14:45:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:01 smithi173 ceph-mon[32443]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T14:45:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:01 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:01 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:01 smithi173 ceph-mon[32443]: from='osd.11 ' entity='osd.11' 2024-04-23T14:45:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:01 smithi173 ceph-mon[32443]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T14:45:02.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.116+0000 7f124dc00700 1 -- 172.21.15.83:0/1261143348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12481046f0 msgr2=0x7f124810afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:02.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.116+0000 7f124dc00700 1 --2- 172.21.15.83:0/1261143348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12481046f0 0x7f124810afa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f124000b0a0 tx=0x7f124000b3b0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.116+0000 7f124dc00700 1 -- 172.21.15.83:0/1261143348 shutdown_connections 2024-04-23T14:45:02.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.116+0000 7f124dc00700 1 --2- 172.21.15.83:0/1261143348 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12481046f0 0x7f124810afa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.116+0000 7f124dc00700 1 --2- 172.21.15.83:0/1261143348 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1248102b50 0x7f1248102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.116+0000 7f124dc00700 1 --2- 172.21.15.83:0/1261143348 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1248103d50 0x7f12481041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.116+0000 7f124dc00700 1 -- 172.21.15.83:0/1261143348 >> 172.21.15.83:0/1261143348 conn(0x7f12480fe0f0 msgr2=0x7f1248100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:02.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.116+0000 7f124dc00700 1 -- 172.21.15.83:0/1261143348 shutdown_connections 2024-04-23T14:45:02.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.116+0000 7f124dc00700 1 -- 172.21.15.83:0/1261143348 wait complete. 2024-04-23T14:45:02.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.117+0000 7f124dc00700 1 Processor -- start 2024-04-23T14:45:02.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.117+0000 7f124dc00700 1 -- start start 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.117+0000 7f124dc00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1248102b50 0x7f124819af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.117+0000 7f124dc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1248103d50 0x7f124819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f124dc00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12481046f0 0x7f12481a2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f124dc00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f124810e790 con 0x7f1248103d50 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f124dc00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f124810e610 con 0x7f1248102b50 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f124dc00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f124810e910 con 0x7f12481046f0 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f1246000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1248103d50 0x7f124819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f1246000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1248103d50 0x7f124819b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42186/0 (socket says 172.21.15.83:42186) 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f1246000700 1 -- 172.21.15.83:0/2242096609 learned_addr learned my addr 172.21.15.83:0/2242096609 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f1246a00700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1248102b50 0x7f124819af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f1247400700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12481046f0 0x7f12481a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f1247400700 1 -- 172.21.15.83:0/2242096609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1248102b50 msgr2=0x7f124819af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f1247400700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1248102b50 0x7f124819af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f1247400700 1 -- 172.21.15.83:0/2242096609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1248103d50 msgr2=0x7f124819b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:02.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f1247400700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1248103d50 0x7f124819b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f1247400700 1 -- 172.21.15.83:0/2242096609 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12481a2bc0 con 0x7f12481046f0 2024-04-23T14:45:02.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.118+0000 7f1246000700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1248103d50 0x7f124819b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:02.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.119+0000 7f1247400700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12481046f0 0x7f12481a2520 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1240007fd0 tx=0x7f1240004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:02.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.119+0000 7f123f400700 1 -- 172.21.15.83:0/2242096609 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240021070 con 0x7f12481046f0 2024-04-23T14:45:02.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.119+0000 7f123f400700 1 -- 172.21.15.83:0/2242096609 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1240007b80 con 0x7f12481046f0 2024-04-23T14:45:02.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.119+0000 7f123f400700 1 -- 172.21.15.83:0/2242096609 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12400052a0 con 0x7f12481046f0 2024-04-23T14:45:02.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.119+0000 7f124dc00700 1 -- 172.21.15.83:0/2242096609 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12481a2e50 con 0x7f12481046f0 2024-04-23T14:45:02.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.119+0000 7f124dc00700 1 -- 172.21.15.83:0/2242096609 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12481a32b0 con 0x7f12481046f0 2024-04-23T14:45:02.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.120+0000 7f124dc00700 1 -- 172.21.15.83:0/2242096609 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12480697e0 con 0x7f12481046f0 2024-04-23T14:45:02.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.122+0000 7f123f400700 1 -- 172.21.15.83:0/2242096609 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1240024070 con 0x7f12481046f0 2024-04-23T14:45:02.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.123+0000 7f123f400700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f122c0a3cc0 0x7f122c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:02.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.123+0000 7f123f400700 1 -- 172.21.15.83:0/2242096609 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f124000f020 con 0x7f12481046f0 2024-04-23T14:45:02.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.124+0000 7f1246a00700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f122c0a3cc0 0x7f122c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:02.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.124+0000 7f1246a00700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f122c0a3cc0 0x7f122c0a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1248103bb0 tx=0x7f1238009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:02.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.127+0000 7f123f400700 1 -- 172.21.15.83:0/2242096609 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f124002b020 con 0x7f12481046f0 2024-04-23T14:45:02.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.355+0000 7f124dc00700 1 -- 172.21.15.83:0/2242096609 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f12481a3c20 con 0x7f12481046f0 2024-04-23T14:45:02.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.355+0000 7f123f400700 1 -- 172.21.15.83:0/2242096609 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f1240007d30 con 0x7f12481046f0 2024-04-23T14:45:02.357 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:02.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.358+0000 7f124dc00700 1 -- 172.21.15.83:0/2242096609 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f122c0a3cc0 msgr2=0x7f122c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:02.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.358+0000 7f124dc00700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f122c0a3cc0 0x7f122c0a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1248103bb0 tx=0x7f1238009260 comp rx=0 tx=0).stop 2024-04-23T14:45:02.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.358+0000 7f124dc00700 1 -- 172.21.15.83:0/2242096609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12481046f0 msgr2=0x7f12481a2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:02.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.358+0000 7f124dc00700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12481046f0 0x7f12481a2520 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1240007fd0 tx=0x7f1240004f40 comp rx=0 tx=0).stop 2024-04-23T14:45:02.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.358+0000 7f124dc00700 1 -- 172.21.15.83:0/2242096609 shutdown_connections 2024-04-23T14:45:02.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.358+0000 7f124dc00700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f122c0a3cc0 0x7f122c0a6180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.358+0000 7f124dc00700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12481046f0 0x7f12481a2520 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.358+0000 7f124dc00700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1248102b50 0x7f124819af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.358+0000 7f124dc00700 1 --2- 172.21.15.83:0/2242096609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1248103d50 0x7f124819b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:02.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.358+0000 7f124dc00700 1 -- 172.21.15.83:0/2242096609 >> 172.21.15.83:0/2242096609 conn(0x7f12480fe0f0 msgr2=0x7f1248074d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:02.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.359+0000 7f124dc00700 1 -- 172.21.15.83:0/2242096609 shutdown_connections 2024-04-23T14:45:02.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:02.359+0000 7f124dc00700 1 -- 172.21.15.83:0/2242096609 wait complete. 2024-04-23T14:45:02.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:02 smithi203 ceph-mon[32800]: purged_snaps scrub starts 2024-04-23T14:45:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:02 smithi203 ceph-mon[32800]: purged_snaps scrub ok 2024-04-23T14:45:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:02 smithi203 ceph-mon[32800]: osd.11 [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923] boot 2024-04-23T14:45:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:02 smithi203 ceph-mon[32800]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T14:45:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:02 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2242096609' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T14:45:02.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:02 smithi083 ceph-mon[27867]: purged_snaps scrub starts 2024-04-23T14:45:02.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:02 smithi083 ceph-mon[27867]: purged_snaps scrub ok 2024-04-23T14:45:02.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:02.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:02 smithi083 ceph-mon[27867]: osd.11 [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923] boot 2024-04-23T14:45:02.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:02 smithi083 ceph-mon[27867]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T14:45:02.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:02.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:02 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2242096609' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T14:45:02.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:02 smithi173 ceph-mon[32443]: purged_snaps scrub starts 2024-04-23T14:45:02.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:02 smithi173 ceph-mon[32443]: purged_snaps scrub ok 2024-04-23T14:45:02.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:02.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:02 smithi173 ceph-mon[32443]: osd.11 [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923] boot 2024-04-23T14:45:02.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:02 smithi173 ceph-mon[32443]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T14:45:02.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:45:02.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:02 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2242096609' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T14:45:03.208 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713883501,"num_in_osds":12,"osd_in_since":1713883481,"num_remapped_pgs":0} 2024-04-23T14:45:03.209 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T14:45:03.632 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:45:04.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T14:45:04.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:04.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:04.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:04.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:45:04.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:45:04.126 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:45:04.127 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:45:04.127 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:04.127 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:04.127 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:45:04.127 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:04.127 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:03 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:45:04.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T14:45:04.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:04.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:04.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:04.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:45:04.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:45:04.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:45:04.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:45:04.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:04.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:04.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:45:04.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:04.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:03 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:04.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:04.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:45:04.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:04.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:03 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:45:05.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:04 smithi083 ceph-mon[27867]: Detected new or changed devices on smithi203 2024-04-23T14:45:05.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:04 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi203 to 4272M 2024-04-23T14:45:05.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:04 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:05.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:04 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:45:05.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:04 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:05.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:04 smithi203 ceph-mon[32800]: Detected new or changed devices on smithi203 2024-04-23T14:45:05.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:04 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi203 to 4272M 2024-04-23T14:45:05.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:04 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:05.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:04 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:45:05.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:04 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:05.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:04 smithi173 ceph-mon[32443]: Detected new or changed devices on smithi203 2024-04-23T14:45:05.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:04 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi203 to 4272M 2024-04-23T14:45:05.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:04 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:05.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:04 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:45:05.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:04 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:05.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.896+0000 7f222a400700 1 -- 172.21.15.83:0/3688781434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222410f5a0 msgr2=0x7f2224111990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:05.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.896+0000 7f222a400700 1 --2- 172.21.15.83:0/3688781434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222410f5a0 0x7f2224111990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f221c00b0a0 tx=0x7f221c00b3b0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.896+0000 7f222a400700 1 -- 172.21.15.83:0/3688781434 shutdown_connections 2024-04-23T14:45:05.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.896+0000 7f222a400700 1 --2- 172.21.15.83:0/3688781434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222410f5a0 0x7f2224111990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.896+0000 7f222a400700 1 --2- 172.21.15.83:0/3688781434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f222406a800 0x7f222406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.896+0000 7f222a400700 1 --2- 172.21.15.83:0/3688781434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f222406b120 0x7f222410f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.896+0000 7f222a400700 1 -- 172.21.15.83:0/3688781434 >> 172.21.15.83:0/3688781434 conn(0x7f22240789a0 msgr2=0x7f2224078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:05.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.896+0000 7f222a400700 1 -- 172.21.15.83:0/3688781434 shutdown_connections 2024-04-23T14:45:05.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.896+0000 7f222a400700 1 -- 172.21.15.83:0/3688781434 wait complete. 2024-04-23T14:45:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.897+0000 7f222a400700 1 Processor -- start 2024-04-23T14:45:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.898+0000 7f222a400700 1 -- start start 2024-04-23T14:45:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.898+0000 7f222a400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222406a800 0x7f222419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.898+0000 7f222a400700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f222406b120 0x7f222419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:05.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.898+0000 7f222a400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f222410f5a0 0x7f222419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.898+0000 7f222a400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2224115010 con 0x7f222406a800 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.898+0000 7f222a400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2224114e90 con 0x7f222410f5a0 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.898+0000 7f222a400700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2224115190 con 0x7f222406b120 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.898+0000 7f2223400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222406a800 0x7f222419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.898+0000 7f2223400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222406a800 0x7f222419b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43624/0 (socket says 172.21.15.83:43624) 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.898+0000 7f2223400700 1 -- 172.21.15.83:0/1311139409 learned_addr learned my addr 172.21.15.83:0/1311139409 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.898+0000 7f2223e00700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f222410f5a0 0x7f222419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.898+0000 7f221be00700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f222406b120 0x7f222419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.899+0000 7f2223400700 1 -- 172.21.15.83:0/1311139409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f222406b120 msgr2=0x7f222419b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.899+0000 7f2223400700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f222406b120 0x7f222419b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.899+0000 7f2223400700 1 -- 172.21.15.83:0/1311139409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f222410f5a0 msgr2=0x7f222419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.899+0000 7f2223400700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f222410f5a0 0x7f222419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.899+0000 7f2223400700 1 -- 172.21.15.83:0/1311139409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22241a0420 con 0x7f222406a800 2024-04-23T14:45:05.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.899+0000 7f2223e00700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f222410f5a0 0x7f222419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:05.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.899+0000 7f2223400700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222406a800 0x7f222419b460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f221400eb30 tx=0x7f221400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:05.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.899+0000 7f2220c00700 1 -- 172.21.15.83:0/1311139409 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221400cbd0 con 0x7f222406a800 2024-04-23T14:45:05.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.899+0000 7f2220c00700 1 -- 172.21.15.83:0/1311139409 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2214004500 con 0x7f222406a800 2024-04-23T14:45:05.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.899+0000 7f2220c00700 1 -- 172.21.15.83:0/1311139409 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214010430 con 0x7f222406a800 2024-04-23T14:45:05.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.899+0000 7f222a400700 1 -- 172.21.15.83:0/1311139409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22241a0710 con 0x7f222406a800 2024-04-23T14:45:05.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.900+0000 7f222a400700 1 -- 172.21.15.83:0/1311139409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22241a7f90 con 0x7f222406a800 2024-04-23T14:45:05.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.901+0000 7f222a400700 1 -- 172.21.15.83:0/1311139409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22240697e0 con 0x7f222406a800 2024-04-23T14:45:05.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.903+0000 7f2220c00700 1 -- 172.21.15.83:0/1311139409 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2214003f00 con 0x7f222406a800 2024-04-23T14:45:05.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.904+0000 7f2220c00700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f22080a3c40 0x7f22080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.904+0000 7f2220c00700 1 -- 172.21.15.83:0/1311139409 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f22140d0330 con 0x7f222406a800 2024-04-23T14:45:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.904+0000 7f221be00700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f22080a3c40 0x7f22080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:05.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.905+0000 7f221be00700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f22080a3c40 0x7f22080a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2224102e00 tx=0x7f220c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:05.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:05.909+0000 7f2220c00700 1 -- 172.21.15.83:0/1311139409 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f221409d110 con 0x7f222406a800 2024-04-23T14:45:06.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.136+0000 7f222a400700 1 -- 172.21.15.83:0/1311139409 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f222402dfe0 con 0x7f222406a800 2024-04-23T14:45:06.138 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:05 smithi083 ceph-mon[27867]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:06.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.138+0000 7f2220c00700 1 -- 172.21.15.83:0/1311139409 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18456 (secure 0 0 0) 0x7f221409ca10 con 0x7f222406a800 2024-04-23T14:45:06.141 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:06.141 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:45:02.842396+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:45:06.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.141+0000 7f222a400700 1 -- 172.21.15.83:0/1311139409 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f22080a3c40 msgr2=0x7f22080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.141+0000 7f222a400700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f22080a3c40 0x7f22080a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2224102e00 tx=0x7f220c009260 comp rx=0 tx=0).stop 2024-04-23T14:45:06.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.141+0000 7f222a400700 1 -- 172.21.15.83:0/1311139409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222406a800 msgr2=0x7f222419b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:06.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.141+0000 7f222a400700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222406a800 0x7f222419b460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f221400eb30 tx=0x7f221400ee40 comp rx=0 tx=0).stop 2024-04-23T14:45:06.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.142+0000 7f222a400700 1 -- 172.21.15.83:0/1311139409 shutdown_connections 2024-04-23T14:45:06.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.142+0000 7f222a400700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f22080a3c40 0x7f22080a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.142+0000 7f222a400700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f222410f5a0 0x7f222419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.142+0000 7f222a400700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f222406a800 0x7f222419b460 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.142+0000 7f222a400700 1 --2- 172.21.15.83:0/1311139409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f222406b120 0x7f222419b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:06.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.142+0000 7f222a400700 1 -- 172.21.15.83:0/1311139409 >> 172.21.15.83:0/1311139409 conn(0x7f22240789a0 msgr2=0x7f2224110990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:06.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.142+0000 7f222a400700 1 -- 172.21.15.83:0/1311139409 shutdown_connections 2024-04-23T14:45:06.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:06.142+0000 7f222a400700 1 -- 172.21.15.83:0/1311139409 wait complete. 2024-04-23T14:45:06.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:05 smithi203 ceph-mon[32800]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:06.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:05 smithi173 ceph-mon[32443]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:06.985 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:40:41.241992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:45:06.985 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-23T14:45:07.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:06 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1311139409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:45:07.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:06 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1311139409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:45:07.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:06 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1311139409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:45:07.419 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:45:08.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:07 smithi203 ceph-mon[32800]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:08.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:07 smithi083 ceph-mon[27867]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:08.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:07 smithi173 ceph-mon[32443]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.711+0000 7f7f92800700 1 -- 172.21.15.83:0/2919535848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8c075640 msgr2=0x7f7f8c10efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.711+0000 7f7f92800700 1 --2- 172.21.15.83:0/2919535848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8c075640 0x7f7f8c10efb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009990 tx=0x7f7f74009ca0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.711+0000 7f7f92800700 1 -- 172.21.15.83:0/2919535848 shutdown_connections 2024-04-23T14:45:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.711+0000 7f7f92800700 1 --2- 172.21.15.83:0/2919535848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c10f4f0 0x7f7f8c1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.711+0000 7f7f92800700 1 --2- 172.21.15.83:0/2919535848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f8c074d20 0x7f7f8c075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.711+0000 7f7f92800700 1 --2- 172.21.15.83:0/2919535848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8c075640 0x7f7f8c10efb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.711+0000 7f7f92800700 1 -- 172.21.15.83:0/2919535848 >> 172.21.15.83:0/2919535848 conn(0x7f7f8c079c50 msgr2=0x7f7f8c07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.711+0000 7f7f92800700 1 -- 172.21.15.83:0/2919535848 shutdown_connections 2024-04-23T14:45:09.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.711+0000 7f7f92800700 1 -- 172.21.15.83:0/2919535848 wait complete. 2024-04-23T14:45:09.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.712+0000 7f7f92800700 1 Processor -- start 2024-04-23T14:45:09.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.712+0000 7f7f92800700 1 -- start start 2024-04-23T14:45:09.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.713+0000 7f7f92800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8c074d20 0x7f7f8c19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.713+0000 7f7f92800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f8c075640 0x7f7f8c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.713+0000 7f7f92800700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c10f4f0 0x7f7f8c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.713+0000 7f7f92800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c115030 con 0x7f7f8c074d20 2024-04-23T14:45:09.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.713+0000 7f7f92800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c114eb0 con 0x7f7f8c075640 2024-04-23T14:45:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.713+0000 7f7f92800700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c1151b0 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.713+0000 7f7f8b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8c074d20 0x7f7f8c19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.713+0000 7f7f8b400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8c074d20 0x7f7f8c19b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43640/0 (socket says 172.21.15.83:43640) 2024-04-23T14:45:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.713+0000 7f7f8b400700 1 -- 172.21.15.83:0/1094175233 learned_addr learned my addr 172.21.15.83:0/1094175233 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.713+0000 7f7f8be00700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c10f4f0 0x7f7f8c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.713+0000 7f7f83e00700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f8c075640 0x7f7f8c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.714+0000 7f7f8be00700 1 -- 172.21.15.83:0/1094175233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f8c075640 msgr2=0x7f7f8c19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.714+0000 7f7f8be00700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f8c075640 0x7f7f8c19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.714+0000 7f7f8be00700 1 -- 172.21.15.83:0/1094175233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8c074d20 msgr2=0x7f7f8c19b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.714+0000 7f7f8be00700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8c074d20 0x7f7f8c19b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.714+0000 7f7f8be00700 1 -- 172.21.15.83:0/1094175233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8c1a04e0 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.714+0000 7f7f8be00700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c10f4f0 0x7f7f8c19fda0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400efc0 tx=0x7f7f8400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.714+0000 7f7f88c00700 1 -- 172.21.15.83:0/1094175233 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f840092e0 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.714+0000 7f7f88c00700 1 -- 172.21.15.83:0/1094175233 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f84004bb0 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.714+0000 7f7f92800700 1 -- 172.21.15.83:0/1094175233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8c1a0770 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.714+0000 7f7f88c00700 1 -- 172.21.15.83:0/1094175233 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84004280 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.714+0000 7f7f92800700 1 -- 172.21.15.83:0/1094175233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f8c1a8050 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.715+0000 7f7f92800700 1 -- 172.21.15.83:0/1094175233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f8c0697e0 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.717+0000 7f7f88c00700 1 -- 172.21.15.83:0/1094175233 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7f8401b030 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.719+0000 7f7f88c00700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f700a3c40 0x7f7f700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.719+0000 7f7f88c00700 1 -- 172.21.15.83:0/1094175233 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7f840cfa50 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.719+0000 7f7f8b400700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f700a3c40 0x7f7f700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:09.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.720+0000 7f7f8b400700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f700a3c40 0x7f7f700a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009d20 tx=0x7f7f7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:09.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.723+0000 7f7f88c00700 1 -- 172.21.15.83:0/1094175233 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f84022020 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.951+0000 7f7f92800700 1 -- 172.21.15.83:0/1094175233 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7f8c02dfe0 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.952+0000 7f7f88c00700 1 -- 172.21.15.83:0/1094175233 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f7f84010040 con 0x7f7f8c10f4f0 2024-04-23T14:45:09.954 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-23T14:45:09.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.956+0000 7f7f92800700 1 -- 172.21.15.83:0/1094175233 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f700a3c40 msgr2=0x7f7f700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.956+0000 7f7f92800700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f700a3c40 0x7f7f700a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c009d20 tx=0x7f7f7c009260 comp rx=0 tx=0).stop 2024-04-23T14:45:09.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.956+0000 7f7f92800700 1 -- 172.21.15.83:0/1094175233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c10f4f0 msgr2=0x7f7f8c19fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:09.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.956+0000 7f7f92800700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c10f4f0 0x7f7f8c19fda0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400efc0 tx=0x7f7f8400c3a0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.956+0000 7f7f92800700 1 -- 172.21.15.83:0/1094175233 shutdown_connections 2024-04-23T14:45:09.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.956+0000 7f7f92800700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f700a3c40 0x7f7f700a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.956+0000 7f7f92800700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f8c10f4f0 0x7f7f8c19fda0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.956+0000 7f7f92800700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f8c074d20 0x7f7f8c19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.956+0000 7f7f92800700 1 --2- 172.21.15.83:0/1094175233 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f8c075640 0x7f7f8c19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:09.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.956+0000 7f7f92800700 1 -- 172.21.15.83:0/1094175233 >> 172.21.15.83:0/1094175233 conn(0x7f7f8c079c50 msgr2=0x7f7f8c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:09.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.957+0000 7f7f92800700 1 -- 172.21.15.83:0/1094175233 shutdown_connections 2024-04-23T14:45:09.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:09.957+0000 7f7f92800700 1 -- 172.21.15.83:0/1094175233 wait complete. 2024-04-23T14:45:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:10 smithi083 ceph-mon[27867]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:10 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1094175233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:45:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:10 smithi173 ceph-mon[32443]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:10 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1094175233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:45:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:10 smithi203 ceph-mon[32800]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:10 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1094175233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:45:10.806 INFO:tasks.cephadm:Adding mds.a on smithi083 2024-04-23T14:45:10.806 INFO:tasks.cephadm:Adding mds.d on smithi083 2024-04-23T14:45:10.806 INFO:tasks.cephadm:Adding mds.g on smithi083 2024-04-23T14:45:10.806 INFO:tasks.cephadm:Adding mds.j on smithi083 2024-04-23T14:45:10.806 INFO:tasks.cephadm:Adding mds.b on smithi173 2024-04-23T14:45:10.807 INFO:tasks.cephadm:Adding mds.e on smithi173 2024-04-23T14:45:10.807 INFO:tasks.cephadm:Adding mds.h on smithi173 2024-04-23T14:45:10.807 INFO:tasks.cephadm:Adding mds.k on smithi173 2024-04-23T14:45:10.807 INFO:tasks.cephadm:Adding mds.c on smithi203 2024-04-23T14:45:10.807 INFO:tasks.cephadm:Adding mds.f on smithi203 2024-04-23T14:45:10.807 INFO:tasks.cephadm:Adding mds.i on smithi203 2024-04-23T14:45:10.807 INFO:tasks.cephadm:Adding mds.l on smithi203 2024-04-23T14:45:10.807 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l' 2024-04-23T14:45:11.055 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:45:12.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:12 smithi083 ceph-mon[27867]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:12.552 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:12 smithi203 ceph-mon[32800]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:12.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:12 smithi173 ceph-mon[32443]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:12.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.781+0000 7f47ee4ad700 1 -- 172.21.15.203:0/3358841179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e81046f0 msgr2=0x7f47e810afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:12.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.781+0000 7f47ee4ad700 1 --2- 172.21.15.203:0/3358841179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e81046f0 0x7f47e810afa0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f47e000b0a0 tx=0x7f47e000b3b0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.781+0000 7f47ee4ad700 1 -- 172.21.15.203:0/3358841179 shutdown_connections 2024-04-23T14:45:12.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.781+0000 7f47ee4ad700 1 --2- 172.21.15.203:0/3358841179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e81046f0 0x7f47e810afa0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.781+0000 7f47ee4ad700 1 --2- 172.21.15.203:0/3358841179 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e8102b50 0x7f47e8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.781+0000 7f47ee4ad700 1 --2- 172.21.15.203:0/3358841179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47e8103d50 0x7f47e81041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.781+0000 7f47ee4ad700 1 -- 172.21.15.203:0/3358841179 >> 172.21.15.203:0/3358841179 conn(0x7f47e80fe0f0 msgr2=0x7f47e8100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:12.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.781+0000 7f47ee4ad700 1 -- 172.21.15.203:0/3358841179 shutdown_connections 2024-04-23T14:45:12.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.781+0000 7f47ee4ad700 1 -- 172.21.15.203:0/3358841179 wait complete. 2024-04-23T14:45:12.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.782+0000 7f47ee4ad700 1 Processor -- start 2024-04-23T14:45:12.783 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.782+0000 7f47ee4ad700 1 -- start start 2024-04-23T14:45:12.785 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.782+0000 7f47ee4ad700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47e8102b50 0x7f47e8198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:12.785 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.782+0000 7f47ee4ad700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e8103d50 0x7f47e8199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:12.785 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.782+0000 7f47ee4ad700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e81046f0 0x7f47e81a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:12.785 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.782+0000 7f47ee4ad700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47e810e880 con 0x7f47e8103d50 2024-04-23T14:45:12.785 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.782+0000 7f47ee4ad700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f47e810e700 con 0x7f47e8102b50 2024-04-23T14:45:12.785 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.782+0000 7f47ee4ad700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f47e810ea00 con 0x7f47e81046f0 2024-04-23T14:45:12.785 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47eca4a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e81046f0 0x7f47e81a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:12.785 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47eca4a700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e81046f0 0x7f47e81a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:51188/0 (socket says 172.21.15.203:51188) 2024-04-23T14:45:12.785 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47eca4a700 1 -- 172.21.15.203:0/2133362463 learned_addr learned my addr 172.21.15.203:0/2133362463 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-23T14:45:12.785 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47e7fff700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47e8102b50 0x7f47e8198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:12.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47eca4a700 1 -- 172.21.15.203:0/2133362463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47e8102b50 msgr2=0x7f47e8198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:12.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47eca4a700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47e8102b50 0x7f47e8198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47eca4a700 1 -- 172.21.15.203:0/2133362463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e8103d50 msgr2=0x7f47e8199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:12.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47eca4a700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e8103d50 0x7f47e8199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:12.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47eca4a700 1 -- 172.21.15.203:0/2133362463 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e81a0b40 con 0x7f47e81046f0 2024-04-23T14:45:12.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47eca4a700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e81046f0 0x7f47e81a04a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f47e00095e0 tx=0x7f47e0007770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:12.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47e57fa700 1 -- 172.21.15.203:0/2133362463 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0021070 con 0x7f47e81046f0 2024-04-23T14:45:12.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47e57fa700 1 -- 172.21.15.203:0/2133362463 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47e0004020 con 0x7f47e81046f0 2024-04-23T14:45:12.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47ee4ad700 1 -- 172.21.15.203:0/2133362463 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e81a0dd0 con 0x7f47e81046f0 2024-04-23T14:45:12.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47e57fa700 1 -- 172.21.15.203:0/2133362463 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0005280 con 0x7f47e81046f0 2024-04-23T14:45:12.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.783+0000 7f47ee4ad700 1 -- 172.21.15.203:0/2133362463 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e81a12a0 con 0x7f47e81046f0 2024-04-23T14:45:12.786 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.784+0000 7f47ee4ad700 1 -- 172.21.15.203:0/2133362463 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e80697e0 con 0x7f47e81046f0 2024-04-23T14:45:12.787 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.785+0000 7f47e57fa700 1 -- 172.21.15.203:0/2133362463 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f47e00041d0 con 0x7f47e81046f0 2024-04-23T14:45:12.787 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.786+0000 7f47e57fa700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f47d00a3cc0 0x7f47d00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:12.787 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.786+0000 7f47e57fa700 1 -- 172.21.15.203:0/2133362463 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f47e00d0ca0 con 0x7f47e81046f0 2024-04-23T14:45:12.787 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.786+0000 7f47e7fff700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f47d00a3cc0 0x7f47d00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:12.788 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.787+0000 7f47e7fff700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f47d00a3cc0 0x7f47d00a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47d8006fd0 tx=0x7f47d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:12.791 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.790+0000 7f47e57fa700 1 -- 172.21.15.203:0/2133362463 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47e009dbb0 con 0x7f47e81046f0 2024-04-23T14:45:13.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:12.999+0000 7f47ee4ad700 1 -- 172.21.15.203:0/2133362463 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}) -- 0x7f47e8063130 con 0x7f47d00a3cc0 2024-04-23T14:45:13.016 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.016+0000 7f47e57fa700 1 -- 172.21.15.203:0/2133362463 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f47e8063130 con 0x7f47d00a3cc0 2024-04-23T14:45:13.017 INFO:teuthology.orchestra.run.smithi203.stdout:Scheduled mds.all update... 2024-04-23T14:45:13.019 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.018+0000 7f47ee4ad700 1 -- 172.21.15.203:0/2133362463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f47d00a3cc0 msgr2=0x7f47d00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:13.019 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.018+0000 7f47ee4ad700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f47d00a3cc0 0x7f47d00a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f47d8006fd0 tx=0x7f47d8008040 comp rx=0 tx=0).stop 2024-04-23T14:45:13.019 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.018+0000 7f47ee4ad700 1 -- 172.21.15.203:0/2133362463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e81046f0 msgr2=0x7f47e81a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:13.019 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.018+0000 7f47ee4ad700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e81046f0 0x7f47e81a04a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f47e00095e0 tx=0x7f47e0007770 comp rx=0 tx=0).stop 2024-04-23T14:45:13.019 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.018+0000 7f47ee4ad700 1 -- 172.21.15.203:0/2133362463 shutdown_connections 2024-04-23T14:45:13.019 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.018+0000 7f47ee4ad700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f47d00a3cc0 0x7f47d00a6180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:13.019 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.018+0000 7f47ee4ad700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47e81046f0 0x7f47e81a04a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:13.019 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.018+0000 7f47ee4ad700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47e8102b50 0x7f47e8198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:13.019 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.018+0000 7f47ee4ad700 1 --2- 172.21.15.203:0/2133362463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47e8103d50 0x7f47e8199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:13.019 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.018+0000 7f47ee4ad700 1 -- 172.21.15.203:0/2133362463 >> 172.21.15.203:0/2133362463 conn(0x7f47e80fe0f0 msgr2=0x7f47e81070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:13.019 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.018+0000 7f47ee4ad700 1 -- 172.21.15.203:0/2133362463 shutdown_connections 2024-04-23T14:45:13.019 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-23T14:45:13.018+0000 7f47ee4ad700 1 -- 172.21.15.203:0/2133362463 wait complete. 2024-04-23T14:45:13.684 DEBUG:teuthology.orchestra.run.smithi083:mds.a> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.a.service 2024-04-23T14:45:13.689 DEBUG:teuthology.orchestra.run.smithi083:mds.d> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.d.service 2024-04-23T14:45:13.736 DEBUG:teuthology.orchestra.run.smithi083:mds.g> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.g.service 2024-04-23T14:45:13.782 DEBUG:teuthology.orchestra.run.smithi083:mds.j> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.j.service 2024-04-23T14:45:13.828 DEBUG:teuthology.orchestra.run.smithi173:mds.b> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.b.service 2024-04-23T14:45:13.831 DEBUG:teuthology.orchestra.run.smithi173:mds.e> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.e.service 2024-04-23T14:45:13.873 DEBUG:teuthology.orchestra.run.smithi173:mds.h> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.h.service 2024-04-23T14:45:13.916 DEBUG:teuthology.orchestra.run.smithi173:mds.k> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.k.service 2024-04-23T14:45:13.959 DEBUG:teuthology.orchestra.run.smithi203:mds.c> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.c.service 2024-04-23T14:45:13.961 DEBUG:teuthology.orchestra.run.smithi203:mds.f> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.f.service 2024-04-23T14:45:14.004 DEBUG:teuthology.orchestra.run.smithi203:mds.i> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.i.service 2024-04-23T14:45:14.047 DEBUG:teuthology.orchestra.run.smithi203:mds.l> sudo journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.l.service 2024-04-23T14:45:14.090 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-23T14:45:14.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:45:14.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.292+0000 7fa8cda00640 1 -- 172.21.15.83:0/2545940137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c8105450 msgr2=0x7fa8c81058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:14.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.292+0000 7fa8cda00640 1 --2- 172.21.15.83:0/2545940137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c8105450 0x7fa8c81058d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009900 tx=0x7fa8b002a410 comp rx=0 tx=0).stop 2024-04-23T14:45:14.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.292+0000 7fa8cda00640 1 -- 172.21.15.83:0/2545940137 shutdown_connections 2024-04-23T14:45:14.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.292+0000 7fa8cda00640 1 --2- 172.21.15.83:0/2545940137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8c8105e10 0x7fa8c810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.292+0000 7fa8cda00640 1 --2- 172.21.15.83:0/2545940137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c8105450 0x7fa8c81058d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.292+0000 7fa8cda00640 1 --2- 172.21.15.83:0/2545940137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8c8104260 0x7fa8c8104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.292+0000 7fa8cda00640 1 -- 172.21.15.83:0/2545940137 >> 172.21.15.83:0/2545940137 conn(0x7fa8c80ff7f0 msgr2=0x7fa8c8101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:14.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.292+0000 7fa8cda00640 1 -- 172.21.15.83:0/2545940137 shutdown_connections 2024-04-23T14:45:14.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.292+0000 7fa8cda00640 1 -- 172.21.15.83:0/2545940137 wait complete. 2024-04-23T14:45:14.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.293+0000 7fa8cda00640 1 Processor -- start 2024-04-23T14:45:14.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.293+0000 7fa8cda00640 1 -- start start 2024-04-23T14:45:14.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.293+0000 7fa8cda00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8c8104260 0x7fa8c819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:14.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.293+0000 7fa8cda00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8c8105450 0x7fa8c819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:14.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.293+0000 7fa8cda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c8105e10 0x7fa8c81a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.293+0000 7fa8cda00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c810fef0 con 0x7fa8c8105e10 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.293+0000 7fa8cda00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c810fd70 con 0x7fa8c8104260 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.293+0000 7fa8cda00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c8110070 con 0x7fa8c8105450 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8be000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8c8105450 0x7fa8c819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8c6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8c8104260 0x7fa8c819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8be000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8c8105450 0x7fa8c819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49164/0 (socket says 172.21.15.83:49164) 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8be000640 1 -- 172.21.15.83:0/371669726 learned_addr learned my addr 172.21.15.83:0/371669726 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8c7400640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c8105e10 0x7fa8c81a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8be000640 1 -- 172.21.15.83:0/371669726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8c8104260 msgr2=0x7fa8c819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8be000640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8c8104260 0x7fa8c819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8be000640 1 -- 172.21.15.83:0/371669726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c8105e10 msgr2=0x7fa8c81a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8be000640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c8105e10 0x7fa8c81a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8be000640 1 -- 172.21.15.83:0/371669726 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c81a2140 con 0x7fa8c8105450 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8c6a00640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8c8104260 0x7fa8c819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:14.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8c7400640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c8105e10 0x7fa8c81a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:14.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.294+0000 7fa8be000640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8c8105450 0x7fa8c819aa60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009230 tx=0x7fa8b0031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:14.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.295+0000 7fa8bfe00640 1 -- 172.21.15.83:0/371669726 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b003f030 con 0x7fa8c8105450 2024-04-23T14:45:14.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.295+0000 7fa8bfe00640 1 -- 172.21.15.83:0/371669726 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b0003ad0 con 0x7fa8c8105450 2024-04-23T14:45:14.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.295+0000 7fa8cda00640 1 -- 172.21.15.83:0/371669726 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c81a23d0 con 0x7fa8c8105450 2024-04-23T14:45:14.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.295+0000 7fa8bfe00640 1 -- 172.21.15.83:0/371669726 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0035070 con 0x7fa8c8105450 2024-04-23T14:45:14.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.295+0000 7fa8cda00640 1 -- 172.21.15.83:0/371669726 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c81a28c0 con 0x7fa8c8105450 2024-04-23T14:45:14.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.296+0000 7fa8cda00640 1 -- 172.21.15.83:0/371669726 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c810b600 con 0x7fa8c8105450 2024-04-23T14:45:14.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.298+0000 7fa8bfe00640 1 -- 172.21.15.83:0/371669726 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa8b0042030 con 0x7fa8c8105450 2024-04-23T14:45:14.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.299+0000 7fa8bfe00640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa8ac0a3bb0 0x7fa8ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:14.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.299+0000 7fa8bfe00640 1 -- 172.21.15.83:0/371669726 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa8b00f0680 con 0x7fa8c8105450 2024-04-23T14:45:14.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.299+0000 7fa8c6a00640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa8ac0a3bb0 0x7fa8ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:14.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.300+0000 7fa8c6a00640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa8ac0a3bb0 0x7fa8ac0a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8005fd0 tx=0x7fa8b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:14.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.302+0000 7fa8bfe00640 1 -- 172.21.15.83:0/371669726 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8b00bce10 con 0x7fa8c8105450 2024-04-23T14:45:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:14 smithi173 ceph-mon[32443]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:45:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:14 smithi173 ceph-mon[32443]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-04-23T14:45:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:14 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:14 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:45:14.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:14 smithi173 ceph-mon[32443]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:14.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:14 smithi203 ceph-mon[32800]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:45:14.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:14 smithi203 ceph-mon[32800]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-04-23T14:45:14.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:14 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:14.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:14 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:45:14.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:14 smithi203 ceph-mon[32800]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:14.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.506+0000 7fa8cda00640 1 -- 172.21.15.83:0/371669726 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8c802d870 con 0x7fa8c8105450 2024-04-23T14:45:14.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:14 smithi083 ceph-mon[27867]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:45:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:14 smithi083 ceph-mon[27867]: Saving service mds.all spec with placement smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-04-23T14:45:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:14 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:14 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:45:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:14 smithi083 ceph-mon[27867]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:14.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.507+0000 7fa8bfe00640 1 -- 172.21.15.83:0/371669726 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18456 (secure 0 0 0) 0x7fa8b00c1110 con 0x7fa8c8105450 2024-04-23T14:45:14.508 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:14.508 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:45:02.842396+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:45:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.511+0000 7fa8cda00640 1 -- 172.21.15.83:0/371669726 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa8ac0a3bb0 msgr2=0x7fa8ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.511+0000 7fa8cda00640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa8ac0a3bb0 0x7fa8ac0a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8005fd0 tx=0x7fa8b800b410 comp rx=0 tx=0).stop 2024-04-23T14:45:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.511+0000 7fa8cda00640 1 -- 172.21.15.83:0/371669726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8c8105450 msgr2=0x7fa8c819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.511+0000 7fa8cda00640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8c8105450 0x7fa8c819aa60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0009230 tx=0x7fa8b0031b60 comp rx=0 tx=0).stop 2024-04-23T14:45:14.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.511+0000 7fa8cda00640 1 -- 172.21.15.83:0/371669726 shutdown_connections 2024-04-23T14:45:14.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.511+0000 7fa8cda00640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa8ac0a3bb0 0x7fa8ac0a6070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.511+0000 7fa8cda00640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa8c8105e10 0x7fa8c81a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.511+0000 7fa8cda00640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8c8105450 0x7fa8c819aa60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.511+0000 7fa8cda00640 1 --2- 172.21.15.83:0/371669726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8c8104260 0x7fa8c819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.511+0000 7fa8cda00640 1 -- 172.21.15.83:0/371669726 >> 172.21.15.83:0/371669726 conn(0x7fa8c80ff7f0 msgr2=0x7fa8c81087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:14.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.511+0000 7fa8cda00640 1 -- 172.21.15.83:0/371669726 shutdown_connections 2024-04-23T14:45:14.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.511+0000 7fa8cda00640 1 -- 172.21.15.83:0/371669726 wait complete. 2024-04-23T14:45:14.531 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:40:41.241992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:45:14.532 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:45:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.738+0000 7fc3f2600640 1 -- 172.21.15.83:0/2260393616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3ec104140 msgr2=0x7fc3ec104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.738+0000 7fc3f2600640 1 --2- 172.21.15.83:0/2260393616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3ec104140 0x7fc3ec104560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009900 tx=0x7fc3e002a410 comp rx=0 tx=0).stop 2024-04-23T14:45:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.739+0000 7fc3f2600640 1 -- 172.21.15.83:0/2260393616 shutdown_connections 2024-04-23T14:45:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.739+0000 7fc3f2600640 1 --2- 172.21.15.83:0/2260393616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3ec105cf0 0x7fc3ec10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.739+0000 7fc3f2600640 1 --2- 172.21.15.83:0/2260393616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3ec105330 0x7fc3ec1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.739+0000 7fc3f2600640 1 --2- 172.21.15.83:0/2260393616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3ec104140 0x7fc3ec104560 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.739+0000 7fc3f2600640 1 -- 172.21.15.83:0/2260393616 >> 172.21.15.83:0/2260393616 conn(0x7fc3ec0ff6b0 msgr2=0x7fc3ec101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.739+0000 7fc3f2600640 1 -- 172.21.15.83:0/2260393616 shutdown_connections 2024-04-23T14:45:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.739+0000 7fc3f2600640 1 -- 172.21.15.83:0/2260393616 wait complete. 2024-04-23T14:45:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.740+0000 7fc3f2600640 1 Processor -- start 2024-04-23T14:45:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.740+0000 7fc3f2600640 1 -- start start 2024-04-23T14:45:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.740+0000 7fc3f2600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3ec104140 0x7fc3ec19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.740+0000 7fc3f2600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3ec105330 0x7fc3ec19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.740+0000 7fc3f2600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3ec105cf0 0x7fc3ec1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.740+0000 7fc3f2600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec10fe20 con 0x7fc3ec105330 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.740+0000 7fc3f2600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec10fca0 con 0x7fc3ec105cf0 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.740+0000 7fc3f2600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec10ffa0 con 0x7fc3ec104140 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.740+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3ec105330 0x7fc3ec19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.740+0000 7fc3ebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3ec105330 0x7fc3ec19a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43680/0 (socket says 172.21.15.83:43680) 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.740+0000 7fc3ebe00640 1 -- 172.21.15.83:0/940624367 learned_addr learned my addr 172.21.15.83:0/940624367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.740+0000 7fc3f1200640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3ec104140 0x7fc3ec19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.741+0000 7fc3f1c00640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3ec105cf0 0x7fc3ec1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.741+0000 7fc3ebe00640 1 -- 172.21.15.83:0/940624367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3ec104140 msgr2=0x7fc3ec19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.741+0000 7fc3ebe00640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3ec104140 0x7fc3ec19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.741+0000 7fc3ebe00640 1 -- 172.21.15.83:0/940624367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3ec105cf0 msgr2=0x7fc3ec1a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.741+0000 7fc3ebe00640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3ec105cf0 0x7fc3ec1a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.741+0000 7fc3ebe00640 1 -- 172.21.15.83:0/940624367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec1a2040 con 0x7fc3ec105330 2024-04-23T14:45:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.741+0000 7fc3ebe00640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3ec105330 0x7fc3ec19a960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800d950 tx=0x7fc3d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.741+0000 7fc3e9600640 1 -- 172.21.15.83:0/940624367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8017070 con 0x7fc3ec105330 2024-04-23T14:45:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.741+0000 7fc3f2600640 1 -- 172.21.15.83:0/940624367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec1a2330 con 0x7fc3ec105330 2024-04-23T14:45:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.741+0000 7fc3e9600640 1 -- 172.21.15.83:0/940624367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d8009c30 con 0x7fc3ec105330 2024-04-23T14:45:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.741+0000 7fc3e9600640 1 -- 172.21.15.83:0/940624367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d800f6b0 con 0x7fc3ec105330 2024-04-23T14:45:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.742+0000 7fc3f2600640 1 -- 172.21.15.83:0/940624367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec1a2880 con 0x7fc3ec105330 2024-04-23T14:45:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.743+0000 7fc3f2600640 1 -- 172.21.15.83:0/940624367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ec10b400 con 0x7fc3ec105330 2024-04-23T14:45:14.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.745+0000 7fc3e9600640 1 -- 172.21.15.83:0/940624367 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc3d80109d0 con 0x7fc3ec105330 2024-04-23T14:45:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.747+0000 7fc3e9600640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fc3d40a3bb0 0x7fc3d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.747+0000 7fc3e9600640 1 -- 172.21.15.83:0/940624367 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc3d80cfb80 con 0x7fc3ec105330 2024-04-23T14:45:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.747+0000 7fc3f1200640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fc3d40a3bb0 0x7fc3d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:14.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.748+0000 7fc3f1200640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fc3d40a3bb0 0x7fc3d40a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc3e002aa00 tx=0x7fc3e00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:14.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.751+0000 7fc3e9600640 1 -- 172.21.15.83:0/940624367 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3d8009de0 con 0x7fc3ec105330 2024-04-23T14:45:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.954+0000 7fc3f2600640 1 -- 172.21.15.83:0/940624367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc3ec062c00 con 0x7fc3ec105330 2024-04-23T14:45:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.955+0000 7fc3e9600640 1 -- 172.21.15.83:0/940624367 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fc3d809c310 con 0x7fc3ec105330 2024-04-23T14:45:14.955 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-23T14:45:14.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.957+0000 7fc3f2600640 1 -- 172.21.15.83:0/940624367 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fc3d40a3bb0 msgr2=0x7fc3d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:14.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.957+0000 7fc3f2600640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fc3d40a3bb0 0x7fc3d40a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc3e002aa00 tx=0x7fc3e00315b0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.957+0000 7fc3f2600640 1 -- 172.21.15.83:0/940624367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3ec105330 msgr2=0x7fc3ec19a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.957+0000 7fc3f2600640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3ec105330 0x7fc3ec19a960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800d950 tx=0x7fc3d800de10 comp rx=0 tx=0).stop 2024-04-23T14:45:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.957+0000 7fc3f2600640 1 -- 172.21.15.83:0/940624367 shutdown_connections 2024-04-23T14:45:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.958+0000 7fc3f2600640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fc3d40a3bb0 0x7fc3d40a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.958+0000 7fc3f2600640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3ec105cf0 0x7fc3ec1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.958+0000 7fc3f2600640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3ec105330 0x7fc3ec19a960 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.958+0000 7fc3f2600640 1 --2- 172.21.15.83:0/940624367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3ec104140 0x7fc3ec19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.958+0000 7fc3f2600640 1 -- 172.21.15.83:0/940624367 >> 172.21.15.83:0/940624367 conn(0x7fc3ec0ff6b0 msgr2=0x7fc3ec1085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.958+0000 7fc3f2600640 1 -- 172.21.15.83:0/940624367 shutdown_connections 2024-04-23T14:45:14.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:14.958+0000 7fc3f2600640 1 -- 172.21.15.83:0/940624367 wait complete. 2024-04-23T14:45:14.977 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:15.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:15.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:45:15.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:15.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:15.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:15.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:15 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:15.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:15 smithi203 ceph-mon[32800]: Deploying daemon mds.l on smithi203 2024-04-23T14:45:15.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:15 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/371669726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:45:15.162 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:15 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/940624367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:45:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.175+0000 7f2a73200640 1 -- 172.21.15.83:0/586754460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a6c108480 msgr2=0x7f2a6c1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.175+0000 7f2a73200640 1 --2- 172.21.15.83:0/586754460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a6c108480 0x7f2a6c1088a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009900 tx=0x7f2a6002a400 comp rx=0 tx=0).stop 2024-04-23T14:45:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.175+0000 7f2a73200640 1 -- 172.21.15.83:0/586754460 shutdown_connections 2024-04-23T14:45:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.175+0000 7f2a73200640 1 --2- 172.21.15.83:0/586754460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a6c10a030 0x7f2a6c1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.175+0000 7f2a73200640 1 --2- 172.21.15.83:0/586754460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c109670 0x7f2a6c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.175+0000 7f2a73200640 1 --2- 172.21.15.83:0/586754460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a6c108480 0x7f2a6c1088a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.175+0000 7f2a73200640 1 -- 172.21.15.83:0/586754460 >> 172.21.15.83:0/586754460 conn(0x7f2a6c077440 msgr2=0x7f2a6c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.175+0000 7f2a73200640 1 -- 172.21.15.83:0/586754460 shutdown_connections 2024-04-23T14:45:15.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.175+0000 7f2a73200640 1 -- 172.21.15.83:0/586754460 wait complete. 2024-04-23T14:45:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.176+0000 7f2a73200640 1 Processor -- start 2024-04-23T14:45:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.176+0000 7f2a73200640 1 -- start start 2024-04-23T14:45:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a73200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a6c108480 0x7f2a6c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a73200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a6c109670 0x7f2a6c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a73200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c10a030 0x7f2a6c1a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a73200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c114160 con 0x7f2a6c108480 2024-04-23T14:45:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a73200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c113fe0 con 0x7f2a6c10a030 2024-04-23T14:45:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a73200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c1142e0 con 0x7f2a6c109670 2024-04-23T14:45:15.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a71e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a6c108480 0x7f2a6c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a71e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a6c108480 0x7f2a6c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48742/0 (socket says 172.21.15.83:48742) 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a71e00640 1 -- 172.21.15.83:0/567023307 learned_addr learned my addr 172.21.15.83:0/567023307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a71400640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a6c109670 0x7f2a6c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a72800640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c10a030 0x7f2a6c1a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a71e00640 1 -- 172.21.15.83:0/567023307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a6c109670 msgr2=0x7f2a6c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a71e00640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a6c109670 0x7f2a6c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a71e00640 1 -- 172.21.15.83:0/567023307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c10a030 msgr2=0x7f2a6c1a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a71e00640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c10a030 0x7f2a6c1a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.177+0000 7f2a71e00640 1 -- 172.21.15.83:0/567023307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c1a63a0 con 0x7f2a6c108480 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.178+0000 7f2a71e00640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a6c108480 0x7f2a6c19e780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2a60031440 tx=0x7f2a60032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.178+0000 7f2a66a00640 1 -- 172.21.15.83:0/567023307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6003f030 con 0x7f2a6c108480 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.178+0000 7f2a66a00640 1 -- 172.21.15.83:0/567023307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a60003ce0 con 0x7f2a6c108480 2024-04-23T14:45:15.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.178+0000 7f2a73200640 1 -- 172.21.15.83:0/567023307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c1a6630 con 0x7f2a6c108480 2024-04-23T14:45:15.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.178+0000 7f2a66a00640 1 -- 172.21.15.83:0/567023307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60035070 con 0x7f2a6c108480 2024-04-23T14:45:15.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.178+0000 7f2a73200640 1 -- 172.21.15.83:0/567023307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c1a6b20 con 0x7f2a6c108480 2024-04-23T14:45:15.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.179+0000 7f2a73200640 1 -- 172.21.15.83:0/567023307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a6c10f7a0 con 0x7f2a6c108480 2024-04-23T14:45:15.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.181+0000 7f2a66a00640 1 -- 172.21.15.83:0/567023307 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2a60042030 con 0x7f2a6c108480 2024-04-23T14:45:15.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.183+0000 7f2a66a00640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2a540a3bb0 0x7f2a540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.184+0000 7f2a66a00640 1 -- 172.21.15.83:0/567023307 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2a600f04b0 con 0x7f2a6c108480 2024-04-23T14:45:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.184+0000 7f2a71400640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2a540a3bb0 0x7f2a540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:15.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.184+0000 7f2a71400640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2a540a3bb0 0x7f2a540a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c19fc90 tx=0x7f2a5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:15.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.188+0000 7f2a66a00640 1 -- 172.21.15.83:0/567023307 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a600bcc40 con 0x7f2a6c108480 2024-04-23T14:45:15.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:15.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:45:15.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:15.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:15.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:15.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:15 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:15.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:15 smithi083 ceph-mon[27867]: Deploying daemon mds.l on smithi203 2024-04-23T14:45:15.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:15 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/371669726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:45:15.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:15 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/940624367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:45:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:45:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:15 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:15 smithi173 ceph-mon[32443]: Deploying daemon mds.l on smithi203 2024-04-23T14:45:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:15 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/371669726' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:45:15.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:15 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/940624367' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:45:15.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.424+0000 7f2a73200640 1 -- 172.21.15.83:0/567023307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a6c02d620 con 0x7f2a6c108480 2024-04-23T14:45:15.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.425+0000 7f2a66a00640 1 -- 172.21.15.83:0/567023307 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f2a600c0f40 con 0x7f2a6c108480 2024-04-23T14:45:15.425 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:15.426 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-23T14:45:15.426 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 1 2024-04-23T14:45:15.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.427+0000 7f2a73200640 1 -- 172.21.15.83:0/567023307 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2a540a3bb0 msgr2=0x7f2a540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:15.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.427+0000 7f2a73200640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2a540a3bb0 0x7f2a540a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c19fc90 tx=0x7f2a5c00b410 comp rx=0 tx=0).stop 2024-04-23T14:45:15.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.427+0000 7f2a73200640 1 -- 172.21.15.83:0/567023307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a6c108480 msgr2=0x7f2a6c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:15.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.427+0000 7f2a73200640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a6c108480 0x7f2a6c19e780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2a60031440 tx=0x7f2a60032ce0 comp rx=0 tx=0).stop 2024-04-23T14:45:15.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.428+0000 7f2a73200640 1 -- 172.21.15.83:0/567023307 shutdown_connections 2024-04-23T14:45:15.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.428+0000 7f2a73200640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2a540a3bb0 0x7f2a540a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:15.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.428+0000 7f2a73200640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c10a030 0x7f2a6c1a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:15.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.428+0000 7f2a73200640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a6c109670 0x7f2a6c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:15.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.428+0000 7f2a73200640 1 --2- 172.21.15.83:0/567023307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2a6c108480 0x7f2a6c19e780 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:15.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.428+0000 7f2a73200640 1 -- 172.21.15.83:0/567023307 >> 172.21.15.83:0/567023307 conn(0x7f2a6c077440 msgr2=0x7f2a6c10c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:15.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.428+0000 7f2a73200640 1 -- 172.21.15.83:0/567023307 shutdown_connections 2024-04-23T14:45:15.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:15.428+0000 7f2a73200640 1 -- 172.21.15.83:0/567023307 wait complete. 2024-04-23T14:45:16.101 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:16 smithi203 ceph-mon[32800]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:16.101 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:16 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/567023307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:16 smithi173 ceph-mon[32443]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:16.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:16 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/567023307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:16 smithi083 ceph-mon[27867]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:16 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/567023307' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:16.721 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 14:45:16 smithi203 podman[62606]: 2024-04-23 14:45:16.711388584 +0000 UTC m=+0.138042997 container create 976d5bb48ff328ff67081bfdc8dd722a7d3c11f274731a75e3d3bf0d9b0f88bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-23T14:45:17.089 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 14:45:16 smithi203 podman[62606]: 2024-04-23 14:45:16.98916767 +0000 UTC m=+0.415822083 container init 976d5bb48ff328ff67081bfdc8dd722a7d3c11f274731a75e3d3bf0d9b0f88bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-04-23T14:45:17.089 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 14:45:16 smithi203 podman[62606]: 2024-04-23 14:45:16.991820687 +0000 UTC m=+0.418475112 container start 976d5bb48ff328ff67081bfdc8dd722a7d3c11f274731a75e3d3bf0d9b0f88bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-23T14:45:17.089 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 14:45:17 smithi203 ceph-mds[62626]: -- 172.21.15.203:0/3926792785 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec9b81fa40 con 0x55ec9b7f3000 2024-04-23T14:45:17.089 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 14:45:17 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-l[62622]: starting mds.l at 2024-04-23T14:45:17.446 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:17.471 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 14:45:17 smithi203 bash[62606]: 976d5bb48ff328ff67081bfdc8dd722a7d3c11f274731a75e3d3bf0d9b0f88bf 2024-04-23T14:45:17.471 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 14:45:17 smithi203 systemd[1]: Started Ceph mds.l for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:45:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.657+0000 7f0c61800640 1 -- 172.21.15.83:0/148362466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c100d80 msgr2=0x7f0c5c1011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.657+0000 7f0c61800640 1 --2- 172.21.15.83:0/148362466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c100d80 0x7f0c5c1011a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009900 tx=0x7f0c4c02a410 comp rx=0 tx=0).stop 2024-04-23T14:45:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.657+0000 7f0c61800640 1 -- 172.21.15.83:0/148362466 shutdown_connections 2024-04-23T14:45:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.657+0000 7f0c61800640 1 --2- 172.21.15.83:0/148362466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c102930 0x7f0c5c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.657+0000 7f0c61800640 1 --2- 172.21.15.83:0/148362466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c5c101f70 0x7f0c5c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.657+0000 7f0c61800640 1 --2- 172.21.15.83:0/148362466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c100d80 0x7f0c5c1011a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.657+0000 7f0c61800640 1 -- 172.21.15.83:0/148362466 >> 172.21.15.83:0/148362466 conn(0x7f0c5c0fc2d0 msgr2=0x7f0c5c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.657+0000 7f0c61800640 1 -- 172.21.15.83:0/148362466 shutdown_connections 2024-04-23T14:45:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.657+0000 7f0c61800640 1 -- 172.21.15.83:0/148362466 wait complete. 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.658+0000 7f0c61800640 1 Processor -- start 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c61800640 1 -- start start 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c61800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c100d80 0x7f0c5c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c61800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c101f70 0x7f0c5c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c61800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c5c102930 0x7f0c5c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c61800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c10db40 con 0x7f0c5c101f70 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c61800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c10d9c0 con 0x7f0c5c100d80 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c61800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5c10dcc0 con 0x7f0c5c102930 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c101f70 0x7f0c5c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c5be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c100d80 0x7f0c5c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c101f70 0x7f0c5c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48758/0 (socket says 172.21.15.83:48758) 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c5be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c100d80 0x7f0c5c1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:42050/0 (socket says 172.21.15.83:42050) 2024-04-23T14:45:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c53e00640 1 -- 172.21.15.83:0/3183946662 learned_addr learned my addr 172.21.15.83:0/3183946662 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.659+0000 7f0c60e00640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c5c102930 0x7f0c5c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.660+0000 7f0c5be00640 1 -- 172.21.15.83:0/3183946662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c5c102930 msgr2=0x7f0c5c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.660+0000 7f0c5be00640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c5c102930 0x7f0c5c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.660+0000 7f0c5be00640 1 -- 172.21.15.83:0/3183946662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c101f70 msgr2=0x7f0c5c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.660+0000 7f0c5be00640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c101f70 0x7f0c5c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.660+0000 7f0c5be00640 1 -- 172.21.15.83:0/3183946662 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5c19fde0 con 0x7f0c5c100d80 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.660+0000 7f0c53e00640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c101f70 0x7f0c5c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.660+0000 7f0c60e00640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c5c102930 0x7f0c5c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.660+0000 7f0c5be00640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c100d80 0x7f0c5c1981c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009230 tx=0x7f0c4c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.660+0000 7f0c59600640 1 -- 172.21.15.83:0/3183946662 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c03f030 con 0x7f0c5c100d80 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.660+0000 7f0c59600640 1 -- 172.21.15.83:0/3183946662 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c4c003ce0 con 0x7f0c5c100d80 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.660+0000 7f0c61800640 1 -- 172.21.15.83:0/3183946662 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5c1a0070 con 0x7f0c5c100d80 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.660+0000 7f0c59600640 1 -- 172.21.15.83:0/3183946662 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c035070 con 0x7f0c5c100d80 2024-04-23T14:45:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.661+0000 7f0c61800640 1 -- 172.21.15.83:0/3183946662 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c5c1a0580 con 0x7f0c5c100d80 2024-04-23T14:45:17.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.662+0000 7f0c61800640 1 -- 172.21.15.83:0/3183946662 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c5c1080a0 con 0x7f0c5c100d80 2024-04-23T14:45:17.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.664+0000 7f0c59600640 1 -- 172.21.15.83:0/3183946662 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0c4c042030 con 0x7f0c5c100d80 2024-04-23T14:45:17.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.665+0000 7f0c59600640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c3c0a3c30 0x7f0c3c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:17.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.665+0000 7f0c59600640 1 -- 172.21.15.83:0/3183946662 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0c4c0f0320 con 0x7f0c5c100d80 2024-04-23T14:45:17.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.665+0000 7f0c53e00640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c3c0a3c30 0x7f0c3c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:17.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.666+0000 7f0c53e00640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c3c0a3c30 0x7f0c3c0a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c440095f0 tx=0x7f0c44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:17.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.669+0000 7f0c59600640 1 -- 172.21.15.83:0/3183946662 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c4c0bcab0 con 0x7f0c5c100d80 2024-04-23T14:45:17.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.913+0000 7f0c61800640 1 -- 172.21.15.83:0/3183946662 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c5c02d620 con 0x7f0c5c100d80 2024-04-23T14:45:17.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.914+0000 7f0c59600640 1 -- 172.21.15.83:0/3183946662 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f0c4c0c0db0 con 0x7f0c5c100d80 2024-04-23T14:45:17.915 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:17.915 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T14:45:17.915 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 2 2024-04-23T14:45:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.917+0000 7f0c61800640 1 -- 172.21.15.83:0/3183946662 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c3c0a3c30 msgr2=0x7f0c3c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.917+0000 7f0c61800640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c3c0a3c30 0x7f0c3c0a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c440095f0 tx=0x7f0c44005f90 comp rx=0 tx=0).stop 2024-04-23T14:45:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.917+0000 7f0c61800640 1 -- 172.21.15.83:0/3183946662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c100d80 msgr2=0x7f0c5c1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.917+0000 7f0c61800640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c100d80 0x7f0c5c1981c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009230 tx=0x7f0c4c032ce0 comp rx=0 tx=0).stop 2024-04-23T14:45:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.917+0000 7f0c61800640 1 -- 172.21.15.83:0/3183946662 shutdown_connections 2024-04-23T14:45:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.917+0000 7f0c61800640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c3c0a3c30 0x7f0c3c0a60f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.917+0000 7f0c61800640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c5c102930 0x7f0c5c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.917+0000 7f0c61800640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c5c101f70 0x7f0c5c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.917+0000 7f0c61800640 1 --2- 172.21.15.83:0/3183946662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c5c100d80 0x7f0c5c1981c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.917+0000 7f0c61800640 1 -- 172.21.15.83:0/3183946662 >> 172.21.15.83:0/3183946662 conn(0x7f0c5c0fc2d0 msgr2=0x7f0c5c105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.917+0000 7f0c61800640 1 -- 172.21.15.83:0/3183946662 shutdown_connections 2024-04-23T14:45:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:17.917+0000 7f0c61800640 1 -- 172.21.15.83:0/3183946662 wait complete. 2024-04-23T14:45:18.127 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:18 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.203:6832/317357729,v1:172.21.15.203:6833/317357729] up:boot 2024-04-23T14:45:18.128 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:18 smithi173 ceph-mon[32443]: fsmap 1 up:standby 2024-04-23T14:45:18.128 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T14:45:18.128 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:18.128 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:18.128 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:18 smithi173 ceph-mon[32443]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:18.128 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:18.128 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:18.128 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:18.128 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:18 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:18.128 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:18 smithi173 ceph-mon[32443]: Deploying daemon mds.k on smithi173 2024-04-23T14:45:18.128 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:18 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3183946662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:18.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:18 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.203:6832/317357729,v1:172.21.15.203:6833/317357729] up:boot 2024-04-23T14:45:18.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:18 smithi083 ceph-mon[27867]: fsmap 1 up:standby 2024-04-23T14:45:18.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T14:45:18.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:18.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:18.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:18 smithi083 ceph-mon[27867]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:18.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:18.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:18.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:18.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:18 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:18.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:18 smithi083 ceph-mon[27867]: Deploying daemon mds.k on smithi173 2024-04-23T14:45:18.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:18 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3183946662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:18.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:18 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.203:6832/317357729,v1:172.21.15.203:6833/317357729] up:boot 2024-04-23T14:45:18.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:18 smithi203 ceph-mon[32800]: fsmap 1 up:standby 2024-04-23T14:45:18.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T14:45:18.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:18.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:18.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:18 smithi203 ceph-mon[32800]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:18.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:18.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:18.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:18.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:18 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:18.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:18 smithi203 ceph-mon[32800]: Deploying daemon mds.k on smithi173 2024-04-23T14:45:18.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:18 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3183946662' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:19.576 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 14:45:19 smithi173 systemd[1]: Starting Ceph mds.k for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:45:19.940 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:19.961 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 14:45:19 smithi173 podman[62166]: 2024-04-23 14:45:19.607309158 +0000 UTC m=+0.018168176 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T14:45:19.961 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 14:45:19 smithi173 podman[62166]: 2024-04-23 14:45:19.724921249 +0000 UTC m=+0.135780300 container create 6db9430fe8edb98a89e013f30b798a6b292ed1f3908e2549c06a358188f50437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-23T14:45:19.961 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 14:45:19 smithi173 podman[62166]: 2024-04-23 14:45:19.960851449 +0000 UTC m=+0.371710454 container init 6db9430fe8edb98a89e013f30b798a6b292ed1f3908e2549c06a358188f50437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-23T14:45:20.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.151+0000 7f4d22a00640 1 -- 172.21.15.83:0/1337559885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c10a030 msgr2=0x7f4d1c1108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:20.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.151+0000 7f4d22a00640 1 --2- 172.21.15.83:0/1337559885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c10a030 0x7f4d1c1108c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009be0 tx=0x7f4d1402a6c0 comp rx=0 tx=0).stop 2024-04-23T14:45:20.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.151+0000 7f4d22a00640 1 -- 172.21.15.83:0/1337559885 shutdown_connections 2024-04-23T14:45:20.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.151+0000 7f4d22a00640 1 --2- 172.21.15.83:0/1337559885 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c10a030 0x7f4d1c1108c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:20.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.151+0000 7f4d22a00640 1 --2- 172.21.15.83:0/1337559885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d1c109670 0x7f4d1c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:20.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.151+0000 7f4d22a00640 1 --2- 172.21.15.83:0/1337559885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d1c108480 0x7f4d1c1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:20.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.151+0000 7f4d22a00640 1 -- 172.21.15.83:0/1337559885 >> 172.21.15.83:0/1337559885 conn(0x7f4d1c077440 msgr2=0x7f4d1c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:20.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.151+0000 7f4d22a00640 1 -- 172.21.15.83:0/1337559885 shutdown_connections 2024-04-23T14:45:20.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.151+0000 7f4d22a00640 1 -- 172.21.15.83:0/1337559885 wait complete. 2024-04-23T14:45:20.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.152+0000 7f4d22a00640 1 Processor -- start 2024-04-23T14:45:20.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.152+0000 7f4d22a00640 1 -- start start 2024-04-23T14:45:20.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d22a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c108480 0x7f4d1c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:20.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d22a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d1c109670 0x7f4d1c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:20.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d22a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d1c10a030 0x7f4d1c1a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:20.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d22a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1140c0 con 0x7f4d1c108480 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d22a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c113f40 con 0x7f4d1c10a030 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d22a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c114240 con 0x7f4d1c109670 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d21600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c108480 0x7f4d1c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d21600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c108480 0x7f4d1c19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48776/0 (socket says 172.21.15.83:48776) 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d21600640 1 -- 172.21.15.83:0/1587536490 learned_addr learned my addr 172.21.15.83:0/1587536490 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d22000640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d1c10a030 0x7f4d1c1a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d20c00640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d1c109670 0x7f4d1c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d21600640 1 -- 172.21.15.83:0/1587536490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d1c109670 msgr2=0x7f4d1c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d21600640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d1c109670 0x7f4d1c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d21600640 1 -- 172.21.15.83:0/1587536490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d1c10a030 msgr2=0x7f4d1c1a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d21600640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d1c10a030 0x7f4d1c1a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.153+0000 7f4d21600640 1 -- 172.21.15.83:0/1587536490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c1a6310 con 0x7f4d1c108480 2024-04-23T14:45:20.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.154+0000 7f4d21600640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c108480 0x7f4d1c19e740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000e9f0 tx=0x7f4d1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:20.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.156+0000 7f4d1a000640 1 -- 172.21.15.83:0/1587536490 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000ccc0 con 0x7f4d1c108480 2024-04-23T14:45:20.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.156+0000 7f4d1a000640 1 -- 172.21.15.83:0/1587536490 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d100035f0 con 0x7f4d1c108480 2024-04-23T14:45:20.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.156+0000 7f4d22a00640 1 -- 172.21.15.83:0/1587536490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1c1a6600 con 0x7f4d1c108480 2024-04-23T14:45:20.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.156+0000 7f4d1a000640 1 -- 172.21.15.83:0/1587536490 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000ce60 con 0x7f4d1c108480 2024-04-23T14:45:20.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.156+0000 7f4d22a00640 1 -- 172.21.15.83:0/1587536490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1c1a6ba0 con 0x7f4d1c108480 2024-04-23T14:45:20.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.158+0000 7f4d22a00640 1 -- 172.21.15.83:0/1587536490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d1c10f740 con 0x7f4d1c108480 2024-04-23T14:45:20.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.160+0000 7f4d1a000640 1 -- 172.21.15.83:0/1587536490 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4d100037a0 con 0x7f4d1c108480 2024-04-23T14:45:20.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.161+0000 7f4d1a000640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4d040a3c30 0x7f4d040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:20.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.161+0000 7f4d1a000640 1 -- 172.21.15.83:0/1587536490 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4d100d0110 con 0x7f4d1c108480 2024-04-23T14:45:20.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.161+0000 7f4d20c00640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4d040a3c30 0x7f4d040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:20.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.162+0000 7f4d20c00640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4d040a3c30 0x7f4d040a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c19fc00 tx=0x7f4d0c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:20.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.166+0000 7f4d1a000640 1 -- 172.21.15.83:0/1587536490 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d1009cf90 con 0x7f4d1c108480 2024-04-23T14:45:20.252 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 14:45:19 smithi173 podman[62166]: 2024-04-23 14:45:19.963698369 +0000 UTC m=+0.374557383 container start 6db9430fe8edb98a89e013f30b798a6b292ed1f3908e2549c06a358188f50437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd) 2024-04-23T14:45:20.252 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 14:45:19 smithi173 ceph-mds[62185]: -- 172.21.15.173:0/2717685718 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556ac9b1da40 con 0x556ac9af0c00 2024-04-23T14:45:20.252 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 14:45:19 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-k[62181]: starting mds.k at 2024-04-23T14:45:20.252 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 14:45:20 smithi173 bash[62166]: 6db9430fe8edb98a89e013f30b798a6b292ed1f3908e2549c06a358188f50437 2024-04-23T14:45:20.252 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 14:45:20 smithi173 systemd[1]: Started Ceph mds.k for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:45:20.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.416+0000 7f4d22a00640 1 -- 172.21.15.83:0/1587536490 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d1c062c00 con 0x7f4d1c108480 2024-04-23T14:45:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.417+0000 7f4d1a000640 1 -- 172.21.15.83:0/1587536490 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2215 (secure 0 0 0) 0x7f4d10085e20 con 0x7f4d1c108480 2024-04-23T14:45:20.418 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:20.418 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T14:45:20.418 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 3 2024-04-23T14:45:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.420+0000 7f4d22a00640 1 -- 172.21.15.83:0/1587536490 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4d040a3c30 msgr2=0x7f4d040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.420+0000 7f4d22a00640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4d040a3c30 0x7f4d040a60f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c19fc00 tx=0x7f4d0c005f90 comp rx=0 tx=0).stop 2024-04-23T14:45:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.420+0000 7f4d22a00640 1 -- 172.21.15.83:0/1587536490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c108480 msgr2=0x7f4d1c19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.420+0000 7f4d22a00640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c108480 0x7f4d1c19e740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000e9f0 tx=0x7f4d1000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:45:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.420+0000 7f4d22a00640 1 -- 172.21.15.83:0/1587536490 shutdown_connections 2024-04-23T14:45:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.420+0000 7f4d22a00640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4d040a3c30 0x7f4d040a60f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:20.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.420+0000 7f4d22a00640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d1c10a030 0x7f4d1c1a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.420+0000 7f4d22a00640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d1c109670 0x7f4d1c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.420+0000 7f4d22a00640 1 --2- 172.21.15.83:0/1587536490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d1c108480 0x7f4d1c19e740 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.420+0000 7f4d22a00640 1 -- 172.21.15.83:0/1587536490 >> 172.21.15.83:0/1587536490 conn(0x7f4d1c077440 msgr2=0x7f4d1c10c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.420+0000 7f4d22a00640 1 -- 172.21.15.83:0/1587536490 shutdown_connections 2024-04-23T14:45:20.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:20.421+0000 7f4d22a00640 1 -- 172.21.15.83:0/1587536490 wait complete. 2024-04-23T14:45:20.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:20 smithi083 ceph-mon[27867]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:20.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:20 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:20.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:20 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:20.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:20 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:20.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:20 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:20.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:20 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:20.437 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:20 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:20 smithi173 ceph-mon[32443]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:20 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:20 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:20 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:20 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:20 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:20 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:20 smithi203 ceph-mon[32800]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:20 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:20 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:20.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:20 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:20.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:20 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:20.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:20 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:20.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:20 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:21.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:21 smithi083 ceph-mon[27867]: Deploying daemon mds.j on smithi083 2024-04-23T14:45:21.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:21 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.173:6832/39034794,v1:172.21.15.173:6833/39034794] up:boot 2024-04-23T14:45:21.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:21 smithi083 ceph-mon[27867]: fsmap 2 up:standby 2024-04-23T14:45:21.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:21 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T14:45:21.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:21 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1587536490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:21 smithi173 ceph-mon[32443]: Deploying daemon mds.j on smithi083 2024-04-23T14:45:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:21 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.173:6832/39034794,v1:172.21.15.173:6833/39034794] up:boot 2024-04-23T14:45:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:21 smithi173 ceph-mon[32443]: fsmap 2 up:standby 2024-04-23T14:45:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:21 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T14:45:21.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:21 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1587536490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:21.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:21 smithi203 ceph-mon[32800]: Deploying daemon mds.j on smithi083 2024-04-23T14:45:21.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:21 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.173:6832/39034794,v1:172.21.15.173:6833/39034794] up:boot 2024-04-23T14:45:21.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:21 smithi203 ceph-mon[32800]: fsmap 2 up:standby 2024-04-23T14:45:21.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:21 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T14:45:21.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:21 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1587536490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:22.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:22.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:22 smithi173 ceph-mon[32443]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:22.720 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:22 smithi083 ceph-mon[27867]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:22.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:22 smithi203 ceph-mon[32800]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.857+0000 7f11e4a00640 1 -- 172.21.15.83:0/2574084242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11e0105c10 msgr2=0x7f11e0106090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.857+0000 7f11e4a00640 1 --2- 172.21.15.83:0/2574084242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11e0105c10 0x7f11e0106090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009900 tx=0x7f11d802a440 comp rx=0 tx=0).stop 2024-04-23T14:45:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.857+0000 7f11e4a00640 1 -- 172.21.15.83:0/2574084242 shutdown_connections 2024-04-23T14:45:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.857+0000 7f11e4a00640 1 --2- 172.21.15.83:0/2574084242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11e0105c10 0x7f11e0106090 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.857+0000 7f11e4a00640 1 --2- 172.21.15.83:0/2574084242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11e0103260 0x7f11e01056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.857+0000 7f11e4a00640 1 --2- 172.21.15.83:0/2574084242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e01008f0 0x7f11e0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:22.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.857+0000 7f11e4a00640 1 -- 172.21.15.83:0/2574084242 >> 172.21.15.83:0/2574084242 conn(0x7f11e00fc4f0 msgr2=0x7f11e00fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.858+0000 7f11e4a00640 1 -- 172.21.15.83:0/2574084242 shutdown_connections 2024-04-23T14:45:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.858+0000 7f11e4a00640 1 -- 172.21.15.83:0/2574084242 wait complete. 2024-04-23T14:45:22.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.859+0000 7f11e4a00640 1 Processor -- start 2024-04-23T14:45:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.859+0000 7f11e4a00640 1 -- start start 2024-04-23T14:45:22.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.860+0000 7f11e4a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11e01008f0 0x7f11e0070990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:22.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.861+0000 7f11e4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e0103260 0x7f11e0070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:22.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.861+0000 7f11e4a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11e0105c10 0x7f11e01aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:22.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.861+0000 7f11e4a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11e0114320 con 0x7f11e0103260 2024-04-23T14:45:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.861+0000 7f11e4a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01141a0 con 0x7f11e01008f0 2024-04-23T14:45:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.861+0000 7f11e4a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f11e01144a0 con 0x7f11e0105c10 2024-04-23T14:45:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.861+0000 7f11dd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e0103260 0x7f11e0070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.861+0000 7f11dde00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11e01008f0 0x7f11e0070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.861+0000 7f11dde00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11e01008f0 0x7f11e0070990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:42096/0 (socket says 172.21.15.83:42096) 2024-04-23T14:45:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.861+0000 7f11dde00640 1 -- 172.21.15.83:0/1598437849 learned_addr learned my addr 172.21.15.83:0/1598437849 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:22.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.861+0000 7f11de800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11e0105c10 0x7f11e01aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.862+0000 7f11dde00640 1 -- 172.21.15.83:0/1598437849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11e0105c10 msgr2=0x7f11e01aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.862+0000 7f11dde00640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11e0105c10 0x7f11e01aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.862+0000 7f11dde00640 1 -- 172.21.15.83:0/1598437849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e0103260 msgr2=0x7f11e0070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.862+0000 7f11dde00640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e0103260 0x7f11e0070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:22.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.862+0000 7f11dde00640 1 -- 172.21.15.83:0/1598437849 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e01aef70 con 0x7f11e01008f0 2024-04-23T14:45:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.862+0000 7f11dd400640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e0103260 0x7f11e0070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.863+0000 7f11dde00640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11e01008f0 0x7f11e0070990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f11c800c890 tx=0x7f11c800cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.863+0000 7f11d6a00640 1 -- 172.21.15.83:0/1598437849 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8004220 con 0x7f11e01008f0 2024-04-23T14:45:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.863+0000 7f11d6a00640 1 -- 172.21.15.83:0/1598437849 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11c80043c0 con 0x7f11e01008f0 2024-04-23T14:45:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.863+0000 7f11e4a00640 1 -- 172.21.15.83:0/1598437849 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e01af140 con 0x7f11e01008f0 2024-04-23T14:45:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.863+0000 7f11d6a00640 1 -- 172.21.15.83:0/1598437849 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c80049e0 con 0x7f11e01008f0 2024-04-23T14:45:22.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.863+0000 7f11e4a00640 1 -- 172.21.15.83:0/1598437849 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e01af630 con 0x7f11e01008f0 2024-04-23T14:45:22.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.866+0000 7f11e4a00640 1 -- 172.21.15.83:0/1598437849 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11e0068590 con 0x7f11e01008f0 2024-04-23T14:45:22.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.867+0000 7f11d6a00640 1 -- 172.21.15.83:0/1598437849 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f11c8004c40 con 0x7f11e01008f0 2024-04-23T14:45:22.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.869+0000 7f11d6a00640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f11c40a3cb0 0x7f11c40a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:22.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.870+0000 7f11d6a00640 1 -- 172.21.15.83:0/1598437849 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f11c80cffe0 con 0x7f11e01008f0 2024-04-23T14:45:22.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.870+0000 7f11dd400640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f11c40a3cb0 0x7f11c40a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:22.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.870+0000 7f11dd400640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f11c40a3cb0 0x7f11c40a6170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f11d00099a0 tx=0x7f11d0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:22.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:22.873+0000 7f11d6a00640 1 -- 172.21.15.83:0/1598437849 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11c8014030 con 0x7f11e01008f0 2024-04-23T14:45:23.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.123+0000 7f11e4a00640 1 -- 172.21.15.83:0/1598437849 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11e00fdfe0 con 0x7f11e01008f0 2024-04-23T14:45:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.123+0000 7f11d6a00640 1 -- 172.21.15.83:0/1598437849 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2215 (secure 0 0 0) 0x7f11e00fdfe0 con 0x7f11e01008f0 2024-04-23T14:45:23.124 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:23.124 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T14:45:23.124 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 3 2024-04-23T14:45:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.126+0000 7f11e4a00640 1 -- 172.21.15.83:0/1598437849 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f11c40a3cb0 msgr2=0x7f11c40a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.126+0000 7f11e4a00640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f11c40a3cb0 0x7f11c40a6170 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f11d00099a0 tx=0x7f11d0009500 comp rx=0 tx=0).stop 2024-04-23T14:45:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.126+0000 7f11e4a00640 1 -- 172.21.15.83:0/1598437849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11e01008f0 msgr2=0x7f11e0070990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:23.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.126+0000 7f11e4a00640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11e01008f0 0x7f11e0070990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f11c800c890 tx=0x7f11c800cd50 comp rx=0 tx=0).stop 2024-04-23T14:45:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.126+0000 7f11e4a00640 1 -- 172.21.15.83:0/1598437849 shutdown_connections 2024-04-23T14:45:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.126+0000 7f11e4a00640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f11c40a3cb0 0x7f11c40a6170 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.126+0000 7f11e4a00640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f11e0105c10 0x7f11e01aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.126+0000 7f11e4a00640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11e0103260 0x7f11e0070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.126+0000 7f11e4a00640 1 --2- 172.21.15.83:0/1598437849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11e01008f0 0x7f11e0070990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.126+0000 7f11e4a00640 1 -- 172.21.15.83:0/1598437849 >> 172.21.15.83:0/1598437849 conn(0x7f11e00fc4f0 msgr2=0x7f11e0101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.126+0000 7f11e4a00640 1 -- 172.21.15.83:0/1598437849 shutdown_connections 2024-04-23T14:45:23.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:23.126+0000 7f11e4a00640 1 -- 172.21.15.83:0/1598437849 wait complete. 2024-04-23T14:45:23.431 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:23 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1598437849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:23.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:23 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1598437849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:23.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:23 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1598437849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:23.755 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 14:45:23 smithi083 podman[67380]: 2024-04-23 14:45:23.456883283 +0000 UTC m=+0.155482638 container create 6f4fa56f68af72204b9b4e90a2b325f8b9038addd3e38318ace42368d5a1053a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-23T14:45:23.755 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 14:45:23 smithi083 podman[67380]: 2024-04-23 14:45:23.752666695 +0000 UTC m=+0.451266053 container init 6f4fa56f68af72204b9b4e90a2b325f8b9038addd3e38318ace42368d5a1053a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-23T14:45:24.007 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 14:45:23 smithi083 podman[67380]: 2024-04-23 14:45:23.763280598 +0000 UTC m=+0.461879949 container start 6f4fa56f68af72204b9b4e90a2b325f8b9038addd3e38318ace42368d5a1053a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-23T14:45:24.008 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 14:45:23 smithi083 ceph-mds[67398]: -- 172.21.15.83:0/4203835962 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579592dda40 con 0x5579592b0c00 2024-04-23T14:45:24.008 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 14:45:23 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-j[67394]: starting mds.j at 2024-04-23T14:45:24.008 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 14:45:23 smithi083 bash[67380]: 6f4fa56f68af72204b9b4e90a2b325f8b9038addd3e38318ace42368d5a1053a 2024-04-23T14:45:24.008 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 14:45:23 smithi083 systemd[1]: Started Ceph mds.j for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:45:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:24 smithi173 ceph-mon[32443]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:24.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:24 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:24 smithi203 ceph-mon[32800]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:24.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:24.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:24.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:24.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:24 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:24.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:24 smithi083 ceph-mon[27867]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:24.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:24.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:24.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:24.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:24.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:24.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:24 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:25.147 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.368+0000 7fe34c200640 1 -- 172.21.15.83:0/917369777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344105450 msgr2=0x7fe3441058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.368+0000 7fe34c200640 1 --2- 172.21.15.83:0/917369777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344105450 0x7fe3441058d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe330009900 tx=0x7fe33002a3f0 comp rx=0 tx=0).stop 2024-04-23T14:45:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.368+0000 7fe34c200640 1 -- 172.21.15.83:0/917369777 shutdown_connections 2024-04-23T14:45:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.368+0000 7fe34c200640 1 --2- 172.21.15.83:0/917369777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe344105e10 0x7fe34410c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.368+0000 7fe34c200640 1 --2- 172.21.15.83:0/917369777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344105450 0x7fe3441058d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.368+0000 7fe34c200640 1 --2- 172.21.15.83:0/917369777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe344104260 0x7fe344104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.368+0000 7fe34c200640 1 -- 172.21.15.83:0/917369777 >> 172.21.15.83:0/917369777 conn(0x7fe3440ff7f0 msgr2=0x7fe344101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.368+0000 7fe34c200640 1 -- 172.21.15.83:0/917369777 shutdown_connections 2024-04-23T14:45:25.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.368+0000 7fe34c200640 1 -- 172.21.15.83:0/917369777 wait complete. 2024-04-23T14:45:25.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.369+0000 7fe34c200640 1 Processor -- start 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.369+0000 7fe34c200640 1 -- start start 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.369+0000 7fe34c200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe344104260 0x7fe34419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe34c200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe344105450 0x7fe34419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe34c200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344105e10 0x7fe3441a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe34c200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe34410ff40 con 0x7fe344105e10 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe34c200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe34410fdc0 con 0x7fe344104260 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe34c200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3441100c0 con 0x7fe344105450 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe34a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344105e10 0x7fe3441a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe349800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe344104260 0x7fe34419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe349800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe344104260 0x7fe34419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:45168/0 (socket says 172.21.15.83:45168) 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe34a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344105e10 0x7fe3441a1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38470/0 (socket says 172.21.15.83:38470) 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe349800640 1 -- 172.21.15.83:0/737867376 learned_addr learned my addr 172.21.15.83:0/737867376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:25.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe348e00640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe344105450 0x7fe34419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe349800640 1 -- 172.21.15.83:0/737867376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe344105450 msgr2=0x7fe34419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe349800640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe344105450 0x7fe34419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe349800640 1 -- 172.21.15.83:0/737867376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344105e10 msgr2=0x7fe3441a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe349800640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344105e10 0x7fe3441a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe349800640 1 -- 172.21.15.83:0/737867376 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3441a2190 con 0x7fe344104260 2024-04-23T14:45:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.370+0000 7fe34a200640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344105e10 0x7fe3441a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.371+0000 7fe349800640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe344104260 0x7fe34419a570 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe33800eb70 tx=0x7fe33800ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.371+0000 7fe342000640 1 -- 172.21.15.83:0/737867376 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338003970 con 0x7fe344104260 2024-04-23T14:45:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.371+0000 7fe342000640 1 -- 172.21.15.83:0/737867376 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe338004020 con 0x7fe344104260 2024-04-23T14:45:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.371+0000 7fe342000640 1 -- 172.21.15.83:0/737867376 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338010670 con 0x7fe344104260 2024-04-23T14:45:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.371+0000 7fe34c200640 1 -- 172.21.15.83:0/737867376 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3441a2480 con 0x7fe344104260 2024-04-23T14:45:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.371+0000 7fe34c200640 1 -- 172.21.15.83:0/737867376 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3441a2950 con 0x7fe344104260 2024-04-23T14:45:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.372+0000 7fe34c200640 1 -- 172.21.15.83:0/737867376 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34410b580 con 0x7fe344104260 2024-04-23T14:45:25.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.374+0000 7fe342000640 1 -- 172.21.15.83:0/737867376 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe3380041d0 con 0x7fe344104260 2024-04-23T14:45:25.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.375+0000 7fe342000640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe32c0a3bb0 0x7fe32c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:25.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.375+0000 7fe342000640 1 -- 172.21.15.83:0/737867376 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe3380d06c0 con 0x7fe344104260 2024-04-23T14:45:25.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.375+0000 7fe348e00640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe32c0a3bb0 0x7fe32c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:25.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.376+0000 7fe348e00640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe32c0a3bb0 0x7fe32c0a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe330031440 tx=0x7fe3300315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:25.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.380+0000 7fe342000640 1 -- 172.21.15.83:0/737867376 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe33809d4a0 con 0x7fe344104260 2024-04-23T14:45:25.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:25 smithi083 ceph-mon[27867]: Deploying daemon mds.d on smithi083 2024-04-23T14:45:25.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:25 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.83:6834/928555402,v1:172.21.15.83:6835/928555402] up:boot 2024-04-23T14:45:25.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:25 smithi083 ceph-mon[27867]: fsmap 3 up:standby 2024-04-23T14:45:25.397 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:25 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T14:45:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:25 smithi173 ceph-mon[32443]: Deploying daemon mds.d on smithi083 2024-04-23T14:45:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:25 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.83:6834/928555402,v1:172.21.15.83:6835/928555402] up:boot 2024-04-23T14:45:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:25 smithi173 ceph-mon[32443]: fsmap 3 up:standby 2024-04-23T14:45:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:25 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T14:45:25.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.624+0000 7fe34c200640 1 -- 172.21.15.83:0/737867376 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe34402d620 con 0x7fe344104260 2024-04-23T14:45:25.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.625+0000 7fe342000640 1 -- 172.21.15.83:0/737867376 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3037 (secure 0 0 0) 0x7fe33809cda0 con 0x7fe344104260 2024-04-23T14:45:25.626 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:25.626 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T14:45:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 4 2024-04-23T14:45:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.628+0000 7fe34c200640 1 -- 172.21.15.83:0/737867376 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe32c0a3bb0 msgr2=0x7fe32c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.628+0000 7fe34c200640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe32c0a3bb0 0x7fe32c0a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe330031440 tx=0x7fe3300315b0 comp rx=0 tx=0).stop 2024-04-23T14:45:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.628+0000 7fe34c200640 1 -- 172.21.15.83:0/737867376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe344104260 msgr2=0x7fe34419a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.628+0000 7fe34c200640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe344104260 0x7fe34419a570 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe33800eb70 tx=0x7fe33800ef70 comp rx=0 tx=0).stop 2024-04-23T14:45:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.628+0000 7fe34c200640 1 -- 172.21.15.83:0/737867376 shutdown_connections 2024-04-23T14:45:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.628+0000 7fe34c200640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe32c0a3bb0 0x7fe32c0a6070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:25.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.628+0000 7fe34c200640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe344105e10 0x7fe3441a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.628+0000 7fe34c200640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe344105450 0x7fe34419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.628+0000 7fe34c200640 1 --2- 172.21.15.83:0/737867376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe344104260 0x7fe34419a570 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.628+0000 7fe34c200640 1 -- 172.21.15.83:0/737867376 >> 172.21.15.83:0/737867376 conn(0x7fe3440ff7f0 msgr2=0x7fe344108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.628+0000 7fe34c200640 1 -- 172.21.15.83:0/737867376 shutdown_connections 2024-04-23T14:45:25.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:25.628+0000 7fe34c200640 1 -- 172.21.15.83:0/737867376 wait complete. 2024-04-23T14:45:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:25 smithi203 ceph-mon[32800]: Deploying daemon mds.d on smithi083 2024-04-23T14:45:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:25 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.83:6834/928555402,v1:172.21.15.83:6835/928555402] up:boot 2024-04-23T14:45:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:25 smithi203 ceph-mon[32800]: fsmap 3 up:standby 2024-04-23T14:45:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:25 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T14:45:26.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:26 smithi083 ceph-mon[27867]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:26.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:26 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/737867376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:26 smithi173 ceph-mon[32443]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:26 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/737867376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:26 smithi203 ceph-mon[32800]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:26 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/737867376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:27.491 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 14:45:27 smithi083 systemd[1]: Starting Ceph mds.d for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:45:27.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:27.757 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 14:45:27 smithi083 podman[67719]: 2024-04-23 14:45:27.395485446 +0000 UTC m=+0.036579608 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T14:45:27.757 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 14:45:27 smithi083 podman[67719]: 2024-04-23 14:45:27.529830937 +0000 UTC m=+0.170925139 container create 024fcdd4320a1f571b7be5cd2be02000aef0117433f60df63169da95bc1ef262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-23T14:45:27.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.846+0000 7fab47e00640 1 -- 172.21.15.83:0/1960769568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab40104280 msgr2=0x7fab401046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:27.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.846+0000 7fab47e00640 1 --2- 172.21.15.83:0/1960769568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab40104280 0x7fab401046a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fab34009900 tx=0x7fab3402a410 comp rx=0 tx=0).stop 2024-04-23T14:45:27.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.846+0000 7fab47e00640 1 -- 172.21.15.83:0/1960769568 shutdown_connections 2024-04-23T14:45:27.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.846+0000 7fab47e00640 1 --2- 172.21.15.83:0/1960769568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab40105e30 0x7fab4010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:27.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.846+0000 7fab47e00640 1 --2- 172.21.15.83:0/1960769568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab40105470 0x7fab401058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:27.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.846+0000 7fab47e00640 1 --2- 172.21.15.83:0/1960769568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab40104280 0x7fab401046a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:27.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.846+0000 7fab47e00640 1 -- 172.21.15.83:0/1960769568 >> 172.21.15.83:0/1960769568 conn(0x7fab400ff7f0 msgr2=0x7fab40101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:27.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.846+0000 7fab47e00640 1 -- 172.21.15.83:0/1960769568 shutdown_connections 2024-04-23T14:45:27.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.846+0000 7fab47e00640 1 -- 172.21.15.83:0/1960769568 wait complete. 2024-04-23T14:45:27.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.847+0000 7fab47e00640 1 Processor -- start 2024-04-23T14:45:27.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.847+0000 7fab47e00640 1 -- start start 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.847+0000 7fab47e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab40104280 0x7fab4019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.847+0000 7fab47e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab40105470 0x7fab4019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab47e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab40105e30 0x7fab401a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab47e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab4010ff10 con 0x7fab40105470 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab47e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fab4010fd90 con 0x7fab40105e30 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab47e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fab40110090 con 0x7fab40104280 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab44a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab40105470 0x7fab4019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab44a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab40105470 0x7fab4019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38482/0 (socket says 172.21.15.83:38482) 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab44a00640 1 -- 172.21.15.83:0/1385608384 learned_addr learned my addr 172.21.15.83:0/1385608384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab45e00640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab40105e30 0x7fab401a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab45400640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab40104280 0x7fab4019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab44a00640 1 -- 172.21.15.83:0/1385608384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab40104280 msgr2=0x7fab4019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab44a00640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab40104280 0x7fab4019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab44a00640 1 -- 172.21.15.83:0/1385608384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab40105e30 msgr2=0x7fab401a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab44a00640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab40105e30 0x7fab401a1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:27.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.848+0000 7fab44a00640 1 -- 172.21.15.83:0/1385608384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab401a21a0 con 0x7fab40105470 2024-04-23T14:45:27.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.849+0000 7fab44a00640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab40105470 0x7fab4019aac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00ea60 tx=0x7fab2c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:27.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.849+0000 7fab3e000640 1 -- 172.21.15.83:0/1385608384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c00cde0 con 0x7fab40105470 2024-04-23T14:45:27.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.849+0000 7fab47e00640 1 -- 172.21.15.83:0/1385608384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab401a2490 con 0x7fab40105470 2024-04-23T14:45:27.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.849+0000 7fab3e000640 1 -- 172.21.15.83:0/1385608384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2c0035f0 con 0x7fab40105470 2024-04-23T14:45:27.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.849+0000 7fab3e000640 1 -- 172.21.15.83:0/1385608384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c010670 con 0x7fab40105470 2024-04-23T14:45:27.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.849+0000 7fab47e00640 1 -- 172.21.15.83:0/1385608384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab401a2a30 con 0x7fab40105470 2024-04-23T14:45:27.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.851+0000 7fab47e00640 1 -- 172.21.15.83:0/1385608384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab4010b5a0 con 0x7fab40105470 2024-04-23T14:45:27.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.853+0000 7fab3e000640 1 -- 172.21.15.83:0/1385608384 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fab2c0037a0 con 0x7fab40105470 2024-04-23T14:45:27.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.854+0000 7fab3e000640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fab280a3c30 0x7fab280a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:27.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.855+0000 7fab3e000640 1 -- 172.21.15.83:0/1385608384 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fab2c0d0230 con 0x7fab40105470 2024-04-23T14:45:27.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.855+0000 7fab45400640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fab280a3c30 0x7fab280a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:27.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.856+0000 7fab45400640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fab280a3c30 0x7fab280a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab34005ab0 tx=0x7fab34000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:27.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:27.859+0000 7fab3e000640 1 -- 172.21.15.83:0/1385608384 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab2c09d010 con 0x7fab40105470 2024-04-23T14:45:28.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.117+0000 7fab47e00640 1 -- 172.21.15.83:0/1385608384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab4002d620 con 0x7fab40105470 2024-04-23T14:45:28.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.118+0000 7fab3e000640 1 -- 172.21.15.83:0/1385608384 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3037 (secure 0 0 0) 0x7fab2c09c910 con 0x7fab40105470 2024-04-23T14:45:28.119 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:28.119 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T14:45:28.119 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 4 2024-04-23T14:45:28.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.121+0000 7fab47e00640 1 -- 172.21.15.83:0/1385608384 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fab280a3c30 msgr2=0x7fab280a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:28.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.121+0000 7fab47e00640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fab280a3c30 0x7fab280a60f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab34005ab0 tx=0x7fab34000bc0 comp rx=0 tx=0).stop 2024-04-23T14:45:28.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.121+0000 7fab47e00640 1 -- 172.21.15.83:0/1385608384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab40105470 msgr2=0x7fab4019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:28.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.121+0000 7fab47e00640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab40105470 0x7fab4019aac0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00ea60 tx=0x7fab2c00ef20 comp rx=0 tx=0).stop 2024-04-23T14:45:28.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.121+0000 7fab47e00640 1 -- 172.21.15.83:0/1385608384 shutdown_connections 2024-04-23T14:45:28.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.121+0000 7fab47e00640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fab280a3c30 0x7fab280a60f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:28.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.121+0000 7fab47e00640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab40105e30 0x7fab401a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:28.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.121+0000 7fab47e00640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab40105470 0x7fab4019aac0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:28.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.121+0000 7fab47e00640 1 --2- 172.21.15.83:0/1385608384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab40104280 0x7fab4019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:28.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.121+0000 7fab47e00640 1 -- 172.21.15.83:0/1385608384 >> 172.21.15.83:0/1385608384 conn(0x7fab400ff7f0 msgr2=0x7fab40108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:28.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.121+0000 7fab47e00640 1 -- 172.21.15.83:0/1385608384 shutdown_connections 2024-04-23T14:45:28.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:28.121+0000 7fab47e00640 1 -- 172.21.15.83:0/1385608384 wait complete. 2024-04-23T14:45:28.139 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 14:45:27 smithi083 podman[67719]: 2024-04-23 14:45:27.817352352 +0000 UTC m=+0.458446558 container init 024fcdd4320a1f571b7be5cd2be02000aef0117433f60df63169da95bc1ef262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-23T14:45:28.139 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 14:45:27 smithi083 podman[67719]: 2024-04-23 14:45:27.826986237 +0000 UTC m=+0.468080443 container start 024fcdd4320a1f571b7be5cd2be02000aef0117433f60df63169da95bc1ef262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-23T14:45:28.139 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 14:45:27 smithi083 ceph-mds[67758]: -- 172.21.15.83:0/1735797369 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be593e3a40 con 0x55be593b7000 2024-04-23T14:45:28.139 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 14:45:27 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-d[67741]: starting mds.d at 2024-04-23T14:45:28.139 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 14:45:27 smithi083 bash[67719]: 024fcdd4320a1f571b7be5cd2be02000aef0117433f60df63169da95bc1ef262 2024-04-23T14:45:28.139 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 14:45:27 smithi083 systemd[1]: Started Ceph mds.d for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:45:28.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:28 smithi083 ceph-mon[27867]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:28.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:28 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:28.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:28 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:28.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:28 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:28.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:28 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:28.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:28 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:28.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:28 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:28.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:28 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1385608384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:28.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:28 smithi173 ceph-mon[32443]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:28.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:28 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:28.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:28 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:28.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:28 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:28.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:28 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:28.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:28 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:28.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:28 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:28.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:28 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1385608384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:28.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:28 smithi203 ceph-mon[32800]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:28.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:28 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:28.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:28 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:28.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:28 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:28.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:28 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:28.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:28 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:28.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:28 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:28.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:28 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1385608384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:29.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:29 smithi173 ceph-mon[32443]: Deploying daemon mds.e on smithi173 2024-04-23T14:45:29.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:29 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.83:6836/3773827425,v1:172.21.15.83:6837/3773827425] up:boot 2024-04-23T14:45:29.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:29 smithi173 ceph-mon[32443]: fsmap 4 up:standby 2024-04-23T14:45:29.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:29 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T14:45:29.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:29 smithi203 ceph-mon[32800]: Deploying daemon mds.e on smithi173 2024-04-23T14:45:29.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:29 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.83:6836/3773827425,v1:172.21.15.83:6837/3773827425] up:boot 2024-04-23T14:45:29.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:29 smithi203 ceph-mon[32800]: fsmap 4 up:standby 2024-04-23T14:45:29.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:29 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T14:45:29.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:29 smithi083 ceph-mon[27867]: Deploying daemon mds.e on smithi173 2024-04-23T14:45:29.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:29 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.83:6836/3773827425,v1:172.21.15.83:6837/3773827425] up:boot 2024-04-23T14:45:29.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:29 smithi083 ceph-mon[27867]: fsmap 4 up:standby 2024-04-23T14:45:29.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:29 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T14:45:30.142 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:30.319 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:45:30 smithi173 systemd[1]: Starting Ceph mds.e for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:45:30.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.358+0000 7f80e9200640 1 -- 172.21.15.83:0/2047542213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80e4105c30 msgr2=0x7f80e410c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:30.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.358+0000 7f80e9200640 1 --2- 172.21.15.83:0/2047542213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80e4105c30 0x7f80e410c4c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009ca0 tx=0x7f80dc02a8c0 comp rx=0 tx=0).stop 2024-04-23T14:45:30.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.358+0000 7f80e9200640 1 -- 172.21.15.83:0/2047542213 shutdown_connections 2024-04-23T14:45:30.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.358+0000 7f80e9200640 1 --2- 172.21.15.83:0/2047542213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80e4105c30 0x7f80e410c4c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:30.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.358+0000 7f80e9200640 1 --2- 172.21.15.83:0/2047542213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e4105270 0x7f80e41056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:30.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.358+0000 7f80e9200640 1 --2- 172.21.15.83:0/2047542213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80e4104080 0x7f80e41044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:30.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.358+0000 7f80e9200640 1 -- 172.21.15.83:0/2047542213 >> 172.21.15.83:0/2047542213 conn(0x7f80e40ff610 msgr2=0x7f80e4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:30.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.358+0000 7f80e9200640 1 -- 172.21.15.83:0/2047542213 shutdown_connections 2024-04-23T14:45:30.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.359+0000 7f80e9200640 1 -- 172.21.15.83:0/2047542213 wait complete. 2024-04-23T14:45:30.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.359+0000 7f80e9200640 1 Processor -- start 2024-04-23T14:45:30.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80e9200640 1 -- start start 2024-04-23T14:45:30.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80e9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80e4104080 0x7f80e419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80e9200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e4105270 0x7f80e419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80e9200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80e4105c30 0x7f80e41a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80e9200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80e410fd60 con 0x7f80e4104080 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80e9200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80e410fbe0 con 0x7f80e4105270 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80e9200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80e410fee0 con 0x7f80e4105c30 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80dbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e4105270 0x7f80e419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80e3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80e4104080 0x7f80e419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80e3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80e4105c30 0x7f80e41a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80dbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e4105270 0x7f80e419a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:45200/0 (socket says 172.21.15.83:45200) 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80dbe00640 1 -- 172.21.15.83:0/585316203 learned_addr learned my addr 172.21.15.83:0/585316203 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80dbe00640 1 -- 172.21.15.83:0/585316203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80e4105c30 msgr2=0x7f80e41a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80dbe00640 1 --2- 172.21.15.83:0/585316203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80e4105c30 0x7f80e41a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80dbe00640 1 -- 172.21.15.83:0/585316203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80e4104080 msgr2=0x7f80e419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80dbe00640 1 --2- 172.21.15.83:0/585316203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80e4104080 0x7f80e419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:30.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.360+0000 7f80dbe00640 1 -- 172.21.15.83:0/585316203 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e41a1f40 con 0x7f80e4105270 2024-04-23T14:45:30.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.361+0000 7f80dbe00640 1 --2- 172.21.15.83:0/585316203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e4105270 0x7f80e419a860 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00e9f0 tx=0x7f80cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:30.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.361+0000 7f80e0c00640 1 -- 172.21.15.83:0/585316203 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc00cd80 con 0x7f80e4105270 2024-04-23T14:45:30.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.361+0000 7f80e0c00640 1 -- 172.21.15.83:0/585316203 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80cc010910 con 0x7f80e4105270 2024-04-23T14:45:30.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.361+0000 7f80e9200640 1 -- 172.21.15.83:0/585316203 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e41a2230 con 0x7f80e4105270 2024-04-23T14:45:30.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.361+0000 7f80e0c00640 1 -- 172.21.15.83:0/585316203 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc018990 con 0x7f80e4105270 2024-04-23T14:45:30.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.361+0000 7f80e9200640 1 -- 172.21.15.83:0/585316203 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e41a27d0 con 0x7f80e4105270 2024-04-23T14:45:30.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.362+0000 7f80e9200640 1 -- 172.21.15.83:0/585316203 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e410b3a0 con 0x7f80e4105270 2024-04-23T14:45:30.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.364+0000 7f80e0c00640 1 -- 172.21.15.83:0/585316203 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f80cc010ac0 con 0x7f80e4105270 2024-04-23T14:45:30.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.365+0000 7f80e0c00640 1 --2- 172.21.15.83:0/585316203 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80c80a3c30 0x7f80c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:30.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.365+0000 7f80e0c00640 1 -- 172.21.15.83:0/585316203 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f80cc0d0270 con 0x7f80e4105270 2024-04-23T14:45:30.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.365+0000 7f80e3400640 1 --2- 172.21.15.83:0/585316203 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80c80a3c30 0x7f80c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:30.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.366+0000 7f80e3400640 1 --2- 172.21.15.83:0/585316203 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80c80a3c30 0x7f80c80a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f80e41050d0 tx=0x7f80d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:30.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.369+0000 7f80e0c00640 1 -- 172.21.15.83:0/585316203 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80cc09ca00 con 0x7f80e4105270 2024-04-23T14:45:30.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:30 smithi083 ceph-mon[27867]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:30.574 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:30 smithi173 ceph-mon[32443]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:30.575 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:45:30 smithi173 podman[62475]: 2024-04-23 14:45:30.311470478 +0000 UTC m=+0.018174301 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T14:45:30.575 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:45:30 smithi173 podman[62475]: 2024-04-23 14:45:30.429320615 +0000 UTC m=+0.136024409 container create 59862320b8af9157b7af44d6a0612237fb8ca5289e93d311ab631a1bacc1b1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-23T14:45:30.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.610+0000 7f80e9200640 1 -- 172.21.15.83:0/585316203 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80e4062c00 con 0x7f80e4105270 2024-04-23T14:45:30.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.611+0000 7f80e0c00640 1 -- 172.21.15.83:0/585316203 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3862 (secure 0 0 0) 0x7f80cc0a0d00 con 0x7f80e4105270 2024-04-23T14:45:30.613 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:30.613 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-23T14:45:30.613 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 5 2024-04-23T14:45:30.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.614+0000 7f80e9200640 1 -- 172.21.15.83:0/585316203 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80c80a3c30 msgr2=0x7f80c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:30.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.614+0000 7f80e9200640 1 --2- 172.21.15.83:0/585316203 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80c80a3c30 0x7f80c80a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f80e41050d0 tx=0x7f80d4005f90 comp rx=0 tx=0).stop 2024-04-23T14:45:30.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.614+0000 7f80e9200640 1 -- 172.21.15.83:0/585316203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e4105270 msgr2=0x7f80e419a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:30.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.614+0000 7f80e9200640 1 --2- 172.21.15.83:0/585316203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e4105270 0x7f80e419a860 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00e9f0 tx=0x7f80cc00eeb0 comp rx=0 tx=0).stop 2024-04-23T14:45:30.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.614+0000 7f80e9200640 1 -- 172.21.15.83:0/585316203 shutdown_connections 2024-04-23T14:45:30.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.614+0000 7f80e9200640 1 --2- 172.21.15.83:0/585316203 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80c80a3c30 0x7f80c80a60f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:30.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.614+0000 7f80e9200640 1 --2- 172.21.15.83:0/585316203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80e4105c30 0x7f80e41a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:30.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.614+0000 7f80e9200640 1 --2- 172.21.15.83:0/585316203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80e4105270 0x7f80e419a860 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:30.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.614+0000 7f80e9200640 1 --2- 172.21.15.83:0/585316203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80e4104080 0x7f80e419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:30.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.614+0000 7f80e9200640 1 -- 172.21.15.83:0/585316203 >> 172.21.15.83:0/585316203 conn(0x7f80e40ff610 msgr2=0x7f80e4108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:30.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.614+0000 7f80e9200640 1 -- 172.21.15.83:0/585316203 shutdown_connections 2024-04-23T14:45:30.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:30.614+0000 7f80e9200640 1 -- 172.21.15.83:0/585316203 wait complete. 2024-04-23T14:45:30.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:30 smithi203 ceph-mon[32800]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:31.076 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:45:30 smithi173 podman[62475]: 2024-04-23 14:45:30.673839816 +0000 UTC m=+0.380543621 container init 59862320b8af9157b7af44d6a0612237fb8ca5289e93d311ab631a1bacc1b1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T14:45:31.076 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:45:30 smithi173 podman[62475]: 2024-04-23 14:45:30.676562027 +0000 UTC m=+0.383265823 container start 59862320b8af9157b7af44d6a0612237fb8ca5289e93d311ab631a1bacc1b1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-23T14:45:31.076 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:45:30 smithi173 ceph-mds[62494]: -- 172.21.15.173:0/617613106 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee34d2da40 con 0x55ee34d00c00 2024-04-23T14:45:31.076 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:45:30 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-e[62490]: starting mds.e at 2024-04-23T14:45:31.076 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:45:30 smithi173 bash[62475]: 59862320b8af9157b7af44d6a0612237fb8ca5289e93d311ab631a1bacc1b1b0 2024-04-23T14:45:31.076 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:45:30 smithi173 systemd[1]: Started Ceph mds.e for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:45:31.573 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:31 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/585316203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:31.573 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:31 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:31.573 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:31 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:31.573 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:31 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:31.573 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:31 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:31.573 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:31 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:31.573 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:31 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:31.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:31 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/585316203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:31.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:31 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:31.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:31 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:31.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:31 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:31.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:31 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:31.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:31 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:31.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:31 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:31.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:31 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/585316203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:31.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:31 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:31.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:31 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:31.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:31 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:31.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:31 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:31.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:31 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:31.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:31 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:32 smithi083 ceph-mon[27867]: Deploying daemon mds.f on smithi203 2024-04-23T14:45:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:32 smithi083 ceph-mon[27867]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:32 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.173:6834/2567101723,v1:172.21.15.173:6835/2567101723] up:boot 2024-04-23T14:45:32.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:32 smithi083 ceph-mon[27867]: fsmap 5 up:standby 2024-04-23T14:45:32.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:32 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T14:45:32.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:32.671 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:32 smithi203 ceph-mon[32800]: Deploying daemon mds.f on smithi203 2024-04-23T14:45:32.671 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:32 smithi203 ceph-mon[32800]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:32.671 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:32 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.173:6834/2567101723,v1:172.21.15.173:6835/2567101723] up:boot 2024-04-23T14:45:32.671 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:32 smithi203 ceph-mon[32800]: fsmap 5 up:standby 2024-04-23T14:45:32.671 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:32 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T14:45:32.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:32 smithi173 ceph-mon[32443]: Deploying daemon mds.f on smithi203 2024-04-23T14:45:32.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:32 smithi173 ceph-mon[32443]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:32.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:32 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.173:6834/2567101723,v1:172.21.15.173:6835/2567101723] up:boot 2024-04-23T14:45:32.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:32 smithi173 ceph-mon[32443]: fsmap 5 up:standby 2024-04-23T14:45:32.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:32 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T14:45:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.850+0000 7f01c4600640 1 -- 172.21.15.83:0/3912769614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01bc105470 msgr2=0x7f01bc1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.850+0000 7f01c4600640 1 --2- 172.21.15.83:0/3912769614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01bc105470 0x7f01bc1058f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009900 tx=0x7f01a802a3f0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.850+0000 7f01c4600640 1 -- 172.21.15.83:0/3912769614 shutdown_connections 2024-04-23T14:45:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.850+0000 7f01c4600640 1 --2- 172.21.15.83:0/3912769614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01bc105e30 0x7f01bc10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.850+0000 7f01c4600640 1 --2- 172.21.15.83:0/3912769614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01bc105470 0x7f01bc1058f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.850+0000 7f01c4600640 1 --2- 172.21.15.83:0/3912769614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01bc104280 0x7f01bc1046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.850+0000 7f01c4600640 1 -- 172.21.15.83:0/3912769614 >> 172.21.15.83:0/3912769614 conn(0x7f01bc0ff7f0 msgr2=0x7f01bc101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.851+0000 7f01c4600640 1 -- 172.21.15.83:0/3912769614 shutdown_connections 2024-04-23T14:45:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.851+0000 7f01c4600640 1 -- 172.21.15.83:0/3912769614 wait complete. 2024-04-23T14:45:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.852+0000 7f01c4600640 1 Processor -- start 2024-04-23T14:45:32.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.852+0000 7f01c4600640 1 -- start start 2024-04-23T14:45:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c4600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01bc104280 0x7f01bc19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c4600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01bc105470 0x7f01bc19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c4600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01bc105e30 0x7f01bc1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:32.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c4600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc10ff60 con 0x7f01bc104280 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c4600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc10fde0 con 0x7f01bc105e30 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c4600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc1100e0 con 0x7f01bc105470 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c1c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01bc104280 0x7f01bc19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01bc105470 0x7f01bc19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01bc105470 0x7f01bc19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:54598/0 (socket says 172.21.15.83:54598) 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c1200640 1 -- 172.21.15.83:0/3424355579 learned_addr learned my addr 172.21.15.83:0/3424355579 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c2600640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01bc105e30 0x7f01bc1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c1c00640 1 -- 172.21.15.83:0/3424355579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01bc105470 msgr2=0x7f01bc19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c1c00640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01bc105470 0x7f01bc19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c1c00640 1 -- 172.21.15.83:0/3424355579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01bc105e30 msgr2=0x7f01bc1a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c1c00640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01bc105e30 0x7f01bc1a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.853+0000 7f01c1c00640 1 -- 172.21.15.83:0/3424355579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc1a2150 con 0x7f01bc104280 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.854+0000 7f01c1c00640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01bc104280 0x7f01bc19a530 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f01b000e9c0 tx=0x7f01b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:32.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.854+0000 7f01ba000640 1 -- 172.21.15.83:0/3424355579 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b000cd80 con 0x7f01bc104280 2024-04-23T14:45:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.854+0000 7f01ba000640 1 -- 172.21.15.83:0/3424355579 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b0010910 con 0x7f01bc104280 2024-04-23T14:45:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.854+0000 7f01ba000640 1 -- 172.21.15.83:0/3424355579 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b0018980 con 0x7f01bc104280 2024-04-23T14:45:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.854+0000 7f01c4600640 1 -- 172.21.15.83:0/3424355579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01bc1a2440 con 0x7f01bc104280 2024-04-23T14:45:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.854+0000 7f01c4600640 1 -- 172.21.15.83:0/3424355579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01bc1a2990 con 0x7f01bc104280 2024-04-23T14:45:32.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.855+0000 7f01c4600640 1 -- 172.21.15.83:0/3424355579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01bc10b5a0 con 0x7f01bc104280 2024-04-23T14:45:32.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.857+0000 7f01ba000640 1 -- 172.21.15.83:0/3424355579 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f01b001f020 con 0x7f01bc104280 2024-04-23T14:45:32.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.859+0000 7f01ba000640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f01a40a3bb0 0x7f01a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:32.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.859+0000 7f01ba000640 1 -- 172.21.15.83:0/3424355579 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f01b00cffa0 con 0x7f01bc104280 2024-04-23T14:45:32.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.859+0000 7f01c1200640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f01a40a3bb0 0x7f01a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:32.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.860+0000 7f01c1200640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f01a40a3bb0 0x7f01a40a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f01bc19ba40 tx=0x7f01a80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:32.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:32.863+0000 7f01ba000640 1 -- 172.21.15.83:0/3424355579 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01b009cd80 con 0x7f01bc104280 2024-04-23T14:45:33.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.106+0000 7f01c4600640 1 -- 172.21.15.83:0/3424355579 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01bc02d620 con 0x7f01bc104280 2024-04-23T14:45:33.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.108+0000 7f01ba000640 1 -- 172.21.15.83:0/3424355579 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7f01b009c680 con 0x7f01bc104280 2024-04-23T14:45:33.110 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:33.110 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T14:45:33.110 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 6 2024-04-23T14:45:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.110+0000 7f01c4600640 1 -- 172.21.15.83:0/3424355579 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f01a40a3bb0 msgr2=0x7f01a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.110+0000 7f01c4600640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f01a40a3bb0 0x7f01a40a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f01bc19ba40 tx=0x7f01a80315b0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.111+0000 7f01c4600640 1 -- 172.21.15.83:0/3424355579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01bc104280 msgr2=0x7f01bc19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:33.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.111+0000 7f01c4600640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01bc104280 0x7f01bc19a530 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f01b000e9c0 tx=0x7f01b000ee80 comp rx=0 tx=0).stop 2024-04-23T14:45:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.111+0000 7f01c4600640 1 -- 172.21.15.83:0/3424355579 shutdown_connections 2024-04-23T14:45:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.111+0000 7f01c4600640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f01a40a3bb0 0x7f01a40a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.111+0000 7f01c4600640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01bc105e30 0x7f01bc1a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.111+0000 7f01c4600640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01bc105470 0x7f01bc19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.111+0000 7f01c4600640 1 --2- 172.21.15.83:0/3424355579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01bc104280 0x7f01bc19a530 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.111+0000 7f01c4600640 1 -- 172.21.15.83:0/3424355579 >> 172.21.15.83:0/3424355579 conn(0x7f01bc0ff7f0 msgr2=0x7f01bc108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.111+0000 7f01c4600640 1 -- 172.21.15.83:0/3424355579 shutdown_connections 2024-04-23T14:45:33.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:33.111+0000 7f01c4600640 1 -- 172.21.15.83:0/3424355579 wait complete. 2024-04-23T14:45:33.350 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:45:33 smithi203 systemd[1]: Starting Ceph mds.f for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:45:33.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:33 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3424355579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:33.687 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:33 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3424355579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:33.687 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:45:33 smithi203 podman[62916]: 2024-04-23 14:45:33.276667466 +0000 UTC m=+0.018050314 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T14:45:33.687 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:45:33 smithi203 podman[62916]: 2024-04-23 14:45:33.41720758 +0000 UTC m=+0.158590412 container create 8108ce5b96073d0971db2d759c6a87ede89ae9102688f08fe07927d2e453bfb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-23T14:45:33.687 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:45:33 smithi203 podman[62916]: 2024-04-23 14:45:33.686892492 +0000 UTC m=+0.428275327 container init 8108ce5b96073d0971db2d759c6a87ede89ae9102688f08fe07927d2e453bfb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5) 2024-04-23T14:45:33.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:33 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3424355579' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:33.971 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:45:33 smithi203 podman[62916]: 2024-04-23 14:45:33.689866138 +0000 UTC m=+0.431248983 container start 8108ce5b96073d0971db2d759c6a87ede89ae9102688f08fe07927d2e453bfb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-23T14:45:33.971 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:45:33 smithi203 ceph-mds[62935]: -- 172.21.15.203:0/1448565639 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5590ade1da40 con 0x5590addf1000 2024-04-23T14:45:33.972 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:45:33 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f[62931]: starting mds.f at 2024-04-23T14:45:33.972 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:45:33 smithi203 bash[62916]: 8108ce5b96073d0971db2d759c6a87ede89ae9102688f08fe07927d2e453bfb0 2024-04-23T14:45:33.972 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:45:33 smithi203 systemd[1]: Started Ceph mds.f for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:45:34.674 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:34 smithi203 ceph-mon[32800]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:34.674 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:34.674 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:34.674 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:34.674 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:34.674 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:34.674 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:34 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:34 smithi083 ceph-mon[27867]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:34 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:34.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:34 smithi173 ceph-mon[32443]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:34.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:34.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:34.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:34.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:34.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:34.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:34 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:35.131 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:35.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.349+0000 7f4388e00640 1 -- 172.21.15.83:0/814095901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4384105470 msgr2=0x7f43841058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:35.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.349+0000 7f4388e00640 1 --2- 172.21.15.83:0/814095901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4384105470 0x7f43841058f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f436c009860 tx=0x7f436c02a340 comp rx=0 tx=0).stop 2024-04-23T14:45:35.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.349+0000 7f4388e00640 1 -- 172.21.15.83:0/814095901 shutdown_connections 2024-04-23T14:45:35.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.349+0000 7f4388e00640 1 --2- 172.21.15.83:0/814095901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4384105e30 0x7f438410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:35.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.349+0000 7f4388e00640 1 --2- 172.21.15.83:0/814095901 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4384105470 0x7f43841058f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:35.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.349+0000 7f4388e00640 1 --2- 172.21.15.83:0/814095901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4384104280 0x7f43841046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:35.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.349+0000 7f4388e00640 1 -- 172.21.15.83:0/814095901 >> 172.21.15.83:0/814095901 conn(0x7f43840ff7f0 msgr2=0x7f4384101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:35.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.349+0000 7f4388e00640 1 -- 172.21.15.83:0/814095901 shutdown_connections 2024-04-23T14:45:35.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.349+0000 7f4388e00640 1 -- 172.21.15.83:0/814095901 wait complete. 2024-04-23T14:45:35.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.350+0000 7f4388e00640 1 Processor -- start 2024-04-23T14:45:35.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.350+0000 7f4388e00640 1 -- start start 2024-04-23T14:45:35.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.351+0000 7f4388e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4384104280 0x7f438419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:35.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.351+0000 7f4388e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4384105470 0x7f438419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:35.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.351+0000 7f4388e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4384105e30 0x7f43841a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:35.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.351+0000 7f4388e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f438410ff10 con 0x7f4384105e30 2024-04-23T14:45:35.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.351+0000 7f4388e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f438410fd90 con 0x7f4384104280 2024-04-23T14:45:35.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.351+0000 7f4388e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4384110090 con 0x7f4384105470 2024-04-23T14:45:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.351+0000 7f4382800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4384105e30 0x7f43841a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.351+0000 7f4381e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4384104280 0x7f438419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.351+0000 7f4381400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4384105470 0x7f438419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.351+0000 7f4381e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4384104280 0x7f438419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33092/0 (socket says 172.21.15.83:33092) 2024-04-23T14:45:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.351+0000 7f4381e00640 1 -- 172.21.15.83:0/2078200216 learned_addr learned my addr 172.21.15.83:0/2078200216 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f4381e00640 1 -- 172.21.15.83:0/2078200216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4384105470 msgr2=0x7f438419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f4381e00640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4384105470 0x7f438419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:35.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f4381e00640 1 -- 172.21.15.83:0/2078200216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4384105e30 msgr2=0x7f43841a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f4381e00640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4384105e30 0x7f43841a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f4381e00640 1 -- 172.21.15.83:0/2078200216 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43841a2190 con 0x7f4384104280 2024-04-23T14:45:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f4381400640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4384105470 0x7f438419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f4382800640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4384105e30 0x7f43841a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f4381e00640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4384104280 0x7f438419a570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f437400e9c0 tx=0x7f437400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f437aa00640 1 -- 172.21.15.83:0/2078200216 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437400cd80 con 0x7f4384104280 2024-04-23T14:45:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f437aa00640 1 -- 172.21.15.83:0/2078200216 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43740035f0 con 0x7f4384104280 2024-04-23T14:45:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f4388e00640 1 -- 172.21.15.83:0/2078200216 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43841a2480 con 0x7f4384104280 2024-04-23T14:45:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f437aa00640 1 -- 172.21.15.83:0/2078200216 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374010690 con 0x7f4384104280 2024-04-23T14:45:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.352+0000 7f4388e00640 1 -- 172.21.15.83:0/2078200216 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43841a2950 con 0x7f4384104280 2024-04-23T14:45:35.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.353+0000 7f4388e00640 1 -- 172.21.15.83:0/2078200216 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f438410b620 con 0x7f4384104280 2024-04-23T14:45:35.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.356+0000 7f437aa00640 1 -- 172.21.15.83:0/2078200216 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f43740037a0 con 0x7f4384104280 2024-04-23T14:45:35.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.357+0000 7f437aa00640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f43680a3bb0 0x7f43680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:35.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.357+0000 7f437aa00640 1 -- 172.21.15.83:0/2078200216 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f43740d0350 con 0x7f4384104280 2024-04-23T14:45:35.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.357+0000 7f4381400640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f43680a3bb0 0x7f43680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:35.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.358+0000 7f4381400640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f43680a3bb0 0x7f43680a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f436c02a930 tx=0x7f436c030570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:35.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.359+0000 7f437aa00640 1 -- 172.21.15.83:0/2078200216 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f437409d130 con 0x7f4384104280 2024-04-23T14:45:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:35 smithi083 ceph-mon[27867]: Deploying daemon mds.c on smithi203 2024-04-23T14:45:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:35 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.203:6834/2384637189,v1:172.21.15.203:6835/2384637189] up:boot 2024-04-23T14:45:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:35 smithi083 ceph-mon[27867]: fsmap 6 up:standby 2024-04-23T14:45:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:35 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T14:45:35.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.603+0000 7f4388e00640 1 -- 172.21.15.83:0/2078200216 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f438402d620 con 0x7f4384104280 2024-04-23T14:45:35.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.604+0000 7f437aa00640 1 -- 172.21.15.83:0/2078200216 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5518 (secure 0 0 0) 0x7f437409ca30 con 0x7f4384104280 2024-04-23T14:45:35.606 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:35.606 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T14:45:35.606 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 7 2024-04-23T14:45:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.607+0000 7f4388e00640 1 -- 172.21.15.83:0/2078200216 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f43680a3bb0 msgr2=0x7f43680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:35.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.607+0000 7f4388e00640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f43680a3bb0 0x7f43680a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f436c02a930 tx=0x7f436c030570 comp rx=0 tx=0).stop 2024-04-23T14:45:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.607+0000 7f4388e00640 1 -- 172.21.15.83:0/2078200216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4384104280 msgr2=0x7f438419a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.607+0000 7f4388e00640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4384104280 0x7f438419a570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f437400e9c0 tx=0x7f437400ee80 comp rx=0 tx=0).stop 2024-04-23T14:45:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.607+0000 7f4388e00640 1 -- 172.21.15.83:0/2078200216 shutdown_connections 2024-04-23T14:45:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.607+0000 7f4388e00640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f43680a3bb0 0x7f43680a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.607+0000 7f4388e00640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4384105e30 0x7f43841a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.607+0000 7f4388e00640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4384105470 0x7f438419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.607+0000 7f4388e00640 1 --2- 172.21.15.83:0/2078200216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4384104280 0x7f438419a570 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.607+0000 7f4388e00640 1 -- 172.21.15.83:0/2078200216 >> 172.21.15.83:0/2078200216 conn(0x7f43840ff7f0 msgr2=0x7f43841087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.607+0000 7f4388e00640 1 -- 172.21.15.83:0/2078200216 shutdown_connections 2024-04-23T14:45:35.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:35.607+0000 7f4388e00640 1 -- 172.21.15.83:0/2078200216 wait complete. 2024-04-23T14:45:35.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:35 smithi203 ceph-mon[32800]: Deploying daemon mds.c on smithi203 2024-04-23T14:45:35.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:35 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.203:6834/2384637189,v1:172.21.15.203:6835/2384637189] up:boot 2024-04-23T14:45:35.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:35 smithi203 ceph-mon[32800]: fsmap 6 up:standby 2024-04-23T14:45:35.707 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:35 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T14:45:35.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:35 smithi173 ceph-mon[32443]: Deploying daemon mds.c on smithi203 2024-04-23T14:45:35.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:35 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.203:6834/2384637189,v1:172.21.15.203:6835/2384637189] up:boot 2024-04-23T14:45:35.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:35 smithi173 ceph-mon[32443]: fsmap 6 up:standby 2024-04-23T14:45:35.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:35 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T14:45:36.255 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:45:36 smithi203 systemd[1]: Starting Ceph mds.c for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:45:36.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:36 smithi203 ceph-mon[32800]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:36.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:36 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2078200216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:36.722 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:45:36 smithi203 podman[63227]: 2024-04-23 14:45:36.334142245 +0000 UTC m=+0.018260359 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T14:45:36.722 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:45:36 smithi203 podman[63227]: 2024-04-23 14:45:36.460254786 +0000 UTC m=+0.144372879 container create b61521142c823471ba8a98eea9a401f7cd1fdf8194eb06799ce1b7aa0a944c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-23T14:45:36.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:36 smithi083 ceph-mon[27867]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:36.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:36 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2078200216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:36.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:36 smithi173 ceph-mon[32443]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:36.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:36 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2078200216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:37.207 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:45:36 smithi203 podman[63227]: 2024-04-23 14:45:36.854677015 +0000 UTC m=+0.538795108 container init b61521142c823471ba8a98eea9a401f7cd1fdf8194eb06799ce1b7aa0a944c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-23T14:45:37.208 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:45:36 smithi203 podman[63227]: 2024-04-23 14:45:36.858000953 +0000 UTC m=+0.542119047 container start b61521142c823471ba8a98eea9a401f7cd1fdf8194eb06799ce1b7aa0a944c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-04-23T14:45:37.208 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:45:36 smithi203 ceph-mds[63256]: -- 172.21.15.203:0/1996781811 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5625e8ac3a40 con 0x5625e8a96c00 2024-04-23T14:45:37.208 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:45:36 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c[63252]: starting mds.c at 2024-04-23T14:45:37.208 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:45:36 smithi203 bash[63227]: b61521142c823471ba8a98eea9a401f7cd1fdf8194eb06799ce1b7aa0a944c42 2024-04-23T14:45:37.208 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:45:36 smithi203 systemd[1]: Started Ceph mds.c for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:45:37.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:37.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.837+0000 7f5e7dc00640 1 -- 172.21.15.83:0/279295534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e78104280 msgr2=0x7f5e781046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:37.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.837+0000 7f5e7dc00640 1 --2- 172.21.15.83:0/279295534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e78104280 0x7f5e781046a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009be0 tx=0x7f5e6802a6c0 comp rx=0 tx=0).stop 2024-04-23T14:45:37.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.837+0000 7f5e7dc00640 1 -- 172.21.15.83:0/279295534 shutdown_connections 2024-04-23T14:45:37.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.837+0000 7f5e7dc00640 1 --2- 172.21.15.83:0/279295534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e78105e30 0x7f5e7810c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:37.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.837+0000 7f5e7dc00640 1 --2- 172.21.15.83:0/279295534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e78105470 0x7f5e781058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:37.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.837+0000 7f5e7dc00640 1 --2- 172.21.15.83:0/279295534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e78104280 0x7f5e781046a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:37.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.837+0000 7f5e7dc00640 1 -- 172.21.15.83:0/279295534 >> 172.21.15.83:0/279295534 conn(0x7f5e780ff7f0 msgr2=0x7f5e78101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:37.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.838+0000 7f5e7dc00640 1 -- 172.21.15.83:0/279295534 shutdown_connections 2024-04-23T14:45:37.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.838+0000 7f5e7dc00640 1 -- 172.21.15.83:0/279295534 wait complete. 2024-04-23T14:45:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.839+0000 7f5e7dc00640 1 Processor -- start 2024-04-23T14:45:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.839+0000 7f5e7dc00640 1 -- start start 2024-04-23T14:45:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.839+0000 7f5e7dc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e78104280 0x7f5e7819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.839+0000 7f5e7dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e78105470 0x7f5e7819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:37.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.839+0000 7f5e7dc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e78105e30 0x7f5e781a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.839+0000 7f5e7dc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7810fec0 con 0x7f5e78105470 2024-04-23T14:45:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.839+0000 7f5e7dc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7810fd40 con 0x7f5e78105e30 2024-04-23T14:45:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.839+0000 7f5e7dc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5e78110040 con 0x7f5e78104280 2024-04-23T14:45:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.840+0000 7f5e76a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e78104280 0x7f5e7819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.840+0000 7f5e6e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e78105470 0x7f5e7819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.840+0000 7f5e76a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e78104280 0x7f5e7819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:58398/0 (socket says 172.21.15.83:58398) 2024-04-23T14:45:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.840+0000 7f5e76a00640 1 -- 172.21.15.83:0/2577191985 learned_addr learned my addr 172.21.15.83:0/2577191985 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.840+0000 7f5e77400640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e78105e30 0x7f5e781a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.840+0000 7f5e76a00640 1 -- 172.21.15.83:0/2577191985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e78105e30 msgr2=0x7f5e781a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.840+0000 7f5e76a00640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e78105e30 0x7f5e781a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.840+0000 7f5e76a00640 1 -- 172.21.15.83:0/2577191985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e78105470 msgr2=0x7f5e7819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:37.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.840+0000 7f5e76a00640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e78105470 0x7f5e7819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:37.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.840+0000 7f5e76a00640 1 -- 172.21.15.83:0/2577191985 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e781a2180 con 0x7f5e78104280 2024-04-23T14:45:37.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.840+0000 7f5e77400640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e78105e30 0x7f5e781a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.840+0000 7f5e76a00640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e78104280 0x7f5e7819a560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e68030960 tx=0x7f5e680305b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.841+0000 7f5e6fe00640 1 -- 172.21.15.83:0/2577191985 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68041020 con 0x7f5e78104280 2024-04-23T14:45:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.841+0000 7f5e6fe00640 1 -- 172.21.15.83:0/2577191985 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e68037040 con 0x7f5e78104280 2024-04-23T14:45:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.841+0000 7f5e7dc00640 1 -- 172.21.15.83:0/2577191985 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e781a2410 con 0x7f5e78104280 2024-04-23T14:45:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.841+0000 7f5e6fe00640 1 -- 172.21.15.83:0/2577191985 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68036070 con 0x7f5e78104280 2024-04-23T14:45:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.841+0000 7f5e7dc00640 1 -- 172.21.15.83:0/2577191985 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e781a2900 con 0x7f5e78104280 2024-04-23T14:45:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.842+0000 7f5e7dc00640 1 -- 172.21.15.83:0/2577191985 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7810b5c0 con 0x7f5e78104280 2024-04-23T14:45:37.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.844+0000 7f5e6fe00640 1 -- 172.21.15.83:0/2577191985 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5e68044020 con 0x7f5e78104280 2024-04-23T14:45:37.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.846+0000 7f5e6fe00640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5e5c0a3bb0 0x7f5e5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:37.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.846+0000 7f5e6fe00640 1 -- 172.21.15.83:0/2577191985 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5e680f1200 con 0x7f5e78104280 2024-04-23T14:45:37.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.846+0000 7f5e6e000640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5e5c0a3bb0 0x7f5e5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:37.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.846+0000 7f5e6e000640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5e5c0a3bb0 0x7f5e5c0a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e7819ba70 tx=0x7f5e60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:37.848+0000 7f5e6fe00640 1 -- 172.21.15.83:0/2577191985 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e6804b020 con 0x7f5e78104280 2024-04-23T14:45:38.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.093+0000 7f5e7dc00640 1 -- 172.21.15.83:0/2577191985 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e7802d620 con 0x7f5e78104280 2024-04-23T14:45:38.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.094+0000 7f5e6fe00640 1 -- 172.21.15.83:0/2577191985 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7f5e680bdfe0 con 0x7f5e78104280 2024-04-23T14:45:38.095 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:38.096 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-23T14:45:38.096 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 8 2024-04-23T14:45:38.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.096+0000 7f5e7dc00640 1 -- 172.21.15.83:0/2577191985 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5e5c0a3bb0 msgr2=0x7f5e5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:38.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.096+0000 7f5e7dc00640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5e5c0a3bb0 0x7f5e5c0a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e7819ba70 tx=0x7f5e60009260 comp rx=0 tx=0).stop 2024-04-23T14:45:38.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.096+0000 7f5e7dc00640 1 -- 172.21.15.83:0/2577191985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e78104280 msgr2=0x7f5e7819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.096+0000 7f5e7dc00640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e78104280 0x7f5e7819a560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e68030960 tx=0x7f5e680305b0 comp rx=0 tx=0).stop 2024-04-23T14:45:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.097+0000 7f5e7dc00640 1 -- 172.21.15.83:0/2577191985 shutdown_connections 2024-04-23T14:45:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.097+0000 7f5e7dc00640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5e5c0a3bb0 0x7f5e5c0a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.097+0000 7f5e7dc00640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e78105e30 0x7f5e781a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.097+0000 7f5e7dc00640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e78105470 0x7f5e7819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.097+0000 7f5e7dc00640 1 --2- 172.21.15.83:0/2577191985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e78104280 0x7f5e7819a560 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.097+0000 7f5e7dc00640 1 -- 172.21.15.83:0/2577191985 >> 172.21.15.83:0/2577191985 conn(0x7f5e780ff7f0 msgr2=0x7f5e78108790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.097+0000 7f5e7dc00640 1 -- 172.21.15.83:0/2577191985 shutdown_connections 2024-04-23T14:45:38.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:38.097+0000 7f5e7dc00640 1 -- 172.21.15.83:0/2577191985 wait complete. 2024-04-23T14:45:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:38 smithi173 ceph-mon[32443]: Deploying daemon mds.h on smithi173 2024-04-23T14:45:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:38 smithi173 ceph-mon[32443]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:38 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.203:6836/258138306,v1:172.21.15.203:6837/258138306] up:boot 2024-04-23T14:45:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:38 smithi173 ceph-mon[32443]: fsmap 7 up:standby 2024-04-23T14:45:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:38 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T14:45:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:38 smithi203 ceph-mon[32800]: Deploying daemon mds.h on smithi173 2024-04-23T14:45:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:38 smithi203 ceph-mon[32800]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:38 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.203:6836/258138306,v1:172.21.15.203:6837/258138306] up:boot 2024-04-23T14:45:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:38 smithi203 ceph-mon[32800]: fsmap 7 up:standby 2024-04-23T14:45:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:38 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T14:45:38.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:38.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:38.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:38.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:38.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:38 smithi083 ceph-mon[27867]: Deploying daemon mds.h on smithi173 2024-04-23T14:45:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:38 smithi083 ceph-mon[27867]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:38 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.203:6836/258138306,v1:172.21.15.203:6837/258138306] up:boot 2024-04-23T14:45:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:38 smithi083 ceph-mon[27867]: fsmap 7 up:standby 2024-04-23T14:45:38.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:38 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T14:45:39.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:39 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2577191985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:39.326 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:45:39 smithi173 systemd[1]: Starting Ceph mds.h for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:45:39.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:39 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2577191985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:39 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2577191985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:39.729 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:45:39 smithi173 podman[62782]: 2024-04-23 14:45:39.356479771 +0000 UTC m=+0.017658981 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T14:45:39.729 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:45:39 smithi173 podman[62782]: 2024-04-23 14:45:39.476182426 +0000 UTC m=+0.137361622 container create df4755f5aa57ec98459237851e4ebe0a9904a38e39b3cec6be112da7340ddbcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-h, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-23T14:45:39.729 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:45:39 smithi173 podman[62782]: 2024-04-23 14:45:39.72905351 +0000 UTC m=+0.390232707 container init df4755f5aa57ec98459237851e4ebe0a9904a38e39b3cec6be112da7340ddbcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-23T14:45:40.076 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:45:39 smithi173 podman[62782]: 2024-04-23 14:45:39.731597915 +0000 UTC m=+0.392777122 container start df4755f5aa57ec98459237851e4ebe0a9904a38e39b3cec6be112da7340ddbcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-23T14:45:40.076 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:45:39 smithi173 ceph-mds[62801]: -- 172.21.15.173:0/2330718317 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc8d5ada40 con 0x55cc8d580800 2024-04-23T14:45:40.076 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:45:39 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-h[62797]: starting mds.h at 2024-04-23T14:45:40.076 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:45:39 smithi173 bash[62782]: df4755f5aa57ec98459237851e4ebe0a9904a38e39b3cec6be112da7340ddbcc 2024-04-23T14:45:40.076 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:45:39 smithi173 systemd[1]: Started Ceph mds.h for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:45:40.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:40.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.318+0000 7ff131800640 1 -- 172.21.15.83:0/3877008214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c077c00 msgr2=0x7ff12c078060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:40.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.318+0000 7ff131800640 1 --2- 172.21.15.83:0/3877008214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c077c00 0x7ff12c078060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff118009900 tx=0x7ff11802a410 comp rx=0 tx=0).stop 2024-04-23T14:45:40.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.318+0000 7ff131800640 1 -- 172.21.15.83:0/3877008214 shutdown_connections 2024-04-23T14:45:40.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.318+0000 7ff131800640 1 --2- 172.21.15.83:0/3877008214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff12c0785a0 0x7ff12c078e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:40.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.318+0000 7ff131800640 1 --2- 172.21.15.83:0/3877008214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c077c00 0x7ff12c078060 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:40.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.318+0000 7ff131800640 1 --2- 172.21.15.83:0/3877008214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff12c0769c0 0x7ff12c076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:40.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.318+0000 7ff131800640 1 -- 172.21.15.83:0/3877008214 >> 172.21.15.83:0/3877008214 conn(0x7ff12c0ff610 msgr2=0x7ff12c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:40.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.318+0000 7ff131800640 1 -- 172.21.15.83:0/3877008214 shutdown_connections 2024-04-23T14:45:40.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.318+0000 7ff131800640 1 -- 172.21.15.83:0/3877008214 wait complete. 2024-04-23T14:45:40.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.319+0000 7ff131800640 1 Processor -- start 2024-04-23T14:45:40.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.319+0000 7ff131800640 1 -- start start 2024-04-23T14:45:40.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.319+0000 7ff131800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff12c0769c0 0x7ff12c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:40.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.319+0000 7ff131800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff12c077c00 0x7ff12c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:40.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.319+0000 7ff131800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c0785a0 0x7ff12c1a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:40.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.319+0000 7ff131800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c114150 con 0x7ff12c077c00 2024-04-23T14:45:40.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.319+0000 7ff131800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c113fd0 con 0x7ff12c0769c0 2024-04-23T14:45:40.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.319+0000 7ff131800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c1142d0 con 0x7ff12c0785a0 2024-04-23T14:45:40.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.320+0000 7ff12b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff12c077c00 0x7ff12c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:40.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.320+0000 7ff12b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff12c077c00 0x7ff12c19ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58110/0 (socket says 172.21.15.83:58110) 2024-04-23T14:45:40.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.320+0000 7ff12b400640 1 -- 172.21.15.83:0/2536284900 learned_addr learned my addr 172.21.15.83:0/2536284900 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:40.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.320+0000 7ff130e00640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c0785a0 0x7ff12c1a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:40.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.320+0000 7ff12be00640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff12c0769c0 0x7ff12c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:40.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.320+0000 7ff12b400640 1 -- 172.21.15.83:0/2536284900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c0785a0 msgr2=0x7ff12c1a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:40.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.320+0000 7ff12b400640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c0785a0 0x7ff12c1a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:40.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.320+0000 7ff12b400640 1 -- 172.21.15.83:0/2536284900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff12c0769c0 msgr2=0x7ff12c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:40.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.320+0000 7ff12b400640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff12c0769c0 0x7ff12c19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:40.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.320+0000 7ff12b400640 1 -- 172.21.15.83:0/2536284900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c1a6370 con 0x7ff12c077c00 2024-04-23T14:45:40.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.321+0000 7ff12b400640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff12c077c00 0x7ff12c19ec90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff118031440 tx=0x7ff118032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:40.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.321+0000 7ff128c00640 1 -- 172.21.15.83:0/2536284900 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11803f030 con 0x7ff12c077c00 2024-04-23T14:45:40.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.321+0000 7ff128c00640 1 -- 172.21.15.83:0/2536284900 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff118003ce0 con 0x7ff12c077c00 2024-04-23T14:45:40.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.321+0000 7ff131800640 1 -- 172.21.15.83:0/2536284900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c1a6600 con 0x7ff12c077c00 2024-04-23T14:45:40.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.321+0000 7ff128c00640 1 -- 172.21.15.83:0/2536284900 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118035070 con 0x7ff12c077c00 2024-04-23T14:45:40.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.321+0000 7ff131800640 1 -- 172.21.15.83:0/2536284900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c1a6b40 con 0x7ff12c077c00 2024-04-23T14:45:40.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.322+0000 7ff131800640 1 -- 172.21.15.83:0/2536284900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12c110b40 con 0x7ff12c077c00 2024-04-23T14:45:40.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.325+0000 7ff128c00640 1 -- 172.21.15.83:0/2536284900 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff118042030 con 0x7ff12c077c00 2024-04-23T14:45:40.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.326+0000 7ff128c00640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1100a3c30 0x7ff1100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:40.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.326+0000 7ff128c00640 1 -- 172.21.15.83:0/2536284900 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff1180f0410 con 0x7ff12c077c00 2024-04-23T14:45:40.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.326+0000 7ff12be00640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1100a3c30 0x7ff1100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:40.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.327+0000 7ff12be00640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1100a3c30 0x7ff1100a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009bd0 tx=0x7ff11c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:40.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.331+0000 7ff128c00640 1 -- 172.21.15.83:0/2536284900 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1180bcba0 con 0x7ff12c077c00 2024-04-23T14:45:40.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:40 smithi203 ceph-mon[32800]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:40.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:40 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:40.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:40 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:40.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:40 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:40.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:40 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:40.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:40 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:40.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:40 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:40 smithi083 ceph-mon[27867]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:40 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:40 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:40 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:40 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:40 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:40 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:40 smithi173 ceph-mon[32443]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:40 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:40 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:40 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:40 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:40 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:40 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:40.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.575+0000 7ff131800640 1 -- 172.21.15.83:0/2536284900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff12c062c00 con 0x7ff12c077c00 2024-04-23T14:45:40.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.576+0000 7ff128c00640 1 -- 172.21.15.83:0/2536284900 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7ff1180c0ea0 con 0x7ff12c077c00 2024-04-23T14:45:40.578 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:40.578 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T14:45:40.579 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-04-23T14:45:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.579+0000 7ff131800640 1 -- 172.21.15.83:0/2536284900 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1100a3c30 msgr2=0x7ff1100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.579+0000 7ff131800640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1100a3c30 0x7ff1100a60f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009bd0 tx=0x7ff11c005f90 comp rx=0 tx=0).stop 2024-04-23T14:45:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.579+0000 7ff131800640 1 -- 172.21.15.83:0/2536284900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff12c077c00 msgr2=0x7ff12c19ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.579+0000 7ff131800640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff12c077c00 0x7ff12c19ec90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff118031440 tx=0x7ff118032ce0 comp rx=0 tx=0).stop 2024-04-23T14:45:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.579+0000 7ff131800640 1 -- 172.21.15.83:0/2536284900 shutdown_connections 2024-04-23T14:45:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.579+0000 7ff131800640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1100a3c30 0x7ff1100a60f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.579+0000 7ff131800640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c0785a0 0x7ff12c1a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.579+0000 7ff131800640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff12c077c00 0x7ff12c19ec90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.579+0000 7ff131800640 1 --2- 172.21.15.83:0/2536284900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff12c0769c0 0x7ff12c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.579+0000 7ff131800640 1 -- 172.21.15.83:0/2536284900 >> 172.21.15.83:0/2536284900 conn(0x7ff12c0ff610 msgr2=0x7ff12c100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.579+0000 7ff131800640 1 -- 172.21.15.83:0/2536284900 shutdown_connections 2024-04-23T14:45:40.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:40.579+0000 7ff131800640 1 -- 172.21.15.83:0/2536284900 wait complete. 2024-04-23T14:45:41.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:41 smithi203 ceph-mon[32800]: Deploying daemon mds.i on smithi203 2024-04-23T14:45:41.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:41 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] up:boot 2024-04-23T14:45:41.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:41 smithi203 ceph-mon[32800]: fsmap 8 up:standby 2024-04-23T14:45:41.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:41 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T14:45:41.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:41 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2536284900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:41.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:41 smithi083 ceph-mon[27867]: Deploying daemon mds.i on smithi203 2024-04-23T14:45:41.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:41 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] up:boot 2024-04-23T14:45:41.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:41 smithi083 ceph-mon[27867]: fsmap 8 up:standby 2024-04-23T14:45:41.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:41 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T14:45:41.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:41 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2536284900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:41.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:41 smithi173 ceph-mon[32443]: Deploying daemon mds.i on smithi203 2024-04-23T14:45:41.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:41 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] up:boot 2024-04-23T14:45:41.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:41 smithi173 ceph-mon[32443]: fsmap 8 up:standby 2024-04-23T14:45:41.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:41 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T14:45:41.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:41 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2536284900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:42.378 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:42 smithi203 ceph-mon[32800]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:42.378 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:45:42 smithi203 systemd[1]: Starting Ceph mds.i for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:45:42.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:42 smithi083 ceph-mon[27867]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:42.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:42 smithi173 ceph-mon[32443]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:42.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:42.671 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:45:42 smithi203 podman[63554]: 2024-04-23 14:45:42.280304467 +0000 UTC m=+0.018407948 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T14:45:42.672 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:45:42 smithi203 podman[63554]: 2024-04-23 14:45:42.426314061 +0000 UTC m=+0.164417576 container create e1125a3808be299cbd7e5b7cb1ba48ecdeb41d797b1985838fe9965f581df481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2) 2024-04-23T14:45:42.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.810+0000 7ff881800640 1 -- 172.21.15.83:0/470352961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff87c109850 msgr2=0x7ff87c109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:42.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.810+0000 7ff881800640 1 --2- 172.21.15.83:0/470352961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff87c109850 0x7ff87c109cd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff864009900 tx=0x7ff86402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:45:42.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.811+0000 7ff881800640 1 -- 172.21.15.83:0/470352961 shutdown_connections 2024-04-23T14:45:42.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.811+0000 7ff881800640 1 --2- 172.21.15.83:0/470352961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff87c10a210 0x7ff87c110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:42.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.811+0000 7ff881800640 1 --2- 172.21.15.83:0/470352961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff87c109850 0x7ff87c109cd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:42.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.811+0000 7ff881800640 1 --2- 172.21.15.83:0/470352961 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff87c108660 0x7ff87c108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:42.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.811+0000 7ff881800640 1 -- 172.21.15.83:0/470352961 >> 172.21.15.83:0/470352961 conn(0x7ff87c077620 msgr2=0x7ff87c079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:42.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.811+0000 7ff881800640 1 -- 172.21.15.83:0/470352961 shutdown_connections 2024-04-23T14:45:42.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.811+0000 7ff881800640 1 -- 172.21.15.83:0/470352961 wait complete. 2024-04-23T14:45:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.812+0000 7ff881800640 1 Processor -- start 2024-04-23T14:45:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.812+0000 7ff881800640 1 -- start start 2024-04-23T14:45:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff881800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff87c108660 0x7ff87c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff881800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff87c109850 0x7ff87c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff881800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff87c10a210 0x7ff87c1a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff881800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c114330 con 0x7ff87c109850 2024-04-23T14:45:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff881800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c1141b0 con 0x7ff87c108660 2024-04-23T14:45:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff881800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c1144b0 con 0x7ff87c10a210 2024-04-23T14:45:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff87aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff87c108660 0x7ff87c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:42.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff87aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff87c108660 0x7ff87c19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33148/0 (socket says 172.21.15.83:33148) 2024-04-23T14:45:42.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff872000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff87c109850 0x7ff87c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:42.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff87aa00640 1 -- 172.21.15.83:0/1854923989 learned_addr learned my addr 172.21.15.83:0/1854923989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:42.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff87b400640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff87c10a210 0x7ff87c1a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:42.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff87aa00640 1 -- 172.21.15.83:0/1854923989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff87c10a210 msgr2=0x7ff87c1a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:42.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff87aa00640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff87c10a210 0x7ff87c1a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:42.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff87aa00640 1 -- 172.21.15.83:0/1854923989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff87c109850 msgr2=0x7ff87c19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:42.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff87aa00640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff87c109850 0x7ff87c19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:42.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.813+0000 7ff87aa00640 1 -- 172.21.15.83:0/1854923989 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c1a6590 con 0x7ff87c108660 2024-04-23T14:45:42.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.814+0000 7ff87aa00640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff87c108660 0x7ff87c19e970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00e9c0 tx=0x7ff86c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:42.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.814+0000 7ff873e00640 1 -- 172.21.15.83:0/1854923989 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c00cd80 con 0x7ff87c108660 2024-04-23T14:45:42.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.814+0000 7ff881800640 1 -- 172.21.15.83:0/1854923989 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87c1a6880 con 0x7ff87c108660 2024-04-23T14:45:42.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.814+0000 7ff873e00640 1 -- 172.21.15.83:0/1854923989 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86c0035f0 con 0x7ff87c108660 2024-04-23T14:45:42.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.814+0000 7ff873e00640 1 -- 172.21.15.83:0/1854923989 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c010690 con 0x7ff87c108660 2024-04-23T14:45:42.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.814+0000 7ff881800640 1 -- 172.21.15.83:0/1854923989 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff87c1a6d50 con 0x7ff87c108660 2024-04-23T14:45:42.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.815+0000 7ff881800640 1 -- 172.21.15.83:0/1854923989 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff87c10f920 con 0x7ff87c108660 2024-04-23T14:45:42.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.818+0000 7ff873e00640 1 -- 172.21.15.83:0/1854923989 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff86c0037a0 con 0x7ff87c108660 2024-04-23T14:45:42.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.820+0000 7ff873e00640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff8600a3bb0 0x7ff8600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:42.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.821+0000 7ff873e00640 1 -- 172.21.15.83:0/1854923989 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff86c0d0360 con 0x7ff87c108660 2024-04-23T14:45:42.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.821+0000 7ff872000640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff8600a3bb0 0x7ff8600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:42.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.821+0000 7ff872000640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff8600a3bb0 0x7ff8600a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff864031440 tx=0x7ff8640315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:42.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:42.824+0000 7ff873e00640 1 -- 172.21.15.83:0/1854923989 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff86c09d140 con 0x7ff87c108660 2024-04-23T14:45:42.971 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:45:42 smithi203 podman[63554]: 2024-04-23 14:45:42.671361428 +0000 UTC m=+0.409464894 container init e1125a3808be299cbd7e5b7cb1ba48ecdeb41d797b1985838fe9965f581df481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-23T14:45:42.972 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:45:42 smithi203 podman[63554]: 2024-04-23 14:45:42.674093759 +0000 UTC m=+0.412197226 container start e1125a3808be299cbd7e5b7cb1ba48ecdeb41d797b1985838fe9965f581df481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-23T14:45:42.972 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:45:42 smithi203 ceph-mds[63575]: -- 172.21.15.203:0/664312078 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eb3cf03a40 con 0x55eb3ced6800 2024-04-23T14:45:42.972 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:45:42 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-i[63571]: starting mds.i at 2024-04-23T14:45:42.972 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:45:42 smithi203 bash[63554]: e1125a3808be299cbd7e5b7cb1ba48ecdeb41d797b1985838fe9965f581df481 2024-04-23T14:45:42.972 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:45:42 smithi203 systemd[1]: Started Ceph mds.i for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:45:43.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.091+0000 7ff881800640 1 -- 172.21.15.83:0/1854923989 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff87c02d620 con 0x7ff87c108660 2024-04-23T14:45:43.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.092+0000 7ff873e00640 1 -- 172.21.15.83:0/1854923989 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7ff86c09ca40 con 0x7ff87c108660 2024-04-23T14:45:43.093 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:43.094 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T14:45:43.094 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 9 2024-04-23T14:45:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.096+0000 7ff881800640 1 -- 172.21.15.83:0/1854923989 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff8600a3bb0 msgr2=0x7ff8600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:43.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.096+0000 7ff881800640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff8600a3bb0 0x7ff8600a6070 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff864031440 tx=0x7ff8640315b0 comp rx=0 tx=0).stop 2024-04-23T14:45:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.096+0000 7ff881800640 1 -- 172.21.15.83:0/1854923989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff87c108660 msgr2=0x7ff87c19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.096+0000 7ff881800640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff87c108660 0x7ff87c19e970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00e9c0 tx=0x7ff86c00ee80 comp rx=0 tx=0).stop 2024-04-23T14:45:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.096+0000 7ff881800640 1 -- 172.21.15.83:0/1854923989 shutdown_connections 2024-04-23T14:45:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.096+0000 7ff881800640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff8600a3bb0 0x7ff8600a6070 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.096+0000 7ff881800640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff87c10a210 0x7ff87c1a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.096+0000 7ff881800640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff87c109850 0x7ff87c19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.096+0000 7ff881800640 1 --2- 172.21.15.83:0/1854923989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff87c108660 0x7ff87c19e970 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.096+0000 7ff881800640 1 -- 172.21.15.83:0/1854923989 >> 172.21.15.83:0/1854923989 conn(0x7ff87c077620 msgr2=0x7ff87c10caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.096+0000 7ff881800640 1 -- 172.21.15.83:0/1854923989 shutdown_connections 2024-04-23T14:45:43.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:43.096+0000 7ff881800640 1 -- 172.21.15.83:0/1854923989 wait complete. 2024-04-23T14:45:44.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:44.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:44.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:44.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:44.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:44.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:44.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:43 smithi173 ceph-mon[32443]: Deploying daemon mds.g on smithi083 2024-04-23T14:45:44.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:43 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1854923989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:44.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:43 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] up:boot 2024-04-23T14:45:44.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:43 smithi173 ceph-mon[32443]: fsmap 9 up:standby 2024-04-23T14:45:44.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:43 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T14:45:44.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:43 smithi173 ceph-mon[32443]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:44.132 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:44.132 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:44.132 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:44.132 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:44.132 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:44.133 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:44.133 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:43 smithi083 ceph-mon[27867]: Deploying daemon mds.g on smithi083 2024-04-23T14:45:44.133 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:43 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1854923989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:44.133 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:43 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] up:boot 2024-04-23T14:45:44.133 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:43 smithi083 ceph-mon[27867]: fsmap 9 up:standby 2024-04-23T14:45:44.133 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:43 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T14:45:44.133 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:43 smithi083 ceph-mon[27867]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:44.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:44.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:44.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:44.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:43 smithi203 ceph-mon[32800]: Deploying daemon mds.g on smithi083 2024-04-23T14:45:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:43 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1854923989' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:43 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] up:boot 2024-04-23T14:45:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:43 smithi203 ceph-mon[32800]: fsmap 9 up:standby 2024-04-23T14:45:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:43 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T14:45:44.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:43 smithi203 ceph-mon[32800]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:45.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:45.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.345+0000 7f339aa00640 1 -- 172.21.15.83:0/533226354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3394103240 msgr2=0x7f33941036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:45.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.345+0000 7f339aa00640 1 --2- 172.21.15.83:0/533226354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3394103240 0x7f33941036c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3380009900 tx=0x7f338002a3f0 comp rx=0 tx=0).stop 2024-04-23T14:45:45.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.350+0000 7f339aa00640 1 -- 172.21.15.83:0/533226354 shutdown_connections 2024-04-23T14:45:45.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.350+0000 7f339aa00640 1 --2- 172.21.15.83:0/533226354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3394103c00 0x7f339410a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:45.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.350+0000 7f339aa00640 1 --2- 172.21.15.83:0/533226354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3394103240 0x7f33941036c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:45.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.350+0000 7f339aa00640 1 --2- 172.21.15.83:0/533226354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3394102050 0x7f3394102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:45.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.350+0000 7f339aa00640 1 -- 172.21.15.83:0/533226354 >> 172.21.15.83:0/533226354 conn(0x7f33940fd600 msgr2=0x7f33940ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:45.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.351+0000 7f339aa00640 1 -- 172.21.15.83:0/533226354 shutdown_connections 2024-04-23T14:45:45.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.351+0000 7f339aa00640 1 -- 172.21.15.83:0/533226354 wait complete. 2024-04-23T14:45:45.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.353+0000 7f339aa00640 1 Processor -- start 2024-04-23T14:45:45.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.353+0000 7f339aa00640 1 -- start start 2024-04-23T14:45:45.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.354+0000 7f339aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3394102050 0x7f3394107e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:45.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.354+0000 7f339aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3394103240 0x7f3394108380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:45.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.354+0000 7f339aa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3394103c00 0x7f33941049b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:45.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.354+0000 7f339aa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f339410dd30 con 0x7f3394103240 2024-04-23T14:45:45.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.354+0000 7f339aa00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f339410dbb0 con 0x7f3394102050 2024-04-23T14:45:45.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.354+0000 7f339aa00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f339410deb0 con 0x7f3394103c00 2024-04-23T14:45:45.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.354+0000 7f3398a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3394103c00 0x7f33941049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:45.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.354+0000 7f3393400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3394103240 0x7f3394108380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:45.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.354+0000 7f3398a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3394103c00 0x7f33941049b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:43736/0 (socket says 172.21.15.83:43736) 2024-04-23T14:45:45.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.355+0000 7f3398a00640 1 -- 172.21.15.83:0/2708937941 learned_addr learned my addr 172.21.15.83:0/2708937941 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:45.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.355+0000 7f3393e00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3394102050 0x7f3394107e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:45.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.355+0000 7f3398a00640 1 -- 172.21.15.83:0/2708937941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3394102050 msgr2=0x7f3394107e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:45.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.355+0000 7f3398a00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3394102050 0x7f3394107e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:45.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.355+0000 7f3398a00640 1 -- 172.21.15.83:0/2708937941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3394103240 msgr2=0x7f3394108380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:45.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.355+0000 7f3398a00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3394103240 0x7f3394108380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:45.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.355+0000 7f3398a00640 1 -- 172.21.15.83:0/2708937941 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3394105270 con 0x7f3394103c00 2024-04-23T14:45:45.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.355+0000 7f3393e00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3394102050 0x7f3394107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:45:45.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.356+0000 7f3398a00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3394103c00 0x7f33941049b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f338c00eaa0 tx=0x7f338c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:45.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.356+0000 7f3390c00640 1 -- 172.21.15.83:0/2708937941 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c00cdb0 con 0x7f3394103c00 2024-04-23T14:45:45.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.356+0000 7f3390c00640 1 -- 172.21.15.83:0/2708937941 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f338c005020 con 0x7f3394103c00 2024-04-23T14:45:45.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.356+0000 7f3390c00640 1 -- 172.21.15.83:0/2708937941 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c0116d0 con 0x7f3394103c00 2024-04-23T14:45:45.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.356+0000 7f339aa00640 1 -- 172.21.15.83:0/2708937941 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339419a130 con 0x7f3394103c00 2024-04-23T14:45:45.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.357+0000 7f339aa00640 1 -- 172.21.15.83:0/2708937941 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339419a600 con 0x7f3394103c00 2024-04-23T14:45:45.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.358+0000 7f339aa00640 1 -- 172.21.15.83:0/2708937941 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3394069270 con 0x7f3394103c00 2024-04-23T14:45:45.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.362+0000 7f3390c00640 1 -- 172.21.15.83:0/2708937941 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f338c005540 con 0x7f3394103c00 2024-04-23T14:45:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.365+0000 7f3390c00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f33780a3c30 0x7f33780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:45.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.366+0000 7f3390c00640 1 -- 172.21.15.83:0/2708937941 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f338c0cfcd0 con 0x7f3394103c00 2024-04-23T14:45:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.367+0000 7f3390c00640 1 -- 172.21.15.83:0/2708937941 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f338c016030 con 0x7f3394103c00 2024-04-23T14:45:45.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.367+0000 7f3393e00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f33780a3c30 0x7f33780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:45.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.368+0000 7f3393e00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f33780a3c30 0x7f33780a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f33941030a0 tx=0x7f3384005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:45.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.615+0000 7f339aa00640 1 -- 172.21.15.83:0/2708937941 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f339419a8c0 con 0x7f3394103c00 2024-04-23T14:45:45.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.616+0000 7f3390c00640 1 -- 172.21.15.83:0/2708937941 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f338c09cbd0 con 0x7f3394103c00 2024-04-23T14:45:45.618 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:45.618 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T14:45:45.618 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 10 2024-04-23T14:45:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.618+0000 7f339aa00640 1 -- 172.21.15.83:0/2708937941 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f33780a3c30 msgr2=0x7f33780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.618+0000 7f339aa00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f33780a3c30 0x7f33780a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f33941030a0 tx=0x7f3384005f90 comp rx=0 tx=0).stop 2024-04-23T14:45:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.618+0000 7f339aa00640 1 -- 172.21.15.83:0/2708937941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3394103c00 msgr2=0x7f33941049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.618+0000 7f339aa00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3394103c00 0x7f33941049b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f338c00eaa0 tx=0x7f338c00ef60 comp rx=0 tx=0).stop 2024-04-23T14:45:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.619+0000 7f339aa00640 1 -- 172.21.15.83:0/2708937941 shutdown_connections 2024-04-23T14:45:45.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.619+0000 7f339aa00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f33780a3c30 0x7f33780a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.619+0000 7f339aa00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3394103c00 0x7f33941049b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.619+0000 7f339aa00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3394103240 0x7f3394108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.619+0000 7f339aa00640 1 --2- 172.21.15.83:0/2708937941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3394102050 0x7f3394107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.619+0000 7f339aa00640 1 -- 172.21.15.83:0/2708937941 >> 172.21.15.83:0/2708937941 conn(0x7f33940fd600 msgr2=0x7f3394074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.619+0000 7f339aa00640 1 -- 172.21.15.83:0/2708937941 shutdown_connections 2024-04-23T14:45:45.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:45.619+0000 7f339aa00640 1 -- 172.21.15.83:0/2708937941 wait complete. 2024-04-23T14:45:46.256 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 14:45:45 smithi083 systemd[1]: Starting Ceph mds.g for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:45:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:46 smithi083 ceph-mon[27867]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:46.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:46 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2708937941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:46.507 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 14:45:46 smithi083 podman[68370]: 2024-04-23 14:45:46.199882444 +0000 UTC m=+0.035130994 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T14:45:46.507 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 14:45:46 smithi083 podman[68370]: 2024-04-23 14:45:46.3290278 +0000 UTC m=+0.164276377 container create 8c789337abcfe3b8ada6b0e3015b42a228fd936b06244e71b4bb367ee8eb0cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T14:45:46.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:46 smithi173 ceph-mon[32443]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:46.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:46 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2708937941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:46.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:46 smithi203 ceph-mon[32800]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:46.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:46 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2708937941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:46.862 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 14:45:46 smithi083 podman[68370]: 2024-04-23 14:45:46.608736699 +0000 UTC m=+0.443985322 container init 8c789337abcfe3b8ada6b0e3015b42a228fd936b06244e71b4bb367ee8eb0cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T14:45:46.862 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 14:45:46 smithi083 podman[68370]: 2024-04-23 14:45:46.6187658 +0000 UTC m=+0.454014345 container start 8c789337abcfe3b8ada6b0e3015b42a228fd936b06244e71b4bb367ee8eb0cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T14:45:46.862 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 14:45:46 smithi083 ceph-mds[68390]: -- 172.21.15.83:0/2923929338 <== mon.0 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d1c9091a40 con 0x55d1c9064c00 2024-04-23T14:45:46.862 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 14:45:46 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-g[68385]: starting mds.g at 2024-04-23T14:45:46.862 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 14:45:46 smithi083 bash[68370]: 8c789337abcfe3b8ada6b0e3015b42a228fd936b06244e71b4bb367ee8eb0cdc 2024-04-23T14:45:46.863 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 14:45:46 smithi083 systemd[1]: Started Ceph mds.g for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:45:47.646 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:47.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.821+0000 7f10a3e00640 1 -- 172.21.15.83:0/3742813242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10a4101ea0 msgr2=0x7f10a41022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:47.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.821+0000 7f10a3e00640 1 --2- 172.21.15.83:0/3742813242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10a4101ea0 0x7f10a41022c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1094009900 tx=0x7f109402a410 comp rx=0 tx=0).stop 2024-04-23T14:45:47.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.822+0000 7f10a3e00640 1 -- 172.21.15.83:0/3742813242 shutdown_connections 2024-04-23T14:45:47.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.822+0000 7f10a3e00640 1 --2- 172.21.15.83:0/3742813242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a41039b0 0x7f10a410a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:47.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.822+0000 7f10a3e00640 1 --2- 172.21.15.83:0/3742813242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10a4102ff0 0x7f10a4103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:47.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.822+0000 7f10a3e00640 1 --2- 172.21.15.83:0/3742813242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10a4101ea0 0x7f10a41022c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:47.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.822+0000 7f10a3e00640 1 -- 172.21.15.83:0/3742813242 >> 172.21.15.83:0/3742813242 conn(0x7f10a40fd410 msgr2=0x7f10a40ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:47.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.822+0000 7f10a3e00640 1 -- 172.21.15.83:0/3742813242 shutdown_connections 2024-04-23T14:45:47.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.822+0000 7f10a3e00640 1 -- 172.21.15.83:0/3742813242 wait complete. 2024-04-23T14:45:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.823+0000 7f10a3e00640 1 Processor -- start 2024-04-23T14:45:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.823+0000 7f10a3e00640 1 -- start start 2024-04-23T14:45:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.823+0000 7f10a3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10a4101ea0 0x7f10a419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.823+0000 7f10a3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10a4102ff0 0x7f10a419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.823+0000 7f10a3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a41039b0 0x7f10a41a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.823+0000 7f10a3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f10a410db70 con 0x7f10a4101ea0 2024-04-23T14:45:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.823+0000 7f10a3e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f10a410d9f0 con 0x7f10a4102ff0 2024-04-23T14:45:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.823+0000 7f10a3e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f10a410dcf0 con 0x7f10a41039b0 2024-04-23T14:45:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.823+0000 7f10a2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10a4101ea0 0x7f10a419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:47.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.823+0000 7f10a2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10a4101ea0 0x7f10a419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52692/0 (socket says 172.21.15.83:52692) 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.823+0000 7f10a2a00640 1 -- 172.21.15.83:0/2942977965 learned_addr learned my addr 172.21.15.83:0/2942977965 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.823+0000 7f10a2000640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10a4102ff0 0x7f10a419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.824+0000 7f10a3400640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a41039b0 0x7f10a41a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.824+0000 7f10a2a00640 1 -- 172.21.15.83:0/2942977965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a41039b0 msgr2=0x7f10a41a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.824+0000 7f10a2a00640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a41039b0 0x7f10a41a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.824+0000 7f10a2a00640 1 -- 172.21.15.83:0/2942977965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10a4102ff0 msgr2=0x7f10a419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.824+0000 7f10a2a00640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10a4102ff0 0x7f10a419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.824+0000 7f10a2a00640 1 -- 172.21.15.83:0/2942977965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a41a1fc0 con 0x7f10a4101ea0 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.824+0000 7f10a3400640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a41039b0 0x7f10a41a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.824+0000 7f10a2a00640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10a4101ea0 0x7f10a419a3a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1094031440 tx=0x7f1094032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.824+0000 7f109b400640 1 -- 172.21.15.83:0/2942977965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109403f030 con 0x7f10a4101ea0 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.824+0000 7f109b400640 1 -- 172.21.15.83:0/2942977965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1094003ce0 con 0x7f10a4101ea0 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.824+0000 7f10a3e00640 1 -- 172.21.15.83:0/2942977965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a41a2250 con 0x7f10a4101ea0 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.824+0000 7f109b400640 1 -- 172.21.15.83:0/2942977965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094035070 con 0x7f10a4101ea0 2024-04-23T14:45:47.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.825+0000 7f10a3e00640 1 -- 172.21.15.83:0/2942977965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a41a2740 con 0x7f10a4101ea0 2024-04-23T14:45:47.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.826+0000 7f10a3e00640 1 -- 172.21.15.83:0/2942977965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a41a29a0 con 0x7f10a4101ea0 2024-04-23T14:45:47.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.827+0000 7f109b400640 1 -- 172.21.15.83:0/2942977965 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1094042030 con 0x7f10a4101ea0 2024-04-23T14:45:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.829+0000 7f109b400640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f10880a3bb0 0x7f10880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.829+0000 7f109b400640 1 -- 172.21.15.83:0/2942977965 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f10940f0680 con 0x7f10a4101ea0 2024-04-23T14:45:47.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.829+0000 7f10a2000640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f10880a3bb0 0x7f10880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:47.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.830+0000 7f10a2000640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f10880a3bb0 0x7f10880a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f10a419b8b0 tx=0x7f109000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:47.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:47.834+0000 7f109b400640 1 -- 172.21.15.83:0/2942977965 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10940bd460 con 0x7f10a4101ea0 2024-04-23T14:45:48.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:47 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:47 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:47 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:47 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:47 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:47 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:47 smithi083 ceph-mon[27867]: Deploying daemon mds.a on smithi083 2024-04-23T14:45:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:47 smithi083 ceph-mon[27867]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:47 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.83:6838/436672914,v1:172.21.15.83:6839/436672914] up:boot 2024-04-23T14:45:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:47 smithi083 ceph-mon[27867]: fsmap 10 up:standby 2024-04-23T14:45:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:47 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T14:45:48.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.077+0000 7f10a3e00640 1 -- 172.21.15.83:0/2942977965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10a41a2b70 con 0x7f10a4101ea0 2024-04-23T14:45:48.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.078+0000 7f109b400640 1 -- 172.21.15.83:0/2942977965 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7f10940bcd60 con 0x7f10a4101ea0 2024-04-23T14:45:48.079 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:48.079 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T14:45:48.079 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 11 2024-04-23T14:45:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.081+0000 7f10a3e00640 1 -- 172.21.15.83:0/2942977965 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f10880a3bb0 msgr2=0x7f10880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.081+0000 7f10a3e00640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f10880a3bb0 0x7f10880a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f10a419b8b0 tx=0x7f109000b410 comp rx=0 tx=0).stop 2024-04-23T14:45:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.081+0000 7f10a3e00640 1 -- 172.21.15.83:0/2942977965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10a4101ea0 msgr2=0x7f10a419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.081+0000 7f10a3e00640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10a4101ea0 0x7f10a419a3a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1094031440 tx=0x7f1094032e20 comp rx=0 tx=0).stop 2024-04-23T14:45:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.081+0000 7f10a3e00640 1 -- 172.21.15.83:0/2942977965 shutdown_connections 2024-04-23T14:45:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.081+0000 7f10a3e00640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f10880a3bb0 0x7f10880a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.081+0000 7f10a3e00640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10a41039b0 0x7f10a41a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.081+0000 7f10a3e00640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10a4102ff0 0x7f10a419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:48.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.081+0000 7f10a3e00640 1 --2- 172.21.15.83:0/2942977965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10a4101ea0 0x7f10a419a3a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.081+0000 7f10a3e00640 1 -- 172.21.15.83:0/2942977965 >> 172.21.15.83:0/2942977965 conn(0x7f10a40fd410 msgr2=0x7f10a4074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.082+0000 7f10a3e00640 1 -- 172.21.15.83:0/2942977965 shutdown_connections 2024-04-23T14:45:48.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:48.082+0000 7f10a3e00640 1 -- 172.21.15.83:0/2942977965 wait complete. 2024-04-23T14:45:48.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:47 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:48.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:47 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:48.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:47 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:48.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:47 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:48.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:47 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:48.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:47 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:48.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:47 smithi203 ceph-mon[32800]: Deploying daemon mds.a on smithi083 2024-04-23T14:45:48.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:47 smithi203 ceph-mon[32800]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:48.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:47 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.83:6838/436672914,v1:172.21.15.83:6839/436672914] up:boot 2024-04-23T14:45:48.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:47 smithi203 ceph-mon[32800]: fsmap 10 up:standby 2024-04-23T14:45:48.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:47 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T14:45:48.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:47 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:48.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:47 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:48.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:47 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:48.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:47 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:48.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:47 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:48.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:47 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:48.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:47 smithi173 ceph-mon[32443]: Deploying daemon mds.a on smithi083 2024-04-23T14:45:48.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:47 smithi173 ceph-mon[32443]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:48.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:47 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.83:6838/436672914,v1:172.21.15.83:6839/436672914] up:boot 2024-04-23T14:45:48.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:47 smithi173 ceph-mon[32443]: fsmap 10 up:standby 2024-04-23T14:45:48.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:47 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T14:45:49.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:48 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2942977965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:49.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:48 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2942977965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:49.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:48 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2942977965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:50.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:49 smithi083 ceph-mon[27867]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:50.104 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:50.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:49 smithi203 ceph-mon[32800]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.289+0000 7fbbb0200640 1 -- 172.21.15.83:0/1296287354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba8109ee0 msgr2=0x7fbba810c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.289+0000 7fbbb0200640 1 --2- 172.21.15.83:0/1296287354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba8109ee0 0x7fbba810c380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009ca0 tx=0x7fbba002a950 comp rx=0 tx=0).stop 2024-04-23T14:45:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.290+0000 7fbbb0200640 1 -- 172.21.15.83:0/1296287354 shutdown_connections 2024-04-23T14:45:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.290+0000 7fbbb0200640 1 --2- 172.21.15.83:0/1296287354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba8109ee0 0x7fbba810c380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.290+0000 7fbbb0200640 1 --2- 172.21.15.83:0/1296287354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba8074a00 0x7fbba8074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.290+0000 7fbbb0200640 1 --2- 172.21.15.83:0/1296287354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbba8075fe0 0x7fbba8074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.290+0000 7fbbb0200640 1 -- 172.21.15.83:0/1296287354 >> 172.21.15.83:0/1296287354 conn(0x7fbba80fd5e0 msgr2=0x7fbba80ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.290+0000 7fbbb0200640 1 -- 172.21.15.83:0/1296287354 shutdown_connections 2024-04-23T14:45:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.290+0000 7fbbb0200640 1 -- 172.21.15.83:0/1296287354 wait complete. 2024-04-23T14:45:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.291+0000 7fbbb0200640 1 Processor -- start 2024-04-23T14:45:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.291+0000 7fbbb0200640 1 -- start start 2024-04-23T14:45:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.291+0000 7fbbb0200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba8074a00 0x7fbba819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.291+0000 7fbbb0200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba8075fe0 0x7fbba819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.291+0000 7fbbb0200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbba8109ee0 0x7fbba81a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.291+0000 7fbbb0200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbba810ff30 con 0x7fbba8074a00 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.291+0000 7fbbb0200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbba810fdb0 con 0x7fbba8075fe0 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.291+0000 7fbbb0200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81100b0 con 0x7fbba8109ee0 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.291+0000 7fbbad800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba8074a00 0x7fbba819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.291+0000 7fbbad800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba8074a00 0x7fbba819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52702/0 (socket says 172.21.15.83:52702) 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.291+0000 7fbbad800640 1 -- 172.21.15.83:0/569741276 learned_addr learned my addr 172.21.15.83:0/569741276 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.292+0000 7fbbace00640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba8075fe0 0x7fbba819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.292+0000 7fbbae200640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbba8109ee0 0x7fbba81a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.292+0000 7fbbad800640 1 -- 172.21.15.83:0/569741276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbba8109ee0 msgr2=0x7fbba81a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.292+0000 7fbbad800640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbba8109ee0 0x7fbba81a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.292+0000 7fbbad800640 1 -- 172.21.15.83:0/569741276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba8075fe0 msgr2=0x7fbba819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.292+0000 7fbbad800640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba8075fe0 0x7fbba819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.292+0000 7fbbad800640 1 -- 172.21.15.83:0/569741276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba81a2120 con 0x7fbba8074a00 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.292+0000 7fbbace00640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba8075fe0 0x7fbba819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:50.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.292+0000 7fbbad800640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba8074a00 0x7fbba819a500 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00e9f0 tx=0x7fbb9c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.292+0000 7fbba6000640 1 -- 172.21.15.83:0/569741276 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c00cd80 con 0x7fbba8074a00 2024-04-23T14:45:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.292+0000 7fbba6000640 1 -- 172.21.15.83:0/569741276 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb9c005500 con 0x7fbba8074a00 2024-04-23T14:45:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.292+0000 7fbba6000640 1 -- 172.21.15.83:0/569741276 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c010690 con 0x7fbba8074a00 2024-04-23T14:45:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.293+0000 7fbbb0200640 1 -- 172.21.15.83:0/569741276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba81a2410 con 0x7fbba8074a00 2024-04-23T14:45:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.293+0000 7fbbb0200640 1 -- 172.21.15.83:0/569741276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba81a2910 con 0x7fbba8074a00 2024-04-23T14:45:50.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.294+0000 7fbbb0200640 1 -- 172.21.15.83:0/569741276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba8108ca0 con 0x7fbba8074a00 2024-04-23T14:45:50.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.296+0000 7fbba6000640 1 -- 172.21.15.83:0/569741276 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbb9c0046c0 con 0x7fbba8074a00 2024-04-23T14:45:50.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.298+0000 7fbba6000640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb900a3bb0 0x7fbb900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:50.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.298+0000 7fbba6000640 1 -- 172.21.15.83:0/569741276 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbb9c0d0400 con 0x7fbba8074a00 2024-04-23T14:45:50.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.298+0000 7fbbace00640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb900a3bb0 0x7fbb900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:50.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.299+0000 7fbbace00640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb900a3bb0 0x7fbb900a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbba819ba10 tx=0x7fbb94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:50.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.302+0000 7fbba6000640 1 -- 172.21.15.83:0/569741276 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb9c09d1e0 con 0x7fbba8074a00 2024-04-23T14:45:50.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:49 smithi173 ceph-mon[32443]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:50.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.552+0000 7fbbb0200640 1 -- 172.21.15.83:0/569741276 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbba802d620 con 0x7fbba8074a00 2024-04-23T14:45:50.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.554+0000 7fbba6000640 1 -- 172.21.15.83:0/569741276 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7fbb9c09cae0 con 0x7fbba8074a00 2024-04-23T14:45:50.556 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:50.556 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T14:45:50.556 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 11 2024-04-23T14:45:50.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.557+0000 7fbbb0200640 1 -- 172.21.15.83:0/569741276 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb900a3bb0 msgr2=0x7fbb900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:50.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.557+0000 7fbbb0200640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb900a3bb0 0x7fbb900a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbba819ba10 tx=0x7fbb94005f90 comp rx=0 tx=0).stop 2024-04-23T14:45:50.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.557+0000 7fbbb0200640 1 -- 172.21.15.83:0/569741276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba8074a00 msgr2=0x7fbba819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:50.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.557+0000 7fbbb0200640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba8074a00 0x7fbba819a500 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00e9f0 tx=0x7fbb9c00edf0 comp rx=0 tx=0).stop 2024-04-23T14:45:50.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.557+0000 7fbbb0200640 1 -- 172.21.15.83:0/569741276 shutdown_connections 2024-04-23T14:45:50.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.557+0000 7fbbb0200640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb900a3bb0 0x7fbb900a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:50.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.557+0000 7fbbb0200640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbba8109ee0 0x7fbba81a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.557+0000 7fbbb0200640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbba8075fe0 0x7fbba819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.557+0000 7fbbb0200640 1 --2- 172.21.15.83:0/569741276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbba8074a00 0x7fbba819a500 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.557+0000 7fbbb0200640 1 -- 172.21.15.83:0/569741276 >> 172.21.15.83:0/569741276 conn(0x7fbba80fd5e0 msgr2=0x7fbba8105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.557+0000 7fbbb0200640 1 -- 172.21.15.83:0/569741276 shutdown_connections 2024-04-23T14:45:50.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:50.557+0000 7fbbb0200640 1 -- 172.21.15.83:0/569741276 wait complete. 2024-04-23T14:45:51.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:50 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/569741276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:51.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:50 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/569741276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:51.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:50 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/569741276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:51.983 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:51 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.83:6840/238596160,v1:172.21.15.83:6841/238596160] up:boot 2024-04-23T14:45:51.983 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:51 smithi173 ceph-mon[32443]: fsmap 11 up:standby 2024-04-23T14:45:51.983 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:51 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T14:45:51.983 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:51 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:51.983 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:51 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:51.983 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:51 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:51.984 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:51 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:51.984 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:51 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:51.984 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:51 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:51.984 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:51 smithi173 ceph-mon[32443]: Deploying daemon mds.b on smithi173 2024-04-23T14:45:51.984 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:51 smithi173 ceph-mon[32443]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:52.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:51 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.83:6840/238596160,v1:172.21.15.83:6841/238596160] up:boot 2024-04-23T14:45:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:51 smithi203 ceph-mon[32800]: fsmap 11 up:standby 2024-04-23T14:45:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T14:45:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:51 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:51 smithi203 ceph-mon[32800]: Deploying daemon mds.b on smithi173 2024-04-23T14:45:52.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:51 smithi203 ceph-mon[32800]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:52.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:51 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.83:6840/238596160,v1:172.21.15.83:6841/238596160] up:boot 2024-04-23T14:45:52.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:51 smithi083 ceph-mon[27867]: fsmap 11 up:standby 2024-04-23T14:45:52.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:51 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T14:45:52.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:51 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:52.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:51 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:52.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:51 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:52.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:51 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T14:45:52.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:51 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T14:45:52.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:51 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:52.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:51 smithi083 ceph-mon[27867]: Deploying daemon mds.b on smithi173 2024-04-23T14:45:52.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:51 smithi083 ceph-mon[27867]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:52.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.787+0000 7fbb66c00640 1 -- 172.21.15.83:0/1219454263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb60108660 msgr2=0x7fbb60108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.787+0000 7fbb66c00640 1 --2- 172.21.15.83:0/1219454263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb60108660 0x7fbb60108a80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009900 tx=0x7fbb5002a410 comp rx=0 tx=0).stop 2024-04-23T14:45:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.787+0000 7fbb66c00640 1 -- 172.21.15.83:0/1219454263 shutdown_connections 2024-04-23T14:45:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.787+0000 7fbb66c00640 1 --2- 172.21.15.83:0/1219454263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb6010a210 0x7fbb60110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.787+0000 7fbb66c00640 1 --2- 172.21.15.83:0/1219454263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb60109850 0x7fbb60109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.787+0000 7fbb66c00640 1 --2- 172.21.15.83:0/1219454263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb60108660 0x7fbb60108a80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.787+0000 7fbb66c00640 1 -- 172.21.15.83:0/1219454263 >> 172.21.15.83:0/1219454263 conn(0x7fbb60077620 msgr2=0x7fbb60079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.787+0000 7fbb66c00640 1 -- 172.21.15.83:0/1219454263 shutdown_connections 2024-04-23T14:45:52.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.787+0000 7fbb66c00640 1 -- 172.21.15.83:0/1219454263 wait complete. 2024-04-23T14:45:52.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.788+0000 7fbb66c00640 1 Processor -- start 2024-04-23T14:45:52.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.788+0000 7fbb66c00640 1 -- start start 2024-04-23T14:45:52.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb66c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb60108660 0x7fbb6019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb66c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb60109850 0x7fbb6019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb66c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb6010a210 0x7fbb601a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb66c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60114330 con 0x7fbb6010a210 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb66c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601141b0 con 0x7fbb60108660 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb66c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601144b0 con 0x7fbb60109850 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb64c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb6010a210 0x7fbb601a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb64c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb6010a210 0x7fbb601a5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52748/0 (socket says 172.21.15.83:52748) 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb64c00640 1 -- 172.21.15.83:0/1160965031 learned_addr learned my addr 172.21.15.83:0/1160965031 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb5f400640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb60109850 0x7fbb6019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb5fe00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb60108660 0x7fbb6019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb64c00640 1 -- 172.21.15.83:0/1160965031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb60109850 msgr2=0x7fbb6019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb64c00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb60109850 0x7fbb6019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb64c00640 1 -- 172.21.15.83:0/1160965031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb60108660 msgr2=0x7fbb6019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb64c00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb60108660 0x7fbb6019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:52.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.789+0000 7fbb64c00640 1 -- 172.21.15.83:0/1160965031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb601a6590 con 0x7fbb6010a210 2024-04-23T14:45:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.790+0000 7fbb64c00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb6010a210 0x7fbb601a5f20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800eb10 tx=0x7fbb5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.790+0000 7fbb5cc00640 1 -- 172.21.15.83:0/1160965031 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58004b20 con 0x7fbb6010a210 2024-04-23T14:45:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.790+0000 7fbb5cc00640 1 -- 172.21.15.83:0/1160965031 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb58004cc0 con 0x7fbb6010a210 2024-04-23T14:45:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.790+0000 7fbb66c00640 1 -- 172.21.15.83:0/1160965031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb601a6880 con 0x7fbb6010a210 2024-04-23T14:45:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.790+0000 7fbb5cc00640 1 -- 172.21.15.83:0/1160965031 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58010760 con 0x7fbb6010a210 2024-04-23T14:45:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.790+0000 7fbb66c00640 1 -- 172.21.15.83:0/1160965031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb601a6e20 con 0x7fbb6010a210 2024-04-23T14:45:52.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.791+0000 7fbb66c00640 1 -- 172.21.15.83:0/1160965031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb6010f980 con 0x7fbb6010a210 2024-04-23T14:45:52.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.793+0000 7fbb5cc00640 1 -- 172.21.15.83:0/1160965031 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbb58005020 con 0x7fbb6010a210 2024-04-23T14:45:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.796+0000 7fbb5cc00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb440a3c30 0x7fbb440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.796+0000 7fbb5cc00640 1 -- 172.21.15.83:0/1160965031 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbb580d06c0 con 0x7fbb6010a210 2024-04-23T14:45:52.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.796+0000 7fbb5fe00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb440a3c30 0x7fbb440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:52.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.797+0000 7fbb5fe00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb440a3c30 0x7fbb440a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbb5002aa00 tx=0x7fbb50031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:52.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:52.801+0000 7fbb5cc00640 1 -- 172.21.15.83:0/1160965031 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb5809d4a0 con 0x7fbb6010a210 2024-04-23T14:45:53.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.043+0000 7fbb66c00640 1 -- 172.21.15.83:0/1160965031 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb6002d620 con 0x7fbb6010a210 2024-04-23T14:45:53.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.044+0000 7fbb5cc00640 1 -- 172.21.15.83:0/1160965031 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9647 (secure 0 0 0) 0x7fbb5809cda0 con 0x7fbb6010a210 2024-04-23T14:45:53.045 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:53.045 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T14:45:53.046 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 12 2024-04-23T14:45:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.047+0000 7fbb66c00640 1 -- 172.21.15.83:0/1160965031 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb440a3c30 msgr2=0x7fbb440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.047+0000 7fbb66c00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb440a3c30 0x7fbb440a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbb5002aa00 tx=0x7fbb50031400 comp rx=0 tx=0).stop 2024-04-23T14:45:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.047+0000 7fbb66c00640 1 -- 172.21.15.83:0/1160965031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb6010a210 msgr2=0x7fbb601a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:53.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.047+0000 7fbb66c00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb6010a210 0x7fbb601a5f20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800eb10 tx=0x7fbb5800c2d0 comp rx=0 tx=0).stop 2024-04-23T14:45:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.047+0000 7fbb66c00640 1 -- 172.21.15.83:0/1160965031 shutdown_connections 2024-04-23T14:45:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.047+0000 7fbb66c00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbb440a3c30 0x7fbb440a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.047+0000 7fbb66c00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb6010a210 0x7fbb601a5f20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.047+0000 7fbb66c00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb60109850 0x7fbb6019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.047+0000 7fbb66c00640 1 --2- 172.21.15.83:0/1160965031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb60108660 0x7fbb6019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.047+0000 7fbb66c00640 1 -- 172.21.15.83:0/1160965031 >> 172.21.15.83:0/1160965031 conn(0x7fbb60077620 msgr2=0x7fbb6010cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.047+0000 7fbb66c00640 1 -- 172.21.15.83:0/1160965031 shutdown_connections 2024-04-23T14:45:53.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:53.047+0000 7fbb66c00640 1 -- 172.21.15.83:0/1160965031 wait complete. 2024-04-23T14:45:53.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:53 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1160965031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:53.326 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:45:53 smithi173 systemd[1]: Starting Ceph mds.b for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T14:45:53.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:53 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1160965031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:53.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:53 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1160965031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:53.676 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:45:53 smithi173 podman[63108]: 2024-04-23 14:45:53.308639097 +0000 UTC m=+0.018291131 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T14:45:53.676 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:45:53 smithi173 podman[63108]: 2024-04-23 14:45:53.423022546 +0000 UTC m=+0.132674570 container create e01408b8ab43d9300d4385bb5d32f071bfb3db63512cd8e6cb73151898726338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T14:45:53.676 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:45:53 smithi173 podman[63108]: 2024-04-23 14:45:53.675750342 +0000 UTC m=+0.385402367 container init e01408b8ab43d9300d4385bb5d32f071bfb3db63512cd8e6cb73151898726338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-23T14:45:54.076 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:45:53 smithi173 podman[63108]: 2024-04-23 14:45:53.678444585 +0000 UTC m=+0.388096614 container start e01408b8ab43d9300d4385bb5d32f071bfb3db63512cd8e6cb73151898726338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-23T14:45:54.076 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:45:53 smithi173 ceph-mds[63127]: -- 172.21.15.173:0/3683275992 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f92d50da40 con 0x55f92d4e0800 2024-04-23T14:45:54.076 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:45:53 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b[63123]: starting mds.b at 2024-04-23T14:45:54.076 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:45:53 smithi173 bash[63108]: e01408b8ab43d9300d4385bb5d32f071bfb3db63512cd8e6cb73151898726338 2024-04-23T14:45:54.076 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:45:53 smithi173 systemd[1]: Started Ceph mds.b for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T14:45:54.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:54 smithi203 ceph-mon[32800]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:54.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:54 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:54 smithi083 ceph-mon[27867]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:54.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:54 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:54 smithi173 ceph-mon[32443]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:54.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:54 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:55.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:45:55.111 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:55 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] up:boot 2024-04-23T14:45:55.111 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:55 smithi203 ceph-mon[32800]: fsmap 12 up:standby 2024-04-23T14:45:55.111 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:55 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T14:45:55.111 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:55 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:45:55.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.264+0000 7f75ecc00640 1 -- 172.21.15.83:0/1869156374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e8108480 msgr2=0x7f75e81088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:55.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.264+0000 7f75ecc00640 1 --2- 172.21.15.83:0/1869156374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e8108480 0x7f75e81088a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009900 tx=0x7f75d802a410 comp rx=0 tx=0).stop 2024-04-23T14:45:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.264+0000 7f75ecc00640 1 -- 172.21.15.83:0/1869156374 shutdown_connections 2024-04-23T14:45:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.264+0000 7f75ecc00640 1 --2- 172.21.15.83:0/1869156374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75e810a030 0x7f75e81108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.264+0000 7f75ecc00640 1 --2- 172.21.15.83:0/1869156374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75e8109670 0x7f75e8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.264+0000 7f75ecc00640 1 --2- 172.21.15.83:0/1869156374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e8108480 0x7f75e81088a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.264+0000 7f75ecc00640 1 -- 172.21.15.83:0/1869156374 >> 172.21.15.83:0/1869156374 conn(0x7f75e8077440 msgr2=0x7f75e8079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.264+0000 7f75ecc00640 1 -- 172.21.15.83:0/1869156374 shutdown_connections 2024-04-23T14:45:55.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.264+0000 7f75ecc00640 1 -- 172.21.15.83:0/1869156374 wait complete. 2024-04-23T14:45:55.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.265+0000 7f75ecc00640 1 Processor -- start 2024-04-23T14:45:55.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.265+0000 7f75ecc00640 1 -- start start 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.265+0000 7f75ecc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e8108480 0x7f75e819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.265+0000 7f75ecc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75e8109670 0x7f75e819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.265+0000 7f75ecc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75e810a030 0x7f75e81a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.265+0000 7f75ecc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75e8114160 con 0x7f75e8109670 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.265+0000 7f75ecc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75e8113fe0 con 0x7f75e810a030 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.265+0000 7f75ecc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75e81142e0 con 0x7f75e8108480 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.265+0000 7f75e6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75e8109670 0x7f75e819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.265+0000 7f75e7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75e810a030 0x7f75e81a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.266+0000 7f75e7400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e8108480 0x7f75e819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.266+0000 7f75e7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75e810a030 0x7f75e81a5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:50134/0 (socket says 172.21.15.83:50134) 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.266+0000 7f75e7e00640 1 -- 172.21.15.83:0/1927391249 learned_addr learned my addr 172.21.15.83:0/1927391249 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.266+0000 7f75e7e00640 1 -- 172.21.15.83:0/1927391249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e8108480 msgr2=0x7f75e819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.266+0000 7f75e7e00640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e8108480 0x7f75e819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.266+0000 7f75e7e00640 1 -- 172.21.15.83:0/1927391249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75e8109670 msgr2=0x7f75e819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:55.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.266+0000 7f75e7e00640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75e8109670 0x7f75e819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.266+0000 7f75e7e00640 1 -- 172.21.15.83:0/1927391249 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e81a63b0 con 0x7f75e810a030 2024-04-23T14:45:55.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.266+0000 7f75e7400640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e8108480 0x7f75e819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:55.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.266+0000 7f75e6a00640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75e8109670 0x7f75e819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:55.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.266+0000 7f75e7e00640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75e810a030 0x7f75e81a5d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75e000eb10 tx=0x7f75e000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:55.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.266+0000 7f75dfe00640 1 -- 172.21.15.83:0/1927391249 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0004b20 con 0x7f75e810a030 2024-04-23T14:45:55.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.267+0000 7f75ecc00640 1 -- 172.21.15.83:0/1927391249 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e81a66a0 con 0x7f75e810a030 2024-04-23T14:45:55.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.267+0000 7f75dfe00640 1 -- 172.21.15.83:0/1927391249 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e0004cc0 con 0x7f75e810a030 2024-04-23T14:45:55.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.267+0000 7f75dfe00640 1 -- 172.21.15.83:0/1927391249 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0010760 con 0x7f75e810a030 2024-04-23T14:45:55.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.267+0000 7f75ecc00640 1 -- 172.21.15.83:0/1927391249 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e81a6ba0 con 0x7f75e810a030 2024-04-23T14:45:55.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.268+0000 7f75ecc00640 1 -- 172.21.15.83:0/1927391249 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e810f7a0 con 0x7f75e810a030 2024-04-23T14:45:55.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.270+0000 7f75dfe00640 1 -- 172.21.15.83:0/1927391249 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f75e0005020 con 0x7f75e810a030 2024-04-23T14:45:55.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.271+0000 7f75dfe00640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f75cc0a3bb0 0x7f75cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:55.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.271+0000 7f75dfe00640 1 -- 172.21.15.83:0/1927391249 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f75e00d0570 con 0x7f75e810a030 2024-04-23T14:45:55.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.271+0000 7f75e7400640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f75cc0a3bb0 0x7f75cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:55.274 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:55 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] up:boot 2024-04-23T14:45:55.274 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:55 smithi173 ceph-mon[32443]: fsmap 12 up:standby 2024-04-23T14:45:55.274 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:55 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T14:45:55.274 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:55 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:45:55.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.272+0000 7f75e7400640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f75cc0a3bb0 0x7f75cc0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f75d802aa00 tx=0x7f75d80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.275+0000 7f75dfe00640 1 -- 172.21.15.83:0/1927391249 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75e009d350 con 0x7f75e810a030 2024-04-23T14:45:55.443 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:55 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] up:boot 2024-04-23T14:45:55.443 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:55 smithi083 ceph-mon[27867]: fsmap 12 up:standby 2024-04-23T14:45:55.443 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:55 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T14:45:55.443 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:55 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:45:55.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.524+0000 7f75ecc00640 1 -- 172.21.15.83:0/1927391249 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75e8062c00 con 0x7f75e810a030 2024-04-23T14:45:55.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.525+0000 7f75dfe00640 1 -- 172.21.15.83:0/1927391249 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7f75e009cc50 con 0x7f75e810a030 2024-04-23T14:45:55.526 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:55.526 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24509,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24532,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24599,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-23T14:45:55.526 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 13 2024-04-23T14:45:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.528+0000 7f75ecc00640 1 -- 172.21.15.83:0/1927391249 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f75cc0a3bb0 msgr2=0x7f75cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.528+0000 7f75ecc00640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f75cc0a3bb0 0x7f75cc0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f75d802aa00 tx=0x7f75d80315b0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.528+0000 7f75ecc00640 1 -- 172.21.15.83:0/1927391249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75e810a030 msgr2=0x7f75e81a5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.528+0000 7f75ecc00640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75e810a030 0x7f75e81a5d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75e000eb10 tx=0x7f75e000c2d0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.529+0000 7f75ecc00640 1 -- 172.21.15.83:0/1927391249 shutdown_connections 2024-04-23T14:45:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.529+0000 7f75ecc00640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f75cc0a3bb0 0x7f75cc0a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.529+0000 7f75ecc00640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75e810a030 0x7f75e81a5d40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.529+0000 7f75ecc00640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75e8109670 0x7f75e819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.529+0000 7f75ecc00640 1 --2- 172.21.15.83:0/1927391249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75e8108480 0x7f75e819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.529+0000 7f75ecc00640 1 -- 172.21.15.83:0/1927391249 >> 172.21.15.83:0/1927391249 conn(0x7f75e8077440 msgr2=0x7f75e810c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:55.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.529+0000 7f75ecc00640 1 -- 172.21.15.83:0/1927391249 shutdown_connections 2024-04-23T14:45:55.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.529+0000 7f75ecc00640 1 -- 172.21.15.83:0/1927391249 wait complete. 2024-04-23T14:45:55.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:45:55.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.735+0000 7f139ea00640 1 -- 172.21.15.83:0/944591358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1398109670 msgr2=0x7f1398109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:55.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.735+0000 7f139ea00640 1 --2- 172.21.15.83:0/944591358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1398109670 0x7f1398109af0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1388009900 tx=0x7f138802a3f0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.735+0000 7f139ea00640 1 -- 172.21.15.83:0/944591358 shutdown_connections 2024-04-23T14:45:55.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.735+0000 7f139ea00640 1 --2- 172.21.15.83:0/944591358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f139810a030 0x7f13981108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.735+0000 7f139ea00640 1 --2- 172.21.15.83:0/944591358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1398109670 0x7f1398109af0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.735+0000 7f139ea00640 1 --2- 172.21.15.83:0/944591358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1398108480 0x7f13981088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.735+0000 7f139ea00640 1 -- 172.21.15.83:0/944591358 >> 172.21.15.83:0/944591358 conn(0x7f1398077440 msgr2=0x7f1398079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:55.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.735+0000 7f139ea00640 1 -- 172.21.15.83:0/944591358 shutdown_connections 2024-04-23T14:45:55.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.735+0000 7f139ea00640 1 -- 172.21.15.83:0/944591358 wait complete. 2024-04-23T14:45:55.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.736+0000 7f139ea00640 1 Processor -- start 2024-04-23T14:45:55.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.736+0000 7f139ea00640 1 -- start start 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1398108480 0x7f139819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398109670 0x7f139819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f139810a030 0x7f13981a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1398114110 con 0x7f1398108480 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139ea00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1398113f90 con 0x7f139810a030 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139ea00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1398114290 con 0x7f1398109670 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398109670 0x7f139819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398109670 0x7f139819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:58612/0 (socket says 172.21.15.83:58612) 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139cc00640 1 -- 172.21.15.83:0/2618931170 learned_addr learned my addr 172.21.15.83:0/2618931170 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139e000640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f139810a030 0x7f13981a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139d600640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1398108480 0x7f139819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139cc00640 1 -- 172.21.15.83:0/2618931170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f139810a030 msgr2=0x7f13981a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139cc00640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f139810a030 0x7f13981a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139cc00640 1 -- 172.21.15.83:0/2618931170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1398108480 msgr2=0x7f139819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:55.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139cc00640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1398108480 0x7f139819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139cc00640 1 -- 172.21.15.83:0/2618931170 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13981a6390 con 0x7f1398109670 2024-04-23T14:45:55.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.737+0000 7f139e000640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f139810a030 0x7f13981a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:55.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.738+0000 7f139cc00640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398109670 0x7f139819ed00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1388009230 tx=0x7f13880042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:55.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.738+0000 7f1396000640 1 -- 172.21.15.83:0/2618931170 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138803f030 con 0x7f1398109670 2024-04-23T14:45:55.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.738+0000 7f139ea00640 1 -- 172.21.15.83:0/2618931170 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13981a6620 con 0x7f1398109670 2024-04-23T14:45:55.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.738+0000 7f1396000640 1 -- 172.21.15.83:0/2618931170 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1388003ce0 con 0x7f1398109670 2024-04-23T14:45:55.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.738+0000 7f1396000640 1 -- 172.21.15.83:0/2618931170 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388035070 con 0x7f1398109670 2024-04-23T14:45:55.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.738+0000 7f139ea00640 1 -- 172.21.15.83:0/2618931170 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13981a6b10 con 0x7f1398109670 2024-04-23T14:45:55.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.739+0000 7f139ea00640 1 -- 172.21.15.83:0/2618931170 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139810f7a0 con 0x7f1398109670 2024-04-23T14:45:55.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.741+0000 7f1396000640 1 -- 172.21.15.83:0/2618931170 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1388042030 con 0x7f1398109670 2024-04-23T14:45:55.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.742+0000 7f1396000640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f13800a3bb0 0x7f13800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:55.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.743+0000 7f1396000640 1 -- 172.21.15.83:0/2618931170 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f13880f0440 con 0x7f1398109670 2024-04-23T14:45:55.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.743+0000 7f139d600640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f13800a3bb0 0x7f13800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:55.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.743+0000 7f139d600640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f13800a3bb0 0x7f13800a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13981094d0 tx=0x7f138c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:55.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.751+0000 7f1396000640 1 -- 172.21.15.83:0/2618931170 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13880bcbd0 con 0x7f1398109670 2024-04-23T14:45:55.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.976+0000 7f139ea00640 1 -- 172.21.15.83:0/2618931170 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1398062c00 con 0x7f1398109670 2024-04-23T14:45:55.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.978+0000 7f1396000640 1 -- 172.21.15.83:0/2618931170 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18456 (secure 0 0 0) 0x7f13880c0ed0 con 0x7f1398109670 2024-04-23T14:45:55.979 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:55.980 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":82,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:45:02.842396+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:45:55.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.981+0000 7f139ea00640 1 -- 172.21.15.83:0/2618931170 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f13800a3bb0 msgr2=0x7f13800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:55.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.981+0000 7f139ea00640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f13800a3bb0 0x7f13800a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13981094d0 tx=0x7f138c005f90 comp rx=0 tx=0).stop 2024-04-23T14:45:55.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.981+0000 7f139ea00640 1 -- 172.21.15.83:0/2618931170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398109670 msgr2=0x7f139819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:55.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.981+0000 7f139ea00640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398109670 0x7f139819ed00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1388009230 tx=0x7f13880042f0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.981+0000 7f139ea00640 1 -- 172.21.15.83:0/2618931170 shutdown_connections 2024-04-23T14:45:55.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.981+0000 7f139ea00640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f13800a3bb0 0x7f13800a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.981+0000 7f139ea00640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f139810a030 0x7f13981a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.981+0000 7f139ea00640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1398109670 0x7f139819ed00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.981+0000 7f139ea00640 1 --2- 172.21.15.83:0/2618931170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1398108480 0x7f139819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:55.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.982+0000 7f139ea00640 1 -- 172.21.15.83:0/2618931170 >> 172.21.15.83:0/2618931170 conn(0x7f1398077440 msgr2=0x7f139810c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:55.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.982+0000 7f139ea00640 1 -- 172.21.15.83:0/2618931170 shutdown_connections 2024-04-23T14:45:55.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:55.982+0000 7f139ea00640 1 -- 172.21.15.83:0/2618931170 wait complete. 2024-04-23T14:45:55.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:40:41.241992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:45:56.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:45:56.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.189+0000 7f80a2c00640 1 -- 172.21.15.83:0/1515501578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f809c104280 msgr2=0x7f809c1046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.189+0000 7f80a2c00640 1 --2- 172.21.15.83:0/1515501578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f809c104280 0x7f809c1046a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f808c009900 tx=0x7f808c02a3f0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.189+0000 7f80a2c00640 1 -- 172.21.15.83:0/1515501578 shutdown_connections 2024-04-23T14:45:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.189+0000 7f80a2c00640 1 --2- 172.21.15.83:0/1515501578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809c105e30 0x7f809c10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.189+0000 7f80a2c00640 1 --2- 172.21.15.83:0/1515501578 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f809c105470 0x7f809c1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.189+0000 7f80a2c00640 1 --2- 172.21.15.83:0/1515501578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f809c104280 0x7f809c1046a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.189+0000 7f80a2c00640 1 -- 172.21.15.83:0/1515501578 >> 172.21.15.83:0/1515501578 conn(0x7f809c0ff7f0 msgr2=0x7f809c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.189+0000 7f80a2c00640 1 -- 172.21.15.83:0/1515501578 shutdown_connections 2024-04-23T14:45:56.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.189+0000 7f80a2c00640 1 -- 172.21.15.83:0/1515501578 wait complete. 2024-04-23T14:45:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.191+0000 7f80a2c00640 1 Processor -- start 2024-04-23T14:45:56.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.191+0000 7f80a2c00640 1 -- start start 2024-04-23T14:45:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.191+0000 7f80a2c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809c104280 0x7f809c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.191+0000 7f80a2c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f809c105470 0x7f809c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.191+0000 7f80a2c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f809c105e30 0x7f809c1a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.191+0000 7f80a2c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f809c10fee0 con 0x7f809c104280 2024-04-23T14:45:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.191+0000 7f80a2c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f809c10fd60 con 0x7f809c105470 2024-04-23T14:45:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.191+0000 7f80a2c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f809c110060 con 0x7f809c105e30 2024-04-23T14:45:56.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.191+0000 7f809be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809c104280 0x7f809c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.192+0000 7f809b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f809c105470 0x7f809c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.192+0000 7f809be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809c104280 0x7f809c19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58854/0 (socket says 172.21.15.83:58854) 2024-04-23T14:45:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.192+0000 7f809be00640 1 -- 172.21.15.83:0/1857477392 learned_addr learned my addr 172.21.15.83:0/1857477392 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.192+0000 7f80a0c00640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f809c105e30 0x7f809c1a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.192+0000 7f809b400640 1 -- 172.21.15.83:0/1857477392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f809c105e30 msgr2=0x7f809c1a1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.192+0000 7f809b400640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f809c105e30 0x7f809c1a1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.192+0000 7f809b400640 1 -- 172.21.15.83:0/1857477392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809c104280 msgr2=0x7f809c19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.192+0000 7f809b400640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809c104280 0x7f809c19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.192+0000 7f809b400640 1 -- 172.21.15.83:0/1857477392 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809c1a2200 con 0x7f809c105470 2024-04-23T14:45:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.192+0000 7f809b400640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f809c105470 0x7f809c19ab20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f808800e9f0 tx=0x7f808800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.193+0000 7f8098c00640 1 -- 172.21.15.83:0/1857477392 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808800cd80 con 0x7f809c105470 2024-04-23T14:45:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.193+0000 7f80a2c00640 1 -- 172.21.15.83:0/1857477392 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809c1a24f0 con 0x7f809c105470 2024-04-23T14:45:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.193+0000 7f8098c00640 1 -- 172.21.15.83:0/1857477392 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80880035f0 con 0x7f809c105470 2024-04-23T14:45:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.193+0000 7f8098c00640 1 -- 172.21.15.83:0/1857477392 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088010670 con 0x7f809c105470 2024-04-23T14:45:56.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.193+0000 7f80a2c00640 1 -- 172.21.15.83:0/1857477392 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809c1a2a90 con 0x7f809c105470 2024-04-23T14:45:56.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.194+0000 7f80a2c00640 1 -- 172.21.15.83:0/1857477392 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f809c10b610 con 0x7f809c105470 2024-04-23T14:45:56.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.197+0000 7f8098c00640 1 -- 172.21.15.83:0/1857477392 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f80880037a0 con 0x7f809c105470 2024-04-23T14:45:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.198+0000 7f8098c00640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80800a3c30 0x7f80800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.198+0000 7f8098c00640 1 -- 172.21.15.83:0/1857477392 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f80880d0250 con 0x7f809c105470 2024-04-23T14:45:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.198+0000 7f809be00640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80800a3c30 0x7f80800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:56.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.199+0000 7f809be00640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80800a3c30 0x7f80800a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f808c000c00 tx=0x7f808c0046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:56.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.202+0000 7f8098c00640 1 -- 172.21.15.83:0/1857477392 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f808809d030 con 0x7f809c105470 2024-04-23T14:45:56.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:56 smithi083 ceph-mon[27867]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:56.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:56 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1927391249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:56.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:56.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:56.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:56 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2618931170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:45:56.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:56.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:56 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.412+0000 7f80a2c00640 1 -- 172.21.15.83:0/1857477392 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f809c062c00 con 0x7f809c105470 2024-04-23T14:45:56.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.413+0000 7f8098c00640 1 -- 172.21.15.83:0/1857477392 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f808809c930 con 0x7f809c105470 2024-04-23T14:45:56.413 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-23T14:45:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.415+0000 7f80a2c00640 1 -- 172.21.15.83:0/1857477392 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80800a3c30 msgr2=0x7f80800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.415+0000 7f80a2c00640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80800a3c30 0x7f80800a60f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f808c000c00 tx=0x7f808c0046c0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.415+0000 7f80a2c00640 1 -- 172.21.15.83:0/1857477392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f809c105470 msgr2=0x7f809c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.415+0000 7f80a2c00640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f809c105470 0x7f809c19ab20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f808800e9f0 tx=0x7f808800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.415+0000 7f80a2c00640 1 -- 172.21.15.83:0/1857477392 shutdown_connections 2024-04-23T14:45:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.415+0000 7f80a2c00640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80800a3c30 0x7f80800a60f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.415+0000 7f80a2c00640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f809c105e30 0x7f809c1a1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.415+0000 7f80a2c00640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f809c105470 0x7f809c19ab20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.415+0000 7f80a2c00640 1 --2- 172.21.15.83:0/1857477392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f809c104280 0x7f809c19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.415+0000 7f80a2c00640 1 -- 172.21.15.83:0/1857477392 >> 172.21.15.83:0/1857477392 conn(0x7f809c0ff7f0 msgr2=0x7f809c1087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.416+0000 7f80a2c00640 1 -- 172.21.15.83:0/1857477392 shutdown_connections 2024-04-23T14:45:56.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.416+0000 7f80a2c00640 1 -- 172.21.15.83:0/1857477392 wait complete. 2024-04-23T14:45:56.435 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T14:45:56.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:56 smithi203 ceph-mon[32800]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:56.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:56 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1927391249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:56.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:56.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:56.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:56 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2618931170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:45:56.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:56.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:56 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:56.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:56 smithi173 ceph-mon[32443]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:56.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:56 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1927391249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:45:56.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:56 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:56.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:56 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:56.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:56 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2618931170' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:45:56.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:56 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:56.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:56 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:56.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.626+0000 7f899c000640 1 -- 172.21.15.83:0/1701812893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8994074a00 msgr2=0x7f8994074e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:56.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.626+0000 7f899c000640 1 --2- 172.21.15.83:0/1701812893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8994074a00 0x7f8994074e60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8980009900 tx=0x7f898002a480 comp rx=0 tx=0).stop 2024-04-23T14:45:56.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.626+0000 7f899c000640 1 -- 172.21.15.83:0/1701812893 shutdown_connections 2024-04-23T14:45:56.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.626+0000 7f899c000640 1 --2- 172.21.15.83:0/1701812893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8994109ee0 0x7f899410c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.626+0000 7f899c000640 1 --2- 172.21.15.83:0/1701812893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8994074a00 0x7f8994074e60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.626+0000 7f899c000640 1 --2- 172.21.15.83:0/1701812893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8994075fe0 0x7f8994074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.626+0000 7f899c000640 1 -- 172.21.15.83:0/1701812893 >> 172.21.15.83:0/1701812893 conn(0x7f89940fd5e0 msgr2=0x7f89940ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:56.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.626+0000 7f899c000640 1 -- 172.21.15.83:0/1701812893 shutdown_connections 2024-04-23T14:45:56.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.626+0000 7f899c000640 1 -- 172.21.15.83:0/1701812893 wait complete. 2024-04-23T14:45:56.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.627+0000 7f899c000640 1 Processor -- start 2024-04-23T14:45:56.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.627+0000 7f899c000640 1 -- start start 2024-04-23T14:45:56.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f899c000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8994074a00 0x7f899419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:56.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f899c000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8994075fe0 0x7f899419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:56.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f899c000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8994109ee0 0x7f89941a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:56.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f899c000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f899410ff30 con 0x7f8994074a00 2024-04-23T14:45:56.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f899c000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f899410fdb0 con 0x7f8994109ee0 2024-04-23T14:45:56.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f899c000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f89941100b0 con 0x7f8994075fe0 2024-04-23T14:45:56.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f8999600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8994074a00 0x7f899419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f8999600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8994074a00 0x7f899419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58866/0 (socket says 172.21.15.83:58866) 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f8999600640 1 -- 172.21.15.83:0/2237010861 learned_addr learned my addr 172.21.15.83:0/2237010861 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f8998c00640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8994075fe0 0x7f899419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f899a000640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8994109ee0 0x7f89941a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f8999600640 1 -- 172.21.15.83:0/2237010861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8994075fe0 msgr2=0x7f899419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f8999600640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8994075fe0 0x7f899419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f8999600640 1 -- 172.21.15.83:0/2237010861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8994109ee0 msgr2=0x7f89941a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f8999600640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8994109ee0 0x7f89941a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f8999600640 1 -- 172.21.15.83:0/2237010861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89941a2190 con 0x7f8994074a00 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.628+0000 7f899a000640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8994109ee0 0x7f89941a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.629+0000 7f8999600640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8994074a00 0x7f899419a570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f898800e9f0 tx=0x7f898800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.629+0000 7f8992000640 1 -- 172.21.15.83:0/2237010861 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898800cd80 con 0x7f8994074a00 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.629+0000 7f8992000640 1 -- 172.21.15.83:0/2237010861 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8988010910 con 0x7f8994074a00 2024-04-23T14:45:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.629+0000 7f8992000640 1 -- 172.21.15.83:0/2237010861 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988018990 con 0x7f8994074a00 2024-04-23T14:45:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.629+0000 7f899c000640 1 -- 172.21.15.83:0/2237010861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89941a2480 con 0x7f8994074a00 2024-04-23T14:45:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.629+0000 7f899c000640 1 -- 172.21.15.83:0/2237010861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89941a2950 con 0x7f8994074a00 2024-04-23T14:45:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.630+0000 7f899c000640 1 -- 172.21.15.83:0/2237010861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8994108ca0 con 0x7f8994074a00 2024-04-23T14:45:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.632+0000 7f8992000640 1 -- 172.21.15.83:0/2237010861 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8988010ac0 con 0x7f8994074a00 2024-04-23T14:45:56.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.634+0000 7f8992000640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f897c0a3bb0 0x7f897c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:56.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.634+0000 7f8992000640 1 -- 172.21.15.83:0/2237010861 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f89880d0330 con 0x7f8994074a00 2024-04-23T14:45:56.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.634+0000 7f8998c00640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f897c0a3bb0 0x7f897c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:56.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.635+0000 7f8998c00640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f897c0a3bb0 0x7f897c0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8980031440 tx=0x7f89800315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:56.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.640+0000 7f8992000640 1 -- 172.21.15.83:0/2237010861 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f898809d1b0 con 0x7f8994074a00 2024-04-23T14:45:56.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.862+0000 7f899c000640 1 -- 172.21.15.83:0/2237010861 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f899402d8d0 con 0x7f8994074a00 2024-04-23T14:45:56.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.863+0000 7f8992000640 1 -- 172.21.15.83:0/2237010861 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f898809cab0 con 0x7f8994074a00 2024-04-23T14:45:56.863 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:45:56.863 INFO:teuthology.orchestra.run.smithi083.stdout:[ 2024-04-23T14:45:56.863 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:45:56.863 INFO:teuthology.orchestra.run.smithi083.stdout: "poolnum": 1, 2024-04-23T14:45:56.863 INFO:teuthology.orchestra.run.smithi083.stdout: "poolname": ".mgr" 2024-04-23T14:45:56.863 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:45:56.864 INFO:teuthology.orchestra.run.smithi083.stdout:] 2024-04-23T14:45:56.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.865+0000 7f899c000640 1 -- 172.21.15.83:0/2237010861 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f897c0a3bb0 msgr2=0x7f897c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.865+0000 7f899c000640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f897c0a3bb0 0x7f897c0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8980031440 tx=0x7f89800315b0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.865+0000 7f899c000640 1 -- 172.21.15.83:0/2237010861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8994074a00 msgr2=0x7f899419a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.865+0000 7f899c000640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8994074a00 0x7f899419a570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f898800e9f0 tx=0x7f898800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.865+0000 7f899c000640 1 -- 172.21.15.83:0/2237010861 shutdown_connections 2024-04-23T14:45:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.865+0000 7f899c000640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f897c0a3bb0 0x7f897c0a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.865+0000 7f899c000640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8994109ee0 0x7f89941a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.865+0000 7f899c000640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8994075fe0 0x7f899419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.865+0000 7f899c000640 1 --2- 172.21.15.83:0/2237010861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8994074a00 0x7f899419a570 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.865+0000 7f899c000640 1 -- 172.21.15.83:0/2237010861 >> 172.21.15.83:0/2237010861 conn(0x7f89940fd5e0 msgr2=0x7f8994105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.865+0000 7f899c000640 1 -- 172.21.15.83:0/2237010861 shutdown_connections 2024-04-23T14:45:56.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:56.865+0000 7f899c000640 1 -- 172.21.15.83:0/2237010861 wait complete. 2024-04-23T14:45:56.884 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T14:45:56.884 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T14:45:57.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.086+0000 7feca6c00640 1 -- 172.21.15.83:0/477067963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feca0103c00 msgr2=0x7feca010a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:57.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.086+0000 7feca6c00640 1 --2- 172.21.15.83:0/477067963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feca0103c00 0x7feca010a490 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fec98009be0 tx=0x7fec9802a6c0 comp rx=0 tx=0).stop 2024-04-23T14:45:57.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.086+0000 7feca6c00640 1 -- 172.21.15.83:0/477067963 shutdown_connections 2024-04-23T14:45:57.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.086+0000 7feca6c00640 1 --2- 172.21.15.83:0/477067963 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feca0103c00 0x7feca010a490 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:57.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.086+0000 7feca6c00640 1 --2- 172.21.15.83:0/477067963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca0103240 0x7feca01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:57.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.086+0000 7feca6c00640 1 --2- 172.21.15.83:0/477067963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca0102050 0x7feca0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:57.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.086+0000 7feca6c00640 1 -- 172.21.15.83:0/477067963 >> 172.21.15.83:0/477067963 conn(0x7feca00fd5e0 msgr2=0x7feca00ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:57.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.086+0000 7feca6c00640 1 -- 172.21.15.83:0/477067963 shutdown_connections 2024-04-23T14:45:57.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.086+0000 7feca6c00640 1 -- 172.21.15.83:0/477067963 wait complete. 2024-04-23T14:45:57.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.087+0000 7feca6c00640 1 Processor -- start 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.087+0000 7feca6c00640 1 -- start start 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7feca6c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca0102050 0x7feca0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7feca6c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca0103240 0x7feca0198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7feca6c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feca0103c00 0x7feca019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7feca6c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feca010dce0 con 0x7feca0103240 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7feca6c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feca010db60 con 0x7feca0103c00 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7feca6c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feca010de60 con 0x7feca0102050 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7fec9f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca0103240 0x7feca0198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7fec9f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca0103240 0x7feca0198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58880/0 (socket says 172.21.15.83:58880) 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7fec9f400640 1 -- 172.21.15.83:0/2867942883 learned_addr learned my addr 172.21.15.83:0/2867942883 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7fec9fe00640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca0102050 0x7feca0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7feca4c00640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feca0103c00 0x7feca019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7fec9f400640 1 -- 172.21.15.83:0/2867942883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca0102050 msgr2=0x7feca0198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:57.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7fec9f400640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca0102050 0x7feca0198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7fec9f400640 1 -- 172.21.15.83:0/2867942883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feca0103c00 msgr2=0x7feca019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7fec9f400640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feca0103c00 0x7feca019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.088+0000 7fec9f400640 1 -- 172.21.15.83:0/2867942883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca019ff30 con 0x7feca0103240 2024-04-23T14:45:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.089+0000 7fec9f400640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca0103240 0x7feca0198850 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00ea60 tx=0x7fec8c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.089+0000 7fec9cc00640 1 -- 172.21.15.83:0/2867942883 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c00cca0 con 0x7feca0103240 2024-04-23T14:45:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.089+0000 7feca6c00640 1 -- 172.21.15.83:0/2867942883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca0074290 con 0x7feca0103240 2024-04-23T14:45:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.089+0000 7fec9cc00640 1 -- 172.21.15.83:0/2867942883 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec8c00ce40 con 0x7feca0103240 2024-04-23T14:45:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.089+0000 7fec9cc00640 1 -- 172.21.15.83:0/2867942883 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c010690 con 0x7feca0103240 2024-04-23T14:45:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.089+0000 7feca6c00640 1 -- 172.21.15.83:0/2867942883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca0074830 con 0x7feca0103240 2024-04-23T14:45:57.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.090+0000 7feca6c00640 1 -- 172.21.15.83:0/2867942883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca0109370 con 0x7feca0103240 2024-04-23T14:45:57.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.093+0000 7fec9cc00640 1 -- 172.21.15.83:0/2867942883 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fec8c003ee0 con 0x7feca0103240 2024-04-23T14:45:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.094+0000 7fec9cc00640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fec840a3bb0 0x7fec840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.095+0000 7fec9cc00640 1 -- 172.21.15.83:0/2867942883 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fec8c0d02f0 con 0x7feca0103240 2024-04-23T14:45:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.095+0000 7fec9fe00640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fec840a3bb0 0x7fec840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.095+0000 7fec9fe00640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fec840a3bb0 0x7fec840a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feca01030a0 tx=0x7fec90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:57.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.097+0000 7fec9cc00640 1 -- 172.21.15.83:0/2867942883 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec8c09d160 con 0x7feca0103240 2024-04-23T14:45:57.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:57 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1857477392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:45:57.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:57 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2237010861' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T14:45:57.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:57.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:57.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:57.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:45:57.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:57.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:57.330+0000 7feca6c00640 1 -- 172.21.15.83:0/2867942883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7feca002d840 con 0x7feca0103240 2024-04-23T14:45:57.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:57 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1857477392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:45:57.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:57 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2237010861' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T14:45:57.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:57.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:57.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:57.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:45:57.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:57 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1857477392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:45:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:57 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2237010861' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T14:45:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:45:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:45:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:58.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.205+0000 7fec9cc00640 1 -- 172.21.15.83:0/2867942883 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fec8c09ca60 con 0x7feca0103240 2024-04-23T14:45:58.206 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' created 2024-04-23T14:45:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.209+0000 7feca6c00640 1 -- 172.21.15.83:0/2867942883 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fec840a3bb0 msgr2=0x7fec840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.209+0000 7feca6c00640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fec840a3bb0 0x7fec840a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feca01030a0 tx=0x7fec90009260 comp rx=0 tx=0).stop 2024-04-23T14:45:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.209+0000 7feca6c00640 1 -- 172.21.15.83:0/2867942883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca0103240 msgr2=0x7feca0198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.209+0000 7feca6c00640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca0103240 0x7feca0198850 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00ea60 tx=0x7fec8c00ee60 comp rx=0 tx=0).stop 2024-04-23T14:45:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.209+0000 7feca6c00640 1 -- 172.21.15.83:0/2867942883 shutdown_connections 2024-04-23T14:45:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.209+0000 7feca6c00640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fec840a3bb0 0x7fec840a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.209+0000 7feca6c00640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feca0103c00 0x7feca019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.209+0000 7feca6c00640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca0103240 0x7feca0198850 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.209+0000 7feca6c00640 1 --2- 172.21.15.83:0/2867942883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feca0102050 0x7feca0198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.209+0000 7feca6c00640 1 -- 172.21.15.83:0/2867942883 >> 172.21.15.83:0/2867942883 conn(0x7feca00fd5e0 msgr2=0x7feca0106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.209+0000 7feca6c00640 1 -- 172.21.15.83:0/2867942883 shutdown_connections 2024-04-23T14:45:58.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.209+0000 7feca6c00640 1 -- 172.21.15.83:0/2867942883 wait complete. 2024-04-23T14:45:58.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T14:45:58.342 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:58 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:58.342 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:58 smithi173 ceph-mon[32443]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:58.342 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:58 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:45:58.342 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:58 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2867942883' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T14:45:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.426+0000 7f806ae00640 1 -- 172.21.15.83:0/867135161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f806410a0f0 msgr2=0x7f8064110980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.426+0000 7f806ae00640 1 --2- 172.21.15.83:0/867135161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f806410a0f0 0x7f8064110980 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f805c009ca0 tx=0x7f805c02a8c0 comp rx=0 tx=0).stop 2024-04-23T14:45:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.426+0000 7f806ae00640 1 -- 172.21.15.83:0/867135161 shutdown_connections 2024-04-23T14:45:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.426+0000 7f806ae00640 1 --2- 172.21.15.83:0/867135161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f806410a0f0 0x7f8064110980 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.426+0000 7f806ae00640 1 --2- 172.21.15.83:0/867135161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8064109730 0x7f8064109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.426+0000 7f806ae00640 1 --2- 172.21.15.83:0/867135161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8064108540 0x7f8064108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.426+0000 7f806ae00640 1 -- 172.21.15.83:0/867135161 >> 172.21.15.83:0/867135161 conn(0x7f80640774d0 msgr2=0x7f8064079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.426+0000 7f806ae00640 1 -- 172.21.15.83:0/867135161 shutdown_connections 2024-04-23T14:45:58.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.426+0000 7f806ae00640 1 -- 172.21.15.83:0/867135161 wait complete. 2024-04-23T14:45:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.427+0000 7f806ae00640 1 Processor -- start 2024-04-23T14:45:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.427+0000 7f806ae00640 1 -- start start 2024-04-23T14:45:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.427+0000 7f806ae00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8064108540 0x7f806419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.427+0000 7f806ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8064109730 0x7f806419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.427+0000 7f806ae00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f806410a0f0 0x7f80641a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.427+0000 7f806ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80641141a0 con 0x7f8064109730 2024-04-23T14:45:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.427+0000 7f806ae00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8064114020 con 0x7f806410a0f0 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.427+0000 7f806ae00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8064114320 con 0x7f8064108540 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.427+0000 7f8069000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8064109730 0x7f806419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.427+0000 7f8069000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8064109730 0x7f806419ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58902/0 (socket says 172.21.15.83:58902) 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.427+0000 7f8069000640 1 -- 172.21.15.83:0/2583071439 learned_addr learned my addr 172.21.15.83:0/2583071439 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.428+0000 7f8069a00640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8064108540 0x7f806419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.428+0000 7f806a400640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f806410a0f0 0x7f80641a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.428+0000 7f8069000640 1 -- 172.21.15.83:0/2583071439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8064108540 msgr2=0x7f806419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.428+0000 7f8069000640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8064108540 0x7f806419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.428+0000 7f8069000640 1 -- 172.21.15.83:0/2583071439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f806410a0f0 msgr2=0x7f80641a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.428+0000 7f8069000640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f806410a0f0 0x7f80641a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.428+0000 7f8069000640 1 -- 172.21.15.83:0/2583071439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80641a63e0 con 0x7f8064109730 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.428+0000 7f8069a00640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8064108540 0x7f806419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.428+0000 7f8069000640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8064109730 0x7f806419ed00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f805400e9f0 tx=0x7f805400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.428+0000 7f8062000640 1 -- 172.21.15.83:0/2583071439 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805400cd80 con 0x7f8064109730 2024-04-23T14:45:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.429+0000 7f806ae00640 1 -- 172.21.15.83:0/2583071439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80641a66d0 con 0x7f8064109730 2024-04-23T14:45:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.429+0000 7f8062000640 1 -- 172.21.15.83:0/2583071439 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8054010910 con 0x7f8064109730 2024-04-23T14:45:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.429+0000 7f8062000640 1 -- 172.21.15.83:0/2583071439 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054018910 con 0x7f8064109730 2024-04-23T14:45:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.429+0000 7f806ae00640 1 -- 172.21.15.83:0/2583071439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80641a6c70 con 0x7f8064109730 2024-04-23T14:45:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.430+0000 7f806ae00640 1 -- 172.21.15.83:0/2583071439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f806410f860 con 0x7f8064109730 2024-04-23T14:45:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.432+0000 7f8062000640 1 -- 172.21.15.83:0/2583071439 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8054010ac0 con 0x7f8064109730 2024-04-23T14:45:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.433+0000 7f8062000640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f804c0a3bb0 0x7f804c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.434+0000 7f8062000640 1 -- 172.21.15.83:0/2583071439 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f80540d07e0 con 0x7f8064109730 2024-04-23T14:45:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.434+0000 7f8069a00640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f804c0a3bb0 0x7f804c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:58.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.434+0000 7f8069a00640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f804c0a3bb0 0x7f804c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8064109590 tx=0x7f805800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.436+0000 7f8062000640 1 -- 172.21.15.83:0/2583071439 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f805409cea0 con 0x7f8064109730 2024-04-23T14:45:58.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:58 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:58.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:58 smithi203 ceph-mon[32800]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:58.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:58 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:45:58.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:58 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2867942883' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T14:45:58.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:58.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:58 smithi083 ceph-mon[27867]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:45:58.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:45:58.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:58 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2867942883' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T14:45:58.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:58.642+0000 7f806ae00640 1 -- 172.21.15.83:0/2583071439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f806402d620 con 0x7f8064109730 2024-04-23T14:45:59.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.212+0000 7f8062000640 1 -- 172.21.15.83:0/2583071439 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f80540a11a0 con 0x7f8064109730 2024-04-23T14:45:59.213 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' created 2024-04-23T14:45:59.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.217+0000 7f806ae00640 1 -- 172.21.15.83:0/2583071439 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f804c0a3bb0 msgr2=0x7f804c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:59.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.217+0000 7f806ae00640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f804c0a3bb0 0x7f804c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8064109590 tx=0x7f805800b390 comp rx=0 tx=0).stop 2024-04-23T14:45:59.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.217+0000 7f806ae00640 1 -- 172.21.15.83:0/2583071439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8064109730 msgr2=0x7f806419ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:59.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.217+0000 7f806ae00640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8064109730 0x7f806419ed00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f805400e9f0 tx=0x7f805400eeb0 comp rx=0 tx=0).stop 2024-04-23T14:45:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.219+0000 7f806ae00640 1 -- 172.21.15.83:0/2583071439 shutdown_connections 2024-04-23T14:45:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.219+0000 7f806ae00640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f804c0a3bb0 0x7f804c0a6070 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8064109590 tx=0x7f805800b390 comp rx=0 tx=0).stop 2024-04-23T14:45:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.219+0000 7f806ae00640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f806410a0f0 0x7f80641a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.219+0000 7f806ae00640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8064109730 0x7f806419ed00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.219+0000 7f806ae00640 1 --2- 172.21.15.83:0/2583071439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8064108540 0x7f806419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:59.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.219+0000 7f806ae00640 1 -- 172.21.15.83:0/2583071439 >> 172.21.15.83:0/2583071439 conn(0x7f80640774d0 msgr2=0x7f806410ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:59.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.222+0000 7f806ae00640 1 -- 172.21.15.83:0/2583071439 shutdown_connections 2024-04-23T14:45:59.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.222+0000 7f806ae00640 1 -- 172.21.15.83:0/2583071439 wait complete. 2024-04-23T14:45:59.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T14:45:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.447+0000 7f28ba000640 1 -- 172.21.15.83:0/2665824497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4102b20 msgr2=0x7f28b4104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.447+0000 7f28ba000640 1 --2- 172.21.15.83:0/2665824497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4102b20 0x7f28b4104f10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009900 tx=0x7f28a402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:45:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.448+0000 7f28ba000640 1 -- 172.21.15.83:0/2665824497 shutdown_connections 2024-04-23T14:45:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.448+0000 7f28ba000640 1 --2- 172.21.15.83:0/2665824497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b4107d80 0x7f28b410a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.448+0000 7f28ba000640 1 --2- 172.21.15.83:0/2665824497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28b4105450 0x7f28b4107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:59.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.448+0000 7f28ba000640 1 --2- 172.21.15.83:0/2665824497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4102b20 0x7f28b4104f10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.448+0000 7f28ba000640 1 -- 172.21.15.83:0/2665824497 >> 172.21.15.83:0/2665824497 conn(0x7f28b40fc4b0 msgr2=0x7f28b40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:45:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.448+0000 7f28ba000640 1 -- 172.21.15.83:0/2665824497 shutdown_connections 2024-04-23T14:45:59.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.448+0000 7f28ba000640 1 -- 172.21.15.83:0/2665824497 wait complete. 2024-04-23T14:45:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28ba000640 1 Processor -- start 2024-04-23T14:45:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28ba000640 1 -- start start 2024-04-23T14:45:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28ba000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28b4102b20 0x7f28b419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28ba000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b4105450 0x7f28b419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28ba000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4107d80 0x7f28b41a1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28ba000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28b410dd20 con 0x7f28b4102b20 2024-04-23T14:45:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28ba000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f28b410dba0 con 0x7f28b4107d80 2024-04-23T14:45:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28ba000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f28b410dea0 con 0x7f28b4105450 2024-04-23T14:45:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28b4102b20 0x7f28b419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28b3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28b4102b20 0x7f28b419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58920/0 (socket says 172.21.15.83:58920) 2024-04-23T14:45:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28b3400640 1 -- 172.21.15.83:0/3038784224 learned_addr learned my addr 172.21.15.83:0/3038784224 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:45:59.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28abe00640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b4105450 0x7f28b419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.449+0000 7f28b3e00640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4107d80 0x7f28b41a1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.450+0000 7f28b3400640 1 -- 172.21.15.83:0/3038784224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b4105450 msgr2=0x7f28b419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.450+0000 7f28b3400640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b4105450 0x7f28b419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.450+0000 7f28b3400640 1 -- 172.21.15.83:0/3038784224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4107d80 msgr2=0x7f28b41a1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.450+0000 7f28b3400640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4107d80 0x7f28b41a1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.450+0000 7f28b3400640 1 -- 172.21.15.83:0/3038784224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b41a2210 con 0x7f28b4102b20 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.450+0000 7f28abe00640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b4105450 0x7f28b419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.450+0000 7f28b3e00640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4107d80 0x7f28b41a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.450+0000 7f28b3400640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28b4102b20 0x7f28b419a5f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f28a4031440 tx=0x7f28a4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.450+0000 7f28b0c00640 1 -- 172.21.15.83:0/3038784224 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a403f030 con 0x7f28b4102b20 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.450+0000 7f28b0c00640 1 -- 172.21.15.83:0/3038784224 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28a4003ce0 con 0x7f28b4102b20 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.450+0000 7f28b0c00640 1 -- 172.21.15.83:0/3038784224 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4035070 con 0x7f28b4102b20 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.451+0000 7f28ba000640 1 -- 172.21.15.83:0/3038784224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b41a24a0 con 0x7f28b4102b20 2024-04-23T14:45:59.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.451+0000 7f28ba000640 1 -- 172.21.15.83:0/3038784224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b41a2990 con 0x7f28b4102b20 2024-04-23T14:45:59.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.452+0000 7f28ba000640 1 -- 172.21.15.83:0/3038784224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b4109870 con 0x7f28b4102b20 2024-04-23T14:45:59.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.456+0000 7f28b0c00640 1 -- 172.21.15.83:0/3038784224 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f28a4042030 con 0x7f28b4102b20 2024-04-23T14:45:59.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.457+0000 7f28b0c00640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f28980a3bb0 0x7f28980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:45:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.457+0000 7f28b0c00640 1 -- 172.21.15.83:0/3038784224 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f28a40f0ad0 con 0x7f28b4102b20 2024-04-23T14:45:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.457+0000 7f28abe00640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f28980a3bb0 0x7f28980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:45:59.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.458+0000 7f28abe00640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f28980a3bb0 0x7f28980a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f28b419bb00 tx=0x7f289c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:45:59.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.461+0000 7f28b0c00640 1 -- 172.21.15.83:0/3038784224 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28a40bd010 con 0x7f28b4102b20 2024-04-23T14:45:59.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:59 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2867942883' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T14:45:59.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:59 smithi203 ceph-mon[32800]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T14:45:59.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:59 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2583071439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T14:45:59.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:59 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:59.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:59 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:59.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:59 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:59.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:45:59 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:59.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:59 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2867942883' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T14:45:59.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:59 smithi083 ceph-mon[27867]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T14:45:59.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:59 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2583071439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T14:45:59.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:59 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:59.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:59 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:59.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:59 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:59.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:45:59 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:59.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:59 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2867942883' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T14:45:59.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:59 smithi173 ceph-mon[32443]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T14:45:59.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:59 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2583071439' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T14:45:59.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:59 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:59.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:59 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:59.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:59 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:59.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:45:59 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:45:59.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:45:59.722+0000 7f28ba000640 1 -- 172.21.15.83:0/3038784224 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f28b402d620 con 0x7f28b4102b20 2024-04-23T14:46:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.216+0000 7f28b0c00640 1 -- 172.21.15.83:0/3038784224 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f28a40c1310 con 0x7f28b4102b20 2024-04-23T14:46:00.217 INFO:teuthology.orchestra.run.smithi083.stderr:new fs with metadata pool 2 and data pool 3 2024-04-23T14:46:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.220+0000 7f28ba000640 1 -- 172.21.15.83:0/3038784224 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f28980a3bb0 msgr2=0x7f28980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.220+0000 7f28ba000640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f28980a3bb0 0x7f28980a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f28b419bb00 tx=0x7f289c005f90 comp rx=0 tx=0).stop 2024-04-23T14:46:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.220+0000 7f28ba000640 1 -- 172.21.15.83:0/3038784224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28b4102b20 msgr2=0x7f28b419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:00.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.220+0000 7f28ba000640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28b4102b20 0x7f28b419a5f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f28a4031440 tx=0x7f28a4032ce0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.220+0000 7f28ba000640 1 -- 172.21.15.83:0/3038784224 shutdown_connections 2024-04-23T14:46:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.220+0000 7f28ba000640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f28980a3bb0 0x7f28980a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.220+0000 7f28ba000640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28b4107d80 0x7f28b41a1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.220+0000 7f28ba000640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28b4105450 0x7f28b419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.220+0000 7f28ba000640 1 --2- 172.21.15.83:0/3038784224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28b4102b20 0x7f28b419a5f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.220+0000 7f28ba000640 1 -- 172.21.15.83:0/3038784224 >> 172.21.15.83:0/3038784224 conn(0x7f28b40fc4b0 msgr2=0x7f28b41022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.221+0000 7f28ba000640 1 -- 172.21.15.83:0/3038784224 shutdown_connections 2024-04-23T14:46:00.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.221+0000 7f28ba000640 1 -- 172.21.15.83:0/3038784224 wait complete. 2024-04-23T14:46:00.250 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:46:00.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.433+0000 7f621f200640 1 -- 172.21.15.83:0/4162301738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6220102980 msgr2=0x7f6220104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:00.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.433+0000 7f621f200640 1 --2- 172.21.15.83:0/4162301738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6220102980 0x7f6220104d70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6210009900 tx=0x7f621002a410 comp rx=0 tx=0).stop 2024-04-23T14:46:00.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.433+0000 7f621f200640 1 -- 172.21.15.83:0/4162301738 shutdown_connections 2024-04-23T14:46:00.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.433+0000 7f621f200640 1 --2- 172.21.15.83:0/4162301738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6220107be0 0x7f6220109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.433+0000 7f621f200640 1 --2- 172.21.15.83:0/4162301738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62201052b0 0x7f62201076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.433+0000 7f621f200640 1 --2- 172.21.15.83:0/4162301738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6220102980 0x7f6220104d70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.433+0000 7f621f200640 1 -- 172.21.15.83:0/4162301738 >> 172.21.15.83:0/4162301738 conn(0x7f62200fc2d0 msgr2=0x7f62200fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:00.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.433+0000 7f621f200640 1 -- 172.21.15.83:0/4162301738 shutdown_connections 2024-04-23T14:46:00.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.434+0000 7f621f200640 1 -- 172.21.15.83:0/4162301738 wait complete. 2024-04-23T14:46:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.435+0000 7f621f200640 1 Processor -- start 2024-04-23T14:46:00.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.435+0000 7f621f200640 1 -- start start 2024-04-23T14:46:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.435+0000 7f621f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6220102980 0x7f62201981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.435+0000 7f621f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62201052b0 0x7f62201986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.435+0000 7f621f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6220107be0 0x7f622019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.435+0000 7f621f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f622010daf0 con 0x7f6220102980 2024-04-23T14:46:00.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.436+0000 7f621f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f622010d970 con 0x7f62201052b0 2024-04-23T14:46:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.436+0000 7f621f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f622010dc70 con 0x7f6220107be0 2024-04-23T14:46:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.436+0000 7f621de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6220102980 0x7f62201981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.436+0000 7f6217e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62201052b0 0x7f62201986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.436+0000 7f621e800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6220107be0 0x7f622019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.436+0000 7f621de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6220102980 0x7f62201981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58924/0 (socket says 172.21.15.83:58924) 2024-04-23T14:46:00.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.436+0000 7f621de00640 1 -- 172.21.15.83:0/1079708045 learned_addr learned my addr 172.21.15.83:0/1079708045 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:00.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.436+0000 7f6217e00640 1 -- 172.21.15.83:0/1079708045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6220107be0 msgr2=0x7f622019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:00.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.436+0000 7f6217e00640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6220107be0 0x7f622019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.436+0000 7f6217e00640 1 -- 172.21.15.83:0/1079708045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6220102980 msgr2=0x7f62201981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:00.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.436+0000 7f6217e00640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6220102980 0x7f62201981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.436+0000 7f6217e00640 1 -- 172.21.15.83:0/1079708045 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622019fdc0 con 0x7f62201052b0 2024-04-23T14:46:00.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.437+0000 7f621e800640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6220107be0 0x7f622019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:00.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.437+0000 7f6217e00640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62201052b0 0x7f62201986e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f620800e9c0 tx=0x7f620800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:00.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.437+0000 7f6216a00640 1 -- 172.21.15.83:0/1079708045 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620800cd80 con 0x7f62201052b0 2024-04-23T14:46:00.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.437+0000 7f621f200640 1 -- 172.21.15.83:0/1079708045 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62201a00b0 con 0x7f62201052b0 2024-04-23T14:46:00.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.437+0000 7f6216a00640 1 -- 172.21.15.83:0/1079708045 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62080035f0 con 0x7f62201052b0 2024-04-23T14:46:00.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.437+0000 7f6216a00640 1 -- 172.21.15.83:0/1079708045 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208010670 con 0x7f62201052b0 2024-04-23T14:46:00.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.437+0000 7f621f200640 1 -- 172.21.15.83:0/1079708045 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62201a0650 con 0x7f62201052b0 2024-04-23T14:46:00.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.438+0000 7f621f200640 1 -- 172.21.15.83:0/1079708045 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62200fddc0 con 0x7f62201052b0 2024-04-23T14:46:00.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.441+0000 7f6216a00640 1 -- 172.21.15.83:0/1079708045 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6208004020 con 0x7f62201052b0 2024-04-23T14:46:00.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.442+0000 7f6216a00640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f62040a3bb0 0x7f62040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:00.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.442+0000 7f6216a00640 1 -- 172.21.15.83:0/1079708045 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f62080d0620 con 0x7f62201052b0 2024-04-23T14:46:00.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.443+0000 7f621de00640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f62040a3bb0 0x7f62040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:00.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.443+0000 7f621de00640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f62040a3bb0 0x7f62040a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6210031440 tx=0x7f62100315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:00.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.446+0000 7f6216a00640 1 -- 172.21.15.83:0/1079708045 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f620809d0b0 con 0x7f62201052b0 2024-04-23T14:46:00.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:00 smithi083 ceph-mon[27867]: pgmap v314: 33 pgs: 14 creating+peering, 18 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:46:00.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:00 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2583071439' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T14:46:00.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:00 smithi083 ceph-mon[27867]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T14:46:00.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:00 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3038784224' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T14:46:00.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:00 smithi083 ceph-mon[27867]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T14:46:00.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:46:00.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:00 smithi083 ceph-mon[27867]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T14:46:00.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:46:00.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:46:00.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:46:00.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:46:00.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:00 smithi083 ceph-mon[27867]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T14:46:00.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:00 smithi173 ceph-mon[32443]: pgmap v314: 33 pgs: 14 creating+peering, 18 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:46:00.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:00 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2583071439' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T14:46:00.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:00 smithi173 ceph-mon[32443]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T14:46:00.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:00 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3038784224' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T14:46:00.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:00 smithi173 ceph-mon[32443]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T14:46:00.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:00 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:46:00.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:00 smithi173 ceph-mon[32443]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T14:46:00.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:00 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:46:00.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:00 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:46:00.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:00 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:46:00.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:00 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:46:00.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:00 smithi173 ceph-mon[32443]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T14:46:00.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.651+0000 7f621f200640 1 -- 172.21.15.83:0/1079708045 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6220062c00 con 0x7f62201052b0 2024-04-23T14:46:00.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.653+0000 7f6216a00640 1 -- 172.21.15.83:0/1079708045 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22232 (secure 0 0 0) 0x7f620809c9b0 con 0x7f62201052b0 2024-04-23T14:46:00.654 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:46:00.654 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":85,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:45:59.723572+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:46:00.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.656+0000 7f621f200640 1 -- 172.21.15.83:0/1079708045 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f62040a3bb0 msgr2=0x7f62040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:00.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.656+0000 7f621f200640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f62040a3bb0 0x7f62040a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6210031440 tx=0x7f62100315b0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.656+0000 7f621f200640 1 -- 172.21.15.83:0/1079708045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62201052b0 msgr2=0x7f62201986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:00.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.656+0000 7f621f200640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62201052b0 0x7f62201986e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f620800e9c0 tx=0x7f620800ee80 comp rx=0 tx=0).stop 2024-04-23T14:46:00.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.656+0000 7f621f200640 1 -- 172.21.15.83:0/1079708045 shutdown_connections 2024-04-23T14:46:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.656+0000 7f621f200640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f62040a3bb0 0x7f62040a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.656+0000 7f621f200640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6220107be0 0x7f622019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.656+0000 7f621f200640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62201052b0 0x7f62201986e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.656+0000 7f621f200640 1 --2- 172.21.15.83:0/1079708045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6220102980 0x7f62201981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.656+0000 7f621f200640 1 -- 172.21.15.83:0/1079708045 >> 172.21.15.83:0/1079708045 conn(0x7f62200fc2d0 msgr2=0x7f6220103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.656+0000 7f621f200640 1 -- 172.21.15.83:0/1079708045 shutdown_connections 2024-04-23T14:46:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.656+0000 7f621f200640 1 -- 172.21.15.83:0/1079708045 wait complete. 2024-04-23T14:46:00.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:46:00.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:00 smithi203 ceph-mon[32800]: pgmap v314: 33 pgs: 14 creating+peering, 18 unknown, 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:46:00.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:00 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2583071439' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T14:46:00.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:00 smithi203 ceph-mon[32800]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T14:46:00.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:00 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3038784224' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T14:46:00.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:00 smithi203 ceph-mon[32800]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T14:46:00.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:00 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:46:00.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:00 smithi203 ceph-mon[32800]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T14:46:00.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:00 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:46:00.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:00 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:46:00.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:00 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:46:00.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:00 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:46:00.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:00 smithi203 ceph-mon[32800]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T14:46:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.864+0000 7f983d600640 1 -- 172.21.15.83:0/2890661238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9838104260 msgr2=0x7f9838104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.864+0000 7f983d600640 1 --2- 172.21.15.83:0/2890661238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9838104260 0x7f9838104680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9828009900 tx=0x7f982802a3f0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.864+0000 7f983d600640 1 -- 172.21.15.83:0/2890661238 shutdown_connections 2024-04-23T14:46:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.864+0000 7f983d600640 1 --2- 172.21.15.83:0/2890661238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9838105e10 0x7f983810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.864+0000 7f983d600640 1 --2- 172.21.15.83:0/2890661238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9838105450 0x7f98381058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.864+0000 7f983d600640 1 --2- 172.21.15.83:0/2890661238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9838104260 0x7f9838104680 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.864+0000 7f983d600640 1 -- 172.21.15.83:0/2890661238 >> 172.21.15.83:0/2890661238 conn(0x7f98380ff7f0 msgr2=0x7f9838101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:00.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.864+0000 7f983d600640 1 -- 172.21.15.83:0/2890661238 shutdown_connections 2024-04-23T14:46:00.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.864+0000 7f983d600640 1 -- 172.21.15.83:0/2890661238 wait complete. 2024-04-23T14:46:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.865+0000 7f983d600640 1 Processor -- start 2024-04-23T14:46:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.865+0000 7f983d600640 1 -- start start 2024-04-23T14:46:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.865+0000 7f983d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9838104260 0x7f983819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.865+0000 7f983d600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9838105450 0x7f983819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.865+0000 7f983d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9838105e10 0x7f98381a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.865+0000 7f983d600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f983810ff10 con 0x7f9838105e10 2024-04-23T14:46:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.865+0000 7f983d600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f983810fd90 con 0x7f9838105450 2024-04-23T14:46:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.865+0000 7f983d600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9838110090 con 0x7f9838104260 2024-04-23T14:46:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.865+0000 7f9837400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9838105e10 0x7f98381a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.865+0000 7f9837400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9838105e10 0x7f98381a1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58942/0 (socket says 172.21.15.83:58942) 2024-04-23T14:46:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.865+0000 7f9837400640 1 -- 172.21.15.83:0/3927463403 learned_addr learned my addr 172.21.15.83:0/3927463403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.866+0000 7f9836a00640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9838104260 0x7f983819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:00.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.866+0000 7f9836000640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9838105450 0x7f983819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:00.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.866+0000 7f9837400640 1 -- 172.21.15.83:0/3927463403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9838104260 msgr2=0x7f983819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.866+0000 7f9837400640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9838104260 0x7f983819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.866+0000 7f9837400640 1 -- 172.21.15.83:0/3927463403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9838105450 msgr2=0x7f983819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.866+0000 7f9837400640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9838105450 0x7f983819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.866+0000 7f9837400640 1 -- 172.21.15.83:0/3927463403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98381a21d0 con 0x7f9838105e10 2024-04-23T14:46:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.866+0000 7f9836a00640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9838104260 0x7f983819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.866+0000 7f9837400640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9838105e10 0x7f98381a1b60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f983000eaa0 tx=0x7f983000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.867+0000 7f982f400640 1 -- 172.21.15.83:0/3927463403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983000cdb0 con 0x7f9838105e10 2024-04-23T14:46:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.867+0000 7f982f400640 1 -- 172.21.15.83:0/3927463403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9830005020 con 0x7f9838105e10 2024-04-23T14:46:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.867+0000 7f983d600640 1 -- 172.21.15.83:0/3927463403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98381a24c0 con 0x7f9838105e10 2024-04-23T14:46:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.867+0000 7f982f400640 1 -- 172.21.15.83:0/3927463403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98300116b0 con 0x7f9838105e10 2024-04-23T14:46:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.867+0000 7f983d600640 1 -- 172.21.15.83:0/3927463403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98381a2a10 con 0x7f9838105e10 2024-04-23T14:46:00.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.868+0000 7f983d600640 1 -- 172.21.15.83:0/3927463403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f983810b580 con 0x7f9838105e10 2024-04-23T14:46:00.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.872+0000 7f982f400640 1 -- 172.21.15.83:0/3927463403 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9830005560 con 0x7f9838105e10 2024-04-23T14:46:00.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.874+0000 7f982f400640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f981c0a3c30 0x7f981c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:00.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.874+0000 7f982f400640 1 -- 172.21.15.83:0/3927463403 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f98300d03f0 con 0x7f9838105e10 2024-04-23T14:46:00.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.874+0000 7f9836a00640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f981c0a3c30 0x7f981c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:00.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.875+0000 7f9836a00640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f981c0a3c30 0x7f981c0a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f982802a9e0 tx=0x7f98280315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:00.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:00.879+0000 7f982f400640 1 -- 172.21.15.83:0/3927463403 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9830016030 con 0x7f9838105e10 2024-04-23T14:46:01.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.093+0000 7f983d600640 1 -- 172.21.15.83:0/3927463403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9838062c00 con 0x7f9838105e10 2024-04-23T14:46:01.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.095+0000 7f982f400640 1 -- 172.21.15.83:0/3927463403 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22219 (secure 0 0 0) 0x7f983009c8d0 con 0x7f9838105e10 2024-04-23T14:46:01.095 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:46:01.096 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:46:00.988678+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:46:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.097+0000 7f983d600640 1 -- 172.21.15.83:0/3927463403 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f981c0a3c30 msgr2=0x7f981c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.097+0000 7f983d600640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f981c0a3c30 0x7f981c0a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f982802a9e0 tx=0x7f98280315b0 comp rx=0 tx=0).stop 2024-04-23T14:46:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.097+0000 7f983d600640 1 -- 172.21.15.83:0/3927463403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9838105e10 msgr2=0x7f98381a1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.097+0000 7f983d600640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9838105e10 0x7f98381a1b60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f983000eaa0 tx=0x7f983000ef60 comp rx=0 tx=0).stop 2024-04-23T14:46:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.097+0000 7f983d600640 1 -- 172.21.15.83:0/3927463403 shutdown_connections 2024-04-23T14:46:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.097+0000 7f983d600640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f981c0a3c30 0x7f981c0a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.097+0000 7f983d600640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9838105e10 0x7f98381a1b60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.097+0000 7f983d600640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9838105450 0x7f983819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:01.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.097+0000 7f983d600640 1 --2- 172.21.15.83:0/3927463403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9838104260 0x7f983819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:01.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.097+0000 7f983d600640 1 -- 172.21.15.83:0/3927463403 >> 172.21.15.83:0/3927463403 conn(0x7f98380ff7f0 msgr2=0x7f9838108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:01.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.098+0000 7f983d600640 1 -- 172.21.15.83:0/3927463403 shutdown_connections 2024-04-23T14:46:01.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.098+0000 7f983d600640 1 -- 172.21.15.83:0/3927463403 wait complete. 2024-04-23T14:46:01.118 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T14:46:01.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.299+0000 7f04baa00640 1 -- 172.21.15.83:0/4255281727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04b4101f90 msgr2=0x7f04b41023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:01.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.299+0000 7f04baa00640 1 --2- 172.21.15.83:0/4255281727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04b4101f90 0x7f04b41023b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009900 tx=0x7f04a802a3f0 comp rx=0 tx=0).stop 2024-04-23T14:46:01.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.299+0000 7f04baa00640 1 -- 172.21.15.83:0/4255281727 shutdown_connections 2024-04-23T14:46:01.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.299+0000 7f04baa00640 1 --2- 172.21.15.83:0/4255281727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04b4103b40 0x7f04b410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:01.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.299+0000 7f04baa00640 1 --2- 172.21.15.83:0/4255281727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04b4103180 0x7f04b4103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:01.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.299+0000 7f04baa00640 1 --2- 172.21.15.83:0/4255281727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04b4101f90 0x7f04b41023b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:01.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.299+0000 7f04baa00640 1 -- 172.21.15.83:0/4255281727 >> 172.21.15.83:0/4255281727 conn(0x7f04b40fd500 msgr2=0x7f04b40ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:01.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.299+0000 7f04baa00640 1 -- 172.21.15.83:0/4255281727 shutdown_connections 2024-04-23T14:46:01.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.299+0000 7f04baa00640 1 -- 172.21.15.83:0/4255281727 wait complete. 2024-04-23T14:46:01.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.300+0000 7f04baa00640 1 Processor -- start 2024-04-23T14:46:01.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.300+0000 7f04baa00640 1 -- start start 2024-04-23T14:46:01.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.300+0000 7f04baa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04b4101f90 0x7f04b419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:01.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.300+0000 7f04baa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04b4103180 0x7f04b419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:01.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.300+0000 7f04baa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04b4103b40 0x7f04b41a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:01.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.300+0000 7f04baa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04b410dc00 con 0x7f04b4103180 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.300+0000 7f04baa00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f04b410da80 con 0x7f04b4103b40 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.300+0000 7f04baa00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f04b410dd80 con 0x7f04b4101f90 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04b9600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04b4101f90 0x7f04b419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04b8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04b4103180 0x7f04b419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04b9600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04b4101f90 0x7f04b419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:58738/0 (socket says 172.21.15.83:58738) 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04b9600640 1 -- 172.21.15.83:0/2777144228 learned_addr learned my addr 172.21.15.83:0/2777144228 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04ba000640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04b4103b40 0x7f04b41a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04b9600640 1 -- 172.21.15.83:0/2777144228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04b4103b40 msgr2=0x7f04b41a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04b9600640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04b4103b40 0x7f04b41a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04b9600640 1 -- 172.21.15.83:0/2777144228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04b4103180 msgr2=0x7f04b419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04b9600640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04b4103180 0x7f04b419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04b9600640 1 -- 172.21.15.83:0/2777144228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b41a2070 con 0x7f04b4101f90 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04b8c00640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04b4103180 0x7f04b419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04b9600640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04b4101f90 0x7f04b419a450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f04a8031440 tx=0x7f04a8032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.301+0000 7f04b2000640 1 -- 172.21.15.83:0/2777144228 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a803f030 con 0x7f04b4101f90 2024-04-23T14:46:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.302+0000 7f04b2000640 1 -- 172.21.15.83:0/2777144228 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04a8003ce0 con 0x7f04b4101f90 2024-04-23T14:46:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.302+0000 7f04b2000640 1 -- 172.21.15.83:0/2777144228 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8035070 con 0x7f04b4101f90 2024-04-23T14:46:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.302+0000 7f04baa00640 1 -- 172.21.15.83:0/2777144228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b41a2300 con 0x7f04b4101f90 2024-04-23T14:46:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.302+0000 7f04baa00640 1 -- 172.21.15.83:0/2777144228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b41a27a0 con 0x7f04b4101f90 2024-04-23T14:46:01.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.303+0000 7f04baa00640 1 -- 172.21.15.83:0/2777144228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b41a2a00 con 0x7f04b4101f90 2024-04-23T14:46:01.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.305+0000 7f04b2000640 1 -- 172.21.15.83:0/2777144228 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f04a8042030 con 0x7f04b4101f90 2024-04-23T14:46:01.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.306+0000 7f04b2000640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f049c0a3c30 0x7f049c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:01.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.306+0000 7f04b2000640 1 -- 172.21.15.83:0/2777144228 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f04a80f0390 con 0x7f04b4101f90 2024-04-23T14:46:01.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.306+0000 7f04b8c00640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f049c0a3c30 0x7f049c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:01.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.307+0000 7f04b8c00640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f049c0a3c30 0x7f049c0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f04a00095f0 tx=0x7f04a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:01.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.311+0000 7f04b2000640 1 -- 172.21.15.83:0/2777144228 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04a80bc7d0 con 0x7f04b4101f90 2024-04-23T14:46:01.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:01 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3038784224' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T14:46:01.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:01 smithi083 ceph-mon[27867]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T14:46:01.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:01 smithi083 ceph-mon[27867]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T14:46:01.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:01 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1079708045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:01.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:01 smithi083 ceph-mon[27867]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T14:46:01.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:01 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3927463403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:01.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:01.554+0000 7f04baa00640 1 -- 172.21.15.83:0/2777144228 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f04b41a2bd0 con 0x7f04b4101f90 2024-04-23T14:46:01.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:01 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3038784224' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T14:46:01.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:01 smithi173 ceph-mon[32443]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T14:46:01.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:01 smithi173 ceph-mon[32443]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T14:46:01.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:01 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1079708045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:01.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:01 smithi173 ceph-mon[32443]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T14:46:01.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:01 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3927463403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:01.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:01 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3038784224' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T14:46:01.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:01 smithi203 ceph-mon[32800]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T14:46:01.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:01 smithi203 ceph-mon[32800]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T14:46:01.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:01 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1079708045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:01.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:01 smithi203 ceph-mon[32800]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T14:46:01.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:01 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3927463403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:02.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:02 smithi083 ceph-mon[27867]: pgmap v318: 97 pgs: 14 creating+peering, 68 unknown, 15 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2024-04-23T14:46:02.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:02 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] up:active 2024-04-23T14:46:02.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:02 smithi083 ceph-mon[27867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T14:46:02.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:02 smithi083 ceph-mon[27867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T14:46:02.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:02 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2777144228' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T14:46:02.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:02 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T14:46:02.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:02 smithi083 ceph-mon[27867]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T14:46:02.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:02 smithi083 ceph-mon[27867]: Cluster is now healthy 2024-04-23T14:46:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.258+0000 7f04b2000640 1 -- 172.21.15.83:0/2777144228 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f04a80c0ad0 con 0x7f04b4101f90 2024-04-23T14:46:02.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.261+0000 7f04baa00640 1 -- 172.21.15.83:0/2777144228 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f049c0a3c30 msgr2=0x7f049c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:02.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.261+0000 7f04baa00640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f049c0a3c30 0x7f049c0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f04a00095f0 tx=0x7f04a0005f90 comp rx=0 tx=0).stop 2024-04-23T14:46:02.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.261+0000 7f04baa00640 1 -- 172.21.15.83:0/2777144228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04b4101f90 msgr2=0x7f04b419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:02.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.261+0000 7f04baa00640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04b4101f90 0x7f04b419a450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f04a8031440 tx=0x7f04a8032ce0 comp rx=0 tx=0).stop 2024-04-23T14:46:02.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.261+0000 7f04baa00640 1 -- 172.21.15.83:0/2777144228 shutdown_connections 2024-04-23T14:46:02.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.261+0000 7f04baa00640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f049c0a3c30 0x7f049c0a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:02.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.262+0000 7f04baa00640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f04b4103b40 0x7f04b41a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:02.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.262+0000 7f04baa00640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04b4103180 0x7f04b419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:02.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.262+0000 7f04baa00640 1 --2- 172.21.15.83:0/2777144228 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04b4101f90 0x7f04b419a450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:02.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.262+0000 7f04baa00640 1 -- 172.21.15.83:0/2777144228 >> 172.21.15.83:0/2777144228 conn(0x7f04b40fd500 msgr2=0x7f04b4074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:02.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.262+0000 7f04baa00640 1 -- 172.21.15.83:0/2777144228 shutdown_connections 2024-04-23T14:46:02.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.262+0000 7f04baa00640 1 -- 172.21.15.83:0/2777144228 wait complete. 2024-04-23T14:46:02.285 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-23T14:46:02.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-23T14:46:02.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.492+0000 7fd9dd600640 1 -- 172.21.15.83:0/1235746331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d8101a90 msgr2=0x7fd9d8101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:02.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.492+0000 7fd9dd600640 1 --2- 172.21.15.83:0/1235746331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d8101a90 0x7fd9d8101f10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009ca0 tx=0x7fd9d002a8c0 comp rx=0 tx=0).stop 2024-04-23T14:46:02.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.492+0000 7fd9dd600640 1 -- 172.21.15.83:0/1235746331 shutdown_connections 2024-04-23T14:46:02.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.492+0000 7fd9dd600640 1 --2- 172.21.15.83:0/1235746331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d8101a90 0x7fd9d8101f10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:02.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.492+0000 7fd9dd600640 1 --2- 172.21.15.83:0/1235746331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d81010f0 0x7fd9d8101550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:02.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.492+0000 7fd9dd600640 1 --2- 172.21.15.83:0/1235746331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9d8100790 0x7fd9d8100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:02.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.492+0000 7fd9dd600640 1 -- 172.21.15.83:0/1235746331 >> 172.21.15.83:0/1235746331 conn(0x7fd9d80fc370 msgr2=0x7fd9d80fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:02.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.492+0000 7fd9dd600640 1 -- 172.21.15.83:0/1235746331 shutdown_connections 2024-04-23T14:46:02.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.492+0000 7fd9dd600640 1 -- 172.21.15.83:0/1235746331 wait complete. 2024-04-23T14:46:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.493+0000 7fd9dd600640 1 Processor -- start 2024-04-23T14:46:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.493+0000 7fd9dd600640 1 -- start start 2024-04-23T14:46:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9dd600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d8100790 0x7fd9d819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9dd600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d81010f0 0x7fd9d819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9dd600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9d8101a90 0x7fd9d81a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9dd600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d810fe10 con 0x7fd9d8100790 2024-04-23T14:46:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9dd600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d810fc90 con 0x7fd9d8101a90 2024-04-23T14:46:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9dd600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d810ff90 con 0x7fd9d81010f0 2024-04-23T14:46:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9d7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d8100790 0x7fd9d819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9d7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d8100790 0x7fd9d819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58984/0 (socket says 172.21.15.83:58984) 2024-04-23T14:46:02.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9d7e00640 1 -- 172.21.15.83:0/161865586 learned_addr learned my addr 172.21.15.83:0/161865586 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9dcc00640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9d8101a90 0x7fd9d81a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9d7400640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d81010f0 0x7fd9d819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9d7e00640 1 -- 172.21.15.83:0/161865586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d81010f0 msgr2=0x7fd9d819a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9d7e00640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d81010f0 0x7fd9d819a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9d7e00640 1 -- 172.21.15.83:0/161865586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9d8101a90 msgr2=0x7fd9d81a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9d7e00640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9d8101a90 0x7fd9d81a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.494+0000 7fd9d7e00640 1 -- 172.21.15.83:0/161865586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d81a1fd0 con 0x7fd9d8100790 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.495+0000 7fd9d7e00640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d8100790 0x7fd9d819a3b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800e9f0 tx=0x7fd9c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.495+0000 7fd9d4c00640 1 -- 172.21.15.83:0/161865586 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c800cd80 con 0x7fd9d8100790 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.495+0000 7fd9d4c00640 1 -- 172.21.15.83:0/161865586 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c8010910 con 0x7fd9d8100790 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.495+0000 7fd9dd600640 1 -- 172.21.15.83:0/161865586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d81a22c0 con 0x7fd9d8100790 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.495+0000 7fd9d4c00640 1 -- 172.21.15.83:0/161865586 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8018990 con 0x7fd9d8100790 2024-04-23T14:46:02.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.495+0000 7fd9dd600640 1 -- 172.21.15.83:0/161865586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d81a2860 con 0x7fd9d8100790 2024-04-23T14:46:02.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.496+0000 7fd9dd600640 1 -- 172.21.15.83:0/161865586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d810b4e0 con 0x7fd9d8100790 2024-04-23T14:46:02.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.498+0000 7fd9d4c00640 1 -- 172.21.15.83:0/161865586 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd9c8010ac0 con 0x7fd9d8100790 2024-04-23T14:46:02.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.499+0000 7fd9d4c00640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd9bc0a3c30 0x7fd9bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:02.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.500+0000 7fd9d4c00640 1 -- 172.21.15.83:0/161865586 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd9c80d09c0 con 0x7fd9d8100790 2024-04-23T14:46:02.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.500+0000 7fd9d7400640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd9bc0a3c30 0x7fd9bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:02.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.500+0000 7fd9d7400640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd9bc0a3c30 0x7fd9bc0a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd9d819b8c0 tx=0x7fd9c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:02.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.502+0000 7fd9d4c00640 1 -- 172.21.15.83:0/161865586 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9c809ce00 con 0x7fd9d8100790 2024-04-23T14:46:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:02 smithi173 ceph-mon[32443]: pgmap v318: 97 pgs: 14 creating+peering, 68 unknown, 15 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2024-04-23T14:46:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:02 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] up:active 2024-04-23T14:46:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:02 smithi173 ceph-mon[32443]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T14:46:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:02 smithi173 ceph-mon[32443]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T14:46:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:02 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2777144228' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T14:46:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:02 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T14:46:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:02 smithi173 ceph-mon[32443]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T14:46:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:02 smithi173 ceph-mon[32443]: Cluster is now healthy 2024-04-23T14:46:02.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:02 smithi203 ceph-mon[32800]: pgmap v318: 97 pgs: 14 creating+peering, 68 unknown, 15 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2024-04-23T14:46:02.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:02 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] up:active 2024-04-23T14:46:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:02 smithi203 ceph-mon[32800]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T14:46:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:02 smithi203 ceph-mon[32800]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T14:46:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:02 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2777144228' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T14:46:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:02 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T14:46:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:02 smithi203 ceph-mon[32800]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T14:46:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:02 smithi203 ceph-mon[32800]: Cluster is now healthy 2024-04-23T14:46:02.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:02.728+0000 7fd9dd600640 1 -- 172.21.15.83:0/161865586 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fd9d8062c00 con 0x7fd9d8100790 2024-04-23T14:46:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.269+0000 7fd9d4c00640 1 -- 172.21.15.83:0/161865586 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fd9c80a1100 con 0x7fd9d8100790 2024-04-23T14:46:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.274+0000 7fd9dd600640 1 -- 172.21.15.83:0/161865586 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd9bc0a3c30 msgr2=0x7fd9bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.274+0000 7fd9dd600640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd9bc0a3c30 0x7fd9bc0a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd9d819b8c0 tx=0x7fd9c4005f90 comp rx=0 tx=0).stop 2024-04-23T14:46:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.274+0000 7fd9dd600640 1 -- 172.21.15.83:0/161865586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d8100790 msgr2=0x7fd9d819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:03.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.274+0000 7fd9dd600640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d8100790 0x7fd9d819a3b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800e9f0 tx=0x7fd9c800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:46:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.274+0000 7fd9dd600640 1 -- 172.21.15.83:0/161865586 shutdown_connections 2024-04-23T14:46:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.274+0000 7fd9dd600640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd9bc0a3c30 0x7fd9bc0a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.274+0000 7fd9dd600640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9d8101a90 0x7fd9d81a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.274+0000 7fd9dd600640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d81010f0 0x7fd9d819a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.274+0000 7fd9dd600640 1 --2- 172.21.15.83:0/161865586 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d8100790 0x7fd9d819a3b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.274+0000 7fd9dd600640 1 -- 172.21.15.83:0/161865586 >> 172.21.15.83:0/161865586 conn(0x7fd9d80fc370 msgr2=0x7fd9d80fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.274+0000 7fd9dd600640 1 -- 172.21.15.83:0/161865586 shutdown_connections 2024-04-23T14:46:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.274+0000 7fd9dd600640 1 -- 172.21.15.83:0/161865586 wait complete. 2024-04-23T14:46:03.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-23T14:46:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.510+0000 7fe214e00640 1 -- 172.21.15.83:0/1958536640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe210105470 msgr2=0x7fe2101058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.510+0000 7fe214e00640 1 --2- 172.21.15.83:0/1958536640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe210105470 0x7fe2101058f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009860 tx=0x7fe1f802a340 comp rx=0 tx=0).stop 2024-04-23T14:46:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.510+0000 7fe214e00640 1 -- 172.21.15.83:0/1958536640 shutdown_connections 2024-04-23T14:46:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.510+0000 7fe214e00640 1 --2- 172.21.15.83:0/1958536640 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210105e30 0x7fe21010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.510+0000 7fe214e00640 1 --2- 172.21.15.83:0/1958536640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe210105470 0x7fe2101058f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.510+0000 7fe214e00640 1 --2- 172.21.15.83:0/1958536640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe210104280 0x7fe2101046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.510+0000 7fe214e00640 1 -- 172.21.15.83:0/1958536640 >> 172.21.15.83:0/1958536640 conn(0x7fe2100ff7f0 msgr2=0x7fe210101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.511+0000 7fe214e00640 1 -- 172.21.15.83:0/1958536640 shutdown_connections 2024-04-23T14:46:03.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.511+0000 7fe214e00640 1 -- 172.21.15.83:0/1958536640 wait complete. 2024-04-23T14:46:03.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.512+0000 7fe214e00640 1 Processor -- start 2024-04-23T14:46:03.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.512+0000 7fe214e00640 1 -- start start 2024-04-23T14:46:03.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.512+0000 7fe214e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe210104280 0x7fe21019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:03.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.512+0000 7fe214e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe210105470 0x7fe21019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:03.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.512+0000 7fe214e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210105e30 0x7fe2101a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:03.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.512+0000 7fe214e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe21010ff10 con 0x7fe210105e30 2024-04-23T14:46:03.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.512+0000 7fe214e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe21010fd90 con 0x7fe210104280 2024-04-23T14:46:03.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.512+0000 7fe214e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe210110090 con 0x7fe210105470 2024-04-23T14:46:03.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.513+0000 7fe20de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe210104280 0x7fe21019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:03.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.513+0000 7fe20de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe210104280 0x7fe21019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:50328/0 (socket says 172.21.15.83:50328) 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.513+0000 7fe20de00640 1 -- 172.21.15.83:0/3146086080 learned_addr learned my addr 172.21.15.83:0/3146086080 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.513+0000 7fe20e800640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210105e30 0x7fe2101a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.513+0000 7fe205400640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe210105470 0x7fe21019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.513+0000 7fe20de00640 1 -- 172.21.15.83:0/3146086080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe210105470 msgr2=0x7fe21019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.513+0000 7fe20de00640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe210105470 0x7fe21019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.513+0000 7fe20de00640 1 -- 172.21.15.83:0/3146086080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210105e30 msgr2=0x7fe2101a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.513+0000 7fe20de00640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210105e30 0x7fe2101a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.513+0000 7fe20de00640 1 -- 172.21.15.83:0/3146086080 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2101a2190 con 0x7fe210104280 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.513+0000 7fe20de00640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe210104280 0x7fe21019a570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe20000e9c0 tx=0x7fe20000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.513+0000 7fe207400640 1 -- 172.21.15.83:0/3146086080 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20000cde0 con 0x7fe210104280 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.514+0000 7fe207400640 1 -- 172.21.15.83:0/3146086080 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe200010910 con 0x7fe210104280 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.514+0000 7fe214e00640 1 -- 172.21.15.83:0/3146086080 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2101a2480 con 0x7fe210104280 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.514+0000 7fe207400640 1 -- 172.21.15.83:0/3146086080 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200018a00 con 0x7fe210104280 2024-04-23T14:46:03.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.514+0000 7fe214e00640 1 -- 172.21.15.83:0/3146086080 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2101a29a0 con 0x7fe210104280 2024-04-23T14:46:03.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.515+0000 7fe214e00640 1 -- 172.21.15.83:0/3146086080 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe21010b670 con 0x7fe210104280 2024-04-23T14:46:03.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.517+0000 7fe207400640 1 -- 172.21.15.83:0/3146086080 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe20001f020 con 0x7fe210104280 2024-04-23T14:46:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.518+0000 7fe207400640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe1f00a3bb0 0x7fe1f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.518+0000 7fe207400640 1 -- 172.21.15.83:0/3146086080 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe2000d0770 con 0x7fe210104280 2024-04-23T14:46:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.518+0000 7fe205400640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe1f00a3bb0 0x7fe1f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:03.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.519+0000 7fe205400640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe1f00a3bb0 0x7fe1f00a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009830 tx=0x7fe1f8030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:03.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.521+0000 7fe207400640 1 -- 172.21.15.83:0/3146086080 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe20009d200 con 0x7fe210104280 2024-04-23T14:46:03.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:03 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T14:46:03.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:03 smithi173 ceph-mon[32443]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T14:46:03.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:03 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/161865586' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-23T14:46:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:03 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T14:46:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:03 smithi203 ceph-mon[32800]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T14:46:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:03 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/161865586' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-23T14:46:03.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:03 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T14:46:03.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:03 smithi083 ceph-mon[27867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T14:46:03.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:03 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/161865586' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-23T14:46:03.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:03.763+0000 7fe214e00640 1 -- 172.21.15.83:0/3146086080 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe21002d620 con 0x7fe210104280 2024-04-23T14:46:04.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.285+0000 7fe207400640 1 -- 172.21.15.83:0/3146086080 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7fe20009cb00 con 0x7fe210104280 2024-04-23T14:46:04.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.289+0000 7fe214e00640 1 -- 172.21.15.83:0/3146086080 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe1f00a3bb0 msgr2=0x7fe1f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:04.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.289+0000 7fe214e00640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe1f00a3bb0 0x7fe1f00a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009830 tx=0x7fe1f8030550 comp rx=0 tx=0).stop 2024-04-23T14:46:04.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.289+0000 7fe214e00640 1 -- 172.21.15.83:0/3146086080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe210104280 msgr2=0x7fe21019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:04.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.289+0000 7fe214e00640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe210104280 0x7fe21019a570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe20000e9c0 tx=0x7fe20000ee80 comp rx=0 tx=0).stop 2024-04-23T14:46:04.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.289+0000 7fe214e00640 1 -- 172.21.15.83:0/3146086080 shutdown_connections 2024-04-23T14:46:04.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.289+0000 7fe214e00640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fe1f00a3bb0 0x7fe1f00a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:04.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.289+0000 7fe214e00640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210105e30 0x7fe2101a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:04.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.289+0000 7fe214e00640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe210105470 0x7fe21019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:04.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.289+0000 7fe214e00640 1 --2- 172.21.15.83:0/3146086080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe210104280 0x7fe21019a570 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:04.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.289+0000 7fe214e00640 1 -- 172.21.15.83:0/3146086080 >> 172.21.15.83:0/3146086080 conn(0x7fe2100ff7f0 msgr2=0x7fe210108840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:04.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.289+0000 7fe214e00640 1 -- 172.21.15.83:0/3146086080 shutdown_connections 2024-04-23T14:46:04.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.290+0000 7fe214e00640 1 -- 172.21.15.83:0/3146086080 wait complete. 2024-04-23T14:46:04.317 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-23T14:46:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.510+0000 7f808fe00640 1 -- 172.21.15.83:0/1322658769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80901010e0 msgr2=0x7f8090101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.510+0000 7f808fe00640 1 --2- 172.21.15.83:0/1322658769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80901010e0 0x7f8090101540 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8078009ca0 tx=0x7f807802a8c0 comp rx=0 tx=0).stop 2024-04-23T14:46:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.510+0000 7f808fe00640 1 -- 172.21.15.83:0/1322658769 shutdown_connections 2024-04-23T14:46:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.510+0000 7f808fe00640 1 --2- 172.21.15.83:0/1322658769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8090101a80 0x7f8090101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.510+0000 7f808fe00640 1 --2- 172.21.15.83:0/1322658769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80901010e0 0x7f8090101540 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.510+0000 7f808fe00640 1 --2- 172.21.15.83:0/1322658769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8090100780 0x7f8090100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.510+0000 7f808fe00640 1 -- 172.21.15.83:0/1322658769 >> 172.21.15.83:0/1322658769 conn(0x7f80900fc3a0 msgr2=0x7f80900fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.510+0000 7f808fe00640 1 -- 172.21.15.83:0/1322658769 shutdown_connections 2024-04-23T14:46:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.510+0000 7f808fe00640 1 -- 172.21.15.83:0/1322658769 wait complete. 2024-04-23T14:46:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.511+0000 7f808fe00640 1 Processor -- start 2024-04-23T14:46:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.511+0000 7f808fe00640 1 -- start start 2024-04-23T14:46:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.511+0000 7f808fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8090100780 0x7f809019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.511+0000 7f808fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80901010e0 0x7f809019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.511+0000 7f808fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8090101a80 0x7f80901a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.511+0000 7f808fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f809010fd80 con 0x7f80901010e0 2024-04-23T14:46:04.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.511+0000 7f808fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f809010fc00 con 0x7f8090100780 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.511+0000 7f808fe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f809010ff00 con 0x7f8090101a80 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.511+0000 7f808ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8090100780 0x7f809019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.511+0000 7f808e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80901010e0 0x7f809019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.511+0000 7f808ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8090100780 0x7f809019a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:50338/0 (socket says 172.21.15.83:50338) 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.511+0000 7f808ea00640 1 -- 172.21.15.83:0/2019427411 learned_addr learned my addr 172.21.15.83:0/2019427411 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.512+0000 7f808f400640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8090101a80 0x7f80901a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.512+0000 7f808ea00640 1 -- 172.21.15.83:0/2019427411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8090101a80 msgr2=0x7f80901a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.512+0000 7f808ea00640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8090101a80 0x7f80901a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.512+0000 7f808ea00640 1 -- 172.21.15.83:0/2019427411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80901010e0 msgr2=0x7f809019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.512+0000 7f808ea00640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80901010e0 0x7f809019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.512+0000 7f808ea00640 1 -- 172.21.15.83:0/2019427411 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80901a2090 con 0x7f8090100780 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.512+0000 7f808e000640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80901010e0 0x7f809019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.512+0000 7f808ea00640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8090100780 0x7f809019a470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f808000e9f0 tx=0x7f808000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.512+0000 7f8094a00640 1 -- 172.21.15.83:0/2019427411 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808000cd80 con 0x7f8090100780 2024-04-23T14:46:04.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.512+0000 7f8094a00640 1 -- 172.21.15.83:0/2019427411 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8080010910 con 0x7f8090100780 2024-04-23T14:46:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.512+0000 7f808fe00640 1 -- 172.21.15.83:0/2019427411 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80901a2380 con 0x7f8090100780 2024-04-23T14:46:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.513+0000 7f8094a00640 1 -- 172.21.15.83:0/2019427411 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080018990 con 0x7f8090100780 2024-04-23T14:46:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.513+0000 7f808fe00640 1 -- 172.21.15.83:0/2019427411 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80901a28d0 con 0x7f8090100780 2024-04-23T14:46:04.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.514+0000 7f808fe00640 1 -- 172.21.15.83:0/2019427411 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f809010b4d0 con 0x7f8090100780 2024-04-23T14:46:04.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.516+0000 7f8094a00640 1 -- 172.21.15.83:0/2019427411 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8080010ac0 con 0x7f8090100780 2024-04-23T14:46:04.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.517+0000 7f8094a00640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80740a3bb0 0x7f80740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:04.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.517+0000 7f8094a00640 1 -- 172.21.15.83:0/2019427411 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f80800d0850 con 0x7f8090100780 2024-04-23T14:46:04.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.517+0000 7f808e000640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80740a3bb0 0x7f80740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:04.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.518+0000 7f808e000640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80740a3bb0 0x7f80740a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f809019b980 tx=0x7f8078000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.523+0000 7f8094a00640 1 -- 172.21.15.83:0/2019427411 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f808009cc90 con 0x7f8090100780 2024-04-23T14:46:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:04 smithi173 ceph-mon[32443]: pgmap v319: 97 pgs: 14 creating+peering, 22 unknown, 61 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 4 op/s 2024-04-23T14:46:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:04 smithi173 ceph-mon[32443]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T14:46:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:04 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/161865586' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-23T14:46:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:04 smithi173 ceph-mon[32443]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T14:46:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:04 smithi173 ceph-mon[32443]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T14:46:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:04 smithi173 ceph-mon[32443]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-23T14:46:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:04 smithi173 ceph-mon[32443]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-23T14:46:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:04 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3146086080' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T14:46:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:04 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T14:46:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:04 smithi203 ceph-mon[32800]: pgmap v319: 97 pgs: 14 creating+peering, 22 unknown, 61 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 4 op/s 2024-04-23T14:46:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:04 smithi203 ceph-mon[32800]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T14:46:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:04 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/161865586' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-23T14:46:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:04 smithi203 ceph-mon[32800]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T14:46:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:04 smithi203 ceph-mon[32800]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T14:46:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:04 smithi203 ceph-mon[32800]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-23T14:46:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:04 smithi203 ceph-mon[32800]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-23T14:46:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:04 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3146086080' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T14:46:04.623 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:04 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T14:46:04.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:04 smithi083 ceph-mon[27867]: pgmap v319: 97 pgs: 14 creating+peering, 22 unknown, 61 active+clean; 578 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 4 op/s 2024-04-23T14:46:04.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:04 smithi083 ceph-mon[27867]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T14:46:04.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:04 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/161865586' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-23T14:46:04.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:04 smithi083 ceph-mon[27867]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T14:46:04.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:04 smithi083 ceph-mon[27867]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T14:46:04.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:04 smithi083 ceph-mon[27867]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-23T14:46:04.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:04 smithi083 ceph-mon[27867]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-23T14:46:04.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:04 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3146086080' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T14:46:04.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:04 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T14:46:04.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:04.769+0000 7f808fe00640 1 -- 172.21.15.83:0/2019427411 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f809002d620 con 0x7f8090100780 2024-04-23T14:46:05.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.301+0000 7f8094a00640 1 -- 172.21.15.83:0/2019427411 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f80800a0f90 con 0x7f8090100780 2024-04-23T14:46:05.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.305+0000 7f808fe00640 1 -- 172.21.15.83:0/2019427411 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80740a3bb0 msgr2=0x7f80740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:05.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.305+0000 7f808fe00640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80740a3bb0 0x7f80740a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f809019b980 tx=0x7f8078000f40 comp rx=0 tx=0).stop 2024-04-23T14:46:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.305+0000 7f808fe00640 1 -- 172.21.15.83:0/2019427411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8090100780 msgr2=0x7f809019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.305+0000 7f808fe00640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8090100780 0x7f809019a470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f808000e9f0 tx=0x7f808000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.306+0000 7f808fe00640 1 -- 172.21.15.83:0/2019427411 shutdown_connections 2024-04-23T14:46:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.306+0000 7f808fe00640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f80740a3bb0 0x7f80740a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.306+0000 7f808fe00640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8090101a80 0x7f80901a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.306+0000 7f808fe00640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f80901010e0 0x7f809019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.306+0000 7f808fe00640 1 --2- 172.21.15.83:0/2019427411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8090100780 0x7f809019a470 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.306+0000 7f808fe00640 1 -- 172.21.15.83:0/2019427411 >> 172.21.15.83:0/2019427411 conn(0x7f80900fc3a0 msgr2=0x7f80900fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.306+0000 7f808fe00640 1 -- 172.21.15.83:0/2019427411 shutdown_connections 2024-04-23T14:46:05.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.306+0000 7f808fe00640 1 -- 172.21.15.83:0/2019427411 wait complete. 2024-04-23T14:46:05.328 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-23T14:46:05.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-23T14:46:05.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.538+0000 7f5b80200640 1 -- 172.21.15.83:0/113088456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b78074a60 msgr2=0x7f5b78074ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.538+0000 7f5b80200640 1 --2- 172.21.15.83:0/113088456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b78074a60 0x7f5b78074ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009900 tx=0x7f5b6402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.538+0000 7f5b80200640 1 -- 172.21.15.83:0/113088456 shutdown_connections 2024-04-23T14:46:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.538+0000 7f5b80200640 1 --2- 172.21.15.83:0/113088456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b78109f10 0x7f5b7810c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.538+0000 7f5b80200640 1 --2- 172.21.15.83:0/113088456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b78074a60 0x7f5b78074ee0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.538+0000 7f5b80200640 1 --2- 172.21.15.83:0/113088456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b78076040 0x7f5b78074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.538+0000 7f5b80200640 1 -- 172.21.15.83:0/113088456 >> 172.21.15.83:0/113088456 conn(0x7f5b780fd5e0 msgr2=0x7f5b780ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.538+0000 7f5b80200640 1 -- 172.21.15.83:0/113088456 shutdown_connections 2024-04-23T14:46:05.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.538+0000 7f5b80200640 1 -- 172.21.15.83:0/113088456 wait complete. 2024-04-23T14:46:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.539+0000 7f5b80200640 1 Processor -- start 2024-04-23T14:46:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.539+0000 7f5b80200640 1 -- start start 2024-04-23T14:46:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.539+0000 7f5b80200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b78074a60 0x7f5b7819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.539+0000 7f5b80200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b78076040 0x7f5b7819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.539+0000 7f5b80200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b78109f10 0x7f5b781a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:05.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.539+0000 7f5b80200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7810ff20 con 0x7f5b78074a60 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.539+0000 7f5b80200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7810fda0 con 0x7f5b78076040 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.539+0000 7f5b80200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b781100a0 con 0x7f5b78109f10 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.540+0000 7f5b7d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b78074a60 0x7f5b7819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.540+0000 7f5b7ce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b78076040 0x7f5b7819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.540+0000 7f5b7d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b78074a60 0x7f5b7819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:59972/0 (socket says 172.21.15.83:59972) 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.540+0000 7f5b7d800640 1 -- 172.21.15.83:0/1055321831 learned_addr learned my addr 172.21.15.83:0/1055321831 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.540+0000 7f5b7e200640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b78109f10 0x7f5b781a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.540+0000 7f5b7ce00640 1 -- 172.21.15.83:0/1055321831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b78109f10 msgr2=0x7f5b781a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.540+0000 7f5b7ce00640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b78109f10 0x7f5b781a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.540+0000 7f5b7ce00640 1 -- 172.21.15.83:0/1055321831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b78074a60 msgr2=0x7f5b7819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.540+0000 7f5b7ce00640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b78074a60 0x7f5b7819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.540+0000 7f5b7ce00640 1 -- 172.21.15.83:0/1055321831 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b781a2190 con 0x7f5b78076040 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.540+0000 7f5b7ce00640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b78076040 0x7f5b7819aab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5b6402a910 tx=0x7f5b64032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:05.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.541+0000 7f5b76000640 1 -- 172.21.15.83:0/1055321831 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6403f030 con 0x7f5b78076040 2024-04-23T14:46:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.541+0000 7f5b76000640 1 -- 172.21.15.83:0/1055321831 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b64003ce0 con 0x7f5b78076040 2024-04-23T14:46:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.541+0000 7f5b76000640 1 -- 172.21.15.83:0/1055321831 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64035070 con 0x7f5b78076040 2024-04-23T14:46:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.541+0000 7f5b80200640 1 -- 172.21.15.83:0/1055321831 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b781a2420 con 0x7f5b78076040 2024-04-23T14:46:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.541+0000 7f5b80200640 1 -- 172.21.15.83:0/1055321831 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b781a2930 con 0x7f5b78076040 2024-04-23T14:46:05.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.542+0000 7f5b80200640 1 -- 172.21.15.83:0/1055321831 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b78108c40 con 0x7f5b78076040 2024-04-23T14:46:05.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.545+0000 7f5b76000640 1 -- 172.21.15.83:0/1055321831 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b64042030 con 0x7f5b78076040 2024-04-23T14:46:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.546+0000 7f5b76000640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5b5c0a3c30 0x7f5b5c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.546+0000 7f5b76000640 1 -- 172.21.15.83:0/1055321831 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5b640f0ee0 con 0x7f5b78076040 2024-04-23T14:46:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.546+0000 7f5b7d800640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5b5c0a3c30 0x7f5b5c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:05.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.547+0000 7f5b7d800640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5b5c0a3c30 0x7f5b5c0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009fd0 tx=0x7f5b70009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:05.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.547+0000 7f5b76000640 1 -- 172.21.15.83:0/1055321831 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b640bd970 con 0x7f5b78076040 2024-04-23T14:46:05.576 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:46:05 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b[63123]: 2024-04-23T14:46:05.115+0000 7f4aabe09700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T14:46:05.576 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:46:05 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b[63123]: 2024-04-23T14:46:05.115+0000 7f4aabe09700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-23T14:46:05.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:05 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T14:46:05.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:05 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] up:active 2024-04-23T14:46:05.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:05 smithi173 ceph-mon[32443]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T14:46:05.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:05 smithi173 ceph-mon[32443]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T14:46:05.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:05 smithi173 ceph-mon[32443]: Cluster is now healthy 2024-04-23T14:46:05.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:05 smithi173 ceph-mon[32443]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-23T14:46:05.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:05 smithi173 ceph-mon[32443]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-23T14:46:05.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:05 smithi173 ceph-mon[32443]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-23T14:46:05.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:05 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2019427411' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T14:46:05.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:05 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T14:46:05.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:05 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T14:46:05.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:05 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] up:active 2024-04-23T14:46:05.635 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:05 smithi203 ceph-mon[32800]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T14:46:05.636 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:05 smithi203 ceph-mon[32800]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T14:46:05.636 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:05 smithi203 ceph-mon[32800]: Cluster is now healthy 2024-04-23T14:46:05.636 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:05 smithi203 ceph-mon[32800]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-23T14:46:05.636 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:05 smithi203 ceph-mon[32800]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-23T14:46:05.636 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:05 smithi203 ceph-mon[32800]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-23T14:46:05.636 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:05 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2019427411' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T14:46:05.636 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:05 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T14:46:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T14:46:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] up:active 2024-04-23T14:46:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-mon[27867]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T14:46:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-mon[27867]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T14:46:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-mon[27867]: Cluster is now healthy 2024-04-23T14:46:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-mon[27867]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-23T14:46:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-mon[27867]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-23T14:46:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-mon[27867]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-23T14:46:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2019427411' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T14:46:05.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T14:46:05.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.776+0000 7f5b80200640 1 -- 172.21.15.83:0/1055321831 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5b78062440 con 0x7f5b5c0a3c30 2024-04-23T14:46:05.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.812+0000 7f5b76000640 1 -- 172.21.15.83:0/1055321831 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5b78062440 con 0x7f5b5c0a3c30 2024-04-23T14:46:05.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.814+0000 7f5b80200640 1 -- 172.21.15.83:0/1055321831 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5b5c0a3c30 msgr2=0x7f5b5c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:05.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.814+0000 7f5b80200640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5b5c0a3c30 0x7f5b5c0a60f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009fd0 tx=0x7f5b70009720 comp rx=0 tx=0).stop 2024-04-23T14:46:05.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.815+0000 7f5b80200640 1 -- 172.21.15.83:0/1055321831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b78076040 msgr2=0x7f5b7819aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:05.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.815+0000 7f5b80200640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b78076040 0x7f5b7819aab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5b6402a910 tx=0x7f5b64032ce0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.815+0000 7f5b80200640 1 -- 172.21.15.83:0/1055321831 shutdown_connections 2024-04-23T14:46:05.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.815+0000 7f5b80200640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5b5c0a3c30 0x7f5b5c0a60f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.815+0000 7f5b80200640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b78109f10 0x7f5b781a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.815+0000 7f5b80200640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b78076040 0x7f5b7819aab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.815+0000 7f5b80200640 1 --2- 172.21.15.83:0/1055321831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b78074a60 0x7f5b7819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:05.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.815+0000 7f5b80200640 1 -- 172.21.15.83:0/1055321831 >> 172.21.15.83:0/1055321831 conn(0x7f5b780fd5e0 msgr2=0x7f5b78105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:05.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.815+0000 7f5b80200640 1 -- 172.21.15.83:0/1055321831 shutdown_connections 2024-04-23T14:46:05.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:05.815+0000 7f5b80200640 1 -- 172.21.15.83:0/1055321831 wait complete. 2024-04-23T14:46:05.835 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-23T14:46:06.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.037+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4181244755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1ec104280 msgr2=0x7fa1ec1046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:06.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.037+0000 7fa1f2c00640 1 --2- 172.21.15.83:0/4181244755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1ec104280 0x7fa1ec1046a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009900 tx=0x7fa1dc02a3f0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.037+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4181244755 shutdown_connections 2024-04-23T14:46:06.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.037+0000 7fa1f2c00640 1 --2- 172.21.15.83:0/4181244755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1ec105e30 0x7fa1ec10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.037+0000 7fa1f2c00640 1 --2- 172.21.15.83:0/4181244755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1ec105470 0x7fa1ec1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.037+0000 7fa1f2c00640 1 --2- 172.21.15.83:0/4181244755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1ec104280 0x7fa1ec1046a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.037+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4181244755 >> 172.21.15.83:0/4181244755 conn(0x7fa1ec0ff7f0 msgr2=0x7fa1ec101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:06.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.037+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4181244755 shutdown_connections 2024-04-23T14:46:06.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.037+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4181244755 wait complete. 2024-04-23T14:46:06.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.038+0000 7fa1f2c00640 1 Processor -- start 2024-04-23T14:46:06.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.039+0000 7fa1f2c00640 1 -- start start 2024-04-23T14:46:06.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.039+0000 7fa1f2c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1ec104280 0x7fa1ec19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:06.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.039+0000 7fa1f2c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1ec105470 0x7fa1ec19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:06.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.039+0000 7fa1f2c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1ec105e30 0x7fa1ec1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:06.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.039+0000 7fa1f2c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec10ff60 con 0x7fa1ec105470 2024-04-23T14:46:06.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.039+0000 7fa1f2c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec10fde0 con 0x7fa1ec104280 2024-04-23T14:46:06.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.039+0000 7fa1f2c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec1100e0 con 0x7fa1ec105e30 2024-04-23T14:46:06.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.039+0000 7fa1ebe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1ec104280 0x7fa1ec19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:06.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.039+0000 7fa1ebe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1ec104280 0x7fa1ec19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:55496/0 (socket says 172.21.15.83:55496) 2024-04-23T14:46:06.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.039+0000 7fa1eb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1ec105470 0x7fa1ec19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:06.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.039+0000 7fa1ebe00640 1 -- 172.21.15.83:0/4174682308 learned_addr learned my addr 172.21.15.83:0/4174682308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:06.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.039+0000 7fa1f0c00640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1ec105e30 0x7fa1ec1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:06.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.040+0000 7fa1ebe00640 1 -- 172.21.15.83:0/4174682308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1ec105e30 msgr2=0x7fa1ec1a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:06.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.040+0000 7fa1ebe00640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1ec105e30 0x7fa1ec1a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.040+0000 7fa1ebe00640 1 -- 172.21.15.83:0/4174682308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1ec105470 msgr2=0x7fa1ec19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:06.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.040+0000 7fa1ebe00640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1ec105470 0x7fa1ec19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.040+0000 7fa1ebe00640 1 -- 172.21.15.83:0/4174682308 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec1a21b0 con 0x7fa1ec104280 2024-04-23T14:46:06.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.040+0000 7fa1ebe00640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1ec104280 0x7fa1ec19a590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc031440 tx=0x7fa1dc032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:06.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.040+0000 7fa1e8c00640 1 -- 172.21.15.83:0/4174682308 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc03f030 con 0x7fa1ec104280 2024-04-23T14:46:06.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.040+0000 7fa1e8c00640 1 -- 172.21.15.83:0/4174682308 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1dc003ce0 con 0x7fa1ec104280 2024-04-23T14:46:06.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.040+0000 7fa1e8c00640 1 -- 172.21.15.83:0/4174682308 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc035070 con 0x7fa1ec104280 2024-04-23T14:46:06.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.040+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4174682308 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec1a2440 con 0x7fa1ec104280 2024-04-23T14:46:06.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.041+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4174682308 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec1a2980 con 0x7fa1ec104280 2024-04-23T14:46:06.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.041+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4174682308 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ec10b5a0 con 0x7fa1ec104280 2024-04-23T14:46:06.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.044+0000 7fa1e8c00640 1 -- 172.21.15.83:0/4174682308 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa1dc042030 con 0x7fa1ec104280 2024-04-23T14:46:06.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.045+0000 7fa1e8c00640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa1d00a3c30 0x7fa1d00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:06.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.045+0000 7fa1e8c00640 1 -- 172.21.15.83:0/4174682308 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa1dc0f08a0 con 0x7fa1ec104280 2024-04-23T14:46:06.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.045+0000 7fa1eb400640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa1d00a3c30 0x7fa1d00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:06.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.046+0000 7fa1eb400640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa1d00a3c30 0x7fa1d00a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec19baa0 tx=0x7fa1d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:06.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.047+0000 7fa1e8c00640 1 -- 172.21.15.83:0/4174682308 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1dc0bcce0 con 0x7fa1ec104280 2024-04-23T14:46:06.258 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:46:05.790+0000 7f9474c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:46:06.258 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:46:05.790+0000 7f9474c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:46:06.258 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:46:05.790+0000 7f9474c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:46:06.258 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:46:05.790+0000 7f9474c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:46:06.258 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:46:05 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:46:05.790+0000 7f9474c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:46:06.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.276+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4174682308 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa1ec0624d0 con 0x7fa1d00a3c30 2024-04-23T14:46:06.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.311+0000 7fa1e8c00640 1 -- 172.21.15.83:0/4174682308 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa1ec0624d0 con 0x7fa1d00a3c30 2024-04-23T14:46:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.313+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4174682308 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa1d00a3c30 msgr2=0x7fa1d00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.313+0000 7fa1f2c00640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa1d00a3c30 0x7fa1d00a60f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec19baa0 tx=0x7fa1d8005f90 comp rx=0 tx=0).stop 2024-04-23T14:46:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.313+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4174682308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1ec104280 msgr2=0x7fa1ec19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.313+0000 7fa1f2c00640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1ec104280 0x7fa1ec19a590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc031440 tx=0x7fa1dc032ce0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.313+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4174682308 shutdown_connections 2024-04-23T14:46:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.313+0000 7fa1f2c00640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa1d00a3c30 0x7fa1d00a60f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.313+0000 7fa1f2c00640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa1ec105e30 0x7fa1ec1a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.313+0000 7fa1f2c00640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1ec105470 0x7fa1ec19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.313+0000 7fa1f2c00640 1 --2- 172.21.15.83:0/4174682308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1ec104280 0x7fa1ec19a590 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.313+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4174682308 >> 172.21.15.83:0/4174682308 conn(0x7fa1ec0ff7f0 msgr2=0x7fa1ec108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.313+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4174682308 shutdown_connections 2024-04-23T14:46:06.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.314+0000 7fa1f2c00640 1 -- 172.21.15.83:0/4174682308 wait complete. 2024-04-23T14:46:06.331 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-23T14:46:06.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.530+0000 7f166f200640 1 -- 172.21.15.83:0/295003866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1670109730 msgr2=0x7f1670109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:06.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.530+0000 7f166f200640 1 --2- 172.21.15.83:0/295003866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1670109730 0x7f1670109bb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1658009900 tx=0x7f165802a3f0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.530+0000 7f166f200640 1 -- 172.21.15.83:0/295003866 shutdown_connections 2024-04-23T14:46:06.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.530+0000 7f166f200640 1 --2- 172.21.15.83:0/295003866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f167010a0f0 0x7f1670110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.530+0000 7f166f200640 1 --2- 172.21.15.83:0/295003866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1670109730 0x7f1670109bb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.530+0000 7f166f200640 1 --2- 172.21.15.83:0/295003866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1670108540 0x7f1670108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.530+0000 7f166f200640 1 -- 172.21.15.83:0/295003866 >> 172.21.15.83:0/295003866 conn(0x7f16700774d0 msgr2=0x7f1670079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:06.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.530+0000 7f166f200640 1 -- 172.21.15.83:0/295003866 shutdown_connections 2024-04-23T14:46:06.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.530+0000 7f166f200640 1 -- 172.21.15.83:0/295003866 wait complete. 2024-04-23T14:46:06.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.531+0000 7f166f200640 1 Processor -- start 2024-04-23T14:46:06.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.531+0000 7f166f200640 1 -- start start 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f166f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1670108540 0x7f167019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f166f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1670109730 0x7f167019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f166f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f167010a0f0 0x7f16701a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f166f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1670114220 con 0x7f1670109730 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f166f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f16701140a0 con 0x7f167010a0f0 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f166f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f16701143a0 con 0x7f1670108540 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f1665400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1670109730 0x7f167019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f166de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1670108540 0x7f167019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f1665400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1670109730 0x7f167019ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60030/0 (socket says 172.21.15.83:60030) 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f1665400640 1 -- 172.21.15.83:0/896064419 learned_addr learned my addr 172.21.15.83:0/896064419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f166e800640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f167010a0f0 0x7f16701a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f1665400640 1 -- 172.21.15.83:0/896064419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1670108540 msgr2=0x7f167019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:06.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f1665400640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1670108540 0x7f167019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f1665400640 1 -- 172.21.15.83:0/896064419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f167010a0f0 msgr2=0x7f16701a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f1665400640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f167010a0f0 0x7f16701a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f1665400640 1 -- 172.21.15.83:0/896064419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16701a6440 con 0x7f1670109730 2024-04-23T14:46:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.532+0000 7f166e800640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f167010a0f0 0x7f16701a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.533+0000 7f1665400640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1670109730 0x7f167019ee50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1658031440 tx=0x7f1658032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.533+0000 7f1667400640 1 -- 172.21.15.83:0/896064419 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165803f030 con 0x7f1670109730 2024-04-23T14:46:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.533+0000 7f1667400640 1 -- 172.21.15.83:0/896064419 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1658003ce0 con 0x7f1670109730 2024-04-23T14:46:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.533+0000 7f1667400640 1 -- 172.21.15.83:0/896064419 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658035070 con 0x7f1670109730 2024-04-23T14:46:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.533+0000 7f166f200640 1 -- 172.21.15.83:0/896064419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16701a66d0 con 0x7f1670109730 2024-04-23T14:46:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.533+0000 7f166f200640 1 -- 172.21.15.83:0/896064419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16701a6b70 con 0x7f1670109730 2024-04-23T14:46:06.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.534+0000 7f166f200640 1 -- 172.21.15.83:0/896064419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f167010f800 con 0x7f1670109730 2024-04-23T14:46:06.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.536+0000 7f1667400640 1 -- 172.21.15.83:0/896064419 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1658042030 con 0x7f1670109730 2024-04-23T14:46:06.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.537+0000 7f1667400640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f16540a3c30 0x7f16540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:06.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.537+0000 7f1667400640 1 -- 172.21.15.83:0/896064419 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f16580f0800 con 0x7f1670109730 2024-04-23T14:46:06.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.538+0000 7f166de00640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f16540a3c30 0x7f16540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:06.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.538+0000 7f166de00640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f16540a3c30 0x7f16540a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1660005fd0 tx=0x7f166000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:06.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.542+0000 7f1667400640 1 -- 172.21.15.83:0/896064419 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16580bd290 con 0x7f1670109730 2024-04-23T14:46:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:06 smithi173 ceph-mon[32443]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-04-23T14:46:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:06 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T14:46:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:06 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] up:active 2024-04-23T14:46:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:06 smithi173 ceph-mon[32443]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-23T14:46:06.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:06 smithi173 ceph-mon[32443]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:46:06.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:06 smithi203 ceph-mon[32800]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-04-23T14:46:06.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:06 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T14:46:06.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:06 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] up:active 2024-04-23T14:46:06.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:06 smithi203 ceph-mon[32800]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-23T14:46:06.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:06 smithi203 ceph-mon[32800]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:46:06.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:06 smithi083 ceph-mon[27867]: pgmap v320: 97 pgs: 97 active+clean; 579 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-04-23T14:46:06.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:06 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T14:46:06.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:06 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] up:active 2024-04-23T14:46:06.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:06 smithi083 ceph-mon[27867]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-23T14:46:06.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:06 smithi083 ceph-mon[27867]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:46:06.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.776+0000 7f166f200640 1 -- 172.21.15.83:0/896064419 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1670062440 con 0x7f16540a3c30 2024-04-23T14:46:06.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.792+0000 7f1667400640 1 -- 172.21.15.83:0/896064419 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1670062440 con 0x7f16540a3c30 2024-04-23T14:46:06.792 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/qa/sv_0/7bdc3e37-a49d-4af3-882d-4ecd97cfa94c 2024-04-23T14:46:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.794+0000 7f166f200640 1 -- 172.21.15.83:0/896064419 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f16540a3c30 msgr2=0x7f16540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.794+0000 7f166f200640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f16540a3c30 0x7f16540a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1660005fd0 tx=0x7f166000b390 comp rx=0 tx=0).stop 2024-04-23T14:46:06.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.794+0000 7f166f200640 1 -- 172.21.15.83:0/896064419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1670109730 msgr2=0x7f167019ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:06.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.794+0000 7f166f200640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1670109730 0x7f167019ee50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1658031440 tx=0x7f1658032ce0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.795+0000 7f166f200640 1 -- 172.21.15.83:0/896064419 shutdown_connections 2024-04-23T14:46:06.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.795+0000 7f166f200640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f16540a3c30 0x7f16540a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.795+0000 7f166f200640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f167010a0f0 0x7f16701a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.795+0000 7f166f200640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1670109730 0x7f167019ee50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.795+0000 7f166f200640 1 --2- 172.21.15.83:0/896064419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1670108540 0x7f167019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:06.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.795+0000 7f166f200640 1 -- 172.21.15.83:0/896064419 >> 172.21.15.83:0/896064419 conn(0x7f16700774d0 msgr2=0x7f167010c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:06.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.795+0000 7f166f200640 1 -- 172.21.15.83:0/896064419 shutdown_connections 2024-04-23T14:46:06.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:06.795+0000 7f166f200640 1 -- 172.21.15.83:0/896064419 wait complete. 2024-04-23T14:46:06.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-23T14:46:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.015+0000 7fa9c3200640 1 -- 172.21.15.83:0/1407401067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c410a0f0 msgr2=0x7fa9c4110980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.015+0000 7fa9c3200640 1 --2- 172.21.15.83:0/1407401067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c410a0f0 0x7fa9c4110980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009ca0 tx=0x7fa9bc02a8c0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.015+0000 7fa9c3200640 1 -- 172.21.15.83:0/1407401067 shutdown_connections 2024-04-23T14:46:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.015+0000 7fa9c3200640 1 --2- 172.21.15.83:0/1407401067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c410a0f0 0x7fa9c4110980 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.015+0000 7fa9c3200640 1 --2- 172.21.15.83:0/1407401067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c4109730 0x7fa9c4109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.015+0000 7fa9c3200640 1 --2- 172.21.15.83:0/1407401067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9c4108540 0x7fa9c4108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.015+0000 7fa9c3200640 1 -- 172.21.15.83:0/1407401067 >> 172.21.15.83:0/1407401067 conn(0x7fa9c40774d0 msgr2=0x7fa9c4079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.015+0000 7fa9c3200640 1 -- 172.21.15.83:0/1407401067 shutdown_connections 2024-04-23T14:46:07.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.015+0000 7fa9c3200640 1 -- 172.21.15.83:0/1407401067 wait complete. 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.016+0000 7fa9c3200640 1 Processor -- start 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.017+0000 7fa9c3200640 1 -- start start 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.017+0000 7fa9c3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c4108540 0x7fa9c419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.017+0000 7fa9c3200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9c4109730 0x7fa9c419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.017+0000 7fa9c3200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c410a0f0 0x7fa9c41a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.017+0000 7fa9c3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c41141d0 con 0x7fa9c4108540 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.017+0000 7fa9c3200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c4114050 con 0x7fa9c4109730 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.017+0000 7fa9c3200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c4114350 con 0x7fa9c410a0f0 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.017+0000 7fa9c1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c4108540 0x7fa9c419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.017+0000 7fa9c1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c4108540 0x7fa9c419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60042/0 (socket says 172.21.15.83:60042) 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.017+0000 7fa9c1e00640 1 -- 172.21.15.83:0/2383049809 learned_addr learned my addr 172.21.15.83:0/2383049809 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.017+0000 7fa9c2800640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c410a0f0 0x7fa9c41a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:07.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.017+0000 7fa9bbe00640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9c4109730 0x7fa9c419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.018+0000 7fa9c1e00640 1 -- 172.21.15.83:0/2383049809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c410a0f0 msgr2=0x7fa9c41a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.018+0000 7fa9c1e00640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c410a0f0 0x7fa9c41a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.018+0000 7fa9c1e00640 1 -- 172.21.15.83:0/2383049809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9c4109730 msgr2=0x7fa9c419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.018+0000 7fa9c1e00640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9c4109730 0x7fa9c419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.018+0000 7fa9c1e00640 1 -- 172.21.15.83:0/2383049809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c41a63c0 con 0x7fa9c4108540 2024-04-23T14:46:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.018+0000 7fa9c1e00640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c4108540 0x7fa9c419e7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400e9c0 tx=0x7fa9b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.018+0000 7fa9baa00640 1 -- 172.21.15.83:0/2383049809 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b400cd80 con 0x7fa9c4108540 2024-04-23T14:46:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.018+0000 7fa9baa00640 1 -- 172.21.15.83:0/2383049809 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b40035f0 con 0x7fa9c4108540 2024-04-23T14:46:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.018+0000 7fa9baa00640 1 -- 172.21.15.83:0/2383049809 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4010690 con 0x7fa9c4108540 2024-04-23T14:46:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.018+0000 7fa9c3200640 1 -- 172.21.15.83:0/2383049809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c41a66b0 con 0x7fa9c4108540 2024-04-23T14:46:07.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.018+0000 7fa9c3200640 1 -- 172.21.15.83:0/2383049809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c41a6bb0 con 0x7fa9c4108540 2024-04-23T14:46:07.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.019+0000 7fa9c3200640 1 -- 172.21.15.83:0/2383049809 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c410f800 con 0x7fa9c4108540 2024-04-23T14:46:07.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.021+0000 7fa9baa00640 1 -- 172.21.15.83:0/2383049809 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa9b40037a0 con 0x7fa9c4108540 2024-04-23T14:46:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.023+0000 7fa9baa00640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa9a80a3bb0 0x7fa9a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.023+0000 7fa9baa00640 1 -- 172.21.15.83:0/2383049809 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa9b40d0800 con 0x7fa9c4108540 2024-04-23T14:46:07.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.023+0000 7fa9bbe00640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa9a80a3bb0 0x7fa9a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:07.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.024+0000 7fa9bbe00640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa9a80a3bb0 0x7fa9a80a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa9c419fcb0 tx=0x7fa9ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:07.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.027+0000 7fa9baa00640 1 -- 172.21.15.83:0/2383049809 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9b409d290 con 0x7fa9c4108540 2024-04-23T14:46:07.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.262+0000 7fa9c3200640 1 -- 172.21.15.83:0/2383049809 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa9c4062440 con 0x7fa9a80a3bb0 2024-04-23T14:46:07.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.298+0000 7fa9baa00640 1 -- 172.21.15.83:0/2383049809 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa9c4062440 con 0x7fa9a80a3bb0 2024-04-23T14:46:07.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.301+0000 7fa9c3200640 1 -- 172.21.15.83:0/2383049809 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa9a80a3bb0 msgr2=0x7fa9a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.301+0000 7fa9c3200640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa9a80a3bb0 0x7fa9a80a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa9c419fcb0 tx=0x7fa9ac005f90 comp rx=0 tx=0).stop 2024-04-23T14:46:07.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.301+0000 7fa9c3200640 1 -- 172.21.15.83:0/2383049809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c4108540 msgr2=0x7fa9c419e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.301+0000 7fa9c3200640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c4108540 0x7fa9c419e7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400e9c0 tx=0x7fa9b400ee80 comp rx=0 tx=0).stop 2024-04-23T14:46:07.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.302+0000 7fa9c3200640 1 -- 172.21.15.83:0/2383049809 shutdown_connections 2024-04-23T14:46:07.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.302+0000 7fa9c3200640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa9a80a3bb0 0x7fa9a80a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.302+0000 7fa9c3200640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9c410a0f0 0x7fa9c41a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.302+0000 7fa9c3200640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9c4109730 0x7fa9c419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.302+0000 7fa9c3200640 1 --2- 172.21.15.83:0/2383049809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9c4108540 0x7fa9c419e7a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.302+0000 7fa9c3200640 1 -- 172.21.15.83:0/2383049809 >> 172.21.15.83:0/2383049809 conn(0x7fa9c40774d0 msgr2=0x7fa9c410c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:07.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.302+0000 7fa9c3200640 1 -- 172.21.15.83:0/2383049809 shutdown_connections 2024-04-23T14:46:07.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.302+0000 7fa9c3200640 1 -- 172.21.15.83:0/2383049809 wait complete. 2024-04-23T14:46:07.321 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-23T14:46:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.518+0000 7f7d3be00640 1 -- 172.21.15.83:0/295068393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c108540 msgr2=0x7f7d3c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.518+0000 7f7d3be00640 1 --2- 172.21.15.83:0/295068393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c108540 0x7f7d3c108960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009900 tx=0x7f7d2c02a3f0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.518+0000 7f7d3be00640 1 -- 172.21.15.83:0/295068393 shutdown_connections 2024-04-23T14:46:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.518+0000 7f7d3be00640 1 --2- 172.21.15.83:0/295068393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d3c10a0f0 0x7f7d3c110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.518+0000 7f7d3be00640 1 --2- 172.21.15.83:0/295068393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d3c109730 0x7f7d3c109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.518+0000 7f7d3be00640 1 --2- 172.21.15.83:0/295068393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c108540 0x7f7d3c108960 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.518+0000 7f7d3be00640 1 -- 172.21.15.83:0/295068393 >> 172.21.15.83:0/295068393 conn(0x7f7d3c0774d0 msgr2=0x7f7d3c079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.518+0000 7f7d3be00640 1 -- 172.21.15.83:0/295068393 shutdown_connections 2024-04-23T14:46:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.518+0000 7f7d3be00640 1 -- 172.21.15.83:0/295068393 wait complete. 2024-04-23T14:46:07.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.519+0000 7f7d3be00640 1 Processor -- start 2024-04-23T14:46:07.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.520+0000 7f7d3be00640 1 -- start start 2024-04-23T14:46:07.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.520+0000 7f7d3be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c108540 0x7f7d3c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:07.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.520+0000 7f7d3be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d3c109730 0x7f7d3c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:07.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.520+0000 7f7d3be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d3c10a0f0 0x7f7d3c1a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.520+0000 7f7d3be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1141f0 con 0x7f7d3c108540 2024-04-23T14:46:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.520+0000 7f7d3be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c114070 con 0x7f7d3c10a0f0 2024-04-23T14:46:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.520+0000 7f7d3be00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c114370 con 0x7f7d3c109730 2024-04-23T14:46:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.520+0000 7f7d3aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c108540 0x7f7d3c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.520+0000 7f7d3aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c108540 0x7f7d3c19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60060/0 (socket says 172.21.15.83:60060) 2024-04-23T14:46:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.520+0000 7f7d3aa00640 1 -- 172.21.15.83:0/2262354737 learned_addr learned my addr 172.21.15.83:0/2262354737 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.520+0000 7f7d32000640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d3c109730 0x7f7d3c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.520+0000 7f7d3b400640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d3c10a0f0 0x7f7d3c1a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.521+0000 7f7d3aa00640 1 -- 172.21.15.83:0/2262354737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d3c109730 msgr2=0x7f7d3c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.521+0000 7f7d3aa00640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d3c109730 0x7f7d3c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.521+0000 7f7d3aa00640 1 -- 172.21.15.83:0/2262354737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d3c10a0f0 msgr2=0x7f7d3c1a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.521+0000 7f7d3aa00640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d3c10a0f0 0x7f7d3c1a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.521+0000 7f7d3aa00640 1 -- 172.21.15.83:0/2262354737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c1a6480 con 0x7f7d3c108540 2024-04-23T14:46:07.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.521+0000 7f7d3aa00640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c108540 0x7f7d3c19e860 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c031440 tx=0x7f7d2c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:07.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.521+0000 7f7d40a00640 1 -- 172.21.15.83:0/2262354737 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c03f030 con 0x7f7d3c108540 2024-04-23T14:46:07.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.521+0000 7f7d40a00640 1 -- 172.21.15.83:0/2262354737 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c003ce0 con 0x7f7d3c108540 2024-04-23T14:46:07.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.521+0000 7f7d40a00640 1 -- 172.21.15.83:0/2262354737 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c035070 con 0x7f7d3c108540 2024-04-23T14:46:07.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.521+0000 7f7d3be00640 1 -- 172.21.15.83:0/2262354737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c1a6710 con 0x7f7d3c108540 2024-04-23T14:46:07.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.521+0000 7f7d3be00640 1 -- 172.21.15.83:0/2262354737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c1a6c50 con 0x7f7d3c108540 2024-04-23T14:46:07.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.523+0000 7f7d3be00640 1 -- 172.21.15.83:0/2262354737 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3c10f860 con 0x7f7d3c108540 2024-04-23T14:46:07.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.525+0000 7f7d40a00640 1 -- 172.21.15.83:0/2262354737 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7d2c042030 con 0x7f7d3c108540 2024-04-23T14:46:07.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.526+0000 7f7d40a00640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7d200a3c30 0x7f7d200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:07.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.526+0000 7f7d40a00640 1 -- 172.21.15.83:0/2262354737 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f7d2c0f0720 con 0x7f7d3c108540 2024-04-23T14:46:07.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.526+0000 7f7d32000640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7d200a3c30 0x7f7d200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:07.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.527+0000 7f7d32000640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7d200a3c30 0x7f7d200a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c19fd70 tx=0x7f7d24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:07.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.531+0000 7f7d40a00640 1 -- 172.21.15.83:0/2262354737 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d2c0bd1b0 con 0x7f7d3c108540 2024-04-23T14:46:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:07 smithi173 ceph-mon[32443]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:07 smithi173 ceph-mon[32443]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:07 smithi203 ceph-mon[32800]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:07 smithi203 ceph-mon[32800]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:07.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:07 smithi083 ceph-mon[27867]: from='client.24683 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:07.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:07 smithi083 ceph-mon[27867]: from='client.24695 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:07.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.768+0000 7f7d3be00640 1 -- 172.21.15.83:0/2262354737 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f7d3c062440 con 0x7f7d200a3c30 2024-04-23T14:46:07.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.776+0000 7f7d40a00640 1 -- 172.21.15.83:0/2262354737 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7d3c062440 con 0x7f7d200a3c30 2024-04-23T14:46:07.776 INFO:teuthology.orchestra.run.smithi083.stdout:/volumes/qa/sv_1/d1dd1b37-f1a3-4aae-99d8-e2fe8203f9d8 2024-04-23T14:46:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.778+0000 7f7d3be00640 1 -- 172.21.15.83:0/2262354737 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7d200a3c30 msgr2=0x7f7d200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.778+0000 7f7d3be00640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7d200a3c30 0x7f7d200a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c19fd70 tx=0x7f7d24005f90 comp rx=0 tx=0).stop 2024-04-23T14:46:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.778+0000 7f7d3be00640 1 -- 172.21.15.83:0/2262354737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c108540 msgr2=0x7f7d3c19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.779+0000 7f7d3be00640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c108540 0x7f7d3c19e860 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c031440 tx=0x7f7d2c032ce0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.779+0000 7f7d3be00640 1 -- 172.21.15.83:0/2262354737 shutdown_connections 2024-04-23T14:46:07.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.779+0000 7f7d3be00640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7d200a3c30 0x7f7d200a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.779+0000 7f7d3be00640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d3c10a0f0 0x7f7d3c1a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.779+0000 7f7d3be00640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d3c109730 0x7f7d3c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.779+0000 7f7d3be00640 1 --2- 172.21.15.83:0/2262354737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d3c108540 0x7f7d3c19e860 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.779+0000 7f7d3be00640 1 -- 172.21.15.83:0/2262354737 >> 172.21.15.83:0/2262354737 conn(0x7f7d3c0774d0 msgr2=0x7f7d3c10ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:07.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.779+0000 7f7d3be00640 1 -- 172.21.15.83:0/2262354737 shutdown_connections 2024-04-23T14:46:07.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.779+0000 7f7d3be00640 1 -- 172.21.15.83:0/2262354737 wait complete. 2024-04-23T14:46:07.798 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:46:07.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.987+0000 7f31efa00640 1 -- 172.21.15.83:0/2143487569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e8108660 msgr2=0x7f31e8108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.987+0000 7f31efa00640 1 --2- 172.21.15.83:0/2143487569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e8108660 0x7f31e8108a80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009900 tx=0x7f31d802a3f0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.987+0000 7f31efa00640 1 -- 172.21.15.83:0/2143487569 shutdown_connections 2024-04-23T14:46:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.987+0000 7f31efa00640 1 --2- 172.21.15.83:0/2143487569 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31e810a210 0x7f31e8110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.987+0000 7f31efa00640 1 --2- 172.21.15.83:0/2143487569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e8109850 0x7f31e8109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.987+0000 7f31efa00640 1 --2- 172.21.15.83:0/2143487569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e8108660 0x7f31e8108a80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.987+0000 7f31efa00640 1 -- 172.21.15.83:0/2143487569 >> 172.21.15.83:0/2143487569 conn(0x7f31e8077620 msgr2=0x7f31e8079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.987+0000 7f31efa00640 1 -- 172.21.15.83:0/2143487569 shutdown_connections 2024-04-23T14:46:07.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.987+0000 7f31efa00640 1 -- 172.21.15.83:0/2143487569 wait complete. 2024-04-23T14:46:07.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.988+0000 7f31efa00640 1 Processor -- start 2024-04-23T14:46:07.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.988+0000 7f31efa00640 1 -- start start 2024-04-23T14:46:07.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31efa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e8108660 0x7f31e819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:07.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31efa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31e8109850 0x7f31e819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:07.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31efa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e810a210 0x7f31e81a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:07.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31efa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31e8114300 con 0x7f31e8108660 2024-04-23T14:46:07.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31efa00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f31e8114180 con 0x7f31e810a210 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31efa00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f31e8114480 con 0x7f31e8109850 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31ed000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e8108660 0x7f31e819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31eda00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e810a210 0x7f31e81a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31dfe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31e8109850 0x7f31e819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31ed000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e8108660 0x7f31e819e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60078/0 (socket says 172.21.15.83:60078) 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31dfe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31e8109850 0x7f31e819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:34146/0 (socket says 172.21.15.83:34146) 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31eda00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e810a210 0x7f31e81a5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:55568/0 (socket says 172.21.15.83:55568) 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31dfe00640 1 -- 172.21.15.83:0/3520241741 learned_addr learned my addr 172.21.15.83:0/3520241741 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31dfe00640 1 -- 172.21.15.83:0/3520241741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e810a210 msgr2=0x7f31e81a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31dfe00640 1 --2- 172.21.15.83:0/3520241741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e810a210 0x7f31e81a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31dfe00640 1 -- 172.21.15.83:0/3520241741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e8108660 msgr2=0x7f31e819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31dfe00640 1 --2- 172.21.15.83:0/3520241741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e8108660 0x7f31e819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.989+0000 7f31dfe00640 1 -- 172.21.15.83:0/3520241741 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e81a6590 con 0x7f31e8109850 2024-04-23T14:46:07.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.990+0000 7f31dfe00640 1 --2- 172.21.15.83:0/3520241741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31e8109850 0x7f31e819eeb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f31d000ea60 tx=0x7f31d000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:07.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.990+0000 7f31e6000640 1 -- 172.21.15.83:0/3520241741 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d000cde0 con 0x7f31e8109850 2024-04-23T14:46:07.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.990+0000 7f31efa00640 1 -- 172.21.15.83:0/3520241741 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e81a6880 con 0x7f31e8109850 2024-04-23T14:46:07.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.990+0000 7f31e6000640 1 -- 172.21.15.83:0/3520241741 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d00035f0 con 0x7f31e8109850 2024-04-23T14:46:07.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.990+0000 7f31e6000640 1 -- 172.21.15.83:0/3520241741 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0010670 con 0x7f31e8109850 2024-04-23T14:46:07.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.990+0000 7f31efa00640 1 -- 172.21.15.83:0/3520241741 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e81a6e20 con 0x7f31e8109850 2024-04-23T14:46:07.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.991+0000 7f31efa00640 1 -- 172.21.15.83:0/3520241741 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e810f9d0 con 0x7f31e8109850 2024-04-23T14:46:07.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.993+0000 7f31e6000640 1 -- 172.21.15.83:0/3520241741 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f31d00037a0 con 0x7f31e8109850 2024-04-23T14:46:07.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.995+0000 7f31e6000640 1 --2- 172.21.15.83:0/3520241741 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f31cc0a3bb0 0x7f31cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:07.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.995+0000 7f31e6000640 1 -- 172.21.15.83:0/3520241741 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f31d00d0600 con 0x7f31e8109850 2024-04-23T14:46:07.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.995+0000 7f31ed000640 1 --2- 172.21.15.83:0/3520241741 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f31cc0a3bb0 0x7f31cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:07.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.996+0000 7f31ed000640 1 --2- 172.21.15.83:0/3520241741 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f31cc0a3bb0 0x7f31cc0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f31d802a9e0 tx=0x7f31d80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:07.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:07.999+0000 7f31e6000640 1 -- 172.21.15.83:0/3520241741 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31d009d090 con 0x7f31e8109850 2024-04-23T14:46:08.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.250+0000 7f31efa00640 1 -- 172.21.15.83:0/3520241741 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31e8062c00 con 0x7f31e8109850 2024-04-23T14:46:08.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.251+0000 7f31e6000640 1 -- 172.21.15.83:0/3520241741 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7f31d009c990 con 0x7f31e8109850 2024-04-23T14:46:08.252 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:46:08.252 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:46:05.301806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:46:08.253 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-04-23T14:46:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.254+0000 7f31efa00640 1 -- 172.21.15.83:0/3520241741 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f31cc0a3bb0 msgr2=0x7f31cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.254+0000 7f31efa00640 1 --2- 172.21.15.83:0/3520241741 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f31cc0a3bb0 0x7f31cc0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f31d802a9e0 tx=0x7f31d80315b0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.254+0000 7f31efa00640 1 -- 172.21.15.83:0/3520241741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31e8109850 msgr2=0x7f31e819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.254+0000 7f31efa00640 1 --2- 172.21.15.83:0/3520241741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31e8109850 0x7f31e819eeb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f31d000ea60 tx=0x7f31d000ef20 comp rx=0 tx=0).stop 2024-04-23T14:46:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.254+0000 7f31efa00640 1 -- 172.21.15.83:0/3520241741 shutdown_connections 2024-04-23T14:46:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.254+0000 7f31efa00640 1 --2- 172.21.15.83:0/3520241741 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f31cc0a3bb0 0x7f31cc0a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.254+0000 7f31efa00640 1 --2- 172.21.15.83:0/3520241741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31e810a210 0x7f31e81a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.254+0000 7f31efa00640 1 --2- 172.21.15.83:0/3520241741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31e8109850 0x7f31e819eeb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.254+0000 7f31efa00640 1 --2- 172.21.15.83:0/3520241741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31e8108660 0x7f31e819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.254+0000 7f31efa00640 1 -- 172.21.15.83:0/3520241741 >> 172.21.15.83:0/3520241741 conn(0x7f31e8077620 msgr2=0x7f31e810cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.255+0000 7f31efa00640 1 -- 172.21.15.83:0/3520241741 shutdown_connections 2024-04-23T14:46:08.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.255+0000 7f31efa00640 1 -- 172.21.15.83:0/3520241741 wait complete. 2024-04-23T14:46:08.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:46:08.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.464+0000 7f0f8ec00640 1 -- 172.21.15.83:0/1039420639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f8810a030 msgr2=0x7f0f881108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.464+0000 7f0f8ec00640 1 --2- 172.21.15.83:0/1039420639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f8810a030 0x7f0f881108c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009ca0 tx=0x7f0f8002a8c0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.464+0000 7f0f8ec00640 1 -- 172.21.15.83:0/1039420639 shutdown_connections 2024-04-23T14:46:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.464+0000 7f0f8ec00640 1 --2- 172.21.15.83:0/1039420639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f8810a030 0x7f0f881108c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.464+0000 7f0f8ec00640 1 --2- 172.21.15.83:0/1039420639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f88109670 0x7f0f88109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.464+0000 7f0f8ec00640 1 --2- 172.21.15.83:0/1039420639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f88108480 0x7f0f881088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.464+0000 7f0f8ec00640 1 -- 172.21.15.83:0/1039420639 >> 172.21.15.83:0/1039420639 conn(0x7f0f88077440 msgr2=0x7f0f88079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.464+0000 7f0f8ec00640 1 -- 172.21.15.83:0/1039420639 shutdown_connections 2024-04-23T14:46:08.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.464+0000 7f0f8ec00640 1 -- 172.21.15.83:0/1039420639 wait complete. 2024-04-23T14:46:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.465+0000 7f0f8ec00640 1 Processor -- start 2024-04-23T14:46:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.465+0000 7f0f8ec00640 1 -- start start 2024-04-23T14:46:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.465+0000 7f0f8ec00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f88108480 0x7f0f8819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.465+0000 7f0f8ec00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f88109670 0x7f0f8819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.465+0000 7f0f8ec00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f8810a030 0x7f0f881a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:08.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.465+0000 7f0f8ec00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88114160 con 0x7f0f88109670 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.465+0000 7f0f8ec00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88113fe0 con 0x7f0f8810a030 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.465+0000 7f0f8ec00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881142e0 con 0x7f0f88108480 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f88109670 0x7f0f8819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f88109670 0x7f0f8819ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60094/0 (socket says 172.21.15.83:60094) 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8ce00640 1 -- 172.21.15.83:0/3255808076 learned_addr learned my addr 172.21.15.83:0/3255808076 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8d800640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f88108480 0x7f0f8819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8e200640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f8810a030 0x7f0f881a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8ce00640 1 -- 172.21.15.83:0/3255808076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f88108480 msgr2=0x7f0f8819e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8ce00640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f88108480 0x7f0f8819e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8ce00640 1 -- 172.21.15.83:0/3255808076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f8810a030 msgr2=0x7f0f881a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8ce00640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f8810a030 0x7f0f881a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8ce00640 1 -- 172.21.15.83:0/3255808076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f881a6300 con 0x7f0f88109670 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8e200640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f8810a030 0x7f0f881a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8d800640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f88108480 0x7f0f8819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:08.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.466+0000 7f0f8ce00640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f88109670 0x7f0f8819ec20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400e9f0 tx=0x7f0f7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.467+0000 7f0f86000640 1 -- 172.21.15.83:0/3255808076 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7400cd80 con 0x7f0f88109670 2024-04-23T14:46:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.467+0000 7f0f86000640 1 -- 172.21.15.83:0/3255808076 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f74010910 con 0x7f0f88109670 2024-04-23T14:46:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.467+0000 7f0f8ec00640 1 -- 172.21.15.83:0/3255808076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f881a65f0 con 0x7f0f88109670 2024-04-23T14:46:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.467+0000 7f0f86000640 1 -- 172.21.15.83:0/3255808076 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74018990 con 0x7f0f88109670 2024-04-23T14:46:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.467+0000 7f0f8ec00640 1 -- 172.21.15.83:0/3255808076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f881a6b40 con 0x7f0f88109670 2024-04-23T14:46:08.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.468+0000 7f0f8ec00640 1 -- 172.21.15.83:0/3255808076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f8810f6d0 con 0x7f0f88109670 2024-04-23T14:46:08.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.471+0000 7f0f86000640 1 -- 172.21.15.83:0/3255808076 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0f74010ac0 con 0x7f0f88109670 2024-04-23T14:46:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.472+0000 7f0f86000640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0f700a3c40 0x7f0f700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.472+0000 7f0f86000640 1 -- 172.21.15.83:0/3255808076 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0f740d0cd0 con 0x7f0f88109670 2024-04-23T14:46:08.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.472+0000 7f0f8d800640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0f700a3c40 0x7f0f700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:08.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.473+0000 7f0f8d800640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0f700a3c40 0x7f0f700a6100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0f881094d0 tx=0x7f0f7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:08.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.474+0000 7f0f86000640 1 -- 172.21.15.83:0/3255808076 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f7409d600 con 0x7f0f88109670 2024-04-23T14:46:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:08 smithi173 ceph-mon[32443]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:08 smithi173 ceph-mon[32443]: pgmap v321: 97 pgs: 97 active+clean; 582 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-04-23T14:46:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:08 smithi173 ceph-mon[32443]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:08.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:08 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3520241741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:46:08.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.676+0000 7f0f8ec00640 1 -- 172.21.15.83:0/3255808076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f8802d890 con 0x7f0f88109670 2024-04-23T14:46:08.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.679+0000 7f0f86000640 1 -- 172.21.15.83:0/3255808076 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22219 (secure 0 0 0) 0x7f0f7409cf00 con 0x7f0f88109670 2024-04-23T14:46:08.680 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:46:08.680 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:46:00.988678+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:46:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.682+0000 7f0f8ec00640 1 -- 172.21.15.83:0/3255808076 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0f700a3c40 msgr2=0x7f0f700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.682+0000 7f0f8ec00640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0f700a3c40 0x7f0f700a6100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0f881094d0 tx=0x7f0f7c005f90 comp rx=0 tx=0).stop 2024-04-23T14:46:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.682+0000 7f0f8ec00640 1 -- 172.21.15.83:0/3255808076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f88109670 msgr2=0x7f0f8819ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.682+0000 7f0f8ec00640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f88109670 0x7f0f8819ec20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400e9f0 tx=0x7f0f7400eeb0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.682+0000 7f0f8ec00640 1 -- 172.21.15.83:0/3255808076 shutdown_connections 2024-04-23T14:46:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.682+0000 7f0f8ec00640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0f700a3c40 0x7f0f700a6100 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.682+0000 7f0f8ec00640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f8810a030 0x7f0f881a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.682+0000 7f0f8ec00640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f88109670 0x7f0f8819ec20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.682+0000 7f0f8ec00640 1 --2- 172.21.15.83:0/3255808076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f88108480 0x7f0f8819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.682+0000 7f0f8ec00640 1 -- 172.21.15.83:0/3255808076 >> 172.21.15.83:0/3255808076 conn(0x7f0f88077440 msgr2=0x7f0f8810c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.682+0000 7f0f8ec00640 1 -- 172.21.15.83:0/3255808076 shutdown_connections 2024-04-23T14:46:08.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.682+0000 7f0f8ec00640 1 -- 172.21.15.83:0/3255808076 wait complete. 2024-04-23T14:46:08.699 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:08 smithi083 ceph-mon[27867]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:08.699 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:08 smithi083 ceph-mon[27867]: pgmap v321: 97 pgs: 97 active+clean; 582 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-04-23T14:46:08.699 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:08 smithi083 ceph-mon[27867]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:08.699 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:08 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3520241741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:46:08.702 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T14:46:08.702 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:46:08.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:08 smithi203 ceph-mon[32800]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:08.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:08 smithi203 ceph-mon[32800]: pgmap v321: 97 pgs: 97 active+clean; 582 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-04-23T14:46:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:08 smithi203 ceph-mon[32800]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:08.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:08 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3520241741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:46:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.903+0000 7fd58d000640 1 -- 172.21.15.83:0/3151788270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd588109670 msgr2=0x7fd588109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.903+0000 7fd58d000640 1 --2- 172.21.15.83:0/3151788270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd588109670 0x7fd588109af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd570009900 tx=0x7fd57002a400 comp rx=0 tx=0).stop 2024-04-23T14:46:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.903+0000 7fd58d000640 1 -- 172.21.15.83:0/3151788270 shutdown_connections 2024-04-23T14:46:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.903+0000 7fd58d000640 1 --2- 172.21.15.83:0/3151788270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd58810a030 0x7fd5881108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.903+0000 7fd58d000640 1 --2- 172.21.15.83:0/3151788270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd588109670 0x7fd588109af0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.903+0000 7fd58d000640 1 --2- 172.21.15.83:0/3151788270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd588108480 0x7fd5881088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.904+0000 7fd58d000640 1 -- 172.21.15.83:0/3151788270 >> 172.21.15.83:0/3151788270 conn(0x7fd588077440 msgr2=0x7fd588079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:08.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.904+0000 7fd58d000640 1 -- 172.21.15.83:0/3151788270 shutdown_connections 2024-04-23T14:46:08.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.904+0000 7fd58d000640 1 -- 172.21.15.83:0/3151788270 wait complete. 2024-04-23T14:46:08.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.905+0000 7fd58d000640 1 Processor -- start 2024-04-23T14:46:08.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.905+0000 7fd58d000640 1 -- start start 2024-04-23T14:46:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.905+0000 7fd58d000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd588108480 0x7fd58819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.905+0000 7fd58d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd588109670 0x7fd58819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:08.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.905+0000 7fd58d000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd58810a030 0x7fd5881a5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.905+0000 7fd58d000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd588114110 con 0x7fd588109670 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.905+0000 7fd58d000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd588113f90 con 0x7fd588108480 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.905+0000 7fd58d000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd588114290 con 0x7fd58810a030 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.906+0000 7fd57ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd588109670 0x7fd58819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.906+0000 7fd57ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd588109670 0x7fd58819ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60108/0 (socket says 172.21.15.83:60108) 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.906+0000 7fd57ea00640 1 -- 172.21.15.83:0/4197555566 learned_addr learned my addr 172.21.15.83:0/4197555566 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.906+0000 7fd587400640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd588108480 0x7fd58819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.906+0000 7fd587e00640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd58810a030 0x7fd5881a5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.906+0000 7fd57ea00640 1 -- 172.21.15.83:0/4197555566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd58810a030 msgr2=0x7fd5881a5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.906+0000 7fd57ea00640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd58810a030 0x7fd5881a5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.906+0000 7fd57ea00640 1 -- 172.21.15.83:0/4197555566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd588108480 msgr2=0x7fd58819e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.906+0000 7fd57ea00640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd588108480 0x7fd58819e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.906+0000 7fd57ea00640 1 -- 172.21.15.83:0/4197555566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5881a6350 con 0x7fd588109670 2024-04-23T14:46:08.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.906+0000 7fd587400640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd588108480 0x7fd58819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.907+0000 7fd57ea00640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd588109670 0x7fd58819ec70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd570009230 tx=0x7fd5700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.907+0000 7fd584c00640 1 -- 172.21.15.83:0/4197555566 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57003f030 con 0x7fd588109670 2024-04-23T14:46:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.907+0000 7fd584c00640 1 -- 172.21.15.83:0/4197555566 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd570003ce0 con 0x7fd588109670 2024-04-23T14:46:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.907+0000 7fd58d000640 1 -- 172.21.15.83:0/4197555566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5881a65e0 con 0x7fd588109670 2024-04-23T14:46:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.907+0000 7fd584c00640 1 -- 172.21.15.83:0/4197555566 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570035070 con 0x7fd588109670 2024-04-23T14:46:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.907+0000 7fd58d000640 1 -- 172.21.15.83:0/4197555566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5881a6ad0 con 0x7fd588109670 2024-04-23T14:46:08.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.908+0000 7fd58d000640 1 -- 172.21.15.83:0/4197555566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58810f7a0 con 0x7fd588109670 2024-04-23T14:46:08.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.910+0000 7fd584c00640 1 -- 172.21.15.83:0/4197555566 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd570042030 con 0x7fd588109670 2024-04-23T14:46:08.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.912+0000 7fd584c00640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd56c0a3c40 0x7fd56c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:08.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.912+0000 7fd584c00640 1 -- 172.21.15.83:0/4197555566 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd5700f0d90 con 0x7fd588109670 2024-04-23T14:46:08.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.912+0000 7fd587400640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd56c0a3c40 0x7fd56c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:08.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.913+0000 7fd587400640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd56c0a3c40 0x7fd56c0a6100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5881094d0 tx=0x7fd578005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:08.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:08.914+0000 7fd584c00640 1 -- 172.21.15.83:0/4197555566 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5700bd070 con 0x7fd588109670 2024-04-23T14:46:09.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.115+0000 7fd58d000640 1 -- 172.21.15.83:0/4197555566 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd588062440 con 0x7fd56c0a3c40 2024-04-23T14:46:09.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.141+0000 7fd584c00640 1 -- 172.21.15.83:0/4197555566 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258895 (secure 0 0 0) 0x7fd588062440 con 0x7fd56c0a3c40 2024-04-23T14:46:09.146 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:46:09.147 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-23T14:46:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.147+0000 7fd58d000640 1 -- 172.21.15.83:0/4197555566 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd56c0a3c40 msgr2=0x7fd56c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.147+0000 7fd58d000640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd56c0a3c40 0x7fd56c0a6100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5881094d0 tx=0x7fd578005f90 comp rx=0 tx=0).stop 2024-04-23T14:46:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.147+0000 7fd58d000640 1 -- 172.21.15.83:0/4197555566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd588109670 msgr2=0x7fd58819ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.147+0000 7fd58d000640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd588109670 0x7fd58819ec70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd570009230 tx=0x7fd5700042f0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.148+0000 7fd58d000640 1 -- 172.21.15.83:0/4197555566 shutdown_connections 2024-04-23T14:46:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.148+0000 7fd58d000640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd56c0a3c40 0x7fd56c0a6100 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.148+0000 7fd58d000640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd58810a030 0x7fd5881a5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.148+0000 7fd58d000640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd588109670 0x7fd58819ec70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.148+0000 7fd58d000640 1 --2- 172.21.15.83:0/4197555566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd588108480 0x7fd58819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.148+0000 7fd58d000640 1 -- 172.21.15.83:0/4197555566 >> 172.21.15.83:0/4197555566 conn(0x7fd588077440 msgr2=0x7fd58810c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:09.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.148+0000 7fd58d000640 1 -- 172.21.15.83:0/4197555566 shutdown_connections 2024-04-23T14:46:09.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.148+0000 7fd58d000640 1 -- 172.21.15.83:0/4197555566 wait complete. 2024-04-23T14:46:09.169 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-23T14:46:07.206240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596238,"num_objects":50,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":157,"num_read_kb":137,"num_write":281,"num_write_kb":2969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":239,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377020,"kb_used_data":11192,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564804,"statfs":{"total":1151940427776,"available":1151554359296,"internally_reserved":0,"allocated":11460608,"data_stored":7118589,"data_compressed":107307,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5870,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":21,"num_read_kb":18,"num_write":100,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.991600"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107436+0000","last_change":"2024-04-23T14:46:00.030934+0000","last_active":"2024-04-23T14:46:01.107436+0000","last_peered":"2024-04-23T14:46:01.107436+0000","last_clean":"2024-04-23T14:46:01.107436+0000","last_became_active":"2024-04-23T14:46:00.030595+0000","last_became_peered":"2024-04-23T14:46:00.030595+0000","last_unstale":"2024-04-23T14:46:01.107436+0000","last_undegraded":"2024-04-23T14:46:01.107436+0000","last_fullsized":"2024-04-23T14:46:01.107436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.814685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.709991+0000","last_change":"2024-04-23T14:46:00.023044+0000","last_active":"2024-04-23T14:46:01.709991+0000","last_peered":"2024-04-23T14:46:01.709991+0000","last_clean":"2024-04-23T14:46:01.709991+0000","last_became_active":"2024-04-23T14:46:00.022441+0000","last_became_peered":"2024-04-23T14:46:00.022441+0000","last_unstale":"2024-04-23T14:46:01.709991+0000","last_undegraded":"2024-04-23T14:46:01.709991+0000","last_fullsized":"2024-04-23T14:46:01.709991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:24:20.976312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.709488+0000","last_change":"2024-04-23T14:46:00.025397+0000","last_active":"2024-04-23T14:46:01.709488+0000","last_peered":"2024-04-23T14:46:01.709488+0000","last_clean":"2024-04-23T14:46:01.709488+0000","last_became_active":"2024-04-23T14:46:00.024477+0000","last_became_peered":"2024-04-23T14:46:00.024477+0000","last_unstale":"2024-04-23T14:46:01.709488+0000","last_undegraded":"2024-04-23T14:46:01.709488+0000","last_fullsized":"2024-04-23T14:46:01.709488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:16.002964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995288+0000","last_change":"2024-04-23T14:46:00.013960+0000","last_active":"2024-04-23T14:46:00.995288+0000","last_peered":"2024-04-23T14:46:00.995288+0000","last_clean":"2024-04-23T14:46:00.995288+0000","last_became_active":"2024-04-23T14:46:00.013656+0000","last_became_peered":"2024-04-23T14:46:00.013656+0000","last_unstale":"2024-04-23T14:46:00.995288+0000","last_undegraded":"2024-04-23T14:46:00.995288+0000","last_fullsized":"2024-04-23T14:46:00.995288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:45:59.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105934+0000","last_change":"2024-04-23T14:46:00.032336+0000","last_active":"2024-04-23T14:46:01.105934+0000","last_peered":"2024-04-23T14:46:01.105934+0000","last_clean":"2024-04-23T14:46:01.105934+0000","last_became_active":"2024-04-23T14:46:00.032008+0000","last_became_peered":"2024-04-23T14:46:00.032008+0000","last_unstale":"2024-04-23T14:46:01.105934+0000","last_undegraded":"2024-04-23T14:46:01.105934+0000","last_fullsized":"2024-04-23T14:46:01.105934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:13:05.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106168+0000","last_change":"2024-04-23T14:46:00.015457+0000","last_active":"2024-04-23T14:46:01.106168+0000","last_peered":"2024-04-23T14:46:01.106168+0000","last_clean":"2024-04-23T14:46:01.106168+0000","last_became_active":"2024-04-23T14:46:00.015199+0000","last_became_peered":"2024-04-23T14:46:00.015199+0000","last_unstale":"2024-04-23T14:46:01.106168+0000","last_undegraded":"2024-04-23T14:46:01.106168+0000","last_fullsized":"2024-04-23T14:46:01.106168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:39.310993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105197+0000","last_change":"2024-04-23T14:46:00.016554+0000","last_active":"2024-04-23T14:46:01.105197+0000","last_peered":"2024-04-23T14:46:01.105197+0000","last_clean":"2024-04-23T14:46:01.105197+0000","last_became_active":"2024-04-23T14:46:00.016242+0000","last_became_peered":"2024-04-23T14:46:00.016242+0000","last_unstale":"2024-04-23T14:46:01.105197+0000","last_undegraded":"2024-04-23T14:46:01.105197+0000","last_fullsized":"2024-04-23T14:46:01.105197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.596637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.996874+0000","last_change":"2024-04-23T14:46:00.014574+0000","last_active":"2024-04-23T14:46:00.996874+0000","last_peered":"2024-04-23T14:46:00.996874+0000","last_clean":"2024-04-23T14:46:00.996874+0000","last_became_active":"2024-04-23T14:46:00.010413+0000","last_became_peered":"2024-04-23T14:46:00.010413+0000","last_unstale":"2024-04-23T14:46:00.996874+0000","last_undegraded":"2024-04-23T14:46:00.996874+0000","last_fullsized":"2024-04-23T14:46:00.996874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:15:59.631552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106615+0000","last_change":"2024-04-23T14:46:00.005588+0000","last_active":"2024-04-23T14:46:01.106615+0000","last_peered":"2024-04-23T14:46:01.106615+0000","last_clean":"2024-04-23T14:46:01.106615+0000","last_became_active":"2024-04-23T14:46:00.005238+0000","last_became_peered":"2024-04-23T14:46:00.005238+0000","last_unstale":"2024-04-23T14:46:01.106615+0000","last_undegraded":"2024-04-23T14:46:01.106615+0000","last_fullsized":"2024-04-23T14:46:01.106615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:53:40.522929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106130+0000","last_change":"2024-04-23T14:46:00.013773+0000","last_active":"2024-04-23T14:46:01.106130+0000","last_peered":"2024-04-23T14:46:01.106130+0000","last_clean":"2024-04-23T14:46:01.106130+0000","last_became_active":"2024-04-23T14:46:00.013390+0000","last_became_peered":"2024-04-23T14:46:00.013390+0000","last_unstale":"2024-04-23T14:46:01.106130+0000","last_undegraded":"2024-04-23T14:46:01.106130+0000","last_fullsized":"2024-04-23T14:46:01.106130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:44.352922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105481+0000","last_change":"2024-04-23T14:46:00.013190+0000","last_active":"2024-04-23T14:46:01.105481+0000","last_peered":"2024-04-23T14:46:01.105481+0000","last_clean":"2024-04-23T14:46:01.105481+0000","last_became_active":"2024-04-23T14:46:00.012880+0000","last_became_peered":"2024-04-23T14:46:00.012880+0000","last_unstale":"2024-04-23T14:46:01.105481+0000","last_undegraded":"2024-04-23T14:46:01.105481+0000","last_fullsized":"2024-04-23T14:46:01.105481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:57.611514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995297+0000","last_change":"2024-04-23T14:46:00.013459+0000","last_active":"2024-04-23T14:46:00.995297+0000","last_peered":"2024-04-23T14:46:00.995297+0000","last_clean":"2024-04-23T14:46:00.995297+0000","last_became_active":"2024-04-23T14:46:00.013106+0000","last_became_peered":"2024-04-23T14:46:00.013106+0000","last_unstale":"2024-04-23T14:46:00.995297+0000","last_undegraded":"2024-04-23T14:46:00.995297+0000","last_fullsized":"2024-04-23T14:46:00.995297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:08:38.849062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106371+0000","last_change":"2024-04-23T14:46:00.030491+0000","last_active":"2024-04-23T14:46:01.106371+0000","last_peered":"2024-04-23T14:46:01.106371+0000","last_clean":"2024-04-23T14:46:01.106371+0000","last_became_active":"2024-04-23T14:46:00.030024+0000","last_became_peered":"2024-04-23T14:46:00.030024+0000","last_unstale":"2024-04-23T14:46:01.106371+0000","last_undegraded":"2024-04-23T14:46:01.106371+0000","last_fullsized":"2024-04-23T14:46:01.106371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:31.289091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.708827+0000","last_change":"2024-04-23T14:46:00.022606+0000","last_active":"2024-04-23T14:46:01.708827+0000","last_peered":"2024-04-23T14:46:01.708827+0000","last_clean":"2024-04-23T14:46:01.708827+0000","last_became_active":"2024-04-23T14:46:00.022155+0000","last_became_peered":"2024-04-23T14:46:00.022155+0000","last_unstale":"2024-04-23T14:46:01.708827+0000","last_undegraded":"2024-04-23T14:46:01.708827+0000","last_fullsized":"2024-04-23T14:46:01.708827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:17:20.250283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.708800+0000","last_change":"2024-04-23T14:46:00.027193+0000","last_active":"2024-04-23T14:46:01.708800+0000","last_peered":"2024-04-23T14:46:01.708800+0000","last_clean":"2024-04-23T14:46:01.708800+0000","last_became_active":"2024-04-23T14:46:00.026211+0000","last_became_peered":"2024-04-23T14:46:00.026211+0000","last_unstale":"2024-04-23T14:46:01.708800+0000","last_undegraded":"2024-04-23T14:46:01.708800+0000","last_fullsized":"2024-04-23T14:46:01.708800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:52.722667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105201+0000","last_change":"2024-04-23T14:46:00.029917+0000","last_active":"2024-04-23T14:46:01.105201+0000","last_peered":"2024-04-23T14:46:01.105201+0000","last_clean":"2024-04-23T14:46:01.105201+0000","last_became_active":"2024-04-23T14:46:00.029563+0000","last_became_peered":"2024-04-23T14:46:00.029563+0000","last_unstale":"2024-04-23T14:46:01.105201+0000","last_undegraded":"2024-04-23T14:46:01.105201+0000","last_fullsized":"2024-04-23T14:46:01.105201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:01.787185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107554+0000","last_change":"2024-04-23T14:46:00.030910+0000","last_active":"2024-04-23T14:46:01.107554+0000","last_peered":"2024-04-23T14:46:01.107554+0000","last_clean":"2024-04-23T14:46:01.107554+0000","last_became_active":"2024-04-23T14:46:00.030555+0000","last_became_peered":"2024-04-23T14:46:00.030555+0000","last_unstale":"2024-04-23T14:46:01.107554+0000","last_undegraded":"2024-04-23T14:46:01.107554+0000","last_fullsized":"2024-04-23T14:46:01.107554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:43.701752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107129+0000","last_change":"2024-04-23T14:46:00.017338+0000","last_active":"2024-04-23T14:46:01.107129+0000","last_peered":"2024-04-23T14:46:01.107129+0000","last_clean":"2024-04-23T14:46:01.107129+0000","last_became_active":"2024-04-23T14:46:00.015381+0000","last_became_peered":"2024-04-23T14:46:00.015381+0000","last_unstale":"2024-04-23T14:46:01.107129+0000","last_undegraded":"2024-04-23T14:46:01.107129+0000","last_fullsized":"2024-04-23T14:46:01.107129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:51:16.762588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995801+0000","last_change":"2024-04-23T14:46:00.033055+0000","last_active":"2024-04-23T14:46:00.995801+0000","last_peered":"2024-04-23T14:46:00.995801+0000","last_clean":"2024-04-23T14:46:00.995801+0000","last_became_active":"2024-04-23T14:46:00.032763+0000","last_became_peered":"2024-04-23T14:46:00.032763+0000","last_unstale":"2024-04-23T14:46:00.995801+0000","last_undegraded":"2024-04-23T14:46:00.995801+0000","last_fullsized":"2024-04-23T14:46:00.995801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:43.793644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106905+0000","last_change":"2024-04-23T14:45:59.232611+0000","last_active":"2024-04-23T14:46:01.106905+0000","last_peered":"2024-04-23T14:46:01.106905+0000","last_clean":"2024-04-23T14:46:01.106905+0000","last_became_active":"2024-04-23T14:45:59.231066+0000","last_became_peered":"2024-04-23T14:45:59.231066+0000","last_unstale":"2024-04-23T14:46:01.106905+0000","last_undegraded":"2024-04-23T14:46:01.106905+0000","last_fullsized":"2024-04-23T14:46:01.106905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:24.204134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.998947+0000","last_change":"2024-04-23T14:46:00.021532+0000","last_active":"2024-04-23T14:46:00.998947+0000","last_peered":"2024-04-23T14:46:00.998947+0000","last_clean":"2024-04-23T14:46:00.998947+0000","last_became_active":"2024-04-23T14:46:00.021125+0000","last_became_peered":"2024-04-23T14:46:00.021125+0000","last_unstale":"2024-04-23T14:46:00.998947+0000","last_undegraded":"2024-04-23T14:46:00.998947+0000","last_fullsized":"2024-04-23T14:46:00.998947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:03.072779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.999100+0000","last_change":"2024-04-23T14:45:59.248925+0000","last_active":"2024-04-23T14:46:00.999100+0000","last_peered":"2024-04-23T14:46:00.999100+0000","last_clean":"2024-04-23T14:46:00.999100+0000","last_became_active":"2024-04-23T14:45:59.248579+0000","last_became_peered":"2024-04-23T14:45:59.248579+0000","last_unstale":"2024-04-23T14:46:00.999100+0000","last_undegraded":"2024-04-23T14:46:00.999100+0000","last_fullsized":"2024-04-23T14:46:00.999100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:17.012610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.710174+0000","last_change":"2024-04-23T14:46:00.028419+0000","last_active":"2024-04-23T14:46:01.710174+0000","last_peered":"2024-04-23T14:46:01.710174+0000","last_clean":"2024-04-23T14:46:01.710174+0000","last_became_active":"2024-04-23T14:46:00.026300+0000","last_became_peered":"2024-04-23T14:46:00.026300+0000","last_unstale":"2024-04-23T14:46:01.710174+0000","last_undegraded":"2024-04-23T14:46:01.710174+0000","last_fullsized":"2024-04-23T14:46:01.710174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:21.784680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106190+0000","last_change":"2024-04-23T14:45:59.225701+0000","last_active":"2024-04-23T14:46:01.106190+0000","last_peered":"2024-04-23T14:46:01.106190+0000","last_clean":"2024-04-23T14:46:01.106190+0000","last_became_active":"2024-04-23T14:45:59.225290+0000","last_became_peered":"2024-04-23T14:45:59.225290+0000","last_unstale":"2024-04-23T14:46:01.106190+0000","last_undegraded":"2024-04-23T14:46:01.106190+0000","last_fullsized":"2024-04-23T14:46:01.106190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:26.877330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105303+0000","last_change":"2024-04-23T14:46:00.035075+0000","last_active":"2024-04-23T14:46:01.105303+0000","last_peered":"2024-04-23T14:46:01.105303+0000","last_clean":"2024-04-23T14:46:01.105303+0000","last_became_active":"2024-04-23T14:46:00.034782+0000","last_became_peered":"2024-04-23T14:46:00.034782+0000","last_unstale":"2024-04-23T14:46:01.105303+0000","last_undegraded":"2024-04-23T14:46:01.105303+0000","last_fullsized":"2024-04-23T14:46:01.105303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:20.747643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106092+0000","last_change":"2024-04-23T14:45:59.227041+0000","last_active":"2024-04-23T14:46:01.106092+0000","last_peered":"2024-04-23T14:46:01.106092+0000","last_clean":"2024-04-23T14:46:01.106092+0000","last_became_active":"2024-04-23T14:45:59.226633+0000","last_became_peered":"2024-04-23T14:45:59.226633+0000","last_unstale":"2024-04-23T14:46:01.106092+0000","last_undegraded":"2024-04-23T14:46:01.106092+0000","last_fullsized":"2024-04-23T14:46:01.106092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:01.548634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107217+0000","last_change":"2024-04-23T14:46:00.008703+0000","last_active":"2024-04-23T14:46:01.107217+0000","last_peered":"2024-04-23T14:46:01.107217+0000","last_clean":"2024-04-23T14:46:01.107217+0000","last_became_active":"2024-04-23T14:46:00.008340+0000","last_became_peered":"2024-04-23T14:46:00.008340+0000","last_unstale":"2024-04-23T14:46:01.107217+0000","last_undegraded":"2024-04-23T14:46:01.107217+0000","last_fullsized":"2024-04-23T14:46:01.107217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:39.824201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:05.314157+0000","last_change":"2024-04-23T14:45:59.226657+0000","last_active":"2024-04-23T14:46:05.314157+0000","last_peered":"2024-04-23T14:46:05.314157+0000","last_clean":"2024-04-23T14:46:05.314157+0000","last_became_active":"2024-04-23T14:45:59.226349+0000","last_became_peered":"2024-04-23T14:45:59.226349+0000","last_unstale":"2024-04-23T14:46:05.314157+0000","last_undegraded":"2024-04-23T14:46:05.314157+0000","last_fullsized":"2024-04-23T14:46:05.314157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:52:36.567060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105922+0000","last_change":"2024-04-23T14:46:00.016700+0000","last_active":"2024-04-23T14:46:01.105922+0000","last_peered":"2024-04-23T14:46:01.105922+0000","last_clean":"2024-04-23T14:46:01.105922+0000","last_became_active":"2024-04-23T14:46:00.015779+0000","last_became_peered":"2024-04-23T14:46:00.015779+0000","last_unstale":"2024-04-23T14:46:01.105922+0000","last_undegraded":"2024-04-23T14:46:01.105922+0000","last_fullsized":"2024-04-23T14:46:01.105922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:19:59.431628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:04.308061+0000","last_change":"2024-04-23T14:45:59.229791+0000","last_active":"2024-04-23T14:46:04.308061+0000","last_peered":"2024-04-23T14:46:04.308061+0000","last_clean":"2024-04-23T14:46:04.308061+0000","last_became_active":"2024-04-23T14:45:59.229054+0000","last_became_peered":"2024-04-23T14:45:59.229054+0000","last_unstale":"2024-04-23T14:46:04.308061+0000","last_undegraded":"2024-04-23T14:46:04.308061+0000","last_fullsized":"2024-04-23T14:46:04.308061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:19:59.385719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106801+0000","last_change":"2024-04-23T14:46:00.021858+0000","last_active":"2024-04-23T14:46:01.106801+0000","last_peered":"2024-04-23T14:46:01.106801+0000","last_clean":"2024-04-23T14:46:01.106801+0000","last_became_active":"2024-04-23T14:46:00.021450+0000","last_became_peered":"2024-04-23T14:46:00.021450+0000","last_unstale":"2024-04-23T14:46:01.106801+0000","last_undegraded":"2024-04-23T14:46:01.106801+0000","last_fullsized":"2024-04-23T14:46:01.106801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:00.270035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:04.311668+0000","last_change":"2024-04-23T14:45:59.238181+0000","last_active":"2024-04-23T14:46:04.311668+0000","last_peered":"2024-04-23T14:46:04.311668+0000","last_clean":"2024-04-23T14:46:04.311668+0000","last_became_active":"2024-04-23T14:45:59.237784+0000","last_became_peered":"2024-04-23T14:45:59.237784+0000","last_unstale":"2024-04-23T14:46:04.311668+0000","last_undegraded":"2024-04-23T14:46:04.311668+0000","last_fullsized":"2024-04-23T14:46:04.311668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:35.016190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.996992+0000","last_change":"2024-04-23T14:46:00.033814+0000","last_active":"2024-04-23T14:46:00.996992+0000","last_peered":"2024-04-23T14:46:00.996992+0000","last_clean":"2024-04-23T14:46:00.996992+0000","last_became_active":"2024-04-23T14:46:00.032128+0000","last_became_peered":"2024-04-23T14:46:00.032128+0000","last_unstale":"2024-04-23T14:46:00.996992+0000","last_undegraded":"2024-04-23T14:46:00.996992+0000","last_fullsized":"2024-04-23T14:46:00.996992+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:32.067418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.250702+0000","last_change":"2024-04-23T14:45:59.228054+0000","last_active":"2024-04-23T14:46:01.250702+0000","last_peered":"2024-04-23T14:46:01.250702+0000","last_clean":"2024-04-23T14:46:01.250702+0000","last_became_active":"2024-04-23T14:45:59.227759+0000","last_became_peered":"2024-04-23T14:45:59.227759+0000","last_unstale":"2024-04-23T14:46:01.250702+0000","last_undegraded":"2024-04-23T14:46:01.250702+0000","last_fullsized":"2024-04-23T14:46:01.250702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:52.185284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106243+0000","last_change":"2024-04-23T14:46:00.028634+0000","last_active":"2024-04-23T14:46:01.106243+0000","last_peered":"2024-04-23T14:46:01.106243+0000","last_clean":"2024-04-23T14:46:01.106243+0000","last_became_active":"2024-04-23T14:46:00.027768+0000","last_became_peered":"2024-04-23T14:46:00.027768+0000","last_unstale":"2024-04-23T14:46:01.106243+0000","last_undegraded":"2024-04-23T14:46:01.106243+0000","last_fullsized":"2024-04-23T14:46:01.106243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:29.869845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.293010+0000","last_change":"2024-04-23T14:45:59.229480+0000","last_active":"2024-04-23T14:46:03.293010+0000","last_peered":"2024-04-23T14:46:03.293010+0000","last_clean":"2024-04-23T14:46:03.293010+0000","last_became_active":"2024-04-23T14:45:59.229124+0000","last_became_peered":"2024-04-23T14:45:59.229124+0000","last_unstale":"2024-04-23T14:46:03.293010+0000","last_undegraded":"2024-04-23T14:46:03.293010+0000","last_fullsized":"2024-04-23T14:46:03.293010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:33.837779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105946+0000","last_change":"2024-04-23T14:46:00.017753+0000","last_active":"2024-04-23T14:46:01.105946+0000","last_peered":"2024-04-23T14:46:01.105946+0000","last_clean":"2024-04-23T14:46:01.105946+0000","last_became_active":"2024-04-23T14:46:00.017370+0000","last_became_peered":"2024-04-23T14:46:00.017370+0000","last_unstale":"2024-04-23T14:46:01.105946+0000","last_undegraded":"2024-04-23T14:46:01.105946+0000","last_fullsized":"2024-04-23T14:46:01.105946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:30:35.460570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.289823+0000","last_change":"2024-04-23T14:45:59.229361+0000","last_active":"2024-04-23T14:46:03.289823+0000","last_peered":"2024-04-23T14:46:03.289823+0000","last_clean":"2024-04-23T14:46:03.289823+0000","last_became_active":"2024-04-23T14:45:59.229043+0000","last_became_peered":"2024-04-23T14:45:59.229043+0000","last_unstale":"2024-04-23T14:46:03.289823+0000","last_undegraded":"2024-04-23T14:46:03.289823+0000","last_fullsized":"2024-04-23T14:46:03.289823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:09.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.710173+0000","last_change":"2024-04-23T14:46:00.030181+0000","last_active":"2024-04-23T14:46:01.710173+0000","last_peered":"2024-04-23T14:46:01.710173+0000","last_clean":"2024-04-23T14:46:01.710173+0000","last_became_active":"2024-04-23T14:46:00.029303+0000","last_became_peered":"2024-04-23T14:46:00.029303+0000","last_unstale":"2024-04-23T14:46:01.710173+0000","last_undegraded":"2024-04-23T14:46:01.710173+0000","last_fullsized":"2024-04-23T14:46:01.710173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:24:53.904501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.289577+0000","last_change":"2024-04-23T14:45:59.231270+0000","last_active":"2024-04-23T14:46:03.289577+0000","last_peered":"2024-04-23T14:46:03.289577+0000","last_clean":"2024-04-23T14:46:03.289577+0000","last_became_active":"2024-04-23T14:45:59.230914+0000","last_became_peered":"2024-04-23T14:45:59.230914+0000","last_unstale":"2024-04-23T14:46:03.289577+0000","last_undegraded":"2024-04-23T14:46:03.289577+0000","last_fullsized":"2024-04-23T14:46:03.289577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.422260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106185+0000","last_change":"2024-04-23T14:46:00.011358+0000","last_active":"2024-04-23T14:46:01.106185+0000","last_peered":"2024-04-23T14:46:01.106185+0000","last_clean":"2024-04-23T14:46:01.106185+0000","last_became_active":"2024-04-23T14:46:00.011040+0000","last_became_peered":"2024-04-23T14:46:00.011040+0000","last_unstale":"2024-04-23T14:46:01.106185+0000","last_undegraded":"2024-04-23T14:46:01.106185+0000","last_fullsized":"2024-04-23T14:46:01.106185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:27.248844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:05.312667+0000","last_change":"2024-04-23T14:45:59.227195+0000","last_active":"2024-04-23T14:46:05.312667+0000","last_peered":"2024-04-23T14:46:05.312667+0000","last_clean":"2024-04-23T14:46:05.312667+0000","last_became_active":"2024-04-23T14:45:59.226874+0000","last_became_peered":"2024-04-23T14:45:59.226874+0000","last_unstale":"2024-04-23T14:46:05.312667+0000","last_undegraded":"2024-04-23T14:46:05.312667+0000","last_fullsized":"2024-04-23T14:46:05.312667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:18:40.203839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106151+0000","last_change":"2024-04-23T14:46:00.010215+0000","last_active":"2024-04-23T14:46:01.106151+0000","last_peered":"2024-04-23T14:46:01.106151+0000","last_clean":"2024-04-23T14:46:01.106151+0000","last_became_active":"2024-04-23T14:46:00.009756+0000","last_became_peered":"2024-04-23T14:46:00.009756+0000","last_unstale":"2024-04-23T14:46:01.106151+0000","last_undegraded":"2024-04-23T14:46:01.106151+0000","last_fullsized":"2024-04-23T14:46:01.106151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.797966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:04.305265+0000","last_change":"2024-04-23T14:45:59.239258+0000","last_active":"2024-04-23T14:46:04.305265+0000","last_peered":"2024-04-23T14:46:04.305265+0000","last_clean":"2024-04-23T14:46:04.305265+0000","last_became_active":"2024-04-23T14:45:59.238899+0000","last_became_peered":"2024-04-23T14:45:59.238899+0000","last_unstale":"2024-04-23T14:46:04.305265+0000","last_undegraded":"2024-04-23T14:46:04.305265+0000","last_fullsized":"2024-04-23T14:46:04.305265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:45.866489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106014+0000","last_change":"2024-04-23T14:46:00.036627+0000","last_active":"2024-04-23T14:46:01.106014+0000","last_peered":"2024-04-23T14:46:01.106014+0000","last_clean":"2024-04-23T14:46:01.106014+0000","last_became_active":"2024-04-23T14:46:00.035397+0000","last_became_peered":"2024-04-23T14:46:00.035397+0000","last_unstale":"2024-04-23T14:46:01.106014+0000","last_undegraded":"2024-04-23T14:46:01.106014+0000","last_fullsized":"2024-04-23T14:46:01.106014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:24:02.038813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.291428+0000","last_change":"2024-04-23T14:45:59.248241+0000","last_active":"2024-04-23T14:46:03.291428+0000","last_peered":"2024-04-23T14:46:03.291428+0000","last_clean":"2024-04-23T14:46:03.291428+0000","last_became_active":"2024-04-23T14:45:59.247895+0000","last_became_peered":"2024-04-23T14:45:59.247895+0000","last_unstale":"2024-04-23T14:46:03.291428+0000","last_undegraded":"2024-04-23T14:46:03.291428+0000","last_fullsized":"2024-04-23T14:46:03.291428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:01.964714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106479+0000","last_change":"2024-04-23T14:46:00.039262+0000","last_active":"2024-04-23T14:46:01.106479+0000","last_peered":"2024-04-23T14:46:01.106479+0000","last_clean":"2024-04-23T14:46:01.106479+0000","last_became_active":"2024-04-23T14:46:00.038906+0000","last_became_peered":"2024-04-23T14:46:00.038906+0000","last_unstale":"2024-04-23T14:46:01.106479+0000","last_undegraded":"2024-04-23T14:46:01.106479+0000","last_fullsized":"2024-04-23T14:46:01.106479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:15:51.444424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:05.314409+0000","last_change":"2024-04-23T14:45:59.227406+0000","last_active":"2024-04-23T14:46:05.314409+0000","last_peered":"2024-04-23T14:46:05.314409+0000","last_clean":"2024-04-23T14:46:05.314409+0000","last_became_active":"2024-04-23T14:45:59.227063+0000","last_became_peered":"2024-04-23T14:45:59.227063+0000","last_unstale":"2024-04-23T14:46:05.314409+0000","last_undegraded":"2024-04-23T14:46:05.314409+0000","last_fullsized":"2024-04-23T14:46:05.314409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:36.845854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105121+0000","last_change":"2024-04-23T14:46:00.006333+0000","last_active":"2024-04-23T14:46:01.105121+0000","last_peered":"2024-04-23T14:46:01.105121+0000","last_clean":"2024-04-23T14:46:01.105121+0000","last_became_active":"2024-04-23T14:46:00.006024+0000","last_became_peered":"2024-04-23T14:46:00.006024+0000","last_unstale":"2024-04-23T14:46:01.105121+0000","last_undegraded":"2024-04-23T14:46:01.105121+0000","last_fullsized":"2024-04-23T14:46:01.105121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:49.389445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105495+0000","last_change":"2024-04-23T14:45:59.226394+0000","last_active":"2024-04-23T14:46:01.105495+0000","last_peered":"2024-04-23T14:46:01.105495+0000","last_clean":"2024-04-23T14:46:01.105495+0000","last_became_active":"2024-04-23T14:45:59.226088+0000","last_became_peered":"2024-04-23T14:45:59.226088+0000","last_unstale":"2024-04-23T14:46:01.105495+0000","last_undegraded":"2024-04-23T14:46:01.105495+0000","last_fullsized":"2024-04-23T14:46:01.105495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:35.139520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106548+0000","last_change":"2024-04-23T14:46:00.009262+0000","last_active":"2024-04-23T14:46:01.106548+0000","last_peered":"2024-04-23T14:46:01.106548+0000","last_clean":"2024-04-23T14:46:01.106548+0000","last_became_active":"2024-04-23T14:46:00.008865+0000","last_became_peered":"2024-04-23T14:46:00.008865+0000","last_unstale":"2024-04-23T14:46:01.106548+0000","last_undegraded":"2024-04-23T14:46:01.106548+0000","last_fullsized":"2024-04-23T14:46:01.106548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:27.662886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106214+0000","last_change":"2024-04-23T14:45:59.223479+0000","last_active":"2024-04-23T14:46:01.106214+0000","last_peered":"2024-04-23T14:46:01.106214+0000","last_clean":"2024-04-23T14:46:01.106214+0000","last_became_active":"2024-04-23T14:45:59.222997+0000","last_became_peered":"2024-04-23T14:45:59.222997+0000","last_unstale":"2024-04-23T14:46:01.106214+0000","last_undegraded":"2024-04-23T14:46:01.106214+0000","last_fullsized":"2024-04-23T14:46:01.106214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:29:58.026918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107161+0000","last_change":"2024-04-23T14:46:00.008050+0000","last_active":"2024-04-23T14:46:01.107161+0000","last_peered":"2024-04-23T14:46:01.107161+0000","last_clean":"2024-04-23T14:46:01.107161+0000","last_became_active":"2024-04-23T14:46:00.007454+0000","last_became_peered":"2024-04-23T14:46:00.007454+0000","last_unstale":"2024-04-23T14:46:01.107161+0000","last_undegraded":"2024-04-23T14:46:01.107161+0000","last_fullsized":"2024-04-23T14:46:01.107161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:38.198952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106653+0000","last_change":"2024-04-23T14:45:59.237813+0000","last_active":"2024-04-23T14:46:01.106653+0000","last_peered":"2024-04-23T14:46:01.106653+0000","last_clean":"2024-04-23T14:46:01.106653+0000","last_became_active":"2024-04-23T14:45:59.237354+0000","last_became_peered":"2024-04-23T14:45:59.237354+0000","last_unstale":"2024-04-23T14:46:01.106653+0000","last_undegraded":"2024-04-23T14:46:01.106653+0000","last_fullsized":"2024-04-23T14:46:01.106653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:35:38.323059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106230+0000","last_change":"2024-04-23T14:46:00.031956+0000","last_active":"2024-04-23T14:46:01.106230+0000","last_peered":"2024-04-23T14:46:01.106230+0000","last_clean":"2024-04-23T14:46:01.106230+0000","last_became_active":"2024-04-23T14:46:00.031666+0000","last_became_peered":"2024-04-23T14:46:00.031666+0000","last_unstale":"2024-04-23T14:46:01.106230+0000","last_undegraded":"2024-04-23T14:46:01.106230+0000","last_fullsized":"2024-04-23T14:46:01.106230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:17.346455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105307+0000","last_change":"2024-04-23T14:45:59.230633+0000","last_active":"2024-04-23T14:46:01.105307+0000","last_peered":"2024-04-23T14:46:01.105307+0000","last_clean":"2024-04-23T14:46:01.105307+0000","last_became_active":"2024-04-23T14:45:59.230300+0000","last_became_peered":"2024-04-23T14:45:59.230300+0000","last_unstale":"2024-04-23T14:46:01.105307+0000","last_undegraded":"2024-04-23T14:46:01.105307+0000","last_fullsized":"2024-04-23T14:46:01.105307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:41.170055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106325+0000","last_change":"2024-04-23T14:46:00.030891+0000","last_active":"2024-04-23T14:46:01.106325+0000","last_peered":"2024-04-23T14:46:01.106325+0000","last_clean":"2024-04-23T14:46:01.106325+0000","last_became_active":"2024-04-23T14:46:00.030438+0000","last_became_peered":"2024-04-23T14:46:00.030438+0000","last_unstale":"2024-04-23T14:46:01.106325+0000","last_undegraded":"2024-04-23T14:46:01.106325+0000","last_fullsized":"2024-04-23T14:46:01.106325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:43.038111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106893+0000","last_change":"2024-04-23T14:46:00.013683+0000","last_active":"2024-04-23T14:46:01.106893+0000","last_peered":"2024-04-23T14:46:01.106893+0000","last_clean":"2024-04-23T14:46:01.106893+0000","last_became_active":"2024-04-23T14:46:00.013356+0000","last_became_peered":"2024-04-23T14:46:00.013356+0000","last_unstale":"2024-04-23T14:46:01.106893+0000","last_undegraded":"2024-04-23T14:46:01.106893+0000","last_fullsized":"2024-04-23T14:46:01.106893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:32.866694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:05.312139+0000","last_change":"2024-04-23T14:45:59.252258+0000","last_active":"2024-04-23T14:46:05.312139+0000","last_peered":"2024-04-23T14:46:05.312139+0000","last_clean":"2024-04-23T14:46:05.312139+0000","last_became_active":"2024-04-23T14:45:59.251513+0000","last_became_peered":"2024-04-23T14:45:59.251513+0000","last_unstale":"2024-04-23T14:46:05.312139+0000","last_undegraded":"2024-04-23T14:46:05.312139+0000","last_fullsized":"2024-04-23T14:46:05.312139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:55:37.087807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.998285+0000","last_change":"2024-04-23T14:46:00.009468+0000","last_active":"2024-04-23T14:46:00.998285+0000","last_peered":"2024-04-23T14:46:00.998285+0000","last_clean":"2024-04-23T14:46:00.998285+0000","last_became_active":"2024-04-23T14:46:00.003118+0000","last_became_peered":"2024-04-23T14:46:00.003118+0000","last_unstale":"2024-04-23T14:46:00.998285+0000","last_undegraded":"2024-04-23T14:46:00.998285+0000","last_fullsized":"2024-04-23T14:46:00.998285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:20:24.488848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:05.310039+0000","last_change":"2024-04-23T14:45:59.225758+0000","last_active":"2024-04-23T14:46:05.310039+0000","last_peered":"2024-04-23T14:46:05.310039+0000","last_clean":"2024-04-23T14:46:05.310039+0000","last_became_active":"2024-04-23T14:45:59.225427+0000","last_became_peered":"2024-04-23T14:45:59.225427+0000","last_unstale":"2024-04-23T14:46:05.310039+0000","last_undegraded":"2024-04-23T14:46:05.310039+0000","last_fullsized":"2024-04-23T14:46:05.310039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:59.495348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":347,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105970+0000","last_change":"2024-04-23T14:44:39.684486+0000","last_active":"2024-04-23T14:46:01.105970+0000","last_peered":"2024-04-23T14:46:01.105970+0000","last_clean":"2024-04-23T14:46:01.105970+0000","last_became_active":"2024-04-23T14:44:39.684074+0000","last_became_peered":"2024-04-23T14:44:39.684074+0000","last_unstale":"2024-04-23T14:46:01.105970+0000","last_undegraded":"2024-04-23T14:46:01.105970+0000","last_fullsized":"2024-04-23T14:46:01.105970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_clean_scrub_stamp":"2024-04-23T14:40:41.333258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:11:16.865449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105713+0000","last_change":"2024-04-23T14:46:00.025018+0000","last_active":"2024-04-23T14:46:01.105713+0000","last_peered":"2024-04-23T14:46:01.105713+0000","last_clean":"2024-04-23T14:46:01.105713+0000","last_became_active":"2024-04-23T14:46:00.024695+0000","last_became_peered":"2024-04-23T14:46:00.024695+0000","last_unstale":"2024-04-23T14:46:01.105713+0000","last_undegraded":"2024-04-23T14:46:01.105713+0000","last_fullsized":"2024-04-23T14:46:01.105713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:00:31.375963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106191+0000","last_change":"2024-04-23T14:45:59.225849+0000","last_active":"2024-04-23T14:46:01.106191+0000","last_peered":"2024-04-23T14:46:01.106191+0000","last_clean":"2024-04-23T14:46:01.106191+0000","last_became_active":"2024-04-23T14:45:59.225402+0000","last_became_peered":"2024-04-23T14:45:59.225402+0000","last_unstale":"2024-04-23T14:46:01.106191+0000","last_undegraded":"2024-04-23T14:46:01.106191+0000","last_fullsized":"2024-04-23T14:46:01.106191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:10:42.217717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105964+0000","last_change":"2024-04-23T14:45:59.240698+0000","last_active":"2024-04-23T14:46:01.105964+0000","last_peered":"2024-04-23T14:46:01.105964+0000","last_clean":"2024-04-23T14:46:01.105964+0000","last_became_active":"2024-04-23T14:45:59.240351+0000","last_became_peered":"2024-04-23T14:45:59.240351+0000","last_unstale":"2024-04-23T14:46:01.105964+0000","last_undegraded":"2024-04-23T14:46:01.105964+0000","last_fullsized":"2024-04-23T14:46:01.105964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:23:51.937010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106284+0000","last_change":"2024-04-23T14:46:00.039566+0000","last_active":"2024-04-23T14:46:01.106284+0000","last_peered":"2024-04-23T14:46:01.106284+0000","last_clean":"2024-04-23T14:46:01.106284+0000","last_became_active":"2024-04-23T14:46:00.039236+0000","last_became_peered":"2024-04-23T14:46:00.039236+0000","last_unstale":"2024-04-23T14:46:01.106284+0000","last_undegraded":"2024-04-23T14:46:01.106284+0000","last_fullsized":"2024-04-23T14:46:01.106284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:44.028367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106941+0000","last_change":"2024-04-23T14:45:59.246192+0000","last_active":"2024-04-23T14:46:01.106941+0000","last_peered":"2024-04-23T14:46:01.106941+0000","last_clean":"2024-04-23T14:46:01.106941+0000","last_became_active":"2024-04-23T14:45:59.245873+0000","last_became_peered":"2024-04-23T14:45:59.245873+0000","last_unstale":"2024-04-23T14:46:01.106941+0000","last_undegraded":"2024-04-23T14:46:01.106941+0000","last_fullsized":"2024-04-23T14:46:01.106941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:22.915746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.999709+0000","last_change":"2024-04-23T14:46:00.029176+0000","last_active":"2024-04-23T14:46:00.999709+0000","last_peered":"2024-04-23T14:46:00.999709+0000","last_clean":"2024-04-23T14:46:00.999709+0000","last_became_active":"2024-04-23T14:46:00.028741+0000","last_became_peered":"2024-04-23T14:46:00.028741+0000","last_unstale":"2024-04-23T14:46:00.999709+0000","last_undegraded":"2024-04-23T14:46:00.999709+0000","last_fullsized":"2024-04-23T14:46:00.999709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:35:22.905348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.290655+0000","last_change":"2024-04-23T14:45:59.231490+0000","last_active":"2024-04-23T14:46:03.290655+0000","last_peered":"2024-04-23T14:46:03.290655+0000","last_clean":"2024-04-23T14:46:03.290655+0000","last_became_active":"2024-04-23T14:45:59.231143+0000","last_became_peered":"2024-04-23T14:45:59.231143+0000","last_unstale":"2024-04-23T14:46:03.290655+0000","last_undegraded":"2024-04-23T14:46:03.290655+0000","last_fullsized":"2024-04-23T14:46:03.290655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:47:26.398150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.709902+0000","last_change":"2024-04-23T14:46:00.035838+0000","last_active":"2024-04-23T14:46:01.709902+0000","last_peered":"2024-04-23T14:46:01.709902+0000","last_clean":"2024-04-23T14:46:01.709902+0000","last_became_active":"2024-04-23T14:46:00.034498+0000","last_became_peered":"2024-04-23T14:46:00.034498+0000","last_unstale":"2024-04-23T14:46:01.709902+0000","last_undegraded":"2024-04-23T14:46:01.709902+0000","last_fullsized":"2024-04-23T14:46:01.709902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:28:15.748503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.290231+0000","last_change":"2024-04-23T14:45:59.240817+0000","last_active":"2024-04-23T14:46:03.290231+0000","last_peered":"2024-04-23T14:46:03.290231+0000","last_clean":"2024-04-23T14:46:03.290231+0000","last_became_active":"2024-04-23T14:45:59.240557+0000","last_became_peered":"2024-04-23T14:45:59.240557+0000","last_unstale":"2024-04-23T14:46:03.290231+0000","last_undegraded":"2024-04-23T14:46:03.290231+0000","last_fullsized":"2024-04-23T14:46:03.290231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:03.099054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105977+0000","last_change":"2024-04-23T14:46:00.009094+0000","last_active":"2024-04-23T14:46:01.105977+0000","last_peered":"2024-04-23T14:46:01.105977+0000","last_clean":"2024-04-23T14:46:01.105977+0000","last_became_active":"2024-04-23T14:46:00.008751+0000","last_became_peered":"2024-04-23T14:46:00.008751+0000","last_unstale":"2024-04-23T14:46:01.105977+0000","last_undegraded":"2024-04-23T14:46:01.105977+0000","last_fullsized":"2024-04-23T14:46:01.105977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:12.480737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.292963+0000","last_change":"2024-04-23T14:45:59.249882+0000","last_active":"2024-04-23T14:46:03.292963+0000","last_peered":"2024-04-23T14:46:03.292963+0000","last_clean":"2024-04-23T14:46:03.292963+0000","last_became_active":"2024-04-23T14:45:59.249072+0000","last_became_peered":"2024-04-23T14:45:59.249072+0000","last_unstale":"2024-04-23T14:46:03.292963+0000","last_undegraded":"2024-04-23T14:46:03.292963+0000","last_fullsized":"2024-04-23T14:46:03.292963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:04:16.573005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106693+0000","last_change":"2024-04-23T14:46:00.030903+0000","last_active":"2024-04-23T14:46:01.106693+0000","last_peered":"2024-04-23T14:46:01.106693+0000","last_clean":"2024-04-23T14:46:01.106693+0000","last_became_active":"2024-04-23T14:46:00.030051+0000","last_became_peered":"2024-04-23T14:46:00.030051+0000","last_unstale":"2024-04-23T14:46:01.106693+0000","last_undegraded":"2024-04-23T14:46:01.106693+0000","last_fullsized":"2024-04-23T14:46:01.106693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:41:15.915795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995532+0000","last_change":"2024-04-23T14:45:59.236446+0000","last_active":"2024-04-23T14:46:00.995532+0000","last_peered":"2024-04-23T14:46:00.995532+0000","last_clean":"2024-04-23T14:46:00.995532+0000","last_became_active":"2024-04-23T14:45:59.236119+0000","last_became_peered":"2024-04-23T14:45:59.236119+0000","last_unstale":"2024-04-23T14:46:00.995532+0000","last_undegraded":"2024-04-23T14:46:00.995532+0000","last_fullsized":"2024-04-23T14:46:00.995532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:40.153626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.997557+0000","last_change":"2024-04-23T14:46:00.035577+0000","last_active":"2024-04-23T14:46:00.997557+0000","last_peered":"2024-04-23T14:46:00.997557+0000","last_clean":"2024-04-23T14:46:00.997557+0000","last_became_active":"2024-04-23T14:46:00.034929+0000","last_became_peered":"2024-04-23T14:46:00.034929+0000","last_unstale":"2024-04-23T14:46:00.997557+0000","last_undegraded":"2024-04-23T14:46:00.997557+0000","last_fullsized":"2024-04-23T14:46:00.997557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:22.844672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105586+0000","last_change":"2024-04-23T14:45:59.227744+0000","last_active":"2024-04-23T14:46:01.105586+0000","last_peered":"2024-04-23T14:46:01.105586+0000","last_clean":"2024-04-23T14:46:01.105586+0000","last_became_active":"2024-04-23T14:45:59.227392+0000","last_became_peered":"2024-04-23T14:45:59.227392+0000","last_unstale":"2024-04-23T14:46:01.105586+0000","last_undegraded":"2024-04-23T14:46:01.105586+0000","last_fullsized":"2024-04-23T14:46:01.105586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:07:52.289620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105609+0000","last_change":"2024-04-23T14:46:00.027934+0000","last_active":"2024-04-23T14:46:01.105609+0000","last_peered":"2024-04-23T14:46:01.105609+0000","last_clean":"2024-04-23T14:46:01.105609+0000","last_became_active":"2024-04-23T14:46:00.027571+0000","last_became_peered":"2024-04-23T14:46:00.027571+0000","last_unstale":"2024-04-23T14:46:01.105609+0000","last_undegraded":"2024-04-23T14:46:01.105609+0000","last_fullsized":"2024-04-23T14:46:01.105609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:54:27.890062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:04.310660+0000","last_change":"2024-04-23T14:45:59.236967+0000","last_active":"2024-04-23T14:46:04.310660+0000","last_peered":"2024-04-23T14:46:04.310660+0000","last_clean":"2024-04-23T14:46:04.310660+0000","last_became_active":"2024-04-23T14:45:59.236612+0000","last_became_peered":"2024-04-23T14:45:59.236612+0000","last_unstale":"2024-04-23T14:46:04.310660+0000","last_undegraded":"2024-04-23T14:46:04.310660+0000","last_fullsized":"2024-04-23T14:46:04.310660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:58:09.422598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106367+0000","last_change":"2024-04-23T14:46:00.009289+0000","last_active":"2024-04-23T14:46:01.106367+0000","last_peered":"2024-04-23T14:46:01.106367+0000","last_clean":"2024-04-23T14:46:01.106367+0000","last_became_active":"2024-04-23T14:46:00.008911+0000","last_became_peered":"2024-04-23T14:46:00.008911+0000","last_unstale":"2024-04-23T14:46:01.106367+0000","last_undegraded":"2024-04-23T14:46:01.106367+0000","last_fullsized":"2024-04-23T14:46:01.106367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:47:55.772442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.250849+0000","last_change":"2024-04-23T14:45:59.239494+0000","last_active":"2024-04-23T14:46:01.250849+0000","last_peered":"2024-04-23T14:46:01.250849+0000","last_clean":"2024-04-23T14:46:01.250849+0000","last_became_active":"2024-04-23T14:45:59.239157+0000","last_became_peered":"2024-04-23T14:45:59.239157+0000","last_unstale":"2024-04-23T14:46:01.250849+0000","last_undegraded":"2024-04-23T14:46:01.250849+0000","last_fullsized":"2024-04-23T14:46:01.250849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:38.275320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105784+0000","last_change":"2024-04-23T14:46:00.029401+0000","last_active":"2024-04-23T14:46:01.105784+0000","last_peered":"2024-04-23T14:46:01.105784+0000","last_clean":"2024-04-23T14:46:01.105784+0000","last_became_active":"2024-04-23T14:46:00.029006+0000","last_became_peered":"2024-04-23T14:46:00.029006+0000","last_unstale":"2024-04-23T14:46:01.105784+0000","last_undegraded":"2024-04-23T14:46:01.105784+0000","last_fullsized":"2024-04-23T14:46:01.105784+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:11.617083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106635+0000","last_change":"2024-04-23T14:45:59.247636+0000","last_active":"2024-04-23T14:46:01.106635+0000","last_peered":"2024-04-23T14:46:01.106635+0000","last_clean":"2024-04-23T14:46:01.106635+0000","last_became_active":"2024-04-23T14:45:59.247231+0000","last_became_peered":"2024-04-23T14:45:59.247231+0000","last_unstale":"2024-04-23T14:46:01.106635+0000","last_undegraded":"2024-04-23T14:46:01.106635+0000","last_fullsized":"2024-04-23T14:46:01.106635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:54.291463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106618+0000","last_change":"2024-04-23T14:46:00.010825+0000","last_active":"2024-04-23T14:46:01.106618+0000","last_peered":"2024-04-23T14:46:01.106618+0000","last_clean":"2024-04-23T14:46:01.106618+0000","last_became_active":"2024-04-23T14:46:00.010470+0000","last_became_peered":"2024-04-23T14:46:00.010470+0000","last_unstale":"2024-04-23T14:46:01.106618+0000","last_undegraded":"2024-04-23T14:46:01.106618+0000","last_fullsized":"2024-04-23T14:46:01.106618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:19.851825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106724+0000","last_change":"2024-04-23T14:46:00.011967+0000","last_active":"2024-04-23T14:46:01.106724+0000","last_peered":"2024-04-23T14:46:01.106724+0000","last_clean":"2024-04-23T14:46:01.106724+0000","last_became_active":"2024-04-23T14:46:00.011580+0000","last_became_peered":"2024-04-23T14:46:00.011580+0000","last_unstale":"2024-04-23T14:46:01.106724+0000","last_undegraded":"2024-04-23T14:46:01.106724+0000","last_fullsized":"2024-04-23T14:46:01.106724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:25:38.909994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995480+0000","last_change":"2024-04-23T14:46:00.015575+0000","last_active":"2024-04-23T14:46:00.995480+0000","last_peered":"2024-04-23T14:46:00.995480+0000","last_clean":"2024-04-23T14:46:00.995480+0000","last_became_active":"2024-04-23T14:46:00.015235+0000","last_became_peered":"2024-04-23T14:46:00.015235+0000","last_unstale":"2024-04-23T14:46:00.995480+0000","last_undegraded":"2024-04-23T14:46:00.995480+0000","last_fullsized":"2024-04-23T14:46:00.995480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:34.910747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995420+0000","last_change":"2024-04-23T14:46:00.029874+0000","last_active":"2024-04-23T14:46:00.995420+0000","last_peered":"2024-04-23T14:46:00.995420+0000","last_clean":"2024-04-23T14:46:00.995420+0000","last_became_active":"2024-04-23T14:46:00.029520+0000","last_became_peered":"2024-04-23T14:46:00.029520+0000","last_unstale":"2024-04-23T14:46:00.995420+0000","last_undegraded":"2024-04-23T14:46:00.995420+0000","last_fullsized":"2024-04-23T14:46:00.995420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:30.711566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.709214+0000","last_change":"2024-04-23T14:46:00.039545+0000","last_active":"2024-04-23T14:46:01.709214+0000","last_peered":"2024-04-23T14:46:01.709214+0000","last_clean":"2024-04-23T14:46:01.709214+0000","last_became_active":"2024-04-23T14:46:00.039193+0000","last_became_peered":"2024-04-23T14:46:00.039193+0000","last_unstale":"2024-04-23T14:46:01.709214+0000","last_undegraded":"2024-04-23T14:46:01.709214+0000","last_fullsized":"2024-04-23T14:46:01.709214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:18:13.528129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106425+0000","last_change":"2024-04-23T14:46:00.036816+0000","last_active":"2024-04-23T14:46:01.106425+0000","last_peered":"2024-04-23T14:46:01.106425+0000","last_clean":"2024-04-23T14:46:01.106425+0000","last_became_active":"2024-04-23T14:46:00.036372+0000","last_became_peered":"2024-04-23T14:46:00.036372+0000","last_unstale":"2024-04-23T14:46:01.106425+0000","last_undegraded":"2024-04-23T14:46:01.106425+0000","last_fullsized":"2024-04-23T14:46:01.106425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:56.179724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107439+0000","last_change":"2024-04-23T14:46:00.012058+0000","last_active":"2024-04-23T14:46:01.107439+0000","last_peered":"2024-04-23T14:46:01.107439+0000","last_clean":"2024-04-23T14:46:01.107439+0000","last_became_active":"2024-04-23T14:46:00.011595+0000","last_became_peered":"2024-04-23T14:46:00.011595+0000","last_unstale":"2024-04-23T14:46:01.107439+0000","last_undegraded":"2024-04-23T14:46:01.107439+0000","last_fullsized":"2024-04-23T14:46:01.107439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:28.341442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995931+0000","last_change":"2024-04-23T14:46:00.011419+0000","last_active":"2024-04-23T14:46:00.995931+0000","last_peered":"2024-04-23T14:46:00.995931+0000","last_clean":"2024-04-23T14:46:00.995931+0000","last_became_active":"2024-04-23T14:46:00.011004+0000","last_became_peered":"2024-04-23T14:46:00.011004+0000","last_unstale":"2024-04-23T14:46:00.995931+0000","last_undegraded":"2024-04-23T14:46:00.995931+0000","last_fullsized":"2024-04-23T14:46:00.995931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:46:57.451648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.999743+0000","last_change":"2024-04-23T14:46:00.023583+0000","last_active":"2024-04-23T14:46:00.999743+0000","last_peered":"2024-04-23T14:46:00.999743+0000","last_clean":"2024-04-23T14:46:00.999743+0000","last_became_active":"2024-04-23T14:46:00.023195+0000","last_became_peered":"2024-04-23T14:46:00.023195+0000","last_unstale":"2024-04-23T14:46:00.999743+0000","last_undegraded":"2024-04-23T14:46:00.999743+0000","last_fullsized":"2024-04-23T14:46:00.999743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:39.992959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106959+0000","last_change":"2024-04-23T14:46:00.014719+0000","last_active":"2024-04-23T14:46:01.106959+0000","last_peered":"2024-04-23T14:46:01.106959+0000","last_clean":"2024-04-23T14:46:01.106959+0000","last_became_active":"2024-04-23T14:46:00.014349+0000","last_became_peered":"2024-04-23T14:46:00.014349+0000","last_unstale":"2024-04-23T14:46:01.106959+0000","last_undegraded":"2024-04-23T14:46:01.106959+0000","last_fullsized":"2024-04-23T14:46:01.106959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:12.789316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.997537+0000","last_change":"2024-04-23T14:46:00.019162+0000","last_active":"2024-04-23T14:46:00.997537+0000","last_peered":"2024-04-23T14:46:00.997537+0000","last_clean":"2024-04-23T14:46:00.997537+0000","last_became_active":"2024-04-23T14:46:00.018814+0000","last_became_peered":"2024-04-23T14:46:00.018814+0000","last_unstale":"2024-04-23T14:46:00.997537+0000","last_undegraded":"2024-04-23T14:46:00.997537+0000","last_fullsized":"2024-04-23T14:46:00.997537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:10.741148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105312+0000","last_change":"2024-04-23T14:46:00.007254+0000","last_active":"2024-04-23T14:46:01.105312+0000","last_peered":"2024-04-23T14:46:01.105312+0000","last_clean":"2024-04-23T14:46:01.105312+0000","last_became_active":"2024-04-23T14:46:00.006826+0000","last_became_peered":"2024-04-23T14:46:00.006826+0000","last_unstale":"2024-04-23T14:46:01.105312+0000","last_undegraded":"2024-04-23T14:46:01.105312+0000","last_fullsized":"2024-04-23T14:46:01.105312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:29.155244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106425+0000","last_change":"2024-04-23T14:46:00.039265+0000","last_active":"2024-04-23T14:46:01.106425+0000","last_peered":"2024-04-23T14:46:01.106425+0000","last_clean":"2024-04-23T14:46:01.106425+0000","last_became_active":"2024-04-23T14:46:00.038848+0000","last_became_peered":"2024-04-23T14:46:00.038848+0000","last_unstale":"2024-04-23T14:46:01.106425+0000","last_undegraded":"2024-04-23T14:46:01.106425+0000","last_fullsized":"2024-04-23T14:46:01.106425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:10.218655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107259+0000","last_change":"2024-04-23T14:46:00.012134+0000","last_active":"2024-04-23T14:46:01.107259+0000","last_peered":"2024-04-23T14:46:01.107259+0000","last_clean":"2024-04-23T14:46:01.107259+0000","last_became_active":"2024-04-23T14:46:00.011796+0000","last_became_peered":"2024-04-23T14:46:00.011796+0000","last_unstale":"2024-04-23T14:46:01.107259+0000","last_undegraded":"2024-04-23T14:46:01.107259+0000","last_fullsized":"2024-04-23T14:46:01.107259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:41.671419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281220,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5870,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":21,"num_read_kb":18,"num_write":100,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":41092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":443879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":1037437,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776153,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":444461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037792,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":444483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":1035451,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":443969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":452091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":445357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":446841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":443983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":444914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":1035723,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34125,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34125,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34125,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":9953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:46:09.173 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:46:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.415+0000 7fa54a400640 1 -- 172.21.15.83:0/376285646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa544108660 msgr2=0x7fa544108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.415+0000 7fa54a400640 1 --2- 172.21.15.83:0/376285646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa544108660 0x7fa544108a80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa534009900 tx=0x7fa53402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.415+0000 7fa54a400640 1 -- 172.21.15.83:0/376285646 shutdown_connections 2024-04-23T14:46:09.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.415+0000 7fa54a400640 1 --2- 172.21.15.83:0/376285646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa54410a210 0x7fa544110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.415+0000 7fa54a400640 1 --2- 172.21.15.83:0/376285646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa544109850 0x7fa544109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.415+0000 7fa54a400640 1 --2- 172.21.15.83:0/376285646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa544108660 0x7fa544108a80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.415+0000 7fa54a400640 1 -- 172.21.15.83:0/376285646 >> 172.21.15.83:0/376285646 conn(0x7fa544077620 msgr2=0x7fa544079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.415+0000 7fa54a400640 1 -- 172.21.15.83:0/376285646 shutdown_connections 2024-04-23T14:46:09.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.415+0000 7fa54a400640 1 -- 172.21.15.83:0/376285646 wait complete. 2024-04-23T14:46:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.416+0000 7fa54a400640 1 Processor -- start 2024-04-23T14:46:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.416+0000 7fa54a400640 1 -- start start 2024-04-23T14:46:09.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa54a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa544108660 0x7fa54419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa54a400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa544109850 0x7fa54419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa54a400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa54410a210 0x7fa5441a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa54a400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa544114330 con 0x7fa544108660 2024-04-23T14:46:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa54a400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441141b0 con 0x7fa54410a210 2024-04-23T14:46:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa54a400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa5441144b0 con 0x7fa544109850 2024-04-23T14:46:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa543400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa544108660 0x7fa54419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa543400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa544108660 0x7fa54419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60130/0 (socket says 172.21.15.83:60130) 2024-04-23T14:46:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa543400640 1 -- 172.21.15.83:0/2587120173 learned_addr learned my addr 172.21.15.83:0/2587120173 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa543e00640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa54410a210 0x7fa5441a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa53be00640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa544109850 0x7fa54419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa543400640 1 -- 172.21.15.83:0/2587120173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa544109850 msgr2=0x7fa54419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:09.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa543400640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa544109850 0x7fa54419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa543400640 1 -- 172.21.15.83:0/2587120173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa54410a210 msgr2=0x7fa5441a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa543400640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa54410a210 0x7fa5441a5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.417+0000 7fa543400640 1 -- 172.21.15.83:0/2587120173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5441a6590 con 0x7fa544108660 2024-04-23T14:46:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.418+0000 7fa543400640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa544108660 0x7fa54419e970 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa534031440 tx=0x7fa534032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.418+0000 7fa540c00640 1 -- 172.21.15.83:0/2587120173 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53403f030 con 0x7fa544108660 2024-04-23T14:46:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.418+0000 7fa540c00640 1 -- 172.21.15.83:0/2587120173 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa534003ce0 con 0x7fa544108660 2024-04-23T14:46:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.418+0000 7fa54a400640 1 -- 172.21.15.83:0/2587120173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5441a6820 con 0x7fa544108660 2024-04-23T14:46:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.418+0000 7fa540c00640 1 -- 172.21.15.83:0/2587120173 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534035070 con 0x7fa544108660 2024-04-23T14:46:09.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.419+0000 7fa54a400640 1 -- 172.21.15.83:0/2587120173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5441a6d10 con 0x7fa544108660 2024-04-23T14:46:09.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.420+0000 7fa54a400640 1 -- 172.21.15.83:0/2587120173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa54410f980 con 0x7fa544108660 2024-04-23T14:46:09.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.422+0000 7fa540c00640 1 -- 172.21.15.83:0/2587120173 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa534042030 con 0x7fa544108660 2024-04-23T14:46:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.423+0000 7fa540c00640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa5280a3c40 0x7fa5280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:09.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.423+0000 7fa540c00640 1 -- 172.21.15.83:0/2587120173 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa5340f0c90 con 0x7fa544108660 2024-04-23T14:46:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.423+0000 7fa53be00640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa5280a3c40 0x7fa5280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:09.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.424+0000 7fa53be00640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa5280a3c40 0x7fa5280a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0095f0 tx=0x7fa52c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:09.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.428+0000 7fa540c00640 1 -- 172.21.15.83:0/2587120173 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5340bcf70 con 0x7fa544108660 2024-04-23T14:46:09.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:09 smithi083 ceph-mon[27867]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:09.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:09 smithi083 ceph-mon[27867]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-23T14:46:09.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:09 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3255808076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:09.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:09 smithi173 ceph-mon[32443]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:09.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:09 smithi173 ceph-mon[32443]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-23T14:46:09.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:09 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3255808076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:09.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.629+0000 7fa54a400640 1 -- 172.21.15.83:0/2587120173 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa544062440 con 0x7fa5280a3c40 2024-04-23T14:46:09.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.652+0000 7fa540c00640 1 -- 172.21.15.83:0/2587120173 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258910 (secure 0 0 0) 0x7fa544062440 con 0x7fa5280a3c40 2024-04-23T14:46:09.657 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:46:09.658 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-23T14:46:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.658+0000 7fa54a400640 1 -- 172.21.15.83:0/2587120173 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa5280a3c40 msgr2=0x7fa5280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.658+0000 7fa54a400640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa5280a3c40 0x7fa5280a6100 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa52c0095f0 tx=0x7fa52c005f90 comp rx=0 tx=0).stop 2024-04-23T14:46:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.658+0000 7fa54a400640 1 -- 172.21.15.83:0/2587120173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa544108660 msgr2=0x7fa54419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.658+0000 7fa54a400640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa544108660 0x7fa54419e970 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa534031440 tx=0x7fa534032ce0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.659+0000 7fa54a400640 1 -- 172.21.15.83:0/2587120173 shutdown_connections 2024-04-23T14:46:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.659+0000 7fa54a400640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa5280a3c40 0x7fa5280a6100 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.659+0000 7fa54a400640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa54410a210 0x7fa5441a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.659+0000 7fa54a400640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa544109850 0x7fa54419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.659+0000 7fa54a400640 1 --2- 172.21.15.83:0/2587120173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa544108660 0x7fa54419e970 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.659+0000 7fa54a400640 1 -- 172.21.15.83:0/2587120173 >> 172.21.15.83:0/2587120173 conn(0x7fa544077620 msgr2=0x7fa54410cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.659+0000 7fa54a400640 1 -- 172.21.15.83:0/2587120173 shutdown_connections 2024-04-23T14:46:09.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:09.659+0000 7fa54a400640 1 -- 172.21.15.83:0/2587120173 wait complete. 2024-04-23T14:46:09.679 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-04-23T14:46:09.207278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596489,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":163,"num_read_kb":143,"num_write":315,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377040,"kb_used_data":11248,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564784,"statfs":{"total":1151940427776,"available":1151554338816,"internally_reserved":0,"allocated":11517952,"data_stored":7121442,"data_compressed":107307,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6121,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":134,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.217035"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107436+0000","last_change":"2024-04-23T14:46:00.030934+0000","last_active":"2024-04-23T14:46:01.107436+0000","last_peered":"2024-04-23T14:46:01.107436+0000","last_clean":"2024-04-23T14:46:01.107436+0000","last_became_active":"2024-04-23T14:46:00.030595+0000","last_became_peered":"2024-04-23T14:46:00.030595+0000","last_unstale":"2024-04-23T14:46:01.107436+0000","last_undegraded":"2024-04-23T14:46:01.107436+0000","last_fullsized":"2024-04-23T14:46:01.107436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.814685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.709991+0000","last_change":"2024-04-23T14:46:00.023044+0000","last_active":"2024-04-23T14:46:01.709991+0000","last_peered":"2024-04-23T14:46:01.709991+0000","last_clean":"2024-04-23T14:46:01.709991+0000","last_became_active":"2024-04-23T14:46:00.022441+0000","last_became_peered":"2024-04-23T14:46:00.022441+0000","last_unstale":"2024-04-23T14:46:01.709991+0000","last_undegraded":"2024-04-23T14:46:01.709991+0000","last_fullsized":"2024-04-23T14:46:01.709991+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:24:20.976312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.709488+0000","last_change":"2024-04-23T14:46:00.025397+0000","last_active":"2024-04-23T14:46:01.709488+0000","last_peered":"2024-04-23T14:46:01.709488+0000","last_clean":"2024-04-23T14:46:01.709488+0000","last_became_active":"2024-04-23T14:46:00.024477+0000","last_became_peered":"2024-04-23T14:46:00.024477+0000","last_unstale":"2024-04-23T14:46:01.709488+0000","last_undegraded":"2024-04-23T14:46:01.709488+0000","last_fullsized":"2024-04-23T14:46:01.709488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:16.002964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995288+0000","last_change":"2024-04-23T14:46:00.013960+0000","last_active":"2024-04-23T14:46:00.995288+0000","last_peered":"2024-04-23T14:46:00.995288+0000","last_clean":"2024-04-23T14:46:00.995288+0000","last_became_active":"2024-04-23T14:46:00.013656+0000","last_became_peered":"2024-04-23T14:46:00.013656+0000","last_unstale":"2024-04-23T14:46:00.995288+0000","last_undegraded":"2024-04-23T14:46:00.995288+0000","last_fullsized":"2024-04-23T14:46:00.995288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:45:59.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105934+0000","last_change":"2024-04-23T14:46:00.032336+0000","last_active":"2024-04-23T14:46:01.105934+0000","last_peered":"2024-04-23T14:46:01.105934+0000","last_clean":"2024-04-23T14:46:01.105934+0000","last_became_active":"2024-04-23T14:46:00.032008+0000","last_became_peered":"2024-04-23T14:46:00.032008+0000","last_unstale":"2024-04-23T14:46:01.105934+0000","last_undegraded":"2024-04-23T14:46:01.105934+0000","last_fullsized":"2024-04-23T14:46:01.105934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:13:05.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106168+0000","last_change":"2024-04-23T14:46:00.015457+0000","last_active":"2024-04-23T14:46:01.106168+0000","last_peered":"2024-04-23T14:46:01.106168+0000","last_clean":"2024-04-23T14:46:01.106168+0000","last_became_active":"2024-04-23T14:46:00.015199+0000","last_became_peered":"2024-04-23T14:46:00.015199+0000","last_unstale":"2024-04-23T14:46:01.106168+0000","last_undegraded":"2024-04-23T14:46:01.106168+0000","last_fullsized":"2024-04-23T14:46:01.106168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:39.310993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105197+0000","last_change":"2024-04-23T14:46:00.016554+0000","last_active":"2024-04-23T14:46:01.105197+0000","last_peered":"2024-04-23T14:46:01.105197+0000","last_clean":"2024-04-23T14:46:01.105197+0000","last_became_active":"2024-04-23T14:46:00.016242+0000","last_became_peered":"2024-04-23T14:46:00.016242+0000","last_unstale":"2024-04-23T14:46:01.105197+0000","last_undegraded":"2024-04-23T14:46:01.105197+0000","last_fullsized":"2024-04-23T14:46:01.105197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.596637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.996874+0000","last_change":"2024-04-23T14:46:00.014574+0000","last_active":"2024-04-23T14:46:00.996874+0000","last_peered":"2024-04-23T14:46:00.996874+0000","last_clean":"2024-04-23T14:46:00.996874+0000","last_became_active":"2024-04-23T14:46:00.010413+0000","last_became_peered":"2024-04-23T14:46:00.010413+0000","last_unstale":"2024-04-23T14:46:00.996874+0000","last_undegraded":"2024-04-23T14:46:00.996874+0000","last_fullsized":"2024-04-23T14:46:00.996874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:15:59.631552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106615+0000","last_change":"2024-04-23T14:46:00.005588+0000","last_active":"2024-04-23T14:46:01.106615+0000","last_peered":"2024-04-23T14:46:01.106615+0000","last_clean":"2024-04-23T14:46:01.106615+0000","last_became_active":"2024-04-23T14:46:00.005238+0000","last_became_peered":"2024-04-23T14:46:00.005238+0000","last_unstale":"2024-04-23T14:46:01.106615+0000","last_undegraded":"2024-04-23T14:46:01.106615+0000","last_fullsized":"2024-04-23T14:46:01.106615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:53:40.522929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106130+0000","last_change":"2024-04-23T14:46:00.013773+0000","last_active":"2024-04-23T14:46:01.106130+0000","last_peered":"2024-04-23T14:46:01.106130+0000","last_clean":"2024-04-23T14:46:01.106130+0000","last_became_active":"2024-04-23T14:46:00.013390+0000","last_became_peered":"2024-04-23T14:46:00.013390+0000","last_unstale":"2024-04-23T14:46:01.106130+0000","last_undegraded":"2024-04-23T14:46:01.106130+0000","last_fullsized":"2024-04-23T14:46:01.106130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:44.352922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105481+0000","last_change":"2024-04-23T14:46:00.013190+0000","last_active":"2024-04-23T14:46:01.105481+0000","last_peered":"2024-04-23T14:46:01.105481+0000","last_clean":"2024-04-23T14:46:01.105481+0000","last_became_active":"2024-04-23T14:46:00.012880+0000","last_became_peered":"2024-04-23T14:46:00.012880+0000","last_unstale":"2024-04-23T14:46:01.105481+0000","last_undegraded":"2024-04-23T14:46:01.105481+0000","last_fullsized":"2024-04-23T14:46:01.105481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:57.611514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995297+0000","last_change":"2024-04-23T14:46:00.013459+0000","last_active":"2024-04-23T14:46:00.995297+0000","last_peered":"2024-04-23T14:46:00.995297+0000","last_clean":"2024-04-23T14:46:00.995297+0000","last_became_active":"2024-04-23T14:46:00.013106+0000","last_became_peered":"2024-04-23T14:46:00.013106+0000","last_unstale":"2024-04-23T14:46:00.995297+0000","last_undegraded":"2024-04-23T14:46:00.995297+0000","last_fullsized":"2024-04-23T14:46:00.995297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:08:38.849062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106371+0000","last_change":"2024-04-23T14:46:00.030491+0000","last_active":"2024-04-23T14:46:01.106371+0000","last_peered":"2024-04-23T14:46:01.106371+0000","last_clean":"2024-04-23T14:46:01.106371+0000","last_became_active":"2024-04-23T14:46:00.030024+0000","last_became_peered":"2024-04-23T14:46:00.030024+0000","last_unstale":"2024-04-23T14:46:01.106371+0000","last_undegraded":"2024-04-23T14:46:01.106371+0000","last_fullsized":"2024-04-23T14:46:01.106371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:31.289091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.708827+0000","last_change":"2024-04-23T14:46:00.022606+0000","last_active":"2024-04-23T14:46:01.708827+0000","last_peered":"2024-04-23T14:46:01.708827+0000","last_clean":"2024-04-23T14:46:01.708827+0000","last_became_active":"2024-04-23T14:46:00.022155+0000","last_became_peered":"2024-04-23T14:46:00.022155+0000","last_unstale":"2024-04-23T14:46:01.708827+0000","last_undegraded":"2024-04-23T14:46:01.708827+0000","last_fullsized":"2024-04-23T14:46:01.708827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:17:20.250283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.708800+0000","last_change":"2024-04-23T14:46:00.027193+0000","last_active":"2024-04-23T14:46:01.708800+0000","last_peered":"2024-04-23T14:46:01.708800+0000","last_clean":"2024-04-23T14:46:01.708800+0000","last_became_active":"2024-04-23T14:46:00.026211+0000","last_became_peered":"2024-04-23T14:46:00.026211+0000","last_unstale":"2024-04-23T14:46:01.708800+0000","last_undegraded":"2024-04-23T14:46:01.708800+0000","last_fullsized":"2024-04-23T14:46:01.708800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:52.722667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105201+0000","last_change":"2024-04-23T14:46:00.029917+0000","last_active":"2024-04-23T14:46:01.105201+0000","last_peered":"2024-04-23T14:46:01.105201+0000","last_clean":"2024-04-23T14:46:01.105201+0000","last_became_active":"2024-04-23T14:46:00.029563+0000","last_became_peered":"2024-04-23T14:46:00.029563+0000","last_unstale":"2024-04-23T14:46:01.105201+0000","last_undegraded":"2024-04-23T14:46:01.105201+0000","last_fullsized":"2024-04-23T14:46:01.105201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:01.787185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107554+0000","last_change":"2024-04-23T14:46:00.030910+0000","last_active":"2024-04-23T14:46:01.107554+0000","last_peered":"2024-04-23T14:46:01.107554+0000","last_clean":"2024-04-23T14:46:01.107554+0000","last_became_active":"2024-04-23T14:46:00.030555+0000","last_became_peered":"2024-04-23T14:46:00.030555+0000","last_unstale":"2024-04-23T14:46:01.107554+0000","last_undegraded":"2024-04-23T14:46:01.107554+0000","last_fullsized":"2024-04-23T14:46:01.107554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:43.701752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107129+0000","last_change":"2024-04-23T14:46:00.017338+0000","last_active":"2024-04-23T14:46:01.107129+0000","last_peered":"2024-04-23T14:46:01.107129+0000","last_clean":"2024-04-23T14:46:01.107129+0000","last_became_active":"2024-04-23T14:46:00.015381+0000","last_became_peered":"2024-04-23T14:46:00.015381+0000","last_unstale":"2024-04-23T14:46:01.107129+0000","last_undegraded":"2024-04-23T14:46:01.107129+0000","last_fullsized":"2024-04-23T14:46:01.107129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:51:16.762588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995801+0000","last_change":"2024-04-23T14:46:00.033055+0000","last_active":"2024-04-23T14:46:00.995801+0000","last_peered":"2024-04-23T14:46:00.995801+0000","last_clean":"2024-04-23T14:46:00.995801+0000","last_became_active":"2024-04-23T14:46:00.032763+0000","last_became_peered":"2024-04-23T14:46:00.032763+0000","last_unstale":"2024-04-23T14:46:00.995801+0000","last_undegraded":"2024-04-23T14:46:00.995801+0000","last_fullsized":"2024-04-23T14:46:00.995801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:43.793644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:04.312149+0000","last_change":"2024-04-23T14:45:59.232611+0000","last_active":"2024-04-23T14:46:04.312149+0000","last_peered":"2024-04-23T14:46:04.312149+0000","last_clean":"2024-04-23T14:46:04.312149+0000","last_became_active":"2024-04-23T14:45:59.231066+0000","last_became_peered":"2024-04-23T14:45:59.231066+0000","last_unstale":"2024-04-23T14:46:04.312149+0000","last_undegraded":"2024-04-23T14:46:04.312149+0000","last_fullsized":"2024-04-23T14:46:04.312149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:24.204134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.998947+0000","last_change":"2024-04-23T14:46:00.021532+0000","last_active":"2024-04-23T14:46:00.998947+0000","last_peered":"2024-04-23T14:46:00.998947+0000","last_clean":"2024-04-23T14:46:00.998947+0000","last_became_active":"2024-04-23T14:46:00.021125+0000","last_became_peered":"2024-04-23T14:46:00.021125+0000","last_unstale":"2024-04-23T14:46:00.998947+0000","last_undegraded":"2024-04-23T14:46:00.998947+0000","last_fullsized":"2024-04-23T14:46:00.998947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:03.072779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.999208+0000","last_change":"2024-04-23T14:45:59.248925+0000","last_active":"2024-04-23T14:46:00.999208+0000","last_peered":"2024-04-23T14:46:00.999208+0000","last_clean":"2024-04-23T14:46:00.999208+0000","last_became_active":"2024-04-23T14:45:59.248579+0000","last_became_peered":"2024-04-23T14:45:59.248579+0000","last_unstale":"2024-04-23T14:46:00.999208+0000","last_undegraded":"2024-04-23T14:46:00.999208+0000","last_fullsized":"2024-04-23T14:46:00.999208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:17.012610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.710174+0000","last_change":"2024-04-23T14:46:00.028419+0000","last_active":"2024-04-23T14:46:01.710174+0000","last_peered":"2024-04-23T14:46:01.710174+0000","last_clean":"2024-04-23T14:46:01.710174+0000","last_became_active":"2024-04-23T14:46:00.026300+0000","last_became_peered":"2024-04-23T14:46:00.026300+0000","last_unstale":"2024-04-23T14:46:01.710174+0000","last_undegraded":"2024-04-23T14:46:01.710174+0000","last_fullsized":"2024-04-23T14:46:01.710174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:21.784680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:04.302224+0000","last_change":"2024-04-23T14:45:59.225701+0000","last_active":"2024-04-23T14:46:04.302224+0000","last_peered":"2024-04-23T14:46:04.302224+0000","last_clean":"2024-04-23T14:46:04.302224+0000","last_became_active":"2024-04-23T14:45:59.225290+0000","last_became_peered":"2024-04-23T14:45:59.225290+0000","last_unstale":"2024-04-23T14:46:04.302224+0000","last_undegraded":"2024-04-23T14:46:04.302224+0000","last_fullsized":"2024-04-23T14:46:04.302224+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:26.877330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105303+0000","last_change":"2024-04-23T14:46:00.035075+0000","last_active":"2024-04-23T14:46:01.105303+0000","last_peered":"2024-04-23T14:46:01.105303+0000","last_clean":"2024-04-23T14:46:01.105303+0000","last_became_active":"2024-04-23T14:46:00.034782+0000","last_became_peered":"2024-04-23T14:46:00.034782+0000","last_unstale":"2024-04-23T14:46:01.105303+0000","last_undegraded":"2024-04-23T14:46:01.105303+0000","last_fullsized":"2024-04-23T14:46:01.105303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:20.747643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106092+0000","last_change":"2024-04-23T14:45:59.227041+0000","last_active":"2024-04-23T14:46:01.106092+0000","last_peered":"2024-04-23T14:46:01.106092+0000","last_clean":"2024-04-23T14:46:01.106092+0000","last_became_active":"2024-04-23T14:45:59.226633+0000","last_became_peered":"2024-04-23T14:45:59.226633+0000","last_unstale":"2024-04-23T14:46:01.106092+0000","last_undegraded":"2024-04-23T14:46:01.106092+0000","last_fullsized":"2024-04-23T14:46:01.106092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:01.548634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107217+0000","last_change":"2024-04-23T14:46:00.008703+0000","last_active":"2024-04-23T14:46:01.107217+0000","last_peered":"2024-04-23T14:46:01.107217+0000","last_clean":"2024-04-23T14:46:01.107217+0000","last_became_active":"2024-04-23T14:46:00.008340+0000","last_became_peered":"2024-04-23T14:46:00.008340+0000","last_unstale":"2024-04-23T14:46:01.107217+0000","last_undegraded":"2024-04-23T14:46:01.107217+0000","last_fullsized":"2024-04-23T14:46:01.107217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:39.824201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:05.314157+0000","last_change":"2024-04-23T14:45:59.226657+0000","last_active":"2024-04-23T14:46:05.314157+0000","last_peered":"2024-04-23T14:46:05.314157+0000","last_clean":"2024-04-23T14:46:05.314157+0000","last_became_active":"2024-04-23T14:45:59.226349+0000","last_became_peered":"2024-04-23T14:45:59.226349+0000","last_unstale":"2024-04-23T14:46:05.314157+0000","last_undegraded":"2024-04-23T14:46:05.314157+0000","last_fullsized":"2024-04-23T14:46:05.314157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:52:36.567060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105922+0000","last_change":"2024-04-23T14:46:00.016700+0000","last_active":"2024-04-23T14:46:01.105922+0000","last_peered":"2024-04-23T14:46:01.105922+0000","last_clean":"2024-04-23T14:46:01.105922+0000","last_became_active":"2024-04-23T14:46:00.015779+0000","last_became_peered":"2024-04-23T14:46:00.015779+0000","last_unstale":"2024-04-23T14:46:01.105922+0000","last_undegraded":"2024-04-23T14:46:01.105922+0000","last_fullsized":"2024-04-23T14:46:01.105922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:19:59.431628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:04.308061+0000","last_change":"2024-04-23T14:45:59.229791+0000","last_active":"2024-04-23T14:46:04.308061+0000","last_peered":"2024-04-23T14:46:04.308061+0000","last_clean":"2024-04-23T14:46:04.308061+0000","last_became_active":"2024-04-23T14:45:59.229054+0000","last_became_peered":"2024-04-23T14:45:59.229054+0000","last_unstale":"2024-04-23T14:46:04.308061+0000","last_undegraded":"2024-04-23T14:46:04.308061+0000","last_fullsized":"2024-04-23T14:46:04.308061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:19:59.385719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106801+0000","last_change":"2024-04-23T14:46:00.021858+0000","last_active":"2024-04-23T14:46:01.106801+0000","last_peered":"2024-04-23T14:46:01.106801+0000","last_clean":"2024-04-23T14:46:01.106801+0000","last_became_active":"2024-04-23T14:46:00.021450+0000","last_became_peered":"2024-04-23T14:46:00.021450+0000","last_unstale":"2024-04-23T14:46:01.106801+0000","last_undegraded":"2024-04-23T14:46:01.106801+0000","last_fullsized":"2024-04-23T14:46:01.106801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:00.270035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:04.311668+0000","last_change":"2024-04-23T14:45:59.238181+0000","last_active":"2024-04-23T14:46:04.311668+0000","last_peered":"2024-04-23T14:46:04.311668+0000","last_clean":"2024-04-23T14:46:04.311668+0000","last_became_active":"2024-04-23T14:45:59.237784+0000","last_became_peered":"2024-04-23T14:45:59.237784+0000","last_unstale":"2024-04-23T14:46:04.311668+0000","last_undegraded":"2024-04-23T14:46:04.311668+0000","last_fullsized":"2024-04-23T14:46:04.311668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:35.016190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.996992+0000","last_change":"2024-04-23T14:46:00.033814+0000","last_active":"2024-04-23T14:46:00.996992+0000","last_peered":"2024-04-23T14:46:00.996992+0000","last_clean":"2024-04-23T14:46:00.996992+0000","last_became_active":"2024-04-23T14:46:00.032128+0000","last_became_peered":"2024-04-23T14:46:00.032128+0000","last_unstale":"2024-04-23T14:46:00.996992+0000","last_undegraded":"2024-04-23T14:46:00.996992+0000","last_fullsized":"2024-04-23T14:46:00.996992+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:32.067418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.250702+0000","last_change":"2024-04-23T14:45:59.228054+0000","last_active":"2024-04-23T14:46:01.250702+0000","last_peered":"2024-04-23T14:46:01.250702+0000","last_clean":"2024-04-23T14:46:01.250702+0000","last_became_active":"2024-04-23T14:45:59.227759+0000","last_became_peered":"2024-04-23T14:45:59.227759+0000","last_unstale":"2024-04-23T14:46:01.250702+0000","last_undegraded":"2024-04-23T14:46:01.250702+0000","last_fullsized":"2024-04-23T14:46:01.250702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:52.185284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106243+0000","last_change":"2024-04-23T14:46:00.028634+0000","last_active":"2024-04-23T14:46:01.106243+0000","last_peered":"2024-04-23T14:46:01.106243+0000","last_clean":"2024-04-23T14:46:01.106243+0000","last_became_active":"2024-04-23T14:46:00.027768+0000","last_became_peered":"2024-04-23T14:46:00.027768+0000","last_unstale":"2024-04-23T14:46:01.106243+0000","last_undegraded":"2024-04-23T14:46:01.106243+0000","last_fullsized":"2024-04-23T14:46:01.106243+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:29.869845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.293010+0000","last_change":"2024-04-23T14:45:59.229480+0000","last_active":"2024-04-23T14:46:03.293010+0000","last_peered":"2024-04-23T14:46:03.293010+0000","last_clean":"2024-04-23T14:46:03.293010+0000","last_became_active":"2024-04-23T14:45:59.229124+0000","last_became_peered":"2024-04-23T14:45:59.229124+0000","last_unstale":"2024-04-23T14:46:03.293010+0000","last_undegraded":"2024-04-23T14:46:03.293010+0000","last_fullsized":"2024-04-23T14:46:03.293010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:33.837779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105946+0000","last_change":"2024-04-23T14:46:00.017753+0000","last_active":"2024-04-23T14:46:01.105946+0000","last_peered":"2024-04-23T14:46:01.105946+0000","last_clean":"2024-04-23T14:46:01.105946+0000","last_became_active":"2024-04-23T14:46:00.017370+0000","last_became_peered":"2024-04-23T14:46:00.017370+0000","last_unstale":"2024-04-23T14:46:01.105946+0000","last_undegraded":"2024-04-23T14:46:01.105946+0000","last_fullsized":"2024-04-23T14:46:01.105946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:30:35.460570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.289823+0000","last_change":"2024-04-23T14:45:59.229361+0000","last_active":"2024-04-23T14:46:03.289823+0000","last_peered":"2024-04-23T14:46:03.289823+0000","last_clean":"2024-04-23T14:46:03.289823+0000","last_became_active":"2024-04-23T14:45:59.229043+0000","last_became_peered":"2024-04-23T14:45:59.229043+0000","last_unstale":"2024-04-23T14:46:03.289823+0000","last_undegraded":"2024-04-23T14:46:03.289823+0000","last_fullsized":"2024-04-23T14:46:03.289823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:09.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.710173+0000","last_change":"2024-04-23T14:46:00.030181+0000","last_active":"2024-04-23T14:46:01.710173+0000","last_peered":"2024-04-23T14:46:01.710173+0000","last_clean":"2024-04-23T14:46:01.710173+0000","last_became_active":"2024-04-23T14:46:00.029303+0000","last_became_peered":"2024-04-23T14:46:00.029303+0000","last_unstale":"2024-04-23T14:46:01.710173+0000","last_undegraded":"2024-04-23T14:46:01.710173+0000","last_fullsized":"2024-04-23T14:46:01.710173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:24:53.904501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.289577+0000","last_change":"2024-04-23T14:45:59.231270+0000","last_active":"2024-04-23T14:46:03.289577+0000","last_peered":"2024-04-23T14:46:03.289577+0000","last_clean":"2024-04-23T14:46:03.289577+0000","last_became_active":"2024-04-23T14:45:59.230914+0000","last_became_peered":"2024-04-23T14:45:59.230914+0000","last_unstale":"2024-04-23T14:46:03.289577+0000","last_undegraded":"2024-04-23T14:46:03.289577+0000","last_fullsized":"2024-04-23T14:46:03.289577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.422260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106185+0000","last_change":"2024-04-23T14:46:00.011358+0000","last_active":"2024-04-23T14:46:01.106185+0000","last_peered":"2024-04-23T14:46:01.106185+0000","last_clean":"2024-04-23T14:46:01.106185+0000","last_became_active":"2024-04-23T14:46:00.011040+0000","last_became_peered":"2024-04-23T14:46:00.011040+0000","last_unstale":"2024-04-23T14:46:01.106185+0000","last_undegraded":"2024-04-23T14:46:01.106185+0000","last_fullsized":"2024-04-23T14:46:01.106185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:27.248844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:05.312667+0000","last_change":"2024-04-23T14:45:59.227195+0000","last_active":"2024-04-23T14:46:05.312667+0000","last_peered":"2024-04-23T14:46:05.312667+0000","last_clean":"2024-04-23T14:46:05.312667+0000","last_became_active":"2024-04-23T14:45:59.226874+0000","last_became_peered":"2024-04-23T14:45:59.226874+0000","last_unstale":"2024-04-23T14:46:05.312667+0000","last_undegraded":"2024-04-23T14:46:05.312667+0000","last_fullsized":"2024-04-23T14:46:05.312667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:18:40.203839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106151+0000","last_change":"2024-04-23T14:46:00.010215+0000","last_active":"2024-04-23T14:46:01.106151+0000","last_peered":"2024-04-23T14:46:01.106151+0000","last_clean":"2024-04-23T14:46:01.106151+0000","last_became_active":"2024-04-23T14:46:00.009756+0000","last_became_peered":"2024-04-23T14:46:00.009756+0000","last_unstale":"2024-04-23T14:46:01.106151+0000","last_undegraded":"2024-04-23T14:46:01.106151+0000","last_fullsized":"2024-04-23T14:46:01.106151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.797966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:04.305265+0000","last_change":"2024-04-23T14:45:59.239258+0000","last_active":"2024-04-23T14:46:04.305265+0000","last_peered":"2024-04-23T14:46:04.305265+0000","last_clean":"2024-04-23T14:46:04.305265+0000","last_became_active":"2024-04-23T14:45:59.238899+0000","last_became_peered":"2024-04-23T14:45:59.238899+0000","last_unstale":"2024-04-23T14:46:04.305265+0000","last_undegraded":"2024-04-23T14:46:04.305265+0000","last_fullsized":"2024-04-23T14:46:04.305265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:45.866489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106014+0000","last_change":"2024-04-23T14:46:00.036627+0000","last_active":"2024-04-23T14:46:01.106014+0000","last_peered":"2024-04-23T14:46:01.106014+0000","last_clean":"2024-04-23T14:46:01.106014+0000","last_became_active":"2024-04-23T14:46:00.035397+0000","last_became_peered":"2024-04-23T14:46:00.035397+0000","last_unstale":"2024-04-23T14:46:01.106014+0000","last_undegraded":"2024-04-23T14:46:01.106014+0000","last_fullsized":"2024-04-23T14:46:01.106014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:24:02.038813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.291428+0000","last_change":"2024-04-23T14:45:59.248241+0000","last_active":"2024-04-23T14:46:03.291428+0000","last_peered":"2024-04-23T14:46:03.291428+0000","last_clean":"2024-04-23T14:46:03.291428+0000","last_became_active":"2024-04-23T14:45:59.247895+0000","last_became_peered":"2024-04-23T14:45:59.247895+0000","last_unstale":"2024-04-23T14:46:03.291428+0000","last_undegraded":"2024-04-23T14:46:03.291428+0000","last_fullsized":"2024-04-23T14:46:03.291428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:01.964714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106479+0000","last_change":"2024-04-23T14:46:00.039262+0000","last_active":"2024-04-23T14:46:01.106479+0000","last_peered":"2024-04-23T14:46:01.106479+0000","last_clean":"2024-04-23T14:46:01.106479+0000","last_became_active":"2024-04-23T14:46:00.038906+0000","last_became_peered":"2024-04-23T14:46:00.038906+0000","last_unstale":"2024-04-23T14:46:01.106479+0000","last_undegraded":"2024-04-23T14:46:01.106479+0000","last_fullsized":"2024-04-23T14:46:01.106479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:15:51.444424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:05.314409+0000","last_change":"2024-04-23T14:45:59.227406+0000","last_active":"2024-04-23T14:46:05.314409+0000","last_peered":"2024-04-23T14:46:05.314409+0000","last_clean":"2024-04-23T14:46:05.314409+0000","last_became_active":"2024-04-23T14:45:59.227063+0000","last_became_peered":"2024-04-23T14:45:59.227063+0000","last_unstale":"2024-04-23T14:46:05.314409+0000","last_undegraded":"2024-04-23T14:46:05.314409+0000","last_fullsized":"2024-04-23T14:46:05.314409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:36.845854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105121+0000","last_change":"2024-04-23T14:46:00.006333+0000","last_active":"2024-04-23T14:46:01.105121+0000","last_peered":"2024-04-23T14:46:01.105121+0000","last_clean":"2024-04-23T14:46:01.105121+0000","last_became_active":"2024-04-23T14:46:00.006024+0000","last_became_peered":"2024-04-23T14:46:00.006024+0000","last_unstale":"2024-04-23T14:46:01.105121+0000","last_undegraded":"2024-04-23T14:46:01.105121+0000","last_fullsized":"2024-04-23T14:46:01.105121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:49.389445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:04.310938+0000","last_change":"2024-04-23T14:45:59.226394+0000","last_active":"2024-04-23T14:46:04.310938+0000","last_peered":"2024-04-23T14:46:04.310938+0000","last_clean":"2024-04-23T14:46:04.310938+0000","last_became_active":"2024-04-23T14:45:59.226088+0000","last_became_peered":"2024-04-23T14:45:59.226088+0000","last_unstale":"2024-04-23T14:46:04.310938+0000","last_undegraded":"2024-04-23T14:46:04.310938+0000","last_fullsized":"2024-04-23T14:46:04.310938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:35.139520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106548+0000","last_change":"2024-04-23T14:46:00.009262+0000","last_active":"2024-04-23T14:46:01.106548+0000","last_peered":"2024-04-23T14:46:01.106548+0000","last_clean":"2024-04-23T14:46:01.106548+0000","last_became_active":"2024-04-23T14:46:00.008865+0000","last_became_peered":"2024-04-23T14:46:00.008865+0000","last_unstale":"2024-04-23T14:46:01.106548+0000","last_undegraded":"2024-04-23T14:46:01.106548+0000","last_fullsized":"2024-04-23T14:46:01.106548+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:27.662886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:05.300039+0000","last_change":"2024-04-23T14:45:59.223479+0000","last_active":"2024-04-23T14:46:05.300039+0000","last_peered":"2024-04-23T14:46:05.300039+0000","last_clean":"2024-04-23T14:46:05.300039+0000","last_became_active":"2024-04-23T14:45:59.222997+0000","last_became_peered":"2024-04-23T14:45:59.222997+0000","last_unstale":"2024-04-23T14:46:05.300039+0000","last_undegraded":"2024-04-23T14:46:05.300039+0000","last_fullsized":"2024-04-23T14:46:05.300039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:29:58.026918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107161+0000","last_change":"2024-04-23T14:46:00.008050+0000","last_active":"2024-04-23T14:46:01.107161+0000","last_peered":"2024-04-23T14:46:01.107161+0000","last_clean":"2024-04-23T14:46:01.107161+0000","last_became_active":"2024-04-23T14:46:00.007454+0000","last_became_peered":"2024-04-23T14:46:00.007454+0000","last_unstale":"2024-04-23T14:46:01.107161+0000","last_undegraded":"2024-04-23T14:46:01.107161+0000","last_fullsized":"2024-04-23T14:46:01.107161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:43:38.198952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106653+0000","last_change":"2024-04-23T14:45:59.237813+0000","last_active":"2024-04-23T14:46:01.106653+0000","last_peered":"2024-04-23T14:46:01.106653+0000","last_clean":"2024-04-23T14:46:01.106653+0000","last_became_active":"2024-04-23T14:45:59.237354+0000","last_became_peered":"2024-04-23T14:45:59.237354+0000","last_unstale":"2024-04-23T14:46:01.106653+0000","last_undegraded":"2024-04-23T14:46:01.106653+0000","last_fullsized":"2024-04-23T14:46:01.106653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:35:38.323059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106230+0000","last_change":"2024-04-23T14:46:00.031956+0000","last_active":"2024-04-23T14:46:01.106230+0000","last_peered":"2024-04-23T14:46:01.106230+0000","last_clean":"2024-04-23T14:46:01.106230+0000","last_became_active":"2024-04-23T14:46:00.031666+0000","last_became_peered":"2024-04-23T14:46:00.031666+0000","last_unstale":"2024-04-23T14:46:01.106230+0000","last_undegraded":"2024-04-23T14:46:01.106230+0000","last_fullsized":"2024-04-23T14:46:01.106230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:17.346455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105355+0000","last_change":"2024-04-23T14:45:59.230633+0000","last_active":"2024-04-23T14:46:01.105355+0000","last_peered":"2024-04-23T14:46:01.105355+0000","last_clean":"2024-04-23T14:46:01.105355+0000","last_became_active":"2024-04-23T14:45:59.230300+0000","last_became_peered":"2024-04-23T14:45:59.230300+0000","last_unstale":"2024-04-23T14:46:01.105355+0000","last_undegraded":"2024-04-23T14:46:01.105355+0000","last_fullsized":"2024-04-23T14:46:01.105355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:41.170055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106325+0000","last_change":"2024-04-23T14:46:00.030891+0000","last_active":"2024-04-23T14:46:01.106325+0000","last_peered":"2024-04-23T14:46:01.106325+0000","last_clean":"2024-04-23T14:46:01.106325+0000","last_became_active":"2024-04-23T14:46:00.030438+0000","last_became_peered":"2024-04-23T14:46:00.030438+0000","last_unstale":"2024-04-23T14:46:01.106325+0000","last_undegraded":"2024-04-23T14:46:01.106325+0000","last_fullsized":"2024-04-23T14:46:01.106325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:43.038111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106893+0000","last_change":"2024-04-23T14:46:00.013683+0000","last_active":"2024-04-23T14:46:01.106893+0000","last_peered":"2024-04-23T14:46:01.106893+0000","last_clean":"2024-04-23T14:46:01.106893+0000","last_became_active":"2024-04-23T14:46:00.013356+0000","last_became_peered":"2024-04-23T14:46:00.013356+0000","last_unstale":"2024-04-23T14:46:01.106893+0000","last_undegraded":"2024-04-23T14:46:01.106893+0000","last_fullsized":"2024-04-23T14:46:01.106893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:32.866694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:05.312139+0000","last_change":"2024-04-23T14:45:59.252258+0000","last_active":"2024-04-23T14:46:05.312139+0000","last_peered":"2024-04-23T14:46:05.312139+0000","last_clean":"2024-04-23T14:46:05.312139+0000","last_became_active":"2024-04-23T14:45:59.251513+0000","last_became_peered":"2024-04-23T14:45:59.251513+0000","last_unstale":"2024-04-23T14:46:05.312139+0000","last_undegraded":"2024-04-23T14:46:05.312139+0000","last_fullsized":"2024-04-23T14:46:05.312139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:55:37.087807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.998285+0000","last_change":"2024-04-23T14:46:00.009468+0000","last_active":"2024-04-23T14:46:00.998285+0000","last_peered":"2024-04-23T14:46:00.998285+0000","last_clean":"2024-04-23T14:46:00.998285+0000","last_became_active":"2024-04-23T14:46:00.003118+0000","last_became_peered":"2024-04-23T14:46:00.003118+0000","last_unstale":"2024-04-23T14:46:00.998285+0000","last_undegraded":"2024-04-23T14:46:00.998285+0000","last_fullsized":"2024-04-23T14:46:00.998285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:20:24.488848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:05.310039+0000","last_change":"2024-04-23T14:45:59.225758+0000","last_active":"2024-04-23T14:46:05.310039+0000","last_peered":"2024-04-23T14:46:05.310039+0000","last_clean":"2024-04-23T14:46:05.310039+0000","last_became_active":"2024-04-23T14:45:59.225427+0000","last_became_peered":"2024-04-23T14:45:59.225427+0000","last_unstale":"2024-04-23T14:46:05.310039+0000","last_undegraded":"2024-04-23T14:46:05.310039+0000","last_fullsized":"2024-04-23T14:46:05.310039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:59.495348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":347,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105970+0000","last_change":"2024-04-23T14:44:39.684486+0000","last_active":"2024-04-23T14:46:01.105970+0000","last_peered":"2024-04-23T14:46:01.105970+0000","last_clean":"2024-04-23T14:46:01.105970+0000","last_became_active":"2024-04-23T14:44:39.684074+0000","last_became_peered":"2024-04-23T14:44:39.684074+0000","last_unstale":"2024-04-23T14:46:01.105970+0000","last_undegraded":"2024-04-23T14:46:01.105970+0000","last_fullsized":"2024-04-23T14:46:01.105970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_clean_scrub_stamp":"2024-04-23T14:40:41.333258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:11:16.865449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105713+0000","last_change":"2024-04-23T14:46:00.025018+0000","last_active":"2024-04-23T14:46:01.105713+0000","last_peered":"2024-04-23T14:46:01.105713+0000","last_clean":"2024-04-23T14:46:01.105713+0000","last_became_active":"2024-04-23T14:46:00.024695+0000","last_became_peered":"2024-04-23T14:46:00.024695+0000","last_unstale":"2024-04-23T14:46:01.105713+0000","last_undegraded":"2024-04-23T14:46:01.105713+0000","last_fullsized":"2024-04-23T14:46:01.105713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:00:31.375963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:04.304149+0000","last_change":"2024-04-23T14:45:59.225849+0000","last_active":"2024-04-23T14:46:04.304149+0000","last_peered":"2024-04-23T14:46:04.304149+0000","last_clean":"2024-04-23T14:46:04.304149+0000","last_became_active":"2024-04-23T14:45:59.225402+0000","last_became_peered":"2024-04-23T14:45:59.225402+0000","last_unstale":"2024-04-23T14:46:04.304149+0000","last_undegraded":"2024-04-23T14:46:04.304149+0000","last_fullsized":"2024-04-23T14:46:04.304149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:10:42.217717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.291067+0000","last_change":"2024-04-23T14:45:59.240698+0000","last_active":"2024-04-23T14:46:03.291067+0000","last_peered":"2024-04-23T14:46:03.291067+0000","last_clean":"2024-04-23T14:46:03.291067+0000","last_became_active":"2024-04-23T14:45:59.240351+0000","last_became_peered":"2024-04-23T14:45:59.240351+0000","last_unstale":"2024-04-23T14:46:03.291067+0000","last_undegraded":"2024-04-23T14:46:03.291067+0000","last_fullsized":"2024-04-23T14:46:03.291067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:23:51.937010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106284+0000","last_change":"2024-04-23T14:46:00.039566+0000","last_active":"2024-04-23T14:46:01.106284+0000","last_peered":"2024-04-23T14:46:01.106284+0000","last_clean":"2024-04-23T14:46:01.106284+0000","last_became_active":"2024-04-23T14:46:00.039236+0000","last_became_peered":"2024-04-23T14:46:00.039236+0000","last_unstale":"2024-04-23T14:46:01.106284+0000","last_undegraded":"2024-04-23T14:46:01.106284+0000","last_fullsized":"2024-04-23T14:46:01.106284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:44.028367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:07.224697+0000","last_change":"2024-04-23T14:45:59.246192+0000","last_active":"2024-04-23T14:46:07.224697+0000","last_peered":"2024-04-23T14:46:07.224697+0000","last_clean":"2024-04-23T14:46:07.224697+0000","last_became_active":"2024-04-23T14:45:59.245873+0000","last_became_peered":"2024-04-23T14:45:59.245873+0000","last_unstale":"2024-04-23T14:46:07.224697+0000","last_undegraded":"2024-04-23T14:46:07.224697+0000","last_fullsized":"2024-04-23T14:46:07.224697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:22.915746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.999709+0000","last_change":"2024-04-23T14:46:00.029176+0000","last_active":"2024-04-23T14:46:00.999709+0000","last_peered":"2024-04-23T14:46:00.999709+0000","last_clean":"2024-04-23T14:46:00.999709+0000","last_became_active":"2024-04-23T14:46:00.028741+0000","last_became_peered":"2024-04-23T14:46:00.028741+0000","last_unstale":"2024-04-23T14:46:00.999709+0000","last_undegraded":"2024-04-23T14:46:00.999709+0000","last_fullsized":"2024-04-23T14:46:00.999709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:35:22.905348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.290655+0000","last_change":"2024-04-23T14:45:59.231490+0000","last_active":"2024-04-23T14:46:03.290655+0000","last_peered":"2024-04-23T14:46:03.290655+0000","last_clean":"2024-04-23T14:46:03.290655+0000","last_became_active":"2024-04-23T14:45:59.231143+0000","last_became_peered":"2024-04-23T14:45:59.231143+0000","last_unstale":"2024-04-23T14:46:03.290655+0000","last_undegraded":"2024-04-23T14:46:03.290655+0000","last_fullsized":"2024-04-23T14:46:03.290655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:47:26.398150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.709902+0000","last_change":"2024-04-23T14:46:00.035838+0000","last_active":"2024-04-23T14:46:01.709902+0000","last_peered":"2024-04-23T14:46:01.709902+0000","last_clean":"2024-04-23T14:46:01.709902+0000","last_became_active":"2024-04-23T14:46:00.034498+0000","last_became_peered":"2024-04-23T14:46:00.034498+0000","last_unstale":"2024-04-23T14:46:01.709902+0000","last_undegraded":"2024-04-23T14:46:01.709902+0000","last_fullsized":"2024-04-23T14:46:01.709902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:28:15.748503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.290231+0000","last_change":"2024-04-23T14:45:59.240817+0000","last_active":"2024-04-23T14:46:03.290231+0000","last_peered":"2024-04-23T14:46:03.290231+0000","last_clean":"2024-04-23T14:46:03.290231+0000","last_became_active":"2024-04-23T14:45:59.240557+0000","last_became_peered":"2024-04-23T14:45:59.240557+0000","last_unstale":"2024-04-23T14:46:03.290231+0000","last_undegraded":"2024-04-23T14:46:03.290231+0000","last_fullsized":"2024-04-23T14:46:03.290231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:03.099054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105977+0000","last_change":"2024-04-23T14:46:00.009094+0000","last_active":"2024-04-23T14:46:01.105977+0000","last_peered":"2024-04-23T14:46:01.105977+0000","last_clean":"2024-04-23T14:46:01.105977+0000","last_became_active":"2024-04-23T14:46:00.008751+0000","last_became_peered":"2024-04-23T14:46:00.008751+0000","last_unstale":"2024-04-23T14:46:01.105977+0000","last_undegraded":"2024-04-23T14:46:01.105977+0000","last_fullsized":"2024-04-23T14:46:01.105977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:12.480737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.292963+0000","last_change":"2024-04-23T14:45:59.249882+0000","last_active":"2024-04-23T14:46:03.292963+0000","last_peered":"2024-04-23T14:46:03.292963+0000","last_clean":"2024-04-23T14:46:03.292963+0000","last_became_active":"2024-04-23T14:45:59.249072+0000","last_became_peered":"2024-04-23T14:45:59.249072+0000","last_unstale":"2024-04-23T14:46:03.292963+0000","last_undegraded":"2024-04-23T14:46:03.292963+0000","last_fullsized":"2024-04-23T14:46:03.292963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:04:16.573005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106693+0000","last_change":"2024-04-23T14:46:00.030903+0000","last_active":"2024-04-23T14:46:01.106693+0000","last_peered":"2024-04-23T14:46:01.106693+0000","last_clean":"2024-04-23T14:46:01.106693+0000","last_became_active":"2024-04-23T14:46:00.030051+0000","last_became_peered":"2024-04-23T14:46:00.030051+0000","last_unstale":"2024-04-23T14:46:01.106693+0000","last_undegraded":"2024-04-23T14:46:01.106693+0000","last_fullsized":"2024-04-23T14:46:01.106693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:41:15.915795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995532+0000","last_change":"2024-04-23T14:45:59.236446+0000","last_active":"2024-04-23T14:46:00.995532+0000","last_peered":"2024-04-23T14:46:00.995532+0000","last_clean":"2024-04-23T14:46:00.995532+0000","last_became_active":"2024-04-23T14:45:59.236119+0000","last_became_peered":"2024-04-23T14:45:59.236119+0000","last_unstale":"2024-04-23T14:46:00.995532+0000","last_undegraded":"2024-04-23T14:46:00.995532+0000","last_fullsized":"2024-04-23T14:46:00.995532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:40.153626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.997557+0000","last_change":"2024-04-23T14:46:00.035577+0000","last_active":"2024-04-23T14:46:00.997557+0000","last_peered":"2024-04-23T14:46:00.997557+0000","last_clean":"2024-04-23T14:46:00.997557+0000","last_became_active":"2024-04-23T14:46:00.034929+0000","last_became_peered":"2024-04-23T14:46:00.034929+0000","last_unstale":"2024-04-23T14:46:00.997557+0000","last_undegraded":"2024-04-23T14:46:00.997557+0000","last_fullsized":"2024-04-23T14:46:00.997557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:22.844672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.290057+0000","last_change":"2024-04-23T14:45:59.227744+0000","last_active":"2024-04-23T14:46:03.290057+0000","last_peered":"2024-04-23T14:46:03.290057+0000","last_clean":"2024-04-23T14:46:03.290057+0000","last_became_active":"2024-04-23T14:45:59.227392+0000","last_became_peered":"2024-04-23T14:45:59.227392+0000","last_unstale":"2024-04-23T14:46:03.290057+0000","last_undegraded":"2024-04-23T14:46:03.290057+0000","last_fullsized":"2024-04-23T14:46:03.290057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:07:52.289620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105609+0000","last_change":"2024-04-23T14:46:00.027934+0000","last_active":"2024-04-23T14:46:01.105609+0000","last_peered":"2024-04-23T14:46:01.105609+0000","last_clean":"2024-04-23T14:46:01.105609+0000","last_became_active":"2024-04-23T14:46:00.027571+0000","last_became_peered":"2024-04-23T14:46:00.027571+0000","last_unstale":"2024-04-23T14:46:01.105609+0000","last_undegraded":"2024-04-23T14:46:01.105609+0000","last_fullsized":"2024-04-23T14:46:01.105609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:54:27.890062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:04.310660+0000","last_change":"2024-04-23T14:45:59.236967+0000","last_active":"2024-04-23T14:46:04.310660+0000","last_peered":"2024-04-23T14:46:04.310660+0000","last_clean":"2024-04-23T14:46:04.310660+0000","last_became_active":"2024-04-23T14:45:59.236612+0000","last_became_peered":"2024-04-23T14:45:59.236612+0000","last_unstale":"2024-04-23T14:46:04.310660+0000","last_undegraded":"2024-04-23T14:46:04.310660+0000","last_fullsized":"2024-04-23T14:46:04.310660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:58:09.422598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106367+0000","last_change":"2024-04-23T14:46:00.009289+0000","last_active":"2024-04-23T14:46:01.106367+0000","last_peered":"2024-04-23T14:46:01.106367+0000","last_clean":"2024-04-23T14:46:01.106367+0000","last_became_active":"2024-04-23T14:46:00.008911+0000","last_became_peered":"2024-04-23T14:46:00.008911+0000","last_unstale":"2024-04-23T14:46:01.106367+0000","last_undegraded":"2024-04-23T14:46:01.106367+0000","last_fullsized":"2024-04-23T14:46:01.106367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:47:55.772442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.250849+0000","last_change":"2024-04-23T14:45:59.239494+0000","last_active":"2024-04-23T14:46:01.250849+0000","last_peered":"2024-04-23T14:46:01.250849+0000","last_clean":"2024-04-23T14:46:01.250849+0000","last_became_active":"2024-04-23T14:45:59.239157+0000","last_became_peered":"2024-04-23T14:45:59.239157+0000","last_unstale":"2024-04-23T14:46:01.250849+0000","last_undegraded":"2024-04-23T14:46:01.250849+0000","last_fullsized":"2024-04-23T14:46:01.250849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:38.275320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:03.544020+0000","last_change":"2024-04-23T14:46:00.029401+0000","last_active":"2024-04-23T14:46:03.544020+0000","last_peered":"2024-04-23T14:46:03.544020+0000","last_clean":"2024-04-23T14:46:03.544020+0000","last_became_active":"2024-04-23T14:46:00.029006+0000","last_became_peered":"2024-04-23T14:46:00.029006+0000","last_unstale":"2024-04-23T14:46:03.544020+0000","last_undegraded":"2024-04-23T14:46:03.544020+0000","last_fullsized":"2024-04-23T14:46:03.544020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:11.617083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106635+0000","last_change":"2024-04-23T14:45:59.247636+0000","last_active":"2024-04-23T14:46:01.106635+0000","last_peered":"2024-04-23T14:46:01.106635+0000","last_clean":"2024-04-23T14:46:01.106635+0000","last_became_active":"2024-04-23T14:45:59.247231+0000","last_became_peered":"2024-04-23T14:45:59.247231+0000","last_unstale":"2024-04-23T14:46:01.106635+0000","last_undegraded":"2024-04-23T14:46:01.106635+0000","last_fullsized":"2024-04-23T14:46:01.106635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:54.291463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106618+0000","last_change":"2024-04-23T14:46:00.010825+0000","last_active":"2024-04-23T14:46:01.106618+0000","last_peered":"2024-04-23T14:46:01.106618+0000","last_clean":"2024-04-23T14:46:01.106618+0000","last_became_active":"2024-04-23T14:46:00.010470+0000","last_became_peered":"2024-04-23T14:46:00.010470+0000","last_unstale":"2024-04-23T14:46:01.106618+0000","last_undegraded":"2024-04-23T14:46:01.106618+0000","last_fullsized":"2024-04-23T14:46:01.106618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:19.851825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106724+0000","last_change":"2024-04-23T14:46:00.011967+0000","last_active":"2024-04-23T14:46:01.106724+0000","last_peered":"2024-04-23T14:46:01.106724+0000","last_clean":"2024-04-23T14:46:01.106724+0000","last_became_active":"2024-04-23T14:46:00.011580+0000","last_became_peered":"2024-04-23T14:46:00.011580+0000","last_unstale":"2024-04-23T14:46:01.106724+0000","last_undegraded":"2024-04-23T14:46:01.106724+0000","last_fullsized":"2024-04-23T14:46:01.106724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:25:38.909994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995480+0000","last_change":"2024-04-23T14:46:00.015575+0000","last_active":"2024-04-23T14:46:00.995480+0000","last_peered":"2024-04-23T14:46:00.995480+0000","last_clean":"2024-04-23T14:46:00.995480+0000","last_became_active":"2024-04-23T14:46:00.015235+0000","last_became_peered":"2024-04-23T14:46:00.015235+0000","last_unstale":"2024-04-23T14:46:00.995480+0000","last_undegraded":"2024-04-23T14:46:00.995480+0000","last_fullsized":"2024-04-23T14:46:00.995480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:14:34.910747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995420+0000","last_change":"2024-04-23T14:46:00.029874+0000","last_active":"2024-04-23T14:46:00.995420+0000","last_peered":"2024-04-23T14:46:00.995420+0000","last_clean":"2024-04-23T14:46:00.995420+0000","last_became_active":"2024-04-23T14:46:00.029520+0000","last_became_peered":"2024-04-23T14:46:00.029520+0000","last_unstale":"2024-04-23T14:46:00.995420+0000","last_undegraded":"2024-04-23T14:46:00.995420+0000","last_fullsized":"2024-04-23T14:46:00.995420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:30.711566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.709214+0000","last_change":"2024-04-23T14:46:00.039545+0000","last_active":"2024-04-23T14:46:01.709214+0000","last_peered":"2024-04-23T14:46:01.709214+0000","last_clean":"2024-04-23T14:46:01.709214+0000","last_became_active":"2024-04-23T14:46:00.039193+0000","last_became_peered":"2024-04-23T14:46:00.039193+0000","last_unstale":"2024-04-23T14:46:01.709214+0000","last_undegraded":"2024-04-23T14:46:01.709214+0000","last_fullsized":"2024-04-23T14:46:01.709214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:18:13.528129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106425+0000","last_change":"2024-04-23T14:46:00.036816+0000","last_active":"2024-04-23T14:46:01.106425+0000","last_peered":"2024-04-23T14:46:01.106425+0000","last_clean":"2024-04-23T14:46:01.106425+0000","last_became_active":"2024-04-23T14:46:00.036372+0000","last_became_peered":"2024-04-23T14:46:00.036372+0000","last_unstale":"2024-04-23T14:46:01.106425+0000","last_undegraded":"2024-04-23T14:46:01.106425+0000","last_fullsized":"2024-04-23T14:46:01.106425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:56.179724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107439+0000","last_change":"2024-04-23T14:46:00.012058+0000","last_active":"2024-04-23T14:46:01.107439+0000","last_peered":"2024-04-23T14:46:01.107439+0000","last_clean":"2024-04-23T14:46:01.107439+0000","last_became_active":"2024-04-23T14:46:00.011595+0000","last_became_peered":"2024-04-23T14:46:00.011595+0000","last_unstale":"2024-04-23T14:46:01.107439+0000","last_undegraded":"2024-04-23T14:46:01.107439+0000","last_fullsized":"2024-04-23T14:46:01.107439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:28.341442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.995931+0000","last_change":"2024-04-23T14:46:00.011419+0000","last_active":"2024-04-23T14:46:00.995931+0000","last_peered":"2024-04-23T14:46:00.995931+0000","last_clean":"2024-04-23T14:46:00.995931+0000","last_became_active":"2024-04-23T14:46:00.011004+0000","last_became_peered":"2024-04-23T14:46:00.011004+0000","last_unstale":"2024-04-23T14:46:00.995931+0000","last_undegraded":"2024-04-23T14:46:00.995931+0000","last_fullsized":"2024-04-23T14:46:00.995931+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:46:57.451648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.999743+0000","last_change":"2024-04-23T14:46:00.023583+0000","last_active":"2024-04-23T14:46:00.999743+0000","last_peered":"2024-04-23T14:46:00.999743+0000","last_clean":"2024-04-23T14:46:00.999743+0000","last_became_active":"2024-04-23T14:46:00.023195+0000","last_became_peered":"2024-04-23T14:46:00.023195+0000","last_unstale":"2024-04-23T14:46:00.999743+0000","last_undegraded":"2024-04-23T14:46:00.999743+0000","last_fullsized":"2024-04-23T14:46:00.999743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:39.992959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106959+0000","last_change":"2024-04-23T14:46:00.014719+0000","last_active":"2024-04-23T14:46:01.106959+0000","last_peered":"2024-04-23T14:46:01.106959+0000","last_clean":"2024-04-23T14:46:01.106959+0000","last_became_active":"2024-04-23T14:46:00.014349+0000","last_became_peered":"2024-04-23T14:46:00.014349+0000","last_unstale":"2024-04-23T14:46:01.106959+0000","last_undegraded":"2024-04-23T14:46:01.106959+0000","last_fullsized":"2024-04-23T14:46:01.106959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:12.789316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:00.997537+0000","last_change":"2024-04-23T14:46:00.019162+0000","last_active":"2024-04-23T14:46:00.997537+0000","last_peered":"2024-04-23T14:46:00.997537+0000","last_clean":"2024-04-23T14:46:00.997537+0000","last_became_active":"2024-04-23T14:46:00.018814+0000","last_became_peered":"2024-04-23T14:46:00.018814+0000","last_unstale":"2024-04-23T14:46:00.997537+0000","last_undegraded":"2024-04-23T14:46:00.997537+0000","last_fullsized":"2024-04-23T14:46:00.997537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:10.741148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.105312+0000","last_change":"2024-04-23T14:46:00.007254+0000","last_active":"2024-04-23T14:46:01.105312+0000","last_peered":"2024-04-23T14:46:01.105312+0000","last_clean":"2024-04-23T14:46:01.105312+0000","last_became_active":"2024-04-23T14:46:00.006826+0000","last_became_peered":"2024-04-23T14:46:00.006826+0000","last_unstale":"2024-04-23T14:46:01.105312+0000","last_undegraded":"2024-04-23T14:46:01.105312+0000","last_fullsized":"2024-04-23T14:46:01.105312+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:29.155244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.106425+0000","last_change":"2024-04-23T14:46:00.039265+0000","last_active":"2024-04-23T14:46:01.106425+0000","last_peered":"2024-04-23T14:46:01.106425+0000","last_clean":"2024-04-23T14:46:01.106425+0000","last_became_active":"2024-04-23T14:46:00.038848+0000","last_became_peered":"2024-04-23T14:46:00.038848+0000","last_unstale":"2024-04-23T14:46:01.106425+0000","last_undegraded":"2024-04-23T14:46:01.106425+0000","last_fullsized":"2024-04-23T14:46:01.106425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:10.218655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T14:46:01.107259+0000","last_change":"2024-04-23T14:46:00.012134+0000","last_active":"2024-04-23T14:46:01.107259+0000","last_peered":"2024-04-23T14:46:01.107259+0000","last_clean":"2024-04-23T14:46:01.107259+0000","last_became_active":"2024-04-23T14:46:00.011796+0000","last_became_peered":"2024-04-23T14:46:00.011796+0000","last_unstale":"2024-04-23T14:46:01.107259+0000","last_undegraded":"2024-04-23T14:46:01.107259+0000","last_fullsized":"2024-04-23T14:46:01.107259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:41.671419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281220,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6004,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":133,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":43504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":444461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":1037437,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776154,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":444697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":444617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299431,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":1036938,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":443969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":452091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":445357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":446841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":444397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920823,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":444914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757297,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":1035723,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34125,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34125,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34125,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":9953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:46:09.683 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T14:46:09.683 INFO:tasks.cephadm:Setting up client nodes... 2024-04-23T14:46:09.683 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-23T14:46:09.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:09 smithi203 ceph-mon[32800]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:46:09.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:09 smithi203 ceph-mon[32800]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-23T14:46:09.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:09 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3255808076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:10.165 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:10.681 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:10 smithi083 ceph-mon[27867]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:46:10.681 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:10 smithi083 ceph-mon[27867]: pgmap v322: 97 pgs: 97 active+clean; 583 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.8 KiB/s wr, 17 op/s 2024-04-23T14:46:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:10 smithi203 ceph-mon[32800]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:46:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:10 smithi203 ceph-mon[32800]: pgmap v322: 97 pgs: 97 active+clean; 583 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.8 KiB/s wr, 17 op/s 2024-04-23T14:46:10.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:10 smithi173 ceph-mon[32443]: from='client.14613 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:46:10.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:10 smithi173 ceph-mon[32443]: pgmap v322: 97 pgs: 97 active+clean; 583 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.8 KiB/s wr, 17 op/s 2024-04-23T14:46:11.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:11 smithi083 ceph-mon[27867]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:46:11.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:11 smithi203 ceph-mon[32800]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:46:11.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:11 smithi173 ceph-mon[32443]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:46:12.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.571+0000 7fc7eba00700 1 -- 172.21.15.83:0/2219390272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e4076b20 msgr2=0x7fc7e4074f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:12.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.571+0000 7fc7eba00700 1 --2- 172.21.15.83:0/2219390272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e4076b20 0x7fc7e4074f80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80098f0 tx=0x7fc7d8009c00 comp rx=0 tx=0).stop 2024-04-23T14:46:12.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.571+0000 7fc7eba00700 1 -- 172.21.15.83:0/2219390272 shutdown_connections 2024-04-23T14:46:12.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.571+0000 7fc7eba00700 1 --2- 172.21.15.83:0/2219390272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e410aa40 0x7fc7e410ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:12.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.571+0000 7fc7eba00700 1 --2- 172.21.15.83:0/2219390272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e4076b20 0x7fc7e4074f80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:12.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.571+0000 7fc7eba00700 1 --2- 172.21.15.83:0/2219390272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4075550 0x7fc7e40759d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:12.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.571+0000 7fc7eba00700 1 -- 172.21.15.83:0/2219390272 >> 172.21.15.83:0/2219390272 conn(0x7fc7e40fe110 msgr2=0x7fc7e4100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:12.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.571+0000 7fc7eba00700 1 -- 172.21.15.83:0/2219390272 shutdown_connections 2024-04-23T14:46:12.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.571+0000 7fc7eba00700 1 -- 172.21.15.83:0/2219390272 wait complete. 2024-04-23T14:46:12.574 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:12 smithi083 ceph-mon[27867]: pgmap v323: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-04-23T14:46:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.572+0000 7fc7eba00700 1 Processor -- start 2024-04-23T14:46:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.572+0000 7fc7eba00700 1 -- start start 2024-04-23T14:46:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.573+0000 7fc7eba00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4075550 0x7fc7e419af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.573+0000 7fc7eba00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e4076b20 0x7fc7e419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.573+0000 7fc7eba00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e410aa40 0x7fc7e41a24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.573+0000 7fc7eba00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e4110950 con 0x7fc7e4075550 2024-04-23T14:46:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.573+0000 7fc7eba00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e41107d0 con 0x7fc7e410aa40 2024-04-23T14:46:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.573+0000 7fc7eba00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e4110ad0 con 0x7fc7e4076b20 2024-04-23T14:46:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.573+0000 7fc7e9000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4075550 0x7fc7e419af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.573+0000 7fc7e9000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4075550 0x7fc7e419af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60146/0 (socket says 172.21.15.83:60146) 2024-04-23T14:46:12.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.573+0000 7fc7e9000700 1 -- 172.21.15.83:0/466009546 learned_addr learned my addr 172.21.15.83:0/466009546 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.573+0000 7fc7e3e00700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e4076b20 0x7fc7e419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.573+0000 7fc7e9a00700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e410aa40 0x7fc7e41a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.574+0000 7fc7e9000700 1 -- 172.21.15.83:0/466009546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e4076b20 msgr2=0x7fc7e419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.574+0000 7fc7e9000700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e4076b20 0x7fc7e419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.574+0000 7fc7e9000700 1 -- 172.21.15.83:0/466009546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e410aa40 msgr2=0x7fc7e41a24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.574+0000 7fc7e9000700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e410aa40 0x7fc7e41a24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.574+0000 7fc7e9000700 1 -- 172.21.15.83:0/466009546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e41a2b60 con 0x7fc7e4075550 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.574+0000 7fc7e9000700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4075550 0x7fc7e419af00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009230 tx=0x7fc7d8005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.574+0000 7fc7e1600700 1 -- 172.21.15.83:0/466009546 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8025070 con 0x7fc7e4075550 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.574+0000 7fc7e1600700 1 -- 172.21.15.83:0/466009546 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d800f950 con 0x7fc7e4075550 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.574+0000 7fc7eba00700 1 -- 172.21.15.83:0/466009546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e41a2df0 con 0x7fc7e4075550 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.575+0000 7fc7e1600700 1 -- 172.21.15.83:0/466009546 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8003c50 con 0x7fc7e4075550 2024-04-23T14:46:12.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.575+0000 7fc7eba00700 1 -- 172.21.15.83:0/466009546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e41a3190 con 0x7fc7e4075550 2024-04-23T14:46:12.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.576+0000 7fc7eba00700 1 -- 172.21.15.83:0/466009546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e402dfe0 con 0x7fc7e4075550 2024-04-23T14:46:12.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.578+0000 7fc7e1600700 1 -- 172.21.15.83:0/466009546 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc7d8028070 con 0x7fc7e4075550 2024-04-23T14:46:12.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.579+0000 7fc7e1600700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fc7cc0a3d80 0x7fc7cc0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:12.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.580+0000 7fc7e1600700 1 -- 172.21.15.83:0/466009546 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc7d80d0d30 con 0x7fc7e4075550 2024-04-23T14:46:12.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.580+0000 7fc7e3e00700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fc7cc0a3d80 0x7fc7cc0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:12.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.580+0000 7fc7e3e00700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fc7cc0a3d80 0x7fc7cc0a6240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009750 tx=0x7fc7d0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:12.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.584+0000 7fc7e1600700 1 -- 172.21.15.83:0/466009546 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7d805cff0 con 0x7fc7e4075550 2024-04-23T14:46:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:12 smithi203 ceph-mon[32800]: pgmap v323: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-04-23T14:46:12.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:12 smithi173 ceph-mon[32443]: pgmap v323: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.7 KiB/s wr, 17 op/s 2024-04-23T14:46:12.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.898+0000 7fc7eba00700 1 -- 172.21.15.83:0/466009546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc7e402e1b0 con 0x7fc7e4075550 2024-04-23T14:46:12.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.902+0000 7fc7e1600700 1 -- 172.21.15.83:0/466009546 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc7d8013030 con 0x7fc7e4075550 2024-04-23T14:46:12.904 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-23T14:46:12.904 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQC0ySdm0TWwNRAAQcczasKd02bH5iisOdzPuw== 2024-04-23T14:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.905+0000 7fc7eba00700 1 -- 172.21.15.83:0/466009546 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fc7cc0a3d80 msgr2=0x7fc7cc0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.905+0000 7fc7eba00700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fc7cc0a3d80 0x7fc7cc0a6240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009750 tx=0x7fc7d0009310 comp rx=0 tx=0).stop 2024-04-23T14:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.905+0000 7fc7eba00700 1 -- 172.21.15.83:0/466009546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4075550 msgr2=0x7fc7e419af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.905+0000 7fc7eba00700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4075550 0x7fc7e419af00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009230 tx=0x7fc7d8005250 comp rx=0 tx=0).stop 2024-04-23T14:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.905+0000 7fc7eba00700 1 -- 172.21.15.83:0/466009546 shutdown_connections 2024-04-23T14:46:12.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.905+0000 7fc7eba00700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fc7cc0a3d80 0x7fc7cc0a6240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:12.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.905+0000 7fc7eba00700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e410aa40 0x7fc7e41a24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:12.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.905+0000 7fc7eba00700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e4075550 0x7fc7e419af00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:12.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.905+0000 7fc7eba00700 1 --2- 172.21.15.83:0/466009546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e4076b20 0x7fc7e419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:12.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.905+0000 7fc7eba00700 1 -- 172.21.15.83:0/466009546 >> 172.21.15.83:0/466009546 conn(0x7fc7e40fe110 msgr2=0x7fc7e4106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:12.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.906+0000 7fc7eba00700 1 -- 172.21.15.83:0/466009546 shutdown_connections 2024-04-23T14:46:12.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:12.906+0000 7fc7eba00700 1 -- 172.21.15.83:0/466009546 wait complete. 2024-04-23T14:46:13.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:13 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/466009546' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T14:46:13.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:13 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/466009546' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T14:46:13.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:13 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/466009546' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T14:46:13.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:13 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/466009546' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T14:46:13.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:13 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/466009546' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T14:46:13.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:13 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/466009546' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T14:46:13.869 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:46:13.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-23T14:46:13.870 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-23T14:46:14.000 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-23T14:46:14.000 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-23T14:46:14.000 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-23T14:46:14.484 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:14.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:14 smithi083 ceph-mon[27867]: pgmap v324: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-04-23T14:46:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:14 smithi203 ceph-mon[32800]: pgmap v324: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-04-23T14:46:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:14 smithi173 ceph-mon[32443]: pgmap v324: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-04-23T14:46:16.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:16 smithi083 ceph-mon[27867]: pgmap v325: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 16 op/s 2024-04-23T14:46:16.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.699+0000 7faa21e00700 1 -- 172.21.15.83:0/4195107480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c06a800 msgr2=0x7faa1c06ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.699+0000 7faa21e00700 1 --2- 172.21.15.83:0/4195107480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c06a800 0x7faa1c06ac60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faa04009990 tx=0x7faa04009ca0 comp rx=0 tx=0).stop 2024-04-23T14:46:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.699+0000 7faa21e00700 1 -- 172.21.15.83:0/4195107480 shutdown_connections 2024-04-23T14:46:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.699+0000 7faa21e00700 1 --2- 172.21.15.83:0/4195107480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa1c06b1a0 0x7faa1c10f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.699+0000 7faa21e00700 1 --2- 172.21.15.83:0/4195107480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa1c108b70 0x7faa1c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.699+0000 7faa21e00700 1 --2- 172.21.15.83:0/4195107480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c06a800 0x7faa1c06ac60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.699+0000 7faa21e00700 1 -- 172.21.15.83:0/4195107480 >> 172.21.15.83:0/4195107480 conn(0x7faa1c0789a0 msgr2=0x7faa1c078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.699+0000 7faa21e00700 1 -- 172.21.15.83:0/4195107480 shutdown_connections 2024-04-23T14:46:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.699+0000 7faa21e00700 1 -- 172.21.15.83:0/4195107480 wait complete. 2024-04-23T14:46:16.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.700+0000 7faa21e00700 1 Processor -- start 2024-04-23T14:46:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.701+0000 7faa21e00700 1 -- start start 2024-04-23T14:46:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.701+0000 7faa21e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c06a800 0x7faa1c1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.701+0000 7faa21e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa1c06b1a0 0x7faa1c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.701+0000 7faa21e00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa1c108b70 0x7faa1c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.701+0000 7faa21e00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c112e20 con 0x7faa1c06a800 2024-04-23T14:46:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.701+0000 7faa21e00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c112ca0 con 0x7faa1c06b1a0 2024-04-23T14:46:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.701+0000 7faa21e00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c112fa0 con 0x7faa1c108b70 2024-04-23T14:46:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.701+0000 7faa1aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c06a800 0x7faa1c1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.701+0000 7faa1aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c06a800 0x7faa1c1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35192/0 (socket says 172.21.15.83:35192) 2024-04-23T14:46:16.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.701+0000 7faa1aa00700 1 -- 172.21.15.83:0/2149480904 learned_addr learned my addr 172.21.15.83:0/2149480904 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.701+0000 7faa12000700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa1c06b1a0 0x7faa1c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.702+0000 7faa1b400700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa1c108b70 0x7faa1c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.702+0000 7faa1aa00700 1 -- 172.21.15.83:0/2149480904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa1c108b70 msgr2=0x7faa1c19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.702+0000 7faa1aa00700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa1c108b70 0x7faa1c19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.702+0000 7faa1aa00700 1 -- 172.21.15.83:0/2149480904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa1c06b1a0 msgr2=0x7faa1c199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.702+0000 7faa1aa00700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa1c06b1a0 0x7faa1c199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.702+0000 7faa1aa00700 1 -- 172.21.15.83:0/2149480904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1c19e2e0 con 0x7faa1c06a800 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.702+0000 7faa12000700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa1c06b1a0 0x7faa1c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.702+0000 7faa1aa00700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c06a800 0x7faa1c1992d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00e9a0 tx=0x7faa0c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.702+0000 7faa20a00700 1 -- 172.21.15.83:0/2149480904 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00cbd0 con 0x7faa1c06a800 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.703+0000 7faa20a00700 1 -- 172.21.15.83:0/2149480904 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa0c00cd70 con 0x7faa1c06a800 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.703+0000 7faa21e00700 1 -- 172.21.15.83:0/2149480904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1c19e5d0 con 0x7faa1c06a800 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.703+0000 7faa21e00700 1 -- 172.21.15.83:0/2149480904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa1c1019f0 con 0x7faa1c06a800 2024-04-23T14:46:16.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.703+0000 7faa20a00700 1 -- 172.21.15.83:0/2149480904 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c018750 con 0x7faa1c06a800 2024-04-23T14:46:16.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.704+0000 7faa21e00700 1 -- 172.21.15.83:0/2149480904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa1c0697e0 con 0x7faa1c06a800 2024-04-23T14:46:16.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.707+0000 7faa20a00700 1 -- 172.21.15.83:0/2149480904 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faa0c010430 con 0x7faa1c06a800 2024-04-23T14:46:16.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.709+0000 7faa20a00700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7faa000a3cd0 0x7faa000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:16.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.709+0000 7faa20a00700 1 -- 172.21.15.83:0/2149480904 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7faa0c0d0bd0 con 0x7faa1c06a800 2024-04-23T14:46:16.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.709+0000 7faa12000700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7faa000a3cd0 0x7faa000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:16.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.710+0000 7faa12000700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7faa000a3cd0 0x7faa000a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7faa04006010 tx=0x7faa0401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:16.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:16.711+0000 7faa20a00700 1 -- 172.21.15.83:0/2149480904 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa0c09d500 con 0x7faa1c06a800 2024-04-23T14:46:16.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:16 smithi203 ceph-mon[32800]: pgmap v325: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 16 op/s 2024-04-23T14:46:16.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:16 smithi173 ceph-mon[32443]: pgmap v325: 97 pgs: 97 active+clean; 623 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 16 op/s 2024-04-23T14:46:17.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.003+0000 7faa21e00700 1 -- 172.21.15.83:0/2149480904 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7faa1c19a630 con 0x7faa1c06a800 2024-04-23T14:46:17.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.018+0000 7faa20a00700 1 -- 172.21.15.83:0/2149480904 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7faa0c09ce00 con 0x7faa1c06a800 2024-04-23T14:46:17.025 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:46:17.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.025+0000 7faa21e00700 1 -- 172.21.15.83:0/2149480904 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7faa000a3cd0 msgr2=0x7faa000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:17.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.025+0000 7faa21e00700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7faa000a3cd0 0x7faa000a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7faa04006010 tx=0x7faa0401a040 comp rx=0 tx=0).stop 2024-04-23T14:46:17.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.025+0000 7faa21e00700 1 -- 172.21.15.83:0/2149480904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c06a800 msgr2=0x7faa1c1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:17.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.025+0000 7faa21e00700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c06a800 0x7faa1c1992d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00e9a0 tx=0x7faa0c00ecb0 comp rx=0 tx=0).stop 2024-04-23T14:46:17.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.025+0000 7faa21e00700 1 -- 172.21.15.83:0/2149480904 shutdown_connections 2024-04-23T14:46:17.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.025+0000 7faa21e00700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7faa000a3cd0 0x7faa000a6190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:17.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.025+0000 7faa21e00700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa1c108b70 0x7faa1c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:17.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.025+0000 7faa21e00700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa1c06a800 0x7faa1c1992d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:17.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.025+0000 7faa21e00700 1 --2- 172.21.15.83:0/2149480904 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa1c06b1a0 0x7faa1c199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:17.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.025+0000 7faa21e00700 1 -- 172.21.15.83:0/2149480904 >> 172.21.15.83:0/2149480904 conn(0x7faa1c0789a0 msgr2=0x7faa1c0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:17.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.026+0000 7faa21e00700 1 -- 172.21.15.83:0/2149480904 shutdown_connections 2024-04-23T14:46:17.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:17.026+0000 7faa21e00700 1 -- 172.21.15.83:0/2149480904 wait complete. 2024-04-23T14:46:17.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:17 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2149480904' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T14:46:17.745 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:17 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2149480904' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T14:46:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:17 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2149480904' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T14:46:17.873 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":1964818199},{"type":"v1","addr":"172.21.15.83:6801","nonce":1964818199}]},"active_addr":"172.21.15.83:6801/1964818199","active_change":"2024-04-23T14:37:57.065312+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.83:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":2255156114}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":4179157882}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":1081276647}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":622708991}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.83:0","nonce":3385644090}]}]} 2024-04-23T14:46:17.880 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-23T14:46:17.880 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-23T14:46:17.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T14:46:18.350 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:18.669 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:18 smithi083 ceph-mon[27867]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-23T14:46:18.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:18 smithi203 ceph-mon[32800]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-23T14:46:18.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:18 smithi173 ceph-mon[32443]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 7.5 KiB/s wr, 16 op/s 2024-04-23T14:46:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.589+0000 7f073a600700 1 -- 172.21.15.83:0/3657523007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0734105920 msgr2=0x7f0734109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.589+0000 7f073a600700 1 --2- 172.21.15.83:0/3657523007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0734105920 0x7f0734109970 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f07200099c0 tx=0x7f0720009cd0 comp rx=0 tx=0).stop 2024-04-23T14:46:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.589+0000 7f073a600700 1 -- 172.21.15.83:0/3657523007 shutdown_connections 2024-04-23T14:46:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.589+0000 7f073a600700 1 --2- 172.21.15.83:0/3657523007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f073410a050 0x7f0734111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.589+0000 7f073a600700 1 --2- 172.21.15.83:0/3657523007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0734104f70 0x7f0734105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:20.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.589+0000 7f073a600700 1 --2- 172.21.15.83:0/3657523007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0734105920 0x7f0734109970 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.589+0000 7f073a600700 1 -- 172.21.15.83:0/3657523007 >> 172.21.15.83:0/3657523007 conn(0x7f0734100800 msgr2=0x7f0734102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.589+0000 7f073a600700 1 -- 172.21.15.83:0/3657523007 shutdown_connections 2024-04-23T14:46:20.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.589+0000 7f073a600700 1 -- 172.21.15.83:0/3657523007 wait complete. 2024-04-23T14:46:20.592 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:20 smithi083 ceph-mon[27867]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-23T14:46:20.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.590+0000 7f073a600700 1 Processor -- start 2024-04-23T14:46:20.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.590+0000 7f073a600700 1 -- start start 2024-04-23T14:46:20.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.591+0000 7f073a600700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0734104f70 0x7f073419b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:20.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.591+0000 7f073a600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0734105920 0x7f073419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:20.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.591+0000 7f073a600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f073410a050 0x7f073419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:20.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.591+0000 7f073a600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0734115060 con 0x7f0734105920 2024-04-23T14:46:20.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.591+0000 7f073a600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0734114ee0 con 0x7f073410a050 2024-04-23T14:46:20.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.591+0000 7f073a600700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f07341151e0 con 0x7f0734104f70 2024-04-23T14:46:20.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.591+0000 7f0732a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0734105920 0x7f073419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.591+0000 7f0733400700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0734104f70 0x7f073419b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.591+0000 7f0733400700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0734104f70 0x7f073419b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:46052/0 (socket says 172.21.15.83:46052) 2024-04-23T14:46:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.591+0000 7f0733400700 1 -- 172.21.15.83:0/2899962897 learned_addr learned my addr 172.21.15.83:0/2899962897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.591+0000 7f0732a00700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0734105920 0x7f073419ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35210/0 (socket says 172.21.15.83:35210) 2024-04-23T14:46:20.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.592+0000 7f0733e00700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f073410a050 0x7f073419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:20.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.592+0000 7f0732a00700 1 -- 172.21.15.83:0/2899962897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0734104f70 msgr2=0x7f073419b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:20.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.592+0000 7f0732a00700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0734104f70 0x7f073419b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:20.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.592+0000 7f0732a00700 1 -- 172.21.15.83:0/2899962897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f073410a050 msgr2=0x7f073419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:20.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.592+0000 7f0732a00700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f073410a050 0x7f073419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:20.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.592+0000 7f0732a00700 1 -- 172.21.15.83:0/2899962897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07341a0500 con 0x7f0734105920 2024-04-23T14:46:20.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.592+0000 7f0733400700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0734104f70 0x7f073419b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:20.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.592+0000 7f0732a00700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0734105920 0x7f073419ba30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f07200098e0 tx=0x7f0720003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:20.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.593+0000 7f072be00700 1 -- 172.21.15.83:0/2899962897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720009e60 con 0x7f0734105920 2024-04-23T14:46:20.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.593+0000 7f072be00700 1 -- 172.21.15.83:0/2899962897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0720003680 con 0x7f0734105920 2024-04-23T14:46:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.593+0000 7f072be00700 1 -- 172.21.15.83:0/2899962897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072000fd60 con 0x7f0734105920 2024-04-23T14:46:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.593+0000 7f073a600700 1 -- 172.21.15.83:0/2899962897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07341a0790 con 0x7f0734105920 2024-04-23T14:46:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.593+0000 7f073a600700 1 -- 172.21.15.83:0/2899962897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07341a8070 con 0x7f0734105920 2024-04-23T14:46:20.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.594+0000 7f073a600700 1 -- 172.21.15.83:0/2899962897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07340697e0 con 0x7f0734105920 2024-04-23T14:46:20.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.596+0000 7f072be00700 1 -- 172.21.15.83:0/2899962897 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f072000f460 con 0x7f0734105920 2024-04-23T14:46:20.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.598+0000 7f072be00700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f07180a3cd0 0x7f07180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.598+0000 7f072be00700 1 -- 172.21.15.83:0/2899962897 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f07200d1360 con 0x7f0734105920 2024-04-23T14:46:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.598+0000 7f0733400700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f07180a3cd0 0x7f07180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:20.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.599+0000 7f0733400700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f07180a3cd0 0x7f07180a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0724011550 tx=0x7f0724009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:20.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.601+0000 7f072be00700 1 -- 172.21.15.83:0/2899962897 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0720013070 con 0x7f0734105920 2024-04-23T14:46:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:20 smithi203 ceph-mon[32800]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-23T14:46:20.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:20 smithi173 ceph-mon[32443]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-23T14:46:20.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.826+0000 7f073a600700 1 -- 172.21.15.83:0/2899962897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f073402dfe0 con 0x7f0734105920 2024-04-23T14:46:20.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.828+0000 7f072be00700 1 -- 172.21.15.83:0/2899962897 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22219 (secure 0 0 0) 0x7f072009dd20 con 0x7f0734105920 2024-04-23T14:46:20.830 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:46:20.830 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:46:00.988678+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:46:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.831+0000 7f073a600700 1 -- 172.21.15.83:0/2899962897 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f07180a3cd0 msgr2=0x7f07180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.831+0000 7f073a600700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f07180a3cd0 0x7f07180a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0724011550 tx=0x7f0724009260 comp rx=0 tx=0).stop 2024-04-23T14:46:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.831+0000 7f073a600700 1 -- 172.21.15.83:0/2899962897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0734105920 msgr2=0x7f073419ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.831+0000 7f073a600700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0734105920 0x7f073419ba30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f07200098e0 tx=0x7f0720003e00 comp rx=0 tx=0).stop 2024-04-23T14:46:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.831+0000 7f073a600700 1 -- 172.21.15.83:0/2899962897 shutdown_connections 2024-04-23T14:46:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.831+0000 7f073a600700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f07180a3cd0 0x7f07180a6190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.831+0000 7f073a600700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f073410a050 0x7f073419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:20.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.831+0000 7f073a600700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0734104f70 0x7f073419b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:20.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.831+0000 7f073a600700 1 --2- 172.21.15.83:0/2899962897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0734105920 0x7f073419ba30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:20.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.831+0000 7f073a600700 1 -- 172.21.15.83:0/2899962897 >> 172.21.15.83:0/2899962897 conn(0x7f0734100800 msgr2=0x7f0734102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:20.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.832+0000 7f073a600700 1 -- 172.21.15.83:0/2899962897 shutdown_connections 2024-04-23T14:46:20.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:20.832+0000 7f073a600700 1 -- 172.21.15.83:0/2899962897 wait complete. 2024-04-23T14:46:21.610 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:21 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2899962897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:21.687 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-23T14:46:21.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T14:46:21.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:21 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2899962897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:21.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:21 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2899962897' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:22.130 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:22.589 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:22 smithi083 ceph-mon[27867]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-04-23T14:46:22.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:22 smithi203 ceph-mon[32800]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-04-23T14:46:22.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:22 smithi173 ceph-mon[32443]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-04-23T14:46:23.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:23 smithi203 ceph-mon[32800]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-23T14:46:23.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:23 smithi083 ceph-mon[27867]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-23T14:46:23.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:23 smithi173 ceph-mon[32443]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-23T14:46:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.323+0000 7fb51f200700 1 -- 172.21.15.83:0/2584239301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5181099f0 msgr2=0x7fb518115e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.323+0000 7fb51f200700 1 --2- 172.21.15.83:0/2584239301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5181099f0 0x7fb518115e90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb51400b0a0 tx=0x7fb51400b3b0 comp rx=0 tx=0).stop 2024-04-23T14:46:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.323+0000 7fb51f200700 1 -- 172.21.15.83:0/2584239301 shutdown_connections 2024-04-23T14:46:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.323+0000 7fb51f200700 1 --2- 172.21.15.83:0/2584239301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5181099f0 0x7fb518115e90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.323+0000 7fb51f200700 1 --2- 172.21.15.83:0/2584239301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb51810f0e0 0x7fb51810f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:24.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.323+0000 7fb51f200700 1 --2- 172.21.15.83:0/2584239301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb518109030 0x7fb5181094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.323+0000 7fb51f200700 1 -- 172.21.15.83:0/2584239301 >> 172.21.15.83:0/2584239301 conn(0x7fb518078880 msgr2=0x7fb518107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.323+0000 7fb51f200700 1 -- 172.21.15.83:0/2584239301 shutdown_connections 2024-04-23T14:46:24.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.323+0000 7fb51f200700 1 -- 172.21.15.83:0/2584239301 wait complete. 2024-04-23T14:46:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.324+0000 7fb51f200700 1 Processor -- start 2024-04-23T14:46:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.324+0000 7fb51f200700 1 -- start start 2024-04-23T14:46:24.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.325+0000 7fb51f200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb518109030 0x7fb51819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.325+0000 7fb51f200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5181099f0 0x7fb51819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.325+0000 7fb51f200700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb51810f0e0 0x7fb5181a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.325+0000 7fb51f200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb518119370 con 0x7fb518109030 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.325+0000 7fb51f200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181191f0 con 0x7fb5181099f0 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.325+0000 7fb51f200700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181194f0 con 0x7fb51810f0e0 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.325+0000 7fb51de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb518109030 0x7fb51819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.325+0000 7fb51d400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5181099f0 0x7fb51819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.325+0000 7fb51de00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb518109030 0x7fb51819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35226/0 (socket says 172.21.15.83:35226) 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.325+0000 7fb51de00700 1 -- 172.21.15.83:0/2849909715 learned_addr learned my addr 172.21.15.83:0/2849909715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.325+0000 7fb51e800700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb51810f0e0 0x7fb5181a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.326+0000 7fb51d400700 1 -- 172.21.15.83:0/2849909715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb51810f0e0 msgr2=0x7fb5181a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.326+0000 7fb51d400700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb51810f0e0 0x7fb5181a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:24.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.326+0000 7fb51d400700 1 -- 172.21.15.83:0/2849909715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb518109030 msgr2=0x7fb51819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:24.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.326+0000 7fb51d400700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb518109030 0x7fb51819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:24.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.326+0000 7fb51d400700 1 -- 172.21.15.83:0/2849909715 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5181a46e0 con 0x7fb5181099f0 2024-04-23T14:46:24.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.326+0000 7fb51de00700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb518109030 0x7fb51819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:24.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.326+0000 7fb51d400700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5181099f0 0x7fb51819fc60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb50800e9f0 tx=0x7fb50800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.326+0000 7fb512a00700 1 -- 172.21.15.83:0/2849909715 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508004220 con 0x7fb5181099f0 2024-04-23T14:46:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.326+0000 7fb512a00700 1 -- 172.21.15.83:0/2849909715 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb508004d10 con 0x7fb5181099f0 2024-04-23T14:46:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.326+0000 7fb51f200700 1 -- 172.21.15.83:0/2849909715 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5181a49d0 con 0x7fb5181099f0 2024-04-23T14:46:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.327+0000 7fb512a00700 1 -- 172.21.15.83:0/2849909715 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508005020 con 0x7fb5181099f0 2024-04-23T14:46:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.327+0000 7fb51f200700 1 -- 172.21.15.83:0/2849909715 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5181ac250 con 0x7fb5181099f0 2024-04-23T14:46:24.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.328+0000 7fb51f200700 1 -- 172.21.15.83:0/2849909715 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5180697e0 con 0x7fb5181099f0 2024-04-23T14:46:24.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.330+0000 7fb512a00700 1 -- 172.21.15.83:0/2849909715 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb508004750 con 0x7fb5181099f0 2024-04-23T14:46:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.331+0000 7fb512a00700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5000a3cd0 0x7fb5000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:24.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.331+0000 7fb512a00700 1 -- 172.21.15.83:0/2849909715 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb5080d0580 con 0x7fb5181099f0 2024-04-23T14:46:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.331+0000 7fb51de00700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5000a3cd0 0x7fb5000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:24.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.332+0000 7fb51de00700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5000a3cd0 0x7fb5000a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0104e0 tx=0x7fb50c007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:24.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.334+0000 7fb512a00700 1 -- 172.21.15.83:0/2849909715 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb50809ceb0 con 0x7fb5181099f0 2024-04-23T14:46:24.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.558+0000 7fb51f200700 1 -- 172.21.15.83:0/2849909715 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5181a0a30 con 0x7fb5181099f0 2024-04-23T14:46:24.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.560+0000 7fb512a00700 1 -- 172.21.15.83:0/2849909715 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22219 (secure 0 0 0) 0x7fb50809c7b0 con 0x7fb5181099f0 2024-04-23T14:46:24.562 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:46:24.562 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":86,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:46:00.988678+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:46:24.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.563+0000 7fb51f200700 1 -- 172.21.15.83:0/2849909715 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5000a3cd0 msgr2=0x7fb5000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:24.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.563+0000 7fb51f200700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5000a3cd0 0x7fb5000a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0104e0 tx=0x7fb50c007480 comp rx=0 tx=0).stop 2024-04-23T14:46:24.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.563+0000 7fb51f200700 1 -- 172.21.15.83:0/2849909715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5181099f0 msgr2=0x7fb51819fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:24.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.563+0000 7fb51f200700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5181099f0 0x7fb51819fc60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb50800e9f0 tx=0x7fb50800ed00 comp rx=0 tx=0).stop 2024-04-23T14:46:24.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.564+0000 7fb51f200700 1 -- 172.21.15.83:0/2849909715 shutdown_connections 2024-04-23T14:46:24.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.564+0000 7fb51f200700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5000a3cd0 0x7fb5000a6190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:24.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.564+0000 7fb51f200700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb51810f0e0 0x7fb5181a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:24.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.564+0000 7fb51f200700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb518109030 0x7fb51819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:24.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.564+0000 7fb51f200700 1 --2- 172.21.15.83:0/2849909715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5181099f0 0x7fb51819fc60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:24.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.564+0000 7fb51f200700 1 -- 172.21.15.83:0/2849909715 >> 172.21.15.83:0/2849909715 conn(0x7fb518078880 msgr2=0x7fb518079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:24.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.564+0000 7fb51f200700 1 -- 172.21.15.83:0/2849909715 shutdown_connections 2024-04-23T14:46:24.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:24.564+0000 7fb51f200700 1 -- 172.21.15.83:0/2849909715 wait complete. 2024-04-23T14:46:24.941 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:24 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2849909715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:24.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:24 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2849909715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:25.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:24 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2849909715' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:46:25.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-23T14:46:25.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-23T14:46:25.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-23T14:46:25.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-23T14:46:25.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-23T14:46:25.343 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-23T14:46:25.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-23T14:46:25.344 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-23T14:46:25.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-23T14:46:25.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-23T14:46:25.345 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-23T14:46:25.346 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-23T14:46:25.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:25 smithi203 ceph-mon[32800]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-23T14:46:26.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:25 smithi083 ceph-mon[27867]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-23T14:46:26.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:25 smithi173 ceph-mon[32443]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-23T14:46:26.365 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:26.380 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:27.957 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:27.962 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:27.962 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:27.968 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:27.996 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:28.001 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:28.002 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:28.004 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:28.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:28 smithi083 ceph-mon[27867]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-23T14:46:28.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:28 smithi173 ceph-mon[32443]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-23T14:46:28.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:28 smithi203 ceph-mon[32800]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-23T14:46:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:30.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:30 smithi173 ceph-mon[32443]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-23T14:46:30.577 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:30 smithi083 ceph-mon[27867]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-23T14:46:30.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:30 smithi203 ceph-mon[32800]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-23T14:46:31.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.018+0000 7f8dee800700 1 -- 172.21.15.83:0/2560065852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de8108c50 msgr2=0x7f8de8109030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:31.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.018+0000 7f8dee800700 1 --2- 172.21.15.83:0/2560065852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de8108c50 0x7f8de8109030 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009990 tx=0x7f8dd8009ca0 comp rx=0 tx=0).stop 2024-04-23T14:46:31.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.018+0000 7f8dee800700 1 -- 172.21.15.83:0/2560065852 shutdown_connections 2024-04-23T14:46:31.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.018+0000 7f8dee800700 1 --2- 172.21.15.83:0/2560065852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de8103500 0x7f8de810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:31.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.018+0000 7f8dee800700 1 --2- 172.21.15.83:0/2560065852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de8108c50 0x7f8de8109030 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:31.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.018+0000 7f8dee800700 1 --2- 172.21.15.83:0/2560065852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8de8102b40 0x7f8de8102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:31.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.018+0000 7f8dee800700 1 -- 172.21.15.83:0/2560065852 >> 172.21.15.83:0/2560065852 conn(0x7f8de8079ca0 msgr2=0x7f8de807a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:31.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.018+0000 7f8dee800700 1 -- 172.21.15.83:0/2560065852 shutdown_connections 2024-04-23T14:46:31.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.018+0000 7f8dee800700 1 -- 172.21.15.83:0/2560065852 wait complete. 2024-04-23T14:46:31.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.019+0000 7f8dee800700 1 Processor -- start 2024-04-23T14:46:31.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.019+0000 7f8dee800700 1 -- start start 2024-04-23T14:46:31.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.020+0000 7f8dee800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de8102b40 0x7f8de8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.020+0000 7f8dee800700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de8103500 0x7f8de8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.020+0000 7f8dee800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8de8108c50 0x7f8de819db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.020+0000 7f8dee800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8de8112e90 con 0x7f8de8102b40 2024-04-23T14:46:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.020+0000 7f8dee800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8de8112d10 con 0x7f8de8108c50 2024-04-23T14:46:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.020+0000 7f8dee800700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8de8113010 con 0x7f8de8103500 2024-04-23T14:46:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.020+0000 7f8de7400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de8102b40 0x7f8de8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.020+0000 7f8ddfe00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de8103500 0x7f8de8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.020+0000 7f8ddfe00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de8103500 0x7f8de8199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:43242/0 (socket says 172.21.15.83:43242) 2024-04-23T14:46:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.020+0000 7f8ddfe00700 1 -- 172.21.15.83:0/347546881 learned_addr learned my addr 172.21.15.83:0/347546881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.020+0000 7f8de7e00700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8de8108c50 0x7f8de819db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.021+0000 7f8ddfe00700 1 -- 172.21.15.83:0/347546881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8de8108c50 msgr2=0x7f8de819db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:31.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.021+0000 7f8ddfe00700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8de8108c50 0x7f8de819db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.021+0000 7f8ddfe00700 1 -- 172.21.15.83:0/347546881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de8102b40 msgr2=0x7f8de8199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.021+0000 7f8ddfe00700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de8102b40 0x7f8de8199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.021+0000 7f8ddfe00700 1 -- 172.21.15.83:0/347546881 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de819e250 con 0x7f8de8103500 2024-04-23T14:46:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.021+0000 7f8de7400700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8de8102b40 0x7f8de8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:31.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.021+0000 7f8ddfe00700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de8103500 0x7f8de8199780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000e9a0 tx=0x7f8dd000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:31.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.021+0000 7f8de4c00700 1 -- 172.21.15.83:0/347546881 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd000ca10 con 0x7f8de8103500 2024-04-23T14:46:31.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.021+0000 7f8dee800700 1 -- 172.21.15.83:0/347546881 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de819e540 con 0x7f8de8103500 2024-04-23T14:46:31.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.022+0000 7f8de4c00700 1 -- 172.21.15.83:0/347546881 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dd000cbb0 con 0x7f8de8103500 2024-04-23T14:46:31.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.022+0000 7f8de4c00700 1 -- 172.21.15.83:0/347546881 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0018680 con 0x7f8de8103500 2024-04-23T14:46:31.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.022+0000 7f8dee800700 1 -- 172.21.15.83:0/347546881 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de81a5dc0 con 0x7f8de8103500 2024-04-23T14:46:31.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.022+0000 7f8dee800700 1 -- 172.21.15.83:0/347546881 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8de8063240 con 0x7f8de8103500 2024-04-23T14:46:31.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.025+0000 7f8de4c00700 1 -- 172.21.15.83:0/347546881 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8dd0010430 con 0x7f8de8103500 2024-04-23T14:46:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.026+0000 7f8de4c00700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8dcc0a3d50 0x7f8dcc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.026+0000 7f8de4c00700 1 -- 172.21.15.83:0/347546881 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8dd00d0ed0 con 0x7f8de8103500 2024-04-23T14:46:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.026+0000 7f8de7400700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8dcc0a3d50 0x7f8dcc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.027+0000 7f8de4c00700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379] conn(0x7f8dcc0ad530 0x7f8dcc0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.027+0000 7f8de4c00700 1 -- 172.21.15.83:0/347546881 --> [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8dd0027e50 con 0x7f8dcc0ad530 2024-04-23T14:46:31.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.027+0000 7f8de4c00700 1 -- 172.21.15.83:0/347546881 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f8dd00d1280 con 0x7f8de8103500 2024-04-23T14:46:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.027+0000 7f8de7e00700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379] conn(0x7f8dcc0ad530 0x7f8dcc0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.027+0000 7f8de7400700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8dcc0a3d50 0x7f8dcc0a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8000c00 tx=0x7f8dd800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:31.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.028+0000 7f8de7e00700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379] conn(0x7f8dcc0ad530 0x7f8dcc0af950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:31.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.029+0000 7f8de4c00700 1 -- 172.21.15.83:0/347546881 <== osd.2 v2:172.21.15.203:6800/4282658379 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8dd0027e50 con 0x7f8dcc0ad530 2024-04-23T14:46:31.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.046+0000 7f8dee800700 1 -- 172.21.15.83:0/347546881 --> [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8de8063670 con 0x7f8dcc0ad530 2024-04-23T14:46:31.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.047+0000 7f8de4c00700 1 -- 172.21.15.83:0/347546881 <== osd.2 v2:172.21.15.203:6800/4282658379 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8de8063670 con 0x7f8dcc0ad530 2024-04-23T14:46:31.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.048+0000 7f8dee800700 1 -- 172.21.15.83:0/347546881 >> [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379] conn(0x7f8dcc0ad530 msgr2=0x7f8dcc0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:31.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.048+0000 7f8dee800700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379] conn(0x7f8dcc0ad530 0x7f8dcc0af950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:31.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.048+0000 7f8dee800700 1 -- 172.21.15.83:0/347546881 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8dcc0a3d50 msgr2=0x7f8dcc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:31.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.048+0000 7f8dee800700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f8dcc0a3d50 0x7f8dcc0a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8000c00 tx=0x7f8dd800b260 comp rx=0 tx=0).stop 2024-04-23T14:46:31.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.048+0000 7f8dee800700 1 -- 172.21.15.83:0/347546881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de8103500 msgr2=0x7f8de8199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:31.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.048+0000 7f8dee800700 1 --2- 172.21.15.83:0/347546881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de8103500 0x7f8de8199780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000e9a0 tx=0x7f8dd000ecb0 comp rx=0 tx=0).stop 2024-04-23T14:46:31.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.048+0000 7f8de7e00700 1 -- 172.21.15.83:0/347546881 reap_dead start 2024-04-23T14:46:31.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.048+0000 7f8dee800700 1 -- 172.21.15.83:0/347546881 shutdown_connections 2024-04-23T14:46:31.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.048+0000 7f8dee800700 1 -- 172.21.15.83:0/347546881 >> 172.21.15.83:0/347546881 conn(0x7f8de8079ca0 msgr2=0x7f8de8101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:31.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.048+0000 7f8dee800700 1 -- 172.21.15.83:0/347546881 shutdown_connections 2024-04-23T14:46:31.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:31.048+0000 7f8dee800700 1 -- 172.21.15.83:0/347546881 wait complete. 2024-04-23T14:46:32.403 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:32 smithi083 ceph-mon[27867]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-23T14:46:32.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:32 smithi173 ceph-mon[32443]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-23T14:46:32.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:32 smithi203 ceph-mon[32800]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-23T14:46:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:34 smithi083 ceph-mon[27867]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-23T14:46:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:34 smithi173 ceph-mon[32443]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-23T14:46:34.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:34 smithi203 ceph-mon[32800]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-23T14:46:36.475 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:36 smithi083 ceph-mon[27867]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-23T14:46:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:36 smithi173 ceph-mon[32443]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-23T14:46:36.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:36 smithi203 ceph-mon[32800]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-23T14:46:38.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:38 smithi083 ceph-mon[27867]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-23T14:46:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:38 smithi173 ceph-mon[32443]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-23T14:46:38.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:38 smithi203 ceph-mon[32800]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-23T14:46:39.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.262+0000 7f31a6c00700 1 -- 172.21.15.83:0/2603886722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31a010a070 msgr2=0x7f31a0111c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:39.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.262+0000 7f31a6c00700 1 --2- 172.21.15.83:0/2603886722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31a010a070 0x7f31a0111c40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f319800b0d0 tx=0x7f319800b3e0 comp rx=0 tx=0).stop 2024-04-23T14:46:39.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.262+0000 7f31a6c00700 1 -- 172.21.15.83:0/2603886722 shutdown_connections 2024-04-23T14:46:39.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.262+0000 7f31a6c00700 1 --2- 172.21.15.83:0/2603886722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31a010a070 0x7f31a0111c40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:39.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.262+0000 7f31a6c00700 1 --2- 172.21.15.83:0/2603886722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31a0104f90 0x7f31a0105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:39.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.262+0000 7f31a6c00700 1 --2- 172.21.15.83:0/2603886722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31a0105940 0x7f31a0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:39.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.262+0000 7f31a6c00700 1 -- 172.21.15.83:0/2603886722 >> 172.21.15.83:0/2603886722 conn(0x7f31a0100820 msgr2=0x7f31a0102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:39.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.262+0000 7f31a6c00700 1 -- 172.21.15.83:0/2603886722 shutdown_connections 2024-04-23T14:46:39.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.262+0000 7f31a6c00700 1 -- 172.21.15.83:0/2603886722 wait complete. 2024-04-23T14:46:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.263+0000 7f31a6c00700 1 Processor -- start 2024-04-23T14:46:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.263+0000 7f31a6c00700 1 -- start start 2024-04-23T14:46:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.264+0000 7f31a6c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31a0104f90 0x7f31a019b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.264+0000 7f31a6c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31a0105940 0x7f31a019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.264+0000 7f31a6c00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31a010a070 0x7f31a019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.264+0000 7f31a6c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0115080 con 0x7f31a0104f90 2024-04-23T14:46:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.264+0000 7f31a6c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0114f00 con 0x7f31a0105940 2024-04-23T14:46:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.264+0000 7f31a6c00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0115200 con 0x7f31a010a070 2024-04-23T14:46:39.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.264+0000 7f319fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31a0104f90 0x7f31a019b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.264+0000 7f319fe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31a0104f90 0x7f31a019b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42284/0 (socket says 172.21.15.83:42284) 2024-04-23T14:46:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.264+0000 7f319fe00700 1 -- 172.21.15.83:0/1189124977 learned_addr learned my addr 172.21.15.83:0/1189124977 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.264+0000 7f31a4c00700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31a010a070 0x7f31a019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.264+0000 7f319f400700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31a0105940 0x7f31a019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.265+0000 7f319fe00700 1 -- 172.21.15.83:0/1189124977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31a010a070 msgr2=0x7f31a019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.265+0000 7f319fe00700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31a010a070 0x7f31a019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.265+0000 7f319fe00700 1 -- 172.21.15.83:0/1189124977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31a0105940 msgr2=0x7f31a019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.265+0000 7f319fe00700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31a0105940 0x7f31a019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.265+0000 7f319fe00700 1 -- 172.21.15.83:0/1189124977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a01a04b0 con 0x7f31a0104f90 2024-04-23T14:46:39.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.265+0000 7f31a4c00700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f31a010a070 0x7f31a019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.265+0000 7f319fe00700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31a0104f90 0x7f31a019b4f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f319000d980 tx=0x7f319000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.265+0000 7f319cc00700 1 -- 172.21.15.83:0/1189124977 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190016070 con 0x7f31a0104f90 2024-04-23T14:46:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.265+0000 7f319cc00700 1 -- 172.21.15.83:0/1189124977 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3190004d10 con 0x7f31a0104f90 2024-04-23T14:46:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.265+0000 7f319cc00700 1 -- 172.21.15.83:0/1189124977 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190005670 con 0x7f31a0104f90 2024-04-23T14:46:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.265+0000 7f31a6c00700 1 -- 172.21.15.83:0/1189124977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a01a07a0 con 0x7f31a0104f90 2024-04-23T14:46:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.265+0000 7f31a6c00700 1 -- 172.21.15.83:0/1189124977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a01a8020 con 0x7f31a0104f90 2024-04-23T14:46:39.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.266+0000 7f31a6c00700 1 -- 172.21.15.83:0/1189124977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31a0063280 con 0x7f31a0104f90 2024-04-23T14:46:39.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.269+0000 7f319cc00700 1 -- 172.21.15.83:0/1189124977 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3190004750 con 0x7f31a0104f90 2024-04-23T14:46:39.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.270+0000 7f319cc00700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f31840a3d50 0x7f31840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:39.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.270+0000 7f319cc00700 1 -- 172.21.15.83:0/1189124977 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f31900d08d0 con 0x7f31a0104f90 2024-04-23T14:46:39.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.270+0000 7f319f400700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f31840a3d50 0x7f31840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:39.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.270+0000 7f319cc00700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908] conn(0x7f31840ad530 0x7f31840af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:39.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.270+0000 7f319cc00700 1 -- 172.21.15.83:0/1189124977 --> [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3190015ec0 con 0x7f31840ad530 2024-04-23T14:46:39.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.270+0000 7f319cc00700 1 -- 172.21.15.83:0/1189124977 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f31900d0c80 con 0x7f31a0104f90 2024-04-23T14:46:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.271+0000 7f31a4c00700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908] conn(0x7f31840ad530 0x7f31840af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.271+0000 7f319f400700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f31840a3d50 0x7f31840a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f31a01099d0 tx=0x7f318c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:39.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.271+0000 7f31a4c00700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908] conn(0x7f31840ad530 0x7f31840af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:39.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.272+0000 7f319cc00700 1 -- 172.21.15.83:0/1189124977 <== osd.4 v2:172.21.15.173:6808/2156601908 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3190015ec0 con 0x7f31840ad530 2024-04-23T14:46:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.290+0000 7f31a6c00700 1 -- 172.21.15.83:0/1189124977 --> [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f31a00636b0 con 0x7f31840ad530 2024-04-23T14:46:39.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.291+0000 7f319cc00700 1 -- 172.21.15.83:0/1189124977 <== osd.4 v2:172.21.15.173:6808/2156601908 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f31a00636b0 con 0x7f31840ad530 2024-04-23T14:46:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.291+0000 7f31a6c00700 1 -- 172.21.15.83:0/1189124977 >> [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908] conn(0x7f31840ad530 msgr2=0x7f31840af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.291+0000 7f31a6c00700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908] conn(0x7f31840ad530 0x7f31840af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.291+0000 7f31a6c00700 1 -- 172.21.15.83:0/1189124977 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f31840a3d50 msgr2=0x7f31840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.291+0000 7f31a6c00700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f31840a3d50 0x7f31840a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f31a01099d0 tx=0x7f318c00b3c0 comp rx=0 tx=0).stop 2024-04-23T14:46:39.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.291+0000 7f31a6c00700 1 -- 172.21.15.83:0/1189124977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31a0104f90 msgr2=0x7f31a019b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:39.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.291+0000 7f31a6c00700 1 --2- 172.21.15.83:0/1189124977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31a0104f90 0x7f31a019b4f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f319000d980 tx=0x7f319000dc90 comp rx=0 tx=0).stop 2024-04-23T14:46:39.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.292+0000 7f31a4c00700 1 -- 172.21.15.83:0/1189124977 reap_dead start 2024-04-23T14:46:39.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.292+0000 7f31a6c00700 1 -- 172.21.15.83:0/1189124977 shutdown_connections 2024-04-23T14:46:39.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.292+0000 7f31a6c00700 1 -- 172.21.15.83:0/1189124977 >> 172.21.15.83:0/1189124977 conn(0x7f31a0100820 msgr2=0x7f31a0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:39.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.292+0000 7f31a6c00700 1 -- 172.21.15.83:0/1189124977 shutdown_connections 2024-04-23T14:46:39.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:39.292+0000 7f31a6c00700 1 -- 172.21.15.83:0/1189124977 wait complete. 2024-04-23T14:46:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:40 smithi173 ceph-mon[32443]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:46:40.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:40 smithi203 ceph-mon[32800]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:46:40.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:40 smithi083 ceph-mon[27867]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:46:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.683+0000 7f3997600700 1 -- 172.21.15.83:0/3166461833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3990108b70 msgr2=0x7f3990108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.683+0000 7f3997600700 1 --2- 172.21.15.83:0/3166461833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3990108b70 0x7f3990108f50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3984009990 tx=0x7f3984009ca0 comp rx=0 tx=0).stop 2024-04-23T14:46:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.684+0000 7f3997600700 1 -- 172.21.15.83:0/3166461833 shutdown_connections 2024-04-23T14:46:41.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.684+0000 7f3997600700 1 --2- 172.21.15.83:0/3166461833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f399006b1a0 0x7f399010f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.684+0000 7f3997600700 1 --2- 172.21.15.83:0/3166461833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3990108b70 0x7f3990108f50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.684+0000 7f3997600700 1 --2- 172.21.15.83:0/3166461833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f399006a800 0x7f399006ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.684+0000 7f3997600700 1 -- 172.21.15.83:0/3166461833 >> 172.21.15.83:0/3166461833 conn(0x7f39900789a0 msgr2=0x7f3990078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.684+0000 7f3997600700 1 -- 172.21.15.83:0/3166461833 shutdown_connections 2024-04-23T14:46:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.684+0000 7f3997600700 1 -- 172.21.15.83:0/3166461833 wait complete. 2024-04-23T14:46:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.685+0000 7f3997600700 1 Processor -- start 2024-04-23T14:46:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.685+0000 7f3997600700 1 -- start start 2024-04-23T14:46:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.685+0000 7f3997600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399006a800 0x7f3990101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.685+0000 7f3997600700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f399006b1a0 0x7f3990102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f3997600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3990108b70 0x7f3990102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f3997600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3990112e00 con 0x7f399006a800 2024-04-23T14:46:41.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f3997600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3990112c80 con 0x7f3990108b70 2024-04-23T14:46:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f3997600700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3990112f80 con 0x7f399006b1a0 2024-04-23T14:46:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f3994c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399006a800 0x7f3990101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f398fe00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f399006b1a0 0x7f3990102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f3994c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399006a800 0x7f3990101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42306/0 (socket says 172.21.15.83:42306) 2024-04-23T14:46:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f3994c00700 1 -- 172.21.15.83:0/597094767 learned_addr learned my addr 172.21.15.83:0/597094767 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f3995600700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3990108b70 0x7f3990102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f398fe00700 1 -- 172.21.15.83:0/597094767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3990108b70 msgr2=0x7f3990102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f398fe00700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3990108b70 0x7f3990102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f398fe00700 1 -- 172.21.15.83:0/597094767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399006a800 msgr2=0x7f3990101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f398fe00700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f399006a800 0x7f3990101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.686+0000 7f398fe00700 1 -- 172.21.15.83:0/597094767 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39901a77e0 con 0x7f399006b1a0 2024-04-23T14:46:41.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.687+0000 7f398fe00700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f399006b1a0 0x7f3990102000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f397c00e9f0 tx=0x7f397c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.687+0000 7f398d600700 1 -- 172.21.15.83:0/597094767 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c00cbd0 con 0x7f399006b1a0 2024-04-23T14:46:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.687+0000 7f398d600700 1 -- 172.21.15.83:0/597094767 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f397c00cd70 con 0x7f399006b1a0 2024-04-23T14:46:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.687+0000 7f3997600700 1 -- 172.21.15.83:0/597094767 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39901a7ad0 con 0x7f399006b1a0 2024-04-23T14:46:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.687+0000 7f398d600700 1 -- 172.21.15.83:0/597094767 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397c018710 con 0x7f399006b1a0 2024-04-23T14:46:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.687+0000 7f3997600700 1 -- 172.21.15.83:0/597094767 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39901a8010 con 0x7f399006b1a0 2024-04-23T14:46:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.688+0000 7f3997600700 1 -- 172.21.15.83:0/597094767 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f39900631c0 con 0x7f399006b1a0 2024-04-23T14:46:41.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.690+0000 7f398d600700 1 -- 172.21.15.83:0/597094767 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f397c010430 con 0x7f399006b1a0 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.691+0000 7f398d600700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f39780a3cd0 0x7f39780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.691+0000 7f398d600700 1 -- 172.21.15.83:0/597094767 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f397c0d0eb0 con 0x7f399006b1a0 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.692+0000 7f3994c00700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f39780a3cd0 0x7f39780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.692+0000 7f398d600700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636] conn(0x7f39780ad4b0 0x7f39780af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.692+0000 7f398d600700 1 -- 172.21.15.83:0/597094767 --> [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f397c001d30 con 0x7f39780ad4b0 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.692+0000 7f398d600700 1 -- 172.21.15.83:0/597094767 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f397c0d1260 con 0x7f399006b1a0 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.692+0000 7f3995600700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636] conn(0x7f39780ad4b0 0x7f39780af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.692+0000 7f3994c00700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f39780a3cd0 0x7f39780a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3984009750 tx=0x7f398400b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.693+0000 7f3995600700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636] conn(0x7f39780ad4b0 0x7f39780af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.694+0000 7f398d600700 1 -- 172.21.15.83:0/597094767 <== osd.5 v2:172.21.15.203:6808/581398636 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f397c001d30 con 0x7f39780ad4b0 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.711+0000 7f3997600700 1 -- 172.21.15.83:0/597094767 --> [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3990063640 con 0x7f39780ad4b0 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.712+0000 7f398d600700 1 -- 172.21.15.83:0/597094767 <== osd.5 v2:172.21.15.203:6808/581398636 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3990063640 con 0x7f39780ad4b0 2024-04-23T14:46:41.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.713+0000 7f3997600700 1 -- 172.21.15.83:0/597094767 >> [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636] conn(0x7f39780ad4b0 msgr2=0x7f39780af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.713+0000 7f3997600700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636] conn(0x7f39780ad4b0 0x7f39780af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.713+0000 7f3997600700 1 -- 172.21.15.83:0/597094767 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f39780a3cd0 msgr2=0x7f39780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.713+0000 7f3997600700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f39780a3cd0 0x7f39780a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3984009750 tx=0x7f398400b280 comp rx=0 tx=0).stop 2024-04-23T14:46:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.713+0000 7f3997600700 1 -- 172.21.15.83:0/597094767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f399006b1a0 msgr2=0x7f3990102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.713+0000 7f3997600700 1 --2- 172.21.15.83:0/597094767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f399006b1a0 0x7f3990102000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f397c00e9f0 tx=0x7f397c00ed00 comp rx=0 tx=0).stop 2024-04-23T14:46:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.713+0000 7f3995600700 1 -- 172.21.15.83:0/597094767 reap_dead start 2024-04-23T14:46:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.713+0000 7f3997600700 1 -- 172.21.15.83:0/597094767 shutdown_connections 2024-04-23T14:46:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.713+0000 7f3997600700 1 -- 172.21.15.83:0/597094767 >> 172.21.15.83:0/597094767 conn(0x7f39900789a0 msgr2=0x7f39900ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.713+0000 7f3997600700 1 -- 172.21.15.83:0/597094767 shutdown_connections 2024-04-23T14:46:41.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:41.713+0000 7f3997600700 1 -- 172.21.15.83:0/597094767 wait complete. 2024-04-23T14:46:42.401 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:42 smithi083 ceph-mon[27867]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:46:42.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:42 smithi173 ceph-mon[32443]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:46:42.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:42 smithi203 ceph-mon[32800]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:46:42.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.472+0000 7fa137200700 1 -- 172.21.15.83:0/1257688435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa130074d20 msgr2=0x7fa130075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:42.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.472+0000 7fa137200700 1 --2- 172.21.15.83:0/1257688435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa130074d20 0x7fa130075100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa1200098f0 tx=0x7fa120009c00 comp rx=0 tx=0).stop 2024-04-23T14:46:42.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.472+0000 7fa137200700 1 -- 172.21.15.83:0/1257688435 shutdown_connections 2024-04-23T14:46:42.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.472+0000 7fa137200700 1 --2- 172.21.15.83:0/1257688435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa13010f4f0 0x7fa1301118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:42.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.472+0000 7fa137200700 1 --2- 172.21.15.83:0/1257688435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa130074d20 0x7fa130075100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:42.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.472+0000 7fa137200700 1 --2- 172.21.15.83:0/1257688435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa130075640 0x7fa13010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.472+0000 7fa137200700 1 -- 172.21.15.83:0/1257688435 >> 172.21.15.83:0/1257688435 conn(0x7fa130079c50 msgr2=0x7fa13007a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.472+0000 7fa137200700 1 -- 172.21.15.83:0/1257688435 shutdown_connections 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.472+0000 7fa137200700 1 -- 172.21.15.83:0/1257688435 wait complete. 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.473+0000 7fa137200700 1 Processor -- start 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.473+0000 7fa137200700 1 -- start start 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.473+0000 7fa137200700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa130074d20 0x7fa13019b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.473+0000 7fa137200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa130075640 0x7fa13019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa137200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa13010f4f0 0x7fa13019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa137200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa130115040 con 0x7fa130075640 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa137200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa130114ec0 con 0x7fa13010f4f0 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa137200700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1301151c0 con 0x7fa130074d20 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa12f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa130075640 0x7fa13019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa12f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa130075640 0x7fa13019ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42326/0 (socket says 172.21.15.83:42326) 2024-04-23T14:46:42.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa12f400700 1 -- 172.21.15.83:0/3994115263 learned_addr learned my addr 172.21.15.83:0/3994115263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa12fe00700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa130074d20 0x7fa13019b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa12f400700 1 -- 172.21.15.83:0/3994115263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa130074d20 msgr2=0x7fa13019b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa12f400700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa130074d20 0x7fa13019b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa135200700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa13010f4f0 0x7fa13019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa12f400700 1 -- 172.21.15.83:0/3994115263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa13010f4f0 msgr2=0x7fa13019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa12f400700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa13010f4f0 0x7fa13019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa12f400700 1 -- 172.21.15.83:0/3994115263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1301a0500 con 0x7fa130075640 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.474+0000 7fa135200700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa13010f4f0 0x7fa13019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.475+0000 7fa12f400700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa130075640 0x7fa13019ba30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00ebf0 tx=0x7fa11c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.475+0000 7fa12cc00700 1 -- 172.21.15.83:0/3994115263 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00cd40 con 0x7fa130075640 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.475+0000 7fa12cc00700 1 -- 172.21.15.83:0/3994115263 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa11c010910 con 0x7fa130075640 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.475+0000 7fa137200700 1 -- 172.21.15.83:0/3994115263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1301a07f0 con 0x7fa130075640 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.475+0000 7fa12cc00700 1 -- 172.21.15.83:0/3994115263 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0189c0 con 0x7fa130075640 2024-04-23T14:46:42.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.475+0000 7fa137200700 1 -- 172.21.15.83:0/3994115263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1301a8070 con 0x7fa130075640 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.476+0000 7fa126000700 1 -- 172.21.15.83:0/3994115263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa130063280 con 0x7fa130075640 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.479+0000 7fa12cc00700 1 -- 172.21.15.83:0/3994115263 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa11c01f020 con 0x7fa130075640 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.480+0000 7fa12cc00700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa1140a3cd0 0x7fa1140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.480+0000 7fa12cc00700 1 -- 172.21.15.83:0/3994115263 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa11c0d1040 con 0x7fa130075640 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.480+0000 7fa12fe00700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa1140a3cd0 0x7fa1140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.480+0000 7fa12cc00700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524] conn(0x7fa1140ad4b0 0x7fa1140af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.480+0000 7fa12cc00700 1 -- 172.21.15.83:0/3994115263 --> [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa11c005750 con 0x7fa1140ad4b0 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.480+0000 7fa12cc00700 1 -- 172.21.15.83:0/3994115263 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fa11c0d1520 con 0x7fa130075640 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.481+0000 7fa135200700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524] conn(0x7fa1140ad4b0 0x7fa1140af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.481+0000 7fa12fe00700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa1140a3cd0 0x7fa1140a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa120009810 tx=0x7fa120019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.482+0000 7fa135200700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524] conn(0x7fa1140ad4b0 0x7fa1140af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.483+0000 7fa12cc00700 1 -- 172.21.15.83:0/3994115263 <== osd.8 v2:172.21.15.203:6816/1549180524 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa11c005750 con 0x7fa1140ad4b0 2024-04-23T14:46:42.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.500+0000 7fa126000700 1 -- 172.21.15.83:0/3994115263 --> [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa130063700 con 0x7fa1140ad4b0 2024-04-23T14:46:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.501+0000 7fa12cc00700 1 -- 172.21.15.83:0/3994115263 <== osd.8 v2:172.21.15.203:6816/1549180524 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa130063700 con 0x7fa1140ad4b0 2024-04-23T14:46:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.502+0000 7fa126000700 1 -- 172.21.15.83:0/3994115263 >> [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524] conn(0x7fa1140ad4b0 msgr2=0x7fa1140af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.502+0000 7fa126000700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524] conn(0x7fa1140ad4b0 0x7fa1140af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.502+0000 7fa126000700 1 -- 172.21.15.83:0/3994115263 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa1140a3cd0 msgr2=0x7fa1140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.502+0000 7fa126000700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fa1140a3cd0 0x7fa1140a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa120009810 tx=0x7fa120019040 comp rx=0 tx=0).stop 2024-04-23T14:46:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.502+0000 7fa126000700 1 -- 172.21.15.83:0/3994115263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa130075640 msgr2=0x7fa13019ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:42.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.502+0000 7fa126000700 1 --2- 172.21.15.83:0/3994115263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa130075640 0x7fa13019ba30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00ebf0 tx=0x7fa11c00ef00 comp rx=0 tx=0).stop 2024-04-23T14:46:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.502+0000 7fa135200700 1 -- 172.21.15.83:0/3994115263 reap_dead start 2024-04-23T14:46:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.502+0000 7fa126000700 1 -- 172.21.15.83:0/3994115263 shutdown_connections 2024-04-23T14:46:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.502+0000 7fa126000700 1 -- 172.21.15.83:0/3994115263 >> 172.21.15.83:0/3994115263 conn(0x7fa130079c50 msgr2=0x7fa1300ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.502+0000 7fa126000700 1 -- 172.21.15.83:0/3994115263 shutdown_connections 2024-04-23T14:46:42.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.502+0000 7fa126000700 1 -- 172.21.15.83:0/3994115263 wait complete. 2024-04-23T14:46:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.566+0000 7fb7f4800700 1 -- 172.21.15.83:0/1001881224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec075640 msgr2=0x7fb7ec10efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.566+0000 7fb7f4800700 1 --2- 172.21.15.83:0/1001881224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec075640 0x7fb7ec10efb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009990 tx=0x7fb7dc009ca0 comp rx=0 tx=0).stop 2024-04-23T14:46:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.567+0000 7fb7f4800700 1 -- 172.21.15.83:0/1001881224 shutdown_connections 2024-04-23T14:46:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.567+0000 7fb7f4800700 1 --2- 172.21.15.83:0/1001881224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7ec10f4f0 0x7fb7ec1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.567+0000 7fb7f4800700 1 --2- 172.21.15.83:0/1001881224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7ec074d20 0x7fb7ec075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.567+0000 7fb7f4800700 1 --2- 172.21.15.83:0/1001881224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec075640 0x7fb7ec10efb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.567+0000 7fb7f4800700 1 -- 172.21.15.83:0/1001881224 >> 172.21.15.83:0/1001881224 conn(0x7fb7ec079c50 msgr2=0x7fb7ec07a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.567+0000 7fb7f4800700 1 -- 172.21.15.83:0/1001881224 shutdown_connections 2024-04-23T14:46:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.567+0000 7fb7f4800700 1 -- 172.21.15.83:0/1001881224 wait complete. 2024-04-23T14:46:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.568+0000 7fb7f4800700 1 Processor -- start 2024-04-23T14:46:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.568+0000 7fb7f4800700 1 -- start start 2024-04-23T14:46:43.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.568+0000 7fb7f4800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7ec074d20 0x7fb7ec19b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.568+0000 7fb7f4800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec075640 0x7fb7ec19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.568+0000 7fb7f4800700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7ec10f4f0 0x7fb7ec19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.568+0000 7fb7f4800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec115060 con 0x7fb7ec075640 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.568+0000 7fb7f4800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec114ee0 con 0x7fb7ec074d20 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.568+0000 7fb7f4800700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1151e0 con 0x7fb7ec10f4f0 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.569+0000 7fb7f1400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec075640 0x7fb7ec19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.569+0000 7fb7f1400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec075640 0x7fb7ec19ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42346/0 (socket says 172.21.15.83:42346) 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.569+0000 7fb7f1400700 1 -- 172.21.15.83:0/3329595264 learned_addr learned my addr 172.21.15.83:0/3329595264 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.569+0000 7fb7f1e00700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7ec074d20 0x7fb7ec19b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.569+0000 7fb7f2800700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7ec10f4f0 0x7fb7ec19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.569+0000 7fb7f1400700 1 -- 172.21.15.83:0/3329595264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7ec10f4f0 msgr2=0x7fb7ec19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.569+0000 7fb7f1400700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7ec10f4f0 0x7fb7ec19fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:43.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.569+0000 7fb7f1400700 1 -- 172.21.15.83:0/3329595264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7ec074d20 msgr2=0x7fb7ec19b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.569+0000 7fb7f1400700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7ec074d20 0x7fb7ec19b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.569+0000 7fb7f1400700 1 -- 172.21.15.83:0/3329595264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec1a0540 con 0x7fb7ec075640 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.569+0000 7fb7f1400700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec075640 0x7fb7ec19ba70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc000c00 tx=0x7fb7dc00ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.570+0000 7fb7e6a00700 1 -- 172.21.15.83:0/3329595264 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc027070 con 0x7fb7ec075640 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.570+0000 7fb7e6a00700 1 -- 172.21.15.83:0/3329595264 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7dc003d70 con 0x7fb7ec075640 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.570+0000 7fb7e6a00700 1 -- 172.21.15.83:0/3329595264 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00f460 con 0x7fb7ec075640 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.570+0000 7fb7f4800700 1 -- 172.21.15.83:0/3329595264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ec1a07d0 con 0x7fb7ec075640 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.570+0000 7fb7f4800700 1 -- 172.21.15.83:0/3329595264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ec1a80b0 con 0x7fb7ec075640 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.571+0000 7fb7dbe00700 1 -- 172.21.15.83:0/3329595264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7ec063280 con 0x7fb7ec075640 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.573+0000 7fb7e6a00700 1 -- 172.21.15.83:0/3329595264 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb7dc02a070 con 0x7fb7ec075640 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.574+0000 7fb7e6a00700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb7d40a3d50 0x7fb7d40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.574+0000 7fb7e6a00700 1 -- 172.21.15.83:0/3329595264 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb7dc0d1e60 con 0x7fb7ec075640 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.575+0000 7fb7f1e00700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb7d40a3d50 0x7fb7d40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.575+0000 7fb7e6a00700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631] conn(0x7fb7d40ad530 0x7fb7d40af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:43.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.575+0000 7fb7e6a00700 1 -- 172.21.15.83:0/3329595264 --> [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7d40b0000 con 0x7fb7d40ad530 2024-04-23T14:46:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.575+0000 7fb7e6a00700 1 -- 172.21.15.83:0/3329595264 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fb7dc0d2210 con 0x7fb7ec075640 2024-04-23T14:46:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.575+0000 7fb7f2800700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631] conn(0x7fb7d40ad530 0x7fb7d40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.575+0000 7fb7f1e00700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb7d40a3d50 0x7fb7d40a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009c60 tx=0x7fb7e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.576+0000 7fb7f2800700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631] conn(0x7fb7d40ad530 0x7fb7d40af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.577+0000 7fb7e6a00700 1 -- 172.21.15.83:0/3329595264 <== osd.6 v2:172.21.15.83:6818/783931631 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7d40b0000 con 0x7fb7d40ad530 2024-04-23T14:46:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.595+0000 7fb7dbe00700 1 -- 172.21.15.83:0/3329595264 --> [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7ec063680 con 0x7fb7d40ad530 2024-04-23T14:46:43.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.595+0000 7fb7e6a00700 1 -- 172.21.15.83:0/3329595264 <== osd.6 v2:172.21.15.83:6818/783931631 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb7ec063680 con 0x7fb7d40ad530 2024-04-23T14:46:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.596+0000 7fb7dbe00700 1 -- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631] conn(0x7fb7d40ad530 msgr2=0x7fb7d40af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.596+0000 7fb7dbe00700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631] conn(0x7fb7d40ad530 0x7fb7d40af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.596+0000 7fb7dbe00700 1 -- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb7d40a3d50 msgr2=0x7fb7d40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.596+0000 7fb7dbe00700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb7d40a3d50 0x7fb7d40a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009c60 tx=0x7fb7e0009260 comp rx=0 tx=0).stop 2024-04-23T14:46:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.596+0000 7fb7dbe00700 1 -- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec075640 msgr2=0x7fb7ec19ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.596+0000 7fb7dbe00700 1 --2- 172.21.15.83:0/3329595264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb7ec075640 0x7fb7ec19ba70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc000c00 tx=0x7fb7dc00ba90 comp rx=0 tx=0).stop 2024-04-23T14:46:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.596+0000 7fb7f2800700 1 -- 172.21.15.83:0/3329595264 reap_dead start 2024-04-23T14:46:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.596+0000 7fb7dbe00700 1 -- 172.21.15.83:0/3329595264 shutdown_connections 2024-04-23T14:46:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.596+0000 7fb7dbe00700 1 -- 172.21.15.83:0/3329595264 >> 172.21.15.83:0/3329595264 conn(0x7fb7ec079c50 msgr2=0x7fb7ec0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.596+0000 7fb7dbe00700 1 -- 172.21.15.83:0/3329595264 shutdown_connections 2024-04-23T14:46:43.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:42.596+0000 7fb7dbe00700 1 -- 172.21.15.83:0/3329595264 wait complete. 2024-04-23T14:46:43.825 INFO:teuthology.orchestra.run.smithi083.stdout:77309411402 2024-04-23T14:46:43.826 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-23T14:46:43.956 INFO:teuthology.orchestra.run.smithi083.stdout:141733920830 2024-04-23T14:46:43.957 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-23T14:46:44.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:44 smithi173 ceph-mon[32443]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-23T14:46:44.576 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:44 smithi083 ceph-mon[27867]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-23T14:46:44.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:44 smithi203 ceph-mon[32800]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-23T14:46:44.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.774+0000 7fcf28c00700 1 -- 172.21.15.83:0/1981525998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf241033e0 msgr2=0x7fcf2410f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:44.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.774+0000 7fcf28c00700 1 --2- 172.21.15.83:0/1981525998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf241033e0 0x7fcf2410f8f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00b0a0 tx=0x7fcf1c00b3b0 comp rx=0 tx=0).stop 2024-04-23T14:46:44.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.774+0000 7fcf28c00700 1 -- 172.21.15.83:0/1981525998 shutdown_connections 2024-04-23T14:46:44.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.774+0000 7fcf28c00700 1 --2- 172.21.15.83:0/1981525998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf241033e0 0x7fcf2410f8f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:44.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.774+0000 7fcf28c00700 1 --2- 172.21.15.83:0/1981525998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf24108a20 0x7fcf24108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:44.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.774+0000 7fcf28c00700 1 --2- 172.21.15.83:0/1981525998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf24102a20 0x7fcf24102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:44.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.774+0000 7fcf28c00700 1 -- 172.21.15.83:0/1981525998 >> 172.21.15.83:0/1981525998 conn(0x7fcf240fe570 msgr2=0x7fcf24100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:44.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.774+0000 7fcf28c00700 1 -- 172.21.15.83:0/1981525998 shutdown_connections 2024-04-23T14:46:44.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.774+0000 7fcf28c00700 1 -- 172.21.15.83:0/1981525998 wait complete. 2024-04-23T14:46:44.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.776+0000 7fcf28c00700 1 Processor -- start 2024-04-23T14:46:44.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.776+0000 7fcf28c00700 1 -- start start 2024-04-23T14:46:44.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.776+0000 7fcf28c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf24102a20 0x7fcf2419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:44.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.776+0000 7fcf28c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf241033e0 0x7fcf2419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.776+0000 7fcf28c00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf24108a20 0x7fcf2419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.776+0000 7fcf28c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf24112db0 con 0x7fcf24102a20 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.776+0000 7fcf28c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcf24112c30 con 0x7fcf241033e0 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.776+0000 7fcf28c00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf24112f30 con 0x7fcf24108a20 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.776+0000 7fcf23400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf24102a20 0x7fcf2419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.777+0000 7fcf23400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf24102a20 0x7fcf2419b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42350/0 (socket says 172.21.15.83:42350) 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.777+0000 7fcf23400700 1 -- 172.21.15.83:0/1609146176 learned_addr learned my addr 172.21.15.83:0/1609146176 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.777+0000 7fcf22a00700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf241033e0 0x7fcf2419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.777+0000 7fcf23e00700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf24108a20 0x7fcf2419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.777+0000 7fcf23400700 1 -- 172.21.15.83:0/1609146176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf24108a20 msgr2=0x7fcf2419fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.777+0000 7fcf23400700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf24108a20 0x7fcf2419fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.777+0000 7fcf23400700 1 -- 172.21.15.83:0/1609146176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf241033e0 msgr2=0x7fcf2419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.777+0000 7fcf23400700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf241033e0 0x7fcf2419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.777+0000 7fcf23400700 1 -- 172.21.15.83:0/1609146176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf241a02f0 con 0x7fcf24102a20 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.777+0000 7fcf23e00700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf24108a20 0x7fcf2419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.778+0000 7fcf23400700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf24102a20 0x7fcf2419b2e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400eb30 tx=0x7fcf1400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.778+0000 7fcf1be00700 1 -- 172.21.15.83:0/1609146176 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1400cc90 con 0x7fcf24102a20 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.778+0000 7fcf28c00700 1 -- 172.21.15.83:0/1609146176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf241a05e0 con 0x7fcf24102a20 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.778+0000 7fcf1be00700 1 -- 172.21.15.83:0/1609146176 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf14004500 con 0x7fcf24102a20 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.778+0000 7fcf1be00700 1 -- 172.21.15.83:0/1609146176 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14010430 con 0x7fcf24102a20 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.778+0000 7fcf28c00700 1 -- 172.21.15.83:0/1609146176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf241a7e60 con 0x7fcf24102a20 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.780+0000 7fcf28c00700 1 -- 172.21.15.83:0/1609146176 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf240ffc30 con 0x7fcf24102a20 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.783+0000 7fcf1be00700 1 -- 172.21.15.83:0/1609146176 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcf14003f70 con 0x7fcf24102a20 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.784+0000 7fcf1be00700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcf080a3d50 0x7fcf080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.784+0000 7fcf1be00700 1 -- 172.21.15.83:0/1609146176 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fcf140d06a0 con 0x7fcf24102a20 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.785+0000 7fcf22a00700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcf080a3d50 0x7fcf080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.785+0000 7fcf1be00700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520] conn(0x7fcf080ad530 0x7fcf080af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.785+0000 7fcf1be00700 1 -- 172.21.15.83:0/1609146176 --> [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf080b0000 con 0x7fcf080ad530 2024-04-23T14:46:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.785+0000 7fcf1be00700 1 -- 172.21.15.83:0/1609146176 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fcf140d7050 con 0x7fcf24102a20 2024-04-23T14:46:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.785+0000 7fcf22a00700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcf080a3d50 0x7fcf080a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcf241082e0 tx=0x7fcf10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.785+0000 7fcf23e00700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520] conn(0x7fcf080ad530 0x7fcf080af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.786+0000 7fcf23e00700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520] conn(0x7fcf080ad530 0x7fcf080af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.787+0000 7fcf1be00700 1 -- 172.21.15.83:0/1609146176 <== osd.10 v2:172.21.15.173:6824/2121565520 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf080b0000 con 0x7fcf080ad530 2024-04-23T14:46:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.805+0000 7fcf28c00700 1 -- 172.21.15.83:0/1609146176 --> [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf24104160 con 0x7fcf080ad530 2024-04-23T14:46:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.806+0000 7fcf1be00700 1 -- 172.21.15.83:0/1609146176 <== osd.10 v2:172.21.15.173:6824/2121565520 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcf24104160 con 0x7fcf080ad530 2024-04-23T14:46:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.806+0000 7fcf28c00700 1 -- 172.21.15.83:0/1609146176 >> [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520] conn(0x7fcf080ad530 msgr2=0x7fcf080af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.806+0000 7fcf28c00700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520] conn(0x7fcf080ad530 0x7fcf080af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.806+0000 7fcf28c00700 1 -- 172.21.15.83:0/1609146176 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcf080a3d50 msgr2=0x7fcf080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.806+0000 7fcf28c00700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcf080a3d50 0x7fcf080a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcf241082e0 tx=0x7fcf10009260 comp rx=0 tx=0).stop 2024-04-23T14:46:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.806+0000 7fcf28c00700 1 -- 172.21.15.83:0/1609146176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf24102a20 msgr2=0x7fcf2419b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.806+0000 7fcf28c00700 1 --2- 172.21.15.83:0/1609146176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf24102a20 0x7fcf2419b2e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400eb30 tx=0x7fcf1400ee40 comp rx=0 tx=0).stop 2024-04-23T14:46:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.806+0000 7fcf23e00700 1 -- 172.21.15.83:0/1609146176 reap_dead start 2024-04-23T14:46:44.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.807+0000 7fcf28c00700 1 -- 172.21.15.83:0/1609146176 shutdown_connections 2024-04-23T14:46:44.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.807+0000 7fcf28c00700 1 -- 172.21.15.83:0/1609146176 >> 172.21.15.83:0/1609146176 conn(0x7fcf240fe570 msgr2=0x7fcf240751a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:44.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.807+0000 7fcf28c00700 1 -- 172.21.15.83:0/1609146176 shutdown_connections 2024-04-23T14:46:44.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.807+0000 7fcf28c00700 1 -- 172.21.15.83:0/1609146176 wait complete. 2024-04-23T14:46:45.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.895+0000 7f5b6f600700 1 -- 172.21.15.83:0/1931666411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68113cc0 msgr2=0x7f5b681160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:45.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.895+0000 7f5b6f600700 1 --2- 172.21.15.83:0/1931666411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68113cc0 0x7f5b681160b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000b0a0 tx=0x7f5b6000b3b0 comp rx=0 tx=0).stop 2024-04-23T14:46:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.895+0000 7f5b6f600700 1 -- 172.21.15.83:0/1931666411 shutdown_connections 2024-04-23T14:46:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.895+0000 7f5b6f600700 1 --2- 172.21.15.83:0/1931666411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68113cc0 0x7f5b681160b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.895+0000 7f5b6f600700 1 --2- 172.21.15.83:0/1931666411 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b68074d20 0x7f5b68075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.895+0000 7f5b6f600700 1 --2- 172.21.15.83:0/1931666411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b68075640 0x7f5b68113780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.895+0000 7f5b6f600700 1 -- 172.21.15.83:0/1931666411 >> 172.21.15.83:0/1931666411 conn(0x7f5b68079c50 msgr2=0x7f5b6807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.895+0000 7f5b6f600700 1 -- 172.21.15.83:0/1931666411 shutdown_connections 2024-04-23T14:46:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.895+0000 7f5b6f600700 1 -- 172.21.15.83:0/1931666411 wait complete. 2024-04-23T14:46:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.896+0000 7f5b6f600700 1 Processor -- start 2024-04-23T14:46:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.896+0000 7f5b6f600700 1 -- start start 2024-04-23T14:46:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.897+0000 7f5b6f600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68074d20 0x7f5b681a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:45.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.897+0000 7f5b6f600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b68075640 0x7f5b681a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.897+0000 7f5b6f600700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b68113cc0 0x7f5b6819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.897+0000 7f5b6f600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68119420 con 0x7f5b68074d20 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.897+0000 7f5b6f600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681192a0 con 0x7f5b68075640 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.897+0000 7f5b6f600700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681195a0 con 0x7f5b68113cc0 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.897+0000 7f5b6cc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68074d20 0x7f5b681a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.897+0000 7f5b6cc00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68074d20 0x7f5b681a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49104/0 (socket says 172.21.15.83:49104) 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.897+0000 7f5b6cc00700 1 -- 172.21.15.83:0/3270399939 learned_addr learned my addr 172.21.15.83:0/3270399939 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.897+0000 7f5b67e00700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b68075640 0x7f5b681a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.897+0000 7f5b6d600700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b68113cc0 0x7f5b6819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.898+0000 7f5b6cc00700 1 -- 172.21.15.83:0/3270399939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b68113cc0 msgr2=0x7f5b6819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.898+0000 7f5b6cc00700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b68113cc0 0x7f5b6819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.898+0000 7f5b6cc00700 1 -- 172.21.15.83:0/3270399939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b68075640 msgr2=0x7f5b681a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.898+0000 7f5b6cc00700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b68075640 0x7f5b681a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:45.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.898+0000 7f5b6cc00700 1 -- 172.21.15.83:0/3270399939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6819ffa0 con 0x7f5b68074d20 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.898+0000 7f5b6d600700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b68113cc0 0x7f5b6819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.898+0000 7f5b67e00700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b68075640 0x7f5b681a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.898+0000 7f5b6cc00700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68074d20 0x7f5b681a55f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00e9a0 tx=0x7f5b5c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.898+0000 7f5b65600700 1 -- 172.21.15.83:0/3270399939 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00ca10 con 0x7f5b68074d20 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.898+0000 7f5b65600700 1 -- 172.21.15.83:0/3270399939 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5c00cbb0 con 0x7f5b68074d20 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.898+0000 7f5b6f600700 1 -- 172.21.15.83:0/3270399939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b681a0290 con 0x7f5b68074d20 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.898+0000 7f5b65600700 1 -- 172.21.15.83:0/3270399939 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c018680 con 0x7f5b68074d20 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.899+0000 7f5b6f600700 1 -- 172.21.15.83:0/3270399939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b681ac570 con 0x7f5b68074d20 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.899+0000 7f5b6f600700 1 -- 172.21.15.83:0/3270399939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b68113430 con 0x7f5b68074d20 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.902+0000 7f5b65600700 1 -- 172.21.15.83:0/3270399939 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5b5c010430 con 0x7f5b68074d20 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.903+0000 7f5b65600700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5b500a3d50 0x7f5b500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.903+0000 7f5b65600700 1 -- 172.21.15.83:0/3270399939 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5b5c0d0ed0 con 0x7f5b68074d20 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.903+0000 7f5b67e00700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5b500a3d50 0x7f5b500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:45.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.903+0000 7f5b65600700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161] conn(0x7f5b500ad530 0x7f5b500af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:45.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.903+0000 7f5b65600700 1 -- 172.21.15.83:0/3270399939 --> [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b5c027e50 con 0x7f5b500ad530 2024-04-23T14:46:45.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.903+0000 7f5b65600700 1 -- 172.21.15.83:0/3270399939 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f5b5c0d1280 con 0x7f5b68074d20 2024-04-23T14:46:45.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.904+0000 7f5b6d600700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161] conn(0x7f5b500ad530 0x7f5b500af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:45.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.904+0000 7f5b67e00700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5b500a3d50 0x7f5b500a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5b681a0e80 tx=0x7f5b54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:45.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.904+0000 7f5b6d600700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161] conn(0x7f5b500ad530 0x7f5b500af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:45.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.905+0000 7f5b65600700 1 -- 172.21.15.83:0/3270399939 <== osd.3 v2:172.21.15.83:6810/3253964161 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b5c027e50 con 0x7f5b500ad530 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.923+0000 7f5b6f600700 1 -- 172.21.15.83:0/3270399939 --> [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b68076410 con 0x7f5b500ad530 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.924+0000 7f5b65600700 1 -- 172.21.15.83:0/3270399939 <== osd.3 v2:172.21.15.83:6810/3253964161 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5b68076410 con 0x7f5b500ad530 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.924+0000 7f5b6f600700 1 -- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161] conn(0x7f5b500ad530 msgr2=0x7f5b500af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.924+0000 7f5b6f600700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161] conn(0x7f5b500ad530 0x7f5b500af950 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.924+0000 7f5b6f600700 1 -- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5b500a3d50 msgr2=0x7f5b500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.924+0000 7f5b6f600700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5b500a3d50 0x7f5b500a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5b681a0e80 tx=0x7f5b54009260 comp rx=0 tx=0).stop 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.924+0000 7f5b6f600700 1 -- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68074d20 msgr2=0x7f5b681a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.924+0000 7f5b6f600700 1 --2- 172.21.15.83:0/3270399939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68074d20 0x7f5b681a55f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00e9a0 tx=0x7f5b5c00ecb0 comp rx=0 tx=0).stop 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.924+0000 7f5b6d600700 1 -- 172.21.15.83:0/3270399939 reap_dead start 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.924+0000 7f5b6f600700 1 -- 172.21.15.83:0/3270399939 shutdown_connections 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.924+0000 7f5b6f600700 1 -- 172.21.15.83:0/3270399939 >> 172.21.15.83:0/3270399939 conn(0x7f5b68079c50 msgr2=0x7f5b68112a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.925+0000 7f5b6f600700 1 -- 172.21.15.83:0/3270399939 shutdown_connections 2024-04-23T14:46:45.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:44.925+0000 7f5b6f600700 1 -- 172.21.15.83:0/3270399939 wait complete. 2024-04-23T14:46:46.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.090+0000 7fbf15800700 1 -- 172.21.15.83:0/1010233749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf10109030 msgr2=0x7fbf101094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:46.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.090+0000 7fbf15800700 1 --2- 172.21.15.83:0/1010233749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf10109030 0x7fbf101094b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009990 tx=0x7fbefc009ca0 comp rx=0 tx=0).stop 2024-04-23T14:46:46.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.090+0000 7fbf15800700 1 -- 172.21.15.83:0/1010233749 shutdown_connections 2024-04-23T14:46:46.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.090+0000 7fbf15800700 1 --2- 172.21.15.83:0/1010233749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf101099f0 0x7fbf10115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:46.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.090+0000 7fbf15800700 1 --2- 172.21.15.83:0/1010233749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf1010f0e0 0x7fbf1010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:46.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.090+0000 7fbf15800700 1 --2- 172.21.15.83:0/1010233749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf10109030 0x7fbf101094b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:46.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.090+0000 7fbf15800700 1 -- 172.21.15.83:0/1010233749 >> 172.21.15.83:0/1010233749 conn(0x7fbf10078880 msgr2=0x7fbf10107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:46.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.090+0000 7fbf15800700 1 -- 172.21.15.83:0/1010233749 shutdown_connections 2024-04-23T14:46:46.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.090+0000 7fbf15800700 1 -- 172.21.15.83:0/1010233749 wait complete. 2024-04-23T14:46:46.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.091+0000 7fbf15800700 1 Processor -- start 2024-04-23T14:46:46.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.092+0000 7fbf15800700 1 -- start start 2024-04-23T14:46:46.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.092+0000 7fbf15800700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf10109030 0x7fbf101a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:46.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.092+0000 7fbf15800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf101099f0 0x7fbf101a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:46.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.092+0000 7fbf15800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1010f0e0 0x7fbf1019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:46.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.092+0000 7fbf15800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf10119360 con 0x7fbf101099f0 2024-04-23T14:46:46.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.092+0000 7fbf15800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbf101191e0 con 0x7fbf1010f0e0 2024-04-23T14:46:46.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.092+0000 7fbf15800700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbf101194e0 con 0x7fbf10109030 2024-04-23T14:46:46.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.092+0000 7fbf0f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf101099f0 0x7fbf101a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:46.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.092+0000 7fbf0fe00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf10109030 0x7fbf101a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:46.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.092+0000 7fbf0fe00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf10109030 0x7fbf101a5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41488/0 (socket says 172.21.15.83:41488) 2024-04-23T14:46:46.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.092+0000 7fbf0fe00700 1 -- 172.21.15.83:0/1920345773 learned_addr learned my addr 172.21.15.83:0/1920345773 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:46.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.093+0000 7fbf14e00700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1010f0e0 0x7fbf1019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:46.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.093+0000 7fbf0fe00700 1 -- 172.21.15.83:0/1920345773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1010f0e0 msgr2=0x7fbf1019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:46.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.093+0000 7fbf0fe00700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1010f0e0 0x7fbf1019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:46.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.093+0000 7fbf0fe00700 1 -- 172.21.15.83:0/1920345773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf101099f0 msgr2=0x7fbf101a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:46.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.093+0000 7fbf0fe00700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf101099f0 0x7fbf101a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:46.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.093+0000 7fbf0fe00700 1 -- 172.21.15.83:0/1920345773 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1019fd20 con 0x7fbf10109030 2024-04-23T14:46:46.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.093+0000 7fbf14e00700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1010f0e0 0x7fbf1019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:46:46.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.093+0000 7fbf0fe00700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf10109030 0x7fbf101a5400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000e9c0 tx=0x7fbf0000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:46.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.094+0000 7fbf0cc00700 1 -- 172.21.15.83:0/1920345773 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0000cbd0 con 0x7fbf10109030 2024-04-23T14:46:46.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.094+0000 7fbf15800700 1 -- 172.21.15.83:0/1920345773 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf101a0010 con 0x7fbf10109030 2024-04-23T14:46:46.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.094+0000 7fbf0cc00700 1 -- 172.21.15.83:0/1920345773 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf0000cd70 con 0x7fbf10109030 2024-04-23T14:46:46.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.094+0000 7fbf0cc00700 1 -- 172.21.15.83:0/1920345773 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00010430 con 0x7fbf10109030 2024-04-23T14:46:46.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.094+0000 7fbf15800700 1 -- 172.21.15.83:0/1920345773 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf101ac2d0 con 0x7fbf10109030 2024-04-23T14:46:46.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.095+0000 7fbf15800700 1 -- 172.21.15.83:0/1920345773 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf10063280 con 0x7fbf10109030 2024-04-23T14:46:46.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.097+0000 7fbf0cc00700 1 -- 172.21.15.83:0/1920345773 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbf00004750 con 0x7fbf10109030 2024-04-23T14:46:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.098+0000 7fbf0cc00700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbef40a3d50 0x7fbef40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.099+0000 7fbf0cc00700 1 -- 172.21.15.83:0/1920345773 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fbf000d0b00 con 0x7fbf10109030 2024-04-23T14:46:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.099+0000 7fbf0f400700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbef40a3d50 0x7fbef40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.099+0000 7fbf0cc00700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936] conn(0x7fbef40ad530 0x7fbef40af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:46.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.099+0000 7fbf0cc00700 1 -- 172.21.15.83:0/1920345773 --> [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf00017e50 con 0x7fbef40ad530 2024-04-23T14:46:46.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.099+0000 7fbf0cc00700 1 -- 172.21.15.83:0/1920345773 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fbf000d0eb0 con 0x7fbf10109030 2024-04-23T14:46:46.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.099+0000 7fbf14e00700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936] conn(0x7fbef40ad530 0x7fbef40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:46.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.099+0000 7fbf0f400700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbef40a3d50 0x7fbef40a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbf1010e840 tx=0x7fbefc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:46.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.100+0000 7fbf14e00700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936] conn(0x7fbef40ad530 0x7fbef40af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:46.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.101+0000 7fbf0cc00700 1 -- 172.21.15.83:0/1920345773 <== osd.7 v2:172.21.15.173:6816/1174963936 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf00017e50 con 0x7fbef40ad530 2024-04-23T14:46:46.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.119+0000 7fbf15800700 1 -- 172.21.15.83:0/1920345773 --> [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf10063680 con 0x7fbef40ad530 2024-04-23T14:46:46.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.120+0000 7fbf0cc00700 1 -- 172.21.15.83:0/1920345773 <== osd.7 v2:172.21.15.173:6816/1174963936 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbf10063680 con 0x7fbef40ad530 2024-04-23T14:46:46.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.120+0000 7fbf15800700 1 -- 172.21.15.83:0/1920345773 >> [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936] conn(0x7fbef40ad530 msgr2=0x7fbef40af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:46.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.120+0000 7fbf15800700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936] conn(0x7fbef40ad530 0x7fbef40af950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.120+0000 7fbf15800700 1 -- 172.21.15.83:0/1920345773 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbef40a3d50 msgr2=0x7fbef40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.120+0000 7fbf15800700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbef40a3d50 0x7fbef40a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbf1010e840 tx=0x7fbefc00b260 comp rx=0 tx=0).stop 2024-04-23T14:46:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.120+0000 7fbf15800700 1 -- 172.21.15.83:0/1920345773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf10109030 msgr2=0x7fbf101a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.120+0000 7fbf15800700 1 --2- 172.21.15.83:0/1920345773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf10109030 0x7fbf101a5400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000e9c0 tx=0x7fbf0000ecd0 comp rx=0 tx=0).stop 2024-04-23T14:46:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.121+0000 7fbf14e00700 1 -- 172.21.15.83:0/1920345773 reap_dead start 2024-04-23T14:46:46.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.121+0000 7fbf15800700 1 -- 172.21.15.83:0/1920345773 shutdown_connections 2024-04-23T14:46:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.121+0000 7fbf15800700 1 -- 172.21.15.83:0/1920345773 >> 172.21.15.83:0/1920345773 conn(0x7fbf10078880 msgr2=0x7fbf10079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.121+0000 7fbf15800700 1 -- 172.21.15.83:0/1920345773 shutdown_connections 2024-04-23T14:46:46.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:46.121+0000 7fbf15800700 1 -- 172.21.15.83:0/1920345773 wait complete. 2024-04-23T14:46:46.385 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:46 smithi083 ceph-mon[27867]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T14:46:46.433 INFO:teuthology.orchestra.run.smithi083.stdout:313532612635 2024-04-23T14:46:46.434 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-23T14:46:46.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:46 smithi173 ceph-mon[32443]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T14:46:46.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:46 smithi203 ceph-mon[32800]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T14:46:48.039 INFO:teuthology.orchestra.run.smithi083.stdout:163208757305 2024-04-23T14:46:48.040 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-23T14:46:48.354 INFO:teuthology.orchestra.run.smithi083.stdout:107374182469 2024-04-23T14:46:48.355 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-23T14:46:48.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:48 smithi173 ceph-mon[32443]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:46:48.600 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:48 smithi083 ceph-mon[27867]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:46:48.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:48 smithi203 ceph-mon[32800]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:46:49.504 INFO:teuthology.orchestra.run.smithi083.stdout:197568495667 2024-04-23T14:46:49.504 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-23T14:46:49.632 INFO:teuthology.orchestra.run.smithi083.stdout:257698037800 2024-04-23T14:46:49.632 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-23T14:46:50.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.526+0000 7f0c6ac00700 1 -- 172.21.15.83:0/1282210534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c64103500 msgr2=0x7f0c6410f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.526+0000 7f0c6ac00700 1 --2- 172.21.15.83:0/1282210534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c64103500 0x7f0c6410f960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c00b0a0 tx=0x7f0c5c00b3b0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.526+0000 7f0c6ac00700 1 -- 172.21.15.83:0/1282210534 shutdown_connections 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.526+0000 7f0c6ac00700 1 --2- 172.21.15.83:0/1282210534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c64103500 0x7f0c6410f960 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.526+0000 7f0c6ac00700 1 --2- 172.21.15.83:0/1282210534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c64108b90 0x7f0c64108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.526+0000 7f0c6ac00700 1 --2- 172.21.15.83:0/1282210534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c64102b40 0x7f0c64102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.526+0000 7f0c6ac00700 1 -- 172.21.15.83:0/1282210534 >> 172.21.15.83:0/1282210534 conn(0x7f0c64079c50 msgr2=0x7f0c6407a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.526+0000 7f0c6ac00700 1 -- 172.21.15.83:0/1282210534 shutdown_connections 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.526+0000 7f0c6ac00700 1 -- 172.21.15.83:0/1282210534 wait complete. 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.528+0000 7f0c6ac00700 1 Processor -- start 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.528+0000 7f0c6ac00700 1 -- start start 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.528+0000 7f0c6ac00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c64102b40 0x7f0c64199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.528+0000 7f0c6ac00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c64103500 0x7f0c641997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.528+0000 7f0c6ac00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c64108b90 0x7f0c6419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:50.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.528+0000 7f0c6ac00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c64112e40 con 0x7f0c64103500 2024-04-23T14:46:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.528+0000 7f0c6ac00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0c64112cc0 con 0x7f0c64108b90 2024-04-23T14:46:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.528+0000 7f0c6ac00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0c64112fc0 con 0x7f0c64102b40 2024-04-23T14:46:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.529+0000 7f0c63400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c64103500 0x7f0c641997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.529+0000 7f0c63e00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c64102b40 0x7f0c64199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.529+0000 7f0c63400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c64103500 0x7f0c641997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49150/0 (socket says 172.21.15.83:49150) 2024-04-23T14:46:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.529+0000 7f0c63400700 1 -- 172.21.15.83:0/115938756 learned_addr learned my addr 172.21.15.83:0/115938756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.529+0000 7f0c68c00700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c64108b90 0x7f0c6419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.529+0000 7f0c63e00700 1 -- 172.21.15.83:0/115938756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c64108b90 msgr2=0x7f0c6419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:50.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.529+0000 7f0c63e00700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c64108b90 0x7f0c6419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.529+0000 7f0c63e00700 1 -- 172.21.15.83:0/115938756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c64103500 msgr2=0x7f0c641997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.529+0000 7f0c63e00700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c64103500 0x7f0c641997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.529+0000 7f0c63e00700 1 -- 172.21.15.83:0/115938756 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6419e2a0 con 0x7f0c64102b40 2024-04-23T14:46:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.529+0000 7f0c63400700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c64103500 0x7f0c641997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.530+0000 7f0c63e00700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c64102b40 0x7f0c64199290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400e9f0 tx=0x7f0c5400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.530+0000 7f0c60c00700 1 -- 172.21.15.83:0/115938756 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400c9c0 con 0x7f0c64102b40 2024-04-23T14:46:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.530+0000 7f0c60c00700 1 -- 172.21.15.83:0/115938756 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c5400cb60 con 0x7f0c64102b40 2024-04-23T14:46:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.530+0000 7f0c6ac00700 1 -- 172.21.15.83:0/115938756 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6419e590 con 0x7f0c64102b40 2024-04-23T14:46:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.530+0000 7f0c60c00700 1 -- 172.21.15.83:0/115938756 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c54018680 con 0x7f0c64102b40 2024-04-23T14:46:50.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.530+0000 7f0c6ac00700 1 -- 172.21.15.83:0/115938756 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c641a5e10 con 0x7f0c64102b40 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.531+0000 7f0c6ac00700 1 -- 172.21.15.83:0/115938756 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c64063200 con 0x7f0c64102b40 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.533+0000 7f0c60c00700 1 -- 172.21.15.83:0/115938756 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0c54010430 con 0x7f0c64102b40 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.535+0000 7f0c60c00700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c480a3d50 0x7f0c480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.535+0000 7f0c60c00700 1 -- 172.21.15.83:0/115938756 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0c540d04f0 con 0x7f0c64102b40 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.535+0000 7f0c63400700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c480a3d50 0x7f0c480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.535+0000 7f0c60c00700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923] conn(0x7f0c480ad530 0x7f0c480af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.535+0000 7f0c60c00700 1 -- 172.21.15.83:0/115938756 --> [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c54015e80 con 0x7f0c480ad530 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.535+0000 7f0c60c00700 1 -- 172.21.15.83:0/115938756 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f0c540d7050 con 0x7f0c64102b40 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.536+0000 7f0c68c00700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923] conn(0x7f0c480ad530 0x7f0c480af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.536+0000 7f0c63400700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c480a3d50 0x7f0c480a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c6407a1e0 tx=0x7f0c50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.537+0000 7f0c68c00700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923] conn(0x7f0c480ad530 0x7f0c480af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.538+0000 7f0c60c00700 1 -- 172.21.15.83:0/115938756 <== osd.11 v2:172.21.15.203:6824/2856576923 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c54015e80 con 0x7f0c480ad530 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.557+0000 7f0c6ac00700 1 -- 172.21.15.83:0/115938756 --> [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c64063600 con 0x7f0c480ad530 2024-04-23T14:46:50.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.558+0000 7f0c60c00700 1 -- 172.21.15.83:0/115938756 <== osd.11 v2:172.21.15.203:6824/2856576923 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0c64063600 con 0x7f0c480ad530 2024-04-23T14:46:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.559+0000 7f0c6ac00700 1 -- 172.21.15.83:0/115938756 >> [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923] conn(0x7f0c480ad530 msgr2=0x7f0c480af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.559+0000 7f0c6ac00700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923] conn(0x7f0c480ad530 0x7f0c480af950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.559+0000 7f0c6ac00700 1 -- 172.21.15.83:0/115938756 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c480a3d50 msgr2=0x7f0c480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.559+0000 7f0c6ac00700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f0c480a3d50 0x7f0c480a6210 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c6407a1e0 tx=0x7f0c50009260 comp rx=0 tx=0).stop 2024-04-23T14:46:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.559+0000 7f0c6ac00700 1 -- 172.21.15.83:0/115938756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c64102b40 msgr2=0x7f0c64199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.559+0000 7f0c6ac00700 1 --2- 172.21.15.83:0/115938756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c64102b40 0x7f0c64199290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400e9f0 tx=0x7f0c5400ed00 comp rx=0 tx=0).stop 2024-04-23T14:46:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.559+0000 7f0c68c00700 1 -- 172.21.15.83:0/115938756 reap_dead start 2024-04-23T14:46:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.559+0000 7f0c6ac00700 1 -- 172.21.15.83:0/115938756 shutdown_connections 2024-04-23T14:46:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.559+0000 7f0c6ac00700 1 -- 172.21.15.83:0/115938756 >> 172.21.15.83:0/115938756 conn(0x7f0c64079c50 msgr2=0x7f0c641016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.559+0000 7f0c6ac00700 1 -- 172.21.15.83:0/115938756 shutdown_connections 2024-04-23T14:46:50.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:49.559+0000 7f0c6ac00700 1 -- 172.21.15.83:0/115938756 wait complete. 2024-04-23T14:46:50.542 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:50 smithi083 ceph-mon[27867]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-23T14:46:50.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.694+0000 7f9bb1600700 1 -- 172.21.15.83:0/4000048492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bac101cf0 msgr2=0x7f9bac107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.694+0000 7f9bb1600700 1 --2- 172.21.15.83:0/4000048492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bac101cf0 0x7f9bac107750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009990 tx=0x7f9b98009ca0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.695+0000 7f9bb1600700 1 -- 172.21.15.83:0/4000048492 shutdown_connections 2024-04-23T14:46:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.695+0000 7f9bb1600700 1 --2- 172.21.15.83:0/4000048492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bac107e30 0x7f9bac10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.695+0000 7f9bb1600700 1 --2- 172.21.15.83:0/4000048492 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bac101340 0x7f9bac101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.695+0000 7f9bb1600700 1 --2- 172.21.15.83:0/4000048492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bac101cf0 0x7f9bac107750 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.695+0000 7f9bb1600700 1 -- 172.21.15.83:0/4000048492 >> 172.21.15.83:0/4000048492 conn(0x7f9bac078090 msgr2=0x7f9bac0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.695+0000 7f9bb1600700 1 -- 172.21.15.83:0/4000048492 shutdown_connections 2024-04-23T14:46:50.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.695+0000 7f9bb1600700 1 -- 172.21.15.83:0/4000048492 wait complete. 2024-04-23T14:46:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.696+0000 7f9bb1600700 1 Processor -- start 2024-04-23T14:46:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.696+0000 7f9bb1600700 1 -- start start 2024-04-23T14:46:50.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.696+0000 7f9bb1600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bac101340 0x7f9bac199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.696+0000 7f9bb1600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bac101cf0 0x7f9bac199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.696+0000 7f9bb1600700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bac107e30 0x7f9bac19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.696+0000 7f9bb1600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac112e40 con 0x7f9bac101340 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.696+0000 7f9bb1600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac112cc0 con 0x7f9bac101cf0 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.696+0000 7f9bb1600700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac112fc0 con 0x7f9bac107e30 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.697+0000 7f9baaa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bac101340 0x7f9bac199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.697+0000 7f9baaa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bac101340 0x7f9bac199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49174/0 (socket says 172.21.15.83:49174) 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.697+0000 7f9baaa00700 1 -- 172.21.15.83:0/3277388463 learned_addr learned my addr 172.21.15.83:0/3277388463 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.697+0000 7f9baa000700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bac101cf0 0x7f9bac199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.697+0000 7f9bab400700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bac107e30 0x7f9bac19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.697+0000 7f9baaa00700 1 -- 172.21.15.83:0/3277388463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bac107e30 msgr2=0x7f9bac19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.697+0000 7f9baaa00700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bac107e30 0x7f9bac19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.697+0000 7f9baaa00700 1 -- 172.21.15.83:0/3277388463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bac101cf0 msgr2=0x7f9bac199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:50.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.697+0000 7f9baaa00700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9bac101cf0 0x7f9bac199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.697+0000 7f9baaa00700 1 -- 172.21.15.83:0/3277388463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac19e310 con 0x7f9bac101340 2024-04-23T14:46:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.698+0000 7f9baaa00700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bac101340 0x7f9bac199350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00eb30 tx=0x7f9b9c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.698+0000 7f9ba3400700 1 -- 172.21.15.83:0/3277388463 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c00cbd0 con 0x7f9bac101340 2024-04-23T14:46:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.698+0000 7f9ba3400700 1 -- 172.21.15.83:0/3277388463 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b9c004500 con 0x7f9bac101340 2024-04-23T14:46:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.698+0000 7f9ba3400700 1 -- 172.21.15.83:0/3277388463 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c010430 con 0x7f9bac101340 2024-04-23T14:46:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.698+0000 7f9bb1600700 1 -- 172.21.15.83:0/3277388463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac19e600 con 0x7f9bac101340 2024-04-23T14:46:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.698+0000 7f9bb1600700 1 -- 172.21.15.83:0/3277388463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac1021f0 con 0x7f9bac101340 2024-04-23T14:46:50.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.699+0000 7f9bb1600700 1 -- 172.21.15.83:0/3277388463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9bac063280 con 0x7f9bac101340 2024-04-23T14:46:50.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.701+0000 7f9ba3400700 1 -- 172.21.15.83:0/3277388463 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9b9c003f00 con 0x7f9bac101340 2024-04-23T14:46:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.702+0000 7f9ba3400700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9b900a3d50 0x7f9b900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.703+0000 7f9ba3400700 1 -- 172.21.15.83:0/3277388463 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9b9c0d0a40 con 0x7f9bac101340 2024-04-23T14:46:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.703+0000 7f9baa000700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9b900a3d50 0x7f9b900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:50.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.703+0000 7f9ba3400700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483] conn(0x7f9b900ad530 0x7f9b900af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.703+0000 7f9ba3400700 1 -- 172.21.15.83:0/3277388463 --> [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b9c01ee50 con 0x7f9b900ad530 2024-04-23T14:46:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.703+0000 7f9ba3400700 1 -- 172.21.15.83:0/3277388463 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9b9c0d0df0 con 0x7f9bac101340 2024-04-23T14:46:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.703+0000 7f9bab400700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483] conn(0x7f9b900ad530 0x7f9b900af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:50.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.703+0000 7f9baa000700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9b900a3d50 0x7f9b900a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9bac105680 tx=0x7f9b98005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:50.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.704+0000 7f9bab400700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483] conn(0x7f9b900ad530 0x7f9b900af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:50.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.705+0000 7f9ba3400700 1 -- 172.21.15.83:0/3277388463 <== osd.1 v2:172.21.15.173:6800/548359483 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b9c01ee50 con 0x7f9b900ad530 2024-04-23T14:46:50.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:50 smithi203 ceph-mon[32800]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-23T14:46:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.723+0000 7f9bb1600700 1 -- 172.21.15.83:0/3277388463 --> [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9bac063750 con 0x7f9b900ad530 2024-04-23T14:46:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.724+0000 7f9ba3400700 1 -- 172.21.15.83:0/3277388463 <== osd.1 v2:172.21.15.173:6800/548359483 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9bac063750 con 0x7f9b900ad530 2024-04-23T14:46:50.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.724+0000 7f9bb1600700 1 -- 172.21.15.83:0/3277388463 >> [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483] conn(0x7f9b900ad530 msgr2=0x7f9b900af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:50.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.724+0000 7f9bb1600700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483] conn(0x7f9b900ad530 0x7f9b900af950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.725+0000 7f9bb1600700 1 -- 172.21.15.83:0/3277388463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9b900a3d50 msgr2=0x7f9b900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:50.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.725+0000 7f9bb1600700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9b900a3d50 0x7f9b900a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9bac105680 tx=0x7f9b98005fd0 comp rx=0 tx=0).stop 2024-04-23T14:46:50.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.725+0000 7f9bb1600700 1 -- 172.21.15.83:0/3277388463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bac101340 msgr2=0x7f9bac199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:50.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.725+0000 7f9bb1600700 1 --2- 172.21.15.83:0/3277388463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9bac101340 0x7f9bac199350 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00eb30 tx=0x7f9b9c00ee40 comp rx=0 tx=0).stop 2024-04-23T14:46:50.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.725+0000 7f9bab400700 1 -- 172.21.15.83:0/3277388463 reap_dead start 2024-04-23T14:46:50.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.725+0000 7f9bb1600700 1 -- 172.21.15.83:0/3277388463 shutdown_connections 2024-04-23T14:46:50.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.725+0000 7f9bb1600700 1 -- 172.21.15.83:0/3277388463 >> 172.21.15.83:0/3277388463 conn(0x7f9bac078090 msgr2=0x7f9bac0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:50.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.725+0000 7f9bb1600700 1 -- 172.21.15.83:0/3277388463 shutdown_connections 2024-04-23T14:46:50.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:50.725+0000 7f9bb1600700 1 -- 172.21.15.83:0/3277388463 wait complete. 2024-04-23T14:46:50.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:50 smithi173 ceph-mon[32443]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-23T14:46:51.704 INFO:teuthology.orchestra.run.smithi083.stdout:55834574929 2024-04-23T14:46:51.704 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-23T14:46:52.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:52 smithi083 ceph-mon[27867]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T14:46:52.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:52 smithi203 ceph-mon[32800]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T14:46:52.748 INFO:teuthology.orchestra.run.smithi083.stdout:347892351000 2024-04-23T14:46:52.748 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-23T14:46:52.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:52 smithi173 ceph-mon[32443]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T14:46:53.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.623+0000 7f88be200700 1 -- 172.21.15.83:0/3318465573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b8109d30 msgr2=0x7f88b810dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:53.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.623+0000 7f88be200700 1 --2- 172.21.15.83:0/3318465573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b8109d30 0x7f88b810dd80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009920 tx=0x7f88a4009c30 comp rx=0 tx=0).stop 2024-04-23T14:46:53.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.623+0000 7f88be200700 1 -- 172.21.15.83:0/3318465573 shutdown_connections 2024-04-23T14:46:53.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.623+0000 7f88be200700 1 --2- 172.21.15.83:0/3318465573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88b810e460 0x7f88b8116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:53.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.623+0000 7f88be200700 1 --2- 172.21.15.83:0/3318465573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b8109380 0x7f88b8109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.623+0000 7f88be200700 1 --2- 172.21.15.83:0/3318465573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b8109d30 0x7f88b810dd80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.623+0000 7f88be200700 1 -- 172.21.15.83:0/3318465573 >> 172.21.15.83:0/3318465573 conn(0x7f88b80762c0 msgr2=0x7f88b8107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.623+0000 7f88be200700 1 -- 172.21.15.83:0/3318465573 shutdown_connections 2024-04-23T14:46:53.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.623+0000 7f88be200700 1 -- 172.21.15.83:0/3318465573 wait complete. 2024-04-23T14:46:53.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.624+0000 7f88be200700 1 Processor -- start 2024-04-23T14:46:53.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.625+0000 7f88be200700 1 -- start start 2024-04-23T14:46:53.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.625+0000 7f88be200700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b8109380 0x7f88b81a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:53.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.625+0000 7f88be200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88b8109d30 0x7f88b81a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:53.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.625+0000 7f88be200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b810e460 0x7f88b819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:53.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.625+0000 7f88be200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88b8119420 con 0x7f88b8109d30 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.625+0000 7f88be200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81192a0 con 0x7f88b810e460 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.625+0000 7f88be200700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f88b81195a0 con 0x7f88b8109380 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.625+0000 7f88b6a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88b8109d30 0x7f88b81a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.625+0000 7f88b7e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b810e460 0x7f88b819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.625+0000 7f88b6a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88b8109d30 0x7f88b81a5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49220/0 (socket says 172.21.15.83:49220) 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.625+0000 7f88b7e00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b810e460 0x7f88b819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33900/0 (socket says 172.21.15.83:33900) 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.625+0000 7f88b6a00700 1 -- 172.21.15.83:0/2985228631 learned_addr learned my addr 172.21.15.83:0/2985228631 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.626+0000 7f88b7400700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b8109380 0x7f88b81a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.626+0000 7f88b7e00700 1 -- 172.21.15.83:0/2985228631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b8109380 msgr2=0x7f88b81a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.626+0000 7f88b7e00700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b8109380 0x7f88b81a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.626+0000 7f88b7e00700 1 -- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88b8109d30 msgr2=0x7f88b81a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.626+0000 7f88b7e00700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88b8109d30 0x7f88b81a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.626+0000 7f88b7e00700 1 -- 172.21.15.83:0/2985228631 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b81a0000 con 0x7f88b810e460 2024-04-23T14:46:53.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.626+0000 7f88b6a00700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88b8109d30 0x7f88b81a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:53.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.626+0000 7f88b7e00700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b810e460 0x7f88b819f720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f88b000efc0 tx=0x7f88b000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:53.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.626+0000 7f88afe00700 1 -- 172.21.15.83:0/2985228631 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b00092e0 con 0x7f88b810e460 2024-04-23T14:46:53.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.626+0000 7f88afe00700 1 -- 172.21.15.83:0/2985228631 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88b0004bb0 con 0x7f88b810e460 2024-04-23T14:46:53.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.626+0000 7f88be200700 1 -- 172.21.15.83:0/2985228631 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b81a0290 con 0x7f88b810e460 2024-04-23T14:46:53.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.627+0000 7f88afe00700 1 -- 172.21.15.83:0/2985228631 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0004280 con 0x7f88b810e460 2024-04-23T14:46:53.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.627+0000 7f88be200700 1 -- 172.21.15.83:0/2985228631 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b81ac5d0 con 0x7f88b810e460 2024-04-23T14:46:53.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.627+0000 7f88be200700 1 -- 172.21.15.83:0/2985228631 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88b81a05d0 con 0x7f88b810e460 2024-04-23T14:46:53.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.630+0000 7f88afe00700 1 -- 172.21.15.83:0/2985228631 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f88b001b030 con 0x7f88b810e460 2024-04-23T14:46:53.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.631+0000 7f88afe00700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f889c0a3d50 0x7f889c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:53.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.631+0000 7f88afe00700 1 -- 172.21.15.83:0/2985228631 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f88b00d07e0 con 0x7f88b810e460 2024-04-23T14:46:53.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.631+0000 7f88b7400700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f889c0a3d50 0x7f889c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:53.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.631+0000 7f88afe00700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722] conn(0x7f889c0ad530 0x7f889c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:53.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.631+0000 7f88afe00700 1 -- 172.21.15.83:0/2985228631 --> [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f889c0b0000 con 0x7f889c0ad530 2024-04-23T14:46:53.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.631+0000 7f88afe00700 1 -- 172.21.15.83:0/2985228631 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f88b00d0b90 con 0x7f88b810e460 2024-04-23T14:46:53.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.632+0000 7f88b6a00700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722] conn(0x7f889c0ad530 0x7f889c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:53.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.632+0000 7f88b7400700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f889c0a3d50 0x7f889c0a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009680 tx=0x7f88a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:53.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.632+0000 7f88b6a00700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722] conn(0x7f889c0ad530 0x7f889c0af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:53.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.633+0000 7f88afe00700 1 -- 172.21.15.83:0/2985228631 <== osd.0 v2:172.21.15.83:6802/1064791722 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f889c0b0000 con 0x7f889c0ad530 2024-04-23T14:46:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.651+0000 7f88be200700 1 -- 172.21.15.83:0/2985228631 --> [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f88b80634d0 con 0x7f889c0ad530 2024-04-23T14:46:53.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.651+0000 7f88afe00700 1 -- 172.21.15.83:0/2985228631 <== osd.0 v2:172.21.15.83:6802/1064791722 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f88b80634d0 con 0x7f889c0ad530 2024-04-23T14:46:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.652+0000 7f88be200700 1 -- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722] conn(0x7f889c0ad530 msgr2=0x7f889c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.652+0000 7f88be200700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722] conn(0x7f889c0ad530 0x7f889c0af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.652+0000 7f88be200700 1 -- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f889c0a3d50 msgr2=0x7f889c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.652+0000 7f88be200700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f889c0a3d50 0x7f889c0a6210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009680 tx=0x7f88a8009260 comp rx=0 tx=0).stop 2024-04-23T14:46:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.652+0000 7f88be200700 1 -- 172.21.15.83:0/2985228631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b810e460 msgr2=0x7f88b819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.652+0000 7f88be200700 1 --2- 172.21.15.83:0/2985228631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88b810e460 0x7f88b819f720 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f88b000efc0 tx=0x7f88b000c3a0 comp rx=0 tx=0).stop 2024-04-23T14:46:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.652+0000 7f88b7e00700 1 -- 172.21.15.83:0/2985228631 reap_dead start 2024-04-23T14:46:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.652+0000 7f88be200700 1 -- 172.21.15.83:0/2985228631 shutdown_connections 2024-04-23T14:46:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.652+0000 7f88be200700 1 -- 172.21.15.83:0/2985228631 >> 172.21.15.83:0/2985228631 conn(0x7f88b80762c0 msgr2=0x7f88b8076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.652+0000 7f88be200700 1 -- 172.21.15.83:0/2985228631 shutdown_connections 2024-04-23T14:46:53.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.652+0000 7f88be200700 1 -- 172.21.15.83:0/2985228631 wait complete. 2024-04-23T14:46:54.428 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:54 smithi083 ceph-mon[27867]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T14:46:54.501 INFO:teuthology.orchestra.run.smithi083.stdout:223338299439 2024-04-23T14:46:54.502 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-23T14:46:54.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:54 smithi203 ceph-mon[32800]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T14:46:54.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:54 smithi173 ceph-mon[32443]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T14:46:54.954 INFO:teuthology.orchestra.run.smithi083.stdout:34359738455 2024-04-23T14:46:54.955 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-23T14:46:55.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.497+0000 7fcc95a00700 1 -- 172.21.15.83:0/115625823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc9010f5a0 msgr2=0x7fcc90111990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:55.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.497+0000 7fcc95a00700 1 --2- 172.21.15.83:0/115625823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc9010f5a0 0x7fcc90111990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800b0a0 tx=0x7fcc8800b3b0 comp rx=0 tx=0).stop 2024-04-23T14:46:55.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.497+0000 7fcc95a00700 1 -- 172.21.15.83:0/115625823 shutdown_connections 2024-04-23T14:46:55.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.497+0000 7fcc95a00700 1 --2- 172.21.15.83:0/115625823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc9010f5a0 0x7fcc90111990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:55.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.497+0000 7fcc95a00700 1 --2- 172.21.15.83:0/115625823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9006a800 0x7fcc9006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:55.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.497+0000 7fcc95a00700 1 --2- 172.21.15.83:0/115625823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc9006b120 0x7fcc9010f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:55.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.497+0000 7fcc95a00700 1 -- 172.21.15.83:0/115625823 >> 172.21.15.83:0/115625823 conn(0x7fcc900789a0 msgr2=0x7fcc90078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.497+0000 7fcc95a00700 1 -- 172.21.15.83:0/115625823 shutdown_connections 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.498+0000 7fcc95a00700 1 -- 172.21.15.83:0/115625823 wait complete. 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.498+0000 7fcc95a00700 1 Processor -- start 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc95a00700 1 -- start start 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc95a00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc9006a800 0x7fcc9019b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc95a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc9006b120 0x7fcc9019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc95a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9010f5a0 0x7fcc9019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc95a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90115010 con 0x7fcc9006b120 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc95a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90114e90 con 0x7fcc9010f5a0 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc95a00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90115190 con 0x7fcc9006a800 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc8e000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc9006b120 0x7fcc9019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc8ea00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc9006a800 0x7fcc9019b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:55.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc8f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9010f5a0 0x7fcc9019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc8ea00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc9006a800 0x7fcc9019b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41564/0 (socket says 172.21.15.83:41564) 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc8f400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9010f5a0 0x7fcc9019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33892/0 (socket says 172.21.15.83:33892) 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc8e000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc9006b120 0x7fcc9019ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49192/0 (socket says 172.21.15.83:49192) 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.499+0000 7fcc8ea00700 1 -- 172.21.15.83:0/3499071782 learned_addr learned my addr 172.21.15.83:0/3499071782 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.500+0000 7fcc8f400700 1 -- 172.21.15.83:0/3499071782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc9006a800 msgr2=0x7fcc9019b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.500+0000 7fcc8f400700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc9006a800 0x7fcc9019b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.500+0000 7fcc8f400700 1 -- 172.21.15.83:0/3499071782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc9006b120 msgr2=0x7fcc9019ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.500+0000 7fcc8f400700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc9006b120 0x7fcc9019ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.500+0000 7fcc8f400700 1 -- 172.21.15.83:0/3499071782 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc901a04a0 con 0x7fcc9010f5a0 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.500+0000 7fcc8ea00700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc9006a800 0x7fcc9019b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.500+0000 7fcc8f400700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9010f5a0 0x7fcc9019fd60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc9006a3c0 tx=0x7fcc88004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.501+0000 7fcc87400700 1 -- 172.21.15.83:0/3499071782 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88021070 con 0x7fcc9010f5a0 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.501+0000 7fcc87400700 1 -- 172.21.15.83:0/3499071782 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc88007b80 con 0x7fcc9010f5a0 2024-04-23T14:46:55.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.501+0000 7fcc87400700 1 -- 172.21.15.83:0/3499071782 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88005040 con 0x7fcc9010f5a0 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.501+0000 7fcc95a00700 1 -- 172.21.15.83:0/3499071782 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc901a0730 con 0x7fcc9010f5a0 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.501+0000 7fcc95a00700 1 -- 172.21.15.83:0/3499071782 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc901a8010 con 0x7fcc9010f5a0 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.502+0000 7fcc95a00700 1 -- 172.21.15.83:0/3499071782 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc90063240 con 0x7fcc9010f5a0 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.505+0000 7fcc87400700 1 -- 172.21.15.83:0/3499071782 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcc88024070 con 0x7fcc9010f5a0 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.506+0000 7fcc87400700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcc740a3cd0 0x7fcc740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.506+0000 7fcc87400700 1 -- 172.21.15.83:0/3499071782 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fcc880d1ba0 con 0x7fcc9010f5a0 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.506+0000 7fcc8ea00700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcc740a3cd0 0x7fcc740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.506+0000 7fcc87400700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956] conn(0x7fcc740ad4b0 0x7fcc740af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.506+0000 7fcc87400700 1 -- 172.21.15.83:0/3499071782 --> [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcc88010ec0 con 0x7fcc740ad4b0 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.506+0000 7fcc87400700 1 -- 172.21.15.83:0/3499071782 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fcc880d1f50 con 0x7fcc9010f5a0 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.507+0000 7fcc8e000700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956] conn(0x7fcc740ad4b0 0x7fcc740af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.507+0000 7fcc8ea00700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcc740a3cd0 0x7fcc740a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc80011550 tx=0x7fcc80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:55.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.507+0000 7fcc8e000700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956] conn(0x7fcc740ad4b0 0x7fcc740af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:46:55.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.508+0000 7fcc87400700 1 -- 172.21.15.83:0/3499071782 <== osd.9 v2:172.21.15.83:6826/3165262956 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcc88010ec0 con 0x7fcc740ad4b0 2024-04-23T14:46:55.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.526+0000 7fcc95a00700 1 -- 172.21.15.83:0/3499071782 --> [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc900636f0 con 0x7fcc740ad4b0 2024-04-23T14:46:55.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.527+0000 7fcc87400700 1 -- 172.21.15.83:0/3499071782 <== osd.9 v2:172.21.15.83:6826/3165262956 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcc900636f0 con 0x7fcc740ad4b0 2024-04-23T14:46:55.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.527+0000 7fcc95a00700 1 -- 172.21.15.83:0/3499071782 >> [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956] conn(0x7fcc740ad4b0 msgr2=0x7fcc740af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:55.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.527+0000 7fcc95a00700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956] conn(0x7fcc740ad4b0 0x7fcc740af8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:46:55.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.527+0000 7fcc95a00700 1 -- 172.21.15.83:0/3499071782 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcc740a3cd0 msgr2=0x7fcc740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:55.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.527+0000 7fcc95a00700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcc740a3cd0 0x7fcc740a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcc80011550 tx=0x7fcc80009260 comp rx=0 tx=0).stop 2024-04-23T14:46:55.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.527+0000 7fcc95a00700 1 -- 172.21.15.83:0/3499071782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9010f5a0 msgr2=0x7fcc9019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:46:55.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.527+0000 7fcc95a00700 1 --2- 172.21.15.83:0/3499071782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc9010f5a0 0x7fcc9019fd60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc9006a3c0 tx=0x7fcc88004d00 comp rx=0 tx=0).stop 2024-04-23T14:46:55.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.528+0000 7fcc8f400700 1 -- 172.21.15.83:0/3499071782 reap_dead start 2024-04-23T14:46:55.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.528+0000 7fcc95a00700 1 -- 172.21.15.83:0/3499071782 shutdown_connections 2024-04-23T14:46:55.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.528+0000 7fcc95a00700 1 -- 172.21.15.83:0/3499071782 >> 172.21.15.83:0/3499071782 conn(0x7fcc900789a0 msgr2=0x7fcc901109f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:46:55.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.528+0000 7fcc95a00700 1 -- 172.21.15.83:0/3499071782 shutdown_connections 2024-04-23T14:46:55.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:46:53.528+0000 7fcc95a00700 1 -- 172.21.15.83:0/3499071782 wait complete. 2024-04-23T14:46:55.987 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:55.989 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:56.004 INFO:teuthology.orchestra.run.smithi083.stdout:292057776164 2024-04-23T14:46:56.005 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-23T14:46:56.005 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:56.016 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:56.017 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:56.169 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:56.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:56 smithi083 ceph-mon[27867]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T14:46:56.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:56 smithi203 ceph-mon[32800]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T14:46:56.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:56 smithi173 ceph-mon[32443]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T14:46:57.095 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:57.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-23T14:46:57.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-23T14:46:57.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-04-23T14:46:57.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-23T14:46:57.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-04-23T14:46:57.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-23T14:46:57.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-04-23T14:46:57.389 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-04-23T14:46:57.390 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:57 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-23T14:46:57.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-23T14:46:57.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-23T14:46:57.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-04-23T14:46:57.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-23T14:46:57.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-04-23T14:46:57.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-23T14:46:57.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-04-23T14:46:57.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-04-23T14:46:57.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:57 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-23T14:46:57.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-23T14:46:57.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-23T14:46:57.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]: dispatch 2024-04-23T14:46:57.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-23T14:46:57.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-04-23T14:46:57.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-23T14:46:57.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-04-23T14:46:57.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-04-23T14:46:57.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:57 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-23T14:46:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:58 smithi083 ceph-mon[27867]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:46:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-23T14:46:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-23T14:46:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-04-23T14:46:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-23T14:46:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-04-23T14:46:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-23T14:46:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-04-23T14:46:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-04-23T14:46:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:58 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-23T14:46:58.553 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:58 smithi083 ceph-mon[27867]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T14:46:58.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:58 smithi203 ceph-mon[32800]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:46:58.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:58 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-23T14:46:58.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:58 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-23T14:46:58.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:58 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-04-23T14:46:58.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:58 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-23T14:46:58.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:58 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-04-23T14:46:58.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:58 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-23T14:46:58.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:58 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-04-23T14:46:58.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:58 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-04-23T14:46:58.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:58 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-23T14:46:58.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:58 smithi203 ceph-mon[32800]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T14:46:58.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:58 smithi173 ceph-mon[32443]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:46:58.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:58 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-23T14:46:58.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:58 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-23T14:46:58.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:58 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2024-04-23T14:46:58.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:58 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-23T14:46:58.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:58 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-04-23T14:46:58.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:58 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-23T14:46:58.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:58 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-04-23T14:46:58.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:58 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-04-23T14:46:58.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:58 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-23T14:46:58.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:58 smithi173 ceph-mon[32443]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T14:46:59.216 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:46:59.503 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:46:59 smithi083 ceph-mon[27867]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T14:46:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:46:59 smithi203 ceph-mon[32800]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T14:46:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:46:59 smithi173 ceph-mon[32443]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T14:46:59.970 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:47:00.671 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:00 smithi083 ceph-mon[27867]: pgmap v349: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-23T14:47:00.671 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:00 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:47:00.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:00 smithi203 ceph-mon[32800]: pgmap v349: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-23T14:47:00.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:00 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:47:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:00 smithi173 ceph-mon[32443]: pgmap v349: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-23T14:47:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:00 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:47:01.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:01 smithi203 ceph-mon[32800]: pgmap v350: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:47:01.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:01 smithi173 ceph-mon[32443]: pgmap v350: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:47:01.709 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:01 smithi083 ceph-mon[27867]: pgmap v350: 97 pgs: 5 peering, 92 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:47:02.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:02 smithi203 ceph-mon[32800]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-23T14:47:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:02.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:02 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:02.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:02 smithi083 ceph-mon[27867]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-23T14:47:02.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:02.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:02.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:02.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:02 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:02.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:02 smithi173 ceph-mon[32443]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-23T14:47:02.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:02.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:02.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:02.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:02 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:03.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:03 smithi083 ceph-mon[27867]: pgmap v351: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s; 129 B/s, 0 objects/s recovering 2024-04-23T14:47:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:03 smithi203 ceph-mon[32800]: pgmap v351: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s; 129 B/s, 0 objects/s recovering 2024-04-23T14:47:03.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:03 smithi173 ceph-mon[32443]: pgmap v351: 97 pgs: 2 peering, 95 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 5 op/s; 129 B/s, 0 objects/s recovering 2024-04-23T14:47:04.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:04 smithi203 ceph-mon[32800]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-23T14:47:04.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:04 smithi203 ceph-mon[32800]: Cluster is now healthy 2024-04-23T14:47:04.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:04 smithi083 ceph-mon[27867]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-23T14:47:04.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:04 smithi083 ceph-mon[27867]: Cluster is now healthy 2024-04-23T14:47:04.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:04 smithi173 ceph-mon[32443]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-23T14:47:04.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:04 smithi173 ceph-mon[32443]: Cluster is now healthy 2024-04-23T14:47:05.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:05 smithi203 ceph-mon[32800]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s; 129 B/s, 0 objects/s recovering 2024-04-23T14:47:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:05 smithi083 ceph-mon[27867]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s; 129 B/s, 0 objects/s recovering 2024-04-23T14:47:05.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:05 smithi173 ceph-mon[32443]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1 op/s; 129 B/s, 0 objects/s recovering 2024-04-23T14:47:05.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.956+0000 7f7ba0000700 1 -- 172.21.15.83:0/977294513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b98104f70 msgr2=0x7f7b98105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:05.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.956+0000 7f7ba0000700 1 --2- 172.21.15.83:0/977294513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b98104f70 0x7f7b98105350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0099c0 tx=0x7f7b8c009cd0 comp rx=0 tx=0).stop 2024-04-23T14:47:05.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.956+0000 7f7ba0000700 1 -- 172.21.15.83:0/977294513 shutdown_connections 2024-04-23T14:47:05.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.956+0000 7f7ba0000700 1 --2- 172.21.15.83:0/977294513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9810a050 0x7f7b98111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:05.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.956+0000 7f7ba0000700 1 --2- 172.21.15.83:0/977294513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b98104f70 0x7f7b98105350 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:05.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.956+0000 7f7ba0000700 1 --2- 172.21.15.83:0/977294513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b98105920 0x7f7b98109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:05.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.956+0000 7f7ba0000700 1 -- 172.21.15.83:0/977294513 >> 172.21.15.83:0/977294513 conn(0x7f7b98100800 msgr2=0x7f7b98102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:05.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.956+0000 7f7ba0000700 1 -- 172.21.15.83:0/977294513 shutdown_connections 2024-04-23T14:47:05.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.956+0000 7f7ba0000700 1 -- 172.21.15.83:0/977294513 wait complete. 2024-04-23T14:47:05.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.957+0000 7f7ba0000700 1 Processor -- start 2024-04-23T14:47:05.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.957+0000 7f7ba0000700 1 -- start start 2024-04-23T14:47:05.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7ba0000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b98104f70 0x7f7b9819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:05.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7ba0000700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b98105920 0x7f7b9819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7ba0000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9810a050 0x7f7b9819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7ba0000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b981150a0 con 0x7f7b98104f70 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7ba0000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7b98114f20 con 0x7f7b9810a050 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7ba0000700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7b98115220 con 0x7f7b98105920 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7b9d600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b98104f70 0x7f7b9819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7b9cc00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b98105920 0x7f7b9819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7b9d600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b98104f70 0x7f7b9819b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47862/0 (socket says 172.21.15.83:47862) 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7b9e000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9810a050 0x7f7b9819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7b9d600700 1 -- 172.21.15.83:0/3429051386 learned_addr learned my addr 172.21.15.83:0/3429051386 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7b9e000700 1 -- 172.21.15.83:0/3429051386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b98105920 msgr2=0x7f7b9819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7b9e000700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b98105920 0x7f7b9819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7b9e000700 1 -- 172.21.15.83:0/3429051386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b98104f70 msgr2=0x7f7b9819b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:05.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.958+0000 7f7b9e000700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b98104f70 0x7f7b9819b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:05.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.959+0000 7f7b9e000700 1 -- 172.21.15.83:0/3429051386 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b981a04d0 con 0x7f7b9810a050 2024-04-23T14:47:05.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.959+0000 7f7b9d600700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b98104f70 0x7f7b9819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:05.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.959+0000 7f7b9e000700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9810a050 0x7f7b9819fd90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000efc0 tx=0x7f7b9000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:05.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.959+0000 7f7b96000700 1 -- 172.21.15.83:0/3429051386 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b900092e0 con 0x7f7b9810a050 2024-04-23T14:47:05.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.959+0000 7f7b96000700 1 -- 172.21.15.83:0/3429051386 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b90004bb0 con 0x7f7b9810a050 2024-04-23T14:47:05.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.959+0000 7f7ba0000700 1 -- 172.21.15.83:0/3429051386 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b981a0760 con 0x7f7b9810a050 2024-04-23T14:47:05.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.960+0000 7f7b96000700 1 -- 172.21.15.83:0/3429051386 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90004280 con 0x7f7b9810a050 2024-04-23T14:47:05.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.960+0000 7f7ba0000700 1 -- 172.21.15.83:0/3429051386 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b981a8040 con 0x7f7b9810a050 2024-04-23T14:47:05.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.961+0000 7f7ba0000700 1 -- 172.21.15.83:0/3429051386 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b980697e0 con 0x7f7b9810a050 2024-04-23T14:47:05.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.963+0000 7f7b96000700 1 -- 172.21.15.83:0/3429051386 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7b9001b030 con 0x7f7b9810a050 2024-04-23T14:47:05.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.964+0000 7f7b96000700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7b800a3d50 0x7f7b800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:05.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.965+0000 7f7b96000700 1 -- 172.21.15.83:0/3429051386 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f7b900d0980 con 0x7f7b9810a050 2024-04-23T14:47:05.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.965+0000 7f7b9d600700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7b800a3d50 0x7f7b800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:05.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.965+0000 7f7b9d600700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7b800a3d50 0x7f7b800a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009990 tx=0x7f7b8c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:05.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:05.967+0000 7f7b96000700 1 -- 172.21.15.83:0/3429051386 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b90022020 con 0x7f7b9810a050 2024-04-23T14:47:06.223 INFO:teuthology.orchestra.run.smithi083.stdout:257698037805 2024-04-23T14:47:06.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.221+0000 7f7ba0000700 1 -- 172.21.15.83:0/3429051386 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f7b9802dfe0 con 0x7f7b9810a050 2024-04-23T14:47:06.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.222+0000 7f7b96000700 1 -- 172.21.15.83:0/3429051386 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7b90010040 con 0x7f7b9810a050 2024-04-23T14:47:06.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.225+0000 7f7ba0000700 1 -- 172.21.15.83:0/3429051386 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7b800a3d50 msgr2=0x7f7b800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:06.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.225+0000 7f7ba0000700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7b800a3d50 0x7f7b800a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009990 tx=0x7f7b8c005ab0 comp rx=0 tx=0).stop 2024-04-23T14:47:06.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.225+0000 7f7ba0000700 1 -- 172.21.15.83:0/3429051386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9810a050 msgr2=0x7f7b9819fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:06.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.225+0000 7f7ba0000700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9810a050 0x7f7b9819fd90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000efc0 tx=0x7f7b9000c3a0 comp rx=0 tx=0).stop 2024-04-23T14:47:06.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.225+0000 7f7ba0000700 1 -- 172.21.15.83:0/3429051386 shutdown_connections 2024-04-23T14:47:06.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.225+0000 7f7ba0000700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7b800a3d50 0x7f7b800a6210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:06.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.225+0000 7f7ba0000700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9810a050 0x7f7b9819fd90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:06.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.225+0000 7f7ba0000700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b98104f70 0x7f7b9819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:06.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.225+0000 7f7ba0000700 1 --2- 172.21.15.83:0/3429051386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b98105920 0x7f7b9819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:06.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.225+0000 7f7ba0000700 1 -- 172.21.15.83:0/3429051386 >> 172.21.15.83:0/3429051386 conn(0x7f7b98100800 msgr2=0x7f7b98101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:06.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.225+0000 7f7ba0000700 1 -- 172.21.15.83:0/3429051386 shutdown_connections 2024-04-23T14:47:06.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:06.225+0000 7f7ba0000700 1 -- 172.21.15.83:0/3429051386 wait complete. 2024-04-23T14:47:06.687 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:06 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3429051386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T14:47:06.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:06 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3429051386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T14:47:06.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:06 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3429051386' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T14:47:07.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:07 smithi083 ceph-mon[27867]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s; 104 B/s, 0 objects/s recovering 2024-04-23T14:47:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:07 smithi203 ceph-mon[32800]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s; 104 B/s, 0 objects/s recovering 2024-04-23T14:47:07.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:07 smithi173 ceph-mon[32443]: pgmap v353: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4 op/s; 104 B/s, 0 objects/s recovering 2024-04-23T14:47:09.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.371+0000 7f9f1aa00700 1 -- 172.21.15.83:0/1560555982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f141022f0 msgr2=0x7f9f141067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:09.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.371+0000 7f9f1aa00700 1 --2- 172.21.15.83:0/1560555982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f141022f0 0x7f9f141067c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009990 tx=0x7f9f00009ca0 comp rx=0 tx=0).stop 2024-04-23T14:47:09.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.371+0000 7f9f1aa00700 1 -- 172.21.15.83:0/1560555982 shutdown_connections 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.371+0000 7f9f1aa00700 1 --2- 172.21.15.83:0/1560555982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f14106de0 0x7f9f14107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.371+0000 7f9f1aa00700 1 --2- 172.21.15.83:0/1560555982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f141019d0 0x7f9f14101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.371+0000 7f9f1aa00700 1 --2- 172.21.15.83:0/1560555982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f141022f0 0x7f9f141067c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.371+0000 7f9f1aa00700 1 -- 172.21.15.83:0/1560555982 >> 172.21.15.83:0/1560555982 conn(0x7f9f140fd410 msgr2=0x7f9f140ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.372+0000 7f9f1aa00700 1 -- 172.21.15.83:0/1560555982 shutdown_connections 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.372+0000 7f9f1aa00700 1 -- 172.21.15.83:0/1560555982 wait complete. 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.373+0000 7f9f1aa00700 1 Processor -- start 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.373+0000 7f9f1aa00700 1 -- start start 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.374+0000 7f9f1aa00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f141019d0 0x7f9f141a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.374+0000 7f9f1aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f141022f0 0x7f9f141a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.374+0000 7f9f1aa00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f14106de0 0x7f9f1419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.374+0000 7f9f1aa00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f141193b0 con 0x7f9f141019d0 2024-04-23T14:47:09.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.374+0000 7f9f1aa00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f14119230 con 0x7f9f141022f0 2024-04-23T14:47:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.374+0000 7f9f1aa00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f14119530 con 0x7f9f14106de0 2024-04-23T14:47:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.374+0000 7f9f19600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f141019d0 0x7f9f141a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.374+0000 7f9f19600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f141019d0 0x7f9f141a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47910/0 (socket says 172.21.15.83:47910) 2024-04-23T14:47:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.374+0000 7f9f19600700 1 -- 172.21.15.83:0/2156120761 learned_addr learned my addr 172.21.15.83:0/2156120761 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:09.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.374+0000 7f9f1a000700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f14106de0 0x7f9f1419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:09.646 INFO:teuthology.orchestra.run.smithi083.stdout:223338299444 2024-04-23T14:47:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f19600700 1 -- 172.21.15.83:0/2156120761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f14106de0 msgr2=0x7f9f1419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f19600700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f14106de0 0x7f9f1419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f19600700 1 -- 172.21.15.83:0/2156120761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f141022f0 msgr2=0x7f9f141a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f18c00700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f141022f0 0x7f9f141a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f19600700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f141022f0 0x7f9f141a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f19600700 1 -- 172.21.15.83:0/2156120761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1419ff10 con 0x7f9f141019d0 2024-04-23T14:47:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f18c00700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f141022f0 0x7f9f141a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:47:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f19600700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f141019d0 0x7f9f141a5560 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800b530 tx=0x7f9f0800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f12000700 1 -- 172.21.15.83:0/2156120761 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08015020 con 0x7f9f141019d0 2024-04-23T14:47:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f12000700 1 -- 172.21.15.83:0/2156120761 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f08011d10 con 0x7f9f141019d0 2024-04-23T14:47:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f1aa00700 1 -- 172.21.15.83:0/2156120761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f141a0200 con 0x7f9f141019d0 2024-04-23T14:47:09.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f12000700 1 -- 172.21.15.83:0/2156120761 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08010e50 con 0x7f9f141019d0 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.375+0000 7f9f1aa00700 1 -- 172.21.15.83:0/2156120761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f141ac4e0 con 0x7f9f141019d0 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.376+0000 7f9f1aa00700 1 -- 172.21.15.83:0/2156120761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f1406b0b0 con 0x7f9f141019d0 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.380+0000 7f9f12000700 1 -- 172.21.15.83:0/2156120761 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9f08018020 con 0x7f9f141019d0 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.382+0000 7f9f12000700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9efc0a3e70 0x7f9efc0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.382+0000 7f9f12000700 1 -- 172.21.15.83:0/2156120761 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f9f080d02e0 con 0x7f9f141019d0 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.382+0000 7f9f18c00700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9efc0a3e70 0x7f9efc0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.383+0000 7f9f18c00700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9efc0a3e70 0x7f9efc0a6330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9f00006010 tx=0x7f9f00005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.383+0000 7f9f12000700 1 -- 172.21.15.83:0/2156120761 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f0801f020 con 0x7f9f141019d0 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.618+0000 7f9f1aa00700 1 -- 172.21.15.83:0/2156120761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9f141a0b60 con 0x7f9f141019d0 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.619+0000 7f9f12000700 1 -- 172.21.15.83:0/2156120761 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9f0809cc30 con 0x7f9f141019d0 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.621+0000 7f9f1aa00700 1 -- 172.21.15.83:0/2156120761 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9efc0a3e70 msgr2=0x7f9efc0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.622+0000 7f9f1aa00700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9efc0a3e70 0x7f9efc0a6330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9f00006010 tx=0x7f9f00005e80 comp rx=0 tx=0).stop 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.622+0000 7f9f1aa00700 1 -- 172.21.15.83:0/2156120761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f141019d0 msgr2=0x7f9f141a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:09.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.622+0000 7f9f1aa00700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f141019d0 0x7f9f141a5560 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800b530 tx=0x7f9f0800b8f0 comp rx=0 tx=0).stop 2024-04-23T14:47:09.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.622+0000 7f9f1aa00700 1 -- 172.21.15.83:0/2156120761 shutdown_connections 2024-04-23T14:47:09.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.622+0000 7f9f1aa00700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9efc0a3e70 0x7f9efc0a6330 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:09.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.622+0000 7f9f1aa00700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f14106de0 0x7f9f1419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:09.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.622+0000 7f9f1aa00700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f141019d0 0x7f9f141a5560 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:09.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.622+0000 7f9f1aa00700 1 --2- 172.21.15.83:0/2156120761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f141022f0 0x7f9f141a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:09.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.622+0000 7f9f1aa00700 1 -- 172.21.15.83:0/2156120761 >> 172.21.15.83:0/2156120761 conn(0x7f9f140fd410 msgr2=0x7f9f14104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:09.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.622+0000 7f9f1aa00700 1 -- 172.21.15.83:0/2156120761 shutdown_connections 2024-04-23T14:47:09.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.622+0000 7f9f1aa00700 1 -- 172.21.15.83:0/2156120761 wait complete. 2024-04-23T14:47:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.262+0000 7f2819a00700 1 -- 172.21.15.83:0/2911858766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281406ad40 msgr2=0x7f281406b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.262+0000 7f2819a00700 1 --2- 172.21.15.83:0/2911858766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281406ad40 0x7f281406b1a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2800009990 tx=0x7f2800009ca0 comp rx=0 tx=0).stop 2024-04-23T14:47:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.263+0000 7f2819a00700 1 -- 172.21.15.83:0/2911858766 shutdown_connections 2024-04-23T14:47:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.263+0000 7f2819a00700 1 --2- 172.21.15.83:0/2911858766 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2814105530 0x7f28141059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.263+0000 7f2819a00700 1 --2- 172.21.15.83:0/2911858766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2814107a20 0x7f281406a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.263+0000 7f2819a00700 1 --2- 172.21.15.83:0/2911858766 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f281406ad40 0x7f281406b1a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.263+0000 7f2819a00700 1 -- 172.21.15.83:0/2911858766 >> 172.21.15.83:0/2911858766 conn(0x7f28140789a0 msgr2=0x7f2814078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.263+0000 7f2819a00700 1 -- 172.21.15.83:0/2911858766 shutdown_connections 2024-04-23T14:47:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.264+0000 7f2819a00700 1 -- 172.21.15.83:0/2911858766 wait complete. 2024-04-23T14:47:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.265+0000 7f2819a00700 1 Processor -- start 2024-04-23T14:47:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.265+0000 7f2819a00700 1 -- start start 2024-04-23T14:47:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.265+0000 7f2819a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281406ad40 0x7f28141a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.265+0000 7f2819a00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2814105530 0x7f28141a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.265+0000 7f2819a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2814107a20 0x7f281419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.265+0000 7f2819a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2814119410 con 0x7f281406ad40 2024-04-23T14:47:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.265+0000 7f2819a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2814119290 con 0x7f2814107a20 2024-04-23T14:47:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.265+0000 7f2819a00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2814119590 con 0x7f2814105530 2024-04-23T14:47:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.266+0000 7f2812a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281406ad40 0x7f28141a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.266+0000 7f2812a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281406ad40 0x7f28141a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47892/0 (socket says 172.21.15.83:47892) 2024-04-23T14:47:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.266+0000 7f2812a00700 1 -- 172.21.15.83:0/3872011595 learned_addr learned my addr 172.21.15.83:0/3872011595 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.266+0000 7f2813400700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2814107a20 0x7f281419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.266+0000 7f2812000700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2814105530 0x7f28141a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.266+0000 7f2813400700 1 -- 172.21.15.83:0/3872011595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2814105530 msgr2=0x7f28141a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.266+0000 7f2813400700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2814105530 0x7f28141a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.266+0000 7f2813400700 1 -- 172.21.15.83:0/3872011595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281406ad40 msgr2=0x7f28141a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.266+0000 7f2813400700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281406ad40 0x7f28141a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.266+0000 7f2813400700 1 -- 172.21.15.83:0/3872011595 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281419fff0 con 0x7f2814107a20 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.266+0000 7f2812a00700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281406ad40 0x7f28141a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.267+0000 7f2813400700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2814107a20 0x7f281419f710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f280c00efc0 tx=0x7f280c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.267+0000 7f280b400700 1 -- 172.21.15.83:0/3872011595 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c0092e0 con 0x7f2814107a20 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.267+0000 7f280b400700 1 -- 172.21.15.83:0/3872011595 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280c004bb0 con 0x7f2814107a20 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.267+0000 7f280b400700 1 -- 172.21.15.83:0/3872011595 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c004280 con 0x7f2814107a20 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.267+0000 7f2819a00700 1 -- 172.21.15.83:0/3872011595 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28141a0280 con 0x7f2814107a20 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.267+0000 7f2819a00700 1 -- 172.21.15.83:0/3872011595 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28141ac3f0 con 0x7f2814107a20 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.268+0000 7f2819a00700 1 -- 172.21.15.83:0/3872011595 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2814102010 con 0x7f2814107a20 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.271+0000 7f280b400700 1 -- 172.21.15.83:0/3872011595 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f280c01b030 con 0x7f2814107a20 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.272+0000 7f280b400700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f27f80a3d50 0x7f27f80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.272+0000 7f280b400700 1 -- 172.21.15.83:0/3872011595 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f280c0d0b20 con 0x7f2814107a20 2024-04-23T14:47:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.272+0000 7f2812a00700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f27f80a3d50 0x7f27f80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.273+0000 7f2812a00700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f27f80a3d50 0x7f27f80a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2804011550 tx=0x7f2804009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.275+0000 7f280b400700 1 -- 172.21.15.83:0/3872011595 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f280c022020 con 0x7f2814107a20 2024-04-23T14:47:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.523+0000 7f2819a00700 1 -- 172.21.15.83:0/3872011595 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f28141ac680 con 0x7f2814107a20 2024-04-23T14:47:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.523+0000 7f280b400700 1 -- 172.21.15.83:0/3872011595 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f280c0bbe20 con 0x7f2814107a20 2024-04-23T14:47:10.136 INFO:teuthology.orchestra.run.smithi083.stdout:313532612640 2024-04-23T14:47:10.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.526+0000 7f2819a00700 1 -- 172.21.15.83:0/3872011595 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f27f80a3d50 msgr2=0x7f27f80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:10.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.526+0000 7f2819a00700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f27f80a3d50 0x7f27f80a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2804011550 tx=0x7f2804009260 comp rx=0 tx=0).stop 2024-04-23T14:47:10.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.526+0000 7f2819a00700 1 -- 172.21.15.83:0/3872011595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2814107a20 msgr2=0x7f281419f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:10.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.526+0000 7f2819a00700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2814107a20 0x7f281419f710 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f280c00efc0 tx=0x7f280c00c3a0 comp rx=0 tx=0).stop 2024-04-23T14:47:10.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.526+0000 7f2819a00700 1 -- 172.21.15.83:0/3872011595 shutdown_connections 2024-04-23T14:47:10.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.526+0000 7f2819a00700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f27f80a3d50 0x7f27f80a6210 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:10.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.526+0000 7f2819a00700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2814107a20 0x7f281419f710 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:10.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.526+0000 7f2819a00700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f281406ad40 0x7f28141a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:10.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.526+0000 7f2819a00700 1 --2- 172.21.15.83:0/3872011595 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2814105530 0x7f28141a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:10.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.526+0000 7f2819a00700 1 -- 172.21.15.83:0/3872011595 >> 172.21.15.83:0/3872011595 conn(0x7f28140789a0 msgr2=0x7f2814104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:10.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.527+0000 7f2819a00700 1 -- 172.21.15.83:0/3872011595 shutdown_connections 2024-04-23T14:47:10.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:09.527+0000 7f2819a00700 1 -- 172.21.15.83:0/3872011595 wait complete. 2024-04-23T14:47:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:10 smithi173 ceph-mon[32443]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s; 95 B/s, 0 objects/s recovering 2024-04-23T14:47:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:10 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3872011595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T14:47:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:10 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2156120761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T14:47:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:10 smithi203 ceph-mon[32800]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s; 95 B/s, 0 objects/s recovering 2024-04-23T14:47:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:10 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3872011595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T14:47:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:10 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2156120761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T14:47:10.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:10 smithi083 ceph-mon[27867]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s; 95 B/s, 0 objects/s recovering 2024-04-23T14:47:10.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:10 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3872011595' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T14:47:10.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:10 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2156120761' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T14:47:11.303 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037800 got 257698037805 for osd.8 2024-04-23T14:47:11.303 DEBUG:teuthology.parallel:result is None 2024-04-23T14:47:12.501 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:12 smithi083 ceph-mon[27867]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-23T14:47:12.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:12 smithi173 ceph-mon[32443]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-23T14:47:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:12 smithi203 ceph-mon[32800]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-23T14:47:13.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.401+0000 7fcc7f200700 1 -- 172.21.15.83:0/3433216074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc78074d20 msgr2=0x7fcc78075100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:13.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.401+0000 7fcc7f200700 1 --2- 172.21.15.83:0/3433216074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc78074d20 0x7fcc78075100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009990 tx=0x7fcc68009ca0 comp rx=0 tx=0).stop 2024-04-23T14:47:13.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.401+0000 7fcc7f200700 1 -- 172.21.15.83:0/3433216074 shutdown_connections 2024-04-23T14:47:13.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.401+0000 7fcc7f200700 1 --2- 172.21.15.83:0/3433216074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc7810f550 0x7fcc78111940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:13.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.401+0000 7fcc7f200700 1 --2- 172.21.15.83:0/3433216074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc78074d20 0x7fcc78075100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:13.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.401+0000 7fcc7f200700 1 --2- 172.21.15.83:0/3433216074 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc78075640 0x7fcc7810f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.401+0000 7fcc7f200700 1 -- 172.21.15.83:0/3433216074 >> 172.21.15.83:0/3433216074 conn(0x7fcc78079c50 msgr2=0x7fcc7807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.401+0000 7fcc7f200700 1 -- 172.21.15.83:0/3433216074 shutdown_connections 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.401+0000 7fcc7f200700 1 -- 172.21.15.83:0/3433216074 wait complete. 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.402+0000 7fcc7f200700 1 Processor -- start 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.403+0000 7fcc7f200700 1 -- start start 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.403+0000 7fcc7f200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc78074d20 0x7fcc7819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.403+0000 7fcc7f200700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc78075640 0x7fcc7819b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.403+0000 7fcc7f200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc7810f550 0x7fcc7819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.403+0000 7fcc7f200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78115090 con 0x7fcc78074d20 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.403+0000 7fcc7f200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78114f10 con 0x7fcc7810f550 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.403+0000 7fcc7f200700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78115210 con 0x7fcc78075640 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.403+0000 7fcc77e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc78074d20 0x7fcc7819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.403+0000 7fcc77e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc78074d20 0x7fcc7819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47928/0 (socket says 172.21.15.83:47928) 2024-04-23T14:47:13.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.403+0000 7fcc77e00700 1 -- 172.21.15.83:0/1487504321 learned_addr learned my addr 172.21.15.83:0/1487504321 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.404+0000 7fcc77400700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc78075640 0x7fcc7819b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.404+0000 7fcc7d200700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc7810f550 0x7fcc7819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.404+0000 7fcc77e00700 1 -- 172.21.15.83:0/1487504321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc78075640 msgr2=0x7fcc7819b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.404+0000 7fcc77e00700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc78075640 0x7fcc7819b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.404+0000 7fcc77e00700 1 -- 172.21.15.83:0/1487504321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc7810f550 msgr2=0x7fcc7819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.404+0000 7fcc77e00700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc7810f550 0x7fcc7819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.404+0000 7fcc77e00700 1 -- 172.21.15.83:0/1487504321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc781a0420 con 0x7fcc78074d20 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.404+0000 7fcc7d200700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc7810f550 0x7fcc7819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.404+0000 7fcc77e00700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc78074d20 0x7fcc7819b460 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcc68000c00 tx=0x7fcc68003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.404+0000 7fcc74c00700 1 -- 172.21.15.83:0/1487504321 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68027070 con 0x7fcc78074d20 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.404+0000 7fcc74c00700 1 -- 172.21.15.83:0/1487504321 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc68005690 con 0x7fcc78074d20 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.404+0000 7fcc74c00700 1 -- 172.21.15.83:0/1487504321 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68022c60 con 0x7fcc78074d20 2024-04-23T14:47:13.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.405+0000 7fcc7f200700 1 -- 172.21.15.83:0/1487504321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc781a06b0 con 0x7fcc78074d20 2024-04-23T14:47:13.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.405+0000 7fcc7f200700 1 -- 172.21.15.83:0/1487504321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc781a7f90 con 0x7fcc78074d20 2024-04-23T14:47:13.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.406+0000 7fcc7f200700 1 -- 172.21.15.83:0/1487504321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc780763e0 con 0x7fcc78074d20 2024-04-23T14:47:13.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.408+0000 7fcc74c00700 1 -- 172.21.15.83:0/1487504321 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcc6802a070 con 0x7fcc78074d20 2024-04-23T14:47:13.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.409+0000 7fcc74c00700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcc5c0a3cd0 0x7fcc5c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:13.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.409+0000 7fcc74c00700 1 -- 172.21.15.83:0/1487504321 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fcc68022e00 con 0x7fcc78074d20 2024-04-23T14:47:13.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.409+0000 7fcc77400700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcc5c0a3cd0 0x7fcc5c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:13.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.410+0000 7fcc77400700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcc5c0a3cd0 0x7fcc5c0a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc78111030 tx=0x7fcc6400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:13.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.414+0000 7fcc74c00700 1 -- 172.21.15.83:0/1487504321 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc6805eff0 con 0x7fcc78074d20 2024-04-23T14:47:13.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.662+0000 7fcc7f200700 1 -- 172.21.15.83:0/1487504321 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcc781a8160 con 0x7fcc78074d20 2024-04-23T14:47:13.472 INFO:teuthology.orchestra.run.smithi083.stdout:107374182474 2024-04-23T14:47:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.663+0000 7fcc74c00700 1 -- 172.21.15.83:0/1487504321 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcc68013070 con 0x7fcc78074d20 2024-04-23T14:47:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.665+0000 7fcc7f200700 1 -- 172.21.15.83:0/1487504321 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcc5c0a3cd0 msgr2=0x7fcc5c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.665+0000 7fcc7f200700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcc5c0a3cd0 0x7fcc5c0a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc78111030 tx=0x7fcc6400b410 comp rx=0 tx=0).stop 2024-04-23T14:47:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.665+0000 7fcc7f200700 1 -- 172.21.15.83:0/1487504321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc78074d20 msgr2=0x7fcc7819b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.665+0000 7fcc7f200700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc78074d20 0x7fcc7819b460 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcc68000c00 tx=0x7fcc68003890 comp rx=0 tx=0).stop 2024-04-23T14:47:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.666+0000 7fcc7f200700 1 -- 172.21.15.83:0/1487504321 shutdown_connections 2024-04-23T14:47:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.666+0000 7fcc7f200700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcc5c0a3cd0 0x7fcc5c0a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.666+0000 7fcc7f200700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc7810f550 0x7fcc7819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:13.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.666+0000 7fcc7f200700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc78074d20 0x7fcc7819b460 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.666+0000 7fcc7f200700 1 --2- 172.21.15.83:0/1487504321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc78075640 0x7fcc7819b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.666+0000 7fcc7f200700 1 -- 172.21.15.83:0/1487504321 >> 172.21.15.83:0/1487504321 conn(0x7fcc78079c50 msgr2=0x7fcc780ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.666+0000 7fcc7f200700 1 -- 172.21.15.83:0/1487504321 shutdown_connections 2024-04-23T14:47:13.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:12.666+0000 7fcc7f200700 1 -- 172.21.15.83:0/1487504321 wait complete. 2024-04-23T14:47:13.557 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:13 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1487504321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T14:47:13.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:13 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1487504321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T14:47:13.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:13 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1487504321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T14:47:14.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.973+0000 7fbc6b600700 1 -- 172.21.15.83:0/1219229671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc64108b60 msgr2=0x7fbc64108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:14.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.973+0000 7fbc6b600700 1 --2- 172.21.15.83:0/1219229671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc64108b60 0x7fbc64108f40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009990 tx=0x7fbc58009ca0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.973+0000 7fbc6b600700 1 -- 172.21.15.83:0/1219229671 shutdown_connections 2024-04-23T14:47:14.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.973+0000 7fbc6b600700 1 --2- 172.21.15.83:0/1219229671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc6406ba30 0x7fbc6410f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.973+0000 7fbc6b600700 1 --2- 172.21.15.83:0/1219229671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc64108b60 0x7fbc64108f40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.973+0000 7fbc6b600700 1 --2- 172.21.15.83:0/1219229671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6406b090 0x7fbc6406b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.973+0000 7fbc6b600700 1 -- 172.21.15.83:0/1219229671 >> 172.21.15.83:0/1219229671 conn(0x7fbc64078960 msgr2=0x7fbc64078d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.973+0000 7fbc6b600700 1 -- 172.21.15.83:0/1219229671 shutdown_connections 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.973+0000 7fbc6b600700 1 -- 172.21.15.83:0/1219229671 wait complete. 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.974+0000 7fbc6b600700 1 Processor -- start 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.975+0000 7fbc6b600700 1 -- start start 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.975+0000 7fbc6b600700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc6406b090 0x7fbc6419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.975+0000 7fbc6b600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6406ba30 0x7fbc6419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.975+0000 7fbc6b600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc64108b60 0x7fbc6419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.975+0000 7fbc6b600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64112e90 con 0x7fbc64108b60 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.975+0000 7fbc6b600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64112d10 con 0x7fbc6406ba30 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.975+0000 7fbc6b600700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64113010 con 0x7fbc6406b090 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.975+0000 7fbc69600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc64108b60 0x7fbc6419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.975+0000 7fbc69600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc64108b60 0x7fbc6419fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47948/0 (socket says 172.21.15.83:47948) 2024-04-23T14:47:14.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.975+0000 7fbc69600700 1 -- 172.21.15.83:0/1877950229 learned_addr learned my addr 172.21.15.83:0/1877950229 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.975+0000 7fbc68c00700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc6406b090 0x7fbc6419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.976+0000 7fbc69600700 1 -- 172.21.15.83:0/1877950229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc6406b090 msgr2=0x7fbc6419b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.976+0000 7fbc69600700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc6406b090 0x7fbc6419b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.976+0000 7fbc69600700 1 -- 172.21.15.83:0/1877950229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6406ba30 msgr2=0x7fbc6419bb00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T14:47:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.976+0000 7fbc69600700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6406ba30 0x7fbc6419bb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.976+0000 7fbc69600700 1 -- 172.21.15.83:0/1877950229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc641a05d0 con 0x7fbc64108b60 2024-04-23T14:47:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.976+0000 7fbc69600700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc64108b60 0x7fbc6419fe90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c007ee0 tx=0x7fbc5c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.976+0000 7fbc61600700 1 -- 172.21.15.83:0/1877950229 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c00ddd0 con 0x7fbc64108b60 2024-04-23T14:47:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.976+0000 7fbc6b600700 1 -- 172.21.15.83:0/1877950229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc641a08c0 con 0x7fbc64108b60 2024-04-23T14:47:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.976+0000 7fbc61600700 1 -- 172.21.15.83:0/1877950229 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc5c004020 con 0x7fbc64108b60 2024-04-23T14:47:14.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.976+0000 7fbc61600700 1 -- 172.21.15.83:0/1877950229 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c015660 con 0x7fbc64108b60 2024-04-23T14:47:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.976+0000 7fbc6b600700 1 -- 172.21.15.83:0/1877950229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc641a8140 con 0x7fbc64108b60 2024-04-23T14:47:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.977+0000 7fbc6b600700 1 -- 172.21.15.83:0/1877950229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc640697e0 con 0x7fbc64108b60 2024-04-23T14:47:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.980+0000 7fbc61600700 1 -- 172.21.15.83:0/1877950229 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbc5c0041d0 con 0x7fbc64108b60 2024-04-23T14:47:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.981+0000 7fbc61600700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbc4c0a3cd0 0x7fbc4c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.981+0000 7fbc61600700 1 -- 172.21.15.83:0/1877950229 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fbc5c0d0790 con 0x7fbc64108b60 2024-04-23T14:47:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.982+0000 7fbc68c00700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbc4c0a3cd0 0x7fbc4c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.982+0000 7fbc68c00700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbc4c0a3cd0 0x7fbc4c0a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbc580098e0 tx=0x7fbc58005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:14.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:13.984+0000 7fbc61600700 1 -- 172.21.15.83:0/1877950229 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc5c010070 con 0x7fbc64108b60 2024-04-23T14:47:14.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.218+0000 7fbc6b600700 1 -- 172.21.15.83:0/1877950229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbc641a83b0 con 0x7fbc64108b60 2024-04-23T14:47:14.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.219+0000 7fbc61600700 1 -- 172.21.15.83:0/1877950229 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbc5c09d010 con 0x7fbc64108b60 2024-04-23T14:47:14.221 INFO:teuthology.orchestra.run.smithi083.stdout:77309411410 2024-04-23T14:47:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.223+0000 7fbc6b600700 1 -- 172.21.15.83:0/1877950229 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbc4c0a3cd0 msgr2=0x7fbc4c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.223+0000 7fbc6b600700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbc4c0a3cd0 0x7fbc4c0a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbc580098e0 tx=0x7fbc58005ab0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.223+0000 7fbc6b600700 1 -- 172.21.15.83:0/1877950229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc64108b60 msgr2=0x7fbc6419fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.223+0000 7fbc6b600700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc64108b60 0x7fbc6419fe90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c007ee0 tx=0x7fbc5c00d3b0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.223+0000 7fbc6b600700 1 -- 172.21.15.83:0/1877950229 shutdown_connections 2024-04-23T14:47:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.223+0000 7fbc6b600700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fbc4c0a3cd0 0x7fbc4c0a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.223+0000 7fbc6b600700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc64108b60 0x7fbc6419fe90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.223+0000 7fbc6b600700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc6406b090 0x7fbc6419b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.223+0000 7fbc6b600700 1 --2- 172.21.15.83:0/1877950229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc6406ba30 0x7fbc6419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.223+0000 7fbc6b600700 1 -- 172.21.15.83:0/1877950229 >> 172.21.15.83:0/1877950229 conn(0x7fbc64078960 msgr2=0x7fbc640ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:14.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.223+0000 7fbc6b600700 1 -- 172.21.15.83:0/1877950229 shutdown_connections 2024-04-23T14:47:14.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.223+0000 7fbc6b600700 1 -- 172.21.15.83:0/1877950229 wait complete. 2024-04-23T14:47:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:14 smithi083 ceph-mon[27867]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-23T14:47:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:14 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1877950229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T14:47:14.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:14 smithi173 ceph-mon[32443]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-23T14:47:14.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:14 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1877950229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T14:47:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:14 smithi203 ceph-mon[32800]: pgmap v356: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 86 B/s, 0 objects/s recovering 2024-04-23T14:47:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:14 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1877950229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T14:47:14.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.733+0000 7fcbd3c00700 1 -- 172.21.15.83:0/1553718909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc102b40 msgr2=0x7fcbcc102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:14.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.733+0000 7fcbd3c00700 1 --2- 172.21.15.83:0/1553718909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc102b40 0x7fcbcc102fc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009990 tx=0x7fcbb8009ca0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.733+0000 7fcbd3c00700 1 -- 172.21.15.83:0/1553718909 shutdown_connections 2024-04-23T14:47:14.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.733+0000 7fcbd3c00700 1 --2- 172.21.15.83:0/1553718909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc103500 0x7fcbcc10f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.733+0000 7fcbd3c00700 1 --2- 172.21.15.83:0/1553718909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbcc108c50 0x7fcbcc109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.733+0000 7fcbd3c00700 1 --2- 172.21.15.83:0/1553718909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc102b40 0x7fcbcc102fc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.733+0000 7fcbd3c00700 1 -- 172.21.15.83:0/1553718909 >> 172.21.15.83:0/1553718909 conn(0x7fcbcc079ca0 msgr2=0x7fcbcc07a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:14.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.733+0000 7fcbd3c00700 1 -- 172.21.15.83:0/1553718909 shutdown_connections 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.733+0000 7fcbd3c00700 1 -- 172.21.15.83:0/1553718909 wait complete. 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.734+0000 7fcbd3c00700 1 Processor -- start 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.734+0000 7fcbd3c00700 1 -- start start 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd3c00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc102b40 0x7fcbcc19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd3c00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbcc103500 0x7fcbcc19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd3c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc108c50 0x7fcbcc19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd3c00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc112e90 con 0x7fcbcc108c50 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd3c00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc112d10 con 0x7fcbcc102b40 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd3c00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc113010 con 0x7fcbcc103500 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd1200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc102b40 0x7fcbcc19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd1c00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc108c50 0x7fcbcc19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd1200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc102b40 0x7fcbcc19b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:57724/0 (socket says 172.21.15.83:57724) 2024-04-23T14:47:14.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd1200700 1 -- 172.21.15.83:0/868094978 learned_addr learned my addr 172.21.15.83:0/868094978 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:14.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbcbe00700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbcc103500 0x7fcbcc19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:14.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd1200700 1 -- 172.21.15.83:0/868094978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbcc103500 msgr2=0x7fcbcc19b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:14.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd1200700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbcc103500 0x7fcbcc19b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd1200700 1 -- 172.21.15.83:0/868094978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc108c50 msgr2=0x7fcbcc19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:14.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd1200700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc108c50 0x7fcbcc19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.735+0000 7fcbd1200700 1 -- 172.21.15.83:0/868094978 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbcc1a04b0 con 0x7fcbcc102b40 2024-04-23T14:47:14.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.736+0000 7fcbd1200700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc102b40 0x7fcbcc19b4a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000e9c0 tx=0x7fcbc000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:14.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.736+0000 7fcbc9600700 1 -- 172.21.15.83:0/868094978 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc000cbd0 con 0x7fcbcc102b40 2024-04-23T14:47:14.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.736+0000 7fcbc9600700 1 -- 172.21.15.83:0/868094978 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc000ee60 con 0x7fcbcc102b40 2024-04-23T14:47:14.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.736+0000 7fcbd3c00700 1 -- 172.21.15.83:0/868094978 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbcc1a07a0 con 0x7fcbcc102b40 2024-04-23T14:47:14.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.736+0000 7fcbc9600700 1 -- 172.21.15.83:0/868094978 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0005630 con 0x7fcbcc102b40 2024-04-23T14:47:14.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.736+0000 7fcbd3c00700 1 -- 172.21.15.83:0/868094978 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbcc1a8020 con 0x7fcbcc102b40 2024-04-23T14:47:14.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.737+0000 7fcbd3c00700 1 -- 172.21.15.83:0/868094978 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbcc0697e0 con 0x7fcbcc102b40 2024-04-23T14:47:14.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.739+0000 7fcbc9600700 1 -- 172.21.15.83:0/868094978 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcbc0004750 con 0x7fcbcc102b40 2024-04-23T14:47:14.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.741+0000 7fcbc9600700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcbb40a3cd0 0x7fcbb40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:14.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.741+0000 7fcbc9600700 1 -- 172.21.15.83:0/868094978 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fcbc00d0a80 con 0x7fcbcc102b40 2024-04-23T14:47:14.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.741+0000 7fcbcbe00700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcbb40a3cd0 0x7fcbb40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:14.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.741+0000 7fcbcbe00700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcbb40a3cd0 0x7fcbb40a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc07a540 tx=0x7fcbb8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:14.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.743+0000 7fcbc9600700 1 -- 172.21.15.83:0/868094978 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcbc0053ff0 con 0x7fcbcc102b40 2024-04-23T14:47:14.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.985+0000 7fcbd3c00700 1 -- 172.21.15.83:0/868094978 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fcbcc19c800 con 0x7fcbcc102b40 2024-04-23T14:47:14.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.986+0000 7fcbc9600700 1 -- 172.21.15.83:0/868094978 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcbc009d2c0 con 0x7fcbcc102b40 2024-04-23T14:47:14.986 INFO:teuthology.orchestra.run.smithi083.stdout:55834574934 2024-04-23T14:47:14.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.988+0000 7fcbd3c00700 1 -- 172.21.15.83:0/868094978 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcbb40a3cd0 msgr2=0x7fcbb40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.988+0000 7fcbd3c00700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcbb40a3cd0 0x7fcbb40a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc07a540 tx=0x7fcbb8005fd0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.989+0000 7fcbd3c00700 1 -- 172.21.15.83:0/868094978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc102b40 msgr2=0x7fcbcc19b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.989+0000 7fcbd3c00700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc102b40 0x7fcbcc19b4a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcbc000e9c0 tx=0x7fcbc000ecd0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.989+0000 7fcbd3c00700 1 -- 172.21.15.83:0/868094978 shutdown_connections 2024-04-23T14:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.989+0000 7fcbd3c00700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fcbb40a3cd0 0x7fcbb40a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.989+0000 7fcbd3c00700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbcc108c50 0x7fcbcc19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.989+0000 7fcbd3c00700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc102b40 0x7fcbcc19b4a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.989+0000 7fcbd3c00700 1 --2- 172.21.15.83:0/868094978 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbcc103500 0x7fcbcc19b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.989+0000 7fcbd3c00700 1 -- 172.21.15.83:0/868094978 >> 172.21.15.83:0/868094978 conn(0x7fcbcc079ca0 msgr2=0x7fcbcc10dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.989+0000 7fcbd3c00700 1 -- 172.21.15.83:0/868094978 shutdown_connections 2024-04-23T14:47:14.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:14.989+0000 7fcbd3c00700 1 -- 172.21.15.83:0/868094978 wait complete. 2024-04-23T14:47:15.452 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:15 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/868094978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T14:47:15.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:15 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/868094978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T14:47:15.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.198+0000 7f1d6b200700 1 -- 172.21.15.83:0/3956023592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d6c06b060 msgr2=0x7f1d6c06b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:15.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.198+0000 7f1d6b200700 1 --2- 172.21.15.83:0/3956023592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d6c06b060 0x7f1d6c06b4e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009f90 tx=0x7f1d5400b330 comp rx=0 tx=0).stop 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.198+0000 7f1d6b200700 1 -- 172.21.15.83:0/3956023592 shutdown_connections 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.198+0000 7f1d6b200700 1 --2- 172.21.15.83:0/3956023592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d6c06ba20 0x7f1d6c10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.198+0000 7f1d6b200700 1 --2- 172.21.15.83:0/3956023592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d6c108ad0 0x7f1d6c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.198+0000 7f1d6b200700 1 --2- 172.21.15.83:0/3956023592 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d6c06b060 0x7f1d6c06b4e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.198+0000 7f1d6b200700 1 -- 172.21.15.83:0/3956023592 >> 172.21.15.83:0/3956023592 conn(0x7f1d6c078830 msgr2=0x7f1d6c078c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.198+0000 7f1d6b200700 1 -- 172.21.15.83:0/3956023592 shutdown_connections 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.198+0000 7f1d6b200700 1 -- 172.21.15.83:0/3956023592 wait complete. 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.200+0000 7f1d6b200700 1 Processor -- start 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.200+0000 7f1d6b200700 1 -- start start 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.200+0000 7f1d6b200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d6c06b060 0x7f1d6c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.200+0000 7f1d6b200700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d6c06ba20 0x7f1d6c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.200+0000 7f1d6b200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d6c108ad0 0x7f1d6c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:15.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.200+0000 7f1d6b200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c112dc0 con 0x7f1d6c108ad0 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.200+0000 7f1d6b200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c112c40 con 0x7f1d6c06b060 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.200+0000 7f1d6b200700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c112f40 con 0x7f1d6c06ba20 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.200+0000 7f1d6a800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d6c108ad0 0x7f1d6c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.200+0000 7f1d6a800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d6c108ad0 0x7f1d6c19fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46122/0 (socket says 172.21.15.83:46122) 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.200+0000 7f1d6a800700 1 -- 172.21.15.83:0/3291349687 learned_addr learned my addr 172.21.15.83:0/3291349687 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.201+0000 7f1d61400700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d6c06ba20 0x7f1d6c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.201+0000 7f1d69e00700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d6c06b060 0x7f1d6c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.201+0000 7f1d6a800700 1 -- 172.21.15.83:0/3291349687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d6c06ba20 msgr2=0x7f1d6c19b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.201+0000 7f1d6a800700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d6c06ba20 0x7f1d6c19b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.201+0000 7f1d6a800700 1 -- 172.21.15.83:0/3291349687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d6c06b060 msgr2=0x7f1d6c19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.201+0000 7f1d6a800700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d6c06b060 0x7f1d6c19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.201+0000 7f1d6a800700 1 -- 172.21.15.83:0/3291349687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6c1a03b0 con 0x7f1d6c108ad0 2024-04-23T14:47:15.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.201+0000 7f1d6a800700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d6c108ad0 0x7f1d6c19fc70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400e970 tx=0x7f1d6400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.201+0000 7f1d63400700 1 -- 172.21.15.83:0/3291349687 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64004a30 con 0x7f1d6c108ad0 2024-04-23T14:47:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.202+0000 7f1d6b200700 1 -- 172.21.15.83:0/3291349687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6c1a06a0 con 0x7f1d6c108ad0 2024-04-23T14:47:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.202+0000 7f1d63400700 1 -- 172.21.15.83:0/3291349687 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d64005500 con 0x7f1d6c108ad0 2024-04-23T14:47:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.202+0000 7f1d63400700 1 -- 172.21.15.83:0/3291349687 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400ccd0 con 0x7f1d6c108ad0 2024-04-23T14:47:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.202+0000 7f1d6b200700 1 -- 172.21.15.83:0/3291349687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6c1a7f20 con 0x7f1d6c108ad0 2024-04-23T14:47:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.203+0000 7f1d6b200700 1 -- 172.21.15.83:0/3291349687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6c0697e0 con 0x7f1d6c108ad0 2024-04-23T14:47:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.205+0000 7f1d63400700 1 -- 172.21.15.83:0/3291349687 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1d64005020 con 0x7f1d6c108ad0 2024-04-23T14:47:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.206+0000 7f1d63400700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1d500a3cd0 0x7f1d500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.206+0000 7f1d63400700 1 -- 172.21.15.83:0/3291349687 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1d640d0800 con 0x7f1d6c108ad0 2024-04-23T14:47:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.206+0000 7f1d69e00700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1d500a3cd0 0x7f1d500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.207+0000 7f1d69e00700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1d500a3cd0 0x7f1d500a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c005fd0 tx=0x7f1d5c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:15.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.210+0000 7f1d63400700 1 -- 172.21.15.83:0/3291349687 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d6409d040 con 0x7f1d6c108ad0 2024-04-23T14:47:15.720 INFO:teuthology.orchestra.run.smithi083.stdout:163208757311 2024-04-23T14:47:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.451+0000 7f1d6b200700 1 -- 172.21.15.83:0/3291349687 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1d6c02dfe0 con 0x7f1d6c108ad0 2024-04-23T14:47:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.452+0000 7f1d63400700 1 -- 172.21.15.83:0/3291349687 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1d6409c940 con 0x7f1d6c108ad0 2024-04-23T14:47:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.454+0000 7f1d6b200700 1 -- 172.21.15.83:0/3291349687 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1d500a3cd0 msgr2=0x7f1d500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.454+0000 7f1d6b200700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1d500a3cd0 0x7f1d500a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c005fd0 tx=0x7f1d5c009310 comp rx=0 tx=0).stop 2024-04-23T14:47:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.454+0000 7f1d6b200700 1 -- 172.21.15.83:0/3291349687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d6c108ad0 msgr2=0x7f1d6c19fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.455+0000 7f1d6b200700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d6c108ad0 0x7f1d6c19fc70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400e970 tx=0x7f1d6400ec80 comp rx=0 tx=0).stop 2024-04-23T14:47:15.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.455+0000 7f1d6b200700 1 -- 172.21.15.83:0/3291349687 shutdown_connections 2024-04-23T14:47:15.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.455+0000 7f1d6b200700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1d500a3cd0 0x7f1d500a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:15.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.455+0000 7f1d6b200700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d6c108ad0 0x7f1d6c19fc70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:15.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.455+0000 7f1d6b200700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d6c06b060 0x7f1d6c19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:15.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.455+0000 7f1d6b200700 1 --2- 172.21.15.83:0/3291349687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d6c06ba20 0x7f1d6c19b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:15.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.455+0000 7f1d6b200700 1 -- 172.21.15.83:0/3291349687 >> 172.21.15.83:0/3291349687 conn(0x7f1d6c078830 msgr2=0x7f1d6c10ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:15.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.455+0000 7f1d6b200700 1 -- 172.21.15.83:0/3291349687 shutdown_connections 2024-04-23T14:47:15.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:15.455+0000 7f1d6b200700 1 -- 172.21.15.83:0/3291349687 wait complete. 2024-04-23T14:47:15.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:15 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/868094978' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T14:47:16.253 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411402 got 77309411410 for osd.2 2024-04-23T14:47:16.253 DEBUG:teuthology.parallel:result is None 2024-04-23T14:47:16.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:16 smithi173 ceph-mon[32443]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-23T14:47:16.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:16 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3291349687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T14:47:16.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:16 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/298374962' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T14:47:16.663 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:16 smithi083 ceph-mon[27867]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-23T14:47:16.663 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:16 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3291349687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T14:47:16.663 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:16 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/298374962' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T14:47:16.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:16 smithi203 ceph-mon[32800]: pgmap v357: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-23T14:47:16.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:16 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3291349687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T14:47:16.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:16 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/298374962' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T14:47:17.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.063+0000 7ff195000700 1 -- 172.21.15.83:0/2259559798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff190104e50 msgr2=0x7ff190105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:17.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.063+0000 7ff195000700 1 --2- 172.21.15.83:0/2259559798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff190104e50 0x7ff190105230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff1800099c0 tx=0x7ff180009cd0 comp rx=0 tx=0).stop 2024-04-23T14:47:17.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.063+0000 7ff195000700 1 -- 172.21.15.83:0/2259559798 shutdown_connections 2024-04-23T14:47:17.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.063+0000 7ff195000700 1 --2- 172.21.15.83:0/2259559798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff190109f30 0x7ff190111b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.063+0000 7ff195000700 1 --2- 172.21.15.83:0/2259559798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff190104e50 0x7ff190105230 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.063+0000 7ff195000700 1 --2- 172.21.15.83:0/2259559798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff190105800 0x7ff190109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.063+0000 7ff195000700 1 -- 172.21.15.83:0/2259559798 >> 172.21.15.83:0/2259559798 conn(0x7ff190100720 msgr2=0x7ff190102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.063+0000 7ff195000700 1 -- 172.21.15.83:0/2259559798 shutdown_connections 2024-04-23T14:47:17.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.063+0000 7ff195000700 1 -- 172.21.15.83:0/2259559798 wait complete. 2024-04-23T14:47:17.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.064+0000 7ff195000700 1 Processor -- start 2024-04-23T14:47:17.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.064+0000 7ff195000700 1 -- start start 2024-04-23T14:47:17.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.065+0000 7ff195000700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff190104e50 0x7ff19019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:17.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.065+0000 7ff195000700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff190105800 0x7ff19019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:17.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.065+0000 7ff195000700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff190109f30 0x7ff19019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:17.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.065+0000 7ff195000700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff190114f60 con 0x7ff190105800 2024-04-23T14:47:17.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.065+0000 7ff195000700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff190114de0 con 0x7ff190109f30 2024-04-23T14:47:17.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.065+0000 7ff195000700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901150e0 con 0x7ff190104e50 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.065+0000 7ff18f400700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff190104e50 0x7ff19019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.065+0000 7ff18f400700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff190104e50 0x7ff19019b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41360/0 (socket says 172.21.15.83:41360) 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.065+0000 7ff18f400700 1 -- 172.21.15.83:0/3550010078 learned_addr learned my addr 172.21.15.83:0/3550010078 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.065+0000 7ff18ea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff190105800 0x7ff19019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.065+0000 7ff18fe00700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff190109f30 0x7ff19019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.066+0000 7ff18ea00700 1 -- 172.21.15.83:0/3550010078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff190104e50 msgr2=0x7ff19019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.066+0000 7ff18ea00700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff190104e50 0x7ff19019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.066+0000 7ff18ea00700 1 -- 172.21.15.83:0/3550010078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff190109f30 msgr2=0x7ff19019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.066+0000 7ff18ea00700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff190109f30 0x7ff19019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.066+0000 7ff18ea00700 1 -- 172.21.15.83:0/3550010078 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1901a03b0 con 0x7ff190105800 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.066+0000 7ff18f400700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff190104e50 0x7ff19019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.066+0000 7ff18ea00700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff190105800 0x7ff19019b930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00e9c0 tx=0x7ff17c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.066+0000 7ff187e00700 1 -- 172.21.15.83:0/3550010078 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00c9c0 con 0x7ff190105800 2024-04-23T14:47:17.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.066+0000 7ff187e00700 1 -- 172.21.15.83:0/3550010078 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff17c00cb60 con 0x7ff190105800 2024-04-23T14:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.066+0000 7ff195000700 1 -- 172.21.15.83:0/3550010078 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1901a06a0 con 0x7ff190105800 2024-04-23T14:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.066+0000 7ff187e00700 1 -- 172.21.15.83:0/3550010078 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c018680 con 0x7ff190105800 2024-04-23T14:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.067+0000 7ff195000700 1 -- 172.21.15.83:0/3550010078 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1901a7f20 con 0x7ff190105800 2024-04-23T14:47:17.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.068+0000 7ff195000700 1 -- 172.21.15.83:0/3550010078 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1900697e0 con 0x7ff190105800 2024-04-23T14:47:17.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.070+0000 7ff187e00700 1 -- 172.21.15.83:0/3550010078 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff17c010430 con 0x7ff190105800 2024-04-23T14:47:17.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.071+0000 7ff187e00700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1740a3cd0 0x7ff1740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.071+0000 7ff187e00700 1 -- 172.21.15.83:0/3550010078 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff17c0d0690 con 0x7ff190105800 2024-04-23T14:47:17.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.071+0000 7ff18f400700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1740a3cd0 0x7ff1740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:17.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.072+0000 7ff18f400700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1740a3cd0 0x7ff1740a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff180009990 tx=0x7ff180005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:17.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.076+0000 7ff187e00700 1 -- 172.21.15.83:0/3550010078 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff17c09ced0 con 0x7ff190105800 2024-04-23T14:47:17.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.309+0000 7ff195000700 1 -- 172.21.15.83:0/3550010078 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff19019c700 con 0x7ff190105800 2024-04-23T14:47:17.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.310+0000 7ff187e00700 1 -- 172.21.15.83:0/3550010078 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff17c09c7d0 con 0x7ff190105800 2024-04-23T14:47:17.310 INFO:teuthology.orchestra.run.smithi083.stdout:141733920837 2024-04-23T14:47:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.312+0000 7ff195000700 1 -- 172.21.15.83:0/3550010078 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1740a3cd0 msgr2=0x7ff1740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.312+0000 7ff195000700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1740a3cd0 0x7ff1740a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff180009990 tx=0x7ff180005ab0 comp rx=0 tx=0).stop 2024-04-23T14:47:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.312+0000 7ff195000700 1 -- 172.21.15.83:0/3550010078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff190105800 msgr2=0x7ff19019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:17.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.312+0000 7ff195000700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff190105800 0x7ff19019b930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00e9c0 tx=0x7ff17c00ecd0 comp rx=0 tx=0).stop 2024-04-23T14:47:17.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.313+0000 7ff195000700 1 -- 172.21.15.83:0/3550010078 shutdown_connections 2024-04-23T14:47:17.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.313+0000 7ff195000700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1740a3cd0 0x7ff1740a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:17.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.313+0000 7ff195000700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff190109f30 0x7ff19019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:17.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.313+0000 7ff195000700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff190104e50 0x7ff19019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:17.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.313+0000 7ff195000700 1 --2- 172.21.15.83:0/3550010078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff190105800 0x7ff19019b930 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:17.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.313+0000 7ff195000700 1 -- 172.21.15.83:0/3550010078 >> 172.21.15.83:0/3550010078 conn(0x7ff190100720 msgr2=0x7ff190102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:17.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.313+0000 7ff195000700 1 -- 172.21.15.83:0/3550010078 shutdown_connections 2024-04-23T14:47:17.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:17.313+0000 7ff195000700 1 -- 172.21.15.83:0/3550010078 wait complete. 2024-04-23T14:47:17.630 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:17 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3550010078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T14:47:17.696 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612635 got 313532612640 for osd.10 2024-04-23T14:47:17.696 DEBUG:teuthology.parallel:result is None 2024-04-23T14:47:17.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:17 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3550010078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T14:47:17.808 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299439 got 223338299444 for osd.7 2024-04-23T14:47:17.808 DEBUG:teuthology.parallel:result is None 2024-04-23T14:47:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:17 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3550010078' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T14:47:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.480+0000 7f2e6f800700 1 -- 172.21.15.83:0/3232885528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e68102b40 msgr2=0x7f2e68102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.480+0000 7f2e6f800700 1 --2- 172.21.15.83:0/3232885528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e68102b40 0x7f2e68102fc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009990 tx=0x7f2e54009ca0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.480+0000 7f2e6f800700 1 -- 172.21.15.83:0/3232885528 shutdown_connections 2024-04-23T14:47:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.480+0000 7f2e6f800700 1 --2- 172.21.15.83:0/3232885528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e68103500 0x7f2e6810f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.480+0000 7f2e6f800700 1 --2- 172.21.15.83:0/3232885528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e68108c50 0x7f2e68109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.480+0000 7f2e6f800700 1 --2- 172.21.15.83:0/3232885528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e68102b40 0x7f2e68102fc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.480+0000 7f2e6f800700 1 -- 172.21.15.83:0/3232885528 >> 172.21.15.83:0/3232885528 conn(0x7f2e68079ca0 msgr2=0x7f2e6807a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.480+0000 7f2e6f800700 1 -- 172.21.15.83:0/3232885528 shutdown_connections 2024-04-23T14:47:18.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.480+0000 7f2e6f800700 1 -- 172.21.15.83:0/3232885528 wait complete. 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.481+0000 7f2e6f800700 1 Processor -- start 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.481+0000 7f2e6f800700 1 -- start start 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.481+0000 7f2e6f800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e68102b40 0x7f2e681992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.481+0000 7f2e6f800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e68103500 0x7f2e68199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.481+0000 7f2e6f800700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e68108c50 0x7f2e6819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.481+0000 7f2e6f800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e68112e70 con 0x7f2e68102b40 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.481+0000 7f2e6f800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2e68112cf0 con 0x7f2e68103500 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.481+0000 7f2e6f800700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2e68112ff0 con 0x7f2e68108c50 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.482+0000 7f2e6ce00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e68102b40 0x7f2e681992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.482+0000 7f2e6ce00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e68102b40 0x7f2e681992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46178/0 (socket says 172.21.15.83:46178) 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.482+0000 7f2e6ce00700 1 -- 172.21.15.83:0/2011840536 learned_addr learned my addr 172.21.15.83:0/2011840536 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.482+0000 7f2e67e00700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e68103500 0x7f2e68199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:18.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.482+0000 7f2e6d800700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e68108c50 0x7f2e6819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.482+0000 7f2e67e00700 1 -- 172.21.15.83:0/2011840536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e68108c50 msgr2=0x7f2e6819db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.482+0000 7f2e67e00700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e68108c50 0x7f2e6819db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.482+0000 7f2e67e00700 1 -- 172.21.15.83:0/2011840536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e68102b40 msgr2=0x7f2e681992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.482+0000 7f2e67e00700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e68102b40 0x7f2e681992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.482+0000 7f2e67e00700 1 -- 172.21.15.83:0/2011840536 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6819e280 con 0x7f2e68103500 2024-04-23T14:47:18.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.483+0000 7f2e67e00700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e68103500 0x7f2e68199800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e540059d0 tx=0x7f2e5400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:18.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.483+0000 7f2e65600700 1 -- 172.21.15.83:0/2011840536 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54027070 con 0x7f2e68103500 2024-04-23T14:47:18.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.483+0000 7f2e65600700 1 -- 172.21.15.83:0/2011840536 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e54003d70 con 0x7f2e68103500 2024-04-23T14:47:18.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.483+0000 7f2e6f800700 1 -- 172.21.15.83:0/2011840536 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6819e510 con 0x7f2e68103500 2024-04-23T14:47:18.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.483+0000 7f2e65600700 1 -- 172.21.15.83:0/2011840536 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5400f460 con 0x7f2e68103500 2024-04-23T14:47:18.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.483+0000 7f2e6f800700 1 -- 172.21.15.83:0/2011840536 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e681a5df0 con 0x7f2e68103500 2024-04-23T14:47:18.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.484+0000 7f2e6f800700 1 -- 172.21.15.83:0/2011840536 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e680697e0 con 0x7f2e68103500 2024-04-23T14:47:18.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.487+0000 7f2e65600700 1 -- 172.21.15.83:0/2011840536 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2e5402a070 con 0x7f2e68103500 2024-04-23T14:47:18.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.488+0000 7f2e65600700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2e500a3d50 0x7f2e500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.488+0000 7f2e65600700 1 -- 172.21.15.83:0/2011840536 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2e540d1c40 con 0x7f2e68103500 2024-04-23T14:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.488+0000 7f2e6ce00700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2e500a3d50 0x7f2e500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:18.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.489+0000 7f2e6ce00700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2e500a3d50 0x7f2e500a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009680 tx=0x7f2e5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:18.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.490+0000 7f2e65600700 1 -- 172.21.15.83:0/2011840536 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e54031070 con 0x7f2e68103500 2024-04-23T14:47:18.581 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:18 smithi083 ceph-mon[27867]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:47:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.018+0000 7fd6bda00700 1 -- 172.21.15.83:0/2593627342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6b8105530 msgr2=0x7fd6b81059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.018+0000 7fd6bda00700 1 --2- 172.21.15.83:0/2593627342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6b8105530 0x7fd6b81059c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000b0a0 tx=0x7fd6b000b3b0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.018+0000 7fd6bda00700 1 -- 172.21.15.83:0/2593627342 shutdown_connections 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.018+0000 7fd6bda00700 1 --2- 172.21.15.83:0/2593627342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6b8105530 0x7fd6b81059c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.018+0000 7fd6bda00700 1 --2- 172.21.15.83:0/2593627342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b8107a20 0x7fd6b806a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.018+0000 7fd6bda00700 1 --2- 172.21.15.83:0/2593627342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b806ad40 0x7fd6b806b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.018+0000 7fd6bda00700 1 -- 172.21.15.83:0/2593627342 >> 172.21.15.83:0/2593627342 conn(0x7fd6b80789a0 msgr2=0x7fd6b8078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.018+0000 7fd6bda00700 1 -- 172.21.15.83:0/2593627342 shutdown_connections 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.018+0000 7fd6bda00700 1 -- 172.21.15.83:0/2593627342 wait complete. 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.019+0000 7fd6bda00700 1 Processor -- start 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.019+0000 7fd6bda00700 1 -- start start 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.020+0000 7fd6bda00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6b806ad40 0x7fd6b81a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.020+0000 7fd6bda00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105530 0x7fd6b81a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.020+0000 7fd6bda00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b8107a20 0x7fd6b819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.020+0000 7fd6bda00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8119430 con 0x7fd6b806ad40 2024-04-23T14:47:18.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.020+0000 7fd6bda00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81192b0 con 0x7fd6b8105530 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.020+0000 7fd6bda00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b81195b0 con 0x7fd6b8107a20 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.020+0000 7fd6b6a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6b806ad40 0x7fd6b81a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.020+0000 7fd6b7400700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b8107a20 0x7fd6b819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.020+0000 7fd6b6a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6b806ad40 0x7fd6b81a55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46138/0 (socket says 172.21.15.83:46138) 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.020+0000 7fd6b6a00700 1 -- 172.21.15.83:0/298374962 learned_addr learned my addr 172.21.15.83:0/298374962 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.020+0000 7fd6b6000700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105530 0x7fd6b81a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.021+0000 7fd6b7400700 1 -- 172.21.15.83:0/298374962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105530 msgr2=0x7fd6b81a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.021+0000 7fd6b7400700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105530 0x7fd6b81a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.021+0000 7fd6b7400700 1 -- 172.21.15.83:0/298374962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6b806ad40 msgr2=0x7fd6b81a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.021+0000 7fd6b7400700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6b806ad40 0x7fd6b81a55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.021+0000 7fd6b7400700 1 -- 172.21.15.83:0/298374962 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b819ff90 con 0x7fd6b8107a20 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.021+0000 7fd6b6a00700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6b806ad40 0x7fd6b81a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.021+0000 7fd6b7400700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b8107a20 0x7fd6b819f6b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000b070 tx=0x7fd6b0004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.021+0000 7fd6af400700 1 -- 172.21.15.83:0/298374962 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0021070 con 0x7fd6b8107a20 2024-04-23T14:47:18.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.021+0000 7fd6af400700 1 -- 172.21.15.83:0/298374962 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6b0007b80 con 0x7fd6b8107a20 2024-04-23T14:47:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.021+0000 7fd6bda00700 1 -- 172.21.15.83:0/298374962 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b81a0220 con 0x7fd6b8107a20 2024-04-23T14:47:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.021+0000 7fd6af400700 1 -- 172.21.15.83:0/298374962 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0005040 con 0x7fd6b8107a20 2024-04-23T14:47:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.021+0000 7fd6bda00700 1 -- 172.21.15.83:0/298374962 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b81ac560 con 0x7fd6b8107a20 2024-04-23T14:47:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.023+0000 7fd6bda00700 1 -- 172.21.15.83:0/298374962 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b81a0590 con 0x7fd6b8107a20 2024-04-23T14:47:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.025+0000 7fd6af400700 1 -- 172.21.15.83:0/298374962 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd6b0024070 con 0x7fd6b8107a20 2024-04-23T14:47:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.026+0000 7fd6af400700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd69c0a3d50 0x7fd69c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.026+0000 7fd6af400700 1 -- 172.21.15.83:0/298374962 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd6b00d13a0 con 0x7fd6b8107a20 2024-04-23T14:47:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.026+0000 7fd6b6a00700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd69c0a3d50 0x7fd69c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.027+0000 7fd6b6a00700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd69c0a3d50 0x7fd69c0a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8011550 tx=0x7fd6a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.028+0000 7fd6af400700 1 -- 172.21.15.83:0/298374962 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6b81a0590 con 0x7fd6b8107a20 2024-04-23T14:47:18.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.259+0000 7fd6bda00700 1 -- 172.21.15.83:0/298374962 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd6b8102010 con 0x7fd6b8107a20 2024-04-23T14:47:18.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.260+0000 7fd6af400700 1 -- 172.21.15.83:0/298374962 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd6b8102010 con 0x7fd6b8107a20 2024-04-23T14:47:18.726 INFO:teuthology.orchestra.run.smithi083.stdout:347892351005 2024-04-23T14:47:18.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.262+0000 7fd6bda00700 1 -- 172.21.15.83:0/298374962 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd69c0a3d50 msgr2=0x7fd69c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:18.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.262+0000 7fd6bda00700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd69c0a3d50 0x7fd69c0a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8011550 tx=0x7fd6a8009260 comp rx=0 tx=0).stop 2024-04-23T14:47:18.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.262+0000 7fd6bda00700 1 -- 172.21.15.83:0/298374962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b8107a20 msgr2=0x7fd6b819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:18.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.262+0000 7fd6bda00700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b8107a20 0x7fd6b819f6b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000b070 tx=0x7fd6b0004c80 comp rx=0 tx=0).stop 2024-04-23T14:47:18.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.263+0000 7fd6bda00700 1 -- 172.21.15.83:0/298374962 shutdown_connections 2024-04-23T14:47:18.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.263+0000 7fd6bda00700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd69c0a3d50 0x7fd69c0a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.263+0000 7fd6bda00700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6b8107a20 0x7fd6b819f6b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.263+0000 7fd6bda00700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6b806ad40 0x7fd6b81a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.263+0000 7fd6bda00700 1 --2- 172.21.15.83:0/298374962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6b8105530 0x7fd6b81a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.263+0000 7fd6bda00700 1 -- 172.21.15.83:0/298374962 >> 172.21.15.83:0/298374962 conn(0x7fd6b80789a0 msgr2=0x7fd6b8104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:18.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.263+0000 7fd6bda00700 1 -- 172.21.15.83:0/298374962 shutdown_connections 2024-04-23T14:47:18.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:16.263+0000 7fd6bda00700 1 -- 172.21.15.83:0/298374962 wait complete. 2024-04-23T14:47:18.728 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:18 smithi173 ceph-mon[32443]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:47:18.729 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:18 smithi203 ceph-mon[32800]: pgmap v358: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:47:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.727+0000 7f2e6f800700 1 -- 172.21.15.83:0/2011840536 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2e6802dfe0 con 0x7f2e68103500 2024-04-23T14:47:18.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.728+0000 7f2e65600700 1 -- 172.21.15.83:0/2011840536 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2e54013070 con 0x7f2e68103500 2024-04-23T14:47:18.729 INFO:teuthology.orchestra.run.smithi083.stdout:197568495674 2024-04-23T14:47:18.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.730+0000 7f2e6f800700 1 -- 172.21.15.83:0/2011840536 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2e500a3d50 msgr2=0x7f2e500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:18.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.730+0000 7f2e6f800700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2e500a3d50 0x7f2e500a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009680 tx=0x7f2e5c009260 comp rx=0 tx=0).stop 2024-04-23T14:47:18.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.730+0000 7f2e6f800700 1 -- 172.21.15.83:0/2011840536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e68103500 msgr2=0x7f2e68199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:18.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.730+0000 7f2e6f800700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e68103500 0x7f2e68199800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2e540059d0 tx=0x7f2e5400bab0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.731+0000 7f2e6f800700 1 -- 172.21.15.83:0/2011840536 shutdown_connections 2024-04-23T14:47:18.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.731+0000 7f2e6f800700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2e500a3d50 0x7f2e500a6210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.731+0000 7f2e6f800700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e68108c50 0x7f2e6819db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.731+0000 7f2e6f800700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e68102b40 0x7f2e681992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.731+0000 7f2e6f800700 1 --2- 172.21.15.83:0/2011840536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e68103500 0x7f2e68199800 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:18.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.731+0000 7f2e6f800700 1 -- 172.21.15.83:0/2011840536 >> 172.21.15.83:0/2011840536 conn(0x7f2e68079ca0 msgr2=0x7f2e68101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:18.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.731+0000 7f2e6f800700 1 -- 172.21.15.83:0/2011840536 shutdown_connections 2024-04-23T14:47:18.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:18.731+0000 7f2e6f800700 1 -- 172.21.15.83:0/2011840536 wait complete. 2024-04-23T14:47:19.596 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:19 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2011840536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T14:47:19.679 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920830 got 141733920837 for osd.4 2024-04-23T14:47:19.679 DEBUG:teuthology.parallel:result is None 2024-04-23T14:47:19.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:19 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2011840536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T14:47:19.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:19 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2011840536' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T14:47:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.853+0000 7fee92800700 1 -- 172.21.15.83:0/1592326335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c105920 msgr2=0x7fee8c109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.853+0000 7fee92800700 1 --2- 172.21.15.83:0/1592326335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c105920 0x7fee8c109970 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fee780099c0 tx=0x7fee78009cd0 comp rx=0 tx=0).stop 2024-04-23T14:47:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.853+0000 7fee92800700 1 -- 172.21.15.83:0/1592326335 shutdown_connections 2024-04-23T14:47:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.853+0000 7fee92800700 1 --2- 172.21.15.83:0/1592326335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee8c10a050 0x7fee8c111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.854+0000 7fee92800700 1 --2- 172.21.15.83:0/1592326335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee8c104f70 0x7fee8c105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.854+0000 7fee92800700 1 --2- 172.21.15.83:0/1592326335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c105920 0x7fee8c109970 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.854+0000 7fee92800700 1 -- 172.21.15.83:0/1592326335 >> 172.21.15.83:0/1592326335 conn(0x7fee8c100800 msgr2=0x7fee8c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.854+0000 7fee92800700 1 -- 172.21.15.83:0/1592326335 shutdown_connections 2024-04-23T14:47:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.854+0000 7fee92800700 1 -- 172.21.15.83:0/1592326335 wait complete. 2024-04-23T14:47:20.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.855+0000 7fee92800700 1 Processor -- start 2024-04-23T14:47:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.855+0000 7fee92800700 1 -- start start 2024-04-23T14:47:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.856+0000 7fee92800700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee8c104f70 0x7fee8c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.856+0000 7fee92800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee8c105920 0x7fee8c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.856+0000 7fee92800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c10a050 0x7fee8c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.856+0000 7fee92800700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c115080 con 0x7fee8c10a050 2024-04-23T14:47:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.856+0000 7fee92800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c114f00 con 0x7fee8c105920 2024-04-23T14:47:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.856+0000 7fee92800700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c115200 con 0x7fee8c104f70 2024-04-23T14:47:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.856+0000 7fee8be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c10a050 0x7fee8c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.856+0000 7fee8b400700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee8c104f70 0x7fee8c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.856+0000 7fee8be00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c10a050 0x7fee8c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46194/0 (socket says 172.21.15.83:46194) 2024-04-23T14:47:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.856+0000 7fee8aa00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee8c105920 0x7fee8c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:20.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.856+0000 7fee8be00700 1 -- 172.21.15.83:0/2283272435 learned_addr learned my addr 172.21.15.83:0/2283272435 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.857+0000 7fee8b400700 1 -- 172.21.15.83:0/2283272435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee8c105920 msgr2=0x7fee8c19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.857+0000 7fee8b400700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee8c105920 0x7fee8c19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.857+0000 7fee8b400700 1 -- 172.21.15.83:0/2283272435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c10a050 msgr2=0x7fee8c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.857+0000 7fee8b400700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c10a050 0x7fee8c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.857+0000 7fee8b400700 1 -- 172.21.15.83:0/2283272435 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8c1a0440 con 0x7fee8c104f70 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.857+0000 7fee8be00700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c10a050 0x7fee8c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.857+0000 7fee8b400700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee8c104f70 0x7fee8c19b430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00c970 tx=0x7fee7c00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.857+0000 7fee83e00700 1 -- 172.21.15.83:0/2283272435 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c014070 con 0x7fee8c104f70 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.857+0000 7fee92800700 1 -- 172.21.15.83:0/2283272435 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8c1a0730 con 0x7fee8c104f70 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.857+0000 7fee83e00700 1 -- 172.21.15.83:0/2283272435 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee7c004440 con 0x7fee8c104f70 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.857+0000 7fee92800700 1 -- 172.21.15.83:0/2283272435 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8c1a7fb0 con 0x7fee8c104f70 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.857+0000 7fee83e00700 1 -- 172.21.15.83:0/2283272435 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c007a30 con 0x7fee8c104f70 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.858+0000 7fee92800700 1 -- 172.21.15.83:0/2283272435 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee8c106450 con 0x7fee8c104f70 2024-04-23T14:47:20.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.860+0000 7fee83e00700 1 -- 172.21.15.83:0/2283272435 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fee7c017070 con 0x7fee8c104f70 2024-04-23T14:47:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.862+0000 7fee83e00700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fee700a3d50 0x7fee700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.862+0000 7fee83e00700 1 -- 172.21.15.83:0/2283272435 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fee7c0d0550 con 0x7fee8c104f70 2024-04-23T14:47:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.862+0000 7fee8aa00700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fee700a3d50 0x7fee700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.863+0000 7fee8aa00700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fee700a3d50 0x7fee700a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fee78000c00 tx=0x7fee78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:20.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:19.864+0000 7fee83e00700 1 -- 172.21.15.83:0/2283272435 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee7c01e030 con 0x7fee8c104f70 2024-04-23T14:47:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.106+0000 7fee92800700 1 -- 172.21.15.83:0/2283272435 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fee8c1a81e0 con 0x7fee8c104f70 2024-04-23T14:47:20.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.107+0000 7fee83e00700 1 -- 172.21.15.83:0/2283272435 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fee7c09ce10 con 0x7fee8c104f70 2024-04-23T14:47:20.108 INFO:teuthology.orchestra.run.smithi083.stdout:34359738460 2024-04-23T14:47:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.110+0000 7fee92800700 1 -- 172.21.15.83:0/2283272435 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fee700a3d50 msgr2=0x7fee700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.110+0000 7fee92800700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fee700a3d50 0x7fee700a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fee78000c00 tx=0x7fee78005fb0 comp rx=0 tx=0).stop 2024-04-23T14:47:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.110+0000 7fee92800700 1 -- 172.21.15.83:0/2283272435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee8c104f70 msgr2=0x7fee8c19b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.110+0000 7fee92800700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee8c104f70 0x7fee8c19b430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00c970 tx=0x7fee7c00cc80 comp rx=0 tx=0).stop 2024-04-23T14:47:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.110+0000 7fee92800700 1 -- 172.21.15.83:0/2283272435 shutdown_connections 2024-04-23T14:47:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.110+0000 7fee92800700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fee700a3d50 0x7fee700a6210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.110+0000 7fee92800700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee8c10a050 0x7fee8c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.110+0000 7fee92800700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fee8c104f70 0x7fee8c19b430 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.110+0000 7fee92800700 1 --2- 172.21.15.83:0/2283272435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee8c105920 0x7fee8c19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.110+0000 7fee92800700 1 -- 172.21.15.83:0/2283272435 >> 172.21.15.83:0/2283272435 conn(0x7fee8c100800 msgr2=0x7fee8c1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.110+0000 7fee92800700 1 -- 172.21.15.83:0/2283272435 shutdown_connections 2024-04-23T14:47:20.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:20.110+0000 7fee92800700 1 -- 172.21.15.83:0/2283272435 wait complete. 2024-04-23T14:47:20.165 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:47:20.194 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182469 got 107374182474 for osd.3 2024-04-23T14:47:20.194 DEBUG:teuthology.parallel:result is None 2024-04-23T14:47:20.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:20 smithi083 ceph-mon[27867]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:20.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:20 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2283272435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T14:47:20.670 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351000 got 347892351005 for osd.11 2024-04-23T14:47:20.671 DEBUG:teuthology.parallel:result is None 2024-04-23T14:47:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:20 smithi203 ceph-mon[32800]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:20 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2283272435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T14:47:20.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:20 smithi173 ceph-mon[32443]: pgmap v359: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:20.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:20 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2283272435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T14:47:21.946 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574929 got 55834574934 for osd.1 2024-04-23T14:47:21.946 DEBUG:teuthology.parallel:result is None 2024-04-23T14:47:22.108 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738455 got 34359738460 for osd.0 2024-04-23T14:47:22.108 DEBUG:teuthology.parallel:result is None 2024-04-23T14:47:22.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:22 smithi203 ceph-mon[32800]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:47:22.789 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:22 smithi083 ceph-mon[27867]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:47:22.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:22 smithi173 ceph-mon[32443]: pgmap v360: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:47:23.662 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757305 got 163208757311 for osd.5 2024-04-23T14:47:23.662 DEBUG:teuthology.parallel:result is None 2024-04-23T14:47:23.759 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495667 got 197568495674 for osd.6 2024-04-23T14:47:23.759 DEBUG:teuthology.parallel:result is None 2024-04-23T14:47:24.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:24 smithi083 ceph-mon[27867]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:47:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:24 smithi203 ceph-mon[32800]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:47:24.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:24 smithi173 ceph-mon[32443]: pgmap v361: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:47:25.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.745+0000 7f41dc200700 1 -- 172.21.15.83:0/2023722718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d410e460 msgr2=0x7f41d4116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.745+0000 7f41dc200700 1 --2- 172.21.15.83:0/2023722718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d410e460 0x7f41d4116030 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00b0d0 tx=0x7f41cc00b3e0 comp rx=0 tx=0).stop 2024-04-23T14:47:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.746+0000 7f41dc200700 1 -- 172.21.15.83:0/2023722718 shutdown_connections 2024-04-23T14:47:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.746+0000 7f41dc200700 1 --2- 172.21.15.83:0/2023722718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d410e460 0x7f41d4116030 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.746+0000 7f41dc200700 1 --2- 172.21.15.83:0/2023722718 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41d4109380 0x7f41d4109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.746+0000 7f41dc200700 1 --2- 172.21.15.83:0/2023722718 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d4109d30 0x7f41d410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.746+0000 7f41dc200700 1 -- 172.21.15.83:0/2023722718 >> 172.21.15.83:0/2023722718 conn(0x7f41d40762c0 msgr2=0x7f41d4107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.746+0000 7f41dc200700 1 -- 172.21.15.83:0/2023722718 shutdown_connections 2024-04-23T14:47:25.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.746+0000 7f41dc200700 1 -- 172.21.15.83:0/2023722718 wait complete. 2024-04-23T14:47:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.747+0000 7f41dc200700 1 Processor -- start 2024-04-23T14:47:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.747+0000 7f41dc200700 1 -- start start 2024-04-23T14:47:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.747+0000 7f41dc200700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d4109380 0x7f41d41a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.747+0000 7f41dc200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d4109d30 0x7f41d41a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.747+0000 7f41dc200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41d410e460 0x7f41d419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.747+0000 7f41dc200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41d41194a0 con 0x7f41d4109d30 2024-04-23T14:47:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.747+0000 7f41dc200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f41d4119320 con 0x7f41d410e460 2024-04-23T14:47:25.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.747+0000 7f41dc200700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41d4119620 con 0x7f41d4109380 2024-04-23T14:47:25.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.748+0000 7f41d8e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d4109d30 0x7f41d41a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:25.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.748+0000 7f41d9800700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d4109380 0x7f41d41a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.748+0000 7f41d9800700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d4109380 0x7f41d41a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:51422/0 (socket says 172.21.15.83:51422) 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.748+0000 7f41d9800700 1 -- 172.21.15.83:0/72410005 learned_addr learned my addr 172.21.15.83:0/72410005 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.748+0000 7f41d8e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d4109d30 0x7f41d41a5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47102/0 (socket says 172.21.15.83:47102) 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.748+0000 7f41da200700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41d410e460 0x7f41d419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.748+0000 7f41d9800700 1 -- 172.21.15.83:0/72410005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41d410e460 msgr2=0x7f41d419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.748+0000 7f41d9800700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41d410e460 0x7f41d419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.748+0000 7f41d9800700 1 -- 172.21.15.83:0/72410005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d4109d30 msgr2=0x7f41d41a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.748+0000 7f41d9800700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d4109d30 0x7f41d41a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.748+0000 7f41d9800700 1 -- 172.21.15.83:0/72410005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d419fec0 con 0x7f41d4109380 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.749+0000 7f41d9800700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d4109380 0x7f41d41a55c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e9a0 tx=0x7f41c800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.749+0000 7f41d2000700 1 -- 172.21.15.83:0/72410005 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c800ca10 con 0x7f41d4109380 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.749+0000 7f41d2000700 1 -- 172.21.15.83:0/72410005 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c800cbb0 con 0x7f41d4109380 2024-04-23T14:47:25.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.749+0000 7f41dc200700 1 -- 172.21.15.83:0/72410005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d41a01b0 con 0x7f41d4109380 2024-04-23T14:47:25.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.749+0000 7f41d2000700 1 -- 172.21.15.83:0/72410005 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8018680 con 0x7f41d4109380 2024-04-23T14:47:25.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.749+0000 7f41dc200700 1 -- 172.21.15.83:0/72410005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d41ac3e0 con 0x7f41d4109380 2024-04-23T14:47:25.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.750+0000 7f41dc200700 1 -- 172.21.15.83:0/72410005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d40697e0 con 0x7f41d4109380 2024-04-23T14:47:25.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.752+0000 7f41d2000700 1 -- 172.21.15.83:0/72410005 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f41c8010430 con 0x7f41d4109380 2024-04-23T14:47:25.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.754+0000 7f41d2000700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f41bc0a3cd0 0x7f41bc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:25.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.755+0000 7f41d2000700 1 -- 172.21.15.83:0/72410005 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f41c80d07c0 con 0x7f41d4109380 2024-04-23T14:47:25.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.755+0000 7f41d8e00700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f41bc0a3cd0 0x7f41bc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:25.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.755+0000 7f41d8e00700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f41bc0a3cd0 0x7f41bc0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41d4108870 tx=0x7f41c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:25.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:25.758+0000 7f41d2000700 1 -- 172.21.15.83:0/72410005 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41c809d000 con 0x7f41d4109380 2024-04-23T14:47:26.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.000+0000 7f41dc200700 1 -- 172.21.15.83:0/72410005 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f41d41ac670 con 0x7f41d4109380 2024-04-23T14:47:26.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.000+0000 7f41d2000700 1 -- 172.21.15.83:0/72410005 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f41c809c900 con 0x7f41d4109380 2024-04-23T14:47:26.001 INFO:teuthology.orchestra.run.smithi083.stdout:292057776170 2024-04-23T14:47:26.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.003+0000 7f41dc200700 1 -- 172.21.15.83:0/72410005 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f41bc0a3cd0 msgr2=0x7f41bc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:26.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.003+0000 7f41dc200700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f41bc0a3cd0 0x7f41bc0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41d4108870 tx=0x7f41c0009260 comp rx=0 tx=0).stop 2024-04-23T14:47:26.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.003+0000 7f41dc200700 1 -- 172.21.15.83:0/72410005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d4109380 msgr2=0x7f41d41a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:26.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.003+0000 7f41dc200700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d4109380 0x7f41d41a55c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e9a0 tx=0x7f41c800ecb0 comp rx=0 tx=0).stop 2024-04-23T14:47:26.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.003+0000 7f41dc200700 1 -- 172.21.15.83:0/72410005 shutdown_connections 2024-04-23T14:47:26.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.003+0000 7f41dc200700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f41bc0a3cd0 0x7f41bc0a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:26.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.003+0000 7f41dc200700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f41d410e460 0x7f41d419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:26.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.003+0000 7f41dc200700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41d4109380 0x7f41d41a55c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:26.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.003+0000 7f41dc200700 1 --2- 172.21.15.83:0/72410005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41d4109d30 0x7f41d41a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:26.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.003+0000 7f41dc200700 1 -- 172.21.15.83:0/72410005 >> 172.21.15.83:0/72410005 conn(0x7f41d40762c0 msgr2=0x7f41d40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:26.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.003+0000 7f41dc200700 1 -- 172.21.15.83:0/72410005 shutdown_connections 2024-04-23T14:47:26.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:26.003+0000 7f41dc200700 1 -- 172.21.15.83:0/72410005 wait complete. 2024-04-23T14:47:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:26 smithi203 ceph-mon[32800]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:26 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/72410005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T14:47:26.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:26 smithi083 ceph-mon[27867]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:26.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:26 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/72410005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T14:47:26.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:26 smithi173 ceph-mon[32443]: pgmap v362: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:26.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:26 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/72410005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T14:47:26.908 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776164 got 292057776170 for osd.9 2024-04-23T14:47:26.908 DEBUG:teuthology.parallel:result is None 2024-04-23T14:47:26.908 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-23T14:47:26.909 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T14:47:27.391 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:47:28.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:28 smithi203 ceph-mon[32800]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:47:28.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:28 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:28.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:28 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:28.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:28 smithi083 ceph-mon[27867]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:47:28.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:28 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:28.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:28 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:28.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:28 smithi173 ceph-mon[32443]: pgmap v363: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:47:28.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:28 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:28.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:28 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:29.496 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:29 smithi083 ceph-mon[27867]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-23T14:47:29.496 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:29 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:47:29.497 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:29 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:47:29.497 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:29 smithi083 ceph-mon[27867]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:29.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:29 smithi203 ceph-mon[32800]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-23T14:47:29.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:29 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:47:29.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:29 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:47:29.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:29 smithi203 ceph-mon[32800]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:29.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.785+0000 7f5947200700 1 -- 172.21.15.83:0/1664409972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5940103500 msgr2=0x7f594010f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:29.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.785+0000 7f5947200700 1 --2- 172.21.15.83:0/1664409972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5940103500 0x7f594010f960 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f593800b0a0 tx=0x7f593800b3b0 comp rx=0 tx=0).stop 2024-04-23T14:47:29.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.785+0000 7f5947200700 1 -- 172.21.15.83:0/1664409972 shutdown_connections 2024-04-23T14:47:29.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.785+0000 7f5947200700 1 --2- 172.21.15.83:0/1664409972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5940103500 0x7f594010f960 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:29.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.785+0000 7f5947200700 1 --2- 172.21.15.83:0/1664409972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5940108b90 0x7f5940108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:29.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.785+0000 7f5947200700 1 --2- 172.21.15.83:0/1664409972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5940102b40 0x7f5940102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:29.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.785+0000 7f5947200700 1 -- 172.21.15.83:0/1664409972 >> 172.21.15.83:0/1664409972 conn(0x7f5940079c50 msgr2=0x7f594007a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:29.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.786+0000 7f5947200700 1 -- 172.21.15.83:0/1664409972 shutdown_connections 2024-04-23T14:47:29.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.786+0000 7f5947200700 1 -- 172.21.15.83:0/1664409972 wait complete. 2024-04-23T14:47:29.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.787+0000 7f5947200700 1 Processor -- start 2024-04-23T14:47:29.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.787+0000 7f5947200700 1 -- start start 2024-04-23T14:47:29.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.788+0000 7f5947200700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5940102b40 0x7f59401992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:29.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.788+0000 7f5947200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5940103500 0x7f59401997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:29.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.788+0000 7f5947200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5940108b90 0x7f594019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.788+0000 7f5947200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5940112e60 con 0x7f5940103500 2024-04-23T14:47:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.788+0000 7f5947200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5940112ce0 con 0x7f5940108b90 2024-04-23T14:47:29.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.788+0000 7f5947200700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5940112fe0 con 0x7f5940102b40 2024-04-23T14:47:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.788+0000 7f593f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5940103500 0x7f59401997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.788+0000 7f593fe00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5940102b40 0x7f59401992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.788+0000 7f593fe00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5940102b40 0x7f59401992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:51446/0 (socket says 172.21.15.83:51446) 2024-04-23T14:47:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.788+0000 7f593fe00700 1 -- 172.21.15.83:0/1388295762 learned_addr learned my addr 172.21.15.83:0/1388295762 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.788+0000 7f5945200700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5940108b90 0x7f594019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.789+0000 7f593f400700 1 -- 172.21.15.83:0/1388295762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5940102b40 msgr2=0x7f59401992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:29.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.789+0000 7f593f400700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5940102b40 0x7f59401992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:29.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.789+0000 7f593f400700 1 -- 172.21.15.83:0/1388295762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5940108b90 msgr2=0x7f594019db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:29.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.789+0000 7f593f400700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5940108b90 0x7f594019db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:29.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.789+0000 7f593f400700 1 -- 172.21.15.83:0/1388295762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f594019e2c0 con 0x7f5940103500 2024-04-23T14:47:29.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.789+0000 7f593f400700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5940103500 0x7f59401997f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f592c00e9f0 tx=0x7f592c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:29.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.790+0000 7f593cc00700 1 -- 172.21.15.83:0/1388295762 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c00c9c0 con 0x7f5940103500 2024-04-23T14:47:29.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.790+0000 7f593cc00700 1 -- 172.21.15.83:0/1388295762 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f592c00cb60 con 0x7f5940103500 2024-04-23T14:47:29.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.790+0000 7f5947200700 1 -- 172.21.15.83:0/1388295762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f594019e5b0 con 0x7f5940103500 2024-04-23T14:47:29.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.790+0000 7f593cc00700 1 -- 172.21.15.83:0/1388295762 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c018680 con 0x7f5940103500 2024-04-23T14:47:29.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.790+0000 7f5947200700 1 -- 172.21.15.83:0/1388295762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59401a5e30 con 0x7f5940103500 2024-04-23T14:47:29.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.791+0000 7f5947200700 1 -- 172.21.15.83:0/1388295762 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59400697e0 con 0x7f5940103500 2024-04-23T14:47:29.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.793+0000 7f593cc00700 1 -- 172.21.15.83:0/1388295762 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f592c010430 con 0x7f5940103500 2024-04-23T14:47:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.796+0000 7f593cc00700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f59240a3c40 0x7f59240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:29.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.796+0000 7f593cc00700 1 -- 172.21.15.83:0/1388295762 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f592c0d0550 con 0x7f5940103500 2024-04-23T14:47:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.796+0000 7f593fe00700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f59240a3c40 0x7f59240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:29.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.797+0000 7f593fe00700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f59240a3c40 0x7f59240a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5930009ce0 tx=0x7f5930009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:29.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:29.800+0000 7f593cc00700 1 -- 172.21.15.83:0/1388295762 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f592c09cef0 con 0x7f5940103500 2024-04-23T14:47:29.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:29 smithi173 ceph-mon[32443]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-23T14:47:29.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:29 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:47:29.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:29 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:47:29.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:29 smithi173 ceph-mon[32443]: from='mgr.14152 172.21.15.83:0/792130291' entity='mgr.x' 2024-04-23T14:47:30.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.032+0000 7f5947200700 1 -- 172.21.15.83:0/1388295762 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5940062f30 con 0x7f59240a3c40 2024-04-23T14:47:30.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.042+0000 7f593cc00700 1 -- 172.21.15.83:0/1388295762 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259138 (secure 0 0 0) 0x7f5940062f30 con 0x7f59240a3c40 2024-04-23T14:47:30.047 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:47:30.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.048+0000 7f5947200700 1 -- 172.21.15.83:0/1388295762 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f59240a3c40 msgr2=0x7f59240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:30.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.048+0000 7f5947200700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f59240a3c40 0x7f59240a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5930009ce0 tx=0x7f5930009260 comp rx=0 tx=0).stop 2024-04-23T14:47:30.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.048+0000 7f5947200700 1 -- 172.21.15.83:0/1388295762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5940103500 msgr2=0x7f59401997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:30.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.048+0000 7f5947200700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5940103500 0x7f59401997f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f592c00e9f0 tx=0x7f592c00ed00 comp rx=0 tx=0).stop 2024-04-23T14:47:30.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.048+0000 7f5947200700 1 -- 172.21.15.83:0/1388295762 shutdown_connections 2024-04-23T14:47:30.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.048+0000 7f5947200700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f59240a3c40 0x7f59240a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:30.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.048+0000 7f5947200700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5940108b90 0x7f594019db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:30.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.048+0000 7f5947200700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5940102b40 0x7f59401992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:30.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.048+0000 7f5947200700 1 --2- 172.21.15.83:0/1388295762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5940103500 0x7f59401997f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:30.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.048+0000 7f5947200700 1 -- 172.21.15.83:0/1388295762 >> 172.21.15.83:0/1388295762 conn(0x7f5940079c50 msgr2=0x7f59401016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:30.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.049+0000 7f5947200700 1 -- 172.21.15.83:0/1388295762 shutdown_connections 2024-04-23T14:47:30.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:30.049+0000 7f5947200700 1 -- 172.21.15.83:0/1388295762 wait complete. 2024-04-23T14:47:30.051 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-23T14:47:30.721 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:30 smithi083 ceph-mon[27867]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-23T14:47:30.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:30 smithi203 ceph-mon[32800]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-23T14:47:30.767 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-04-23T14:47:29.243027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653835,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":634,"num_read_kb":439,"num_write":342,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181771,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":480,"ondisk_log_size":480,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377544,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564280,"statfs":{"total":1151940427776,"available":1151553822720,"internally_reserved":0,"allocated":11980800,"data_stored":7494603,"data_compressed":182499,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005127"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.250184+0000","last_change":"2024-04-23T14:46:00.030934+0000","last_active":"2024-04-23T14:47:27.250184+0000","last_peered":"2024-04-23T14:47:27.250184+0000","last_clean":"2024-04-23T14:47:27.250184+0000","last_became_active":"2024-04-23T14:46:00.030595+0000","last_became_peered":"2024-04-23T14:46:00.030595+0000","last_unstale":"2024-04-23T14:47:27.250184+0000","last_undegraded":"2024-04-23T14:47:27.250184+0000","last_fullsized":"2024-04-23T14:47:27.250184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.814685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.361496+0000","last_change":"2024-04-23T14:46:00.023044+0000","last_active":"2024-04-23T14:46:58.361496+0000","last_peered":"2024-04-23T14:46:58.361496+0000","last_clean":"2024-04-23T14:46:58.361496+0000","last_became_active":"2024-04-23T14:46:00.022441+0000","last_became_peered":"2024-04-23T14:46:00.022441+0000","last_unstale":"2024-04-23T14:46:58.361496+0000","last_undegraded":"2024-04-23T14:46:58.361496+0000","last_fullsized":"2024-04-23T14:46:58.361496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:24:20.976312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.360136+0000","last_change":"2024-04-23T14:46:00.025397+0000","last_active":"2024-04-23T14:46:58.360136+0000","last_peered":"2024-04-23T14:46:58.360136+0000","last_clean":"2024-04-23T14:46:58.360136+0000","last_became_active":"2024-04-23T14:46:00.024477+0000","last_became_peered":"2024-04-23T14:46:00.024477+0000","last_unstale":"2024-04-23T14:46:58.360136+0000","last_undegraded":"2024-04-23T14:46:58.360136+0000","last_fullsized":"2024-04-23T14:46:58.360136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:16.002964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.435954+0000","last_change":"2024-04-23T14:46:00.013960+0000","last_active":"2024-04-23T14:46:58.435954+0000","last_peered":"2024-04-23T14:46:58.435954+0000","last_clean":"2024-04-23T14:46:58.435954+0000","last_became_active":"2024-04-23T14:46:00.013656+0000","last_became_peered":"2024-04-23T14:46:00.013656+0000","last_unstale":"2024-04-23T14:46:58.435954+0000","last_undegraded":"2024-04-23T14:46:58.435954+0000","last_fullsized":"2024-04-23T14:46:58.435954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:45:59.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.251484+0000","last_change":"2024-04-23T14:46:00.032336+0000","last_active":"2024-04-23T14:47:27.251484+0000","last_peered":"2024-04-23T14:47:27.251484+0000","last_clean":"2024-04-23T14:47:27.251484+0000","last_became_active":"2024-04-23T14:46:00.032008+0000","last_became_peered":"2024-04-23T14:46:00.032008+0000","last_unstale":"2024-04-23T14:47:27.251484+0000","last_undegraded":"2024-04-23T14:47:27.251484+0000","last_fullsized":"2024-04-23T14:47:27.251484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:13:05.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.358245+0000","last_change":"2024-04-23T14:46:00.015457+0000","last_active":"2024-04-23T14:46:58.358245+0000","last_peered":"2024-04-23T14:46:58.358245+0000","last_clean":"2024-04-23T14:46:58.358245+0000","last_became_active":"2024-04-23T14:46:00.015199+0000","last_became_peered":"2024-04-23T14:46:00.015199+0000","last_unstale":"2024-04-23T14:46:58.358245+0000","last_undegraded":"2024-04-23T14:46:58.358245+0000","last_fullsized":"2024-04-23T14:46:58.358245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:39.310993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.246446+0000","last_change":"2024-04-23T14:46:00.016554+0000","last_active":"2024-04-23T14:47:27.246446+0000","last_peered":"2024-04-23T14:47:27.246446+0000","last_clean":"2024-04-23T14:47:27.246446+0000","last_became_active":"2024-04-23T14:46:00.016242+0000","last_became_peered":"2024-04-23T14:46:00.016242+0000","last_unstale":"2024-04-23T14:47:27.246446+0000","last_undegraded":"2024-04-23T14:47:27.246446+0000","last_fullsized":"2024-04-23T14:47:27.246446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.596637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.241613+0000","last_change":"2024-04-23T14:46:58.373050+0000","last_active":"2024-04-23T14:47:27.241613+0000","last_peered":"2024-04-23T14:47:27.241613+0000","last_clean":"2024-04-23T14:47:27.241613+0000","last_became_active":"2024-04-23T14:46:58.372507+0000","last_became_peered":"2024-04-23T14:46:58.372507+0000","last_unstale":"2024-04-23T14:47:27.241613+0000","last_undegraded":"2024-04-23T14:47:27.241613+0000","last_fullsized":"2024-04-23T14:47:27.241613+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:47:08.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367234+0000","last_change":"2024-04-23T14:46:00.005588+0000","last_active":"2024-04-23T14:46:58.367234+0000","last_peered":"2024-04-23T14:46:58.367234+0000","last_clean":"2024-04-23T14:46:58.367234+0000","last_became_active":"2024-04-23T14:46:00.005238+0000","last_became_peered":"2024-04-23T14:46:00.005238+0000","last_unstale":"2024-04-23T14:46:58.367234+0000","last_undegraded":"2024-04-23T14:46:58.367234+0000","last_fullsized":"2024-04-23T14:46:58.367234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:53:40.522929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.358507+0000","last_change":"2024-04-23T14:46:00.013773+0000","last_active":"2024-04-23T14:46:58.358507+0000","last_peered":"2024-04-23T14:46:58.358507+0000","last_clean":"2024-04-23T14:46:58.358507+0000","last_became_active":"2024-04-23T14:46:00.013390+0000","last_became_peered":"2024-04-23T14:46:00.013390+0000","last_unstale":"2024-04-23T14:46:58.358507+0000","last_undegraded":"2024-04-23T14:46:58.358507+0000","last_fullsized":"2024-04-23T14:46:58.358507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:44.352922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.242072+0000","last_change":"2024-04-23T14:46:00.013190+0000","last_active":"2024-04-23T14:47:27.242072+0000","last_peered":"2024-04-23T14:47:27.242072+0000","last_clean":"2024-04-23T14:47:27.242072+0000","last_became_active":"2024-04-23T14:46:00.012880+0000","last_became_peered":"2024-04-23T14:46:00.012880+0000","last_unstale":"2024-04-23T14:47:27.242072+0000","last_undegraded":"2024-04-23T14:47:27.242072+0000","last_fullsized":"2024-04-23T14:47:27.242072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:57.611514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.435934+0000","last_change":"2024-04-23T14:46:00.013459+0000","last_active":"2024-04-23T14:46:58.435934+0000","last_peered":"2024-04-23T14:46:58.435934+0000","last_clean":"2024-04-23T14:46:58.435934+0000","last_became_active":"2024-04-23T14:46:00.013106+0000","last_became_peered":"2024-04-23T14:46:00.013106+0000","last_unstale":"2024-04-23T14:46:58.435934+0000","last_undegraded":"2024-04-23T14:46:58.435934+0000","last_fullsized":"2024-04-23T14:46:58.435934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:08:38.849062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.432751+0000","last_change":"2024-04-23T14:46:00.030491+0000","last_active":"2024-04-23T14:46:58.432751+0000","last_peered":"2024-04-23T14:46:58.432751+0000","last_clean":"2024-04-23T14:46:58.432751+0000","last_became_active":"2024-04-23T14:46:00.030024+0000","last_became_peered":"2024-04-23T14:46:00.030024+0000","last_unstale":"2024-04-23T14:46:58.432751+0000","last_undegraded":"2024-04-23T14:46:58.432751+0000","last_fullsized":"2024-04-23T14:46:58.432751+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:31.289091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.360577+0000","last_change":"2024-04-23T14:46:00.022606+0000","last_active":"2024-04-23T14:46:58.360577+0000","last_peered":"2024-04-23T14:46:58.360577+0000","last_clean":"2024-04-23T14:46:58.360577+0000","last_became_active":"2024-04-23T14:46:00.022155+0000","last_became_peered":"2024-04-23T14:46:00.022155+0000","last_unstale":"2024-04-23T14:46:58.360577+0000","last_undegraded":"2024-04-23T14:46:58.360577+0000","last_fullsized":"2024-04-23T14:46:58.360577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:17:20.250283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.359868+0000","last_change":"2024-04-23T14:46:00.027193+0000","last_active":"2024-04-23T14:46:58.359868+0000","last_peered":"2024-04-23T14:46:58.359868+0000","last_clean":"2024-04-23T14:46:58.359868+0000","last_became_active":"2024-04-23T14:46:00.026211+0000","last_became_peered":"2024-04-23T14:46:00.026211+0000","last_unstale":"2024-04-23T14:46:58.359868+0000","last_undegraded":"2024-04-23T14:46:58.359868+0000","last_fullsized":"2024-04-23T14:46:58.359868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:52.722667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366160+0000","last_change":"2024-04-23T14:46:00.029917+0000","last_active":"2024-04-23T14:46:58.366160+0000","last_peered":"2024-04-23T14:46:58.366160+0000","last_clean":"2024-04-23T14:46:58.366160+0000","last_became_active":"2024-04-23T14:46:00.029563+0000","last_became_peered":"2024-04-23T14:46:00.029563+0000","last_unstale":"2024-04-23T14:46:58.366160+0000","last_undegraded":"2024-04-23T14:46:58.366160+0000","last_fullsized":"2024-04-23T14:46:58.366160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:01.787185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.255603+0000","last_change":"2024-04-23T14:46:00.030910+0000","last_active":"2024-04-23T14:47:27.255603+0000","last_peered":"2024-04-23T14:47:27.255603+0000","last_clean":"2024-04-23T14:47:27.255603+0000","last_became_active":"2024-04-23T14:46:00.030555+0000","last_became_peered":"2024-04-23T14:46:00.030555+0000","last_unstale":"2024-04-23T14:47:27.255603+0000","last_undegraded":"2024-04-23T14:47:27.255603+0000","last_fullsized":"2024-04-23T14:47:27.255603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:43.701752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.247395+0000","last_change":"2024-04-23T14:46:00.017338+0000","last_active":"2024-04-23T14:47:27.247395+0000","last_peered":"2024-04-23T14:47:27.247395+0000","last_clean":"2024-04-23T14:47:27.247395+0000","last_became_active":"2024-04-23T14:46:00.015381+0000","last_became_peered":"2024-04-23T14:46:00.015381+0000","last_unstale":"2024-04-23T14:47:27.247395+0000","last_undegraded":"2024-04-23T14:47:27.247395+0000","last_fullsized":"2024-04-23T14:47:27.247395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:51:16.762588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.368337+0000","last_change":"2024-04-23T14:46:00.033055+0000","last_active":"2024-04-23T14:46:58.368337+0000","last_peered":"2024-04-23T14:46:58.368337+0000","last_clean":"2024-04-23T14:46:58.368337+0000","last_became_active":"2024-04-23T14:46:00.032763+0000","last_became_peered":"2024-04-23T14:46:00.032763+0000","last_unstale":"2024-04-23T14:46:58.368337+0000","last_undegraded":"2024-04-23T14:46:58.368337+0000","last_fullsized":"2024-04-23T14:46:58.368337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:43.793644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.368181+0000","last_change":"2024-04-23T14:45:59.232611+0000","last_active":"2024-04-23T14:46:58.368181+0000","last_peered":"2024-04-23T14:46:58.368181+0000","last_clean":"2024-04-23T14:46:58.368181+0000","last_became_active":"2024-04-23T14:45:59.231066+0000","last_became_peered":"2024-04-23T14:45:59.231066+0000","last_unstale":"2024-04-23T14:46:58.368181+0000","last_undegraded":"2024-04-23T14:46:58.368181+0000","last_fullsized":"2024-04-23T14:46:58.368181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:24.204134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.226454+0000","last_change":"2024-04-23T14:46:00.021532+0000","last_active":"2024-04-23T14:47:27.226454+0000","last_peered":"2024-04-23T14:47:27.226454+0000","last_clean":"2024-04-23T14:47:27.226454+0000","last_became_active":"2024-04-23T14:46:00.021125+0000","last_became_peered":"2024-04-23T14:46:00.021125+0000","last_unstale":"2024-04-23T14:47:27.226454+0000","last_undegraded":"2024-04-23T14:47:27.226454+0000","last_fullsized":"2024-04-23T14:47:27.226454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:03.072779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.364399+0000","last_change":"2024-04-23T14:45:59.248925+0000","last_active":"2024-04-23T14:46:58.364399+0000","last_peered":"2024-04-23T14:46:58.364399+0000","last_clean":"2024-04-23T14:46:58.364399+0000","last_became_active":"2024-04-23T14:45:59.248579+0000","last_became_peered":"2024-04-23T14:45:59.248579+0000","last_unstale":"2024-04-23T14:46:58.364399+0000","last_undegraded":"2024-04-23T14:46:58.364399+0000","last_fullsized":"2024-04-23T14:46:58.364399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:17.012610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.361078+0000","last_change":"2024-04-23T14:46:00.028419+0000","last_active":"2024-04-23T14:46:58.361078+0000","last_peered":"2024-04-23T14:46:58.361078+0000","last_clean":"2024-04-23T14:46:58.361078+0000","last_became_active":"2024-04-23T14:46:00.026300+0000","last_became_peered":"2024-04-23T14:46:00.026300+0000","last_unstale":"2024-04-23T14:46:58.361078+0000","last_undegraded":"2024-04-23T14:46:58.361078+0000","last_fullsized":"2024-04-23T14:46:58.361078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:21.784680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367277+0000","last_change":"2024-04-23T14:45:59.225701+0000","last_active":"2024-04-23T14:46:58.367277+0000","last_peered":"2024-04-23T14:46:58.367277+0000","last_clean":"2024-04-23T14:46:58.367277+0000","last_became_active":"2024-04-23T14:45:59.225290+0000","last_became_peered":"2024-04-23T14:45:59.225290+0000","last_unstale":"2024-04-23T14:46:58.367277+0000","last_undegraded":"2024-04-23T14:46:58.367277+0000","last_fullsized":"2024-04-23T14:46:58.367277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:26.877330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366434+0000","last_change":"2024-04-23T14:46:00.035075+0000","last_active":"2024-04-23T14:46:58.366434+0000","last_peered":"2024-04-23T14:46:58.366434+0000","last_clean":"2024-04-23T14:46:58.366434+0000","last_became_active":"2024-04-23T14:46:00.034782+0000","last_became_peered":"2024-04-23T14:46:00.034782+0000","last_unstale":"2024-04-23T14:46:58.366434+0000","last_undegraded":"2024-04-23T14:46:58.366434+0000","last_fullsized":"2024-04-23T14:46:58.366434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:20.747643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.380160+0000","last_change":"2024-04-23T14:46:58.380160+0000","last_active":"2024-04-23T14:46:58.380160+0000","last_peered":"2024-04-23T14:46:58.380160+0000","last_clean":"2024-04-23T14:46:58.380160+0000","last_became_active":"2024-04-23T14:46:58.371279+0000","last_became_peered":"2024-04-23T14:46:58.371279+0000","last_unstale":"2024-04-23T14:46:58.380160+0000","last_undegraded":"2024-04-23T14:46:58.380160+0000","last_fullsized":"2024-04-23T14:46:58.380160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:19:44.330147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.372972+0000","last_change":"2024-04-23T14:46:58.372972+0000","last_active":"2024-04-23T14:46:58.372972+0000","last_peered":"2024-04-23T14:46:58.372972+0000","last_clean":"2024-04-23T14:46:58.372972+0000","last_became_active":"2024-04-23T14:46:58.372374+0000","last_became_peered":"2024-04-23T14:46:58.372374+0000","last_unstale":"2024-04-23T14:46:58.372972+0000","last_undegraded":"2024-04-23T14:46:58.372972+0000","last_fullsized":"2024-04-23T14:46:58.372972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:15.638664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":120,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:24.409609+0000","last_change":"2024-04-23T14:46:58.376256+0000","last_active":"2024-04-23T14:47:24.409609+0000","last_peered":"2024-04-23T14:47:24.409609+0000","last_clean":"2024-04-23T14:47:24.409609+0000","last_became_active":"2024-04-23T14:46:58.371767+0000","last_became_peered":"2024-04-23T14:46:58.371767+0000","last_unstale":"2024-04-23T14:47:24.409609+0000","last_undegraded":"2024-04-23T14:47:24.409609+0000","last_fullsized":"2024-04-23T14:47:24.409609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:01:47.773660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.242377+0000","last_change":"2024-04-23T14:46:00.016700+0000","last_active":"2024-04-23T14:47:27.242377+0000","last_peered":"2024-04-23T14:47:27.242377+0000","last_clean":"2024-04-23T14:47:27.242377+0000","last_became_active":"2024-04-23T14:46:00.015779+0000","last_became_peered":"2024-04-23T14:46:00.015779+0000","last_unstale":"2024-04-23T14:47:27.242377+0000","last_undegraded":"2024-04-23T14:47:27.242377+0000","last_fullsized":"2024-04-23T14:47:27.242377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:19:59.431628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.358048+0000","last_change":"2024-04-23T14:45:59.229791+0000","last_active":"2024-04-23T14:46:58.358048+0000","last_peered":"2024-04-23T14:46:58.358048+0000","last_clean":"2024-04-23T14:46:58.358048+0000","last_became_active":"2024-04-23T14:45:59.229054+0000","last_became_peered":"2024-04-23T14:45:59.229054+0000","last_unstale":"2024-04-23T14:46:58.358048+0000","last_undegraded":"2024-04-23T14:46:58.358048+0000","last_fullsized":"2024-04-23T14:46:58.358048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:19:59.385719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.369168+0000","last_change":"2024-04-23T14:46:00.021858+0000","last_active":"2024-04-23T14:46:58.369168+0000","last_peered":"2024-04-23T14:46:58.369168+0000","last_clean":"2024-04-23T14:46:58.369168+0000","last_became_active":"2024-04-23T14:46:00.021450+0000","last_became_peered":"2024-04-23T14:46:00.021450+0000","last_unstale":"2024-04-23T14:46:58.369168+0000","last_undegraded":"2024-04-23T14:46:58.369168+0000","last_fullsized":"2024-04-23T14:46:58.369168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:00.270035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.431920+0000","last_change":"2024-04-23T14:45:59.238181+0000","last_active":"2024-04-23T14:46:58.431920+0000","last_peered":"2024-04-23T14:46:58.431920+0000","last_clean":"2024-04-23T14:46:58.431920+0000","last_became_active":"2024-04-23T14:45:59.237784+0000","last_became_peered":"2024-04-23T14:45:59.237784+0000","last_unstale":"2024-04-23T14:46:58.431920+0000","last_undegraded":"2024-04-23T14:46:58.431920+0000","last_fullsized":"2024-04-23T14:46:58.431920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:35.016190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.249771+0000","last_change":"2024-04-23T14:46:00.033814+0000","last_active":"2024-04-23T14:47:27.249771+0000","last_peered":"2024-04-23T14:47:27.249771+0000","last_clean":"2024-04-23T14:47:27.249771+0000","last_became_active":"2024-04-23T14:46:00.032128+0000","last_became_peered":"2024-04-23T14:46:00.032128+0000","last_unstale":"2024-04-23T14:47:27.249771+0000","last_undegraded":"2024-04-23T14:47:27.249771+0000","last_fullsized":"2024-04-23T14:47:27.249771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:32.067418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.369713+0000","last_change":"2024-04-23T14:45:59.228054+0000","last_active":"2024-04-23T14:46:58.369713+0000","last_peered":"2024-04-23T14:46:58.369713+0000","last_clean":"2024-04-23T14:46:58.369713+0000","last_became_active":"2024-04-23T14:45:59.227759+0000","last_became_peered":"2024-04-23T14:45:59.227759+0000","last_unstale":"2024-04-23T14:46:58.369713+0000","last_undegraded":"2024-04-23T14:46:58.369713+0000","last_fullsized":"2024-04-23T14:46:58.369713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:52.185284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.243065+0000","last_change":"2024-04-23T14:46:00.028634+0000","last_active":"2024-04-23T14:47:27.243065+0000","last_peered":"2024-04-23T14:47:27.243065+0000","last_clean":"2024-04-23T14:47:27.243065+0000","last_became_active":"2024-04-23T14:46:00.027768+0000","last_became_peered":"2024-04-23T14:46:00.027768+0000","last_unstale":"2024-04-23T14:47:27.243065+0000","last_undegraded":"2024-04-23T14:47:27.243065+0000","last_fullsized":"2024-04-23T14:47:27.243065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:29.869845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":102,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:24.408928+0000","last_change":"2024-04-23T14:45:59.229480+0000","last_active":"2024-04-23T14:47:24.408928+0000","last_peered":"2024-04-23T14:47:24.408928+0000","last_clean":"2024-04-23T14:47:24.408928+0000","last_became_active":"2024-04-23T14:45:59.229124+0000","last_became_peered":"2024-04-23T14:45:59.229124+0000","last_unstale":"2024-04-23T14:47:24.408928+0000","last_undegraded":"2024-04-23T14:47:24.408928+0000","last_fullsized":"2024-04-23T14:47:24.408928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:33.837779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366928+0000","last_change":"2024-04-23T14:46:00.017753+0000","last_active":"2024-04-23T14:46:58.366928+0000","last_peered":"2024-04-23T14:46:58.366928+0000","last_clean":"2024-04-23T14:46:58.366928+0000","last_became_active":"2024-04-23T14:46:00.017370+0000","last_became_peered":"2024-04-23T14:46:00.017370+0000","last_unstale":"2024-04-23T14:46:58.366928+0000","last_undegraded":"2024-04-23T14:46:58.366928+0000","last_fullsized":"2024-04-23T14:46:58.366928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:30:35.460570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367839+0000","last_change":"2024-04-23T14:45:59.229361+0000","last_active":"2024-04-23T14:46:58.367839+0000","last_peered":"2024-04-23T14:46:58.367839+0000","last_clean":"2024-04-23T14:46:58.367839+0000","last_became_active":"2024-04-23T14:45:59.229043+0000","last_became_peered":"2024-04-23T14:45:59.229043+0000","last_unstale":"2024-04-23T14:46:58.367839+0000","last_undegraded":"2024-04-23T14:46:58.367839+0000","last_fullsized":"2024-04-23T14:46:58.367839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:09.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.360803+0000","last_change":"2024-04-23T14:46:00.030181+0000","last_active":"2024-04-23T14:46:58.360803+0000","last_peered":"2024-04-23T14:46:58.360803+0000","last_clean":"2024-04-23T14:46:58.360803+0000","last_became_active":"2024-04-23T14:46:00.029303+0000","last_became_peered":"2024-04-23T14:46:00.029303+0000","last_unstale":"2024-04-23T14:46:58.360803+0000","last_undegraded":"2024-04-23T14:46:58.360803+0000","last_fullsized":"2024-04-23T14:46:58.360803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:24:53.904501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.370426+0000","last_change":"2024-04-23T14:45:59.231270+0000","last_active":"2024-04-23T14:46:58.370426+0000","last_peered":"2024-04-23T14:46:58.370426+0000","last_clean":"2024-04-23T14:46:58.370426+0000","last_became_active":"2024-04-23T14:45:59.230914+0000","last_became_peered":"2024-04-23T14:45:59.230914+0000","last_unstale":"2024-04-23T14:46:58.370426+0000","last_undegraded":"2024-04-23T14:46:58.370426+0000","last_fullsized":"2024-04-23T14:46:58.370426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.422260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366598+0000","last_change":"2024-04-23T14:46:00.011358+0000","last_active":"2024-04-23T14:46:58.366598+0000","last_peered":"2024-04-23T14:46:58.366598+0000","last_clean":"2024-04-23T14:46:58.366598+0000","last_became_active":"2024-04-23T14:46:00.011040+0000","last_became_peered":"2024-04-23T14:46:00.011040+0000","last_unstale":"2024-04-23T14:46:58.366598+0000","last_undegraded":"2024-04-23T14:46:58.366598+0000","last_fullsized":"2024-04-23T14:46:58.366598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:27.248844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'26","reported_seq":212,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:25.410497+0000","last_change":"2024-04-23T14:45:59.227195+0000","last_active":"2024-04-23T14:47:25.410497+0000","last_peered":"2024-04-23T14:47:25.410497+0000","last_clean":"2024-04-23T14:47:25.410497+0000","last_became_active":"2024-04-23T14:45:59.226874+0000","last_became_peered":"2024-04-23T14:45:59.226874+0000","last_unstale":"2024-04-23T14:47:25.410497+0000","last_undegraded":"2024-04-23T14:47:25.410497+0000","last_fullsized":"2024-04-23T14:47:25.410497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:18:40.203839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58841,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":167,"num_read_kb":140,"num_write":27,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.231789+0000","last_change":"2024-04-23T14:46:00.010215+0000","last_active":"2024-04-23T14:47:27.231789+0000","last_peered":"2024-04-23T14:47:27.231789+0000","last_clean":"2024-04-23T14:47:27.231789+0000","last_became_active":"2024-04-23T14:46:00.009756+0000","last_became_peered":"2024-04-23T14:46:00.009756+0000","last_unstale":"2024-04-23T14:47:27.231789+0000","last_undegraded":"2024-04-23T14:47:27.231789+0000","last_fullsized":"2024-04-23T14:47:27.231789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.797966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.369370+0000","last_change":"2024-04-23T14:45:59.239258+0000","last_active":"2024-04-23T14:46:58.369370+0000","last_peered":"2024-04-23T14:46:58.369370+0000","last_clean":"2024-04-23T14:46:58.369370+0000","last_became_active":"2024-04-23T14:45:59.238899+0000","last_became_peered":"2024-04-23T14:45:59.238899+0000","last_unstale":"2024-04-23T14:46:58.369370+0000","last_undegraded":"2024-04-23T14:46:58.369370+0000","last_fullsized":"2024-04-23T14:46:58.369370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:45.866489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.432126+0000","last_change":"2024-04-23T14:46:00.036627+0000","last_active":"2024-04-23T14:46:58.432126+0000","last_peered":"2024-04-23T14:46:58.432126+0000","last_clean":"2024-04-23T14:46:58.432126+0000","last_became_active":"2024-04-23T14:46:00.035397+0000","last_became_peered":"2024-04-23T14:46:00.035397+0000","last_unstale":"2024-04-23T14:46:58.432126+0000","last_undegraded":"2024-04-23T14:46:58.432126+0000","last_fullsized":"2024-04-23T14:46:58.432126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:24:02.038813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.369141+0000","last_change":"2024-04-23T14:45:59.248241+0000","last_active":"2024-04-23T14:46:58.369141+0000","last_peered":"2024-04-23T14:46:58.369141+0000","last_clean":"2024-04-23T14:46:58.369141+0000","last_became_active":"2024-04-23T14:45:59.247895+0000","last_became_peered":"2024-04-23T14:45:59.247895+0000","last_unstale":"2024-04-23T14:46:58.369141+0000","last_undegraded":"2024-04-23T14:46:58.369141+0000","last_fullsized":"2024-04-23T14:46:58.369141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:01.964714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.252604+0000","last_change":"2024-04-23T14:46:58.433781+0000","last_active":"2024-04-23T14:47:27.252604+0000","last_peered":"2024-04-23T14:47:27.252604+0000","last_clean":"2024-04-23T14:47:27.252604+0000","last_became_active":"2024-04-23T14:46:58.433460+0000","last_became_peered":"2024-04-23T14:46:58.433460+0000","last_unstale":"2024-04-23T14:47:27.252604+0000","last_undegraded":"2024-04-23T14:47:27.252604+0000","last_fullsized":"2024-04-23T14:47:27.252604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:44.149949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":107,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:24.490067+0000","last_change":"2024-04-23T14:45:59.227406+0000","last_active":"2024-04-23T14:47:24.490067+0000","last_peered":"2024-04-23T14:47:24.490067+0000","last_clean":"2024-04-23T14:47:24.490067+0000","last_became_active":"2024-04-23T14:45:59.227063+0000","last_became_peered":"2024-04-23T14:45:59.227063+0000","last_unstale":"2024-04-23T14:47:24.490067+0000","last_undegraded":"2024-04-23T14:47:24.490067+0000","last_fullsized":"2024-04-23T14:47:24.490067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:36.845854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.234534+0000","last_change":"2024-04-23T14:46:00.006333+0000","last_active":"2024-04-23T14:47:27.234534+0000","last_peered":"2024-04-23T14:47:27.234534+0000","last_clean":"2024-04-23T14:47:27.234534+0000","last_became_active":"2024-04-23T14:46:00.006024+0000","last_became_peered":"2024-04-23T14:46:00.006024+0000","last_unstale":"2024-04-23T14:47:27.234534+0000","last_undegraded":"2024-04-23T14:47:27.234534+0000","last_fullsized":"2024-04-23T14:47:27.234534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:49.389445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366404+0000","last_change":"2024-04-23T14:45:59.226394+0000","last_active":"2024-04-23T14:46:58.366404+0000","last_peered":"2024-04-23T14:46:58.366404+0000","last_clean":"2024-04-23T14:46:58.366404+0000","last_became_active":"2024-04-23T14:45:59.226088+0000","last_became_peered":"2024-04-23T14:45:59.226088+0000","last_unstale":"2024-04-23T14:46:58.366404+0000","last_undegraded":"2024-04-23T14:46:58.366404+0000","last_fullsized":"2024-04-23T14:46:58.366404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:35.139520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.369001+0000","last_change":"2024-04-23T14:46:00.009262+0000","last_active":"2024-04-23T14:46:58.369001+0000","last_peered":"2024-04-23T14:46:58.369001+0000","last_clean":"2024-04-23T14:46:58.369001+0000","last_became_active":"2024-04-23T14:46:00.008865+0000","last_became_peered":"2024-04-23T14:46:00.008865+0000","last_unstale":"2024-04-23T14:46:58.369001+0000","last_undegraded":"2024-04-23T14:46:58.369001+0000","last_fullsized":"2024-04-23T14:46:58.369001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:27.662886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367049+0000","last_change":"2024-04-23T14:45:59.223479+0000","last_active":"2024-04-23T14:46:58.367049+0000","last_peered":"2024-04-23T14:46:58.367049+0000","last_clean":"2024-04-23T14:46:58.367049+0000","last_became_active":"2024-04-23T14:45:59.222997+0000","last_became_peered":"2024-04-23T14:45:59.222997+0000","last_unstale":"2024-04-23T14:46:58.367049+0000","last_undegraded":"2024-04-23T14:46:58.367049+0000","last_fullsized":"2024-04-23T14:46:58.367049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:29:58.026918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.248492+0000","last_change":"2024-04-23T14:46:58.434121+0000","last_active":"2024-04-23T14:47:27.248492+0000","last_peered":"2024-04-23T14:47:27.248492+0000","last_clean":"2024-04-23T14:47:27.248492+0000","last_became_active":"2024-04-23T14:46:58.433746+0000","last_became_peered":"2024-04-23T14:46:58.433746+0000","last_unstale":"2024-04-23T14:47:27.248492+0000","last_undegraded":"2024-04-23T14:47:27.248492+0000","last_fullsized":"2024-04-23T14:47:27.248492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:33:22.493604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367356+0000","last_change":"2024-04-23T14:45:59.237813+0000","last_active":"2024-04-23T14:46:58.367356+0000","last_peered":"2024-04-23T14:46:58.367356+0000","last_clean":"2024-04-23T14:46:58.367356+0000","last_became_active":"2024-04-23T14:45:59.237354+0000","last_became_peered":"2024-04-23T14:45:59.237354+0000","last_unstale":"2024-04-23T14:46:58.367356+0000","last_undegraded":"2024-04-23T14:46:58.367356+0000","last_fullsized":"2024-04-23T14:46:58.367356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:35:38.323059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.358242+0000","last_change":"2024-04-23T14:46:00.031956+0000","last_active":"2024-04-23T14:46:58.358242+0000","last_peered":"2024-04-23T14:46:58.358242+0000","last_clean":"2024-04-23T14:46:58.358242+0000","last_became_active":"2024-04-23T14:46:00.031666+0000","last_became_peered":"2024-04-23T14:46:00.031666+0000","last_unstale":"2024-04-23T14:46:58.358242+0000","last_undegraded":"2024-04-23T14:46:58.358242+0000","last_fullsized":"2024-04-23T14:46:58.358242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:17.346455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367284+0000","last_change":"2024-04-23T14:45:59.230633+0000","last_active":"2024-04-23T14:46:58.367284+0000","last_peered":"2024-04-23T14:46:58.367284+0000","last_clean":"2024-04-23T14:46:58.367284+0000","last_became_active":"2024-04-23T14:45:59.230300+0000","last_became_peered":"2024-04-23T14:45:59.230300+0000","last_unstale":"2024-04-23T14:46:58.367284+0000","last_undegraded":"2024-04-23T14:46:58.367284+0000","last_fullsized":"2024-04-23T14:46:58.367284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:41.170055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.358261+0000","last_change":"2024-04-23T14:46:00.030891+0000","last_active":"2024-04-23T14:46:58.358261+0000","last_peered":"2024-04-23T14:46:58.358261+0000","last_clean":"2024-04-23T14:46:58.358261+0000","last_became_active":"2024-04-23T14:46:00.030438+0000","last_became_peered":"2024-04-23T14:46:00.030438+0000","last_unstale":"2024-04-23T14:46:58.358261+0000","last_undegraded":"2024-04-23T14:46:58.358261+0000","last_fullsized":"2024-04-23T14:46:58.358261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:43.038111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.370630+0000","last_change":"2024-04-23T14:46:00.013683+0000","last_active":"2024-04-23T14:46:58.370630+0000","last_peered":"2024-04-23T14:46:58.370630+0000","last_clean":"2024-04-23T14:46:58.370630+0000","last_became_active":"2024-04-23T14:46:00.013356+0000","last_became_peered":"2024-04-23T14:46:00.013356+0000","last_unstale":"2024-04-23T14:46:58.370630+0000","last_undegraded":"2024-04-23T14:46:58.370630+0000","last_fullsized":"2024-04-23T14:46:58.370630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:32.866694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:24.489066+0000","last_change":"2024-04-23T14:45:59.252258+0000","last_active":"2024-04-23T14:47:24.489066+0000","last_peered":"2024-04-23T14:47:24.489066+0000","last_clean":"2024-04-23T14:47:24.489066+0000","last_became_active":"2024-04-23T14:45:59.251513+0000","last_became_peered":"2024-04-23T14:45:59.251513+0000","last_unstale":"2024-04-23T14:47:24.489066+0000","last_undegraded":"2024-04-23T14:47:24.489066+0000","last_fullsized":"2024-04-23T14:47:24.489066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:55:37.087807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.252125+0000","last_change":"2024-04-23T14:46:00.009468+0000","last_active":"2024-04-23T14:47:27.252125+0000","last_peered":"2024-04-23T14:47:27.252125+0000","last_clean":"2024-04-23T14:47:27.252125+0000","last_became_active":"2024-04-23T14:46:00.003118+0000","last_became_peered":"2024-04-23T14:46:00.003118+0000","last_unstale":"2024-04-23T14:47:27.252125+0000","last_undegraded":"2024-04-23T14:47:27.252125+0000","last_fullsized":"2024-04-23T14:47:27.252125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:20:24.488848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.435770+0000","last_change":"2024-04-23T14:45:59.225758+0000","last_active":"2024-04-23T14:46:58.435770+0000","last_peered":"2024-04-23T14:46:58.435770+0000","last_clean":"2024-04-23T14:46:58.435770+0000","last_became_active":"2024-04-23T14:45:59.225427+0000","last_became_peered":"2024-04-23T14:45:59.225427+0000","last_unstale":"2024-04-23T14:46:58.435770+0000","last_undegraded":"2024-04-23T14:46:58.435770+0000","last_fullsized":"2024-04-23T14:46:58.435770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:59.495348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366533+0000","last_change":"2024-04-23T14:44:39.684486+0000","last_active":"2024-04-23T14:46:58.366533+0000","last_peered":"2024-04-23T14:46:58.366533+0000","last_clean":"2024-04-23T14:46:58.366533+0000","last_became_active":"2024-04-23T14:44:39.684074+0000","last_became_peered":"2024-04-23T14:44:39.684074+0000","last_unstale":"2024-04-23T14:46:58.366533+0000","last_undegraded":"2024-04-23T14:46:58.366533+0000","last_fullsized":"2024-04-23T14:46:58.366533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_clean_scrub_stamp":"2024-04-23T14:40:41.333258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:11:16.865449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.431621+0000","last_change":"2024-04-23T14:46:00.025018+0000","last_active":"2024-04-23T14:46:58.431621+0000","last_peered":"2024-04-23T14:46:58.431621+0000","last_clean":"2024-04-23T14:46:58.431621+0000","last_became_active":"2024-04-23T14:46:00.024695+0000","last_became_peered":"2024-04-23T14:46:00.024695+0000","last_unstale":"2024-04-23T14:46:58.431621+0000","last_undegraded":"2024-04-23T14:46:58.431621+0000","last_fullsized":"2024-04-23T14:46:58.431621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:00:31.375963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366613+0000","last_change":"2024-04-23T14:45:59.225849+0000","last_active":"2024-04-23T14:46:58.366613+0000","last_peered":"2024-04-23T14:46:58.366613+0000","last_clean":"2024-04-23T14:46:58.366613+0000","last_became_active":"2024-04-23T14:45:59.225402+0000","last_became_peered":"2024-04-23T14:45:59.225402+0000","last_unstale":"2024-04-23T14:46:58.366613+0000","last_undegraded":"2024-04-23T14:46:58.366613+0000","last_fullsized":"2024-04-23T14:46:58.366613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:10:42.217717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.377666+0000","last_change":"2024-04-23T14:46:58.377666+0000","last_active":"2024-04-23T14:46:58.377666+0000","last_peered":"2024-04-23T14:46:58.377666+0000","last_clean":"2024-04-23T14:46:58.377666+0000","last_became_active":"2024-04-23T14:46:58.370533+0000","last_became_peered":"2024-04-23T14:46:58.370533+0000","last_unstale":"2024-04-23T14:46:58.377666+0000","last_undegraded":"2024-04-23T14:46:58.377666+0000","last_fullsized":"2024-04-23T14:46:58.377666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:39:58.606732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.432523+0000","last_change":"2024-04-23T14:46:00.039566+0000","last_active":"2024-04-23T14:46:58.432523+0000","last_peered":"2024-04-23T14:46:58.432523+0000","last_clean":"2024-04-23T14:46:58.432523+0000","last_became_active":"2024-04-23T14:46:00.039236+0000","last_became_peered":"2024-04-23T14:46:00.039236+0000","last_unstale":"2024-04-23T14:46:58.432523+0000","last_undegraded":"2024-04-23T14:46:58.432523+0000","last_fullsized":"2024-04-23T14:46:58.432523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:44.028367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367735+0000","last_change":"2024-04-23T14:45:59.246192+0000","last_active":"2024-04-23T14:46:58.367735+0000","last_peered":"2024-04-23T14:46:58.367735+0000","last_clean":"2024-04-23T14:46:58.367735+0000","last_became_active":"2024-04-23T14:45:59.245873+0000","last_became_peered":"2024-04-23T14:45:59.245873+0000","last_unstale":"2024-04-23T14:46:58.367735+0000","last_undegraded":"2024-04-23T14:46:58.367735+0000","last_fullsized":"2024-04-23T14:46:58.367735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:22.915746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.241236+0000","last_change":"2024-04-23T14:46:00.029176+0000","last_active":"2024-04-23T14:47:27.241236+0000","last_peered":"2024-04-23T14:47:27.241236+0000","last_clean":"2024-04-23T14:47:27.241236+0000","last_became_active":"2024-04-23T14:46:00.028741+0000","last_became_peered":"2024-04-23T14:46:00.028741+0000","last_unstale":"2024-04-23T14:47:27.241236+0000","last_undegraded":"2024-04-23T14:47:27.241236+0000","last_fullsized":"2024-04-23T14:47:27.241236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:35:22.905348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.436084+0000","last_change":"2024-04-23T14:45:59.231490+0000","last_active":"2024-04-23T14:46:58.436084+0000","last_peered":"2024-04-23T14:46:58.436084+0000","last_clean":"2024-04-23T14:46:58.436084+0000","last_became_active":"2024-04-23T14:45:59.231143+0000","last_became_peered":"2024-04-23T14:45:59.231143+0000","last_unstale":"2024-04-23T14:46:58.436084+0000","last_undegraded":"2024-04-23T14:46:58.436084+0000","last_fullsized":"2024-04-23T14:46:58.436084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:47:26.398150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.361778+0000","last_change":"2024-04-23T14:46:00.035838+0000","last_active":"2024-04-23T14:46:58.361778+0000","last_peered":"2024-04-23T14:46:58.361778+0000","last_clean":"2024-04-23T14:46:58.361778+0000","last_became_active":"2024-04-23T14:46:00.034498+0000","last_became_peered":"2024-04-23T14:46:00.034498+0000","last_unstale":"2024-04-23T14:46:58.361778+0000","last_undegraded":"2024-04-23T14:46:58.361778+0000","last_fullsized":"2024-04-23T14:46:58.361778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:28:15.748503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366294+0000","last_change":"2024-04-23T14:45:59.240817+0000","last_active":"2024-04-23T14:46:58.366294+0000","last_peered":"2024-04-23T14:46:58.366294+0000","last_clean":"2024-04-23T14:46:58.366294+0000","last_became_active":"2024-04-23T14:45:59.240557+0000","last_became_peered":"2024-04-23T14:45:59.240557+0000","last_unstale":"2024-04-23T14:46:58.366294+0000","last_undegraded":"2024-04-23T14:46:58.366294+0000","last_fullsized":"2024-04-23T14:46:58.366294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:03.099054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.197125+0000","last_change":"2024-04-23T14:46:00.009094+0000","last_active":"2024-04-23T14:47:27.197125+0000","last_peered":"2024-04-23T14:47:27.197125+0000","last_clean":"2024-04-23T14:47:27.197125+0000","last_became_active":"2024-04-23T14:46:00.008751+0000","last_became_peered":"2024-04-23T14:46:00.008751+0000","last_unstale":"2024-04-23T14:47:27.197125+0000","last_undegraded":"2024-04-23T14:47:27.197125+0000","last_fullsized":"2024-04-23T14:47:27.197125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:12.480737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.361163+0000","last_change":"2024-04-23T14:45:59.249882+0000","last_active":"2024-04-23T14:46:58.361163+0000","last_peered":"2024-04-23T14:46:58.361163+0000","last_clean":"2024-04-23T14:46:58.361163+0000","last_became_active":"2024-04-23T14:45:59.249072+0000","last_became_peered":"2024-04-23T14:45:59.249072+0000","last_unstale":"2024-04-23T14:46:58.361163+0000","last_undegraded":"2024-04-23T14:46:58.361163+0000","last_fullsized":"2024-04-23T14:46:58.361163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:04:16.573005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.255710+0000","last_change":"2024-04-23T14:46:00.030903+0000","last_active":"2024-04-23T14:47:27.255710+0000","last_peered":"2024-04-23T14:47:27.255710+0000","last_clean":"2024-04-23T14:47:27.255710+0000","last_became_active":"2024-04-23T14:46:00.030051+0000","last_became_peered":"2024-04-23T14:46:00.030051+0000","last_unstale":"2024-04-23T14:47:27.255710+0000","last_undegraded":"2024-04-23T14:47:27.255710+0000","last_fullsized":"2024-04-23T14:47:27.255710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:41:15.915795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367777+0000","last_change":"2024-04-23T14:45:59.236446+0000","last_active":"2024-04-23T14:46:58.367777+0000","last_peered":"2024-04-23T14:46:58.367777+0000","last_clean":"2024-04-23T14:46:58.367777+0000","last_became_active":"2024-04-23T14:45:59.236119+0000","last_became_peered":"2024-04-23T14:45:59.236119+0000","last_unstale":"2024-04-23T14:46:58.367777+0000","last_undegraded":"2024-04-23T14:46:58.367777+0000","last_fullsized":"2024-04-23T14:46:58.367777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:40.153626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.251722+0000","last_change":"2024-04-23T14:46:00.035577+0000","last_active":"2024-04-23T14:47:27.251722+0000","last_peered":"2024-04-23T14:47:27.251722+0000","last_clean":"2024-04-23T14:47:27.251722+0000","last_became_active":"2024-04-23T14:46:00.034929+0000","last_became_peered":"2024-04-23T14:46:00.034929+0000","last_unstale":"2024-04-23T14:47:27.251722+0000","last_undegraded":"2024-04-23T14:47:27.251722+0000","last_fullsized":"2024-04-23T14:47:27.251722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:22.844672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.440882+0000","last_change":"2024-04-23T14:46:58.440882+0000","last_active":"2024-04-23T14:46:58.440882+0000","last_peered":"2024-04-23T14:46:58.440882+0000","last_clean":"2024-04-23T14:46:58.440882+0000","last_became_active":"2024-04-23T14:46:58.436914+0000","last_became_peered":"2024-04-23T14:46:58.436914+0000","last_unstale":"2024-04-23T14:46:58.440882+0000","last_undegraded":"2024-04-23T14:46:58.440882+0000","last_fullsized":"2024-04-23T14:46:58.440882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:36.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.248604+0000","last_change":"2024-04-23T14:46:00.027934+0000","last_active":"2024-04-23T14:47:27.248604+0000","last_peered":"2024-04-23T14:47:27.248604+0000","last_clean":"2024-04-23T14:47:27.248604+0000","last_became_active":"2024-04-23T14:46:00.027571+0000","last_became_peered":"2024-04-23T14:46:00.027571+0000","last_unstale":"2024-04-23T14:47:27.248604+0000","last_undegraded":"2024-04-23T14:47:27.248604+0000","last_fullsized":"2024-04-23T14:47:27.248604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:54:27.890062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367884+0000","last_change":"2024-04-23T14:45:59.236967+0000","last_active":"2024-04-23T14:46:58.367884+0000","last_peered":"2024-04-23T14:46:58.367884+0000","last_clean":"2024-04-23T14:46:58.367884+0000","last_became_active":"2024-04-23T14:45:59.236612+0000","last_became_peered":"2024-04-23T14:45:59.236612+0000","last_unstale":"2024-04-23T14:46:58.367884+0000","last_undegraded":"2024-04-23T14:46:58.367884+0000","last_fullsized":"2024-04-23T14:46:58.367884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:58:09.422598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.242612+0000","last_change":"2024-04-23T14:46:00.009289+0000","last_active":"2024-04-23T14:47:27.242612+0000","last_peered":"2024-04-23T14:47:27.242612+0000","last_clean":"2024-04-23T14:47:27.242612+0000","last_became_active":"2024-04-23T14:46:00.008911+0000","last_became_peered":"2024-04-23T14:46:00.008911+0000","last_unstale":"2024-04-23T14:47:27.242612+0000","last_undegraded":"2024-04-23T14:47:27.242612+0000","last_fullsized":"2024-04-23T14:47:27.242612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:47:55.772442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.370270+0000","last_change":"2024-04-23T14:45:59.239494+0000","last_active":"2024-04-23T14:46:58.370270+0000","last_peered":"2024-04-23T14:46:58.370270+0000","last_clean":"2024-04-23T14:46:58.370270+0000","last_became_active":"2024-04-23T14:45:59.239157+0000","last_became_peered":"2024-04-23T14:45:59.239157+0000","last_unstale":"2024-04-23T14:46:58.370270+0000","last_undegraded":"2024-04-23T14:46:58.370270+0000","last_fullsized":"2024-04-23T14:46:58.370270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:38.275320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.243521+0000","last_change":"2024-04-23T14:46:00.029401+0000","last_active":"2024-04-23T14:47:27.243521+0000","last_peered":"2024-04-23T14:47:27.243521+0000","last_clean":"2024-04-23T14:47:27.243521+0000","last_became_active":"2024-04-23T14:46:00.029006+0000","last_became_peered":"2024-04-23T14:46:00.029006+0000","last_unstale":"2024-04-23T14:47:27.243521+0000","last_undegraded":"2024-04-23T14:47:27.243521+0000","last_fullsized":"2024-04-23T14:47:27.243521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:11.617083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.433003+0000","last_change":"2024-04-23T14:45:59.247636+0000","last_active":"2024-04-23T14:46:58.433003+0000","last_peered":"2024-04-23T14:46:58.433003+0000","last_clean":"2024-04-23T14:46:58.433003+0000","last_became_active":"2024-04-23T14:45:59.247231+0000","last_became_peered":"2024-04-23T14:45:59.247231+0000","last_unstale":"2024-04-23T14:46:58.433003+0000","last_undegraded":"2024-04-23T14:46:58.433003+0000","last_fullsized":"2024-04-23T14:46:58.433003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:54.291463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.236936+0000","last_change":"2024-04-23T14:46:00.010825+0000","last_active":"2024-04-23T14:47:27.236936+0000","last_peered":"2024-04-23T14:47:27.236936+0000","last_clean":"2024-04-23T14:47:27.236936+0000","last_became_active":"2024-04-23T14:46:00.010470+0000","last_became_peered":"2024-04-23T14:46:00.010470+0000","last_unstale":"2024-04-23T14:47:27.236936+0000","last_undegraded":"2024-04-23T14:47:27.236936+0000","last_fullsized":"2024-04-23T14:47:27.236936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:19.851825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.236271+0000","last_change":"2024-04-23T14:46:00.011967+0000","last_active":"2024-04-23T14:47:27.236271+0000","last_peered":"2024-04-23T14:47:27.236271+0000","last_clean":"2024-04-23T14:47:27.236271+0000","last_became_active":"2024-04-23T14:46:00.011580+0000","last_became_peered":"2024-04-23T14:46:00.011580+0000","last_unstale":"2024-04-23T14:47:27.236271+0000","last_undegraded":"2024-04-23T14:47:27.236271+0000","last_fullsized":"2024-04-23T14:47:27.236271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:25:38.909994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.436615+0000","last_change":"2024-04-23T14:46:58.436615+0000","last_active":"2024-04-23T14:46:58.436615+0000","last_peered":"2024-04-23T14:46:58.436615+0000","last_clean":"2024-04-23T14:46:58.436615+0000","last_became_active":"2024-04-23T14:46:58.436288+0000","last_became_peered":"2024-04-23T14:46:58.436288+0000","last_unstale":"2024-04-23T14:46:58.436615+0000","last_undegraded":"2024-04-23T14:46:58.436615+0000","last_fullsized":"2024-04-23T14:46:58.436615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:49.614004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367623+0000","last_change":"2024-04-23T14:46:00.029874+0000","last_active":"2024-04-23T14:46:58.367623+0000","last_peered":"2024-04-23T14:46:58.367623+0000","last_clean":"2024-04-23T14:46:58.367623+0000","last_became_active":"2024-04-23T14:46:00.029520+0000","last_became_peered":"2024-04-23T14:46:00.029520+0000","last_unstale":"2024-04-23T14:46:58.367623+0000","last_undegraded":"2024-04-23T14:46:58.367623+0000","last_fullsized":"2024-04-23T14:46:58.367623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:30.711566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.361370+0000","last_change":"2024-04-23T14:46:00.039545+0000","last_active":"2024-04-23T14:46:58.361370+0000","last_peered":"2024-04-23T14:46:58.361370+0000","last_clean":"2024-04-23T14:46:58.361370+0000","last_became_active":"2024-04-23T14:46:00.039193+0000","last_became_peered":"2024-04-23T14:46:00.039193+0000","last_unstale":"2024-04-23T14:46:58.361370+0000","last_undegraded":"2024-04-23T14:46:58.361370+0000","last_fullsized":"2024-04-23T14:46:58.361370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:18:13.528129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.432934+0000","last_change":"2024-04-23T14:46:00.036816+0000","last_active":"2024-04-23T14:46:58.432934+0000","last_peered":"2024-04-23T14:46:58.432934+0000","last_clean":"2024-04-23T14:46:58.432934+0000","last_became_active":"2024-04-23T14:46:00.036372+0000","last_became_peered":"2024-04-23T14:46:00.036372+0000","last_unstale":"2024-04-23T14:46:58.432934+0000","last_undegraded":"2024-04-23T14:46:58.432934+0000","last_fullsized":"2024-04-23T14:46:58.432934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:56.179724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.235570+0000","last_change":"2024-04-23T14:46:00.012058+0000","last_active":"2024-04-23T14:47:27.235570+0000","last_peered":"2024-04-23T14:47:27.235570+0000","last_clean":"2024-04-23T14:47:27.235570+0000","last_became_active":"2024-04-23T14:46:00.011595+0000","last_became_peered":"2024-04-23T14:46:00.011595+0000","last_unstale":"2024-04-23T14:47:27.235570+0000","last_undegraded":"2024-04-23T14:47:27.235570+0000","last_fullsized":"2024-04-23T14:47:27.235570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:28.341442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.368222+0000","last_change":"2024-04-23T14:46:00.011419+0000","last_active":"2024-04-23T14:46:58.368222+0000","last_peered":"2024-04-23T14:46:58.368222+0000","last_clean":"2024-04-23T14:46:58.368222+0000","last_became_active":"2024-04-23T14:46:00.011004+0000","last_became_peered":"2024-04-23T14:46:00.011004+0000","last_unstale":"2024-04-23T14:46:58.368222+0000","last_undegraded":"2024-04-23T14:46:58.368222+0000","last_fullsized":"2024-04-23T14:46:58.368222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:46:57.451648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.236187+0000","last_change":"2024-04-23T14:46:00.023583+0000","last_active":"2024-04-23T14:47:27.236187+0000","last_peered":"2024-04-23T14:47:27.236187+0000","last_clean":"2024-04-23T14:47:27.236187+0000","last_became_active":"2024-04-23T14:46:00.023195+0000","last_became_peered":"2024-04-23T14:46:00.023195+0000","last_unstale":"2024-04-23T14:47:27.236187+0000","last_undegraded":"2024-04-23T14:47:27.236187+0000","last_fullsized":"2024-04-23T14:47:27.236187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:39.992959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.369847+0000","last_change":"2024-04-23T14:46:00.014719+0000","last_active":"2024-04-23T14:46:58.369847+0000","last_peered":"2024-04-23T14:46:58.369847+0000","last_clean":"2024-04-23T14:46:58.369847+0000","last_became_active":"2024-04-23T14:46:00.014349+0000","last_became_peered":"2024-04-23T14:46:00.014349+0000","last_unstale":"2024-04-23T14:46:58.369847+0000","last_undegraded":"2024-04-23T14:46:58.369847+0000","last_fullsized":"2024-04-23T14:46:58.369847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:12.789316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.230619+0000","last_change":"2024-04-23T14:46:00.019162+0000","last_active":"2024-04-23T14:47:27.230619+0000","last_peered":"2024-04-23T14:47:27.230619+0000","last_clean":"2024-04-23T14:47:27.230619+0000","last_became_active":"2024-04-23T14:46:00.018814+0000","last_became_peered":"2024-04-23T14:46:00.018814+0000","last_unstale":"2024-04-23T14:47:27.230619+0000","last_undegraded":"2024-04-23T14:47:27.230619+0000","last_fullsized":"2024-04-23T14:47:27.230619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:10.741148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366494+0000","last_change":"2024-04-23T14:46:00.007254+0000","last_active":"2024-04-23T14:46:58.366494+0000","last_peered":"2024-04-23T14:46:58.366494+0000","last_clean":"2024-04-23T14:46:58.366494+0000","last_became_active":"2024-04-23T14:46:00.006826+0000","last_became_peered":"2024-04-23T14:46:00.006826+0000","last_unstale":"2024-04-23T14:46:58.366494+0000","last_undegraded":"2024-04-23T14:46:58.366494+0000","last_fullsized":"2024-04-23T14:46:58.366494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:29.155244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.432555+0000","last_change":"2024-04-23T14:46:00.039265+0000","last_active":"2024-04-23T14:46:58.432555+0000","last_peered":"2024-04-23T14:46:58.432555+0000","last_clean":"2024-04-23T14:46:58.432555+0000","last_became_active":"2024-04-23T14:46:00.038848+0000","last_became_peered":"2024-04-23T14:46:00.038848+0000","last_unstale":"2024-04-23T14:46:58.432555+0000","last_undegraded":"2024-04-23T14:46:58.432555+0000","last_fullsized":"2024-04-23T14:46:58.432555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:10.218655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.235347+0000","last_change":"2024-04-23T14:46:00.012134+0000","last_active":"2024-04-23T14:47:27.235347+0000","last_peered":"2024-04-23T14:47:27.235347+0000","last_clean":"2024-04-23T14:47:27.235347+0000","last_became_active":"2024-04-23T14:46:00.011796+0000","last_became_peered":"2024-04-23T14:46:00.011796+0000","last_unstale":"2024-04-23T14:47:27.235347+0000","last_undegraded":"2024-04-23T14:47:27.235347+0000","last_fullsized":"2024-04-23T14:47:27.235347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:41.671419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281220,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63233,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":498,"num_read_kb":320,"num_write":159,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1035,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":189981,"data_compressed":5556,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":958464,"data_stored":461672,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612644,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":1112363,"data_compressed":43424,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776171,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":461886,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":461784,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":1054409,"data_compressed":41572,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":461275,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1024000,"data_stored":520922,"data_compressed":7655,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":462546,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":520637,"data_compressed":7655,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":461586,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":462103,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":1053420,"data_compressed":41572,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34125,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34125,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34125,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":59854,"data_compressed":1852,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":59569,"data_compressed":1852,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":60810,"data_compressed":1852,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:47:30.772 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T14:47:30.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:30 smithi173 ceph-mon[32443]: pgmap v364: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-23T14:47:31.243 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:47:31.709 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:31 smithi083 ceph-mon[27867]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:47:31.709 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:31 smithi083 ceph-mon[27867]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-23T14:47:31.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:31 smithi203 ceph-mon[32800]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:47:31.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:31 smithi203 ceph-mon[32800]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-23T14:47:31.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:31 smithi173 ceph-mon[32443]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:47:31.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:31 smithi173 ceph-mon[32443]: pgmap v365: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-23T14:47:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.582+0000 7f2ffca00700 1 -- 172.21.15.83:0/3204848474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff810f4f0 msgr2=0x7f2ff81118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.582+0000 7f2ffca00700 1 --2- 172.21.15.83:0/3204848474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff810f4f0 0x7f2ff81118e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000b0a0 tx=0x7f2ff000b3b0 comp rx=0 tx=0).stop 2024-04-23T14:47:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.583+0000 7f2ffca00700 1 -- 172.21.15.83:0/3204848474 shutdown_connections 2024-04-23T14:47:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.583+0000 7f2ffca00700 1 --2- 172.21.15.83:0/3204848474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff810f4f0 0x7f2ff81118e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.583+0000 7f2ffca00700 1 --2- 172.21.15.83:0/3204848474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ff8074d20 0x7f2ff8075100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.583+0000 7f2ffca00700 1 --2- 172.21.15.83:0/3204848474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ff8075640 0x7f2ff810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.583+0000 7f2ffca00700 1 -- 172.21.15.83:0/3204848474 >> 172.21.15.83:0/3204848474 conn(0x7f2ff8079c50 msgr2=0x7f2ff807a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.583+0000 7f2ffca00700 1 -- 172.21.15.83:0/3204848474 shutdown_connections 2024-04-23T14:47:33.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.583+0000 7f2ffca00700 1 -- 172.21.15.83:0/3204848474 wait complete. 2024-04-23T14:47:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.584+0000 7f2ffca00700 1 Processor -- start 2024-04-23T14:47:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.584+0000 7f2ffca00700 1 -- start start 2024-04-23T14:47:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ffca00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ff8074d20 0x7f2ff819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ffca00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ff8075640 0x7f2ff819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ffca00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff810f4f0 0x7f2ff819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ffca00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff8115040 con 0x7f2ff810f4f0 2024-04-23T14:47:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ffca00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff8114ec0 con 0x7f2ff8074d20 2024-04-23T14:47:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ffca00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff81151c0 con 0x7f2ff8075640 2024-04-23T14:47:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ff6800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff810f4f0 0x7f2ff819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ff6800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff810f4f0 0x7f2ff819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47138/0 (socket says 172.21.15.83:47138) 2024-04-23T14:47:33.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ff6800700 1 -- 172.21.15.83:0/4202940374 learned_addr learned my addr 172.21.15.83:0/4202940374 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ff5e00700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ff8074d20 0x7f2ff819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ff5400700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ff8075640 0x7f2ff819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ff6800700 1 -- 172.21.15.83:0/4202940374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ff8075640 msgr2=0x7f2ff819b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ff6800700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ff8075640 0x7f2ff819b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ff6800700 1 -- 172.21.15.83:0/4202940374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ff8074d20 msgr2=0x7f2ff819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ff6800700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ff8074d20 0x7f2ff819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:33.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.585+0000 7f2ff6800700 1 -- 172.21.15.83:0/4202940374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff81a0460 con 0x7f2ff810f4f0 2024-04-23T14:47:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.586+0000 7f2ff6800700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff810f4f0 0x7f2ff819fd20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0006070 tx=0x7f2ff0004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.586+0000 7f2feea00700 1 -- 172.21.15.83:0/4202940374 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0021070 con 0x7f2ff810f4f0 2024-04-23T14:47:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.586+0000 7f2feea00700 1 -- 172.21.15.83:0/4202940374 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ff0003c00 con 0x7f2ff810f4f0 2024-04-23T14:47:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.586+0000 7f2ffca00700 1 -- 172.21.15.83:0/4202940374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff81a06f0 con 0x7f2ff810f4f0 2024-04-23T14:47:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.586+0000 7f2feea00700 1 -- 172.21.15.83:0/4202940374 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00046c0 con 0x7f2ff810f4f0 2024-04-23T14:47:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.586+0000 7f2ffca00700 1 -- 172.21.15.83:0/4202940374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff81a7fd0 con 0x7f2ff810f4f0 2024-04-23T14:47:33.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.588+0000 7f2ffca00700 1 -- 172.21.15.83:0/4202940374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff80697e0 con 0x7f2ff810f4f0 2024-04-23T14:47:33.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.589+0000 7f2feea00700 1 -- 172.21.15.83:0/4202940374 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2ff0007980 con 0x7f2ff810f4f0 2024-04-23T14:47:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.591+0000 7f2feea00700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2fdc0a3c40 0x7f2fdc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.591+0000 7f2feea00700 1 -- 172.21.15.83:0/4202940374 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2ff000f020 con 0x7f2ff810f4f0 2024-04-23T14:47:33.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.591+0000 7f2ff5e00700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2fdc0a3c40 0x7f2fdc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:33.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.592+0000 7f2ff5e00700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2fdc0a3c40 0x7f2fdc0a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009c60 tx=0x7f2fe8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:33.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.596+0000 7f2feea00700 1 -- 172.21.15.83:0/4202940374 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ff002a020 con 0x7f2ff810f4f0 2024-04-23T14:47:33.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.819+0000 7f2ffca00700 1 -- 172.21.15.83:0/4202940374 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ff81003d0 con 0x7f2fdc0a3c40 2024-04-23T14:47:33.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.828+0000 7f2feea00700 1 -- 172.21.15.83:0/4202940374 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259277 (secure 0 0 0) 0x7f2ff81003d0 con 0x7f2fdc0a3c40 2024-04-23T14:47:33.836 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:47:33.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.836+0000 7f2ffca00700 1 -- 172.21.15.83:0/4202940374 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2fdc0a3c40 msgr2=0x7f2fdc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.836+0000 7f2ffca00700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2fdc0a3c40 0x7f2fdc0a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009c60 tx=0x7f2fe8009260 comp rx=0 tx=0).stop 2024-04-23T14:47:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.836+0000 7f2ffca00700 1 -- 172.21.15.83:0/4202940374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff810f4f0 msgr2=0x7f2ff819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.836+0000 7f2ffca00700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff810f4f0 0x7f2ff819fd20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0006070 tx=0x7f2ff0004c60 comp rx=0 tx=0).stop 2024-04-23T14:47:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.836+0000 7f2ffca00700 1 -- 172.21.15.83:0/4202940374 shutdown_connections 2024-04-23T14:47:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.836+0000 7f2ffca00700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2fdc0a3c40 0x7f2fdc0a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.836+0000 7f2ffca00700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ff810f4f0 0x7f2ff819fd20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.836+0000 7f2ffca00700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ff8074d20 0x7f2ff819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.836+0000 7f2ffca00700 1 --2- 172.21.15.83:0/4202940374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ff8075640 0x7f2ff819b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.836+0000 7f2ffca00700 1 -- 172.21.15.83:0/4202940374 >> 172.21.15.83:0/4202940374 conn(0x7f2ff8079c50 msgr2=0x7f2ff80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:33.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.837+0000 7f2ffca00700 1 -- 172.21.15.83:0/4202940374 shutdown_connections 2024-04-23T14:47:33.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:33.837+0000 7f2ffca00700 1 -- 172.21.15.83:0/4202940374 wait complete. 2024-04-23T14:47:33.840 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-23T14:47:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:34 smithi083 ceph-mon[27867]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T14:47:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:34 smithi173 ceph-mon[32443]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T14:47:34.674 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-04-23T14:47:33.244139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654366,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":667,"num_read_kb":457,"num_write":349,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181771,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":782,"ondisk_log_size":782,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377492,"kb_used_data":11700,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124564332,"statfs":{"total":1151940427776,"available":1151553875968,"internally_reserved":0,"allocated":11980800,"data_stored":7495914,"data_compressed":182499,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":78,"num_read_kb":38,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004488"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.250184+0000","last_change":"2024-04-23T14:46:00.030934+0000","last_active":"2024-04-23T14:47:27.250184+0000","last_peered":"2024-04-23T14:47:27.250184+0000","last_clean":"2024-04-23T14:47:27.250184+0000","last_became_active":"2024-04-23T14:46:00.030595+0000","last_became_peered":"2024-04-23T14:46:00.030595+0000","last_unstale":"2024-04-23T14:47:27.250184+0000","last_undegraded":"2024-04-23T14:47:27.250184+0000","last_fullsized":"2024-04-23T14:47:27.250184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.814685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.247314+0000","last_change":"2024-04-23T14:46:00.023044+0000","last_active":"2024-04-23T14:47:27.247314+0000","last_peered":"2024-04-23T14:47:27.247314+0000","last_clean":"2024-04-23T14:47:27.247314+0000","last_became_active":"2024-04-23T14:46:00.022441+0000","last_became_peered":"2024-04-23T14:46:00.022441+0000","last_unstale":"2024-04-23T14:47:27.247314+0000","last_undegraded":"2024-04-23T14:47:27.247314+0000","last_fullsized":"2024-04-23T14:47:27.247314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:24:20.976312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.248181+0000","last_change":"2024-04-23T14:46:00.025397+0000","last_active":"2024-04-23T14:47:27.248181+0000","last_peered":"2024-04-23T14:47:27.248181+0000","last_clean":"2024-04-23T14:47:27.248181+0000","last_became_active":"2024-04-23T14:46:00.024477+0000","last_became_peered":"2024-04-23T14:46:00.024477+0000","last_unstale":"2024-04-23T14:47:27.248181+0000","last_undegraded":"2024-04-23T14:47:27.248181+0000","last_fullsized":"2024-04-23T14:47:27.248181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:16.002964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.222101+0000","last_change":"2024-04-23T14:46:00.013960+0000","last_active":"2024-04-23T14:47:27.222101+0000","last_peered":"2024-04-23T14:47:27.222101+0000","last_clean":"2024-04-23T14:47:27.222101+0000","last_became_active":"2024-04-23T14:46:00.013656+0000","last_became_peered":"2024-04-23T14:46:00.013656+0000","last_unstale":"2024-04-23T14:47:27.222101+0000","last_undegraded":"2024-04-23T14:47:27.222101+0000","last_fullsized":"2024-04-23T14:47:27.222101+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:45:59.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.251484+0000","last_change":"2024-04-23T14:46:00.032336+0000","last_active":"2024-04-23T14:47:27.251484+0000","last_peered":"2024-04-23T14:47:27.251484+0000","last_clean":"2024-04-23T14:47:27.251484+0000","last_became_active":"2024-04-23T14:46:00.032008+0000","last_became_peered":"2024-04-23T14:46:00.032008+0000","last_unstale":"2024-04-23T14:47:27.251484+0000","last_undegraded":"2024-04-23T14:47:27.251484+0000","last_fullsized":"2024-04-23T14:47:27.251484+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:13:05.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.242458+0000","last_change":"2024-04-23T14:46:00.015457+0000","last_active":"2024-04-23T14:47:27.242458+0000","last_peered":"2024-04-23T14:47:27.242458+0000","last_clean":"2024-04-23T14:47:27.242458+0000","last_became_active":"2024-04-23T14:46:00.015199+0000","last_became_peered":"2024-04-23T14:46:00.015199+0000","last_unstale":"2024-04-23T14:47:27.242458+0000","last_undegraded":"2024-04-23T14:47:27.242458+0000","last_fullsized":"2024-04-23T14:47:27.242458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:39.310993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.246446+0000","last_change":"2024-04-23T14:46:00.016554+0000","last_active":"2024-04-23T14:47:27.246446+0000","last_peered":"2024-04-23T14:47:27.246446+0000","last_clean":"2024-04-23T14:47:27.246446+0000","last_became_active":"2024-04-23T14:46:00.016242+0000","last_became_peered":"2024-04-23T14:46:00.016242+0000","last_unstale":"2024-04-23T14:47:27.246446+0000","last_undegraded":"2024-04-23T14:47:27.246446+0000","last_fullsized":"2024-04-23T14:47:27.246446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.596637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.241613+0000","last_change":"2024-04-23T14:46:58.373050+0000","last_active":"2024-04-23T14:47:27.241613+0000","last_peered":"2024-04-23T14:47:27.241613+0000","last_clean":"2024-04-23T14:47:27.241613+0000","last_became_active":"2024-04-23T14:46:58.372507+0000","last_became_peered":"2024-04-23T14:46:58.372507+0000","last_unstale":"2024-04-23T14:47:27.241613+0000","last_undegraded":"2024-04-23T14:47:27.241613+0000","last_fullsized":"2024-04-23T14:47:27.241613+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:47:08.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.243824+0000","last_change":"2024-04-23T14:46:00.005588+0000","last_active":"2024-04-23T14:47:27.243824+0000","last_peered":"2024-04-23T14:47:27.243824+0000","last_clean":"2024-04-23T14:47:27.243824+0000","last_became_active":"2024-04-23T14:46:00.005238+0000","last_became_peered":"2024-04-23T14:46:00.005238+0000","last_unstale":"2024-04-23T14:47:27.243824+0000","last_undegraded":"2024-04-23T14:47:27.243824+0000","last_fullsized":"2024-04-23T14:47:27.243824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:53:40.522929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.234078+0000","last_change":"2024-04-23T14:46:00.013773+0000","last_active":"2024-04-23T14:47:27.234078+0000","last_peered":"2024-04-23T14:47:27.234078+0000","last_clean":"2024-04-23T14:47:27.234078+0000","last_became_active":"2024-04-23T14:46:00.013390+0000","last_became_peered":"2024-04-23T14:46:00.013390+0000","last_unstale":"2024-04-23T14:47:27.234078+0000","last_undegraded":"2024-04-23T14:47:27.234078+0000","last_fullsized":"2024-04-23T14:47:27.234078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:44.352922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.242072+0000","last_change":"2024-04-23T14:46:00.013190+0000","last_active":"2024-04-23T14:47:27.242072+0000","last_peered":"2024-04-23T14:47:27.242072+0000","last_clean":"2024-04-23T14:47:27.242072+0000","last_became_active":"2024-04-23T14:46:00.012880+0000","last_became_peered":"2024-04-23T14:46:00.012880+0000","last_unstale":"2024-04-23T14:47:27.242072+0000","last_undegraded":"2024-04-23T14:47:27.242072+0000","last_fullsized":"2024-04-23T14:47:27.242072+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:57.611514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.235683+0000","last_change":"2024-04-23T14:46:00.013459+0000","last_active":"2024-04-23T14:47:27.235683+0000","last_peered":"2024-04-23T14:47:27.235683+0000","last_clean":"2024-04-23T14:47:27.235683+0000","last_became_active":"2024-04-23T14:46:00.013106+0000","last_became_peered":"2024-04-23T14:46:00.013106+0000","last_unstale":"2024-04-23T14:47:27.235683+0000","last_undegraded":"2024-04-23T14:47:27.235683+0000","last_fullsized":"2024-04-23T14:47:27.235683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:08:38.849062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.260146+0000","last_change":"2024-04-23T14:46:00.030491+0000","last_active":"2024-04-23T14:47:27.260146+0000","last_peered":"2024-04-23T14:47:27.260146+0000","last_clean":"2024-04-23T14:47:27.260146+0000","last_became_active":"2024-04-23T14:46:00.030024+0000","last_became_peered":"2024-04-23T14:46:00.030024+0000","last_unstale":"2024-04-23T14:47:27.260146+0000","last_undegraded":"2024-04-23T14:47:27.260146+0000","last_fullsized":"2024-04-23T14:47:27.260146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:31.289091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.247039+0000","last_change":"2024-04-23T14:46:00.022606+0000","last_active":"2024-04-23T14:47:27.247039+0000","last_peered":"2024-04-23T14:47:27.247039+0000","last_clean":"2024-04-23T14:47:27.247039+0000","last_became_active":"2024-04-23T14:46:00.022155+0000","last_became_peered":"2024-04-23T14:46:00.022155+0000","last_unstale":"2024-04-23T14:47:27.247039+0000","last_undegraded":"2024-04-23T14:47:27.247039+0000","last_fullsized":"2024-04-23T14:47:27.247039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:17:20.250283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.246256+0000","last_change":"2024-04-23T14:46:00.027193+0000","last_active":"2024-04-23T14:47:27.246256+0000","last_peered":"2024-04-23T14:47:27.246256+0000","last_clean":"2024-04-23T14:47:27.246256+0000","last_became_active":"2024-04-23T14:46:00.026211+0000","last_became_peered":"2024-04-23T14:46:00.026211+0000","last_unstale":"2024-04-23T14:47:27.246256+0000","last_undegraded":"2024-04-23T14:47:27.246256+0000","last_fullsized":"2024-04-23T14:47:27.246256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:52.722667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.255948+0000","last_change":"2024-04-23T14:46:00.029917+0000","last_active":"2024-04-23T14:47:27.255948+0000","last_peered":"2024-04-23T14:47:27.255948+0000","last_clean":"2024-04-23T14:47:27.255948+0000","last_became_active":"2024-04-23T14:46:00.029563+0000","last_became_peered":"2024-04-23T14:46:00.029563+0000","last_unstale":"2024-04-23T14:47:27.255948+0000","last_undegraded":"2024-04-23T14:47:27.255948+0000","last_fullsized":"2024-04-23T14:47:27.255948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:01.787185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.255603+0000","last_change":"2024-04-23T14:46:00.030910+0000","last_active":"2024-04-23T14:47:27.255603+0000","last_peered":"2024-04-23T14:47:27.255603+0000","last_clean":"2024-04-23T14:47:27.255603+0000","last_became_active":"2024-04-23T14:46:00.030555+0000","last_became_peered":"2024-04-23T14:46:00.030555+0000","last_unstale":"2024-04-23T14:47:27.255603+0000","last_undegraded":"2024-04-23T14:47:27.255603+0000","last_fullsized":"2024-04-23T14:47:27.255603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:43.701752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.247395+0000","last_change":"2024-04-23T14:46:00.017338+0000","last_active":"2024-04-23T14:47:27.247395+0000","last_peered":"2024-04-23T14:47:27.247395+0000","last_clean":"2024-04-23T14:47:27.247395+0000","last_became_active":"2024-04-23T14:46:00.015381+0000","last_became_peered":"2024-04-23T14:46:00.015381+0000","last_unstale":"2024-04-23T14:47:27.247395+0000","last_undegraded":"2024-04-23T14:47:27.247395+0000","last_fullsized":"2024-04-23T14:47:27.247395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:51:16.762588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.239405+0000","last_change":"2024-04-23T14:46:00.033055+0000","last_active":"2024-04-23T14:47:27.239405+0000","last_peered":"2024-04-23T14:47:27.239405+0000","last_clean":"2024-04-23T14:47:27.239405+0000","last_became_active":"2024-04-23T14:46:00.032763+0000","last_became_peered":"2024-04-23T14:46:00.032763+0000","last_unstale":"2024-04-23T14:47:27.239405+0000","last_undegraded":"2024-04-23T14:47:27.239405+0000","last_fullsized":"2024-04-23T14:47:27.239405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:43.793644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.368181+0000","last_change":"2024-04-23T14:45:59.232611+0000","last_active":"2024-04-23T14:46:58.368181+0000","last_peered":"2024-04-23T14:46:58.368181+0000","last_clean":"2024-04-23T14:46:58.368181+0000","last_became_active":"2024-04-23T14:45:59.231066+0000","last_became_peered":"2024-04-23T14:45:59.231066+0000","last_unstale":"2024-04-23T14:46:58.368181+0000","last_undegraded":"2024-04-23T14:46:58.368181+0000","last_fullsized":"2024-04-23T14:46:58.368181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:24.204134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.226454+0000","last_change":"2024-04-23T14:46:00.021532+0000","last_active":"2024-04-23T14:47:27.226454+0000","last_peered":"2024-04-23T14:47:27.226454+0000","last_clean":"2024-04-23T14:47:27.226454+0000","last_became_active":"2024-04-23T14:46:00.021125+0000","last_became_peered":"2024-04-23T14:46:00.021125+0000","last_unstale":"2024-04-23T14:47:27.226454+0000","last_undegraded":"2024-04-23T14:47:27.226454+0000","last_fullsized":"2024-04-23T14:47:27.226454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:03.072779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.364399+0000","last_change":"2024-04-23T14:45:59.248925+0000","last_active":"2024-04-23T14:46:58.364399+0000","last_peered":"2024-04-23T14:46:58.364399+0000","last_clean":"2024-04-23T14:46:58.364399+0000","last_became_active":"2024-04-23T14:45:59.248579+0000","last_became_peered":"2024-04-23T14:45:59.248579+0000","last_unstale":"2024-04-23T14:46:58.364399+0000","last_undegraded":"2024-04-23T14:46:58.364399+0000","last_fullsized":"2024-04-23T14:46:58.364399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:17.012610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.255232+0000","last_change":"2024-04-23T14:46:00.028419+0000","last_active":"2024-04-23T14:47:27.255232+0000","last_peered":"2024-04-23T14:47:27.255232+0000","last_clean":"2024-04-23T14:47:27.255232+0000","last_became_active":"2024-04-23T14:46:00.026300+0000","last_became_peered":"2024-04-23T14:46:00.026300+0000","last_unstale":"2024-04-23T14:47:27.255232+0000","last_undegraded":"2024-04-23T14:47:27.255232+0000","last_fullsized":"2024-04-23T14:47:27.255232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:21.784680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367277+0000","last_change":"2024-04-23T14:45:59.225701+0000","last_active":"2024-04-23T14:46:58.367277+0000","last_peered":"2024-04-23T14:46:58.367277+0000","last_clean":"2024-04-23T14:46:58.367277+0000","last_became_active":"2024-04-23T14:45:59.225290+0000","last_became_peered":"2024-04-23T14:45:59.225290+0000","last_unstale":"2024-04-23T14:46:58.367277+0000","last_undegraded":"2024-04-23T14:46:58.367277+0000","last_fullsized":"2024-04-23T14:46:58.367277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:26.877330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.255486+0000","last_change":"2024-04-23T14:46:00.035075+0000","last_active":"2024-04-23T14:47:27.255486+0000","last_peered":"2024-04-23T14:47:27.255486+0000","last_clean":"2024-04-23T14:47:27.255486+0000","last_became_active":"2024-04-23T14:46:00.034782+0000","last_became_peered":"2024-04-23T14:46:00.034782+0000","last_unstale":"2024-04-23T14:47:27.255486+0000","last_undegraded":"2024-04-23T14:47:27.255486+0000","last_fullsized":"2024-04-23T14:47:27.255486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:20.747643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.380160+0000","last_change":"2024-04-23T14:46:58.380160+0000","last_active":"2024-04-23T14:46:58.380160+0000","last_peered":"2024-04-23T14:46:58.380160+0000","last_clean":"2024-04-23T14:46:58.380160+0000","last_became_active":"2024-04-23T14:46:58.371279+0000","last_became_peered":"2024-04-23T14:46:58.371279+0000","last_unstale":"2024-04-23T14:46:58.380160+0000","last_undegraded":"2024-04-23T14:46:58.380160+0000","last_fullsized":"2024-04-23T14:46:58.380160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:19:44.330147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.240866+0000","last_change":"2024-04-23T14:46:58.372972+0000","last_active":"2024-04-23T14:47:27.240866+0000","last_peered":"2024-04-23T14:47:27.240866+0000","last_clean":"2024-04-23T14:47:27.240866+0000","last_became_active":"2024-04-23T14:46:58.372374+0000","last_became_peered":"2024-04-23T14:46:58.372374+0000","last_unstale":"2024-04-23T14:47:27.240866+0000","last_undegraded":"2024-04-23T14:47:27.240866+0000","last_fullsized":"2024-04-23T14:47:27.240866+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:15.638664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":127,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:29.417748+0000","last_change":"2024-04-23T14:46:58.376256+0000","last_active":"2024-04-23T14:47:29.417748+0000","last_peered":"2024-04-23T14:47:29.417748+0000","last_clean":"2024-04-23T14:47:29.417748+0000","last_became_active":"2024-04-23T14:46:58.371767+0000","last_became_peered":"2024-04-23T14:46:58.371767+0000","last_unstale":"2024-04-23T14:47:29.417748+0000","last_undegraded":"2024-04-23T14:47:29.417748+0000","last_fullsized":"2024-04-23T14:47:29.417748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:01:47.773660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.242377+0000","last_change":"2024-04-23T14:46:00.016700+0000","last_active":"2024-04-23T14:47:27.242377+0000","last_peered":"2024-04-23T14:47:27.242377+0000","last_clean":"2024-04-23T14:47:27.242377+0000","last_became_active":"2024-04-23T14:46:00.015779+0000","last_became_peered":"2024-04-23T14:46:00.015779+0000","last_unstale":"2024-04-23T14:47:27.242377+0000","last_undegraded":"2024-04-23T14:47:27.242377+0000","last_fullsized":"2024-04-23T14:47:27.242377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:19:59.431628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.358048+0000","last_change":"2024-04-23T14:45:59.229791+0000","last_active":"2024-04-23T14:46:58.358048+0000","last_peered":"2024-04-23T14:46:58.358048+0000","last_clean":"2024-04-23T14:46:58.358048+0000","last_became_active":"2024-04-23T14:45:59.229054+0000","last_became_peered":"2024-04-23T14:45:59.229054+0000","last_unstale":"2024-04-23T14:46:58.358048+0000","last_undegraded":"2024-04-23T14:46:58.358048+0000","last_fullsized":"2024-04-23T14:46:58.358048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:19:59.385719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.238196+0000","last_change":"2024-04-23T14:46:00.021858+0000","last_active":"2024-04-23T14:47:27.238196+0000","last_peered":"2024-04-23T14:47:27.238196+0000","last_clean":"2024-04-23T14:47:27.238196+0000","last_became_active":"2024-04-23T14:46:00.021450+0000","last_became_peered":"2024-04-23T14:46:00.021450+0000","last_unstale":"2024-04-23T14:47:27.238196+0000","last_undegraded":"2024-04-23T14:47:27.238196+0000","last_fullsized":"2024-04-23T14:47:27.238196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:00.270035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.431920+0000","last_change":"2024-04-23T14:45:59.238181+0000","last_active":"2024-04-23T14:46:58.431920+0000","last_peered":"2024-04-23T14:46:58.431920+0000","last_clean":"2024-04-23T14:46:58.431920+0000","last_became_active":"2024-04-23T14:45:59.237784+0000","last_became_peered":"2024-04-23T14:45:59.237784+0000","last_unstale":"2024-04-23T14:46:58.431920+0000","last_undegraded":"2024-04-23T14:46:58.431920+0000","last_fullsized":"2024-04-23T14:46:58.431920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:35.016190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.249771+0000","last_change":"2024-04-23T14:46:00.033814+0000","last_active":"2024-04-23T14:47:27.249771+0000","last_peered":"2024-04-23T14:47:27.249771+0000","last_clean":"2024-04-23T14:47:27.249771+0000","last_became_active":"2024-04-23T14:46:00.032128+0000","last_became_peered":"2024-04-23T14:46:00.032128+0000","last_unstale":"2024-04-23T14:47:27.249771+0000","last_undegraded":"2024-04-23T14:47:27.249771+0000","last_fullsized":"2024-04-23T14:47:27.249771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:32.067418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.369713+0000","last_change":"2024-04-23T14:45:59.228054+0000","last_active":"2024-04-23T14:46:58.369713+0000","last_peered":"2024-04-23T14:46:58.369713+0000","last_clean":"2024-04-23T14:46:58.369713+0000","last_became_active":"2024-04-23T14:45:59.227759+0000","last_became_peered":"2024-04-23T14:45:59.227759+0000","last_unstale":"2024-04-23T14:46:58.369713+0000","last_undegraded":"2024-04-23T14:46:58.369713+0000","last_fullsized":"2024-04-23T14:46:58.369713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:52.185284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.243065+0000","last_change":"2024-04-23T14:46:00.028634+0000","last_active":"2024-04-23T14:47:27.243065+0000","last_peered":"2024-04-23T14:47:27.243065+0000","last_clean":"2024-04-23T14:47:27.243065+0000","last_became_active":"2024-04-23T14:46:00.027768+0000","last_became_peered":"2024-04-23T14:46:00.027768+0000","last_unstale":"2024-04-23T14:47:27.243065+0000","last_undegraded":"2024-04-23T14:47:27.243065+0000","last_fullsized":"2024-04-23T14:47:27.243065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:29.869845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:29.417027+0000","last_change":"2024-04-23T14:45:59.229480+0000","last_active":"2024-04-23T14:47:29.417027+0000","last_peered":"2024-04-23T14:47:29.417027+0000","last_clean":"2024-04-23T14:47:29.417027+0000","last_became_active":"2024-04-23T14:45:59.229124+0000","last_became_peered":"2024-04-23T14:45:59.229124+0000","last_unstale":"2024-04-23T14:47:29.417027+0000","last_undegraded":"2024-04-23T14:47:29.417027+0000","last_fullsized":"2024-04-23T14:47:29.417027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:33.837779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.251005+0000","last_change":"2024-04-23T14:46:00.017753+0000","last_active":"2024-04-23T14:47:27.251005+0000","last_peered":"2024-04-23T14:47:27.251005+0000","last_clean":"2024-04-23T14:47:27.251005+0000","last_became_active":"2024-04-23T14:46:00.017370+0000","last_became_peered":"2024-04-23T14:46:00.017370+0000","last_unstale":"2024-04-23T14:47:27.251005+0000","last_undegraded":"2024-04-23T14:47:27.251005+0000","last_fullsized":"2024-04-23T14:47:27.251005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:30:35.460570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367839+0000","last_change":"2024-04-23T14:45:59.229361+0000","last_active":"2024-04-23T14:46:58.367839+0000","last_peered":"2024-04-23T14:46:58.367839+0000","last_clean":"2024-04-23T14:46:58.367839+0000","last_became_active":"2024-04-23T14:45:59.229043+0000","last_became_peered":"2024-04-23T14:45:59.229043+0000","last_unstale":"2024-04-23T14:46:58.367839+0000","last_undegraded":"2024-04-23T14:46:58.367839+0000","last_fullsized":"2024-04-23T14:46:58.367839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:09.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.244216+0000","last_change":"2024-04-23T14:46:00.030181+0000","last_active":"2024-04-23T14:47:27.244216+0000","last_peered":"2024-04-23T14:47:27.244216+0000","last_clean":"2024-04-23T14:47:27.244216+0000","last_became_active":"2024-04-23T14:46:00.029303+0000","last_became_peered":"2024-04-23T14:46:00.029303+0000","last_unstale":"2024-04-23T14:47:27.244216+0000","last_undegraded":"2024-04-23T14:47:27.244216+0000","last_fullsized":"2024-04-23T14:47:27.244216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:24:53.904501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.370426+0000","last_change":"2024-04-23T14:45:59.231270+0000","last_active":"2024-04-23T14:46:58.370426+0000","last_peered":"2024-04-23T14:46:58.370426+0000","last_clean":"2024-04-23T14:46:58.370426+0000","last_became_active":"2024-04-23T14:45:59.230914+0000","last_became_peered":"2024-04-23T14:45:59.230914+0000","last_unstale":"2024-04-23T14:46:58.370426+0000","last_undegraded":"2024-04-23T14:46:58.370426+0000","last_fullsized":"2024-04-23T14:46:58.370426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.422260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.245387+0000","last_change":"2024-04-23T14:46:00.011358+0000","last_active":"2024-04-23T14:47:27.245387+0000","last_peered":"2024-04-23T14:47:27.245387+0000","last_clean":"2024-04-23T14:47:27.245387+0000","last_became_active":"2024-04-23T14:46:00.011040+0000","last_became_peered":"2024-04-23T14:46:00.011040+0000","last_unstale":"2024-04-23T14:47:27.245387+0000","last_undegraded":"2024-04-23T14:47:27.245387+0000","last_fullsized":"2024-04-23T14:47:27.245387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:27.248844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'29","reported_seq":226,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:30.418719+0000","last_change":"2024-04-23T14:45:59.227195+0000","last_active":"2024-04-23T14:47:30.418719+0000","last_peered":"2024-04-23T14:47:30.418719+0000","last_clean":"2024-04-23T14:47:30.418719+0000","last_became_active":"2024-04-23T14:45:59.226874+0000","last_became_peered":"2024-04-23T14:45:59.226874+0000","last_unstale":"2024-04-23T14:47:30.418719+0000","last_undegraded":"2024-04-23T14:47:30.418719+0000","last_fullsized":"2024-04-23T14:47:30.418719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:18:40.203839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59090,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":178,"num_read_kb":146,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.231789+0000","last_change":"2024-04-23T14:46:00.010215+0000","last_active":"2024-04-23T14:47:27.231789+0000","last_peered":"2024-04-23T14:47:27.231789+0000","last_clean":"2024-04-23T14:47:27.231789+0000","last_became_active":"2024-04-23T14:46:00.009756+0000","last_became_peered":"2024-04-23T14:46:00.009756+0000","last_unstale":"2024-04-23T14:47:27.231789+0000","last_undegraded":"2024-04-23T14:47:27.231789+0000","last_fullsized":"2024-04-23T14:47:27.231789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.797966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.369370+0000","last_change":"2024-04-23T14:45:59.239258+0000","last_active":"2024-04-23T14:46:58.369370+0000","last_peered":"2024-04-23T14:46:58.369370+0000","last_clean":"2024-04-23T14:46:58.369370+0000","last_became_active":"2024-04-23T14:45:59.238899+0000","last_became_peered":"2024-04-23T14:45:59.238899+0000","last_unstale":"2024-04-23T14:46:58.369370+0000","last_undegraded":"2024-04-23T14:46:58.369370+0000","last_fullsized":"2024-04-23T14:46:58.369370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:45.866489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.256547+0000","last_change":"2024-04-23T14:46:00.036627+0000","last_active":"2024-04-23T14:47:27.256547+0000","last_peered":"2024-04-23T14:47:27.256547+0000","last_clean":"2024-04-23T14:47:27.256547+0000","last_became_active":"2024-04-23T14:46:00.035397+0000","last_became_peered":"2024-04-23T14:46:00.035397+0000","last_unstale":"2024-04-23T14:47:27.256547+0000","last_undegraded":"2024-04-23T14:47:27.256547+0000","last_fullsized":"2024-04-23T14:47:27.256547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:24:02.038813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.369141+0000","last_change":"2024-04-23T14:45:59.248241+0000","last_active":"2024-04-23T14:46:58.369141+0000","last_peered":"2024-04-23T14:46:58.369141+0000","last_clean":"2024-04-23T14:46:58.369141+0000","last_became_active":"2024-04-23T14:45:59.247895+0000","last_became_peered":"2024-04-23T14:45:59.247895+0000","last_unstale":"2024-04-23T14:46:58.369141+0000","last_undegraded":"2024-04-23T14:46:58.369141+0000","last_fullsized":"2024-04-23T14:46:58.369141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:01.964714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.252604+0000","last_change":"2024-04-23T14:46:58.433781+0000","last_active":"2024-04-23T14:47:27.252604+0000","last_peered":"2024-04-23T14:47:27.252604+0000","last_clean":"2024-04-23T14:47:27.252604+0000","last_became_active":"2024-04-23T14:46:58.433460+0000","last_became_peered":"2024-04-23T14:46:58.433460+0000","last_unstale":"2024-04-23T14:47:27.252604+0000","last_undegraded":"2024-04-23T14:47:27.252604+0000","last_fullsized":"2024-04-23T14:47:27.252604+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:44.149949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":114,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:29.501303+0000","last_change":"2024-04-23T14:45:59.227406+0000","last_active":"2024-04-23T14:47:29.501303+0000","last_peered":"2024-04-23T14:47:29.501303+0000","last_clean":"2024-04-23T14:47:29.501303+0000","last_became_active":"2024-04-23T14:45:59.227063+0000","last_became_peered":"2024-04-23T14:45:59.227063+0000","last_unstale":"2024-04-23T14:47:29.501303+0000","last_undegraded":"2024-04-23T14:47:29.501303+0000","last_fullsized":"2024-04-23T14:47:29.501303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:36.845854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.234534+0000","last_change":"2024-04-23T14:46:00.006333+0000","last_active":"2024-04-23T14:47:27.234534+0000","last_peered":"2024-04-23T14:47:27.234534+0000","last_clean":"2024-04-23T14:47:27.234534+0000","last_became_active":"2024-04-23T14:46:00.006024+0000","last_became_peered":"2024-04-23T14:46:00.006024+0000","last_unstale":"2024-04-23T14:47:27.234534+0000","last_undegraded":"2024-04-23T14:47:27.234534+0000","last_fullsized":"2024-04-23T14:47:27.234534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:49.389445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366404+0000","last_change":"2024-04-23T14:45:59.226394+0000","last_active":"2024-04-23T14:46:58.366404+0000","last_peered":"2024-04-23T14:46:58.366404+0000","last_clean":"2024-04-23T14:46:58.366404+0000","last_became_active":"2024-04-23T14:45:59.226088+0000","last_became_peered":"2024-04-23T14:45:59.226088+0000","last_unstale":"2024-04-23T14:46:58.366404+0000","last_undegraded":"2024-04-23T14:46:58.366404+0000","last_fullsized":"2024-04-23T14:46:58.366404+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:35.139520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.240024+0000","last_change":"2024-04-23T14:46:00.009262+0000","last_active":"2024-04-23T14:47:27.240024+0000","last_peered":"2024-04-23T14:47:27.240024+0000","last_clean":"2024-04-23T14:47:27.240024+0000","last_became_active":"2024-04-23T14:46:00.008865+0000","last_became_peered":"2024-04-23T14:46:00.008865+0000","last_unstale":"2024-04-23T14:47:27.240024+0000","last_undegraded":"2024-04-23T14:47:27.240024+0000","last_fullsized":"2024-04-23T14:47:27.240024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:27.662886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367049+0000","last_change":"2024-04-23T14:45:59.223479+0000","last_active":"2024-04-23T14:46:58.367049+0000","last_peered":"2024-04-23T14:46:58.367049+0000","last_clean":"2024-04-23T14:46:58.367049+0000","last_became_active":"2024-04-23T14:45:59.222997+0000","last_became_peered":"2024-04-23T14:45:59.222997+0000","last_unstale":"2024-04-23T14:46:58.367049+0000","last_undegraded":"2024-04-23T14:46:58.367049+0000","last_fullsized":"2024-04-23T14:46:58.367049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:29:58.026918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.248492+0000","last_change":"2024-04-23T14:46:58.434121+0000","last_active":"2024-04-23T14:47:27.248492+0000","last_peered":"2024-04-23T14:47:27.248492+0000","last_clean":"2024-04-23T14:47:27.248492+0000","last_became_active":"2024-04-23T14:46:58.433746+0000","last_became_peered":"2024-04-23T14:46:58.433746+0000","last_unstale":"2024-04-23T14:47:27.248492+0000","last_undegraded":"2024-04-23T14:47:27.248492+0000","last_fullsized":"2024-04-23T14:47:27.248492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:33:22.493604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367356+0000","last_change":"2024-04-23T14:45:59.237813+0000","last_active":"2024-04-23T14:46:58.367356+0000","last_peered":"2024-04-23T14:46:58.367356+0000","last_clean":"2024-04-23T14:46:58.367356+0000","last_became_active":"2024-04-23T14:45:59.237354+0000","last_became_peered":"2024-04-23T14:45:59.237354+0000","last_unstale":"2024-04-23T14:46:58.367356+0000","last_undegraded":"2024-04-23T14:46:58.367356+0000","last_fullsized":"2024-04-23T14:46:58.367356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:35:38.323059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.228885+0000","last_change":"2024-04-23T14:46:00.031956+0000","last_active":"2024-04-23T14:47:27.228885+0000","last_peered":"2024-04-23T14:47:27.228885+0000","last_clean":"2024-04-23T14:47:27.228885+0000","last_became_active":"2024-04-23T14:46:00.031666+0000","last_became_peered":"2024-04-23T14:46:00.031666+0000","last_unstale":"2024-04-23T14:47:27.228885+0000","last_undegraded":"2024-04-23T14:47:27.228885+0000","last_fullsized":"2024-04-23T14:47:27.228885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:17.346455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367284+0000","last_change":"2024-04-23T14:45:59.230633+0000","last_active":"2024-04-23T14:46:58.367284+0000","last_peered":"2024-04-23T14:46:58.367284+0000","last_clean":"2024-04-23T14:46:58.367284+0000","last_became_active":"2024-04-23T14:45:59.230300+0000","last_became_peered":"2024-04-23T14:45:59.230300+0000","last_unstale":"2024-04-23T14:46:58.367284+0000","last_undegraded":"2024-04-23T14:46:58.367284+0000","last_fullsized":"2024-04-23T14:46:58.367284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:41.170055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.249517+0000","last_change":"2024-04-23T14:46:00.030891+0000","last_active":"2024-04-23T14:47:27.249517+0000","last_peered":"2024-04-23T14:47:27.249517+0000","last_clean":"2024-04-23T14:47:27.249517+0000","last_became_active":"2024-04-23T14:46:00.030438+0000","last_became_peered":"2024-04-23T14:46:00.030438+0000","last_unstale":"2024-04-23T14:47:27.249517+0000","last_undegraded":"2024-04-23T14:47:27.249517+0000","last_fullsized":"2024-04-23T14:47:27.249517+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:43.038111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.234759+0000","last_change":"2024-04-23T14:46:00.013683+0000","last_active":"2024-04-23T14:47:27.234759+0000","last_peered":"2024-04-23T14:47:27.234759+0000","last_clean":"2024-04-23T14:47:27.234759+0000","last_became_active":"2024-04-23T14:46:00.013356+0000","last_became_peered":"2024-04-23T14:46:00.013356+0000","last_unstale":"2024-04-23T14:47:27.234759+0000","last_undegraded":"2024-04-23T14:47:27.234759+0000","last_fullsized":"2024-04-23T14:47:27.234759+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:32.866694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":110,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:29.500503+0000","last_change":"2024-04-23T14:45:59.252258+0000","last_active":"2024-04-23T14:47:29.500503+0000","last_peered":"2024-04-23T14:47:29.500503+0000","last_clean":"2024-04-23T14:47:29.500503+0000","last_became_active":"2024-04-23T14:45:59.251513+0000","last_became_peered":"2024-04-23T14:45:59.251513+0000","last_unstale":"2024-04-23T14:47:29.500503+0000","last_undegraded":"2024-04-23T14:47:29.500503+0000","last_fullsized":"2024-04-23T14:47:29.500503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:55:37.087807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.252125+0000","last_change":"2024-04-23T14:46:00.009468+0000","last_active":"2024-04-23T14:47:27.252125+0000","last_peered":"2024-04-23T14:47:27.252125+0000","last_clean":"2024-04-23T14:47:27.252125+0000","last_became_active":"2024-04-23T14:46:00.003118+0000","last_became_peered":"2024-04-23T14:46:00.003118+0000","last_unstale":"2024-04-23T14:47:27.252125+0000","last_undegraded":"2024-04-23T14:47:27.252125+0000","last_fullsized":"2024-04-23T14:47:27.252125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:20:24.488848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.435770+0000","last_change":"2024-04-23T14:45:59.225758+0000","last_active":"2024-04-23T14:46:58.435770+0000","last_peered":"2024-04-23T14:46:58.435770+0000","last_clean":"2024-04-23T14:46:58.435770+0000","last_became_active":"2024-04-23T14:45:59.225427+0000","last_became_peered":"2024-04-23T14:45:59.225427+0000","last_unstale":"2024-04-23T14:46:58.435770+0000","last_undegraded":"2024-04-23T14:46:58.435770+0000","last_fullsized":"2024-04-23T14:46:58.435770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:59.495348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366533+0000","last_change":"2024-04-23T14:44:39.684486+0000","last_active":"2024-04-23T14:46:58.366533+0000","last_peered":"2024-04-23T14:46:58.366533+0000","last_clean":"2024-04-23T14:46:58.366533+0000","last_became_active":"2024-04-23T14:44:39.684074+0000","last_became_peered":"2024-04-23T14:44:39.684074+0000","last_unstale":"2024-04-23T14:46:58.366533+0000","last_undegraded":"2024-04-23T14:46:58.366533+0000","last_fullsized":"2024-04-23T14:46:58.366533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_clean_scrub_stamp":"2024-04-23T14:40:41.333258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:11:16.865449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.249568+0000","last_change":"2024-04-23T14:46:00.025018+0000","last_active":"2024-04-23T14:47:27.249568+0000","last_peered":"2024-04-23T14:47:27.249568+0000","last_clean":"2024-04-23T14:47:27.249568+0000","last_became_active":"2024-04-23T14:46:00.024695+0000","last_became_peered":"2024-04-23T14:46:00.024695+0000","last_unstale":"2024-04-23T14:47:27.249568+0000","last_undegraded":"2024-04-23T14:47:27.249568+0000","last_fullsized":"2024-04-23T14:47:27.249568+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:00:31.375963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366613+0000","last_change":"2024-04-23T14:45:59.225849+0000","last_active":"2024-04-23T14:46:58.366613+0000","last_peered":"2024-04-23T14:46:58.366613+0000","last_clean":"2024-04-23T14:46:58.366613+0000","last_became_active":"2024-04-23T14:45:59.225402+0000","last_became_peered":"2024-04-23T14:45:59.225402+0000","last_unstale":"2024-04-23T14:46:58.366613+0000","last_undegraded":"2024-04-23T14:46:58.366613+0000","last_fullsized":"2024-04-23T14:46:58.366613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:10:42.217717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.377666+0000","last_change":"2024-04-23T14:46:58.377666+0000","last_active":"2024-04-23T14:46:58.377666+0000","last_peered":"2024-04-23T14:46:58.377666+0000","last_clean":"2024-04-23T14:46:58.377666+0000","last_became_active":"2024-04-23T14:46:58.370533+0000","last_became_peered":"2024-04-23T14:46:58.370533+0000","last_unstale":"2024-04-23T14:46:58.377666+0000","last_undegraded":"2024-04-23T14:46:58.377666+0000","last_fullsized":"2024-04-23T14:46:58.377666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:39:58.606732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.249812+0000","last_change":"2024-04-23T14:46:00.039566+0000","last_active":"2024-04-23T14:47:27.249812+0000","last_peered":"2024-04-23T14:47:27.249812+0000","last_clean":"2024-04-23T14:47:27.249812+0000","last_became_active":"2024-04-23T14:46:00.039236+0000","last_became_peered":"2024-04-23T14:46:00.039236+0000","last_unstale":"2024-04-23T14:47:27.249812+0000","last_undegraded":"2024-04-23T14:47:27.249812+0000","last_fullsized":"2024-04-23T14:47:27.249812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:44.028367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367735+0000","last_change":"2024-04-23T14:45:59.246192+0000","last_active":"2024-04-23T14:46:58.367735+0000","last_peered":"2024-04-23T14:46:58.367735+0000","last_clean":"2024-04-23T14:46:58.367735+0000","last_became_active":"2024-04-23T14:45:59.245873+0000","last_became_peered":"2024-04-23T14:45:59.245873+0000","last_unstale":"2024-04-23T14:46:58.367735+0000","last_undegraded":"2024-04-23T14:46:58.367735+0000","last_fullsized":"2024-04-23T14:46:58.367735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:22.915746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.241236+0000","last_change":"2024-04-23T14:46:00.029176+0000","last_active":"2024-04-23T14:47:27.241236+0000","last_peered":"2024-04-23T14:47:27.241236+0000","last_clean":"2024-04-23T14:47:27.241236+0000","last_became_active":"2024-04-23T14:46:00.028741+0000","last_became_peered":"2024-04-23T14:46:00.028741+0000","last_unstale":"2024-04-23T14:47:27.241236+0000","last_undegraded":"2024-04-23T14:47:27.241236+0000","last_fullsized":"2024-04-23T14:47:27.241236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:35:22.905348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.436084+0000","last_change":"2024-04-23T14:45:59.231490+0000","last_active":"2024-04-23T14:46:58.436084+0000","last_peered":"2024-04-23T14:46:58.436084+0000","last_clean":"2024-04-23T14:46:58.436084+0000","last_became_active":"2024-04-23T14:45:59.231143+0000","last_became_peered":"2024-04-23T14:45:59.231143+0000","last_unstale":"2024-04-23T14:46:58.436084+0000","last_undegraded":"2024-04-23T14:46:58.436084+0000","last_fullsized":"2024-04-23T14:46:58.436084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:47:26.398150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.244030+0000","last_change":"2024-04-23T14:46:00.035838+0000","last_active":"2024-04-23T14:47:27.244030+0000","last_peered":"2024-04-23T14:47:27.244030+0000","last_clean":"2024-04-23T14:47:27.244030+0000","last_became_active":"2024-04-23T14:46:00.034498+0000","last_became_peered":"2024-04-23T14:46:00.034498+0000","last_unstale":"2024-04-23T14:47:27.244030+0000","last_undegraded":"2024-04-23T14:47:27.244030+0000","last_fullsized":"2024-04-23T14:47:27.244030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:28:15.748503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.366294+0000","last_change":"2024-04-23T14:45:59.240817+0000","last_active":"2024-04-23T14:46:58.366294+0000","last_peered":"2024-04-23T14:46:58.366294+0000","last_clean":"2024-04-23T14:46:58.366294+0000","last_became_active":"2024-04-23T14:45:59.240557+0000","last_became_peered":"2024-04-23T14:45:59.240557+0000","last_unstale":"2024-04-23T14:46:58.366294+0000","last_undegraded":"2024-04-23T14:46:58.366294+0000","last_fullsized":"2024-04-23T14:46:58.366294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:03.099054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.197125+0000","last_change":"2024-04-23T14:46:00.009094+0000","last_active":"2024-04-23T14:47:27.197125+0000","last_peered":"2024-04-23T14:47:27.197125+0000","last_clean":"2024-04-23T14:47:27.197125+0000","last_became_active":"2024-04-23T14:46:00.008751+0000","last_became_peered":"2024-04-23T14:46:00.008751+0000","last_unstale":"2024-04-23T14:47:27.197125+0000","last_undegraded":"2024-04-23T14:47:27.197125+0000","last_fullsized":"2024-04-23T14:47:27.197125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:12.480737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.361163+0000","last_change":"2024-04-23T14:45:59.249882+0000","last_active":"2024-04-23T14:46:58.361163+0000","last_peered":"2024-04-23T14:46:58.361163+0000","last_clean":"2024-04-23T14:46:58.361163+0000","last_became_active":"2024-04-23T14:45:59.249072+0000","last_became_peered":"2024-04-23T14:45:59.249072+0000","last_unstale":"2024-04-23T14:46:58.361163+0000","last_undegraded":"2024-04-23T14:46:58.361163+0000","last_fullsized":"2024-04-23T14:46:58.361163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:04:16.573005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.255710+0000","last_change":"2024-04-23T14:46:00.030903+0000","last_active":"2024-04-23T14:47:27.255710+0000","last_peered":"2024-04-23T14:47:27.255710+0000","last_clean":"2024-04-23T14:47:27.255710+0000","last_became_active":"2024-04-23T14:46:00.030051+0000","last_became_peered":"2024-04-23T14:46:00.030051+0000","last_unstale":"2024-04-23T14:47:27.255710+0000","last_undegraded":"2024-04-23T14:47:27.255710+0000","last_fullsized":"2024-04-23T14:47:27.255710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:41:15.915795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367777+0000","last_change":"2024-04-23T14:45:59.236446+0000","last_active":"2024-04-23T14:46:58.367777+0000","last_peered":"2024-04-23T14:46:58.367777+0000","last_clean":"2024-04-23T14:46:58.367777+0000","last_became_active":"2024-04-23T14:45:59.236119+0000","last_became_peered":"2024-04-23T14:45:59.236119+0000","last_unstale":"2024-04-23T14:46:58.367777+0000","last_undegraded":"2024-04-23T14:46:58.367777+0000","last_fullsized":"2024-04-23T14:46:58.367777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:40.153626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.251722+0000","last_change":"2024-04-23T14:46:00.035577+0000","last_active":"2024-04-23T14:47:27.251722+0000","last_peered":"2024-04-23T14:47:27.251722+0000","last_clean":"2024-04-23T14:47:27.251722+0000","last_became_active":"2024-04-23T14:46:00.034929+0000","last_became_peered":"2024-04-23T14:46:00.034929+0000","last_unstale":"2024-04-23T14:47:27.251722+0000","last_undegraded":"2024-04-23T14:47:27.251722+0000","last_fullsized":"2024-04-23T14:47:27.251722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:22.844672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.440882+0000","last_change":"2024-04-23T14:46:58.440882+0000","last_active":"2024-04-23T14:46:58.440882+0000","last_peered":"2024-04-23T14:46:58.440882+0000","last_clean":"2024-04-23T14:46:58.440882+0000","last_became_active":"2024-04-23T14:46:58.436914+0000","last_became_peered":"2024-04-23T14:46:58.436914+0000","last_unstale":"2024-04-23T14:46:58.440882+0000","last_undegraded":"2024-04-23T14:46:58.440882+0000","last_fullsized":"2024-04-23T14:46:58.440882+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:36.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.248604+0000","last_change":"2024-04-23T14:46:00.027934+0000","last_active":"2024-04-23T14:47:27.248604+0000","last_peered":"2024-04-23T14:47:27.248604+0000","last_clean":"2024-04-23T14:47:27.248604+0000","last_became_active":"2024-04-23T14:46:00.027571+0000","last_became_peered":"2024-04-23T14:46:00.027571+0000","last_unstale":"2024-04-23T14:47:27.248604+0000","last_undegraded":"2024-04-23T14:47:27.248604+0000","last_fullsized":"2024-04-23T14:47:27.248604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:54:27.890062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.367884+0000","last_change":"2024-04-23T14:45:59.236967+0000","last_active":"2024-04-23T14:46:58.367884+0000","last_peered":"2024-04-23T14:46:58.367884+0000","last_clean":"2024-04-23T14:46:58.367884+0000","last_became_active":"2024-04-23T14:45:59.236612+0000","last_became_peered":"2024-04-23T14:45:59.236612+0000","last_unstale":"2024-04-23T14:46:58.367884+0000","last_undegraded":"2024-04-23T14:46:58.367884+0000","last_fullsized":"2024-04-23T14:46:58.367884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:58:09.422598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.242612+0000","last_change":"2024-04-23T14:46:00.009289+0000","last_active":"2024-04-23T14:47:27.242612+0000","last_peered":"2024-04-23T14:47:27.242612+0000","last_clean":"2024-04-23T14:47:27.242612+0000","last_became_active":"2024-04-23T14:46:00.008911+0000","last_became_peered":"2024-04-23T14:46:00.008911+0000","last_unstale":"2024-04-23T14:47:27.242612+0000","last_undegraded":"2024-04-23T14:47:27.242612+0000","last_fullsized":"2024-04-23T14:47:27.242612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:47:55.772442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.370270+0000","last_change":"2024-04-23T14:45:59.239494+0000","last_active":"2024-04-23T14:46:58.370270+0000","last_peered":"2024-04-23T14:46:58.370270+0000","last_clean":"2024-04-23T14:46:58.370270+0000","last_became_active":"2024-04-23T14:45:59.239157+0000","last_became_peered":"2024-04-23T14:45:59.239157+0000","last_unstale":"2024-04-23T14:46:58.370270+0000","last_undegraded":"2024-04-23T14:46:58.370270+0000","last_fullsized":"2024-04-23T14:46:58.370270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:38.275320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.243521+0000","last_change":"2024-04-23T14:46:00.029401+0000","last_active":"2024-04-23T14:47:27.243521+0000","last_peered":"2024-04-23T14:47:27.243521+0000","last_clean":"2024-04-23T14:47:27.243521+0000","last_became_active":"2024-04-23T14:46:00.029006+0000","last_became_peered":"2024-04-23T14:46:00.029006+0000","last_unstale":"2024-04-23T14:47:27.243521+0000","last_undegraded":"2024-04-23T14:47:27.243521+0000","last_fullsized":"2024-04-23T14:47:27.243521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:11.617083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:46:58.433003+0000","last_change":"2024-04-23T14:45:59.247636+0000","last_active":"2024-04-23T14:46:58.433003+0000","last_peered":"2024-04-23T14:46:58.433003+0000","last_clean":"2024-04-23T14:46:58.433003+0000","last_became_active":"2024-04-23T14:45:59.247231+0000","last_became_peered":"2024-04-23T14:45:59.247231+0000","last_unstale":"2024-04-23T14:46:58.433003+0000","last_undegraded":"2024-04-23T14:46:58.433003+0000","last_fullsized":"2024-04-23T14:46:58.433003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:54.291463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.236936+0000","last_change":"2024-04-23T14:46:00.010825+0000","last_active":"2024-04-23T14:47:27.236936+0000","last_peered":"2024-04-23T14:47:27.236936+0000","last_clean":"2024-04-23T14:47:27.236936+0000","last_became_active":"2024-04-23T14:46:00.010470+0000","last_became_peered":"2024-04-23T14:46:00.010470+0000","last_unstale":"2024-04-23T14:47:27.236936+0000","last_undegraded":"2024-04-23T14:47:27.236936+0000","last_fullsized":"2024-04-23T14:47:27.236936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:19.851825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.236271+0000","last_change":"2024-04-23T14:46:00.011967+0000","last_active":"2024-04-23T14:47:27.236271+0000","last_peered":"2024-04-23T14:47:27.236271+0000","last_clean":"2024-04-23T14:47:27.236271+0000","last_became_active":"2024-04-23T14:46:00.011580+0000","last_became_peered":"2024-04-23T14:46:00.011580+0000","last_unstale":"2024-04-23T14:47:27.236271+0000","last_undegraded":"2024-04-23T14:47:27.236271+0000","last_fullsized":"2024-04-23T14:47:27.236271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:25:38.909994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.235718+0000","last_change":"2024-04-23T14:46:58.436615+0000","last_active":"2024-04-23T14:47:27.235718+0000","last_peered":"2024-04-23T14:47:27.235718+0000","last_clean":"2024-04-23T14:47:27.235718+0000","last_became_active":"2024-04-23T14:46:58.436288+0000","last_became_peered":"2024-04-23T14:46:58.436288+0000","last_unstale":"2024-04-23T14:47:27.235718+0000","last_undegraded":"2024-04-23T14:47:27.235718+0000","last_fullsized":"2024-04-23T14:47:27.235718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:49.614004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.231943+0000","last_change":"2024-04-23T14:46:00.029874+0000","last_active":"2024-04-23T14:47:27.231943+0000","last_peered":"2024-04-23T14:47:27.231943+0000","last_clean":"2024-04-23T14:47:27.231943+0000","last_became_active":"2024-04-23T14:46:00.029520+0000","last_became_peered":"2024-04-23T14:46:00.029520+0000","last_unstale":"2024-04-23T14:47:27.231943+0000","last_undegraded":"2024-04-23T14:47:27.231943+0000","last_fullsized":"2024-04-23T14:47:27.231943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:30.711566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.257305+0000","last_change":"2024-04-23T14:46:00.039545+0000","last_active":"2024-04-23T14:47:27.257305+0000","last_peered":"2024-04-23T14:47:27.257305+0000","last_clean":"2024-04-23T14:47:27.257305+0000","last_became_active":"2024-04-23T14:46:00.039193+0000","last_became_peered":"2024-04-23T14:46:00.039193+0000","last_unstale":"2024-04-23T14:47:27.257305+0000","last_undegraded":"2024-04-23T14:47:27.257305+0000","last_fullsized":"2024-04-23T14:47:27.257305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:18:13.528129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.257277+0000","last_change":"2024-04-23T14:46:00.036816+0000","last_active":"2024-04-23T14:47:27.257277+0000","last_peered":"2024-04-23T14:47:27.257277+0000","last_clean":"2024-04-23T14:47:27.257277+0000","last_became_active":"2024-04-23T14:46:00.036372+0000","last_became_peered":"2024-04-23T14:46:00.036372+0000","last_unstale":"2024-04-23T14:47:27.257277+0000","last_undegraded":"2024-04-23T14:47:27.257277+0000","last_fullsized":"2024-04-23T14:47:27.257277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:56.179724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.235570+0000","last_change":"2024-04-23T14:46:00.012058+0000","last_active":"2024-04-23T14:47:27.235570+0000","last_peered":"2024-04-23T14:47:27.235570+0000","last_clean":"2024-04-23T14:47:27.235570+0000","last_became_active":"2024-04-23T14:46:00.011595+0000","last_became_peered":"2024-04-23T14:46:00.011595+0000","last_unstale":"2024-04-23T14:47:27.235570+0000","last_undegraded":"2024-04-23T14:47:27.235570+0000","last_fullsized":"2024-04-23T14:47:27.235570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:28.341442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.241193+0000","last_change":"2024-04-23T14:46:00.011419+0000","last_active":"2024-04-23T14:47:27.241193+0000","last_peered":"2024-04-23T14:47:27.241193+0000","last_clean":"2024-04-23T14:47:27.241193+0000","last_became_active":"2024-04-23T14:46:00.011004+0000","last_became_peered":"2024-04-23T14:46:00.011004+0000","last_unstale":"2024-04-23T14:47:27.241193+0000","last_undegraded":"2024-04-23T14:47:27.241193+0000","last_fullsized":"2024-04-23T14:47:27.241193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:46:57.451648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.236187+0000","last_change":"2024-04-23T14:46:00.023583+0000","last_active":"2024-04-23T14:47:27.236187+0000","last_peered":"2024-04-23T14:47:27.236187+0000","last_clean":"2024-04-23T14:47:27.236187+0000","last_became_active":"2024-04-23T14:46:00.023195+0000","last_became_peered":"2024-04-23T14:46:00.023195+0000","last_unstale":"2024-04-23T14:47:27.236187+0000","last_undegraded":"2024-04-23T14:47:27.236187+0000","last_fullsized":"2024-04-23T14:47:27.236187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:39.992959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.243898+0000","last_change":"2024-04-23T14:46:00.014719+0000","last_active":"2024-04-23T14:47:27.243898+0000","last_peered":"2024-04-23T14:47:27.243898+0000","last_clean":"2024-04-23T14:47:27.243898+0000","last_became_active":"2024-04-23T14:46:00.014349+0000","last_became_peered":"2024-04-23T14:46:00.014349+0000","last_unstale":"2024-04-23T14:47:27.243898+0000","last_undegraded":"2024-04-23T14:47:27.243898+0000","last_fullsized":"2024-04-23T14:47:27.243898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:12.789316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.230619+0000","last_change":"2024-04-23T14:46:00.019162+0000","last_active":"2024-04-23T14:47:27.230619+0000","last_peered":"2024-04-23T14:47:27.230619+0000","last_clean":"2024-04-23T14:47:27.230619+0000","last_became_active":"2024-04-23T14:46:00.018814+0000","last_became_peered":"2024-04-23T14:46:00.018814+0000","last_unstale":"2024-04-23T14:47:27.230619+0000","last_undegraded":"2024-04-23T14:47:27.230619+0000","last_fullsized":"2024-04-23T14:47:27.230619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:10.741148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.256318+0000","last_change":"2024-04-23T14:46:00.007254+0000","last_active":"2024-04-23T14:47:27.256318+0000","last_peered":"2024-04-23T14:47:27.256318+0000","last_clean":"2024-04-23T14:47:27.256318+0000","last_became_active":"2024-04-23T14:46:00.006826+0000","last_became_peered":"2024-04-23T14:46:00.006826+0000","last_unstale":"2024-04-23T14:47:27.256318+0000","last_undegraded":"2024-04-23T14:47:27.256318+0000","last_fullsized":"2024-04-23T14:47:27.256318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:29.155244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.259825+0000","last_change":"2024-04-23T14:46:00.039265+0000","last_active":"2024-04-23T14:47:27.259825+0000","last_peered":"2024-04-23T14:47:27.259825+0000","last_clean":"2024-04-23T14:47:27.259825+0000","last_became_active":"2024-04-23T14:46:00.038848+0000","last_became_peered":"2024-04-23T14:46:00.038848+0000","last_unstale":"2024-04-23T14:47:27.259825+0000","last_undegraded":"2024-04-23T14:47:27.259825+0000","last_fullsized":"2024-04-23T14:47:27.259825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:10.218655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T14:47:27.235347+0000","last_change":"2024-04-23T14:46:00.012134+0000","last_active":"2024-04-23T14:47:27.235347+0000","last_peered":"2024-04-23T14:47:27.235347+0000","last_clean":"2024-04-23T14:47:27.235347+0000","last_became_active":"2024-04-23T14:46:00.011796+0000","last_became_peered":"2024-04-23T14:46:00.011796+0000","last_unstale":"2024-04-23T14:47:27.235347+0000","last_undegraded":"2024-04-23T14:47:27.235347+0000","last_fullsized":"2024-04-23T14:47:27.235347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:41.671419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281220,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63764,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":531,"num_read_kb":338,"num_write":166,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1035,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":191292,"data_compressed":5556,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351009,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":461672,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612645,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":1112753,"data_compressed":43424,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776171,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":461886,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":461784,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":1054409,"data_compressed":41572,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495677,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":461275,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":521312,"data_compressed":7655,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":462546,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":520886,"data_compressed":7655,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":461586,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":462244,"data_compressed":5803,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1053561,"data_compressed":41572,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34125,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34125,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34125,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":60244,"data_compressed":1852,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":59818,"data_compressed":1852,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":61200,"data_compressed":1852,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:47:34.678 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-23T14:47:34.678 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-23T14:47:34.678 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-23T14:47:34.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-23T14:47:34.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:34 smithi203 ceph-mon[32800]: pgmap v366: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T14:47:35.173 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:47:35.564 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:35 smithi083 ceph-mon[27867]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:47:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:35 smithi173 ceph-mon[32443]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:47:35.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:35 smithi203 ceph-mon[32800]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:47:36.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:36 smithi083 ceph-mon[27867]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T14:47:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:36 smithi173 ceph-mon[32443]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T14:47:36.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:36 smithi203 ceph-mon[32800]: pgmap v367: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T14:47:37.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.549+0000 7f7a8e600700 1 -- 172.21.15.83:0/505662344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a88109380 msgr2=0x7f7a88109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:37.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.549+0000 7f7a8e600700 1 --2- 172.21.15.83:0/505662344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a88109380 0x7f7a88109760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009920 tx=0x7f7a78009c30 comp rx=0 tx=0).stop 2024-04-23T14:47:37.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.549+0000 7f7a8e600700 1 -- 172.21.15.83:0/505662344 shutdown_connections 2024-04-23T14:47:37.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.549+0000 7f7a8e600700 1 --2- 172.21.15.83:0/505662344 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8810e460 0x7f7a88116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:37.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.549+0000 7f7a8e600700 1 --2- 172.21.15.83:0/505662344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a88109380 0x7f7a88109760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:37.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.549+0000 7f7a8e600700 1 --2- 172.21.15.83:0/505662344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a88109d30 0x7f7a8810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:37.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.550+0000 7f7a8e600700 1 -- 172.21.15.83:0/505662344 >> 172.21.15.83:0/505662344 conn(0x7f7a880762c0 msgr2=0x7f7a88107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:37.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.550+0000 7f7a8e600700 1 -- 172.21.15.83:0/505662344 shutdown_connections 2024-04-23T14:47:37.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.550+0000 7f7a8e600700 1 -- 172.21.15.83:0/505662344 wait complete. 2024-04-23T14:47:37.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.551+0000 7f7a8e600700 1 Processor -- start 2024-04-23T14:47:37.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.551+0000 7f7a8e600700 1 -- start start 2024-04-23T14:47:37.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.551+0000 7f7a8e600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a88109380 0x7f7a881a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:37.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.551+0000 7f7a8e600700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a88109d30 0x7f7a881a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:37.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.551+0000 7f7a8e600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8810e460 0x7f7a8819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:37.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.551+0000 7f7a8e600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881194c0 con 0x7f7a88109380 2024-04-23T14:47:37.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.551+0000 7f7a8e600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7a88119340 con 0x7f7a8810e460 2024-04-23T14:47:37.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.551+0000 7f7a8e600700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7a88119640 con 0x7f7a88109d30 2024-04-23T14:47:37.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.551+0000 7f7a87400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a88109380 0x7f7a881a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.551+0000 7f7a87400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a88109380 0x7f7a881a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53788/0 (socket says 172.21.15.83:53788) 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.552+0000 7f7a87400700 1 -- 172.21.15.83:0/2470181507 learned_addr learned my addr 172.21.15.83:0/2470181507 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.552+0000 7f7a86a00700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a88109d30 0x7f7a881a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.552+0000 7f7a87e00700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8810e460 0x7f7a8819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.552+0000 7f7a87400700 1 -- 172.21.15.83:0/2470181507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a88109d30 msgr2=0x7f7a881a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.552+0000 7f7a87400700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a88109d30 0x7f7a881a5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.552+0000 7f7a87400700 1 -- 172.21.15.83:0/2470181507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8810e460 msgr2=0x7f7a8819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.552+0000 7f7a87400700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8810e460 0x7f7a8819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.552+0000 7f7a87400700 1 -- 172.21.15.83:0/2470181507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8819ffe0 con 0x7f7a88109380 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.552+0000 7f7a87400700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a88109380 0x7f7a881a5630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7a78000c00 tx=0x7f7a7800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.553+0000 7f7a7fe00700 1 -- 172.21.15.83:0/2470181507 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78027070 con 0x7f7a88109380 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.553+0000 7f7a8e600700 1 -- 172.21.15.83:0/2470181507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a881a0270 con 0x7f7a88109380 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.553+0000 7f7a7fe00700 1 -- 172.21.15.83:0/2470181507 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a78005660 con 0x7f7a88109380 2024-04-23T14:47:37.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.553+0000 7f7a7fe00700 1 -- 172.21.15.83:0/2470181507 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78004930 con 0x7f7a88109380 2024-04-23T14:47:37.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.553+0000 7f7a8e600700 1 -- 172.21.15.83:0/2470181507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a881ac490 con 0x7f7a88109380 2024-04-23T14:47:37.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.554+0000 7f7a8e600700 1 -- 172.21.15.83:0/2470181507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8810a860 con 0x7f7a88109380 2024-04-23T14:47:37.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.556+0000 7f7a7fe00700 1 -- 172.21.15.83:0/2470181507 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7a7802a070 con 0x7f7a88109380 2024-04-23T14:47:37.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.557+0000 7f7a7fe00700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7a6c0a3c40 0x7f7a6c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:37.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.557+0000 7f7a7fe00700 1 -- 172.21.15.83:0/2470181507 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f7a780d0df0 con 0x7f7a88109380 2024-04-23T14:47:37.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.557+0000 7f7a86a00700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7a6c0a3c40 0x7f7a6c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:37.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.558+0000 7f7a86a00700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7a6c0a3c40 0x7f7a6c0a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009750 tx=0x7f7a74009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:37.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.562+0000 7f7a7fe00700 1 -- 172.21.15.83:0/2470181507 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a7805eff0 con 0x7f7a88109380 2024-04-23T14:47:37.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.838+0000 7f7a8e600700 1 -- 172.21.15.83:0/2470181507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7a881a0ce0 con 0x7f7a88109380 2024-04-23T14:47:37.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.839+0000 7f7a7fe00700 1 -- 172.21.15.83:0/2470181507 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f7a78013030 con 0x7f7a88109380 2024-04-23T14:47:37.841 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:47:37.841 INFO:teuthology.orchestra.run.smithi083.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T14:47:37.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.841+0000 7f7a8e600700 1 -- 172.21.15.83:0/2470181507 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7a6c0a3c40 msgr2=0x7f7a6c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.842+0000 7f7a8e600700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7a6c0a3c40 0x7f7a6c0a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009750 tx=0x7f7a74009310 comp rx=0 tx=0).stop 2024-04-23T14:47:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.842+0000 7f7a8e600700 1 -- 172.21.15.83:0/2470181507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a88109380 msgr2=0x7f7a881a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.842+0000 7f7a8e600700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a88109380 0x7f7a881a5630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7a78000c00 tx=0x7f7a7800fa70 comp rx=0 tx=0).stop 2024-04-23T14:47:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.842+0000 7f7a8e600700 1 -- 172.21.15.83:0/2470181507 shutdown_connections 2024-04-23T14:47:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.842+0000 7f7a8e600700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7a6c0a3c40 0x7f7a6c0a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.842+0000 7f7a8e600700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a8810e460 0x7f7a8819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.842+0000 7f7a8e600700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a88109380 0x7f7a881a5630 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.842+0000 7f7a8e600700 1 --2- 172.21.15.83:0/2470181507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a88109d30 0x7f7a881a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.842+0000 7f7a8e600700 1 -- 172.21.15.83:0/2470181507 >> 172.21.15.83:0/2470181507 conn(0x7f7a880762c0 msgr2=0x7f7a88077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.842+0000 7f7a8e600700 1 -- 172.21.15.83:0/2470181507 shutdown_connections 2024-04-23T14:47:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:37.842+0000 7f7a8e600700 1 -- 172.21.15.83:0/2470181507 wait complete. 2024-04-23T14:47:38.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:38 smithi083 ceph-mon[27867]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-23T14:47:38.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:38 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2470181507' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T14:47:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:38 smithi173 ceph-mon[32443]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-23T14:47:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:38 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2470181507' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T14:47:38.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:38 smithi203 ceph-mon[32800]: pgmap v368: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-23T14:47:38.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:38 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2470181507' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T14:47:38.740 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-23T14:47:38.741 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:47:38.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.942+0000 7f834ee00640 1 -- 172.21.15.83:0/1633702213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8348103240 msgr2=0x7f83481036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:38.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.942+0000 7f834ee00640 1 --2- 172.21.15.83:0/1633702213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8348103240 0x7f83481036c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8330009860 tx=0x7f833002a340 comp rx=0 tx=0).stop 2024-04-23T14:47:38.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.942+0000 7f834ee00640 1 -- 172.21.15.83:0/1633702213 shutdown_connections 2024-04-23T14:47:38.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.942+0000 7f834ee00640 1 --2- 172.21.15.83:0/1633702213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348103c00 0x7f834810a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:38.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.942+0000 7f834ee00640 1 --2- 172.21.15.83:0/1633702213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8348103240 0x7f83481036c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:38.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.942+0000 7f834ee00640 1 --2- 172.21.15.83:0/1633702213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8348102050 0x7f8348102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:38.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.942+0000 7f834ee00640 1 -- 172.21.15.83:0/1633702213 >> 172.21.15.83:0/1633702213 conn(0x7f83480fd5e0 msgr2=0x7f83480ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:38.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.942+0000 7f834ee00640 1 -- 172.21.15.83:0/1633702213 shutdown_connections 2024-04-23T14:47:38.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.942+0000 7f834ee00640 1 -- 172.21.15.83:0/1633702213 wait complete. 2024-04-23T14:47:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.943+0000 7f834ee00640 1 Processor -- start 2024-04-23T14:47:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.944+0000 7f834ee00640 1 -- start start 2024-04-23T14:47:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.944+0000 7f834ee00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8348102050 0x7f8348075db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:38.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.944+0000 7f834ee00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8348103240 0x7f83480743f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.944+0000 7f834ee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348103c00 0x7f8348074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.944+0000 7f834ee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f834810dce0 con 0x7f8348103c00 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.944+0000 7f834ee00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f834810db60 con 0x7f8348103240 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.944+0000 7f834ee00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f834810de60 con 0x7f8348102050 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.944+0000 7f834ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348103c00 0x7f8348074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.944+0000 7f8347e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8348102050 0x7f8348075db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.944+0000 7f834ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348103c00 0x7f8348074930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53808/0 (socket says 172.21.15.83:53808) 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.944+0000 7f834ce00640 1 -- 172.21.15.83:0/1680725269 learned_addr learned my addr 172.21.15.83:0/1680725269 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.944+0000 7f833f400640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8348103240 0x7f83480743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.945+0000 7f8347e00640 1 -- 172.21.15.83:0/1680725269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8348103240 msgr2=0x7f83480743f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.945+0000 7f8347e00640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8348103240 0x7f83480743f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.945+0000 7f8347e00640 1 -- 172.21.15.83:0/1680725269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348103c00 msgr2=0x7f8348074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:38.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.945+0000 7f8347e00640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348103c00 0x7f8348074930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.945+0000 7f8347e00640 1 -- 172.21.15.83:0/1680725269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83480751c0 con 0x7f8348102050 2024-04-23T14:47:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.945+0000 7f834ce00640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348103c00 0x7f8348074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.945+0000 7f8347e00640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8348102050 0x7f8348075db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f833800e9c0 tx=0x7f833800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.945+0000 7f8345600640 1 -- 172.21.15.83:0/1680725269 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833800ccc0 con 0x7f8348102050 2024-04-23T14:47:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.946+0000 7f8345600640 1 -- 172.21.15.83:0/1680725269 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83380035f0 con 0x7f8348102050 2024-04-23T14:47:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.946+0000 7f8345600640 1 -- 172.21.15.83:0/1680725269 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833800ce60 con 0x7f8348102050 2024-04-23T14:47:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.946+0000 7f834ee00640 1 -- 172.21.15.83:0/1680725269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83481a2440 con 0x7f8348102050 2024-04-23T14:47:38.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.946+0000 7f834ee00640 1 -- 172.21.15.83:0/1680725269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83481a2910 con 0x7f8348102050 2024-04-23T14:47:38.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.947+0000 7f834ee00640 1 -- 172.21.15.83:0/1680725269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8348069270 con 0x7f8348102050 2024-04-23T14:47:38.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.949+0000 7f8345600640 1 -- 172.21.15.83:0/1680725269 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f83380037a0 con 0x7f8348102050 2024-04-23T14:47:38.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.950+0000 7f8345600640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f832c0a3bb0 0x7f832c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:38.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.950+0000 7f8345600640 1 -- 172.21.15.83:0/1680725269 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f83380d0830 con 0x7f8348102050 2024-04-23T14:47:38.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.950+0000 7f833f400640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f832c0a3bb0 0x7f832c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:38.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.951+0000 7f833f400640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f832c0a3bb0 0x7f832c0a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f833002a930 tx=0x7f8330030570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:38.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:38.953+0000 7f8345600640 1 -- 172.21.15.83:0/1680725269 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f833809d1d0 con 0x7f8348102050 2024-04-23T14:47:39.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.160+0000 7f834ee00640 1 -- 172.21.15.83:0/1680725269 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f834802d6d0 con 0x7f8348102050 2024-04-23T14:47:39.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.162+0000 7f8345600640 1 -- 172.21.15.83:0/1680725269 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22699 (secure 0 0 0) 0x7f833809cad0 con 0x7f8348102050 2024-04-23T14:47:39.164 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:47:39.164 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:46:58.351541+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:47:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.165+0000 7f834ee00640 1 -- 172.21.15.83:0/1680725269 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f832c0a3bb0 msgr2=0x7f832c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.165+0000 7f834ee00640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f832c0a3bb0 0x7f832c0a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f833002a930 tx=0x7f8330030570 comp rx=0 tx=0).stop 2024-04-23T14:47:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.165+0000 7f834ee00640 1 -- 172.21.15.83:0/1680725269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8348102050 msgr2=0x7f8348075db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.165+0000 7f834ee00640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8348102050 0x7f8348075db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f833800e9c0 tx=0x7f833800ee80 comp rx=0 tx=0).stop 2024-04-23T14:47:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.165+0000 7f834ee00640 1 -- 172.21.15.83:0/1680725269 shutdown_connections 2024-04-23T14:47:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.165+0000 7f834ee00640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f832c0a3bb0 0x7f832c0a6070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.165+0000 7f834ee00640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8348103c00 0x7f8348074930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.165+0000 7f834ee00640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8348103240 0x7f83480743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.165+0000 7f834ee00640 1 --2- 172.21.15.83:0/1680725269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8348102050 0x7f8348075db0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.165+0000 7f834ee00640 1 -- 172.21.15.83:0/1680725269 >> 172.21.15.83:0/1680725269 conn(0x7f83480fd5e0 msgr2=0x7f83481065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.166+0000 7f834ee00640 1 -- 172.21.15.83:0/1680725269 shutdown_connections 2024-04-23T14:47:39.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.166+0000 7f834ee00640 1 -- 172.21.15.83:0/1680725269 wait complete. 2024-04-23T14:47:39.187 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:40:41.241992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:45:57.341602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:45:58.654798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:47:39.187 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:47:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.387+0000 7f7fa5e00640 1 -- 172.21.15.83:0/1115583686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fa0109730 msgr2=0x7f7fa0109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.387+0000 7f7fa5e00640 1 --2- 172.21.15.83:0/1115583686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fa0109730 0x7f7fa0109bb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009900 tx=0x7f7f8c02a3f0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.387+0000 7f7fa5e00640 1 -- 172.21.15.83:0/1115583686 shutdown_connections 2024-04-23T14:47:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.387+0000 7f7fa5e00640 1 --2- 172.21.15.83:0/1115583686 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fa010a0f0 0x7f7fa0110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.387+0000 7f7fa5e00640 1 --2- 172.21.15.83:0/1115583686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fa0109730 0x7f7fa0109bb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.387+0000 7f7fa5e00640 1 --2- 172.21.15.83:0/1115583686 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa0108540 0x7f7fa0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.387+0000 7f7fa5e00640 1 -- 172.21.15.83:0/1115583686 >> 172.21.15.83:0/1115583686 conn(0x7f7fa00774d0 msgr2=0x7f7fa0079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.387+0000 7f7fa5e00640 1 -- 172.21.15.83:0/1115583686 shutdown_connections 2024-04-23T14:47:39.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.387+0000 7f7fa5e00640 1 -- 172.21.15.83:0/1115583686 wait complete. 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.388+0000 7f7fa5e00640 1 Processor -- start 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.388+0000 7f7fa5e00640 1 -- start start 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.388+0000 7f7fa5e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fa0108540 0x7f7fa019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.388+0000 7f7fa5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fa0109730 0x7f7fa019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.388+0000 7f7fa5e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa010a0f0 0x7f7fa01a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7fa5e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa01141d0 con 0x7f7fa0109730 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7fa5e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0114050 con 0x7f7fa0108540 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7fa5e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0114350 con 0x7f7fa010a0f0 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7f9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fa0109730 0x7f7fa019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7f9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fa0109730 0x7f7fa019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53822/0 (socket says 172.21.15.83:53822) 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7f9fe00640 1 -- 172.21.15.83:0/2638692217 learned_addr learned my addr 172.21.15.83:0/2638692217 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7fa4a00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fa0108540 0x7f7fa019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:39.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7fa5400640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa010a0f0 0x7f7fa01a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7f9fe00640 1 -- 172.21.15.83:0/2638692217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa010a0f0 msgr2=0x7f7fa01a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7f9fe00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa010a0f0 0x7f7fa01a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7f9fe00640 1 -- 172.21.15.83:0/2638692217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fa0108540 msgr2=0x7f7fa019e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7f9fe00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fa0108540 0x7f7fa019e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7f9fe00640 1 -- 172.21.15.83:0/2638692217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa01a63e0 con 0x7f7fa0109730 2024-04-23T14:47:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.389+0000 7f7fa4a00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fa0108540 0x7f7fa019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.390+0000 7f7f9fe00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fa0109730 0x7f7fa019eda0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009230 tx=0x7f7f8c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.390+0000 7f7f9d600640 1 -- 172.21.15.83:0/2638692217 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c03f030 con 0x7f7fa0109730 2024-04-23T14:47:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.390+0000 7f7f9d600640 1 -- 172.21.15.83:0/2638692217 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f8c003ce0 con 0x7f7fa0109730 2024-04-23T14:47:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.390+0000 7f7fa5e00640 1 -- 172.21.15.83:0/2638692217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa01a6670 con 0x7f7fa0109730 2024-04-23T14:47:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.390+0000 7f7f9d600640 1 -- 172.21.15.83:0/2638692217 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c035070 con 0x7f7fa0109730 2024-04-23T14:47:39.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.390+0000 7f7fa5e00640 1 -- 172.21.15.83:0/2638692217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa01a6b60 con 0x7f7fa0109730 2024-04-23T14:47:39.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.391+0000 7f7fa5e00640 1 -- 172.21.15.83:0/2638692217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa010f860 con 0x7f7fa0109730 2024-04-23T14:47:39.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.393+0000 7f7f9d600640 1 -- 172.21.15.83:0/2638692217 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7f8c042030 con 0x7f7fa0109730 2024-04-23T14:47:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.395+0000 7f7f9d600640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f880a3bb0 0x7f7f880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.395+0000 7f7f9d600640 1 -- 172.21.15.83:0/2638692217 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f7f8c0f0d70 con 0x7f7fa0109730 2024-04-23T14:47:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.396+0000 7f7fa4a00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f880a3bb0 0x7f7f880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:39.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.396+0000 7f7fa4a00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f880a3bb0 0x7f7f880a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0109590 tx=0x7f7f9400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:39.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.399+0000 7f7f9d600640 1 -- 172.21.15.83:0/2638692217 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f8c0bd0c0 con 0x7f7fa0109730 2024-04-23T14:47:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:39 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1680725269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:47:39.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.601+0000 7f7fa5e00640 1 -- 172.21.15.83:0/2638692217 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7fa0062c00 con 0x7f7fa0109730 2024-04-23T14:47:39.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.602+0000 7f7f9d600640 1 -- 172.21.15.83:0/2638692217 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f7f8c0c13c0 con 0x7f7fa0109730 2024-04-23T14:47:39.603 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-23T14:47:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.604+0000 7f7fa5e00640 1 -- 172.21.15.83:0/2638692217 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f880a3bb0 msgr2=0x7f7f880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.604+0000 7f7fa5e00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f880a3bb0 0x7f7f880a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0109590 tx=0x7f7f9400b390 comp rx=0 tx=0).stop 2024-04-23T14:47:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.604+0000 7f7fa5e00640 1 -- 172.21.15.83:0/2638692217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fa0109730 msgr2=0x7f7fa019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.604+0000 7f7fa5e00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fa0109730 0x7f7fa019eda0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c009230 tx=0x7f7f8c004360 comp rx=0 tx=0).stop 2024-04-23T14:47:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.604+0000 7f7fa5e00640 1 -- 172.21.15.83:0/2638692217 shutdown_connections 2024-04-23T14:47:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.604+0000 7f7fa5e00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7f880a3bb0 0x7f7f880a6070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.604+0000 7f7fa5e00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7fa010a0f0 0x7f7fa01a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.604+0000 7f7fa5e00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fa0109730 0x7f7fa019eda0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.604+0000 7f7fa5e00640 1 --2- 172.21.15.83:0/2638692217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7fa0108540 0x7f7fa019e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.604+0000 7f7fa5e00640 1 -- 172.21.15.83:0/2638692217 >> 172.21.15.83:0/2638692217 conn(0x7f7fa00774d0 msgr2=0x7f7fa010ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:39.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.604+0000 7f7fa5e00640 1 -- 172.21.15.83:0/2638692217 shutdown_connections 2024-04-23T14:47:39.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.605+0000 7f7fa5e00640 1 -- 172.21.15.83:0/2638692217 wait complete. 2024-04-23T14:47:39.625 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:47:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:39 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1680725269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:47:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:39 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1680725269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:47:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.828+0000 7f6719600640 1 -- 172.21.15.83:0/3667102908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6714101170 msgr2=0x7f67141015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.828+0000 7f6719600640 1 --2- 172.21.15.83:0/3667102908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6714101170 0x7f67141015d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6700009860 tx=0x7f670002a3d0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.828+0000 7f6719600640 1 -- 172.21.15.83:0/3667102908 shutdown_connections 2024-04-23T14:47:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.828+0000 7f6719600640 1 --2- 172.21.15.83:0/3667102908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6714101b10 0x7f6714101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.828+0000 7f6719600640 1 --2- 172.21.15.83:0/3667102908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6714101170 0x7f67141015d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.828+0000 7f6719600640 1 --2- 172.21.15.83:0/3667102908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6714100810 0x7f6714100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.829+0000 7f6719600640 1 -- 172.21.15.83:0/3667102908 >> 172.21.15.83:0/3667102908 conn(0x7f67140fc3b0 msgr2=0x7f67140fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.829+0000 7f6719600640 1 -- 172.21.15.83:0/3667102908 shutdown_connections 2024-04-23T14:47:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.829+0000 7f6719600640 1 -- 172.21.15.83:0/3667102908 wait complete. 2024-04-23T14:47:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.830+0000 7f6719600640 1 Processor -- start 2024-04-23T14:47:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.830+0000 7f6719600640 1 -- start start 2024-04-23T14:47:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.830+0000 7f6719600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6714100810 0x7f671419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.830+0000 7f6719600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6714101170 0x7f671419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.830+0000 7f6719600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6714101b10 0x7f67141a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.830+0000 7f6719600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f671410fdd0 con 0x7f6714101170 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.830+0000 7f6719600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f671410fc50 con 0x7f6714101b10 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.830+0000 7f6719600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f671410ff50 con 0x7f6714100810 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.830+0000 7f6713400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6714101170 0x7f671419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.830+0000 7f6713400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6714101170 0x7f671419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53840/0 (socket says 172.21.15.83:53840) 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.830+0000 7f6713400640 1 -- 172.21.15.83:0/21527980 learned_addr learned my addr 172.21.15.83:0/21527980 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.830+0000 7f6718c00640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6714101b10 0x7f67141a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.831+0000 7f6713e00640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6714100810 0x7f671419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.831+0000 7f6713400640 1 -- 172.21.15.83:0/21527980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6714100810 msgr2=0x7f671419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.831+0000 7f6713400640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6714100810 0x7f671419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.831+0000 7f6713400640 1 -- 172.21.15.83:0/21527980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6714101b10 msgr2=0x7f67141a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.831+0000 7f6713400640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6714101b10 0x7f67141a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.831+0000 7f6713400640 1 -- 172.21.15.83:0/21527980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67141a20a0 con 0x7f6714101170 2024-04-23T14:47:39.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.831+0000 7f6713400640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6714101170 0x7f671419a9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6700009980 tx=0x7f67000052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.831+0000 7f6710c00640 1 -- 172.21.15.83:0/21527980 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700040070 con 0x7f6714101170 2024-04-23T14:47:39.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.832+0000 7f6710c00640 1 -- 172.21.15.83:0/21527980 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6700030780 con 0x7f6714101170 2024-04-23T14:47:39.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.832+0000 7f6710c00640 1 -- 172.21.15.83:0/21527980 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700004a40 con 0x7f6714101170 2024-04-23T14:47:39.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.832+0000 7f6719600640 1 -- 172.21.15.83:0/21527980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67141a2330 con 0x7f6714101170 2024-04-23T14:47:39.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.832+0000 7f6719600640 1 -- 172.21.15.83:0/21527980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67141a27d0 con 0x7f6714101170 2024-04-23T14:47:39.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.833+0000 7f6719600640 1 -- 172.21.15.83:0/21527980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f671410b6e0 con 0x7f6714101170 2024-04-23T14:47:39.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.835+0000 7f6710c00640 1 -- 172.21.15.83:0/21527980 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6700043070 con 0x7f6714101170 2024-04-23T14:47:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.836+0000 7f6710c00640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f66f80a3c30 0x7f66f80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:39.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.836+0000 7f6710c00640 1 -- 172.21.15.83:0/21527980 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f67000f1220 con 0x7f6714101170 2024-04-23T14:47:39.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.836+0000 7f6713e00640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f66f80a3c30 0x7f66f80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:39.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.837+0000 7f6713e00640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f66f80a3c30 0x7f66f80a60f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6704005fd0 tx=0x7f6704009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:39.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:39.841+0000 7f6710c00640 1 -- 172.21.15.83:0/21527980 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67000bdbc0 con 0x7f6714101170 2024-04-23T14:47:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.041+0000 7f6719600640 1 -- 172.21.15.83:0/21527980 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6714062c00 con 0x7f6714101170 2024-04-23T14:47:40.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.042+0000 7f6710c00640 1 -- 172.21.15.83:0/21527980 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f67000bd4c0 con 0x7f6714101170 2024-04-23T14:47:40.043 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-23T14:47:40.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.044+0000 7f6719600640 1 -- 172.21.15.83:0/21527980 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f66f80a3c30 msgr2=0x7f66f80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:40.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.044+0000 7f6719600640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f66f80a3c30 0x7f66f80a60f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6704005fd0 tx=0x7f6704009260 comp rx=0 tx=0).stop 2024-04-23T14:47:40.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.044+0000 7f6719600640 1 -- 172.21.15.83:0/21527980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6714101170 msgr2=0x7f671419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:40.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.044+0000 7f6719600640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6714101170 0x7f671419a9c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6700009980 tx=0x7f67000052f0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.045+0000 7f6719600640 1 -- 172.21.15.83:0/21527980 shutdown_connections 2024-04-23T14:47:40.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.045+0000 7f6719600640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f66f80a3c30 0x7f66f80a60f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.045+0000 7f6719600640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6714101b10 0x7f67141a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.045+0000 7f6719600640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6714101170 0x7f671419a9c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.045+0000 7f6719600640 1 --2- 172.21.15.83:0/21527980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6714100810 0x7f671419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.045+0000 7f6719600640 1 -- 172.21.15.83:0/21527980 >> 172.21.15.83:0/21527980 conn(0x7f67140fc3b0 msgr2=0x7f67140fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:40.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.045+0000 7f6719600640 1 -- 172.21.15.83:0/21527980 shutdown_connections 2024-04-23T14:47:40.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.045+0000 7f6719600640 1 -- 172.21.15.83:0/21527980 wait complete. 2024-04-23T14:47:40.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:47:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.270+0000 7f7d98c00640 1 -- 172.21.15.83:0/2790315661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d94103a40 msgr2=0x7f7d9410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.270+0000 7f7d98c00640 1 --2- 172.21.15.83:0/2790315661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d94103a40 0x7f7d9410a360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009ca0 tx=0x7f7d8c02a8c0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.270+0000 7f7d98c00640 1 -- 172.21.15.83:0/2790315661 shutdown_connections 2024-04-23T14:47:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.270+0000 7f7d98c00640 1 --2- 172.21.15.83:0/2790315661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d94103a40 0x7f7d9410a360 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.270+0000 7f7d98c00640 1 --2- 172.21.15.83:0/2790315661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d94103080 0x7f7d94103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.270+0000 7f7d98c00640 1 --2- 172.21.15.83:0/2790315661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d94101f30 0x7f7d94102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.270+0000 7f7d98c00640 1 -- 172.21.15.83:0/2790315661 >> 172.21.15.83:0/2790315661 conn(0x7f7d940fd4a0 msgr2=0x7f7d940ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.270+0000 7f7d98c00640 1 -- 172.21.15.83:0/2790315661 shutdown_connections 2024-04-23T14:47:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.270+0000 7f7d98c00640 1 -- 172.21.15.83:0/2790315661 wait complete. 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.271+0000 7f7d98c00640 1 Processor -- start 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.271+0000 7f7d98c00640 1 -- start start 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d98c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d94101f30 0x7f7d9419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d98c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d94103080 0x7f7d9419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d98c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d94103a40 0x7f7d941a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d98c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9410dc10 con 0x7f7d94103080 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d98c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9410da90 con 0x7f7d94101f30 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d98c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9410dd90 con 0x7f7d94103a40 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d94103080 0x7f7d9419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d93400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d94101f30 0x7f7d9419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d8be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d94103080 0x7f7d9419a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53850/0 (socket says 172.21.15.83:53850) 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d8be00640 1 -- 172.21.15.83:0/3841711126 learned_addr learned my addr 172.21.15.83:0/3841711126 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d93400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d94101f30 0x7f7d9419a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:34510/0 (socket says 172.21.15.83:34510) 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d93e00640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d94103a40 0x7f7d941a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d93400640 1 -- 172.21.15.83:0/3841711126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d94103a40 msgr2=0x7f7d941a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d93400640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d94103a40 0x7f7d941a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d93400640 1 -- 172.21.15.83:0/3841711126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d94103080 msgr2=0x7f7d9419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.272+0000 7f7d93400640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d94103080 0x7f7d9419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.273+0000 7f7d93400640 1 -- 172.21.15.83:0/3841711126 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d941a1fb0 con 0x7f7d94101f30 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.273+0000 7f7d8be00640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d94103080 0x7f7d9419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.273+0000 7f7d93400640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d94101f30 0x7f7d9419a390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400e9f0 tx=0x7f7d8400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.273+0000 7f7d90c00640 1 -- 172.21.15.83:0/3841711126 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8400cd80 con 0x7f7d94101f30 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.273+0000 7f7d98c00640 1 -- 172.21.15.83:0/3841711126 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d941a22a0 con 0x7f7d94101f30 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.273+0000 7f7d90c00640 1 -- 172.21.15.83:0/3841711126 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d840035f0 con 0x7f7d94101f30 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.273+0000 7f7d90c00640 1 -- 172.21.15.83:0/3841711126 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84010670 con 0x7f7d94101f30 2024-04-23T14:47:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.273+0000 7f7d98c00640 1 -- 172.21.15.83:0/3841711126 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d941a27f0 con 0x7f7d94101f30 2024-04-23T14:47:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.274+0000 7f7d98c00640 1 -- 172.21.15.83:0/3841711126 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d941a2ab0 con 0x7f7d94101f30 2024-04-23T14:47:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.276+0000 7f7d90c00640 1 -- 172.21.15.83:0/3841711126 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7d840037a0 con 0x7f7d94101f30 2024-04-23T14:47:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.278+0000 7f7d90c00640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7d780a3bb0 0x7f7d780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.278+0000 7f7d90c00640 1 -- 172.21.15.83:0/3841711126 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f7d840d0890 con 0x7f7d94101f30 2024-04-23T14:47:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.278+0000 7f7d8be00640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7d780a3bb0 0x7f7d780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.278+0000 7f7d8be00640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7d780a3bb0 0x7f7d780a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7d9419b8a0 tx=0x7f7d7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.280+0000 7f7d90c00640 1 -- 172.21.15.83:0/3841711126 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d8409d230 con 0x7f7d94101f30 2024-04-23T14:47:40.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.478+0000 7f7d98c00640 1 -- 172.21.15.83:0/3841711126 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7d941a2c80 con 0x7f7d94101f30 2024-04-23T14:47:40.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.480+0000 7f7d90c00640 1 -- 172.21.15.83:0/3841711126 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f7d8409cb30 con 0x7f7d94101f30 2024-04-23T14:47:40.481 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-23T14:47:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.481+0000 7f7d98c00640 1 -- 172.21.15.83:0/3841711126 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7d780a3bb0 msgr2=0x7f7d780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.481+0000 7f7d98c00640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7d780a3bb0 0x7f7d780a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7d9419b8a0 tx=0x7f7d7c005f90 comp rx=0 tx=0).stop 2024-04-23T14:47:40.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.482+0000 7f7d98c00640 1 -- 172.21.15.83:0/3841711126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d94101f30 msgr2=0x7f7d9419a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.482+0000 7f7d98c00640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d94101f30 0x7f7d9419a390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400e9f0 tx=0x7f7d8400eeb0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.482+0000 7f7d98c00640 1 -- 172.21.15.83:0/3841711126 shutdown_connections 2024-04-23T14:47:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.482+0000 7f7d98c00640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7d780a3bb0 0x7f7d780a6070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.482+0000 7f7d98c00640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d94103a40 0x7f7d941a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.482+0000 7f7d98c00640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d94103080 0x7f7d9419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.482+0000 7f7d98c00640 1 --2- 172.21.15.83:0/3841711126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d94101f30 0x7f7d9419a390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.482+0000 7f7d98c00640 1 -- 172.21.15.83:0/3841711126 >> 172.21.15.83:0/3841711126 conn(0x7f7d940fd4a0 msgr2=0x7f7d94074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.482+0000 7f7d98c00640 1 -- 172.21.15.83:0/3841711126 shutdown_connections 2024-04-23T14:47:40.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.482+0000 7f7d98c00640 1 -- 172.21.15.83:0/3841711126 wait complete. 2024-04-23T14:47:40.501 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:47:40.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.695+0000 7fd9a3200640 1 -- 172.21.15.83:0/4063678710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a4104080 msgr2=0x7fd9a41044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.695+0000 7fd9a3200640 1 --2- 172.21.15.83:0/4063678710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a4104080 0x7fd9a41044a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd994009900 tx=0x7fd99402a410 comp rx=0 tx=0).stop 2024-04-23T14:47:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.695+0000 7fd9a3200640 1 -- 172.21.15.83:0/4063678710 shutdown_connections 2024-04-23T14:47:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.695+0000 7fd9a3200640 1 --2- 172.21.15.83:0/4063678710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a4105c30 0x7fd9a410c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.695+0000 7fd9a3200640 1 --2- 172.21.15.83:0/4063678710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a4105270 0x7fd9a41056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.695+0000 7fd9a3200640 1 --2- 172.21.15.83:0/4063678710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a4104080 0x7fd9a41044a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.695+0000 7fd9a3200640 1 -- 172.21.15.83:0/4063678710 >> 172.21.15.83:0/4063678710 conn(0x7fd9a40ff610 msgr2=0x7fd9a4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.695+0000 7fd9a3200640 1 -- 172.21.15.83:0/4063678710 shutdown_connections 2024-04-23T14:47:40.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.695+0000 7fd9a3200640 1 -- 172.21.15.83:0/4063678710 wait complete. 2024-04-23T14:47:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.696+0000 7fd9a3200640 1 Processor -- start 2024-04-23T14:47:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.696+0000 7fd9a3200640 1 -- start start 2024-04-23T14:47:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.696+0000 7fd9a3200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a4104080 0x7fd9a419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.696+0000 7fd9a3200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a4105270 0x7fd9a419a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.696+0000 7fd9a3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a4105c30 0x7fd9a41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:40.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.696+0000 7fd9a3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a410fd10 con 0x7fd9a4105c30 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.696+0000 7fd9a3200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a410fb90 con 0x7fd9a4105270 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.696+0000 7fd9a3200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a410fe90 con 0x7fd9a4104080 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.696+0000 7fd9a2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a4105c30 0x7fd9a41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.697+0000 7fd9a2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a4105c30 0x7fd9a41a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53880/0 (socket says 172.21.15.83:53880) 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.697+0000 7fd9a1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a4104080 0x7fd9a419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.697+0000 7fd9a2800640 1 -- 172.21.15.83:0/1502875688 learned_addr learned my addr 172.21.15.83:0/1502875688 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.697+0000 7fd99be00640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a4105270 0x7fd9a419a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.697+0000 7fd9a2800640 1 -- 172.21.15.83:0/1502875688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a4104080 msgr2=0x7fd9a419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.697+0000 7fd9a2800640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a4104080 0x7fd9a419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.697+0000 7fd9a2800640 1 -- 172.21.15.83:0/1502875688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a4105270 msgr2=0x7fd9a419a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.697+0000 7fd9a2800640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a4105270 0x7fd9a419a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.697+0000 7fd9a2800640 1 -- 172.21.15.83:0/1502875688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a41a1fa0 con 0x7fd9a4105c30 2024-04-23T14:47:40.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.697+0000 7fd9a2800640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a4105c30 0x7fd9a41a1930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00eb10 tx=0x7fd99c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:40.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.697+0000 7fd99aa00640 1 -- 172.21.15.83:0/1502875688 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c004b20 con 0x7fd9a4105c30 2024-04-23T14:47:40.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.698+0000 7fd9a3200640 1 -- 172.21.15.83:0/1502875688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a41a2290 con 0x7fd9a4105c30 2024-04-23T14:47:40.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.698+0000 7fd99aa00640 1 -- 172.21.15.83:0/1502875688 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd99c004cc0 con 0x7fd9a4105c30 2024-04-23T14:47:40.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.698+0000 7fd99aa00640 1 -- 172.21.15.83:0/1502875688 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c010760 con 0x7fd9a4105c30 2024-04-23T14:47:40.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.698+0000 7fd9a3200640 1 -- 172.21.15.83:0/1502875688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a41a2790 con 0x7fd9a4105c30 2024-04-23T14:47:40.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.699+0000 7fd9a3200640 1 -- 172.21.15.83:0/1502875688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a410b420 con 0x7fd9a4105c30 2024-04-23T14:47:40.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.701+0000 7fd99aa00640 1 -- 172.21.15.83:0/1502875688 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd99c005020 con 0x7fd9a4105c30 2024-04-23T14:47:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.702+0000 7fd99aa00640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd9840a3bb0 0x7fd9840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.702+0000 7fd99aa00640 1 -- 172.21.15.83:0/1502875688 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd99c0d0970 con 0x7fd9a4105c30 2024-04-23T14:47:40.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.702+0000 7fd9a1e00640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd9840a3bb0 0x7fd9840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:40.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.703+0000 7fd9a1e00640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd9840a3bb0 0x7fd9840a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd994005ab0 tx=0x7fd994000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:40.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.706+0000 7fd99aa00640 1 -- 172.21.15.83:0/1502875688 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd99c09d350 con 0x7fd9a4105c30 2024-04-23T14:47:40.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:40 smithi203 ceph-mon[32800]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T14:47:40.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:40 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2638692217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:47:40.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:40 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/21527980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:47:40.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:40 smithi083 ceph-mon[27867]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T14:47:40.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:40 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2638692217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:47:40.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:40 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/21527980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:47:40.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:40 smithi173 ceph-mon[32443]: pgmap v369: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-23T14:47:40.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:40 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2638692217' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:47:40.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:40 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/21527980' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:47:40.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.949+0000 7fd9a3200640 1 -- 172.21.15.83:0/1502875688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9a4062c00 con 0x7fd9a4105c30 2024-04-23T14:47:40.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.951+0000 7fd99aa00640 1 -- 172.21.15.83:0/1502875688 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7fd99c09cc50 con 0x7fd9a4105c30 2024-04-23T14:47:40.952 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:47:40.953 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:46:05.301806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:47:40.953 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-04-23T14:47:40.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.953+0000 7fd9a3200640 1 -- 172.21.15.83:0/1502875688 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd9840a3bb0 msgr2=0x7fd9840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:40.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.953+0000 7fd9a3200640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd9840a3bb0 0x7fd9840a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd994005ab0 tx=0x7fd994000bc0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.953+0000 7fd9a3200640 1 -- 172.21.15.83:0/1502875688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a4105c30 msgr2=0x7fd9a41a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:40.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.953+0000 7fd9a3200640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a4105c30 0x7fd9a41a1930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00eb10 tx=0x7fd99c00c2d0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.954+0000 7fd9a3200640 1 -- 172.21.15.83:0/1502875688 shutdown_connections 2024-04-23T14:47:40.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.954+0000 7fd9a3200640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fd9840a3bb0 0x7fd9840a6070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.954+0000 7fd9a3200640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9a4105c30 0x7fd9a41a1930 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.954+0000 7fd9a3200640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9a4105270 0x7fd9a419a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.954+0000 7fd9a3200640 1 --2- 172.21.15.83:0/1502875688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9a4104080 0x7fd9a419a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:40.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.954+0000 7fd9a3200640 1 -- 172.21.15.83:0/1502875688 >> 172.21.15.83:0/1502875688 conn(0x7fd9a40ff610 msgr2=0x7fd9a41085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:40.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.954+0000 7fd9a3200640 1 -- 172.21.15.83:0/1502875688 shutdown_connections 2024-04-23T14:47:40.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:40.954+0000 7fd9a3200640 1 -- 172.21.15.83:0/1502875688 wait complete. 2024-04-23T14:47:40.975 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:47:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.172+0000 7f5d0b200640 1 -- 172.21.15.83:0/2791412167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c109670 msgr2=0x7f5d0c109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.172+0000 7f5d0b200640 1 --2- 172.21.15.83:0/2791412167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c109670 0x7f5d0c109af0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009900 tx=0x7f5cf402a400 comp rx=0 tx=0).stop 2024-04-23T14:47:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.172+0000 7f5d0b200640 1 -- 172.21.15.83:0/2791412167 shutdown_connections 2024-04-23T14:47:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.172+0000 7f5d0b200640 1 --2- 172.21.15.83:0/2791412167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d0c10a030 0x7f5d0c1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.172+0000 7f5d0b200640 1 --2- 172.21.15.83:0/2791412167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c109670 0x7f5d0c109af0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.172+0000 7f5d0b200640 1 --2- 172.21.15.83:0/2791412167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0c108480 0x7f5d0c1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.172+0000 7f5d0b200640 1 -- 172.21.15.83:0/2791412167 >> 172.21.15.83:0/2791412167 conn(0x7f5d0c077440 msgr2=0x7f5d0c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.172+0000 7f5d0b200640 1 -- 172.21.15.83:0/2791412167 shutdown_connections 2024-04-23T14:47:41.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.172+0000 7f5d0b200640 1 -- 172.21.15.83:0/2791412167 wait complete. 2024-04-23T14:47:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.173+0000 7f5d0b200640 1 Processor -- start 2024-04-23T14:47:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.173+0000 7f5d0b200640 1 -- start start 2024-04-23T14:47:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.173+0000 7f5d0b200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d0c108480 0x7f5d0c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.173+0000 7f5d0b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c109670 0x7f5d0c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.173+0000 7f5d0b200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0c10a030 0x7f5d0c1a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d0b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c114160 con 0x7f5d0c109670 2024-04-23T14:47:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d0b200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c113fe0 con 0x7f5d0c10a030 2024-04-23T14:47:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d0b200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0c1142e0 con 0x7f5d0c108480 2024-04-23T14:47:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d03e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c109670 0x7f5d0c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d03e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c109670 0x7f5d0c19ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53896/0 (socket says 172.21.15.83:53896) 2024-04-23T14:47:41.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d03e00640 1 -- 172.21.15.83:0/1150908746 learned_addr learned my addr 172.21.15.83:0/1150908746 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d09e00640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d0c108480 0x7f5d0c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d0a800640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0c10a030 0x7f5d0c1a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d03e00640 1 -- 172.21.15.83:0/1150908746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d0c108480 msgr2=0x7f5d0c19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d03e00640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d0c108480 0x7f5d0c19e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d03e00640 1 -- 172.21.15.83:0/1150908746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0c10a030 msgr2=0x7f5d0c1a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d03e00640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0c10a030 0x7f5d0c1a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.174+0000 7f5d03e00640 1 -- 172.21.15.83:0/1150908746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0c1a63b0 con 0x7f5d0c109670 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.175+0000 7f5d03e00640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c109670 0x7f5d0c19ecd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4031440 tx=0x7f5cf4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.175+0000 7f5d02a00640 1 -- 172.21.15.83:0/1150908746 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf403f030 con 0x7f5d0c109670 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.175+0000 7f5d02a00640 1 -- 172.21.15.83:0/1150908746 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf4003ce0 con 0x7f5d0c109670 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.175+0000 7f5d0b200640 1 -- 172.21.15.83:0/1150908746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0c1a6640 con 0x7f5d0c109670 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.175+0000 7f5d02a00640 1 -- 172.21.15.83:0/1150908746 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4035070 con 0x7f5d0c109670 2024-04-23T14:47:41.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.175+0000 7f5d0b200640 1 -- 172.21.15.83:0/1150908746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d0c1a6b30 con 0x7f5d0c109670 2024-04-23T14:47:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.176+0000 7f5d0b200640 1 -- 172.21.15.83:0/1150908746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0c10f7a0 con 0x7f5d0c109670 2024-04-23T14:47:41.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.178+0000 7f5d02a00640 1 -- 172.21.15.83:0/1150908746 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5cf4042030 con 0x7f5d0c109670 2024-04-23T14:47:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.180+0000 7f5d02a00640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5cec0a3bb0 0x7f5cec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.180+0000 7f5d02a00640 1 -- 172.21.15.83:0/1150908746 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5cf40f08c0 con 0x7f5d0c109670 2024-04-23T14:47:41.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.180+0000 7f5d09e00640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5cec0a3bb0 0x7f5cec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:41.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.181+0000 7f5d09e00640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5cec0a3bb0 0x7f5cec0a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c1094d0 tx=0x7f5cfc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:41.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.184+0000 7f5d02a00640 1 -- 172.21.15.83:0/1150908746 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5cf40bd260 con 0x7f5d0c109670 2024-04-23T14:47:41.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.375+0000 7f5d0b200640 1 -- 172.21.15.83:0/1150908746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d0c062c00 con 0x7f5d0c109670 2024-04-23T14:47:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.378+0000 7f5d02a00640 1 -- 172.21.15.83:0/1150908746 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22699 (secure 0 0 0) 0x7f5cf40bcb60 con 0x7f5d0c109670 2024-04-23T14:47:41.379 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:47:41.380 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:46:58.351541+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:47:41.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.380+0000 7f5d0b200640 1 -- 172.21.15.83:0/1150908746 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5cec0a3bb0 msgr2=0x7f5cec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:41.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.380+0000 7f5d0b200640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5cec0a3bb0 0x7f5cec0a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c1094d0 tx=0x7f5cfc005f90 comp rx=0 tx=0).stop 2024-04-23T14:47:41.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.380+0000 7f5d0b200640 1 -- 172.21.15.83:0/1150908746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c109670 msgr2=0x7f5d0c19ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:41.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.380+0000 7f5d0b200640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c109670 0x7f5d0c19ecd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4031440 tx=0x7f5cf4032ce0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.380+0000 7f5d0b200640 1 -- 172.21.15.83:0/1150908746 shutdown_connections 2024-04-23T14:47:41.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.380+0000 7f5d0b200640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f5cec0a3bb0 0x7f5cec0a6070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.380+0000 7f5d0b200640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d0c10a030 0x7f5d0c1a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.380+0000 7f5d0b200640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0c109670 0x7f5d0c19ecd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.380+0000 7f5d0b200640 1 --2- 172.21.15.83:0/1150908746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d0c108480 0x7f5d0c19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.380+0000 7f5d0b200640 1 -- 172.21.15.83:0/1150908746 >> 172.21.15.83:0/1150908746 conn(0x7f5d0c077440 msgr2=0x7f5d0c10c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:41.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.381+0000 7f5d0b200640 1 -- 172.21.15.83:0/1150908746 shutdown_connections 2024-04-23T14:47:41.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.381+0000 7f5d0b200640 1 -- 172.21.15.83:0/1150908746 wait complete. 2024-04-23T14:47:41.400 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:40:41.241992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:45:57.341602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:45:58.654798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:47:41.400 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:47:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.605+0000 7fefbbe00640 1 -- 172.21.15.83:0/1543904838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefbc100780 msgr2=0x7fefbc102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.605+0000 7fefbbe00640 1 --2- 172.21.15.83:0/1543904838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefbc100780 0x7fefbc102bb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fefac009900 tx=0x7fefac02a3f0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.605+0000 7fefbbe00640 1 -- 172.21.15.83:0/1543904838 shutdown_connections 2024-04-23T14:47:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.605+0000 7fefbbe00640 1 --2- 172.21.15.83:0/1543904838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc105aa0 0x7fefbc105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.605+0000 7fefbbe00640 1 --2- 172.21.15.83:0/1543904838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefbc1030f0 0x7fefbc105560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.605+0000 7fefbbe00640 1 --2- 172.21.15.83:0/1543904838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefbc100780 0x7fefbc102bb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.605+0000 7fefbbe00640 1 -- 172.21.15.83:0/1543904838 >> 172.21.15.83:0/1543904838 conn(0x7fefbc0fc3a0 msgr2=0x7fefbc0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.605+0000 7fefbbe00640 1 -- 172.21.15.83:0/1543904838 shutdown_connections 2024-04-23T14:47:41.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.605+0000 7fefbbe00640 1 -- 172.21.15.83:0/1543904838 wait complete. 2024-04-23T14:47:41.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.606+0000 7fefbbe00640 1 Processor -- start 2024-04-23T14:47:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.606+0000 7fefbbe00640 1 -- start start 2024-04-23T14:47:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.606+0000 7fefbbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefbc100780 0x7fefbc19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.606+0000 7fefbbe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefbc1030f0 0x7fefbc19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.606+0000 7fefbbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc105aa0 0x7fefbc1a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.606+0000 7fefbbe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc114200 con 0x7fefbc105aa0 2024-04-23T14:47:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.606+0000 7fefbbe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc114080 con 0x7fefbc100780 2024-04-23T14:47:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.606+0000 7fefbbe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fefbc114380 con 0x7fefbc1030f0 2024-04-23T14:47:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefbb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc105aa0 0x7fefbc1a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefb2000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefbc1030f0 0x7fefbc19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefbaa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefbc100780 0x7fefbc19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefb2000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefbc1030f0 0x7fefbc19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59882/0 (socket says 172.21.15.83:59882) 2024-04-23T14:47:41.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefbb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc105aa0 0x7fefbc1a5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53916/0 (socket says 172.21.15.83:53916) 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefb2000640 1 -- 172.21.15.83:0/2516720350 learned_addr learned my addr 172.21.15.83:0/2516720350 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefb2000640 1 -- 172.21.15.83:0/2516720350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefbc100780 msgr2=0x7fefbc19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefb2000640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefbc100780 0x7fefbc19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefb2000640 1 -- 172.21.15.83:0/2516720350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc105aa0 msgr2=0x7fefbc1a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefb2000640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc105aa0 0x7fefbc1a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefb2000640 1 -- 172.21.15.83:0/2516720350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefbc1a63f0 con 0x7fefbc1030f0 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefbb400640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc105aa0 0x7fefbc1a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.607+0000 7fefb2000640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefbc1030f0 0x7fefbc19ed10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fefa400e9f0 tx=0x7fefa400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.608+0000 7fefb3e00640 1 -- 172.21.15.83:0/2516720350 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa400cd80 con 0x7fefbc1030f0 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.608+0000 7fefb3e00640 1 -- 172.21.15.83:0/2516720350 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa4005500 con 0x7fefbc1030f0 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.608+0000 7fefb3e00640 1 -- 172.21.15.83:0/2516720350 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4010690 con 0x7fefbc1030f0 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.608+0000 7fefbbe00640 1 -- 172.21.15.83:0/2516720350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefbc1a66e0 con 0x7fefbc1030f0 2024-04-23T14:47:41.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.608+0000 7fefbbe00640 1 -- 172.21.15.83:0/2516720350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefbc1a6be0 con 0x7fefbc1030f0 2024-04-23T14:47:41.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.609+0000 7fefbbe00640 1 -- 172.21.15.83:0/2516720350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefbc0fde90 con 0x7fefbc1030f0 2024-04-23T14:47:41.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.611+0000 7fefb3e00640 1 -- 172.21.15.83:0/2516720350 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fefa40046c0 con 0x7fefbc1030f0 2024-04-23T14:47:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.612+0000 7fefb3e00640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fefa00a3bb0 0x7fefa00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.612+0000 7fefb3e00640 1 -- 172.21.15.83:0/2516720350 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fefa40d0840 con 0x7fefbc1030f0 2024-04-23T14:47:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.612+0000 7fefbaa00640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fefa00a3bb0 0x7fefa00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.613+0000 7fefbaa00640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fefa00a3bb0 0x7fefa00a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fefac031440 tx=0x7fefac0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.616+0000 7fefb3e00640 1 -- 172.21.15.83:0/2516720350 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefa409d1e0 con 0x7fefbc1030f0 2024-04-23T14:47:41.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:41 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3841711126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:47:41.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:41 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1502875688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:47:41.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:41 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3841711126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:47:41.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:41 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1502875688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:47:41.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.820+0000 7fefbbe00640 1 -- 172.21.15.83:0/2516720350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fefbc062c00 con 0x7fefbc1030f0 2024-04-23T14:47:41.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.820+0000 7fefb3e00640 1 -- 172.21.15.83:0/2516720350 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fefa409cae0 con 0x7fefbc1030f0 2024-04-23T14:47:41.822 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-23T14:47:41.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.823+0000 7fefbbe00640 1 -- 172.21.15.83:0/2516720350 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fefa00a3bb0 msgr2=0x7fefa00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:41.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.823+0000 7fefbbe00640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fefa00a3bb0 0x7fefa00a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fefac031440 tx=0x7fefac0315b0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.823+0000 7fefbbe00640 1 -- 172.21.15.83:0/2516720350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefbc1030f0 msgr2=0x7fefbc19ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:41.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.823+0000 7fefbbe00640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefbc1030f0 0x7fefbc19ed10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fefa400e9f0 tx=0x7fefa400edf0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.824+0000 7fefbbe00640 1 -- 172.21.15.83:0/2516720350 shutdown_connections 2024-04-23T14:47:41.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.824+0000 7fefbbe00640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fefa00a3bb0 0x7fefa00a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.824+0000 7fefbbe00640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fefbc105aa0 0x7fefbc1a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.824+0000 7fefbbe00640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fefbc1030f0 0x7fefbc19ed10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.824+0000 7fefbbe00640 1 --2- 172.21.15.83:0/2516720350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefbc100780 0x7fefbc19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:41.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.824+0000 7fefbbe00640 1 -- 172.21.15.83:0/2516720350 >> 172.21.15.83:0/2516720350 conn(0x7fefbc0fc3a0 msgr2=0x7fefbc103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:41.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.824+0000 7fefbbe00640 1 -- 172.21.15.83:0/2516720350 shutdown_connections 2024-04-23T14:47:41.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:41.824+0000 7fefbbe00640 1 -- 172.21.15.83:0/2516720350 wait complete. 2024-04-23T14:47:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:41 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3841711126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:47:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:41 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1502875688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:47:41.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:47:42.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.033+0000 7f386ae00640 1 -- 172.21.15.83:0/1502538505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3864104280 msgr2=0x7f38641046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.033+0000 7f386ae00640 1 --2- 172.21.15.83:0/1502538505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3864104280 0x7f38641046a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f385c009ca0 tx=0x7f385c02a8c0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.033+0000 7f386ae00640 1 -- 172.21.15.83:0/1502538505 shutdown_connections 2024-04-23T14:47:42.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.033+0000 7f386ae00640 1 --2- 172.21.15.83:0/1502538505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3864105e30 0x7f386410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.033+0000 7f386ae00640 1 --2- 172.21.15.83:0/1502538505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3864105470 0x7f38641058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.033+0000 7f386ae00640 1 --2- 172.21.15.83:0/1502538505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3864104280 0x7f38641046a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.033+0000 7f386ae00640 1 -- 172.21.15.83:0/1502538505 >> 172.21.15.83:0/1502538505 conn(0x7f38640ff7f0 msgr2=0x7f3864101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:42.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.033+0000 7f386ae00640 1 -- 172.21.15.83:0/1502538505 shutdown_connections 2024-04-23T14:47:42.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.033+0000 7f386ae00640 1 -- 172.21.15.83:0/1502538505 wait complete. 2024-04-23T14:47:42.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.034+0000 7f386ae00640 1 Processor -- start 2024-04-23T14:47:42.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.034+0000 7f386ae00640 1 -- start start 2024-04-23T14:47:42.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.034+0000 7f386ae00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3864104280 0x7f386419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:42.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.034+0000 7f386ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3864105470 0x7f386419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:42.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.034+0000 7f386ae00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3864105e30 0x7f38641a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:42.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.034+0000 7f386ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f386410fee0 con 0x7f3864105470 2024-04-23T14:47:42.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.034+0000 7f386ae00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f386410fd60 con 0x7f3864105e30 2024-04-23T14:47:42.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.034+0000 7f386ae00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3864110060 con 0x7f3864104280 2024-04-23T14:47:42.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.035+0000 7f3863e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3864104280 0x7f386419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:42.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.035+0000 7f3863400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3864105470 0x7f386419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:42.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.035+0000 7f3863e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3864104280 0x7f386419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59902/0 (socket says 172.21.15.83:59902) 2024-04-23T14:47:42.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.035+0000 7f3863e00640 1 -- 172.21.15.83:0/3966163320 learned_addr learned my addr 172.21.15.83:0/3966163320 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:42.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.035+0000 7f3868e00640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3864105e30 0x7f38641a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:42.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.035+0000 7f3863e00640 1 -- 172.21.15.83:0/3966163320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3864105e30 msgr2=0x7f38641a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.035+0000 7f3863e00640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3864105e30 0x7f38641a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.035+0000 7f3863e00640 1 -- 172.21.15.83:0/3966163320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3864105470 msgr2=0x7f386419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.035+0000 7f3863e00640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3864105470 0x7f386419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.035+0000 7f3863e00640 1 -- 172.21.15.83:0/3966163320 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38641a21e0 con 0x7f3864104280 2024-04-23T14:47:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.035+0000 7f3863400640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3864105470 0x7f386419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.036+0000 7f3863e00640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3864104280 0x7f386419a5c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f385c009bf0 tx=0x7f385c031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.036+0000 7f3860c00640 1 -- 172.21.15.83:0/3966163320 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c02fa20 con 0x7f3864104280 2024-04-23T14:47:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.036+0000 7f386ae00640 1 -- 172.21.15.83:0/3966163320 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38641a2470 con 0x7f3864104280 2024-04-23T14:47:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.036+0000 7f3860c00640 1 -- 172.21.15.83:0/3966163320 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f385c02fbc0 con 0x7f3864104280 2024-04-23T14:47:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.036+0000 7f3860c00640 1 -- 172.21.15.83:0/3966163320 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c046750 con 0x7f3864104280 2024-04-23T14:47:42.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.036+0000 7f386ae00640 1 -- 172.21.15.83:0/3966163320 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38641a2a10 con 0x7f3864104280 2024-04-23T14:47:42.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.037+0000 7f386ae00640 1 -- 172.21.15.83:0/3966163320 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f386410b620 con 0x7f3864104280 2024-04-23T14:47:42.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.039+0000 7f3860c00640 1 -- 172.21.15.83:0/3966163320 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f385c02fd70 con 0x7f3864104280 2024-04-23T14:47:42.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.040+0000 7f3860c00640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f38480a3bb0 0x7f38480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:42.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.041+0000 7f3860c00640 1 -- 172.21.15.83:0/3966163320 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f385c0f1790 con 0x7f3864104280 2024-04-23T14:47:42.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.041+0000 7f3863400640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f38480a3bb0 0x7f38480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:42.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.041+0000 7f3863400640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f38480a3bb0 0x7f38480a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f384c0095f0 tx=0x7f384c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:42.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.045+0000 7f3860c00640 1 -- 172.21.15.83:0/3966163320 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f385c0be130 con 0x7f3864104280 2024-04-23T14:47:42.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.250+0000 7f386ae00640 1 -- 172.21.15.83:0/3966163320 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3864062c00 con 0x7f3864104280 2024-04-23T14:47:42.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.251+0000 7f3860c00640 1 -- 172.21.15.83:0/3966163320 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f385c0bda30 con 0x7f3864104280 2024-04-23T14:47:42.252 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-23T14:47:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.253+0000 7f386ae00640 1 -- 172.21.15.83:0/3966163320 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f38480a3bb0 msgr2=0x7f38480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.253+0000 7f386ae00640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f38480a3bb0 0x7f38480a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f384c0095f0 tx=0x7f384c005f90 comp rx=0 tx=0).stop 2024-04-23T14:47:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.253+0000 7f386ae00640 1 -- 172.21.15.83:0/3966163320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3864104280 msgr2=0x7f386419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.253+0000 7f386ae00640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3864104280 0x7f386419a5c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f385c009bf0 tx=0x7f385c031ee0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.254+0000 7f386ae00640 1 -- 172.21.15.83:0/3966163320 shutdown_connections 2024-04-23T14:47:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.254+0000 7f386ae00640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f38480a3bb0 0x7f38480a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.254+0000 7f386ae00640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3864105e30 0x7f38641a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.254+0000 7f386ae00640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3864105470 0x7f386419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.254+0000 7f386ae00640 1 --2- 172.21.15.83:0/3966163320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3864104280 0x7f386419a5c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.254+0000 7f386ae00640 1 -- 172.21.15.83:0/3966163320 >> 172.21.15.83:0/3966163320 conn(0x7f38640ff7f0 msgr2=0x7f38641087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:42.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.254+0000 7f386ae00640 1 -- 172.21.15.83:0/3966163320 shutdown_connections 2024-04-23T14:47:42.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.254+0000 7f386ae00640 1 -- 172.21.15.83:0/3966163320 wait complete. 2024-04-23T14:47:42.275 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:47:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.466+0000 7f79aee00640 1 -- 172.21.15.83:0/1399726460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8105330 msgr2=0x7f79a81057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.466+0000 7f79aee00640 1 --2- 172.21.15.83:0/1399726460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8105330 0x7f79a81057b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7994009900 tx=0x7f799402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.466+0000 7f79aee00640 1 -- 172.21.15.83:0/1399726460 shutdown_connections 2024-04-23T14:47:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.466+0000 7f79aee00640 1 --2- 172.21.15.83:0/1399726460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79a8105cf0 0x7f79a810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.466+0000 7f79aee00640 1 --2- 172.21.15.83:0/1399726460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8105330 0x7f79a81057b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.466+0000 7f79aee00640 1 --2- 172.21.15.83:0/1399726460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8104140 0x7f79a8104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.466+0000 7f79aee00640 1 -- 172.21.15.83:0/1399726460 >> 172.21.15.83:0/1399726460 conn(0x7f79a80ff6b0 msgr2=0x7f79a8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.466+0000 7f79aee00640 1 -- 172.21.15.83:0/1399726460 shutdown_connections 2024-04-23T14:47:42.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.466+0000 7f79aee00640 1 -- 172.21.15.83:0/1399726460 wait complete. 2024-04-23T14:47:42.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.467+0000 7f79aee00640 1 Processor -- start 2024-04-23T14:47:42.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.467+0000 7f79aee00640 1 -- start start 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79aee00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8104140 0x7f79a819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79aee00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8105330 0x7f79a819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79aee00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79a8105cf0 0x7f79a81a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79aee00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79a810fe20 con 0x7f79a8105cf0 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79aee00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f79a810fca0 con 0x7f79a8104140 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79aee00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79a810ffa0 con 0x7f79a8105330 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79ada00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8104140 0x7f79a819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79ada00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8104140 0x7f79a819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:34608/0 (socket says 172.21.15.83:34608) 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79ae400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79a8105cf0 0x7f79a81a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79ada00640 1 -- 172.21.15.83:0/461148029 learned_addr learned my addr 172.21.15.83:0/461148029 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79ad000640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8105330 0x7f79a819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79ada00640 1 -- 172.21.15.83:0/461148029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8105330 msgr2=0x7f79a819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79ada00640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8105330 0x7f79a819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79ada00640 1 -- 172.21.15.83:0/461148029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79a8105cf0 msgr2=0x7f79a81a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79ada00640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79a8105cf0 0x7f79a81a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.468+0000 7f79ada00640 1 -- 172.21.15.83:0/461148029 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a81a2030 con 0x7f79a8104140 2024-04-23T14:47:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.469+0000 7f79ad000640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8105330 0x7f79a819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:47:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.469+0000 7f79ada00640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8104140 0x7f79a819a410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f799c00e9c0 tx=0x7f799c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.469+0000 7f79a6000640 1 -- 172.21.15.83:0/461148029 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c00cd80 con 0x7f79a8104140 2024-04-23T14:47:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.469+0000 7f79a6000640 1 -- 172.21.15.83:0/461148029 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c0035f0 con 0x7f79a8104140 2024-04-23T14:47:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.469+0000 7f79aee00640 1 -- 172.21.15.83:0/461148029 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a81a2320 con 0x7f79a8104140 2024-04-23T14:47:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.469+0000 7f79a6000640 1 -- 172.21.15.83:0/461148029 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c010690 con 0x7f79a8104140 2024-04-23T14:47:42.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.469+0000 7f79aee00640 1 -- 172.21.15.83:0/461148029 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a81a27f0 con 0x7f79a8104140 2024-04-23T14:47:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.471+0000 7f79aee00640 1 -- 172.21.15.83:0/461148029 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a810b400 con 0x7f79a8104140 2024-04-23T14:47:42.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.472+0000 7f79a6000640 1 -- 172.21.15.83:0/461148029 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f799c0037a0 con 0x7f79a8104140 2024-04-23T14:47:42.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.473+0000 7f79a6000640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f79900a3bb0 0x7f79900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:42.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.474+0000 7f79a6000640 1 -- 172.21.15.83:0/461148029 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f799c0d0880 con 0x7f79a8104140 2024-04-23T14:47:42.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.474+0000 7f79ad000640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f79900a3bb0 0x7f79900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:42.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.475+0000 7f79ad000640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f79900a3bb0 0x7f79900a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7994006010 tx=0x7f7994000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:42.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.478+0000 7f79a6000640 1 -- 172.21.15.83:0/461148029 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f799c09d220 con 0x7f79a8104140 2024-04-23T14:47:42.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.679+0000 7f79aee00640 1 -- 172.21.15.83:0/461148029 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f79a8062c00 con 0x7f79a8104140 2024-04-23T14:47:42.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.680+0000 7f79a6000640 1 -- 172.21.15.83:0/461148029 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f799c09cb20 con 0x7f79a8104140 2024-04-23T14:47:42.682 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-23T14:47:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.683+0000 7f79aee00640 1 -- 172.21.15.83:0/461148029 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f79900a3bb0 msgr2=0x7f79900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.683+0000 7f79aee00640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f79900a3bb0 0x7f79900a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7994006010 tx=0x7f7994000bc0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.683+0000 7f79aee00640 1 -- 172.21.15.83:0/461148029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8104140 msgr2=0x7f79a819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.683+0000 7f79aee00640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8104140 0x7f79a819a410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f799c00e9c0 tx=0x7f799c00ee80 comp rx=0 tx=0).stop 2024-04-23T14:47:42.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.683+0000 7f79aee00640 1 -- 172.21.15.83:0/461148029 shutdown_connections 2024-04-23T14:47:42.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.683+0000 7f79aee00640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f79900a3bb0 0x7f79900a6070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.683+0000 7f79aee00640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79a8105cf0 0x7f79a81a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.683+0000 7f79aee00640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79a8105330 0x7f79a819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.683+0000 7f79aee00640 1 --2- 172.21.15.83:0/461148029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79a8104140 0x7f79a819a410 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.683+0000 7f79aee00640 1 -- 172.21.15.83:0/461148029 >> 172.21.15.83:0/461148029 conn(0x7f79a80ff6b0 msgr2=0x7f79a81085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:42.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.683+0000 7f79aee00640 1 -- 172.21.15.83:0/461148029 shutdown_connections 2024-04-23T14:47:42.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.683+0000 7f79aee00640 1 -- 172.21.15.83:0/461148029 wait complete. 2024-04-23T14:47:42.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:42 smithi083 ceph-mon[27867]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T14:47:42.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:42 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1150908746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:47:42.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:42 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2516720350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:47:42.700 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:42 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3966163320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:47:42.703 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:47:42.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:42 smithi203 ceph-mon[32800]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T14:47:42.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:42 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1150908746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:47:42.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:42 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2516720350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:47:42.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:42 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3966163320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:47:42.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:42 smithi173 ceph-mon[32443]: pgmap v370: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-23T14:47:42.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:42 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1150908746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:47:42.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:42 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2516720350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:47:42.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:42 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3966163320' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:47:42.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.894+0000 7f9893200640 1 -- 172.21.15.83:0/1587406322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9894105270 msgr2=0x7f98941056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.894+0000 7f9893200640 1 --2- 172.21.15.83:0/1587406322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9894105270 0x7f98941056f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f987c009900 tx=0x7f987c02a3f0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.894+0000 7f9893200640 1 -- 172.21.15.83:0/1587406322 shutdown_connections 2024-04-23T14:47:42.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.894+0000 7f9893200640 1 --2- 172.21.15.83:0/1587406322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9894105c30 0x7f989410c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.894+0000 7f9893200640 1 --2- 172.21.15.83:0/1587406322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9894105270 0x7f98941056f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.894+0000 7f9893200640 1 --2- 172.21.15.83:0/1587406322 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9894104080 0x7f98941044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.894+0000 7f9893200640 1 -- 172.21.15.83:0/1587406322 >> 172.21.15.83:0/1587406322 conn(0x7f98940ff610 msgr2=0x7f9894101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:42.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.894+0000 7f9893200640 1 -- 172.21.15.83:0/1587406322 shutdown_connections 2024-04-23T14:47:42.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.894+0000 7f9893200640 1 -- 172.21.15.83:0/1587406322 wait complete. 2024-04-23T14:47:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.895+0000 7f9893200640 1 Processor -- start 2024-04-23T14:47:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.895+0000 7f9893200640 1 -- start start 2024-04-23T14:47:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.895+0000 7f9893200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9894104080 0x7f989419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.895+0000 7f9893200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9894105270 0x7f989419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:42.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.895+0000 7f9893200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9894105c30 0x7f98941a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.895+0000 7f9893200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f989410fd10 con 0x7f9894105c30 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.895+0000 7f9893200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f989410fb90 con 0x7f9894105270 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.895+0000 7f9893200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f989410fe90 con 0x7f9894104080 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.896+0000 7f988be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9894105270 0x7f989419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.896+0000 7f9892800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9894105c30 0x7f98941a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.896+0000 7f988be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9894105270 0x7f989419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:34626/0 (socket says 172.21.15.83:34626) 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.896+0000 7f988be00640 1 -- 172.21.15.83:0/1050698778 learned_addr learned my addr 172.21.15.83:0/1050698778 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.896+0000 7f9891e00640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9894104080 0x7f989419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.896+0000 7f9892800640 1 -- 172.21.15.83:0/1050698778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9894104080 msgr2=0x7f989419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.896+0000 7f9892800640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9894104080 0x7f989419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.896+0000 7f9892800640 1 -- 172.21.15.83:0/1050698778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9894105270 msgr2=0x7f989419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.896+0000 7f9892800640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9894105270 0x7f989419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.896+0000 7f9892800640 1 -- 172.21.15.83:0/1050698778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98941a1f20 con 0x7f9894105c30 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.896+0000 7f988be00640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9894105270 0x7f989419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.896+0000 7f9892800640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9894105c30 0x7f98941a18b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f988c00e940 tx=0x7f988c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:42.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.897+0000 7f988aa00640 1 -- 172.21.15.83:0/1050698778 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c004d20 con 0x7f9894105c30 2024-04-23T14:47:42.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.897+0000 7f9893200640 1 -- 172.21.15.83:0/1050698778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98941a2210 con 0x7f9894105c30 2024-04-23T14:47:42.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.897+0000 7f988aa00640 1 -- 172.21.15.83:0/1050698778 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f988c005020 con 0x7f9894105c30 2024-04-23T14:47:42.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.897+0000 7f988aa00640 1 -- 172.21.15.83:0/1050698778 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c0116d0 con 0x7f9894105c30 2024-04-23T14:47:42.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.897+0000 7f9893200640 1 -- 172.21.15.83:0/1050698778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98941a2780 con 0x7f9894105c30 2024-04-23T14:47:42.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.898+0000 7f9893200640 1 -- 172.21.15.83:0/1050698778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f989410b3a0 con 0x7f9894105c30 2024-04-23T14:47:42.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.900+0000 7f988aa00640 1 -- 172.21.15.83:0/1050698778 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f988c0054d0 con 0x7f9894105c30 2024-04-23T14:47:42.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.902+0000 7f988aa00640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f98780a3bb0 0x7f98780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:42.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.903+0000 7f988aa00640 1 -- 172.21.15.83:0/1050698778 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f988c0d0820 con 0x7f9894105c30 2024-04-23T14:47:42.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.903+0000 7f9891e00640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f98780a3bb0 0x7f98780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:42.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.903+0000 7f9891e00640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f98780a3bb0 0x7f98780a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f98941050d0 tx=0x7f988400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:42.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:42.905+0000 7f988aa00640 1 -- 172.21.15.83:0/1050698778 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f988c016030 con 0x7f9894105c30 2024-04-23T14:47:43.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.131+0000 7f9893200640 1 -- 172.21.15.83:0/1050698778 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f989402d620 con 0x7f9894105c30 2024-04-23T14:47:43.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.132+0000 7f988aa00640 1 -- 172.21.15.83:0/1050698778 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7f988c09cb70 con 0x7f9894105c30 2024-04-23T14:47:43.134 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:47:43.134 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:46:05.301806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:47:43.134 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-04-23T14:47:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.134+0000 7f9893200640 1 -- 172.21.15.83:0/1050698778 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f98780a3bb0 msgr2=0x7f98780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.134+0000 7f9893200640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f98780a3bb0 0x7f98780a6070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f98941050d0 tx=0x7f988400b390 comp rx=0 tx=0).stop 2024-04-23T14:47:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.135+0000 7f9893200640 1 -- 172.21.15.83:0/1050698778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9894105c30 msgr2=0x7f98941a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:43.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.135+0000 7f9893200640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9894105c30 0x7f98941a18b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f988c00e940 tx=0x7f988c00ee00 comp rx=0 tx=0).stop 2024-04-23T14:47:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.135+0000 7f9893200640 1 -- 172.21.15.83:0/1050698778 shutdown_connections 2024-04-23T14:47:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.135+0000 7f9893200640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f98780a3bb0 0x7f98780a6070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.135+0000 7f9893200640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9894105c30 0x7f98941a18b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.135+0000 7f9893200640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9894105270 0x7f989419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.135+0000 7f9893200640 1 --2- 172.21.15.83:0/1050698778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9894104080 0x7f989419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.135+0000 7f9893200640 1 -- 172.21.15.83:0/1050698778 >> 172.21.15.83:0/1050698778 conn(0x7f98940ff610 msgr2=0x7f9894108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.135+0000 7f9893200640 1 -- 172.21.15.83:0/1050698778 shutdown_connections 2024-04-23T14:47:43.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.135+0000 7f9893200640 1 -- 172.21.15.83:0/1050698778 wait complete. 2024-04-23T14:47:43.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:47:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.352+0000 7ff231600640 1 -- 172.21.15.83:0/397026741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c104260 msgr2=0x7ff22c104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.352+0000 7ff231600640 1 --2- 172.21.15.83:0/397026741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c104260 0x7ff22c104680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009900 tx=0x7ff21c02a400 comp rx=0 tx=0).stop 2024-04-23T14:47:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.352+0000 7ff231600640 1 -- 172.21.15.83:0/397026741 shutdown_connections 2024-04-23T14:47:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.352+0000 7ff231600640 1 --2- 172.21.15.83:0/397026741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c105e10 0x7ff22c10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.352+0000 7ff231600640 1 --2- 172.21.15.83:0/397026741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff22c105450 0x7ff22c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.352+0000 7ff231600640 1 --2- 172.21.15.83:0/397026741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c104260 0x7ff22c104680 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.352+0000 7ff231600640 1 -- 172.21.15.83:0/397026741 >> 172.21.15.83:0/397026741 conn(0x7ff22c0ff7f0 msgr2=0x7ff22c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:43.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.352+0000 7ff231600640 1 -- 172.21.15.83:0/397026741 shutdown_connections 2024-04-23T14:47:43.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.352+0000 7ff231600640 1 -- 172.21.15.83:0/397026741 wait complete. 2024-04-23T14:47:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.353+0000 7ff231600640 1 Processor -- start 2024-04-23T14:47:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.353+0000 7ff231600640 1 -- start start 2024-04-23T14:47:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.354+0000 7ff231600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c104260 0x7ff22c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.354+0000 7ff231600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c105450 0x7ff22c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:43.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.354+0000 7ff231600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff22c105e10 0x7ff22c1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.354+0000 7ff231600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c10ff40 con 0x7ff22c105e10 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.354+0000 7ff231600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c10fdc0 con 0x7ff22c104260 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.354+0000 7ff231600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff22c1100c0 con 0x7ff22c105450 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.354+0000 7ff22a000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c105450 0x7ff22c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.354+0000 7ff22aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c104260 0x7ff22c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.354+0000 7ff22a000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c105450 0x7ff22c19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59946/0 (socket says 172.21.15.83:59946) 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.354+0000 7ff22a000640 1 -- 172.21.15.83:0/2141676400 learned_addr learned my addr 172.21.15.83:0/2141676400 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.354+0000 7ff22b400640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff22c105e10 0x7ff22c1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.355+0000 7ff22aa00640 1 -- 172.21.15.83:0/2141676400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c105450 msgr2=0x7ff22c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.355+0000 7ff22aa00640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c105450 0x7ff22c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.355+0000 7ff22aa00640 1 -- 172.21.15.83:0/2141676400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff22c105e10 msgr2=0x7ff22c1a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.355+0000 7ff22aa00640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff22c105e10 0x7ff22c1a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.355+0000 7ff22aa00640 1 -- 172.21.15.83:0/2141676400 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff22c1a2150 con 0x7ff22c104260 2024-04-23T14:47:43.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.355+0000 7ff22a000640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c105450 0x7ff22c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.355+0000 7ff22aa00640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c104260 0x7ff22c19a530 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009230 tx=0x7ff21c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.356+0000 7ff223400640 1 -- 172.21.15.83:0/2141676400 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c03f030 con 0x7ff22c104260 2024-04-23T14:47:43.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.356+0000 7ff223400640 1 -- 172.21.15.83:0/2141676400 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff21c003ce0 con 0x7ff22c104260 2024-04-23T14:47:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.356+0000 7ff231600640 1 -- 172.21.15.83:0/2141676400 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22c1a23e0 con 0x7ff22c104260 2024-04-23T14:47:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.356+0000 7ff223400640 1 -- 172.21.15.83:0/2141676400 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21c035070 con 0x7ff22c104260 2024-04-23T14:47:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.356+0000 7ff231600640 1 -- 172.21.15.83:0/2141676400 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff22c1a28f0 con 0x7ff22c104260 2024-04-23T14:47:43.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.357+0000 7ff231600640 1 -- 172.21.15.83:0/2141676400 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff22c10b580 con 0x7ff22c104260 2024-04-23T14:47:43.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.359+0000 7ff223400640 1 -- 172.21.15.83:0/2141676400 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff21c042030 con 0x7ff22c104260 2024-04-23T14:47:43.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.360+0000 7ff223400640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff2100a3c30 0x7ff2100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:43.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.361+0000 7ff223400640 1 -- 172.21.15.83:0/2141676400 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff21c0f0770 con 0x7ff22c104260 2024-04-23T14:47:43.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.361+0000 7ff22a000640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff2100a3c30 0x7ff2100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:43.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.361+0000 7ff22a000640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff2100a3c30 0x7ff2100a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2180095f0 tx=0x7ff218005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:43.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.365+0000 7ff223400640 1 -- 172.21.15.83:0/2141676400 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff21c0bcac0 con 0x7ff22c104260 2024-04-23T14:47:43.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:43 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/461148029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:47:43.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:43 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1050698778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:47:43.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.555+0000 7ff231600640 1 -- 172.21.15.83:0/2141676400 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff22c062c00 con 0x7ff22c104260 2024-04-23T14:47:43.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.558+0000 7ff223400640 1 -- 172.21.15.83:0/2141676400 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22699 (secure 0 0 0) 0x7ff21c0c0dc0 con 0x7ff22c104260 2024-04-23T14:47:43.559 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:47:43.559 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:46:58.351541+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:47:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.560+0000 7ff231600640 1 -- 172.21.15.83:0/2141676400 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff2100a3c30 msgr2=0x7ff2100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:43.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.560+0000 7ff231600640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff2100a3c30 0x7ff2100a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2180095f0 tx=0x7ff218005f90 comp rx=0 tx=0).stop 2024-04-23T14:47:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.560+0000 7ff231600640 1 -- 172.21.15.83:0/2141676400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c104260 msgr2=0x7ff22c19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.560+0000 7ff231600640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c104260 0x7ff22c19a530 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff21c009230 tx=0x7ff21c032ce0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.560+0000 7ff231600640 1 -- 172.21.15.83:0/2141676400 shutdown_connections 2024-04-23T14:47:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.560+0000 7ff231600640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff2100a3c30 0x7ff2100a60f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.560+0000 7ff231600640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff22c105e10 0x7ff22c1a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.560+0000 7ff231600640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff22c105450 0x7ff22c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.560+0000 7ff231600640 1 --2- 172.21.15.83:0/2141676400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff22c104260 0x7ff22c19a530 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.560+0000 7ff231600640 1 -- 172.21.15.83:0/2141676400 >> 172.21.15.83:0/2141676400 conn(0x7ff22c0ff7f0 msgr2=0x7ff22c108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.560+0000 7ff231600640 1 -- 172.21.15.83:0/2141676400 shutdown_connections 2024-04-23T14:47:43.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.560+0000 7ff231600640 1 -- 172.21.15.83:0/2141676400 wait complete. 2024-04-23T14:47:43.581 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:47:43.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:43 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/461148029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:47:43.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:43 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1050698778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:47:43.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.775+0000 7fdf57e00640 1 -- 172.21.15.83:0/4010850733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf58108480 msgr2=0x7fdf581088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:43.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.775+0000 7fdf57e00640 1 --2- 172.21.15.83:0/4010850733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf58108480 0x7fdf581088a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009900 tx=0x7fdf4802a3f0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.775+0000 7fdf57e00640 1 -- 172.21.15.83:0/4010850733 shutdown_connections 2024-04-23T14:47:43.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.775+0000 7fdf57e00640 1 --2- 172.21.15.83:0/4010850733 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf5810a030 0x7fdf581108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.775+0000 7fdf57e00640 1 --2- 172.21.15.83:0/4010850733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf58109670 0x7fdf58109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.775+0000 7fdf57e00640 1 --2- 172.21.15.83:0/4010850733 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf58108480 0x7fdf581088a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.775+0000 7fdf57e00640 1 -- 172.21.15.83:0/4010850733 >> 172.21.15.83:0/4010850733 conn(0x7fdf58077440 msgr2=0x7fdf58079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:43.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.775+0000 7fdf57e00640 1 -- 172.21.15.83:0/4010850733 shutdown_connections 2024-04-23T14:47:43.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.775+0000 7fdf57e00640 1 -- 172.21.15.83:0/4010850733 wait complete. 2024-04-23T14:47:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.776+0000 7fdf57e00640 1 Processor -- start 2024-04-23T14:47:43.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.776+0000 7fdf57e00640 1 -- start start 2024-04-23T14:47:43.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.776+0000 7fdf57e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf58108480 0x7fdf5819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:43.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.776+0000 7fdf57e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf58109670 0x7fdf5819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:43.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.776+0000 7fdf57e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf5810a030 0x7fdf581a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:43.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.776+0000 7fdf57e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58114160 con 0x7fdf58108480 2024-04-23T14:47:43.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.776+0000 7fdf57e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdf58113fe0 con 0x7fdf5810a030 2024-04-23T14:47:43.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.776+0000 7fdf57e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdf581142e0 con 0x7fdf58109670 2024-04-23T14:47:43.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.777+0000 7fdf56a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf58108480 0x7fdf5819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:43.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.777+0000 7fdf56a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf58108480 0x7fdf5819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54022/0 (socket says 172.21.15.83:54022) 2024-04-23T14:47:43.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.777+0000 7fdf56a00640 1 -- 172.21.15.83:0/633432596 learned_addr learned my addr 172.21.15.83:0/633432596 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:43.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.777+0000 7fdf4e000640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf58109670 0x7fdf5819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:43.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.777+0000 7fdf57400640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf5810a030 0x7fdf581a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:43.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.777+0000 7fdf56a00640 1 -- 172.21.15.83:0/633432596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf58109670 msgr2=0x7fdf5819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:43.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.777+0000 7fdf56a00640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf58109670 0x7fdf5819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.777+0000 7fdf56a00640 1 -- 172.21.15.83:0/633432596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf5810a030 msgr2=0x7fdf581a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:43.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.777+0000 7fdf56a00640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf5810a030 0x7fdf581a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:43.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.777+0000 7fdf56a00640 1 -- 172.21.15.83:0/633432596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf581a63b0 con 0x7fdf58108480 2024-04-23T14:47:43.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.777+0000 7fdf56a00640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf58108480 0x7fdf5819e7e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf48031440 tx=0x7fdf48032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:43.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.778+0000 7fdf4fe00640 1 -- 172.21.15.83:0/633432596 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4803f030 con 0x7fdf58108480 2024-04-23T14:47:43.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.778+0000 7fdf4fe00640 1 -- 172.21.15.83:0/633432596 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf48003ce0 con 0x7fdf58108480 2024-04-23T14:47:43.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.778+0000 7fdf4fe00640 1 -- 172.21.15.83:0/633432596 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf48035070 con 0x7fdf58108480 2024-04-23T14:47:43.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.778+0000 7fdf57e00640 1 -- 172.21.15.83:0/633432596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf581a6640 con 0x7fdf58108480 2024-04-23T14:47:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.778+0000 7fdf57e00640 1 -- 172.21.15.83:0/633432596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf581a6ae0 con 0x7fdf58108480 2024-04-23T14:47:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.779+0000 7fdf57e00640 1 -- 172.21.15.83:0/633432596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5810f7a0 con 0x7fdf58108480 2024-04-23T14:47:43.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.781+0000 7fdf4fe00640 1 -- 172.21.15.83:0/633432596 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdf48042030 con 0x7fdf58108480 2024-04-23T14:47:43.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.782+0000 7fdf4fe00640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fdf3c0a3bb0 0x7fdf3c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:43.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.782+0000 7fdf4fe00640 1 -- 172.21.15.83:0/633432596 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fdf480f09e0 con 0x7fdf58108480 2024-04-23T14:47:43.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.782+0000 7fdf4e000640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fdf3c0a3bb0 0x7fdf3c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:43.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.783+0000 7fdf4e000640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fdf3c0a3bb0 0x7fdf3c0a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdf400095f0 tx=0x7fdf40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:43.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:43.787+0000 7fdf4fe00640 1 -- 172.21.15.83:0/633432596 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf480bd380 con 0x7fdf58108480 2024-04-23T14:47:43.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:43 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/461148029' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:47:43.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:43 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1050698778' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:47:44.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.029+0000 7fdf57e00640 1 -- 172.21.15.83:0/633432596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf58062c30 con 0x7fdf58108480 2024-04-23T14:47:44.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.030+0000 7fdf4fe00640 1 -- 172.21.15.83:0/633432596 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7fdf480bcc80 con 0x7fdf58108480 2024-04-23T14:47:44.032 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:47:44.032 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:46:05.301806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:47:44.033 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-04-23T14:47:44.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.033+0000 7fdf57e00640 1 -- 172.21.15.83:0/633432596 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fdf3c0a3bb0 msgr2=0x7fdf3c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.033+0000 7fdf57e00640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fdf3c0a3bb0 0x7fdf3c0a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdf400095f0 tx=0x7fdf40005f90 comp rx=0 tx=0).stop 2024-04-23T14:47:44.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.033+0000 7fdf57e00640 1 -- 172.21.15.83:0/633432596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf58108480 msgr2=0x7fdf5819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.033+0000 7fdf57e00640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf58108480 0x7fdf5819e7e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf48031440 tx=0x7fdf48032ce0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.034+0000 7fdf57e00640 1 -- 172.21.15.83:0/633432596 shutdown_connections 2024-04-23T14:47:44.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.034+0000 7fdf57e00640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fdf3c0a3bb0 0x7fdf3c0a6070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.034+0000 7fdf57e00640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf5810a030 0x7fdf581a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.034+0000 7fdf57e00640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf58109670 0x7fdf5819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.034+0000 7fdf57e00640 1 --2- 172.21.15.83:0/633432596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdf58108480 0x7fdf5819e7e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.034+0000 7fdf57e00640 1 -- 172.21.15.83:0/633432596 >> 172.21.15.83:0/633432596 conn(0x7fdf58077440 msgr2=0x7fdf5810c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.034+0000 7fdf57e00640 1 -- 172.21.15.83:0/633432596 shutdown_connections 2024-04-23T14:47:44.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.034+0000 7fdf57e00640 1 -- 172.21.15.83:0/633432596 wait complete. 2024-04-23T14:47:44.054 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T14:45:59.723532+0000', 'modified': '2024-04-23T14:46:05.301806+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24599, 'mds_1': 24542, 'mds_2': 24532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/2384637189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 2384637189}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 2384637189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/2567101723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2567101723}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2567101723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6837/258138306', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 258138306}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 258138306}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24532': {'gid': 24532, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.203:6839/2628587422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 2628587422}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 2628587422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.173:6837/3951862333', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 3951862333}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 3951862333}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24599': {'gid': 24599, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6839/2485025097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 2485025097}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 2485025097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24599, 'qdb_cluster': [24532, 24542, 24599]} 2024-04-23T14:47:44.054 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-23T14:47:44.055 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T14:47:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.247+0000 7f0961e00640 1 -- 172.21.15.83:0/3365323531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f095c105450 msgr2=0x7f095c1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.247+0000 7f0961e00640 1 --2- 172.21.15.83:0/3365323531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f095c105450 0x7f095c1058d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0944009ca0 tx=0x7f094402a8c0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.247+0000 7f0961e00640 1 -- 172.21.15.83:0/3365323531 shutdown_connections 2024-04-23T14:47:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.247+0000 7f0961e00640 1 --2- 172.21.15.83:0/3365323531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f095c105e10 0x7f095c10c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.247+0000 7f0961e00640 1 --2- 172.21.15.83:0/3365323531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f095c105450 0x7f095c1058d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.247+0000 7f0961e00640 1 --2- 172.21.15.83:0/3365323531 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095c104260 0x7f095c104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.247+0000 7f0961e00640 1 -- 172.21.15.83:0/3365323531 >> 172.21.15.83:0/3365323531 conn(0x7f095c0ff7f0 msgr2=0x7f095c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.247+0000 7f0961e00640 1 -- 172.21.15.83:0/3365323531 shutdown_connections 2024-04-23T14:47:44.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.247+0000 7f0961e00640 1 -- 172.21.15.83:0/3365323531 wait complete. 2024-04-23T14:47:44.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.248+0000 7f0961e00640 1 Processor -- start 2024-04-23T14:47:44.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.248+0000 7f0961e00640 1 -- start start 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.248+0000 7f0961e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095c104260 0x7f095c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.248+0000 7f0961e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f095c105450 0x7f095c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.248+0000 7f0961e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f095c105e10 0x7f095c1a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.248+0000 7f0961e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10fef0 con 0x7f095c105e10 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.248+0000 7f0961e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10fd70 con 0x7f095c105450 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.248+0000 7f0961e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f095c110070 con 0x7f095c104260 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.249+0000 7f095b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f095c105e10 0x7f095c1a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.249+0000 7f095aa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095c104260 0x7f095c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.249+0000 7f095b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f095c105e10 0x7f095c1a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54044/0 (socket says 172.21.15.83:54044) 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.249+0000 7f095b400640 1 -- 172.21.15.83:0/4100320677 learned_addr learned my addr 172.21.15.83:0/4100320677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.249+0000 7f0952000640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f095c105450 0x7f095c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.249+0000 7f095b400640 1 -- 172.21.15.83:0/4100320677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095c104260 msgr2=0x7f095c19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.249+0000 7f095b400640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095c104260 0x7f095c19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.249+0000 7f095b400640 1 -- 172.21.15.83:0/4100320677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f095c105450 msgr2=0x7f095c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.249+0000 7f095b400640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f095c105450 0x7f095c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.249+0000 7f095b400640 1 -- 172.21.15.83:0/4100320677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c1a2140 con 0x7f095c105e10 2024-04-23T14:47:44.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.249+0000 7f095b400640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f095c105e10 0x7f095c1a1ad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f095400e970 tx=0x7f095400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.250+0000 7f0960a00640 1 -- 172.21.15.83:0/4100320677 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095400ccc0 con 0x7f095c105e10 2024-04-23T14:47:44.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.250+0000 7f0960a00640 1 -- 172.21.15.83:0/4100320677 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f095400ce60 con 0x7f095c105e10 2024-04-23T14:47:44.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.250+0000 7f0960a00640 1 -- 172.21.15.83:0/4100320677 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954010490 con 0x7f095c105e10 2024-04-23T14:47:44.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.250+0000 7f0961e00640 1 -- 172.21.15.83:0/4100320677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c1a2430 con 0x7f095c105e10 2024-04-23T14:47:44.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.250+0000 7f0961e00640 1 -- 172.21.15.83:0/4100320677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c1a2930 con 0x7f095c105e10 2024-04-23T14:47:44.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.253+0000 7f0964dcf740 1 Processor -- start 2024-04-23T14:47:44.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.253+0000 7f0960a00640 1 -- 172.21.15.83:0/4100320677 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0954005020 con 0x7f095c105e10 2024-04-23T14:47:44.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.253+0000 7f0964dcf740 1 -- start start 2024-04-23T14:47:44.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.253+0000 7f0964dcf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49b090 0x55ea0c49b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.253+0000 7f0964dcf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 0x55ea0c4a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.253+0000 7f0964dcf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c4a3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.253+0000 7f0964dcf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c400b10 con 0x55ea0c4a0bf0 2024-04-23T14:47:44.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.253+0000 7f0964dcf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c3a72b0 con 0x55ea0c49b090 2024-04-23T14:47:44.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.253+0000 7f0964dcf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c3fcad0 con 0x55ea0c49ba80 2024-04-23T14:47:44.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.254+0000 7f0952000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49b090 0x55ea0c49b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.254+0000 7f095aa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 0x55ea0c4a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.254+0000 7f095aa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 0x55ea0c4a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59992/0 (socket says 172.21.15.83:59992) 2024-04-23T14:47:44.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.254+0000 7f0952000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49b090 0x55ea0c49b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:34680/0 (socket says 172.21.15.83:34680) 2024-04-23T14:47:44.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.254+0000 7f095aa00640 1 -- 172.21.15.83:0/3554084369 learned_addr learned my addr 172.21.15.83:0/3554084369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.254+0000 7f0952000640 1 --2- 172.21.15.83:0/3554084369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c4a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.254+0000 7f0952000640 1 -- 172.21.15.83:0/3554084369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 msgr2=0x55ea0c4a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.254+0000 7f0952000640 1 --2- 172.21.15.83:0/3554084369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 0x55ea0c4a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.254+0000 7f0952000640 1 -- 172.21.15.83:0/3554084369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c4a0bf0 msgr2=0x55ea0c4a3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.254+0000 7f0952000640 1 --2- 172.21.15.83:0/3554084369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c4a3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.254+0000 7f0952000640 1 -- 172.21.15.83:0/3554084369 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0c1132d0 con 0x55ea0c49b090 2024-04-23T14:47:44.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.254+0000 7f095aa00640 1 --2- 172.21.15.83:0/3554084369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 0x55ea0c4a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:44.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.255+0000 7f0952000640 1 --2- 172.21.15.83:0/3554084369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49b090 0x55ea0c49b4b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0944031ae0 tx=0x7f0944031bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d6e496431716d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.255+0000 7f0960a00640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f09400a3bb0 0x7f09400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.255+0000 7f0960a00640 1 -- 172.21.15.83:0/4100320677 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f09540d0910 con 0x7f095c105e10 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.255+0000 7f094be00640 1 -- 172.21.15.83:0/3554084369 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944031db0 con 0x55ea0c49b090 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.255+0000 7f094be00640 1 -- 172.21.15.83:0/3554084369 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0944036040 con 0x55ea0c49b090 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.255+0000 7f094be00640 1 -- 172.21.15.83:0/3554084369 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440043d0 con 0x55ea0c49b090 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.255+0000 7f095aa00640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f09400a3bb0 0x7f09400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.256+0000 7f095aa00640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f09400a3bb0 0x7f09400a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55ea0c49be80 tx=0x7f094c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.256+0000 7f0964dcf740 1 -- 172.21.15.83:0/3554084369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49b090 msgr2=0x55ea0c49b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.256+0000 7f0964dcf740 1 --2- 172.21.15.83:0/3554084369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49b090 0x55ea0c49b4b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0944031ae0 tx=0x7f0944031bb0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.256+0000 7f0964dcf740 1 -- 172.21.15.83:0/3554084369 shutdown_connections 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.256+0000 7f0964dcf740 1 --2- 172.21.15.83:0/3554084369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c4a3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.256+0000 7f0964dcf740 1 --2- 172.21.15.83:0/3554084369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 0x55ea0c4a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.256+0000 7f0964dcf740 1 --2- 172.21.15.83:0/3554084369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49b090 0x55ea0c49b4b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.256+0000 7f0964dcf740 1 -- 172.21.15.83:0/3554084369 >> 172.21.15.83:0/3554084369 conn(0x55ea0c496700 msgr2=0x55ea0c496b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.256+0000 7f0964dcf740 1 -- 172.21.15.83:0/3554084369 shutdown_connections 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.256+0000 7f0964dcf740 1 -- 172.21.15.83:0/3554084369 wait complete. 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.257+0000 7f0964dcf740 1 -- start start 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.257+0000 7f0964dcf740 1 Processor -- start 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f0964dcf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4a5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f0964dcf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 0x55ea0c4a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f0964dcf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c4ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f0964dcf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c4a39c0 con 0x55ea0c49b090 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f0964dcf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c3ba590 con 0x55ea0c4a0bf0 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f0964dcf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c4a3b40 con 0x55ea0c49ba80 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f0964dcf740 10 client.0 _ll_register_callbacks cb 0x55ea0c39d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f095aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f095aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4a5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54060/0 (socket says 172.21.15.83:54060) 2024-04-23T14:47:44.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f095aa00640 1 -- 172.21.15.83:0/885829552 learned_addr learned my addr 172.21.15.83:0/885829552 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f0952000640 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 0x55ea0c4a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f095b400640 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c4ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f095aa00640 1 -- 172.21.15.83:0/885829552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 msgr2=0x55ea0c4a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f095aa00640 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 0x55ea0c4a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f095aa00640 1 -- 172.21.15.83:0/885829552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c4a0bf0 msgr2=0x55ea0c4ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f095aa00640 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c4ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f095aa00640 1 -- 172.21.15.83:0/885829552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0c1dd6b0 con 0x55ea0c49b090 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.258+0000 7f0952000640 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 0x55ea0c4a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.259+0000 7f095aa00640 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4a5370 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f094c004610 tx=0x7f094c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3ac9da5e49d954 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.259+0000 7f0948c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.259+0000 7f0948c00640 1 -- 172.21.15.83:0/885829552 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c00f690 con 0x55ea0c49b090 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.259+0000 7f0948c00640 1 -- 172.21.15.83:0/885829552 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f094c00f830 con 0x55ea0c49b090 2024-04-23T14:47:44.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.259+0000 7f0964dcf740 1 -- 172.21.15.83:0/885829552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0944036040 con 0x55ea0c49b090 2024-04-23T14:47:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.259+0000 7f0948c00640 1 -- 172.21.15.83:0/885829552 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f094c00fa30 con 0x55ea0c49b090 2024-04-23T14:47:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.259+0000 7f0964dcf740 10 client.14889 fetch_fsmap learned FSMap version 23 2024-04-23T14:47:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.259+0000 7f0964dcf740 1 -- 172.21.15.83:0/885829552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ea0c4afd30 con 0x55ea0c49b090 2024-04-23T14:47:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.260+0000 7f0948c00640 1 -- 172.21.15.83:0/885829552 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0944036040 con 0x55ea0c49b090 2024-04-23T14:47:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.260+0000 7f0948c00640 1 -- 172.21.15.83:0/885829552 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f094c027070 con 0x55ea0c49b090 2024-04-23T14:47:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.260+0000 7f0964dcf740 10 client.14889 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T14:47:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.260+0000 7f0964dcf740 10 client.14889 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T14:47:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.260+0000 7f0964dcf740 20 client.14889 populate_metadata read hostname 'smithi083' 2024-04-23T14:47:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.260+0000 7f0964dcf740 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ea0c4b23f0 0x55ea0c4b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.260+0000 7f0964dcf740 4 client.14889 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:47:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.260+0000 7f0964dcf740 1 -- 172.21.15.83:0/885829552 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea0c40e530 con 0x55ea0c4b23f0 2024-04-23T14:47:44.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.260+0000 7f0952000640 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ea0c4b23f0 0x55ea0c4b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.261+0000 7f0952000640 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ea0c4b23f0 0x55ea0c4b3860 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac4afd98ad751580 server_cookie=969a4367f9181a16 in_seq=0 out_seq=0 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.261+0000 7f0948c00640 10 client.14889 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.262+0000 7f0948c00640 1 -- 172.21.15.83:0/885829552 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea0c40e530 con 0x55ea0c4b23f0 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.262+0000 7f0948c00640 10 client.14889 handle_command_reply: tid=0 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.262+0000 7f0964dcf740 1 client.14889 shutdown 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.262+0000 7f0964dcf740 1 -- 172.21.15.83:0/885829552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 msgr2=0x55ea0c4a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.262+0000 7f0964dcf740 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4a5370 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f094c004610 tx=0x7f094c004730 comp rx=0 tx=0).stop 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.263+0000 7f0964dcf740 1 -- 172.21.15.83:0/885829552 shutdown_connections 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.263+0000 7f0964dcf740 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ea0c4b23f0 0x55ea0c4b3860 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.263+0000 7f0964dcf740 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c4ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.263+0000 7f0964dcf740 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c49ba80 0x55ea0c4a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.263+0000 7f0948c00640 0 client.14889 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:47:44.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.263+0000 7f0964dcf740 1 --2- 172.21.15.83:0/885829552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4a5370 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.263+0000 7f0964dcf740 1 -- 172.21.15.83:0/885829552 >> 172.21.15.83:0/885829552 conn(0x55ea0c496700 msgr2=0x55ea0c4958c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.263+0000 7f0964dcf740 1 -- 172.21.15.83:0/885829552 shutdown_connections 2024-04-23T14:47:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.263+0000 7f0964dcf740 1 -- 172.21.15.83:0/885829552 wait complete. 2024-04-23T14:47:44.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.263+0000 7f0964dcf740 20 client.14889 trim_cache size 0 max 16384 2024-04-23T14:47:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.282+0000 7f0964dcf740 1 Processor -- start 2024-04-23T14:47:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.282+0000 7f0964dcf740 1 -- start start 2024-04-23T14:47:44.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.282+0000 7f0964dcf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.282+0000 7f0964dcf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49ba80 0x55ea0c4ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.282+0000 7f0964dcf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c49cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.282+0000 7f0964dcf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c4b1ba0 con 0x55ea0c49b090 2024-04-23T14:47:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.282+0000 7f0964dcf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c4b2270 con 0x55ea0c49ba80 2024-04-23T14:47:44.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.282+0000 7f0964dcf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c4b2940 con 0x55ea0c4a0bf0 2024-04-23T14:47:44.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.282+0000 7f095aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.282+0000 7f095aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4a2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54064/0 (socket says 172.21.15.83:54064) 2024-04-23T14:47:44.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.282+0000 7f095aa00640 1 -- 172.21.15.83:0/2777204500 learned_addr learned my addr 172.21.15.83:0/2777204500 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.282+0000 7f0952000640 1 --2- 172.21.15.83:0/2777204500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49ba80 0x55ea0c4ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.283+0000 7f095b400640 1 --2- 172.21.15.83:0/2777204500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c49cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.283+0000 7f095aa00640 1 -- 172.21.15.83:0/2777204500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c4a0bf0 msgr2=0x55ea0c49cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.283+0000 7f095aa00640 1 --2- 172.21.15.83:0/2777204500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c49cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.283+0000 7f095aa00640 1 -- 172.21.15.83:0/2777204500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49ba80 msgr2=0x55ea0c4ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.283+0000 7f095aa00640 1 --2- 172.21.15.83:0/2777204500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49ba80 0x55ea0c4ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.283+0000 7f095aa00640 1 -- 172.21.15.83:0/2777204500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea0c496530 con 0x55ea0c49b090 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.283+0000 7f095aa00640 1 --2- 172.21.15.83:0/2777204500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4a2fe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f094c009b50 tx=0x7f094c009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a6d8fd79451aac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.283+0000 7f094b400640 1 -- 172.21.15.83:0/2777204500 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c01f430 con 0x55ea0c49b090 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.283+0000 7f094b400640 1 -- 172.21.15.83:0/2777204500 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f094c038070 con 0x55ea0c49b090 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.283+0000 7f094b400640 1 -- 172.21.15.83:0/2777204500 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c0296a0 con 0x55ea0c49b090 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.284+0000 7f0964dcf740 1 -- 172.21.15.83:0/2777204500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 msgr2=0x55ea0c4a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.284+0000 7f0964dcf740 1 --2- 172.21.15.83:0/2777204500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4a2fe0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f094c009b50 tx=0x7f094c009df0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.284+0000 7f0964dcf740 1 -- 172.21.15.83:0/2777204500 shutdown_connections 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.284+0000 7f0964dcf740 1 --2- 172.21.15.83:0/2777204500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c49cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.284+0000 7f0964dcf740 1 --2- 172.21.15.83:0/2777204500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49ba80 0x55ea0c4ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.284+0000 7f0964dcf740 1 --2- 172.21.15.83:0/2777204500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4a2fe0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.284+0000 7f0964dcf740 1 -- 172.21.15.83:0/2777204500 >> 172.21.15.83:0/2777204500 conn(0x55ea0c496700 msgr2=0x55ea0c491750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.284+0000 7f0964dcf740 1 -- 172.21.15.83:0/2777204500 shutdown_connections 2024-04-23T14:47:44.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.284+0000 7f0964dcf740 1 -- 172.21.15.83:0/2777204500 wait complete. 2024-04-23T14:47:44.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.285+0000 7f0964dcf740 1 -- start start 2024-04-23T14:47:44.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.285+0000 7f0964dcf740 1 Processor -- start 2024-04-23T14:47:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.285+0000 7f0964dcf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.285+0000 7f0964dcf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49ba80 0x55ea0c4aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.285+0000 7f0964dcf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c4ab520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.285+0000 7f0964dcf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c4b36e0 con 0x55ea0c49b090 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.285+0000 7f0964dcf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c4b3010 con 0x55ea0c49ba80 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.285+0000 7f0964dcf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ea0c4b3db0 con 0x55ea0c4a0bf0 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.285+0000 7f0964dcf740 10 client.0 _ll_register_callbacks cb 0x7f094c0044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f095aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f095aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4aaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54074/0 (socket says 172.21.15.83:54074) 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f095aa00640 1 -- 172.21.15.83:0/3677217034 learned_addr learned my addr 172.21.15.83:0/3677217034 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f095b400640 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c4ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f0952000640 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49ba80 0x55ea0c4aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f095aa00640 1 -- 172.21.15.83:0/3677217034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c4a0bf0 msgr2=0x55ea0c4ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f095aa00640 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c4ab520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f095aa00640 1 -- 172.21.15.83:0/3677217034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49ba80 msgr2=0x55ea0c4aafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f095aa00640 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49ba80 0x55ea0c4aafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f095aa00640 1 -- 172.21.15.83:0/3677217034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea0c487e20 con 0x55ea0c49b090 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f095aa00640 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4aaaa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f094c00f9c0 tx=0x7f094c00fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1fb5c0312d7fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f0948c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f0948c00640 1 -- 172.21.15.83:0/3677217034 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c041070 con 0x55ea0c49b090 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f0948c00640 1 -- 172.21.15.83:0/3677217034 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f094c005450 con 0x55ea0c49b090 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.286+0000 7f0964dcf740 1 -- 172.21.15.83:0/3677217034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f094c038070 con 0x55ea0c49b090 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.287+0000 7f0948c00640 1 -- 172.21.15.83:0/3677217034 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f094c04d050 con 0x55ea0c49b090 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.287+0000 7f0964dcf740 10 client.14895 fetch_fsmap learned FSMap version 23 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.287+0000 7f0964dcf740 1 -- 172.21.15.83:0/3677217034 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ea0c5183a0 con 0x55ea0c49b090 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.287+0000 7f0948c00640 1 -- 172.21.15.83:0/3677217034 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f094c038070 con 0x55ea0c49b090 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.287+0000 7f0948c00640 1 -- 172.21.15.83:0/3677217034 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f094c027050 con 0x55ea0c49b090 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.287+0000 7f0964dcf740 10 client.14895 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.287+0000 7f0964dcf740 10 client.14895 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T14:47:44.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.287+0000 7f0964dcf740 20 client.14895 populate_metadata read hostname 'smithi083' 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.287+0000 7f0964dcf740 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ea0c51cb70 0x55ea0c519680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.287+0000 7f0964dcf740 4 client.14895 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.287+0000 7f0964dcf740 1 -- 172.21.15.83:0/3677217034 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "status"}) -- 0x55ea0c4083a0 con 0x55ea0c51cb70 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.288+0000 7f0952000640 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ea0c51cb70 0x55ea0c519680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.288+0000 7f0948c00640 10 client.14895 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.288+0000 7f0952000640 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ea0c51cb70 0x55ea0c519680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5521a048d0bf0e06 server_cookie=81b3ae97f04a4ee0 in_seq=0 out_seq=0 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.288+0000 7f0948c00640 1 -- 172.21.15.83:0/3677217034 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ea0c4083a0 con 0x55ea0c51cb70 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.288+0000 7f0948c00640 10 client.14895 handle_command_reply: tid=0 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.288+0000 7f0964dcf740 1 client.14895 shutdown 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.288+0000 7f0964dcf740 1 -- 172.21.15.83:0/3677217034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 msgr2=0x55ea0c4aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.288+0000 7f0964dcf740 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4aaaa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f094c00f9c0 tx=0x7f094c00fbd0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.288+0000 7f0964dcf740 1 -- 172.21.15.83:0/3677217034 shutdown_connections 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.288+0000 7f0964dcf740 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ea0c51cb70 0x55ea0c519680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.288+0000 7f0964dcf740 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ea0c4a0bf0 0x55ea0c4ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.289+0000 7f0964dcf740 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ea0c49ba80 0x55ea0c4aafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.289+0000 7f0948c00640 0 client.14895 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:47:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.289+0000 7f0964dcf740 1 --2- 172.21.15.83:0/3677217034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ea0c49b090 0x55ea0c4aaaa0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.289+0000 7f0964dcf740 1 -- 172.21.15.83:0/3677217034 >> 172.21.15.83:0/3677217034 conn(0x55ea0c496700 msgr2=0x55ea0c4a0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.289+0000 7f0964dcf740 1 -- 172.21.15.83:0/3677217034 shutdown_connections 2024-04-23T14:47:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.289+0000 7f0964dcf740 1 -- 172.21.15.83:0/3677217034 wait complete. 2024-04-23T14:47:44.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.289+0000 7f0964dcf740 20 client.14895 trim_cache size 0 max 16384 2024-04-23T14:47:44.295 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:47:44.295 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5f4fe708-017e-11ef-bc93-c7b262605968", 2024-04-23T14:47:44.295 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 2, 2024-04-23T14:47:44.296 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24532, 2024-04-23T14:47:44.296 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-23T14:47:44.296 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-23T14:47:44.296 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-23T14:47:44.296 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 99.994946640999999, 2024-04-23T14:47:44.296 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 23, 2024-04-23T14:47:44.296 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-23T14:47:44.296 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T14:47:44.296 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 121.590494185 2024-04-23T14:47:44.296 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:47:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.289+0000 7f0961e00640 1 -- 172.21.15.83:0/4100320677 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f09400a3bb0 msgr2=0x7f09400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.289+0000 7f0961e00640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f09400a3bb0 0x7f09400a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55ea0c49be80 tx=0x7f094c005f90 comp rx=0 tx=0).stop 2024-04-23T14:47:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.289+0000 7f0961e00640 1 -- 172.21.15.83:0/4100320677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f095c105e10 msgr2=0x7f095c1a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.289+0000 7f0961e00640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f095c105e10 0x7f095c1a1ad0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f095400e970 tx=0x7f095400ed70 comp rx=0 tx=0).stop 2024-04-23T14:47:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.290+0000 7f0961e00640 1 -- 172.21.15.83:0/4100320677 shutdown_connections 2024-04-23T14:47:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.290+0000 7f0961e00640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f09400a3bb0 0x7f09400a6070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.290+0000 7f0961e00640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f095c105e10 0x7f095c1a1ad0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.290+0000 7f0961e00640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f095c105450 0x7f095c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.290+0000 7f0961e00640 1 --2- 172.21.15.83:0/4100320677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095c104260 0x7f095c19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.290+0000 7f0961e00640 1 -- 172.21.15.83:0/4100320677 >> 172.21.15.83:0/4100320677 conn(0x7f095c0ff7f0 msgr2=0x7f095c108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.290+0000 7f0961e00640 1 -- 172.21.15.83:0/4100320677 shutdown_connections 2024-04-23T14:47:44.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.290+0000 7f0961e00640 1 -- 172.21.15.83:0/4100320677 wait complete. 2024-04-23T14:47:44.310 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T14:47:44.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.503+0000 7f6a5f000640 1 -- 172.21.15.83:0/4294725146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a58103060 msgr2=0x7f6a581034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.503+0000 7f6a5f000640 1 --2- 172.21.15.83:0/4294725146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a58103060 0x7f6a581034e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009900 tx=0x7f6a4402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.503+0000 7f6a5f000640 1 -- 172.21.15.83:0/4294725146 shutdown_connections 2024-04-23T14:47:44.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.503+0000 7f6a5f000640 1 --2- 172.21.15.83:0/4294725146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a58103a20 0x7f6a5810a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.503+0000 7f6a5f000640 1 --2- 172.21.15.83:0/4294725146 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a58103060 0x7f6a581034e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.503+0000 7f6a5f000640 1 --2- 172.21.15.83:0/4294725146 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58101f10 0x7f6a58102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.503+0000 7f6a5f000640 1 -- 172.21.15.83:0/4294725146 >> 172.21.15.83:0/4294725146 conn(0x7f6a580fd4a0 msgr2=0x7f6a580ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.503+0000 7f6a5f000640 1 -- 172.21.15.83:0/4294725146 shutdown_connections 2024-04-23T14:47:44.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.503+0000 7f6a5f000640 1 -- 172.21.15.83:0/4294725146 wait complete. 2024-04-23T14:47:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.504+0000 7f6a5f000640 1 Processor -- start 2024-04-23T14:47:44.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.504+0000 7f6a5f000640 1 -- start start 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5f000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a58101f10 0x7f6a5819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5f000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58103060 0x7f6a5819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5f000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a58103a20 0x7f6a581a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5f000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5810dc10 con 0x7f6a58103060 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5f000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5810da90 con 0x7f6a58103a20 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5f000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5810dd90 con 0x7f6a58101f10 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58103060 0x7f6a5819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58103060 0x7f6a5819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54096/0 (socket says 172.21.15.83:54096) 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5d200640 1 -- 172.21.15.83:0/1451135242 learned_addr learned my addr 172.21.15.83:0/1451135242 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5e600640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a58103a20 0x7f6a581a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5dc00640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a58101f10 0x7f6a5819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5d200640 1 -- 172.21.15.83:0/1451135242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a58101f10 msgr2=0x7f6a5819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5d200640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a58101f10 0x7f6a5819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5d200640 1 -- 172.21.15.83:0/1451135242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a58103a20 msgr2=0x7f6a581a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5d200640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a58103a20 0x7f6a581a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5d200640 1 -- 172.21.15.83:0/1451135242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a581a2090 con 0x7f6a58103060 2024-04-23T14:47:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.505+0000 7f6a5e600640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a58103a20 0x7f6a581a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:47:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.506+0000 7f6a5d200640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58103060 0x7f6a5819a9b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6a44031440 tx=0x7f6a44032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.506+0000 7f6a56000640 1 -- 172.21.15.83:0/1451135242 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4403f030 con 0x7f6a58103060 2024-04-23T14:47:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.506+0000 7f6a56000640 1 -- 172.21.15.83:0/1451135242 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a44003ce0 con 0x7f6a58103060 2024-04-23T14:47:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.506+0000 7f6a5f000640 1 -- 172.21.15.83:0/1451135242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a581a2320 con 0x7f6a58103060 2024-04-23T14:47:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.506+0000 7f6a56000640 1 -- 172.21.15.83:0/1451135242 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44035070 con 0x7f6a58103060 2024-04-23T14:47:44.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.506+0000 7f6a5f000640 1 -- 172.21.15.83:0/1451135242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a581a2810 con 0x7f6a58103060 2024-04-23T14:47:44.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.509+0000 7f6a6341c740 1 Processor -- start 2024-04-23T14:47:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.509+0000 7f6a56000640 1 -- 172.21.15.83:0/1451135242 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6a44042030 con 0x7f6a58103060 2024-04-23T14:47:44.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.509+0000 7f6a6341c740 1 -- start start 2024-04-23T14:47:44.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a6341c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3921a70 0x5654a3921e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a6341c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a3922460 0x5654a3926f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a6341c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a39275d0 0x5654a3929a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a6341c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5654a387eb10 con 0x5654a3921a70 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a6341c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5654a38252b0 con 0x5654a3922460 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a6341c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5654a387aad0 con 0x5654a39275d0 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a5e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3921a70 0x5654a3921e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a5e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3921a70 0x5654a3921e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54102/0 (socket says 172.21.15.83:54102) 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a5d200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a39275d0 0x5654a3929a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a5e600640 1 -- 172.21.15.83:0/1011759271 learned_addr learned my addr 172.21.15.83:0/1011759271 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a5dc00640 1 --2- 172.21.15.83:0/1011759271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a3922460 0x5654a3926f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a5e600640 1 -- 172.21.15.83:0/1011759271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a39275d0 msgr2=0x5654a3929a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.510+0000 7f6a5e600640 1 --2- 172.21.15.83:0/1011759271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a39275d0 0x5654a3929a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.511+0000 7f6a5e600640 1 -- 172.21.15.83:0/1011759271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a3922460 msgr2=0x5654a3926f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.511+0000 7f6a5e600640 1 --2- 172.21.15.83:0/1011759271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a3922460 0x5654a3926f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.511+0000 7f6a5e600640 1 -- 172.21.15.83:0/1011759271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a35912d0 con 0x5654a3921a70 2024-04-23T14:47:44.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.511+0000 7f6a5d200640 1 --2- 172.21.15.83:0/1011759271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a39275d0 0x5654a3929a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:44.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.511+0000 7f6a5e600640 1 --2- 172.21.15.83:0/1011759271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3921a70 0x5654a3921e90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009cc0 tx=0x7f6a500117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed68d516c55be1a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.511+0000 7f6a56000640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6a400a3bb0 0x7f6a400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.511+0000 7f6a4aa00640 1 -- 172.21.15.83:0/1011759271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50012450 con 0x5654a3921a70 2024-04-23T14:47:44.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.511+0000 7f6a4aa00640 1 -- 172.21.15.83:0/1011759271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a50012ad0 con 0x5654a3921a70 2024-04-23T14:47:44.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.511+0000 7f6a4aa00640 1 -- 172.21.15.83:0/1011759271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5000fc90 con 0x5654a3921a70 2024-04-23T14:47:44.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.511+0000 7f6a56000640 1 -- 172.21.15.83:0/1451135242 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6a440f0850 con 0x7f6a58103060 2024-04-23T14:47:44.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.511+0000 7f6a5dc00640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6a400a3bb0 0x7f6a400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.512+0000 7f6a6341c740 1 -- 172.21.15.83:0/1011759271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3921a70 msgr2=0x5654a3921e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.512+0000 7f6a6341c740 1 --2- 172.21.15.83:0/1011759271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3921a70 0x5654a3921e90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009cc0 tx=0x7f6a500117b0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.512+0000 7f6a5dc00640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6a400a3bb0 0x7f6a400a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x5654a3922860 tx=0x7f6a4c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.512+0000 7f6a6341c740 1 -- 172.21.15.83:0/1011759271 shutdown_connections 2024-04-23T14:47:44.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.512+0000 7f6a6341c740 1 --2- 172.21.15.83:0/1011759271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a39275d0 0x5654a3929a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.512+0000 7f6a6341c740 1 --2- 172.21.15.83:0/1011759271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a3922460 0x5654a3926f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.512+0000 7f6a6341c740 1 --2- 172.21.15.83:0/1011759271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3921a70 0x5654a3921e90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.512+0000 7f6a6341c740 1 -- 172.21.15.83:0/1011759271 >> 172.21.15.83:0/1011759271 conn(0x5654a39148d0 msgr2=0x5654a3914cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.512+0000 7f6a6341c740 1 -- 172.21.15.83:0/1011759271 shutdown_connections 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.512+0000 7f6a6341c740 1 -- 172.21.15.83:0/1011759271 wait complete. 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.513+0000 7f6a6341c740 1 -- start start 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.513+0000 7f6a6341c740 1 Processor -- start 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.513+0000 7f6a6341c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3921a70 0x5654a392be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.513+0000 7f6a6341c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 0x5654a392c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.513+0000 7f6a6341c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a39275d0 0x5654a39333f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.513+0000 7f6a6341c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5654a392a580 con 0x5654a3922460 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.513+0000 7f6a6341c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5654a3838590 con 0x5654a39275d0 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.513+0000 7f6a6341c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5654a392a700 con 0x5654a3921a70 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.513+0000 7f6a6341c740 10 client.0 _ll_register_callbacks cb 0x5654a381b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 0x5654a392c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 0x5654a392c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54104/0 (socket says 172.21.15.83:54104) 2024-04-23T14:47:44.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5d200640 1 -- 172.21.15.83:0/268215875 learned_addr learned my addr 172.21.15.83:0/268215875 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5dc00640 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3921a70 0x5654a392be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5e600640 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a39275d0 0x5654a39333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5d200640 1 -- 172.21.15.83:0/268215875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3921a70 msgr2=0x5654a392be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5d200640 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3921a70 0x5654a392be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5d200640 1 -- 172.21.15.83:0/268215875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a39275d0 msgr2=0x5654a39333f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5d200640 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a39275d0 0x5654a39333f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5d200640 1 -- 172.21.15.83:0/268215875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a365b6b0 con 0x5654a3922460 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5dc00640 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3921a70 0x5654a392be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5e600640 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a39275d0 0x5654a39333f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a5d200640 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 0x5654a392c3b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a440bca80 tx=0x7f6a44009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2956cef55f53fd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a3f400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a3f400640 1 -- 172.21.15.83:0/268215875 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44049070 con 0x5654a3922460 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a3f400640 1 -- 172.21.15.83:0/268215875 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6a44003800 con 0x5654a3922460 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.514+0000 7f6a6341c740 1 -- 172.21.15.83:0/268215875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a50012ad0 con 0x5654a3922460 2024-04-23T14:47:44.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.515+0000 7f6a3f400640 1 -- 172.21.15.83:0/268215875 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6a440bc630 con 0x5654a3922460 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.515+0000 7f6a6341c740 10 client.14907 fetch_fsmap learned FSMap version 23 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.515+0000 7f6a6341c740 1 -- 172.21.15.83:0/268215875 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5654a3936760 con 0x5654a3922460 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.515+0000 7f6a3f400640 1 -- 172.21.15.83:0/268215875 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a50012ad0 con 0x5654a3922460 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.515+0000 7f6a3f400640 1 -- 172.21.15.83:0/268215875 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f6a440b9160 con 0x5654a3922460 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.515+0000 7f6a6341c740 10 client.14907 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.515+0000 7f6a6341c740 10 client.14907 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.515+0000 7f6a6341c740 20 client.14907 populate_metadata read hostname 'smithi083' 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.515+0000 7f6a6341c740 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5654a3938dc0 0x5654a393a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.515+0000 7f6a6341c740 4 client.14907 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.515+0000 7f6a6341c740 1 -- 172.21.15.83:0/268215875 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654a388c530 con 0x5654a3938dc0 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.516+0000 7f6a5dc00640 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5654a3938dc0 0x5654a393a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.516+0000 7f6a5dc00640 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5654a3938dc0 0x5654a393a230 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=888cfa681ce5f1ac server_cookie=98fef633b1b22f83 in_seq=0 out_seq=0 2024-04-23T14:47:44.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.516+0000 7f6a3f400640 10 client.14907 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:47:44.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.517+0000 7f6a3f400640 1 -- 172.21.15.83:0/268215875 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654a388c530 con 0x5654a3938dc0 2024-04-23T14:47:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.517+0000 7f6a3f400640 10 client.14907 handle_command_reply: tid=0 2024-04-23T14:47:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.517+0000 7f6a6341c740 1 client.14907 shutdown 2024-04-23T14:47:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.518+0000 7f6a6341c740 1 -- 172.21.15.83:0/268215875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 msgr2=0x5654a392c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.518+0000 7f6a6341c740 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 0x5654a392c3b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6a440bca80 tx=0x7f6a44009620 comp rx=0 tx=0).stop 2024-04-23T14:47:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.518+0000 7f6a6341c740 1 -- 172.21.15.83:0/268215875 shutdown_connections 2024-04-23T14:47:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.518+0000 7f6a6341c740 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a39275d0 0x5654a39333f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.518+0000 7f6a6341c740 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 0x5654a392c3b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.518+0000 7f6a6341c740 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5654a3938dc0 0x5654a393a230 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.518+0000 7f6a6341c740 1 --2- 172.21.15.83:0/268215875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3921a70 0x5654a392be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.518+0000 7f6a3f400640 0 client.14907 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:47:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.518+0000 7f6a6341c740 1 -- 172.21.15.83:0/268215875 >> 172.21.15.83:0/268215875 conn(0x5654a39148d0 msgr2=0x5654a3913970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.518+0000 7f6a6341c740 1 -- 172.21.15.83:0/268215875 shutdown_connections 2024-04-23T14:47:44.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.518+0000 7f6a6341c740 1 -- 172.21.15.83:0/268215875 wait complete. 2024-04-23T14:47:44.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.518+0000 7f6a6341c740 20 client.14907 trim_cache size 0 max 16384 2024-04-23T14:47:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.538+0000 7f6a6341c740 1 Processor -- start 2024-04-23T14:47:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.538+0000 7f6a6341c740 1 -- start start 2024-04-23T14:47:44.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.538+0000 7f6a6341c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3921a70 0x5654a3926e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.538+0000 7f6a6341c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 0x5654a39333b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.538+0000 7f6a6341c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a39275d0 0x5654a3937ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.538+0000 7f6a6341c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5654a3938570 con 0x5654a3922460 2024-04-23T14:47:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.538+0000 7f6a6341c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5654a3938c40 con 0x5654a39275d0 2024-04-23T14:47:44.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.538+0000 7f6a6341c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5654a3939310 con 0x5654a3921a70 2024-04-23T14:47:44.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.538+0000 7f6a5d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 0x5654a39333b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a5d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 0x5654a39333b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54114/0 (socket says 172.21.15.83:54114) 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a5d200640 1 -- 172.21.15.83:0/4062163829 learned_addr learned my addr 172.21.15.83:0/4062163829 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a5dc00640 1 --2- 172.21.15.83:0/4062163829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3921a70 0x5654a3926e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a5e600640 1 --2- 172.21.15.83:0/4062163829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a39275d0 0x5654a3937ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a5d200640 1 -- 172.21.15.83:0/4062163829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3921a70 msgr2=0x5654a3926e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a5d200640 1 --2- 172.21.15.83:0/4062163829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3921a70 0x5654a3926e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a5d200640 1 -- 172.21.15.83:0/4062163829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a39275d0 msgr2=0x5654a3937ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a5d200640 1 --2- 172.21.15.83:0/4062163829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a39275d0 0x5654a3937ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a5d200640 1 -- 172.21.15.83:0/4062163829 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a3922cf0 con 0x5654a3922460 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a5e600640 1 --2- 172.21.15.83:0/4062163829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a39275d0 0x5654a3937ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a5d200640 1 --2- 172.21.15.83:0/4062163829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 0x5654a39333b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6a440bada0 tx=0x7f6a4402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689e40adb8d92e35 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a4a000640 1 -- 172.21.15.83:0/4062163829 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44049070 con 0x5654a3922460 2024-04-23T14:47:44.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a4a000640 1 -- 172.21.15.83:0/4062163829 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a440bd1f0 con 0x5654a3922460 2024-04-23T14:47:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.539+0000 7f6a4a000640 1 -- 172.21.15.83:0/4062163829 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44005d50 con 0x5654a3922460 2024-04-23T14:47:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.540+0000 7f6a6341c740 1 -- 172.21.15.83:0/4062163829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 msgr2=0x5654a39333b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.540+0000 7f6a6341c740 1 --2- 172.21.15.83:0/4062163829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 0x5654a39333b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6a440bada0 tx=0x7f6a4402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.540+0000 7f6a6341c740 1 -- 172.21.15.83:0/4062163829 shutdown_connections 2024-04-23T14:47:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.540+0000 7f6a6341c740 1 --2- 172.21.15.83:0/4062163829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a39275d0 0x5654a3937ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.540+0000 7f6a6341c740 1 --2- 172.21.15.83:0/4062163829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a3922460 0x5654a39333b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.540+0000 7f6a6341c740 1 --2- 172.21.15.83:0/4062163829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3921a70 0x5654a3926e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.540+0000 7f6a6341c740 1 -- 172.21.15.83:0/4062163829 >> 172.21.15.83:0/4062163829 conn(0x5654a39148d0 msgr2=0x5654a3914cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.540+0000 7f6a6341c740 1 -- 172.21.15.83:0/4062163829 shutdown_connections 2024-04-23T14:47:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.540+0000 7f6a6341c740 1 -- 172.21.15.83:0/4062163829 wait complete. 2024-04-23T14:47:44.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.541+0000 7f6a6341c740 1 -- start start 2024-04-23T14:47:44.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.541+0000 7f6a6341c740 1 Processor -- start 2024-04-23T14:47:44.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.541+0000 7f6a6341c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a3921a70 0x5654a392f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.541+0000 7f6a6341c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3922460 0x5654a392faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.541+0000 7f6a6341c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a39275d0 0x5654a39323a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.541+0000 7f6a6341c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5654a393a0b0 con 0x5654a39275d0 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.541+0000 7f6a6341c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5654a39399e0 con 0x5654a3921a70 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.541+0000 7f6a6341c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5654a393a780 con 0x5654a3922460 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.541+0000 7f6a6341c740 10 client.0 _ll_register_callbacks cb 0x7f6a44003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a5e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a39275d0 0x5654a39323a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a5d200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3922460 0x5654a392faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a5d200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3922460 0x5654a392faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:60076/0 (socket says 172.21.15.83:60076) 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a5d200640 1 -- 172.21.15.83:0/1140995874 learned_addr learned my addr 172.21.15.83:0/1140995874 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a5dc00640 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a3921a70 0x5654a392f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a5d200640 1 -- 172.21.15.83:0/1140995874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a3921a70 msgr2=0x5654a392f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a5d200640 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a3921a70 0x5654a392f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a5d200640 1 -- 172.21.15.83:0/1140995874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a39275d0 msgr2=0x5654a39323a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a5d200640 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a39275d0 0x5654a39323a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a5d200640 1 -- 172.21.15.83:0/1140995874 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a39329e0 con 0x5654a3922460 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a5e600640 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a39275d0 0x5654a39323a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a5d200640 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3922460 0x5654a392faa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6a440f62d0 tx=0x7f6a440b9390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ae996a12f9b40fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a3f400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.542+0000 7f6a3f400640 1 -- 172.21.15.83:0/1140995874 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44049050 con 0x5654a3922460 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a6341c740 1 -- 172.21.15.83:0/1140995874 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a440bd1f0 con 0x5654a3922460 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a3f400640 1 -- 172.21.15.83:0/1140995874 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f6a4403bdc0 con 0x5654a3922460 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a3f400640 1 -- 172.21.15.83:0/1140995874 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6a440bc450 con 0x5654a3922460 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a6341c740 10 client.24997 fetch_fsmap learned FSMap version 23 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a6341c740 1 -- 172.21.15.83:0/1140995874 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5654a399ef80 con 0x5654a3922460 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a3f400640 1 -- 172.21.15.83:0/1140995874 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a440bd1f0 con 0x5654a3922460 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a3f400640 1 -- 172.21.15.83:0/1140995874 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f6a440a5050 con 0x5654a3922460 2024-04-23T14:47:44.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a6341c740 10 client.24997 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a6341c740 10 client.24997 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a6341c740 20 client.24997 populate_metadata read hostname 'smithi083' 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a6341c740 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5654a39a3450 0x5654a39a2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a6341c740 4 client.24997 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.543+0000 7f6a6341c740 1 -- 172.21.15.83:0/1140995874 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "status"}) -- 0x5654a38863a0 con 0x5654a39a3450 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.544+0000 7f6a5dc00640 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5654a39a3450 0x5654a39a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.544+0000 7f6a5dc00640 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5654a39a3450 0x5654a39a2160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9ce55bf4b8893a3 server_cookie=c2d342cdec43d03b in_seq=0 out_seq=0 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.544+0000 7f6a3f400640 10 client.24997 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.544+0000 7f6a3f400640 1 -- 172.21.15.83:0/1140995874 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5654a38863a0 con 0x5654a39a3450 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.544+0000 7f6a3f400640 10 client.24997 handle_command_reply: tid=0 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.544+0000 7f6a6341c740 1 client.24997 shutdown 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.545+0000 7f6a6341c740 1 -- 172.21.15.83:0/1140995874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3922460 msgr2=0x5654a392faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.545+0000 7f6a6341c740 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3922460 0x5654a392faa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6a440f62d0 tx=0x7f6a440b9390 comp rx=0 tx=0).stop 2024-04-23T14:47:44.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.545+0000 7f6a6341c740 1 -- 172.21.15.83:0/1140995874 shutdown_connections 2024-04-23T14:47:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.545+0000 7f6a6341c740 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5654a39275d0 0x5654a39323a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.545+0000 7f6a6341c740 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5654a39a3450 0x5654a39a2160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.545+0000 7f6a6341c740 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5654a3922460 0x5654a392faa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.545+0000 7f6a3f400640 0 client.24997 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:47:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.545+0000 7f6a6341c740 1 --2- 172.21.15.83:0/1140995874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5654a3921a70 0x5654a392f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.545+0000 7f6a6341c740 1 -- 172.21.15.83:0/1140995874 >> 172.21.15.83:0/1140995874 conn(0x5654a39148d0 msgr2=0x5654a3926a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.545+0000 7f6a6341c740 1 -- 172.21.15.83:0/1140995874 shutdown_connections 2024-04-23T14:47:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.545+0000 7f6a6341c740 1 -- 172.21.15.83:0/1140995874 wait complete. 2024-04-23T14:47:44.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.545+0000 7f6a6341c740 20 client.24997 trim_cache size 0 max 16384 2024-04-23T14:47:44.551 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:47:44.551 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5f4fe708-017e-11ef-bc93-c7b262605968", 2024-04-23T14:47:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-04-23T14:47:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24542, 2024-04-23T14:47:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-23T14:47:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-23T14:47:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-23T14:47:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 101.265495636, 2024-04-23T14:47:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 23, 2024-04-23T14:47:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-23T14:47:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T14:47:44.552 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 124.78938255200001 2024-04-23T14:47:44.552 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:47:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.546+0000 7f6a5f000640 1 -- 172.21.15.83:0/1451135242 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6a400a3bb0 msgr2=0x7f6a400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.546+0000 7f6a5f000640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6a400a3bb0 0x7f6a400a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x5654a3922860 tx=0x7f6a4c009500 comp rx=0 tx=0).stop 2024-04-23T14:47:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.546+0000 7f6a5f000640 1 -- 172.21.15.83:0/1451135242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58103060 msgr2=0x7f6a5819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.546+0000 7f6a5f000640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58103060 0x7f6a5819a9b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6a44031440 tx=0x7f6a44032ce0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.546+0000 7f6a5f000640 1 -- 172.21.15.83:0/1451135242 shutdown_connections 2024-04-23T14:47:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.546+0000 7f6a5f000640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f6a400a3bb0 0x7f6a400a6070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.546+0000 7f6a5f000640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a58103a20 0x7f6a581a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.546+0000 7f6a5f000640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a58103060 0x7f6a5819a9b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.546+0000 7f6a5f000640 1 --2- 172.21.15.83:0/1451135242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a58101f10 0x7f6a5819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.546+0000 7f6a5f000640 1 -- 172.21.15.83:0/1451135242 >> 172.21.15.83:0/1451135242 conn(0x7f6a580fd4a0 msgr2=0x7f6a58074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.546+0000 7f6a5f000640 1 -- 172.21.15.83:0/1451135242 shutdown_connections 2024-04-23T14:47:44.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.546+0000 7f6a5f000640 1 -- 172.21.15.83:0/1451135242 wait complete. 2024-04-23T14:47:44.563 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:44 smithi083 ceph-mon[27867]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-23T14:47:44.563 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:44 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2141676400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:47:44.563 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:44 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/633432596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:47:44.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T14:47:44.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:44 smithi203 ceph-mon[32800]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-23T14:47:44.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:44 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2141676400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:47:44.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:44 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/633432596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:47:44.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.770+0000 7ff203e00640 1 -- 172.21.15.83:0/4074858565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff204105cf0 msgr2=0x7ff20410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.770+0000 7ff203e00640 1 --2- 172.21.15.83:0/4074858565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff204105cf0 0x7ff20410c580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009ca0 tx=0x7ff1fc02a8c0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.770+0000 7ff203e00640 1 -- 172.21.15.83:0/4074858565 shutdown_connections 2024-04-23T14:47:44.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.770+0000 7ff203e00640 1 --2- 172.21.15.83:0/4074858565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff204105cf0 0x7ff20410c580 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.770+0000 7ff203e00640 1 --2- 172.21.15.83:0/4074858565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff204105330 0x7ff2041057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.770+0000 7ff203e00640 1 --2- 172.21.15.83:0/4074858565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff204104140 0x7ff204104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.770+0000 7ff203e00640 1 -- 172.21.15.83:0/4074858565 >> 172.21.15.83:0/4074858565 conn(0x7ff2040ff6b0 msgr2=0x7ff204101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.770+0000 7ff203e00640 1 -- 172.21.15.83:0/4074858565 shutdown_connections 2024-04-23T14:47:44.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.771+0000 7ff203e00640 1 -- 172.21.15.83:0/4074858565 wait complete. 2024-04-23T14:47:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.771+0000 7ff203e00640 1 Processor -- start 2024-04-23T14:47:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff203e00640 1 -- start start 2024-04-23T14:47:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff203e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff204104140 0x7ff20419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff203e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff204105330 0x7ff20419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff203e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff204105cf0 0x7ff2041a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff203e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff20410fdd0 con 0x7ff204104140 2024-04-23T14:47:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff203e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff20410fc50 con 0x7ff204105cf0 2024-04-23T14:47:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff203e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff20410ff50 con 0x7ff204105330 2024-04-23T14:47:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff202a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff204104140 0x7ff20419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff202000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff204105330 0x7ff20419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff202a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff204104140 0x7ff20419a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54134/0 (socket says 172.21.15.83:54134) 2024-04-23T14:47:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff202a00640 1 -- 172.21.15.83:0/3996838948 learned_addr learned my addr 172.21.15.83:0/3996838948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff203400640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff204105cf0 0x7ff2041a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff202000640 1 -- 172.21.15.83:0/3996838948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff204105cf0 msgr2=0x7ff2041a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.772+0000 7ff202000640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff204105cf0 0x7ff2041a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.773+0000 7ff202000640 1 -- 172.21.15.83:0/3996838948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff204104140 msgr2=0x7ff20419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.773+0000 7ff202000640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff204104140 0x7ff20419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.773+0000 7ff202000640 1 -- 172.21.15.83:0/3996838948 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2041a2010 con 0x7ff204105330 2024-04-23T14:47:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.773+0000 7ff202000640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff204105330 0x7ff20419a930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000e9f0 tx=0x7ff1f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.773+0000 7ff1fb400640 1 -- 172.21.15.83:0/3996838948 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000ccc0 con 0x7ff204105330 2024-04-23T14:47:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.773+0000 7ff1fb400640 1 -- 172.21.15.83:0/3996838948 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f0010910 con 0x7ff204105330 2024-04-23T14:47:44.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.773+0000 7ff1fb400640 1 -- 172.21.15.83:0/3996838948 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0018960 con 0x7ff204105330 2024-04-23T14:47:44.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.773+0000 7ff203e00640 1 -- 172.21.15.83:0/3996838948 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2041a2300 con 0x7ff204105330 2024-04-23T14:47:44.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.773+0000 7ff203e00640 1 -- 172.21.15.83:0/3996838948 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2041a28a0 con 0x7ff204105330 2024-04-23T14:47:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.776+0000 7ff20c672740 1 Processor -- start 2024-04-23T14:47:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff20c672740 1 -- start start 2024-04-23T14:47:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff20c672740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b1260 0x55dd677b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff1fb400640 1 -- 172.21.15.83:0/3996838948 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff1f0010ac0 con 0x7ff204105330 2024-04-23T14:47:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff20c672740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 0x55dd677b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff20c672740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b6dc0 0x55dd677b9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff20c672740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd67716b10 con 0x55dd677b1c50 2024-04-23T14:47:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff20c672740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dd676bd2b0 con 0x55dd677b1260 2024-04-23T14:47:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff20c672740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55dd67712ad0 con 0x55dd677b6dc0 2024-04-23T14:47:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff202a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 0x55dd677b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff202a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 0x55dd677b6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54136/0 (socket says 172.21.15.83:54136) 2024-04-23T14:47:44.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff202a00640 1 -- 172.21.15.83:0/84806017 learned_addr learned my addr 172.21.15.83:0/84806017 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff202000640 1 --2- 172.21.15.83:0/84806017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b6dc0 0x55dd677b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.777+0000 7ff203400640 1 --2- 172.21.15.83:0/84806017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b1260 0x55dd677b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.778+0000 7ff202a00640 1 -- 172.21.15.83:0/84806017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b6dc0 msgr2=0x55dd677b9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.778+0000 7ff202a00640 1 --2- 172.21.15.83:0/84806017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b6dc0 0x55dd677b9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.778+0000 7ff202a00640 1 -- 172.21.15.83:0/84806017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b1260 msgr2=0x55dd677b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.778+0000 7ff202a00640 1 --2- 172.21.15.83:0/84806017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b1260 0x55dd677b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.778+0000 7ff202a00640 1 -- 172.21.15.83:0/84806017 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd674292d0 con 0x55dd677b1c50 2024-04-23T14:47:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.778+0000 7ff202a00640 1 --2- 172.21.15.83:0/84806017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 0x55dd677b6750 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009d70 tx=0x7ff1f4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df91af60acdecadb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.778+0000 7ff1efe00640 1 -- 172.21.15.83:0/84806017 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4004220 con 0x55dd677b1c50 2024-04-23T14:47:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.778+0000 7ff1efe00640 1 -- 172.21.15.83:0/84806017 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f40043c0 con 0x55dd677b1c50 2024-04-23T14:47:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.778+0000 7ff1efe00640 1 -- 172.21.15.83:0/84806017 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4019950 con 0x55dd677b1c50 2024-04-23T14:47:44.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.779+0000 7ff1fb400640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1e80a3c30 0x7ff1e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.779+0000 7ff1fb400640 1 -- 172.21.15.83:0/3996838948 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff1f00d0740 con 0x7ff204105330 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.779+0000 7ff203400640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1e80a3c30 0x7ff1e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.779+0000 7ff20c672740 1 -- 172.21.15.83:0/84806017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 msgr2=0x55dd677b6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.779+0000 7ff20c672740 1 --2- 172.21.15.83:0/84806017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 0x55dd677b6750 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009d70 tx=0x7ff1f4011810 comp rx=0 tx=0).stop 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.779+0000 7ff20c672740 1 -- 172.21.15.83:0/84806017 shutdown_connections 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.779+0000 7ff20c672740 1 --2- 172.21.15.83:0/84806017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b6dc0 0x55dd677b9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.779+0000 7ff20c672740 1 --2- 172.21.15.83:0/84806017 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 0x55dd677b6750 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.779+0000 7ff20c672740 1 --2- 172.21.15.83:0/84806017 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b1260 0x55dd677b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.779+0000 7ff20c672740 1 -- 172.21.15.83:0/84806017 >> 172.21.15.83:0/84806017 conn(0x55dd677ac8d0 msgr2=0x55dd677accd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.779+0000 7ff20c672740 1 -- 172.21.15.83:0/84806017 shutdown_connections 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.779+0000 7ff20c672740 1 -- 172.21.15.83:0/84806017 wait complete. 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.780+0000 7ff203400640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1e80a3c30 0x7ff1e80a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc005e30 tx=0x7ff1fc03d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.780+0000 7ff20c672740 1 -- start start 2024-04-23T14:47:44.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.780+0000 7ff20c672740 1 Processor -- start 2024-04-23T14:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.780+0000 7ff20c672740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1260 0x55dd677bb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.780+0000 7ff20c672740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 0x55dd677bbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.780+0000 7ff20c672740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b6dc0 0x55dd677c2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.780+0000 7ff20c672740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd677b9d70 con 0x55dd677b1c50 2024-04-23T14:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff20c672740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dd676d0590 con 0x55dd677b6dc0 2024-04-23T14:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff20c672740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55dd677b9ef0 con 0x55dd677b1260 2024-04-23T14:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff20c672740 10 client.0 _ll_register_callbacks cb 0x55dd676b33a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff202000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 0x55dd677bbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff202000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 0x55dd677bbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54150/0 (socket says 172.21.15.83:54150) 2024-04-23T14:47:44.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff202000640 1 -- 172.21.15.83:0/2669986271 learned_addr learned my addr 172.21.15.83:0/2669986271 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff202a00640 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1260 0x55dd677bb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff203400640 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b6dc0 0x55dd677c2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff202000640 1 -- 172.21.15.83:0/2669986271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1260 msgr2=0x55dd677bb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff202000640 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1260 0x55dd677bb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff202000640 1 -- 172.21.15.83:0/2669986271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b6dc0 msgr2=0x55dd677c2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff202000640 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b6dc0 0x55dd677c2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff202000640 1 -- 172.21.15.83:0/2669986271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd674f36b0 con 0x55dd677b1c50 2024-04-23T14:47:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff202000640 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 0x55dd677bbb40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00a5120 tx=0x7ff1f0098df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dc5789c67f639e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.781+0000 7ff1ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:47:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.782+0000 7ff1ecc00640 1 -- 172.21.15.83:0/2669986271 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f009c3f0 con 0x55dd677b1c50 2024-04-23T14:47:44.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.782+0000 7ff1ecc00640 1 -- 172.21.15.83:0/2669986271 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff1f00bbda0 con 0x55dd677b1c50 2024-04-23T14:47:44.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.782+0000 7ff20c672740 1 -- 172.21.15.83:0/2669986271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f40043c0 con 0x55dd677b1c50 2024-04-23T14:47:44.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.782+0000 7ff1ecc00640 1 -- 172.21.15.83:0/2669986271 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff1f009ca90 con 0x55dd677b1c50 2024-04-23T14:47:44.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.782+0000 7ff20c672740 10 client.14925 fetch_fsmap learned FSMap version 23 2024-04-23T14:47:44.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.782+0000 7ff20c672740 1 -- 172.21.15.83:0/2669986271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55dd677c5f60 con 0x55dd677b1c50 2024-04-23T14:47:44.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.782+0000 7ff1ecc00640 1 -- 172.21.15.83:0/2669986271 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f40043c0 con 0x55dd677b1c50 2024-04-23T14:47:44.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.782+0000 7ff1ecc00640 1 -- 172.21.15.83:0/2669986271 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff1f009bee0 con 0x55dd677b1c50 2024-04-23T14:47:44.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.782+0000 7ff20c672740 10 client.14925 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T14:47:44.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.782+0000 7ff20c672740 10 client.14925 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T14:47:44.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.783+0000 7ff20c672740 20 client.14925 populate_metadata read hostname 'smithi083' 2024-04-23T14:47:44.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.783+0000 7ff20c672740 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55dd677c8570 0x55dd677c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.783+0000 7ff20c672740 4 client.14925 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:47:44.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.783+0000 7ff20c672740 1 -- 172.21.15.83:0/2669986271 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd67724530 con 0x55dd677c8570 2024-04-23T14:47:44.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.783+0000 7ff202a00640 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55dd677c8570 0x55dd677c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.783+0000 7ff202a00640 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55dd677c8570 0x55dd677c99e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e85bb00e15b4ae7 server_cookie=4953031dac9850d6 in_seq=0 out_seq=0 2024-04-23T14:47:44.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.783+0000 7ff1ecc00640 10 client.14925 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:47:44.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.784+0000 7ff1ecc00640 1 -- 172.21.15.83:0/2669986271 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd67724530 con 0x55dd677c8570 2024-04-23T14:47:44.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.784+0000 7ff1ecc00640 10 client.14925 handle_command_reply: tid=0 2024-04-23T14:47:44.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.784+0000 7ff20c672740 1 client.14925 shutdown 2024-04-23T14:47:44.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.784+0000 7ff20c672740 1 -- 172.21.15.83:0/2669986271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 msgr2=0x55dd677bbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.784+0000 7ff20c672740 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 0x55dd677bbb40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00a5120 tx=0x7ff1f0098df0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.785+0000 7ff20c672740 1 -- 172.21.15.83:0/2669986271 shutdown_connections 2024-04-23T14:47:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.785+0000 7ff20c672740 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55dd677c8570 0x55dd677c99e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.785+0000 7ff20c672740 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b6dc0 0x55dd677c2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.785+0000 7ff20c672740 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1c50 0x55dd677bbb40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.785+0000 7ff1ecc00640 0 client.14925 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:47:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.785+0000 7ff20c672740 1 --2- 172.21.15.83:0/2669986271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1260 0x55dd677bb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.785+0000 7ff20c672740 1 -- 172.21.15.83:0/2669986271 >> 172.21.15.83:0/2669986271 conn(0x55dd677ac8d0 msgr2=0x55dd677ab970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.785+0000 7ff20c672740 1 -- 172.21.15.83:0/2669986271 shutdown_connections 2024-04-23T14:47:44.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.785+0000 7ff20c672740 1 -- 172.21.15.83:0/2669986271 wait complete. 2024-04-23T14:47:44.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.785+0000 7ff20c672740 20 client.14925 trim_cache size 0 max 16384 2024-04-23T14:47:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.805+0000 7ff20c672740 1 Processor -- start 2024-04-23T14:47:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.805+0000 7ff20c672740 1 -- start start 2024-04-23T14:47:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.805+0000 7ff20c672740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1260 0x55dd677c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.805+0000 7ff20c672740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1c50 0x55dd677c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.805+0000 7ff20c672740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b6dc0 0x55dd677b91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.805+0000 7ff20c672740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd677c7d20 con 0x55dd677b1260 2024-04-23T14:47:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.805+0000 7ff20c672740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dd677c83f0 con 0x55dd677b6dc0 2024-04-23T14:47:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.805+0000 7ff20c672740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55dd677c8ac0 con 0x55dd677b1c50 2024-04-23T14:47:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.805+0000 7ff202a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1260 0x55dd677c35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.805+0000 7ff202a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1260 0x55dd677c35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54152/0 (socket says 172.21.15.83:54152) 2024-04-23T14:47:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.805+0000 7ff202a00640 1 -- 172.21.15.83:0/2766934625 learned_addr learned my addr 172.21.15.83:0/2766934625 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.806+0000 7ff203400640 1 --2- 172.21.15.83:0/2766934625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b6dc0 0x55dd677b91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.806+0000 7ff202000640 1 --2- 172.21.15.83:0/2766934625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1c50 0x55dd677c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.806+0000 7ff202a00640 1 -- 172.21.15.83:0/2766934625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1c50 msgr2=0x55dd677c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.806+0000 7ff202a00640 1 --2- 172.21.15.83:0/2766934625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1c50 0x55dd677c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.806+0000 7ff202a00640 1 -- 172.21.15.83:0/2766934625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b6dc0 msgr2=0x55dd677b91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.806+0000 7ff202a00640 1 --2- 172.21.15.83:0/2766934625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b6dc0 0x55dd677b91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.806+0000 7ff202a00640 1 -- 172.21.15.83:0/2766934625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd677a8230 con 0x55dd677b1260 2024-04-23T14:47:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.806+0000 7ff202000640 1 --2- 172.21.15.83:0/2766934625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1c50 0x55dd677c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:47:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.806+0000 7ff202a00640 1 --2- 172.21.15.83:0/2766934625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1260 0x55dd677c35d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff1f401b040 tx=0x7ff1f40052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6bca7a73c8ab8c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.806+0000 7ff1ef400640 1 -- 172.21.15.83:0/2766934625 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4030070 con 0x55dd677b1260 2024-04-23T14:47:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.806+0000 7ff1ef400640 1 -- 172.21.15.83:0/2766934625 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f4003ad0 con 0x55dd677b1260 2024-04-23T14:47:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.806+0000 7ff1ef400640 1 -- 172.21.15.83:0/2766934625 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4019780 con 0x55dd677b1260 2024-04-23T14:47:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.807+0000 7ff20c672740 1 -- 172.21.15.83:0/2766934625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1260 msgr2=0x55dd677c35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.807+0000 7ff20c672740 1 --2- 172.21.15.83:0/2766934625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1260 0x55dd677c35d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff1f401b040 tx=0x7ff1f40052e0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.807+0000 7ff20c672740 1 -- 172.21.15.83:0/2766934625 shutdown_connections 2024-04-23T14:47:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.807+0000 7ff20c672740 1 --2- 172.21.15.83:0/2766934625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b6dc0 0x55dd677b91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.807+0000 7ff20c672740 1 --2- 172.21.15.83:0/2766934625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1c50 0x55dd677c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.807+0000 7ff20c672740 1 --2- 172.21.15.83:0/2766934625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b1260 0x55dd677c35d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.807+0000 7ff20c672740 1 -- 172.21.15.83:0/2766934625 >> 172.21.15.83:0/2766934625 conn(0x55dd677ac8d0 msgr2=0x55dd677accb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.807+0000 7ff20c672740 1 -- 172.21.15.83:0/2766934625 shutdown_connections 2024-04-23T14:47:44.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.807+0000 7ff20c672740 1 -- 172.21.15.83:0/2766934625 wait complete. 2024-04-23T14:47:44.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.808+0000 7ff20c672740 1 -- start start 2024-04-23T14:47:44.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.808+0000 7ff20c672740 1 Processor -- start 2024-04-23T14:47:44.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.808+0000 7ff20c672740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1260 0x55dd677be4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.808+0000 7ff20c672740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b1c50 0x55dd677be9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.808+0000 7ff20c672740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b6dc0 0x55dd677c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.808+0000 7ff20c672740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55dd677c9860 con 0x55dd677b6dc0 2024-04-23T14:47:44.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.808+0000 7ff20c672740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55dd677c9190 con 0x55dd677b1c50 2024-04-23T14:47:44.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.808+0000 7ff20c672740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55dd677c9f30 con 0x55dd677b1260 2024-04-23T14:47:44.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.808+0000 7ff20c672740 10 client.0 _ll_register_callbacks cb 0x55dd676b33a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:47:44.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.808+0000 7ff203400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b6dc0 0x55dd677c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff203400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b6dc0 0x55dd677c1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:54160/0 (socket says 172.21.15.83:54160) 2024-04-23T14:47:44.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff203400640 1 -- 172.21.15.83:0/3754777615 learned_addr learned my addr 172.21.15.83:0/3754777615 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:44.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff202a00640 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1260 0x55dd677be4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff202000640 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b1c50 0x55dd677be9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff203400640 1 -- 172.21.15.83:0/3754777615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1260 msgr2=0x55dd677be4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff203400640 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1260 0x55dd677be4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff203400640 1 -- 172.21.15.83:0/3754777615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b1c50 msgr2=0x55dd677be9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff203400640 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b1c50 0x55dd677be9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff203400640 1 -- 172.21.15.83:0/3754777615 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd677c1ff0 con 0x55dd677b6dc0 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff203400640 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b6dc0 0x55dd677c1a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc035000 tx=0x7ff1fc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f932656dd27c6d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff1ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff1ecc00640 1 -- 172.21.15.83:0/3754777615 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc030c40 con 0x55dd677b6dc0 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.809+0000 7ff1ecc00640 1 -- 172.21.15.83:0/3754777615 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff1fc030de0 con 0x55dd677b6dc0 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.810+0000 7ff20c672740 1 -- 172.21.15.83:0/3754777615 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f4003ad0 con 0x55dd677b6dc0 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.810+0000 7ff1ecc00640 1 -- 172.21.15.83:0/3754777615 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff1fc062050 con 0x55dd677b6dc0 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.810+0000 7ff20c672740 10 client.14931 fetch_fsmap learned FSMap version 23 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.810+0000 7ff20c672740 1 -- 172.21.15.83:0/3754777615 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55dd6782e7a0 con 0x55dd677b6dc0 2024-04-23T14:47:44.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.810+0000 7ff1ecc00640 1 -- 172.21.15.83:0/3754777615 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f4003ad0 con 0x55dd677b6dc0 2024-04-23T14:47:44.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.810+0000 7ff1ecc00640 1 -- 172.21.15.83:0/3754777615 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff1fc053410 con 0x55dd677b6dc0 2024-04-23T14:47:44.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.810+0000 7ff20c672740 10 client.14931 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T14:47:44.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.810+0000 7ff20c672740 10 client.14931 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T14:47:44.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.810+0000 7ff20c672740 20 client.14931 populate_metadata read hostname 'smithi083' 2024-04-23T14:47:44.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.810+0000 7ff20c672740 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55dd67833490 0x55dd678321a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:44.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.810+0000 7ff20c672740 4 client.14931 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:47:44.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.810+0000 7ff20c672740 1 -- 172.21.15.83:0/3754777615 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "status"}) -- 0x55dd6771e3a0 con 0x55dd67833490 2024-04-23T14:47:44.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.811+0000 7ff202a00640 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55dd67833490 0x55dd678321a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:44.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.811+0000 7ff202a00640 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55dd67833490 0x55dd678321a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7057441da1168481 server_cookie=b17b2f08d6aff6d8 in_seq=0 out_seq=0 2024-04-23T14:47:44.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.811+0000 7ff1ecc00640 10 client.14931 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:47:44.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.811+0000 7ff1ecc00640 1 -- 172.21.15.83:0/3754777615 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dd6771e3a0 con 0x55dd67833490 2024-04-23T14:47:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.811+0000 7ff1ecc00640 10 client.14931 handle_command_reply: tid=0 2024-04-23T14:47:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.811+0000 7ff20c672740 1 client.14931 shutdown 2024-04-23T14:47:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.812+0000 7ff20c672740 1 -- 172.21.15.83:0/3754777615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b6dc0 msgr2=0x55dd677c1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.812+0000 7ff20c672740 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b6dc0 0x55dd677c1a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc035000 tx=0x7ff1fc02a8c0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.812+0000 7ff20c672740 1 -- 172.21.15.83:0/3754777615 shutdown_connections 2024-04-23T14:47:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.812+0000 7ff20c672740 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55dd67833490 0x55dd678321a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.812+0000 7ff20c672740 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55dd677b6dc0 0x55dd677c1a00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.812+0000 7ff20c672740 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55dd677b1c50 0x55dd677be9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.812+0000 7ff20c672740 1 --2- 172.21.15.83:0/3754777615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55dd677b1260 0x55dd677be4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.812+0000 7ff1ecc00640 0 client.14931 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:47:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.812+0000 7ff20c672740 1 -- 172.21.15.83:0/3754777615 >> 172.21.15.83:0/3754777615 conn(0x55dd677ac8d0 msgr2=0x55dd677b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.812+0000 7ff20c672740 1 -- 172.21.15.83:0/3754777615 shutdown_connections 2024-04-23T14:47:44.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.812+0000 7ff20c672740 1 -- 172.21.15.83:0/3754777615 wait complete. 2024-04-23T14:47:44.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.812+0000 7ff20c672740 20 client.14931 trim_cache size 0 max 16384 2024-04-23T14:47:44.824 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:47:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5f4fe708-017e-11ef-bc93-c7b262605968", 2024-04-23T14:47:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-23T14:47:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24599, 2024-04-23T14:47:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-23T14:47:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-23T14:47:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-23T14:47:44.824 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 104.59513868099999, 2024-04-23T14:47:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 23, 2024-04-23T14:47:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-23T14:47:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T14:47:44.825 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 111.109426984 2024-04-23T14:47:44.825 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:47:44.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.813+0000 7ff203e00640 1 -- 172.21.15.83:0/3996838948 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1e80a3c30 msgr2=0x7ff1e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.813+0000 7ff203e00640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1e80a3c30 0x7ff1e80a60f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc005e30 tx=0x7ff1fc03d040 comp rx=0 tx=0).stop 2024-04-23T14:47:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.813+0000 7ff203e00640 1 -- 172.21.15.83:0/3996838948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff204105330 msgr2=0x7ff20419a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.813+0000 7ff203e00640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff204105330 0x7ff20419a930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000e9f0 tx=0x7ff1f000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.813+0000 7ff203e00640 1 -- 172.21.15.83:0/3996838948 shutdown_connections 2024-04-23T14:47:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.813+0000 7ff203e00640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff1e80a3c30 0x7ff1e80a60f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.813+0000 7ff203e00640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff204105cf0 0x7ff2041a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.813+0000 7ff203e00640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff204105330 0x7ff20419a930 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.813+0000 7ff203e00640 1 --2- 172.21.15.83:0/3996838948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff204104140 0x7ff20419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.813+0000 7ff203e00640 1 -- 172.21.15.83:0/3996838948 >> 172.21.15.83:0/3996838948 conn(0x7ff2040ff6b0 msgr2=0x7ff204108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.813+0000 7ff203e00640 1 -- 172.21.15.83:0/3996838948 shutdown_connections 2024-04-23T14:47:44.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:44.813+0000 7ff203e00640 1 -- 172.21.15.83:0/3996838948 wait complete. 2024-04-23T14:47:44.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:44 smithi173 ceph-mon[32443]: pgmap v371: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-23T14:47:44.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:44 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2141676400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:47:44.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:44 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/633432596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:47:44.831 INFO:tasks.cephadm:Setup complete, yielding 2024-04-23T14:47:44.832 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T14:47:44.842 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-23T14:47:44.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-23T14:47:45.284 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:47:46.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:46 smithi203 ceph-mon[32800]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:46 smithi083 ceph-mon[27867]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:46.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:46 smithi173 ceph-mon[32443]: pgmap v372: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.729+0000 7f1cac600700 1 -- 172.21.15.83:0/3504142739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca4105f00 msgr2=0x7f1ca41082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.729+0000 7f1cac600700 1 --2- 172.21.15.83:0/3504142739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca4105f00 0x7f1ca41082f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1c900098f0 tx=0x7f1c90009c00 comp rx=0 tx=0).stop 2024-04-23T14:47:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.729+0000 7f1cac600700 1 -- 172.21.15.83:0/3504142739 shutdown_connections 2024-04-23T14:47:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.729+0000 7f1cac600700 1 --2- 172.21.15.83:0/3504142739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ca4108830 0x7f1ca410aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.729+0000 7f1cac600700 1 --2- 172.21.15.83:0/3504142739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca41035d0 0x7f1ca41059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.729+0000 7f1cac600700 1 --2- 172.21.15.83:0/3504142739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca4105f00 0x7f1ca41082f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:47.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.729+0000 7f1cac600700 1 -- 172.21.15.83:0/3504142739 >> 172.21.15.83:0/3504142739 conn(0x7f1ca40fcfe0 msgr2=0x7f1ca40ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:47.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.729+0000 7f1cac600700 1 -- 172.21.15.83:0/3504142739 shutdown_connections 2024-04-23T14:47:47.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.729+0000 7f1cac600700 1 -- 172.21.15.83:0/3504142739 wait complete. 2024-04-23T14:47:47.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.730+0000 7f1cac600700 1 Processor -- start 2024-04-23T14:47:47.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.730+0000 7f1cac600700 1 -- start start 2024-04-23T14:47:47.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.731+0000 7f1cac600700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca41035d0 0x7f1ca4102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:47.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.731+0000 7f1cac600700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ca4105f00 0x7f1ca4101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:47.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.731+0000 7f1cac600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca4108830 0x7f1ca4101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:47.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.731+0000 7f1cac600700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca410e770 con 0x7f1ca4108830 2024-04-23T14:47:47.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.731+0000 7f1cac600700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca410e5f0 con 0x7f1ca4105f00 2024-04-23T14:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.731+0000 7f1cac600700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca410e8f0 con 0x7f1ca41035d0 2024-04-23T14:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.731+0000 7f1ca9200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ca4105f00 0x7f1ca4101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.731+0000 7f1caa600700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca4108830 0x7f1ca4101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.731+0000 7f1ca9200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ca4105f00 0x7f1ca4101580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:43878/0 (socket says 172.21.15.83:43878) 2024-04-23T14:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.731+0000 7f1ca9200700 1 -- 172.21.15.83:0/3883231512 learned_addr learned my addr 172.21.15.83:0/3883231512 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.731+0000 7f1ca9c00700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca41035d0 0x7f1ca4102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.731+0000 7f1ca9200700 1 -- 172.21.15.83:0/3883231512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca41035d0 msgr2=0x7f1ca4102f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.732+0000 7f1ca9200700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca41035d0 0x7f1ca4102f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.732+0000 7f1ca9200700 1 -- 172.21.15.83:0/3883231512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca4108830 msgr2=0x7f1ca4101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.732+0000 7f1ca9200700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca4108830 0x7f1ca4101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.732+0000 7f1ca9200700 1 -- 172.21.15.83:0/3883231512 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca4102370 con 0x7f1ca4105f00 2024-04-23T14:47:47.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.732+0000 7f1caa600700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca4108830 0x7f1ca4101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:47.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.732+0000 7f1ca9200700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ca4105f00 0x7f1ca4101580 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009810 tx=0x7f1c90005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:47.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.732+0000 7f1ca2000700 1 -- 172.21.15.83:0/3883231512 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90005440 con 0x7f1ca4105f00 2024-04-23T14:47:47.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.732+0000 7f1ca2000700 1 -- 172.21.15.83:0/3883231512 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c900055e0 con 0x7f1ca4105f00 2024-04-23T14:47:47.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.732+0000 7f1cac600700 1 -- 172.21.15.83:0/3883231512 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca41a2f70 con 0x7f1ca4105f00 2024-04-23T14:47:47.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.732+0000 7f1ca2000700 1 -- 172.21.15.83:0/3883231512 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90003c00 con 0x7f1ca4105f00 2024-04-23T14:47:47.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.732+0000 7f1cac600700 1 -- 172.21.15.83:0/3883231512 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca41a3360 con 0x7f1ca4105f00 2024-04-23T14:47:47.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.733+0000 7f1cac600700 1 -- 172.21.15.83:0/3883231512 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca40697e0 con 0x7f1ca4105f00 2024-04-23T14:47:47.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.736+0000 7f1ca2000700 1 -- 172.21.15.83:0/3883231512 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1c9000b560 con 0x7f1ca4105f00 2024-04-23T14:47:47.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.737+0000 7f1ca2000700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1c8c0a3cc0 0x7f1c8c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:47.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.737+0000 7f1ca2000700 1 -- 172.21.15.83:0/3883231512 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1c900d1470 con 0x7f1ca4105f00 2024-04-23T14:47:47.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.737+0000 7f1ca9c00700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1c8c0a3cc0 0x7f1c8c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:47.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.738+0000 7f1ca9c00700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1c8c0a3cc0 0x7f1c8c0a6180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009680 tx=0x7f1c98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:47.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.739+0000 7f1ca2000700 1 -- 172.21.15.83:0/3883231512 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c90013030 con 0x7f1ca4105f00 2024-04-23T14:47:47.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.988+0000 7f1cac600700 1 -- 172.21.15.83:0/3883231512 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1ca4062f30 con 0x7f1c8c0a3cc0 2024-04-23T14:47:47.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.990+0000 7f1ca2000700 1 -- 172.21.15.83:0/3883231512 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1ca4062f30 con 0x7f1c8c0a3cc0 2024-04-23T14:47:47.992 INFO:teuthology.orchestra.run.smithi083.stdout:Backend: cephadm 2024-04-23T14:47:47.992 INFO:teuthology.orchestra.run.smithi083.stdout:Available: Yes 2024-04-23T14:47:47.992 INFO:teuthology.orchestra.run.smithi083.stdout:Paused: No 2024-04-23T14:47:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.993+0000 7f1cac600700 1 -- 172.21.15.83:0/3883231512 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1c8c0a3cc0 msgr2=0x7f1c8c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.993+0000 7f1cac600700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1c8c0a3cc0 0x7f1c8c0a6180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009680 tx=0x7f1c98009260 comp rx=0 tx=0).stop 2024-04-23T14:47:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.993+0000 7f1cac600700 1 -- 172.21.15.83:0/3883231512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ca4105f00 msgr2=0x7f1ca4101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.993+0000 7f1cac600700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ca4105f00 0x7f1ca4101580 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009810 tx=0x7f1c90005250 comp rx=0 tx=0).stop 2024-04-23T14:47:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.994+0000 7f1cac600700 1 -- 172.21.15.83:0/3883231512 shutdown_connections 2024-04-23T14:47:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.994+0000 7f1cac600700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1c8c0a3cc0 0x7f1c8c0a6180 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.994+0000 7f1cac600700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ca4108830 0x7f1ca4101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.994+0000 7f1cac600700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca41035d0 0x7f1ca4102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.994+0000 7f1cac600700 1 --2- 172.21.15.83:0/3883231512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ca4105f00 0x7f1ca4101580 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.994+0000 7f1cac600700 1 -- 172.21.15.83:0/3883231512 >> 172.21.15.83:0/3883231512 conn(0x7f1ca40fcfe0 msgr2=0x7f1ca4103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.994+0000 7f1cac600700 1 -- 172.21.15.83:0/3883231512 shutdown_connections 2024-04-23T14:47:47.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:47.994+0000 7f1cac600700 1 -- 172.21.15.83:0/3883231512 wait complete. 2024-04-23T14:47:48.628 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:48 smithi173 ceph-mon[32443]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:47:48.706 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:48 smithi083 ceph-mon[27867]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:47:48.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:48 smithi203 ceph-mon[32800]: pgmap v373: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:47:48.794 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-23T14:47:49.245 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:47:49.636 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:49 smithi083 ceph-mon[27867]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:47:49.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:49 smithi203 ceph-mon[32800]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:47:49.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:49 smithi173 ceph-mon[32443]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:47:50.715 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:50 smithi083 ceph-mon[27867]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:50.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:50 smithi203 ceph-mon[32800]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:50.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:50 smithi173 ceph-mon[32443]: pgmap v374: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.524+0000 7f50d1400700 1 -- 172.21.15.83:0/30045373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50cc102b60 msgr2=0x7f50cc102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.524+0000 7f50d1400700 1 --2- 172.21.15.83:0/30045373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50cc102b60 0x7f50cc102f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0099c0 tx=0x7f50bc009cd0 comp rx=0 tx=0).stop 2024-04-23T14:47:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.524+0000 7f50d1400700 1 -- 172.21.15.83:0/30045373 shutdown_connections 2024-04-23T14:47:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.524+0000 7f50d1400700 1 --2- 172.21.15.83:0/30045373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50cc104700 0x7f50cc10afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:51.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.524+0000 7f50d1400700 1 --2- 172.21.15.83:0/30045373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50cc102b60 0x7f50cc102f80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.524+0000 7f50d1400700 1 --2- 172.21.15.83:0/30045373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50cc103d60 0x7f50cc1041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.524+0000 7f50d1400700 1 -- 172.21.15.83:0/30045373 >> 172.21.15.83:0/30045373 conn(0x7f50cc0fe0c0 msgr2=0x7f50cc100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.524+0000 7f50d1400700 1 -- 172.21.15.83:0/30045373 shutdown_connections 2024-04-23T14:47:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.525+0000 7f50d1400700 1 -- 172.21.15.83:0/30045373 wait complete. 2024-04-23T14:47:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.526+0000 7f50d1400700 1 Processor -- start 2024-04-23T14:47:51.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.526+0000 7f50d1400700 1 -- start start 2024-04-23T14:47:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.526+0000 7f50d1400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50cc102b60 0x7f50cc076890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.526+0000 7f50d1400700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50cc103d60 0x7f50cc074ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.526+0000 7f50d1400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50cc104700 0x7f50cc075420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.526+0000 7f50d1400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc10e740 con 0x7f50cc102b60 2024-04-23T14:47:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.526+0000 7f50d1400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc10e5c0 con 0x7f50cc104700 2024-04-23T14:47:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.526+0000 7f50d1400700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc10e8c0 con 0x7f50cc103d60 2024-04-23T14:47:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.526+0000 7f50ca800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50cc102b60 0x7f50cc076890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.527+0000 7f50ca800700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50cc102b60 0x7f50cc076890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58016/0 (socket says 172.21.15.83:58016) 2024-04-23T14:47:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.527+0000 7f50ca800700 1 -- 172.21.15.83:0/360048057 learned_addr learned my addr 172.21.15.83:0/360048057 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.527+0000 7f50cb200700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50cc104700 0x7f50cc075420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.527+0000 7f50c3e00700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50cc103d60 0x7f50cc074ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:51.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.527+0000 7f50ca800700 1 -- 172.21.15.83:0/360048057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50cc103d60 msgr2=0x7f50cc074ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.527+0000 7f50ca800700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50cc103d60 0x7f50cc074ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.527+0000 7f50ca800700 1 -- 172.21.15.83:0/360048057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50cc104700 msgr2=0x7f50cc075420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.527+0000 7f50ca800700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50cc104700 0x7f50cc075420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.527+0000 7f50ca800700 1 -- 172.21.15.83:0/360048057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc075cd0 con 0x7f50cc102b60 2024-04-23T14:47:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.527+0000 7f50ca800700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50cc102b60 0x7f50cc076890 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00bf60 tx=0x7f50bc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.528+0000 7f50c3400700 1 -- 172.21.15.83:0/360048057 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc009e60 con 0x7f50cc102b60 2024-04-23T14:47:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.528+0000 7f50c3400700 1 -- 172.21.15.83:0/360048057 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50bc003e30 con 0x7f50cc102b60 2024-04-23T14:47:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.528+0000 7f50c3400700 1 -- 172.21.15.83:0/360048057 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50bc00fdd0 con 0x7f50cc102b60 2024-04-23T14:47:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.528+0000 7f50d1400700 1 -- 172.21.15.83:0/360048057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50cc1a2e80 con 0x7f50cc102b60 2024-04-23T14:47:51.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.528+0000 7f50d1400700 1 -- 172.21.15.83:0/360048057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50cc1a3220 con 0x7f50cc102b60 2024-04-23T14:47:51.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.529+0000 7f50d1400700 1 -- 172.21.15.83:0/360048057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50cc0697e0 con 0x7f50cc102b60 2024-04-23T14:47:51.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.533+0000 7f50c3400700 1 -- 172.21.15.83:0/360048057 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f50bc00f4f0 con 0x7f50cc102b60 2024-04-23T14:47:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.534+0000 7f50c3400700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f50b00a3cc0 0x7f50b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.534+0000 7f50c3400700 1 -- 172.21.15.83:0/360048057 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f50bc0d11c0 con 0x7f50cc102b60 2024-04-23T14:47:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.534+0000 7f50c3e00700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f50b00a3cc0 0x7f50b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:51.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.535+0000 7f50c3e00700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f50b00a3cc0 0x7f50b00a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009c60 tx=0x7f50b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:51.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.536+0000 7f50c3400700 1 -- 172.21.15.83:0/360048057 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50bc013070 con 0x7f50cc102b60 2024-04-23T14:47:51.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.760+0000 7f50d1400700 1 -- 172.21.15.83:0/360048057 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f50cc062f30 con 0x7f50b00a3cc0 2024-04-23T14:47:51.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.802+0000 7f50c3400700 1 -- 172.21.15.83:0/360048057 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f50cc062f30 con 0x7f50b00a3cc0 2024-04-23T14:47:51.804 INFO:teuthology.orchestra.run.smithi083.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T14:47:51.804 INFO:teuthology.orchestra.run.smithi083.stdout:mds.a smithi083 running (2m) 24s ago 2m 12.4M - 19.0.0-3126-gd44fddd9 76e77f280c74 9d9eb3f563f2 2024-04-23T14:47:51.804 INFO:teuthology.orchestra.run.smithi083.stdout:mds.b smithi173 running (118s) 49s ago 117s 24.9M - 19.0.0-3126-gd44fddd9 76e77f280c74 e01408b8ab43 2024-04-23T14:47:51.804 INFO:teuthology.orchestra.run.smithi083.stdout:mds.c smithi203 running (2m) 49s ago 2m 19.4M - 19.0.0-3126-gd44fddd9 76e77f280c74 b61521142c82 2024-04-23T14:47:51.804 INFO:teuthology.orchestra.run.smithi083.stdout:mds.d smithi083 running (2m) 24s ago 2m 12.5M - 19.0.0-3126-gd44fddd9 76e77f280c74 024fcdd4320a 2024-04-23T14:47:51.804 INFO:teuthology.orchestra.run.smithi083.stdout:mds.e smithi173 running (2m) 49s ago 2m 15.8M - 19.0.0-3126-gd44fddd9 76e77f280c74 59862320b8af 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mds.f smithi203 running (2m) 49s ago 2m 19.4M - 19.0.0-3126-gd44fddd9 76e77f280c74 8108ce5b9607 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mds.g smithi083 running (2m) 24s ago 2m 12.4M - 19.0.0-3126-gd44fddd9 76e77f280c74 8c789337abcf 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mds.h smithi173 running (2m) 49s ago 2m 18.1M - 19.0.0-3126-gd44fddd9 76e77f280c74 df4755f5aa57 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mds.i smithi203 running (2m) 49s ago 2m 16.6M - 19.0.0-3126-gd44fddd9 76e77f280c74 e1125a3808be 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mds.j smithi083 running (2m) 24s ago 2m 12.5M - 19.0.0-3126-gd44fddd9 76e77f280c74 6f4fa56f68af 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mds.k smithi173 running (2m) 49s ago 2m 13.1M - 19.0.0-3126-gd44fddd9 76e77f280c74 6db9430fe8ed 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mds.l smithi203 running (2m) 49s ago 2m 16.6M - 19.0.0-3126-gd44fddd9 76e77f280c74 976d5bb48ff3 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.x smithi083 *:9283,8765 running (10m) 24s ago 10m 516M - 19.0.0-3126-gd44fddd9 76e77f280c74 00a122261f10 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.y smithi173 *:8443,8765 running (8m) 49s ago 8m 452M - 19.0.0-3126-gd44fddd9 76e77f280c74 0c1434fbabb1 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mgr.z smithi203 *:8443,8765 running (8m) 49s ago 8m 451M - 19.0.0-3126-gd44fddd9 76e77f280c74 96b8fc56554b 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mon.a smithi083 running (10m) 24s ago 11m 49.3M 2048M 19.0.0-3126-gd44fddd9 76e77f280c74 4c2a91033a2a 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mon.b smithi173 running (8m) 49s ago 8m 42.4M 2048M 19.0.0-3126-gd44fddd9 76e77f280c74 8d63442d7a98 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:mon.c smithi203 running (9m) 49s ago 8m 41.2M 2048M 19.0.0-3126-gd44fddd9 76e77f280c74 06fa1ec79286 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 smithi083 running (8m) 24s ago 8m 55.0M 4272M 19.0.0-3126-gd44fddd9 76e77f280c74 f638f6fc65b6 2024-04-23T14:47:51.805 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 smithi173 running (7m) 49s ago 7m 57.9M 4272M 19.0.0-3126-gd44fddd9 76e77f280c74 36ee470295f3 2024-04-23T14:47:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 smithi203 running (7m) 49s ago 7m 56.8M 4272M 19.0.0-3126-gd44fddd9 76e77f280c74 2aa7ac21b3cb 2024-04-23T14:47:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 smithi083 running (6m) 24s ago 6m 57.1M 4272M 19.0.0-3126-gd44fddd9 76e77f280c74 eeda600cd284 2024-04-23T14:47:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 smithi173 running (6m) 49s ago 6m 42.2M 4272M 19.0.0-3126-gd44fddd9 76e77f280c74 580aec3dc358 2024-04-23T14:47:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 smithi203 running (5m) 49s ago 5m 45.7M 4272M 19.0.0-3126-gd44fddd9 76e77f280c74 22e8caf1f34e 2024-04-23T14:47:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 smithi083 running (5m) 24s ago 5m 42.7M 4272M 19.0.0-3126-gd44fddd9 76e77f280c74 56a52a582e43 2024-04-23T14:47:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 smithi173 running (4m) 49s ago 4m 54.7M 4272M 19.0.0-3126-gd44fddd9 76e77f280c74 bc0668cb881c 2024-04-23T14:47:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.8 smithi203 running (4m) 49s ago 4m 56.2M 4272M 19.0.0-3126-gd44fddd9 76e77f280c74 df94c9991848 2024-04-23T14:47:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.9 smithi083 running (3m) 24s ago 3m 51.5M 4272M 19.0.0-3126-gd44fddd9 76e77f280c74 ac994596b8b6 2024-04-23T14:47:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.10 smithi173 running (3m) 49s ago 3m 53.1M 4272M 19.0.0-3126-gd44fddd9 76e77f280c74 3d36e1d4b669 2024-04-23T14:47:51.806 INFO:teuthology.orchestra.run.smithi083.stdout:osd.11 smithi203 running (2m) 49s ago 2m 52.6M 4272M 19.0.0-3126-gd44fddd9 76e77f280c74 54e31043b974 2024-04-23T14:47:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.805+0000 7f50d1400700 1 -- 172.21.15.83:0/360048057 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f50b00a3cc0 msgr2=0x7f50b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.805+0000 7f50d1400700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f50b00a3cc0 0x7f50b00a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009c60 tx=0x7f50b4009260 comp rx=0 tx=0).stop 2024-04-23T14:47:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.805+0000 7f50d1400700 1 -- 172.21.15.83:0/360048057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50cc102b60 msgr2=0x7f50cc076890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.805+0000 7f50d1400700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50cc102b60 0x7f50cc076890 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00bf60 tx=0x7f50bc00bf90 comp rx=0 tx=0).stop 2024-04-23T14:47:51.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.805+0000 7f50d1400700 1 -- 172.21.15.83:0/360048057 shutdown_connections 2024-04-23T14:47:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.805+0000 7f50d1400700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f50b00a3cc0 0x7f50b00a6180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.805+0000 7f50d1400700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50cc104700 0x7f50cc075420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.805+0000 7f50d1400700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50cc102b60 0x7f50cc076890 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.805+0000 7f50d1400700 1 --2- 172.21.15.83:0/360048057 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50cc103d60 0x7f50cc074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.805+0000 7f50d1400700 1 -- 172.21.15.83:0/360048057 >> 172.21.15.83:0/360048057 conn(0x7f50cc0fe0c0 msgr2=0x7f50cc1070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.806+0000 7f50d1400700 1 -- 172.21.15.83:0/360048057 shutdown_connections 2024-04-23T14:47:51.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:51.806+0000 7f50d1400700 1 -- 172.21.15.83:0/360048057 wait complete. 2024-04-23T14:47:52.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:52 smithi083 ceph-mon[27867]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:47:52.666 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-23T14:47:52.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:52 smithi203 ceph-mon[32800]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:47:52.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:52 smithi173 ceph-mon[32443]: pgmap v375: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:47:53.119 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:47:53.518 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:53 smithi083 ceph-mon[27867]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:47:53.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:53 smithi203 ceph-mon[32800]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:47:53.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:53 smithi173 ceph-mon[32443]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:47:54.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:54 smithi083 ceph-mon[27867]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:47:54.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:54 smithi203 ceph-mon[32800]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:47:54.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:54 smithi173 ceph-mon[32443]: pgmap v376: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:47:55.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.431+0000 7f4ed0a00700 1 -- 172.21.15.83:0/3125040219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ecc103510 msgr2=0x7f4ecc105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.431+0000 7f4ed0a00700 1 --2- 172.21.15.83:0/3125040219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ecc103510 0x7f4ecc105900 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009990 tx=0x7f4ebc009ca0 comp rx=0 tx=0).stop 2024-04-23T14:47:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.431+0000 7f4ed0a00700 1 -- 172.21.15.83:0/3125040219 shutdown_connections 2024-04-23T14:47:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.431+0000 7f4ed0a00700 1 --2- 172.21.15.83:0/3125040219 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ecc108770 0x7f4ecc10abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.431+0000 7f4ed0a00700 1 --2- 172.21.15.83:0/3125040219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ecc103510 0x7f4ecc105900 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.431+0000 7f4ed0a00700 1 --2- 172.21.15.83:0/3125040219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ecc105e40 0x7f4ecc108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.431+0000 7f4ed0a00700 1 -- 172.21.15.83:0/3125040219 >> 172.21.15.83:0/3125040219 conn(0x7f4ecc0fcf00 msgr2=0x7f4ecc0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.431+0000 7f4ed0a00700 1 -- 172.21.15.83:0/3125040219 shutdown_connections 2024-04-23T14:47:55.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.432+0000 7f4ed0a00700 1 -- 172.21.15.83:0/3125040219 wait complete. 2024-04-23T14:47:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.433+0000 7f4ed0a00700 1 Processor -- start 2024-04-23T14:47:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.433+0000 7f4ed0a00700 1 -- start start 2024-04-23T14:47:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.433+0000 7f4ed0a00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ecc103510 0x7f4ecc198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.433+0000 7f4ed0a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ecc105e40 0x7f4ecc199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4ed0a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ecc108770 0x7f4ecc1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4ed0a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc10e680 con 0x7f4ecc105e40 2024-04-23T14:47:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4ed0a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc10e500 con 0x7f4ecc108770 2024-04-23T14:47:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4ed0a00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc10e800 con 0x7f4ecc103510 2024-04-23T14:47:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4ec9e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ecc105e40 0x7f4ecc199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:55.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4ec9e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ecc105e40 0x7f4ecc199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:53858/0 (socket says 172.21.15.83:53858) 2024-04-23T14:47:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4ec9e00700 1 -- 172.21.15.83:0/2041994799 learned_addr learned my addr 172.21.15.83:0/2041994799 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4eca800700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ecc103510 0x7f4ecc198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4ecb200700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ecc108770 0x7f4ecc1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4eca800700 1 -- 172.21.15.83:0/2041994799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ecc108770 msgr2=0x7f4ecc1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4eca800700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ecc108770 0x7f4ecc1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4eca800700 1 -- 172.21.15.83:0/2041994799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ecc105e40 msgr2=0x7f4ecc199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4eca800700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ecc105e40 0x7f4ecc199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4eca800700 1 -- 172.21.15.83:0/2041994799 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc1a0870 con 0x7f4ecc103510 2024-04-23T14:47:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4ec9e00700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ecc105e40 0x7f4ecc199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.434+0000 7f4ecb200700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ecc108770 0x7f4ecc1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:55.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.435+0000 7f4eca800700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ecc103510 0x7f4ecc198c20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc000c00 tx=0x7f4ebc00bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.435+0000 7f4ec3400700 1 -- 172.21.15.83:0/2041994799 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc026070 con 0x7f4ecc103510 2024-04-23T14:47:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.435+0000 7f4ec3400700 1 -- 172.21.15.83:0/2041994799 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ebc00f460 con 0x7f4ecc103510 2024-04-23T14:47:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.435+0000 7f4ed0a00700 1 -- 172.21.15.83:0/2041994799 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc1a0b00 con 0x7f4ecc103510 2024-04-23T14:47:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.435+0000 7f4ec3400700 1 -- 172.21.15.83:0/2041994799 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc021670 con 0x7f4ecc103510 2024-04-23T14:47:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.435+0000 7f4ed0a00700 1 -- 172.21.15.83:0/2041994799 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc1a0f50 con 0x7f4ecc103510 2024-04-23T14:47:55.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.436+0000 7f4ed0a00700 1 -- 172.21.15.83:0/2041994799 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ecc0697e0 con 0x7f4ecc103510 2024-04-23T14:47:55.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.438+0000 7f4ec3400700 1 -- 172.21.15.83:0/2041994799 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4ebc029070 con 0x7f4ecc103510 2024-04-23T14:47:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.440+0000 7f4ec3400700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4eb00a3c40 0x7f4eb00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.440+0000 7f4ec3400700 1 -- 172.21.15.83:0/2041994799 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4ebc0d1150 con 0x7f4ecc103510 2024-04-23T14:47:55.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.440+0000 7f4ec9e00700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4eb00a3c40 0x7f4eb00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:55.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.441+0000 7f4ec9e00700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4eb00a3c40 0x7f4eb00a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8011550 tx=0x7f4eb8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:55.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.449+0000 7f4ec3400700 1 -- 172.21.15.83:0/2041994799 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ebc05dff0 con 0x7f4ecc103510 2024-04-23T14:47:55.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.683+0000 7f4ed0a00700 1 -- 172.21.15.83:0/2041994799 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f4ecc062f30 con 0x7f4eb00a3c40 2024-04-23T14:47:55.685 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:55 smithi083 ceph-mon[27867]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:55.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.700+0000 7f4ec3400700 1 -- 172.21.15.83:0/2041994799 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f4ecc062f30 con 0x7f4eb00a3c40 2024-04-23T14:47:55.701 INFO:teuthology.orchestra.run.smithi083.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-23T14:47:55.701 INFO:teuthology.orchestra.run.smithi083.stdout:mds.all 12/12 53s ago 2m smithi083=a;smithi083=d;smithi083=g;smithi083=j;smithi173=b;smithi173=e;smithi173=h;smithi173=k;smithi203=c;smithi203=f;smithi203=i;smithi203=l;count:12 2024-04-23T14:47:55.702 INFO:teuthology.orchestra.run.smithi083.stdout:mgr 3/3 53s ago 8m smithi083=x;smithi173=y;smithi203=z;count:3 2024-04-23T14:47:55.702 INFO:teuthology.orchestra.run.smithi083.stdout:mon 3/3 53s ago 9m smithi083:172.21.15.83=a;smithi173:172.21.15.173=b;smithi203:172.21.15.203=c;count:3 2024-04-23T14:47:55.702 INFO:teuthology.orchestra.run.smithi083.stdout:osd 12 53s ago - 2024-04-23T14:47:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.703+0000 7f4ed0a00700 1 -- 172.21.15.83:0/2041994799 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4eb00a3c40 msgr2=0x7f4eb00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.703+0000 7f4ed0a00700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4eb00a3c40 0x7f4eb00a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8011550 tx=0x7f4eb8009260 comp rx=0 tx=0).stop 2024-04-23T14:47:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.703+0000 7f4ed0a00700 1 -- 172.21.15.83:0/2041994799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ecc103510 msgr2=0x7f4ecc198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.703+0000 7f4ed0a00700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ecc103510 0x7f4ecc198c20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc000c00 tx=0x7f4ebc00bbb0 comp rx=0 tx=0).stop 2024-04-23T14:47:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.703+0000 7f4ed0a00700 1 -- 172.21.15.83:0/2041994799 shutdown_connections 2024-04-23T14:47:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.703+0000 7f4ed0a00700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4eb00a3c40 0x7f4eb00a6100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.703+0000 7f4ed0a00700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ecc108770 0x7f4ecc1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.703+0000 7f4ed0a00700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ecc103510 0x7f4ecc198c20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.703+0000 7f4ed0a00700 1 --2- 172.21.15.83:0/2041994799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ecc105e40 0x7f4ecc199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.703+0000 7f4ed0a00700 1 -- 172.21.15.83:0/2041994799 >> 172.21.15.83:0/2041994799 conn(0x7f4ecc0fcf00 msgr2=0x7f4ecc103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.703+0000 7f4ed0a00700 1 -- 172.21.15.83:0/2041994799 shutdown_connections 2024-04-23T14:47:55.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:55.703+0000 7f4ed0a00700 1 -- 172.21.15.83:0/2041994799 wait complete. 2024-04-23T14:47:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:55 smithi203 ceph-mon[32800]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:55.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:55 smithi173 ceph-mon[32443]: pgmap v377: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:47:56.558 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-23T14:47:56.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:56 smithi203 ceph-mon[32800]: from='client.25033 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:47:56.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:56 smithi083 ceph-mon[27867]: from='client.25033 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:47:56.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:56 smithi173 ceph-mon[32443]: from='client.25033 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:47:57.007 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:47:57.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:47:57 smithi203 ceph-mon[32800]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:47:57.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:47:57 smithi083 ceph-mon[27867]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:47:57.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:47:57 smithi173 ceph-mon[32443]: pgmap v378: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:47:59.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.272+0000 7f506ba00700 1 -- 172.21.15.83:0/3185059582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5064109190 msgr2=0x7f50641095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:59.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.272+0000 7f506ba00700 1 --2- 172.21.15.83:0/3185059582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5064109190 0x7f50641095b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f50580099c0 tx=0x7f5058009cd0 comp rx=0 tx=0).stop 2024-04-23T14:47:59.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.273+0000 7f506ba00700 1 -- 172.21.15.83:0/3185059582 shutdown_connections 2024-04-23T14:47:59.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.273+0000 7f506ba00700 1 --2- 172.21.15.83:0/3185059582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506410ad30 0x7f50641115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:59.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.273+0000 7f506ba00700 1 --2- 172.21.15.83:0/3185059582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5064109190 0x7f50641095b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:59.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.273+0000 7f506ba00700 1 --2- 172.21.15.83:0/3185059582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f506410a390 0x7f506410a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:59.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.273+0000 7f506ba00700 1 -- 172.21.15.83:0/3185059582 >> 172.21.15.83:0/3185059582 conn(0x7f5064078160 msgr2=0x7f506407a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:59.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.273+0000 7f506ba00700 1 -- 172.21.15.83:0/3185059582 shutdown_connections 2024-04-23T14:47:59.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.273+0000 7f506ba00700 1 -- 172.21.15.83:0/3185059582 wait complete. 2024-04-23T14:47:59.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.274+0000 7f506ba00700 1 Processor -- start 2024-04-23T14:47:59.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.274+0000 7f506ba00700 1 -- start start 2024-04-23T14:47:59.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.275+0000 7f506ba00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5064109190 0x7f506419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:59.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.275+0000 7f506ba00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f506410a390 0x7f506419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:59.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.275+0000 7f506ba00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506410ad30 0x7f50641a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:59.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.275+0000 7f506ba00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5064114d70 con 0x7f506410a390 2024-04-23T14:47:59.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.275+0000 7f506ba00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5064114bf0 con 0x7f506410ad30 2024-04-23T14:47:59.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.275+0000 7f506ba00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5064114ef0 con 0x7f5064109190 2024-04-23T14:47:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.275+0000 7f5063e00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f506410a390 0x7f506419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.275+0000 7f5069000700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5064109190 0x7f506419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.275+0000 7f5069000700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5064109190 0x7f506419f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42766/0 (socket says 172.21.15.83:42766) 2024-04-23T14:47:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.275+0000 7f5069000700 1 -- 172.21.15.83:0/2434937052 learned_addr learned my addr 172.21.15.83:0/2434937052 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:47:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.275+0000 7f5069a00700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506410ad30 0x7f50641a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f5069000700 1 -- 172.21.15.83:0/2434937052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506410ad30 msgr2=0x7f50641a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f5069000700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506410ad30 0x7f50641a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f5069000700 1 -- 172.21.15.83:0/2434937052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f506410a390 msgr2=0x7f506419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f5069000700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f506410a390 0x7f506419f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f5069000700 1 -- 172.21.15.83:0/2434937052 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50641a6fb0 con 0x7f5064109190 2024-04-23T14:47:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f5063e00700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f506410a390 0x7f506419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:59.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f5069a00700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506410ad30 0x7f50641a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:47:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f5069000700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5064109190 0x7f506419f350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f505800bf90 tx=0x7f505800bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f5061600700 1 -- 172.21.15.83:0/2434937052 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058009e60 con 0x7f5064109190 2024-04-23T14:47:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f506ba00700 1 -- 172.21.15.83:0/2434937052 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50641a7240 con 0x7f5064109190 2024-04-23T14:47:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f5061600700 1 -- 172.21.15.83:0/2434937052 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5058003e30 con 0x7f5064109190 2024-04-23T14:47:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f5061600700 1 -- 172.21.15.83:0/2434937052 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505800fd70 con 0x7f5064109190 2024-04-23T14:47:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.276+0000 7f506ba00700 1 -- 172.21.15.83:0/2434937052 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50641a7770 con 0x7f5064109190 2024-04-23T14:47:59.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.277+0000 7f506ba00700 1 -- 172.21.15.83:0/2434937052 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50640697e0 con 0x7f5064109190 2024-04-23T14:47:59.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.280+0000 7f5061600700 1 -- 172.21.15.83:0/2434937052 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f505800f460 con 0x7f5064109190 2024-04-23T14:47:59.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.281+0000 7f5061600700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f504c0a3cc0 0x7f504c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:47:59.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.281+0000 7f5061600700 1 -- 172.21.15.83:0/2434937052 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f50580d1250 con 0x7f5064109190 2024-04-23T14:47:59.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.281+0000 7f5063e00700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f504c0a3cc0 0x7f504c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:47:59.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.282+0000 7f5063e00700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f504c0a3cc0 0x7f504c0a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5050011490 tx=0x7f5050009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:47:59.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.283+0000 7f5061600700 1 -- 172.21.15.83:0/2434937052 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5058013070 con 0x7f5064109190 2024-04-23T14:47:59.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.521+0000 7f506ba00700 1 -- 172.21.15.83:0/2434937052 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f5064062f30 con 0x7f504c0a3cc0 2024-04-23T14:47:59.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.525+0000 7f5061600700 1 -- 172.21.15.83:0/2434937052 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f5064062f30 con 0x7f504c0a3cc0 2024-04-23T14:47:59.526 INFO:teuthology.orchestra.run.smithi083.stdout:HOST ADDR LABELS STATUS 2024-04-23T14:47:59.526 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083 172.21.15.83 2024-04-23T14:47:59.526 INFO:teuthology.orchestra.run.smithi083.stdout:smithi173 172.21.15.173 2024-04-23T14:47:59.526 INFO:teuthology.orchestra.run.smithi083.stdout:smithi203 172.21.15.203 2024-04-23T14:47:59.526 INFO:teuthology.orchestra.run.smithi083.stdout:3 hosts in cluster 2024-04-23T14:47:59.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.528+0000 7f506ba00700 1 -- 172.21.15.83:0/2434937052 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f504c0a3cc0 msgr2=0x7f504c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:59.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.528+0000 7f506ba00700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f504c0a3cc0 0x7f504c0a6180 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5050011490 tx=0x7f5050009260 comp rx=0 tx=0).stop 2024-04-23T14:47:59.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.528+0000 7f506ba00700 1 -- 172.21.15.83:0/2434937052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5064109190 msgr2=0x7f506419f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:47:59.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.528+0000 7f506ba00700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5064109190 0x7f506419f350 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f505800bf90 tx=0x7f505800bfc0 comp rx=0 tx=0).stop 2024-04-23T14:47:59.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.528+0000 7f506ba00700 1 -- 172.21.15.83:0/2434937052 shutdown_connections 2024-04-23T14:47:59.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.528+0000 7f506ba00700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f504c0a3cc0 0x7f504c0a6180 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:59.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.528+0000 7f506ba00700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f506410ad30 0x7f50641a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:59.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.528+0000 7f506ba00700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5064109190 0x7f506419f350 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:59.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.528+0000 7f506ba00700 1 --2- 172.21.15.83:0/2434937052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f506410a390 0x7f506419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:47:59.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.528+0000 7f506ba00700 1 -- 172.21.15.83:0/2434937052 >> 172.21.15.83:0/2434937052 conn(0x7f5064078160 msgr2=0x7f506410d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:47:59.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.528+0000 7f506ba00700 1 -- 172.21.15.83:0/2434937052 shutdown_connections 2024-04-23T14:47:59.529 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:47:59.528+0000 7f506ba00700 1 -- 172.21.15.83:0/2434937052 wait complete. 2024-04-23T14:48:00.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-23T14:48:00.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:00 smithi173 ceph-mon[32443]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:00.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:00 smithi203 ceph-mon[32800]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:00.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:00 smithi083 ceph-mon[27867]: pgmap v379: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:00.857 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:48:01.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:01 smithi173 ceph-mon[32443]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:01.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:01 smithi203 ceph-mon[32800]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:01.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:01 smithi083 ceph-mon[27867]: from='client.25039 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:02.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:02 smithi083 ceph-mon[27867]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:48:02.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:02 smithi173 ceph-mon[32443]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:48:02.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:02 smithi203 ceph-mon[32800]: pgmap v380: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:48:03.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.064+0000 7f19f0400700 1 -- 172.21.15.83:0/1801702580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e810a390 msgr2=0x7f19e810a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:03.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.064+0000 7f19f0400700 1 --2- 172.21.15.83:0/1801702580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e810a390 0x7f19e810a7f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f19d40099c0 tx=0x7f19d4009cd0 comp rx=0 tx=0).stop 2024-04-23T14:48:03.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.064+0000 7f19f0400700 1 -- 172.21.15.83:0/1801702580 shutdown_connections 2024-04-23T14:48:03.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.064+0000 7f19f0400700 1 --2- 172.21.15.83:0/1801702580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e810ad30 0x7f19e81115e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:03.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.064+0000 7f19f0400700 1 --2- 172.21.15.83:0/1801702580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19e8109190 0x7f19e81095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:03.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.064+0000 7f19f0400700 1 --2- 172.21.15.83:0/1801702580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e810a390 0x7f19e810a7f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:03.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.064+0000 7f19f0400700 1 -- 172.21.15.83:0/1801702580 >> 172.21.15.83:0/1801702580 conn(0x7f19e8078160 msgr2=0x7f19e807a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.064+0000 7f19f0400700 1 -- 172.21.15.83:0/1801702580 shutdown_connections 2024-04-23T14:48:03.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.064+0000 7f19f0400700 1 -- 172.21.15.83:0/1801702580 wait complete. 2024-04-23T14:48:03.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.065+0000 7f19f0400700 1 Processor -- start 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.065+0000 7f19f0400700 1 -- start start 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19f0400700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e8109190 0x7f19e819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19f0400700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19e810a390 0x7f19e819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19f0400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e810ad30 0x7f19e81a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19f0400700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8114d80 con 0x7f19e810ad30 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19f0400700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8114c00 con 0x7f19e810a390 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19f0400700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8114f00 con 0x7f19e8109190 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19eda00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e8109190 0x7f19e819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19ee400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e810ad30 0x7f19e81a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19eda00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e8109190 0x7f19e819f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42782/0 (socket says 172.21.15.83:42782) 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19eda00700 1 -- 172.21.15.83:0/3988533050 learned_addr learned my addr 172.21.15.83:0/3988533050 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19ed000700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19e810a390 0x7f19e819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19eda00700 1 -- 172.21.15.83:0/3988533050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19e810a390 msgr2=0x7f19e819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:03.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19eda00700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19e810a390 0x7f19e819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:03.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19eda00700 1 -- 172.21.15.83:0/3988533050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e810ad30 msgr2=0x7f19e81a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:03.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.066+0000 7f19eda00700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e810ad30 0x7f19e81a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:03.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.067+0000 7f19eda00700 1 -- 172.21.15.83:0/3988533050 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e81a6f90 con 0x7f19e8109190 2024-04-23T14:48:03.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.067+0000 7f19eda00700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e8109190 0x7f19e819f330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d9b0 tx=0x7f19dc00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:03.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.067+0000 7f19e6000700 1 -- 172.21.15.83:0/3988533050 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc016070 con 0x7f19e8109190 2024-04-23T14:48:03.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.067+0000 7f19f0400700 1 -- 172.21.15.83:0/3988533050 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e81a7280 con 0x7f19e8109190 2024-04-23T14:48:03.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.067+0000 7f19e6000700 1 -- 172.21.15.83:0/3988533050 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19dc00be30 con 0x7f19e8109190 2024-04-23T14:48:03.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.067+0000 7f19e6000700 1 -- 172.21.15.83:0/3988533050 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00f3c0 con 0x7f19e8109190 2024-04-23T14:48:03.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.067+0000 7f19f0400700 1 -- 172.21.15.83:0/3988533050 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e81a77c0 con 0x7f19e8109190 2024-04-23T14:48:03.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.068+0000 7f19f0400700 1 -- 172.21.15.83:0/3988533050 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e80697e0 con 0x7f19e8109190 2024-04-23T14:48:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.070+0000 7f19e6000700 1 -- 172.21.15.83:0/3988533050 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f19dc019070 con 0x7f19e8109190 2024-04-23T14:48:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.072+0000 7f19e6000700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f19d00a3c40 0x7f19d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.072+0000 7f19e6000700 1 -- 172.21.15.83:0/3988533050 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f19dc0cff60 con 0x7f19e8109190 2024-04-23T14:48:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.072+0000 7f19ed000700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f19d00a3c40 0x7f19d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:03.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.073+0000 7f19ed000700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f19d00a3c40 0x7f19d00a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f19d4000c00 tx=0x7f19d4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:03.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.076+0000 7f19e6000700 1 -- 172.21.15.83:0/3988533050 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f19dc020030 con 0x7f19e8109190 2024-04-23T14:48:03.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.310+0000 7f19f0400700 1 -- 172.21.15.83:0/3988533050 --> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f19e8062f30 con 0x7f19d00a3c40 2024-04-23T14:48:03.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.315+0000 7f19e6000700 1 -- 172.21.15.83:0/3988533050 <== mgr.14152 v2:172.21.15.83:6800/1964818199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f19e8062f30 con 0x7f19d00a3c40 2024-04-23T14:48:03.316 INFO:teuthology.orchestra.run.smithi083.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-23T14:48:03.316 INFO:teuthology.orchestra.run.smithi083.stdout:smithi083 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040148400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-23T14:48:03.316 INFO:teuthology.orchestra.run.smithi083.stdout:smithi173 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204019S400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-23T14:48:03.316 INFO:teuthology.orchestra.run.smithi083.stdout:smithi203 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-23T14:48:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.318+0000 7f19f0400700 1 -- 172.21.15.83:0/3988533050 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f19d00a3c40 msgr2=0x7f19d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.318+0000 7f19f0400700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f19d00a3c40 0x7f19d00a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f19d4000c00 tx=0x7f19d4005fb0 comp rx=0 tx=0).stop 2024-04-23T14:48:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.318+0000 7f19f0400700 1 -- 172.21.15.83:0/3988533050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e8109190 msgr2=0x7f19e819f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.318+0000 7f19f0400700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e8109190 0x7f19e819f330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d9b0 tx=0x7f19dc00dcc0 comp rx=0 tx=0).stop 2024-04-23T14:48:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.318+0000 7f19f0400700 1 -- 172.21.15.83:0/3988533050 shutdown_connections 2024-04-23T14:48:03.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.318+0000 7f19f0400700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f19d00a3c40 0x7f19d00a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:03.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.318+0000 7f19f0400700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19e810ad30 0x7f19e81a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:03.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.318+0000 7f19f0400700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e8109190 0x7f19e819f330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:03.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.318+0000 7f19f0400700 1 --2- 172.21.15.83:0/3988533050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19e810a390 0x7f19e819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:03.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.318+0000 7f19f0400700 1 -- 172.21.15.83:0/3988533050 >> 172.21.15.83:0/3988533050 conn(0x7f19e8078160 msgr2=0x7f19e810d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:03.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.318+0000 7f19f0400700 1 -- 172.21.15.83:0/3988533050 shutdown_connections 2024-04-23T14:48:03.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:03.318+0000 7f19f0400700 1 -- 172.21.15.83:0/3988533050 wait complete. 2024-04-23T14:48:04.119 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T14:48:04.129 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-23T14:48:04.129 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-23T14:48:04.578 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:48:04.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:04 smithi203 ceph-mon[32800]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:48:04.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:04 smithi203 ceph-mon[32800]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:04.728 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:04 smithi083 ceph-mon[27867]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:48:04.728 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:04 smithi083 ceph-mon[27867]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:04.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:04 smithi173 ceph-mon[32443]: pgmap v381: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:48:04.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:04 smithi173 ceph-mon[32443]: from='client.25045 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:06.613 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:06 smithi083 ceph-mon[27867]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:06.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:06 smithi203 ceph-mon[32800]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:06.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:06 smithi173 ceph-mon[32443]: pgmap v382: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:06.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.942+0000 7f4bfea00700 1 -- 172.21.15.83:0/4024017070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bf81013d0 msgr2=0x7f4bf81017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:06.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.942+0000 7f4bfea00700 1 --2- 172.21.15.83:0/4024017070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bf81013d0 0x7f4bf81017f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009990 tx=0x7f4be8009ca0 comp rx=0 tx=0).stop 2024-04-23T14:48:06.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.942+0000 7f4bfea00700 1 -- 172.21.15.83:0/4024017070 shutdown_connections 2024-04-23T14:48:06.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.942+0000 7f4bfea00700 1 --2- 172.21.15.83:0/4024017070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bf81026f0 0x7f4bf8102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:06.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.942+0000 7f4bfea00700 1 --2- 172.21.15.83:0/4024017070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bf81013d0 0x7f4bf81017f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:06.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.942+0000 7f4bfea00700 1 --2- 172.21.15.83:0/4024017070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf8101d30 0x7f4bf81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:06.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.942+0000 7f4bfea00700 1 -- 172.21.15.83:0/4024017070 >> 172.21.15.83:0/4024017070 conn(0x7f4bf80fcff0 msgr2=0x7f4bf80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:06.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.942+0000 7f4bfea00700 1 -- 172.21.15.83:0/4024017070 shutdown_connections 2024-04-23T14:48:06.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.942+0000 7f4bfea00700 1 -- 172.21.15.83:0/4024017070 wait complete. 2024-04-23T14:48:06.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.943+0000 7f4bfea00700 1 Processor -- start 2024-04-23T14:48:06.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.943+0000 7f4bfea00700 1 -- start start 2024-04-23T14:48:06.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bfea00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf81013d0 0x7f4bf819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:06.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bfea00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bf8101d30 0x7f4bf819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:06.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bfea00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bf81026f0 0x7f4bf81a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:06.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bfea00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf8110940 con 0x7f4bf8101d30 2024-04-23T14:48:06.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bfea00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf81107c0 con 0x7f4bf81026f0 2024-04-23T14:48:06.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bfea00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf8110ac0 con 0x7f4bf81013d0 2024-04-23T14:48:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4befe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bf8101d30 0x7f4bf819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bf7e00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf81013d0 0x7f4bf819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4befe00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bf8101d30 0x7f4bf819b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37402/0 (socket says 172.21.15.83:37402) 2024-04-23T14:48:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4befe00700 1 -- 172.21.15.83:0/182557280 learned_addr learned my addr 172.21.15.83:0/182557280 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bfca00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bf81026f0 0x7f4bf81a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bf7e00700 1 -- 172.21.15.83:0/182557280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bf81026f0 msgr2=0x7f4bf81a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bf7e00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bf81026f0 0x7f4bf81a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bf7e00700 1 -- 172.21.15.83:0/182557280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bf8101d30 msgr2=0x7f4bf819b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bf7e00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bf8101d30 0x7f4bf819b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4bf7e00700 1 -- 172.21.15.83:0/182557280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf81a2c50 con 0x7f4bf81013d0 2024-04-23T14:48:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.944+0000 7f4befe00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bf8101d30 0x7f4bf819b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:06.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.945+0000 7f4bf7e00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf81013d0 0x7f4bf819aff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4be8000c00 tx=0x7f4be8005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:06.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.945+0000 7f4bf5600700 1 -- 172.21.15.83:0/182557280 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be8027070 con 0x7f4bf81013d0 2024-04-23T14:48:06.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.945+0000 7f4bf5600700 1 -- 172.21.15.83:0/182557280 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be8003d70 con 0x7f4bf81013d0 2024-04-23T14:48:06.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.945+0000 7f4bfea00700 1 -- 172.21.15.83:0/182557280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf81a2ee0 con 0x7f4bf81013d0 2024-04-23T14:48:06.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.945+0000 7f4bf5600700 1 -- 172.21.15.83:0/182557280 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be800f460 con 0x7f4bf81013d0 2024-04-23T14:48:06.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.945+0000 7f4bfea00700 1 -- 172.21.15.83:0/182557280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf81a3280 con 0x7f4bf81013d0 2024-04-23T14:48:06.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.946+0000 7f4bfea00700 1 -- 172.21.15.83:0/182557280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf802dfe0 con 0x7f4bf81013d0 2024-04-23T14:48:06.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.948+0000 7f4bf5600700 1 -- 172.21.15.83:0/182557280 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4be802a070 con 0x7f4bf81013d0 2024-04-23T14:48:06.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.950+0000 7f4bf5600700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4bd80a3c40 0x7f4bd80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:06.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.950+0000 7f4bf5600700 1 -- 172.21.15.83:0/182557280 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4be80d1df0 con 0x7f4bf81013d0 2024-04-23T14:48:06.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.950+0000 7f4befe00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4bd80a3c40 0x7f4bd80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:06.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.951+0000 7f4befe00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4bd80a3c40 0x7f4bd80a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4be0011550 tx=0x7f4be0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:06.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:06.952+0000 7f4bf5600700 1 -- 172.21.15.83:0/182557280 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4be8013070 con 0x7f4bf81013d0 2024-04-23T14:48:07.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.224+0000 7f4bfea00700 1 -- 172.21.15.83:0/182557280 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f4bf802e1b0 con 0x7f4bf81013d0 2024-04-23T14:48:07.227 INFO:teuthology.orchestra.run.smithi083.stdout:e23 2024-04-23T14:48:07.227 INFO:teuthology.orchestra.run.smithi083.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T14:48:07.227 INFO:teuthology.orchestra.run.smithi083.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T14:48:07.227 INFO:teuthology.orchestra.run.smithi083.stdout:legacy client fscid: 1 2024-04-23T14:48:07.227 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:07.227 INFO:teuthology.orchestra.run.smithi083.stdout:Filesystem 'cephfs' (1) 2024-04-23T14:48:07.227 INFO:teuthology.orchestra.run.smithi083.stdout:fs_name cephfs 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 23 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-04-23T14:45:59.723532+0000 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-04-23T14:46:05.301806+0000 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:tableserver 0 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:root 0 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:session_timeout 300 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:session_autoclose 300 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:max_file_size 1099511627776 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:max_xattr_size 65536 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:required_client_features {} 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:last_failure 0 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:last_failure_osd_epoch 0 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:max_mds 3 2024-04-23T14:48:07.228 INFO:teuthology.orchestra.run.smithi083.stdout:in 0,1,2 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:up {0=24599,1=24542,2=24532} 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:failed 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:damaged 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:stopped 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:data_pools [3] 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:metadata_pool 2 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:inline_data disabled 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:balancer 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:bal_rank_mask -1 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:standby_count_wanted 0 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:qdb_cluster leader: 24599 members: 24532,24542,24599 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b{0:24599} state up:active seq 3 addr [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c{0:24511} state up:standby-replay seq 1 addr [v2:172.21.15.203:6836/258138306,v1:172.21.15.203:6837/258138306] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.h{1:24542} state up:active seq 7 addr [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T14:48:07.229 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.e{1:24509} state up:standby-replay seq 1 addr [v2:172.21.15.173:6834/2567101723,v1:172.21.15.173:6835/2567101723] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T14:48:07.230 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.i{2:24532} state up:active seq 7 addr [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T14:48:07.230 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.f{2:24499} state up:standby-replay seq 1 addr [v2:172.21.15.203:6834/2384637189,v1:172.21.15.203:6835/2384637189] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T14:48:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:07.230 INFO:teuthology.orchestra.run.smithi083.stdout:Standby daemons: 2024-04-23T14:48:07.230 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:07.230 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.j{-1:14388} state up:standby seq 1 addr [v2:172.21.15.83:6834/928555402,v1:172.21.15.83:6835/928555402] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T14:48:07.230 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.g{-1:14457} state up:standby seq 1 addr [v2:172.21.15.83:6838/436672914,v1:172.21.15.83:6839/436672914] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T14:48:07.230 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a{-1:14475} state up:standby seq 1 addr [v2:172.21.15.83:6840/238596160,v1:172.21.15.83:6841/238596160] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T14:48:07.230 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.l{-1:24436} state up:standby seq 1 addr [v2:172.21.15.203:6832/317357729,v1:172.21.15.203:6833/317357729] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T14:48:07.230 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.k{-1:24461} state up:standby seq 1 addr [v2:172.21.15.173:6832/39034794,v1:172.21.15.173:6833/39034794] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T14:48:07.230 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d{-1:24497} state up:standby seq 1 addr [v2:172.21.15.83:6836/3773827425,v1:172.21.15.83:6837/3773827425] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T14:48:07.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.225+0000 7f4bf5600700 1 -- 172.21.15.83:0/182557280 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3044 (secure 0 0 0) 0x7f4be809e820 con 0x7f4bf81013d0 2024-04-23T14:48:07.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.227+0000 7f4bfea00700 1 -- 172.21.15.83:0/182557280 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4bd80a3c40 msgr2=0x7f4bd80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:07.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.227+0000 7f4bfea00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4bd80a3c40 0x7f4bd80a6100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4be0011550 tx=0x7f4be0009260 comp rx=0 tx=0).stop 2024-04-23T14:48:07.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.227+0000 7f4bfea00700 1 -- 172.21.15.83:0/182557280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf81013d0 msgr2=0x7f4bf819aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:07.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.227+0000 7f4bfea00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf81013d0 0x7f4bf819aff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4be8000c00 tx=0x7f4be8005f10 comp rx=0 tx=0).stop 2024-04-23T14:48:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.228+0000 7f4bfea00700 1 -- 172.21.15.83:0/182557280 shutdown_connections 2024-04-23T14:48:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.228+0000 7f4bfea00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f4bd80a3c40 0x7f4bd80a6100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.228+0000 7f4bfea00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bf81026f0 0x7f4bf81a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.228+0000 7f4bfea00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bf81013d0 0x7f4bf819aff0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.228+0000 7f4bfea00700 1 --2- 172.21.15.83:0/182557280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bf8101d30 0x7f4bf819b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.228+0000 7f4bfea00700 1 -- 172.21.15.83:0/182557280 >> 172.21.15.83:0/182557280 conn(0x7f4bf80fcff0 msgr2=0x7f4bf80feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.228+0000 7f4bfea00700 1 -- 172.21.15.83:0/182557280 shutdown_connections 2024-04-23T14:48:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:07.228+0000 7f4bfea00700 1 -- 172.21.15.83:0/182557280 wait complete. 2024-04-23T14:48:07.232 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-04-23T14:48:07.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:07 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/182557280' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T14:48:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:07 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/182557280' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T14:48:07.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:07 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/182557280' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T14:48:08.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-23T14:48:08.564 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:48:08.708 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:08 smithi083 ceph-mon[27867]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:48:08.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:08 smithi203 ceph-mon[32800]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:48:08.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:08 smithi173 ceph-mon[32443]: pgmap v383: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:48:10.543 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:10 smithi083 ceph-mon[27867]: pgmap v384: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:10 smithi203 ceph-mon[32800]: pgmap v384: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:10.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.821+0000 7f0136200700 1 -- 172.21.15.83:0/2634233991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0130075550 msgr2=0x7f01300759d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:10.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.821+0000 7f0136200700 1 --2- 172.21.15.83:0/2634233991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0130075550 0x7f01300759d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0118009990 tx=0x7f0118009ca0 comp rx=0 tx=0).stop 2024-04-23T14:48:10.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.822+0000 7f0136200700 1 -- 172.21.15.83:0/2634233991 shutdown_connections 2024-04-23T14:48:10.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.822+0000 7f0136200700 1 --2- 172.21.15.83:0/2634233991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f013010aa30 0x7f013010cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:10.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.822+0000 7f0136200700 1 --2- 172.21.15.83:0/2634233991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0130076b20 0x7f0130074f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:10.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.822+0000 7f0136200700 1 --2- 172.21.15.83:0/2634233991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0130075550 0x7f01300759d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:10.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.822+0000 7f0136200700 1 -- 172.21.15.83:0/2634233991 >> 172.21.15.83:0/2634233991 conn(0x7f01300fe0c0 msgr2=0x7f0130100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:10.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.822+0000 7f0136200700 1 -- 172.21.15.83:0/2634233991 shutdown_connections 2024-04-23T14:48:10.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.822+0000 7f0136200700 1 -- 172.21.15.83:0/2634233991 wait complete. 2024-04-23T14:48:10.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.823+0000 7f0136200700 1 Processor -- start 2024-04-23T14:48:10.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.823+0000 7f0136200700 1 -- start start 2024-04-23T14:48:10.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.823+0000 7f0136200700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0130075550 0x7f0130105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:10.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.823+0000 7f0136200700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0130076b20 0x7f01301063d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:10.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.823+0000 7f0136200700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f013010aa30 0x7f0130102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:10.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.823+0000 7f0136200700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f01301109b0 con 0x7f0130075550 2024-04-23T14:48:10.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.824+0000 7f0136200700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0130110830 con 0x7f0130076b20 2024-04-23T14:48:10.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.824+0000 7f0136200700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0130110b30 con 0x7f013010aa30 2024-04-23T14:48:10.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.824+0000 7f012f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0130075550 0x7f0130105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:10.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.824+0000 7f012f400700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0130075550 0x7f0130105e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37416/0 (socket says 172.21.15.83:37416) 2024-04-23T14:48:10.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.824+0000 7f012f400700 1 -- 172.21.15.83:0/4285201860 learned_addr learned my addr 172.21.15.83:0/4285201860 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:10.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.824+0000 7f0126a00700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0130076b20 0x7f01301063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:10.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.824+0000 7f012fe00700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f013010aa30 0x7f0130102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:10.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.824+0000 7f012f400700 1 -- 172.21.15.83:0/4285201860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f013010aa30 msgr2=0x7f0130102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:10.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.824+0000 7f012f400700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f013010aa30 0x7f0130102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:10.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.824+0000 7f012f400700 1 -- 172.21.15.83:0/4285201860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0130076b20 msgr2=0x7f01301063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:10.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.824+0000 7f012f400700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0130076b20 0x7f01301063d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:10.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.824+0000 7f012f400700 1 -- 172.21.15.83:0/4285201860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01301032e0 con 0x7f0130075550 2024-04-23T14:48:10.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.825+0000 7f012f400700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0130075550 0x7f0130105e90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f012000e9f0 tx=0x7f012000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:10.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.825+0000 7f012cc00700 1 -- 172.21.15.83:0/4285201860 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120004a30 con 0x7f0130075550 2024-04-23T14:48:10.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.825+0000 7f012cc00700 1 -- 172.21.15.83:0/4285201860 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f012000cca0 con 0x7f0130075550 2024-04-23T14:48:10.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.825+0000 7f012cc00700 1 -- 172.21.15.83:0/4285201860 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01200056c0 con 0x7f0130075550 2024-04-23T14:48:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.825+0000 7f0136200700 1 -- 172.21.15.83:0/4285201860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01301a7440 con 0x7f0130075550 2024-04-23T14:48:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.825+0000 7f0136200700 1 -- 172.21.15.83:0/4285201860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01301a7840 con 0x7f0130075550 2024-04-23T14:48:10.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:10 smithi173 ceph-mon[32443]: pgmap v384: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:10.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.826+0000 7f0136200700 1 -- 172.21.15.83:0/4285201860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01300697e0 con 0x7f0130075550 2024-04-23T14:48:10.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.828+0000 7f012cc00700 1 -- 172.21.15.83:0/4285201860 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f012000c7c0 con 0x7f0130075550 2024-04-23T14:48:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.829+0000 7f012cc00700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f01140a3c40 0x7f01140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.830+0000 7f012cc00700 1 -- 172.21.15.83:0/4285201860 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f01200d0620 con 0x7f0130075550 2024-04-23T14:48:10.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.830+0000 7f0126a00700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f01140a3c40 0x7f01140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:10.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.830+0000 7f0126a00700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f01140a3c40 0x7f01140a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f011800b7b0 tx=0x7f0118005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:10.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:10.835+0000 7f012cc00700 1 -- 172.21.15.83:0/4285201860 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f012009cfc0 con 0x7f0130075550 2024-04-23T14:48:11.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.064+0000 7f0136200700 1 -- 172.21.15.83:0/4285201860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f0130063730 con 0x7f0130075550 2024-04-23T14:48:11.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.066+0000 7f012cc00700 1 -- 172.21.15.83:0/4285201860 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5309 (secure 0 0 0) 0x7f012009c8c0 con 0x7f0130075550 2024-04-23T14:48:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:epoch 88 2024-04-23T14:48:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:fsid 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T14:48:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:created 2024-04-23T14:36:49.928506+0000 2024-04-23T14:48:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:modified 2024-04-23T14:46:58.351541+0000 2024-04-23T14:48:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-23T14:48:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:crush_version 26 2024-04-23T14:48:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:full_ratio 0.95 2024-04-23T14:48:11.067 INFO:teuthology.orchestra.run.smithi083.stdout:backfillfull_ratio 0.9 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:nearfull_ratio 0.85 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:require_min_compat_client luminous 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:min_compat_client luminous 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:require_osd_release squid 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:stretch_mode_enabled false 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:max_osd 12 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6802/1064791722,v1:172.21.15.83:6803/1064791722] [v2:172.21.15.83:6804/1064791722,v1:172.21.15.83:6805/1064791722] exists,up 8f0dcc33-b313-4e7b-8cbf-d541c92624bb 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6800/548359483,v1:172.21.15.173:6801/548359483] [v2:172.21.15.173:6802/548359483,v1:172.21.15.173:6803/548359483] exists,up 50d1440d-148d-4828-8607-63ebd9076ccb 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6800/4282658379,v1:172.21.15.203:6801/4282658379] [v2:172.21.15.203:6802/4282658379,v1:172.21.15.203:6803/4282658379] exists,up f7fd9255-ac0b-47bb-813f-d29f1ffa131b 2024-04-23T14:48:11.068 INFO:teuthology.orchestra.run.smithi083.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6810/3253964161,v1:172.21.15.83:6811/3253964161] [v2:172.21.15.83:6812/3253964161,v1:172.21.15.83:6813/3253964161] exists,up 8caf9f27-bac9-4ff3-b700-bcca3c250e69 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6808/2156601908,v1:172.21.15.173:6809/2156601908] [v2:172.21.15.173:6810/2156601908,v1:172.21.15.173:6811/2156601908] exists,up 07401fed-df44-4869-92ff-72c233457a2d 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6808/581398636,v1:172.21.15.203:6809/581398636] [v2:172.21.15.203:6810/581398636,v1:172.21.15.203:6811/581398636] exists,up 04c3ad4f-1bd5-45fa-a333-26b242b2ba1e 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:osd.6 up in weight 1 up_from 46 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6818/783931631,v1:172.21.15.83:6819/783931631] [v2:172.21.15.83:6820/783931631,v1:172.21.15.83:6821/783931631] exists,up 835bb798-4cbd-4689-af88-4da9744dd601 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6816/1174963936,v1:172.21.15.173:6817/1174963936] [v2:172.21.15.173:6818/1174963936,v1:172.21.15.173:6819/1174963936] exists,up e0ac95e1-5d76-4f47-9483-1420f8c834c1 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6816/1549180524,v1:172.21.15.203:6817/1549180524] [v2:172.21.15.203:6818/1549180524,v1:172.21.15.203:6819/1549180524] exists,up 75266f49-b8fb-46c3-aa62-bef7957954dc 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6826/3165262956,v1:172.21.15.83:6827/3165262956] [v2:172.21.15.83:6828/3165262956,v1:172.21.15.83:6829/3165262956] exists,up bf897b76-ece1-4df5-8531-ed3aab716d5f 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6824/2121565520,v1:172.21.15.173:6825/2121565520] [v2:172.21.15.173:6826/2121565520,v1:172.21.15.173:6827/2121565520] exists,up 1b95d0e9-c1de-4607-b765-2eb61bb91faf 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.203:6824/2856576923,v1:172.21.15.203:6825/2856576923] [v2:172.21.15.203:6826/2856576923,v1:172.21.15.203:6827/2856576923] exists,up 636b2be4-2656-4ec1-89c4-739c158b6221 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.5 [8,11] 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.b [2,4] 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.1b [2,1] 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 2.1c [8,11] 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.e [4,6] 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.13 [4,9] 2024-04-23T14:48:11.069 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.1d [4,8] 2024-04-23T14:48:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.21 [5,9] 2024-04-23T14:48:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:pg_upmap_items 3.38 [3,9] 2024-04-23T14:48:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3063977912 expires 2024-04-24T14:37:57.064987+0000 2024-04-23T14:48:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/1987378267 expires 2024-04-24T14:37:57.064987+0000 2024-04-23T14:48:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/3974620988 expires 2024-04-24T14:37:57.064987+0000 2024-04-23T14:48:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/313104778 expires 2024-04-24T14:37:23.112444+0000 2024-04-23T14:48:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3743883304 expires 2024-04-24T14:37:23.112444+0000 2024-04-23T14:48:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/871735489 expires 2024-04-24T14:37:23.112444+0000 2024-04-23T14:48:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/1316070089 expires 2024-04-24T14:37:23.112444+0000 2024-04-23T14:48:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6801/1316070089 expires 2024-04-24T14:37:23.112444+0000 2024-04-23T14:48:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:0/3707212986 expires 2024-04-24T14:37:57.064987+0000 2024-04-23T14:48:11.070 INFO:teuthology.orchestra.run.smithi083.stdout:blocklist 172.21.15.83:6800/3974620988 expires 2024-04-24T14:37:57.064987+0000 2024-04-23T14:48:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.069+0000 7f0136200700 1 -- 172.21.15.83:0/4285201860 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f01140a3c40 msgr2=0x7f01140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.069+0000 7f0136200700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f01140a3c40 0x7f01140a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f011800b7b0 tx=0x7f0118005fd0 comp rx=0 tx=0).stop 2024-04-23T14:48:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.069+0000 7f0136200700 1 -- 172.21.15.83:0/4285201860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0130075550 msgr2=0x7f0130105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.069+0000 7f0136200700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0130075550 0x7f0130105e90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f012000e9f0 tx=0x7f012000ed00 comp rx=0 tx=0).stop 2024-04-23T14:48:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.069+0000 7f0136200700 1 -- 172.21.15.83:0/4285201860 shutdown_connections 2024-04-23T14:48:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.069+0000 7f0136200700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f01140a3c40 0x7f01140a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.069+0000 7f0136200700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f013010aa30 0x7f0130102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.069+0000 7f0136200700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0130075550 0x7f0130105e90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.069+0000 7f0136200700 1 --2- 172.21.15.83:0/4285201860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0130076b20 0x7f01301063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.069+0000 7f0136200700 1 -- 172.21.15.83:0/4285201860 >> 172.21.15.83:0/4285201860 conn(0x7f01300fe0c0 msgr2=0x7f0130106930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.069+0000 7f0136200700 1 -- 172.21.15.83:0/4285201860 shutdown_connections 2024-04-23T14:48:11.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:11.069+0000 7f0136200700 1 -- 172.21.15.83:0/4285201860 wait complete. 2024-04-23T14:48:11.604 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:11 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/4285201860' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T14:48:11.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:11 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/4285201860' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T14:48:11.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:11 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/4285201860' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T14:48:12.033 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-23T14:48:12.045 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:48:12.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.253+0000 7fefa2a00640 1 -- 172.21.15.83:0/1156562570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c1019f0 msgr2=0x7fef9c101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:12.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.253+0000 7fefa2a00640 1 --2- 172.21.15.83:0/1156562570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c1019f0 0x7fef9c101e70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fef94009ca0 tx=0x7fef9402a8c0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.253+0000 7fefa2a00640 1 -- 172.21.15.83:0/1156562570 shutdown_connections 2024-04-23T14:48:12.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.253+0000 7fefa2a00640 1 --2- 172.21.15.83:0/1156562570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c1019f0 0x7fef9c101e70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.253+0000 7fefa2a00640 1 --2- 172.21.15.83:0/1156562570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9c101050 0x7fef9c1014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.253+0000 7fefa2a00640 1 --2- 172.21.15.83:0/1156562570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef9c1006f0 0x7fef9c100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.253+0000 7fefa2a00640 1 -- 172.21.15.83:0/1156562570 >> 172.21.15.83:0/1156562570 conn(0x7fef9c0fc310 msgr2=0x7fef9c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:12.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.253+0000 7fefa2a00640 1 -- 172.21.15.83:0/1156562570 shutdown_connections 2024-04-23T14:48:12.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.253+0000 7fefa2a00640 1 -- 172.21.15.83:0/1156562570 wait complete. 2024-04-23T14:48:12.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.254+0000 7fefa2a00640 1 Processor -- start 2024-04-23T14:48:12.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.254+0000 7fefa2a00640 1 -- start start 2024-04-23T14:48:12.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.255+0000 7fefa2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c1006f0 0x7fef9c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:12.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.255+0000 7fefa2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef9c101050 0x7fef9c19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:12.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.255+0000 7fefa2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9c1019f0 0x7fef9c1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.255+0000 7fefa2a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c10fd20 con 0x7fef9c1006f0 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.255+0000 7fefa2a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c10fba0 con 0x7fef9c101050 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.255+0000 7fefa2a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c10fea0 con 0x7fef9c1019f0 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.255+0000 7fefa1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c1006f0 0x7fef9c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.255+0000 7fefa1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c1006f0 0x7fef9c19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37442/0 (socket says 172.21.15.83:37442) 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.255+0000 7fefa1600640 1 -- 172.21.15.83:0/1834975948 learned_addr learned my addr 172.21.15.83:0/1834975948 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.255+0000 7fefa0c00640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef9c101050 0x7fef9c19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.255+0000 7fefa2000640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9c1019f0 0x7fef9c1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.256+0000 7fefa1600640 1 -- 172.21.15.83:0/1834975948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9c1019f0 msgr2=0x7fef9c1a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.256+0000 7fefa1600640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9c1019f0 0x7fef9c1a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.256+0000 7fefa1600640 1 -- 172.21.15.83:0/1834975948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef9c101050 msgr2=0x7fef9c19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.256+0000 7fefa1600640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef9c101050 0x7fef9c19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.256+0000 7fefa1600640 1 -- 172.21.15.83:0/1834975948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c1a1f20 con 0x7fef9c1006f0 2024-04-23T14:48:12.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.256+0000 7fefa1600640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c1006f0 0x7fef9c19a300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fef9000d950 tx=0x7fef9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:12.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.256+0000 7fef9a000640 1 -- 172.21.15.83:0/1834975948 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90017070 con 0x7fef9c1006f0 2024-04-23T14:48:12.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.256+0000 7fef9a000640 1 -- 172.21.15.83:0/1834975948 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef90009c30 con 0x7fef9c1006f0 2024-04-23T14:48:12.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.256+0000 7fefa2a00640 1 -- 172.21.15.83:0/1834975948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9c1a2210 con 0x7fef9c1006f0 2024-04-23T14:48:12.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.256+0000 7fef9a000640 1 -- 172.21.15.83:0/1834975948 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9000f6b0 con 0x7fef9c1006f0 2024-04-23T14:48:12.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.256+0000 7fefa2a00640 1 -- 172.21.15.83:0/1834975948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9c1a27b0 con 0x7fef9c1006f0 2024-04-23T14:48:12.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.258+0000 7fefa2a00640 1 -- 172.21.15.83:0/1834975948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9c10b5a0 con 0x7fef9c1006f0 2024-04-23T14:48:12.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.260+0000 7fef9a000640 1 -- 172.21.15.83:0/1834975948 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fef900109d0 con 0x7fef9c1006f0 2024-04-23T14:48:12.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.262+0000 7fef9a000640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fef840a3c30 0x7fef840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.262+0000 7fef9a000640 1 -- 172.21.15.83:0/1834975948 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fef900cff30 con 0x7fef9c1006f0 2024-04-23T14:48:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.262+0000 7fefa0c00640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fef840a3c30 0x7fef840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:12.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.263+0000 7fefa0c00640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fef840a3c30 0x7fef840a60f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fef9c19b810 tx=0x7fef8800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:12.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.267+0000 7fef9a000640 1 -- 172.21.15.83:0/1834975948 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef90009de0 con 0x7fef9c1006f0 2024-04-23T14:48:12.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.470+0000 7fefa2a00640 1 -- 172.21.15.83:0/1834975948 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef9c062c00 con 0x7fef9c1006f0 2024-04-23T14:48:12.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.472+0000 7fef9a000640 1 -- 172.21.15.83:0/1834975948 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22699 (secure 0 0 0) 0x7fef9009c280 con 0x7fef9c1006f0 2024-04-23T14:48:12.473 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:12.473 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:46:58.351541+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:48:12.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.474+0000 7fefa2a00640 1 -- 172.21.15.83:0/1834975948 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fef840a3c30 msgr2=0x7fef840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:12.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.475+0000 7fefa2a00640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fef840a3c30 0x7fef840a60f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fef9c19b810 tx=0x7fef8800b390 comp rx=0 tx=0).stop 2024-04-23T14:48:12.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.475+0000 7fefa2a00640 1 -- 172.21.15.83:0/1834975948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c1006f0 msgr2=0x7fef9c19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:12.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.475+0000 7fefa2a00640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c1006f0 0x7fef9c19a300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fef9000d950 tx=0x7fef9000de10 comp rx=0 tx=0).stop 2024-04-23T14:48:12.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.475+0000 7fefa2a00640 1 -- 172.21.15.83:0/1834975948 shutdown_connections 2024-04-23T14:48:12.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.475+0000 7fefa2a00640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fef840a3c30 0x7fef840a60f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.475+0000 7fefa2a00640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef9c1019f0 0x7fef9c1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.475+0000 7fefa2a00640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef9c101050 0x7fef9c19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.475+0000 7fefa2a00640 1 --2- 172.21.15.83:0/1834975948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef9c1006f0 0x7fef9c19a300 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.475+0000 7fefa2a00640 1 -- 172.21.15.83:0/1834975948 >> 172.21.15.83:0/1834975948 conn(0x7fef9c0fc310 msgr2=0x7fef9c0fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:12.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.475+0000 7fefa2a00640 1 -- 172.21.15.83:0/1834975948 shutdown_connections 2024-04-23T14:48:12.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.475+0000 7fefa2a00640 1 -- 172.21.15.83:0/1834975948 wait complete. 2024-04-23T14:48:12.496 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:40:41.241992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:45:57.341602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:45:58.654798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:48:12.496 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:48:12.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.697+0000 7f7603e00640 1 -- 172.21.15.83:0/2263839367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7604104140 msgr2=0x7f7604104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:12.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.697+0000 7f7603e00640 1 --2- 172.21.15.83:0/2263839367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7604104140 0x7f7604104560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009be0 tx=0x7f75f402a6c0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.697+0000 7f7603e00640 1 -- 172.21.15.83:0/2263839367 shutdown_connections 2024-04-23T14:48:12.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.697+0000 7f7603e00640 1 --2- 172.21.15.83:0/2263839367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7604105cf0 0x7f760410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.697+0000 7f7603e00640 1 --2- 172.21.15.83:0/2263839367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7604105330 0x7f76041057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.697+0000 7f7603e00640 1 --2- 172.21.15.83:0/2263839367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7604104140 0x7f7604104560 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.697+0000 7f7603e00640 1 -- 172.21.15.83:0/2263839367 >> 172.21.15.83:0/2263839367 conn(0x7f76040ff6b0 msgr2=0x7f7604101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:12.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.697+0000 7f7603e00640 1 -- 172.21.15.83:0/2263839367 shutdown_connections 2024-04-23T14:48:12.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.697+0000 7f7603e00640 1 -- 172.21.15.83:0/2263839367 wait complete. 2024-04-23T14:48:12.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.698+0000 7f7603e00640 1 Processor -- start 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.698+0000 7f7603e00640 1 -- start start 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.698+0000 7f7603e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7604104140 0x7f760419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.698+0000 7f7603e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7604105330 0x7f760419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.698+0000 7f7603e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7604105cf0 0x7f76041a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f7603e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f760410fdd0 con 0x7f7604105330 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f7603e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f760410fc50 con 0x7f7604104140 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f7603e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f760410ff50 con 0x7f7604105cf0 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f75fa000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7604105330 0x7f760419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f75fa000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7604105330 0x7f760419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37458/0 (socket says 172.21.15.83:37458) 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f75fa000640 1 -- 172.21.15.83:0/3696593663 learned_addr learned my addr 172.21.15.83:0/3696593663 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f7602a00640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7604104140 0x7f760419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f7603400640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7604105cf0 0x7f76041a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:12.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f75fa000640 1 -- 172.21.15.83:0/3696593663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7604105cf0 msgr2=0x7f76041a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f75fa000640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7604105cf0 0x7f76041a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f75fa000640 1 -- 172.21.15.83:0/3696593663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7604104140 msgr2=0x7f760419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f75fa000640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7604104140 0x7f760419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f75fa000640 1 -- 172.21.15.83:0/3696593663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76041a20a0 con 0x7f7604105330 2024-04-23T14:48:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.699+0000 7f75fa000640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7604105330 0x7f760419a9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00e9f0 tx=0x7f75ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.700+0000 7f75fbe00640 1 -- 172.21.15.83:0/3696593663 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec00ccc0 con 0x7f7604105330 2024-04-23T14:48:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.700+0000 7f75fbe00640 1 -- 172.21.15.83:0/3696593663 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75ec005500 con 0x7f7604105330 2024-04-23T14:48:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.700+0000 7f7603e00640 1 -- 172.21.15.83:0/3696593663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76041a2390 con 0x7f7604105330 2024-04-23T14:48:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.700+0000 7f75fbe00640 1 -- 172.21.15.83:0/3696593663 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec010690 con 0x7f7604105330 2024-04-23T14:48:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.700+0000 7f7603e00640 1 -- 172.21.15.83:0/3696593663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76041a2890 con 0x7f7604105330 2024-04-23T14:48:12.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.701+0000 7f7603e00640 1 -- 172.21.15.83:0/3696593663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760410b4b0 con 0x7f7604105330 2024-04-23T14:48:12.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.703+0000 7f75fbe00640 1 -- 172.21.15.83:0/3696593663 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f75ec0046c0 con 0x7f7604105330 2024-04-23T14:48:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.705+0000 7f75fbe00640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f75e80a3bb0 0x7f75e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.706+0000 7f75fbe00640 1 -- 172.21.15.83:0/3696593663 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f75ec0d06c0 con 0x7f7604105330 2024-04-23T14:48:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.706+0000 7f7602a00640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f75e80a3bb0 0x7f75e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:12.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.706+0000 7f7602a00640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f75e80a3bb0 0x7f75e80a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f75f402abe0 tx=0x7f75f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:12.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.710+0000 7f75fbe00640 1 -- 172.21.15.83:0/3696593663 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75ec09d060 con 0x7f7604105330 2024-04-23T14:48:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:12 smithi203 ceph-mon[32800]: pgmap v385: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:48:12.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:12 smithi083 ceph-mon[27867]: pgmap v385: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:48:12.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:12 smithi173 ceph-mon[32443]: pgmap v385: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:48:12.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.914+0000 7f7603e00640 1 -- 172.21.15.83:0/3696593663 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7604062c00 con 0x7f7604105330 2024-04-23T14:48:12.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.915+0000 7f75fbe00640 1 -- 172.21.15.83:0/3696593663 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f75ec09c960 con 0x7f7604105330 2024-04-23T14:48:12.916 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-23T14:48:12.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.918+0000 7f7603e00640 1 -- 172.21.15.83:0/3696593663 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f75e80a3bb0 msgr2=0x7f75e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:12.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.918+0000 7f7603e00640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f75e80a3bb0 0x7f75e80a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f75f402abe0 tx=0x7f75f4005fd0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.918+0000 7f7603e00640 1 -- 172.21.15.83:0/3696593663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7604105330 msgr2=0x7f760419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:12.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.918+0000 7f7603e00640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7604105330 0x7f760419a9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00e9f0 tx=0x7f75ec00eeb0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.918+0000 7f7603e00640 1 -- 172.21.15.83:0/3696593663 shutdown_connections 2024-04-23T14:48:12.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.918+0000 7f7603e00640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f75e80a3bb0 0x7f75e80a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.919+0000 7f7603e00640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7604105cf0 0x7f76041a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.919+0000 7f7603e00640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7604105330 0x7f760419a9c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.919+0000 7f7603e00640 1 --2- 172.21.15.83:0/3696593663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7604104140 0x7f760419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:12.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.919+0000 7f7603e00640 1 -- 172.21.15.83:0/3696593663 >> 172.21.15.83:0/3696593663 conn(0x7f76040ff6b0 msgr2=0x7f7604108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:12.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.919+0000 7f7603e00640 1 -- 172.21.15.83:0/3696593663 shutdown_connections 2024-04-23T14:48:12.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:12.919+0000 7f7603e00640 1 -- 172.21.15.83:0/3696593663 wait complete. 2024-04-23T14:48:12.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:48:13.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.140+0000 7f554f200640 1 -- 172.21.15.83:0/2653547972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5550108540 msgr2=0x7f5550108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:13.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.140+0000 7f554f200640 1 --2- 172.21.15.83:0/2653547972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5550108540 0x7f5550108960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5540009900 tx=0x7f554002a410 comp rx=0 tx=0).stop 2024-04-23T14:48:13.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.140+0000 7f554f200640 1 -- 172.21.15.83:0/2653547972 shutdown_connections 2024-04-23T14:48:13.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.140+0000 7f554f200640 1 --2- 172.21.15.83:0/2653547972 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f555010a0f0 0x7f5550110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.140+0000 7f554f200640 1 --2- 172.21.15.83:0/2653547972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5550109730 0x7f5550109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.140+0000 7f554f200640 1 --2- 172.21.15.83:0/2653547972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5550108540 0x7f5550108960 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.140+0000 7f554f200640 1 -- 172.21.15.83:0/2653547972 >> 172.21.15.83:0/2653547972 conn(0x7f55500774d0 msgr2=0x7f5550079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:13.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.140+0000 7f554f200640 1 -- 172.21.15.83:0/2653547972 shutdown_connections 2024-04-23T14:48:13.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.140+0000 7f554f200640 1 -- 172.21.15.83:0/2653547972 wait complete. 2024-04-23T14:48:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.141+0000 7f554f200640 1 Processor -- start 2024-04-23T14:48:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.141+0000 7f554f200640 1 -- start start 2024-04-23T14:48:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.142+0000 7f554f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5550108540 0x7f555019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.142+0000 7f554f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5550109730 0x7f555019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.142+0000 7f554f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f555010a0f0 0x7f55501a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.142+0000 7f554f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f55501141f0 con 0x7f5550109730 2024-04-23T14:48:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.142+0000 7f554f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5550114070 con 0x7f5550108540 2024-04-23T14:48:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.142+0000 7f554f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5550114370 con 0x7f555010a0f0 2024-04-23T14:48:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.142+0000 7f554d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5550109730 0x7f555019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:13.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.142+0000 7f554de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5550108540 0x7f555019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.142+0000 7f554d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5550109730 0x7f555019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37466/0 (socket says 172.21.15.83:37466) 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.142+0000 7f554d400640 1 -- 172.21.15.83:0/115534091 learned_addr learned my addr 172.21.15.83:0/115534091 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.142+0000 7f554e800640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f555010a0f0 0x7f55501a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.143+0000 7f554d400640 1 -- 172.21.15.83:0/115534091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f555010a0f0 msgr2=0x7f55501a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.143+0000 7f554d400640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f555010a0f0 0x7f55501a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.143+0000 7f554d400640 1 -- 172.21.15.83:0/115534091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5550108540 msgr2=0x7f555019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.143+0000 7f554d400640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5550108540 0x7f555019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.143+0000 7f554d400640 1 -- 172.21.15.83:0/115534091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55501a6470 con 0x7f5550109730 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.143+0000 7f554de00640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5550108540 0x7f555019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.143+0000 7f554d400640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5550109730 0x7f555019ed90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f553800e9f0 tx=0x7f553800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.143+0000 7f5546a00640 1 -- 172.21.15.83:0/115534091 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553800cd80 con 0x7f5550109730 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.143+0000 7f554f200640 1 -- 172.21.15.83:0/115534091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55501a6760 con 0x7f5550109730 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.143+0000 7f5546a00640 1 -- 172.21.15.83:0/115534091 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5538010910 con 0x7f5550109730 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.143+0000 7f5546a00640 1 -- 172.21.15.83:0/115534091 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55380188f0 con 0x7f5550109730 2024-04-23T14:48:13.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.143+0000 7f554f200640 1 -- 172.21.15.83:0/115534091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55501a6d00 con 0x7f5550109730 2024-04-23T14:48:13.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.145+0000 7f554f200640 1 -- 172.21.15.83:0/115534091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f555010f800 con 0x7f5550109730 2024-04-23T14:48:13.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.147+0000 7f5546a00640 1 -- 172.21.15.83:0/115534091 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5538010ac0 con 0x7f5550109730 2024-04-23T14:48:13.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.148+0000 7f5546a00640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f55340a3c30 0x7f55340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.148+0000 7f5546a00640 1 -- 172.21.15.83:0/115534091 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f55380d0750 con 0x7f5550109730 2024-04-23T14:48:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.148+0000 7f554de00640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f55340a3c30 0x7f55340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:13.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.149+0000 7f554de00640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f55340a3c30 0x7f55340a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f554002aa00 tx=0x7f55400315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:13.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.153+0000 7f5546a00640 1 -- 172.21.15.83:0/115534091 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f553809caa0 con 0x7f5550109730 2024-04-23T14:48:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.358+0000 7f554f200640 1 -- 172.21.15.83:0/115534091 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5550062c00 con 0x7f5550109730 2024-04-23T14:48:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.359+0000 7f5546a00640 1 -- 172.21.15.83:0/115534091 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f55380a0da0 con 0x7f5550109730 2024-04-23T14:48:13.359 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-23T14:48:13.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.362+0000 7f554f200640 1 -- 172.21.15.83:0/115534091 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f55340a3c30 msgr2=0x7f55340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.362+0000 7f554f200640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f55340a3c30 0x7f55340a60f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f554002aa00 tx=0x7f55400315b0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.362+0000 7f554f200640 1 -- 172.21.15.83:0/115534091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5550109730 msgr2=0x7f555019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.362+0000 7f554f200640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5550109730 0x7f555019ed90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f553800e9f0 tx=0x7f553800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.362+0000 7f554f200640 1 -- 172.21.15.83:0/115534091 shutdown_connections 2024-04-23T14:48:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.362+0000 7f554f200640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f55340a3c30 0x7f55340a60f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.362+0000 7f554f200640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f555010a0f0 0x7f55501a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.362+0000 7f554f200640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5550109730 0x7f555019ed90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.362+0000 7f554f200640 1 --2- 172.21.15.83:0/115534091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5550108540 0x7f555019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.362+0000 7f554f200640 1 -- 172.21.15.83:0/115534091 >> 172.21.15.83:0/115534091 conn(0x7f55500774d0 msgr2=0x7f555010c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.362+0000 7f554f200640 1 -- 172.21.15.83:0/115534091 shutdown_connections 2024-04-23T14:48:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.362+0000 7f554f200640 1 -- 172.21.15.83:0/115534091 wait complete. 2024-04-23T14:48:13.382 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:48:13.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.584+0000 7efc87400640 1 -- 172.21.15.83:0/2332569214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc80105450 msgr2=0x7efc801058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:13.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.584+0000 7efc87400640 1 --2- 172.21.15.83:0/2332569214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc80105450 0x7efc801058d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efc68009900 tx=0x7efc6802a3f0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.585+0000 7efc87400640 1 -- 172.21.15.83:0/2332569214 shutdown_connections 2024-04-23T14:48:13.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.585+0000 7efc87400640 1 --2- 172.21.15.83:0/2332569214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc80105e10 0x7efc8010c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.585+0000 7efc87400640 1 --2- 172.21.15.83:0/2332569214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc80105450 0x7efc801058d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.585+0000 7efc87400640 1 --2- 172.21.15.83:0/2332569214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc80104260 0x7efc80104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.585+0000 7efc87400640 1 -- 172.21.15.83:0/2332569214 >> 172.21.15.83:0/2332569214 conn(0x7efc800ff7f0 msgr2=0x7efc80101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:13.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.585+0000 7efc87400640 1 -- 172.21.15.83:0/2332569214 shutdown_connections 2024-04-23T14:48:13.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.585+0000 7efc87400640 1 -- 172.21.15.83:0/2332569214 wait complete. 2024-04-23T14:48:13.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc87400640 1 Processor -- start 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc87400640 1 -- start start 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc87400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc80104260 0x7efc8019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc87400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc80105450 0x7efc8019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc87400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc80105e10 0x7efc801a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc87400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010ff40 con 0x7efc80105e10 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc87400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efc8010fdc0 con 0x7efc80105450 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc87400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efc801100c0 con 0x7efc80104260 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc84a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc80104260 0x7efc8019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc85400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc80105e10 0x7efc801a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc85400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc80105e10 0x7efc801a1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37472/0 (socket says 172.21.15.83:37472) 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc84a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc80104260 0x7efc8019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:43912/0 (socket says 172.21.15.83:43912) 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.586+0000 7efc85400640 1 -- 172.21.15.83:0/2571500332 learned_addr learned my addr 172.21.15.83:0/2571500332 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:13.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.587+0000 7efc77e00640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc80105450 0x7efc8019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.587+0000 7efc84a00640 1 -- 172.21.15.83:0/2571500332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc80105450 msgr2=0x7efc8019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.587+0000 7efc84a00640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc80105450 0x7efc8019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.587+0000 7efc84a00640 1 -- 172.21.15.83:0/2571500332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc80105e10 msgr2=0x7efc801a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.587+0000 7efc84a00640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc80105e10 0x7efc801a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.587+0000 7efc84a00640 1 -- 172.21.15.83:0/2571500332 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc801a2170 con 0x7efc80104260 2024-04-23T14:48:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.587+0000 7efc85400640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc80105e10 0x7efc801a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.587+0000 7efc84a00640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc80104260 0x7efc8019a550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efc7000e9c0 tx=0x7efc7000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.587+0000 7efc7e000640 1 -- 172.21.15.83:0/2571500332 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7000ccc0 con 0x7efc80104260 2024-04-23T14:48:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.587+0000 7efc7e000640 1 -- 172.21.15.83:0/2571500332 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc700035f0 con 0x7efc80104260 2024-04-23T14:48:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.587+0000 7efc87400640 1 -- 172.21.15.83:0/2571500332 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc801a2460 con 0x7efc80104260 2024-04-23T14:48:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.588+0000 7efc7e000640 1 -- 172.21.15.83:0/2571500332 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7000ce60 con 0x7efc80104260 2024-04-23T14:48:13.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.588+0000 7efc87400640 1 -- 172.21.15.83:0/2571500332 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc801a2980 con 0x7efc80104260 2024-04-23T14:48:13.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.589+0000 7efc87400640 1 -- 172.21.15.83:0/2571500332 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc8010b580 con 0x7efc80104260 2024-04-23T14:48:13.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.591+0000 7efc7e000640 1 -- 172.21.15.83:0/2571500332 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efc700037a0 con 0x7efc80104260 2024-04-23T14:48:13.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.592+0000 7efc7e000640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7efc600a3bb0 0x7efc600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:13.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.592+0000 7efc7e000640 1 -- 172.21.15.83:0/2571500332 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7efc700d0780 con 0x7efc80104260 2024-04-23T14:48:13.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.592+0000 7efc77e00640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7efc600a3bb0 0x7efc600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:13.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.593+0000 7efc77e00640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7efc600a3bb0 0x7efc600a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efc68031440 tx=0x7efc680315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:13.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.596+0000 7efc7e000640 1 -- 172.21.15.83:0/2571500332 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc7009d120 con 0x7efc80104260 2024-04-23T14:48:13.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:13 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1834975948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:13.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:13 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3696593663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:13.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:13 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/115534091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:13.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:13 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1834975948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:13.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:13 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3696593663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:13.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:13 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/115534091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:13.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.797+0000 7efc87400640 1 -- 172.21.15.83:0/2571500332 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efc80062c00 con 0x7efc80104260 2024-04-23T14:48:13.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.798+0000 7efc7e000640 1 -- 172.21.15.83:0/2571500332 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7efc7009ca20 con 0x7efc80104260 2024-04-23T14:48:13.798 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-23T14:48:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.800+0000 7efc87400640 1 -- 172.21.15.83:0/2571500332 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7efc600a3bb0 msgr2=0x7efc600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.800+0000 7efc87400640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7efc600a3bb0 0x7efc600a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efc68031440 tx=0x7efc680315b0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.800+0000 7efc87400640 1 -- 172.21.15.83:0/2571500332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc80104260 msgr2=0x7efc8019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.800+0000 7efc87400640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc80104260 0x7efc8019a550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efc7000e9c0 tx=0x7efc7000ee80 comp rx=0 tx=0).stop 2024-04-23T14:48:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.800+0000 7efc87400640 1 -- 172.21.15.83:0/2571500332 shutdown_connections 2024-04-23T14:48:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.800+0000 7efc87400640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7efc600a3bb0 0x7efc600a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.800+0000 7efc87400640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc80105e10 0x7efc801a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.800+0000 7efc87400640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc80105450 0x7efc8019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.800+0000 7efc87400640 1 --2- 172.21.15.83:0/2571500332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc80104260 0x7efc8019a550 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.800+0000 7efc87400640 1 -- 172.21.15.83:0/2571500332 >> 172.21.15.83:0/2571500332 conn(0x7efc800ff7f0 msgr2=0x7efc80108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.801+0000 7efc87400640 1 -- 172.21.15.83:0/2571500332 shutdown_connections 2024-04-23T14:48:13.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:13.801+0000 7efc87400640 1 -- 172.21.15.83:0/2571500332 wait complete. 2024-04-23T14:48:13.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:48:13.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:13 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1834975948' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:13.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:13 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3696593663' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:13.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:13 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/115534091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:14.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.021+0000 7efc83400640 1 -- 172.21.15.83:0/144731032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc7c105e10 msgr2=0x7efc7c10c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.021+0000 7efc83400640 1 --2- 172.21.15.83:0/144731032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc7c105e10 0x7efc7c10c6a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efc74009ca0 tx=0x7efc7402a8c0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.021+0000 7efc83400640 1 -- 172.21.15.83:0/144731032 shutdown_connections 2024-04-23T14:48:14.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.021+0000 7efc83400640 1 --2- 172.21.15.83:0/144731032 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc7c105e10 0x7efc7c10c6a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.021+0000 7efc83400640 1 --2- 172.21.15.83:0/144731032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc7c105450 0x7efc7c1058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.021+0000 7efc83400640 1 --2- 172.21.15.83:0/144731032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc7c104260 0x7efc7c104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.021+0000 7efc83400640 1 -- 172.21.15.83:0/144731032 >> 172.21.15.83:0/144731032 conn(0x7efc7c0ff7f0 msgr2=0x7efc7c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:14.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.021+0000 7efc83400640 1 -- 172.21.15.83:0/144731032 shutdown_connections 2024-04-23T14:48:14.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.021+0000 7efc83400640 1 -- 172.21.15.83:0/144731032 wait complete. 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.022+0000 7efc83400640 1 Processor -- start 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.022+0000 7efc83400640 1 -- start start 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.023+0000 7efc83400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc7c104260 0x7efc7c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.023+0000 7efc83400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc7c105450 0x7efc7c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.023+0000 7efc83400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc7c105e10 0x7efc7c1a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.023+0000 7efc83400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c10fec0 con 0x7efc7c104260 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.023+0000 7efc83400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c10fd40 con 0x7efc7c105450 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.023+0000 7efc83400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7efc7c110040 con 0x7efc7c105e10 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.023+0000 7efc80a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc7c104260 0x7efc7c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.023+0000 7efc81400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc7c105e10 0x7efc7c1a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.023+0000 7efc80a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc7c104260 0x7efc7c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37484/0 (socket says 172.21.15.83:37484) 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.023+0000 7efc80a00640 1 -- 172.21.15.83:0/1046909351 learned_addr learned my addr 172.21.15.83:0/1046909351 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:14.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.023+0000 7efc73e00640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc7c105450 0x7efc7c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.024+0000 7efc80a00640 1 -- 172.21.15.83:0/1046909351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc7c105e10 msgr2=0x7efc7c1a1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.024+0000 7efc80a00640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc7c105e10 0x7efc7c1a1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.024+0000 7efc80a00640 1 -- 172.21.15.83:0/1046909351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc7c105450 msgr2=0x7efc7c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.024+0000 7efc80a00640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc7c105450 0x7efc7c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.024+0000 7efc80a00640 1 -- 172.21.15.83:0/1046909351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7c1a2120 con 0x7efc7c104260 2024-04-23T14:48:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.024+0000 7efc80a00640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc7c104260 0x7efc7c19a500 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00e9c0 tx=0x7efc6c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.024+0000 7efc7a000640 1 -- 172.21.15.83:0/1046909351 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c00cd80 con 0x7efc7c104260 2024-04-23T14:48:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.024+0000 7efc83400640 1 -- 172.21.15.83:0/1046909351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7c1a2410 con 0x7efc7c104260 2024-04-23T14:48:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.024+0000 7efc7a000640 1 -- 172.21.15.83:0/1046909351 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc6c0035f0 con 0x7efc7c104260 2024-04-23T14:48:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.024+0000 7efc7a000640 1 -- 172.21.15.83:0/1046909351 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c010690 con 0x7efc7c104260 2024-04-23T14:48:14.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.024+0000 7efc83400640 1 -- 172.21.15.83:0/1046909351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7c1a29b0 con 0x7efc7c104260 2024-04-23T14:48:14.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.026+0000 7efc83400640 1 -- 172.21.15.83:0/1046909351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc7c10b520 con 0x7efc7c104260 2024-04-23T14:48:14.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.027+0000 7efc7a000640 1 -- 172.21.15.83:0/1046909351 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efc6c0037a0 con 0x7efc7c104260 2024-04-23T14:48:14.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.029+0000 7efc7a000640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7efc5c0a3c30 0x7efc5c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:14.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.029+0000 7efc7a000640 1 -- 172.21.15.83:0/1046909351 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7efc6c0d0740 con 0x7efc7c104260 2024-04-23T14:48:14.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.029+0000 7efc73e00640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7efc5c0a3c30 0x7efc5c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:14.030 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.030+0000 7efc73e00640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7efc5c0a3c30 0x7efc5c0a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc640095f0 tx=0x7efc64005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:14.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.033+0000 7efc7a000640 1 -- 172.21.15.83:0/1046909351 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc6c09d120 con 0x7efc7c104260 2024-04-23T14:48:14.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.269+0000 7efc83400640 1 -- 172.21.15.83:0/1046909351 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc7c02d620 con 0x7efc7c104260 2024-04-23T14:48:14.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.270+0000 7efc7a000640 1 -- 172.21.15.83:0/1046909351 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7efc6c09ca20 con 0x7efc7c104260 2024-04-23T14:48:14.272 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:14.272 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:46:05.301806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:48:14.273 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-04-23T14:48:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.273+0000 7efc83400640 1 -- 172.21.15.83:0/1046909351 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7efc5c0a3c30 msgr2=0x7efc5c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.273+0000 7efc83400640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7efc5c0a3c30 0x7efc5c0a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc640095f0 tx=0x7efc64005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.273+0000 7efc83400640 1 -- 172.21.15.83:0/1046909351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc7c104260 msgr2=0x7efc7c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.273+0000 7efc83400640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc7c104260 0x7efc7c19a500 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00e9c0 tx=0x7efc6c00ee80 comp rx=0 tx=0).stop 2024-04-23T14:48:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.273+0000 7efc83400640 1 -- 172.21.15.83:0/1046909351 shutdown_connections 2024-04-23T14:48:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.273+0000 7efc83400640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7efc5c0a3c30 0x7efc5c0a60f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.273+0000 7efc83400640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc7c105e10 0x7efc7c1a1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.273+0000 7efc83400640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc7c105450 0x7efc7c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.273+0000 7efc83400640 1 --2- 172.21.15.83:0/1046909351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc7c104260 0x7efc7c19a500 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.273+0000 7efc83400640 1 -- 172.21.15.83:0/1046909351 >> 172.21.15.83:0/1046909351 conn(0x7efc7c0ff7f0 msgr2=0x7efc7c1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.273+0000 7efc83400640 1 -- 172.21.15.83:0/1046909351 shutdown_connections 2024-04-23T14:48:14.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.273+0000 7efc83400640 1 -- 172.21.15.83:0/1046909351 wait complete. 2024-04-23T14:48:14.293 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:48:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.485+0000 7f9eebe00640 1 -- 172.21.15.83:0/2435323315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eec104080 msgr2=0x7f9eec1044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.485+0000 7f9eebe00640 1 --2- 172.21.15.83:0/2435323315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eec104080 0x7f9eec1044a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009900 tx=0x7f9edc02a3f0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.485+0000 7f9eebe00640 1 -- 172.21.15.83:0/2435323315 shutdown_connections 2024-04-23T14:48:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.485+0000 7f9eebe00640 1 --2- 172.21.15.83:0/2435323315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9eec105c30 0x7f9eec10c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.485+0000 7f9eebe00640 1 --2- 172.21.15.83:0/2435323315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9eec105270 0x7f9eec1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.485+0000 7f9eebe00640 1 --2- 172.21.15.83:0/2435323315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eec104080 0x7f9eec1044a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.485+0000 7f9eebe00640 1 -- 172.21.15.83:0/2435323315 >> 172.21.15.83:0/2435323315 conn(0x7f9eec0ff610 msgr2=0x7f9eec101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.485+0000 7f9eebe00640 1 -- 172.21.15.83:0/2435323315 shutdown_connections 2024-04-23T14:48:14.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.485+0000 7f9eebe00640 1 -- 172.21.15.83:0/2435323315 wait complete. 2024-04-23T14:48:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.486+0000 7f9eebe00640 1 Processor -- start 2024-04-23T14:48:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.486+0000 7f9eebe00640 1 -- start start 2024-04-23T14:48:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.486+0000 7f9eebe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9eec104080 0x7f9eec19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.486+0000 7f9eebe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9eec105270 0x7f9eec19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.486+0000 7f9eebe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eec105c30 0x7f9eec1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.486+0000 7f9eebe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec10fd60 con 0x7f9eec105c30 2024-04-23T14:48:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.486+0000 7f9eebe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec10fbe0 con 0x7f9eec105270 2024-04-23T14:48:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.486+0000 7f9eebe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec10fee0 con 0x7f9eec104080 2024-04-23T14:48:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.487+0000 7f9eeaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9eec104080 0x7f9eec19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:14.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.487+0000 7f9ee2000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9eec105270 0x7f9eec19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.487+0000 7f9eeaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9eec104080 0x7f9eec19a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:43940/0 (socket says 172.21.15.83:43940) 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.487+0000 7f9eeaa00640 1 -- 172.21.15.83:0/2270384379 learned_addr learned my addr 172.21.15.83:0/2270384379 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.487+0000 7f9eeb400640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eec105c30 0x7f9eec1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.487+0000 7f9eeaa00640 1 -- 172.21.15.83:0/2270384379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9eec105270 msgr2=0x7f9eec19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.487+0000 7f9eeaa00640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9eec105270 0x7f9eec19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.487+0000 7f9eeaa00640 1 -- 172.21.15.83:0/2270384379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eec105c30 msgr2=0x7f9eec1a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.487+0000 7f9eeaa00640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eec105c30 0x7f9eec1a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.487+0000 7f9eeaa00640 1 -- 172.21.15.83:0/2270384379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec1a1f80 con 0x7f9eec104080 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.487+0000 7f9eeb400640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eec105c30 0x7f9eec1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.487+0000 7f9eeaa00640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9eec104080 0x7f9eec19a360 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9edc031440 tx=0x7f9edc032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.488+0000 7f9ef0a00640 1 -- 172.21.15.83:0/2270384379 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc03f030 con 0x7f9eec104080 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.488+0000 7f9ef0a00640 1 -- 172.21.15.83:0/2270384379 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9edc003ce0 con 0x7f9eec104080 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.488+0000 7f9eebe00640 1 -- 172.21.15.83:0/2270384379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec1a2210 con 0x7f9eec104080 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.488+0000 7f9ef0a00640 1 -- 172.21.15.83:0/2270384379 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc035070 con 0x7f9eec104080 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.488+0000 7f9eebe00640 1 -- 172.21.15.83:0/2270384379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec1a2700 con 0x7f9eec104080 2024-04-23T14:48:14.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.489+0000 7f9eebe00640 1 -- 172.21.15.83:0/2270384379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eec10b3a0 con 0x7f9eec104080 2024-04-23T14:48:14.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.491+0000 7f9ef0a00640 1 -- 172.21.15.83:0/2270384379 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9edc042030 con 0x7f9eec104080 2024-04-23T14:48:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.492+0000 7f9ef0a00640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9ed00a3bb0 0x7f9ed00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.492+0000 7f9ef0a00640 1 -- 172.21.15.83:0/2270384379 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f9edc0f0be0 con 0x7f9eec104080 2024-04-23T14:48:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.492+0000 7f9ee2000640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9ed00a3bb0 0x7f9ed00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:14.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.493+0000 7f9ee2000640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9ed00a3bb0 0x7f9ed00a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9eec19b870 tx=0x7f9ed4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.496+0000 7f9ef0a00640 1 -- 172.21.15.83:0/2270384379 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9edc0bcf30 con 0x7f9eec104080 2024-04-23T14:48:14.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.683+0000 7f9eebe00640 1 -- 172.21.15.83:0/2270384379 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9eec062c00 con 0x7f9eec104080 2024-04-23T14:48:14.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.685+0000 7f9ef0a00640 1 -- 172.21.15.83:0/2270384379 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22699 (secure 0 0 0) 0x7f9edc0c1230 con 0x7f9eec104080 2024-04-23T14:48:14.687 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:14.687 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:46:58.351541+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:48:14.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.687+0000 7f9eebe00640 1 -- 172.21.15.83:0/2270384379 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9ed00a3bb0 msgr2=0x7f9ed00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.687+0000 7f9eebe00640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9ed00a3bb0 0x7f9ed00a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9eec19b870 tx=0x7f9ed4005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:14.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.687+0000 7f9eebe00640 1 -- 172.21.15.83:0/2270384379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9eec104080 msgr2=0x7f9eec19a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.687+0000 7f9eebe00640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9eec104080 0x7f9eec19a360 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9edc031440 tx=0x7f9edc032ce0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.687+0000 7f9eebe00640 1 -- 172.21.15.83:0/2270384379 shutdown_connections 2024-04-23T14:48:14.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.687+0000 7f9eebe00640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9ed00a3bb0 0x7f9ed00a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.687+0000 7f9eebe00640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9eec105c30 0x7f9eec1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.687+0000 7f9eebe00640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9eec105270 0x7f9eec19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.687+0000 7f9eebe00640 1 --2- 172.21.15.83:0/2270384379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9eec104080 0x7f9eec19a360 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.687+0000 7f9eebe00640 1 -- 172.21.15.83:0/2270384379 >> 172.21.15.83:0/2270384379 conn(0x7f9eec0ff610 msgr2=0x7f9eec108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:14.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.688+0000 7f9eebe00640 1 -- 172.21.15.83:0/2270384379 shutdown_connections 2024-04-23T14:48:14.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.688+0000 7f9eebe00640 1 -- 172.21.15.83:0/2270384379 wait complete. 2024-04-23T14:48:14.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:14 smithi083 ceph-mon[27867]: pgmap v386: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:48:14.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:14 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2571500332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:14.704 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:14 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1046909351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:48:14.706 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:40:41.241992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:45:57.341602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:45:58.654798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:48:14.707 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:48:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:14 smithi203 ceph-mon[32800]: pgmap v386: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:48:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:14 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2571500332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:14 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1046909351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:48:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:14 smithi173 ceph-mon[32443]: pgmap v386: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:48:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:14 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2571500332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:14 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1046909351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:48:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.908+0000 7fef4a400640 1 -- 172.21.15.83:0/3387474422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44105cf0 msgr2=0x7fef4410c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.908+0000 7fef4a400640 1 --2- 172.21.15.83:0/3387474422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44105cf0 0x7fef4410c580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009ca0 tx=0x7fef3c02a8c0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.908+0000 7fef4a400640 1 -- 172.21.15.83:0/3387474422 shutdown_connections 2024-04-23T14:48:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.908+0000 7fef4a400640 1 --2- 172.21.15.83:0/3387474422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44105cf0 0x7fef4410c580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.908+0000 7fef4a400640 1 --2- 172.21.15.83:0/3387474422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef44105330 0x7fef441057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.908+0000 7fef4a400640 1 --2- 172.21.15.83:0/3387474422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef44104140 0x7fef44104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.908+0000 7fef4a400640 1 -- 172.21.15.83:0/3387474422 >> 172.21.15.83:0/3387474422 conn(0x7fef440ff6b0 msgr2=0x7fef44101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.908+0000 7fef4a400640 1 -- 172.21.15.83:0/3387474422 shutdown_connections 2024-04-23T14:48:14.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.908+0000 7fef4a400640 1 -- 172.21.15.83:0/3387474422 wait complete. 2024-04-23T14:48:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.909+0000 7fef4a400640 1 Processor -- start 2024-04-23T14:48:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.909+0000 7fef4a400640 1 -- start start 2024-04-23T14:48:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef4a400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef44104140 0x7fef4419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef4a400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef44105330 0x7fef4419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef4a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44105cf0 0x7fef441a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef4a400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410fe20 con 0x7fef44105cf0 2024-04-23T14:48:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef4a400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410fca0 con 0x7fef44105330 2024-04-23T14:48:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef4a400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef4410ffa0 con 0x7fef44104140 2024-04-23T14:48:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef3be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef44105330 0x7fef4419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef49a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44105cf0 0x7fef441a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:14.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef3be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef44105330 0x7fef4419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:57004/0 (socket says 172.21.15.83:57004) 2024-04-23T14:48:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef3be00640 1 -- 172.21.15.83:0/3712277943 learned_addr learned my addr 172.21.15.83:0/3712277943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef49000640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef44104140 0x7fef4419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef3be00640 1 -- 172.21.15.83:0/3712277943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef44104140 msgr2=0x7fef4419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef3be00640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef44104140 0x7fef4419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef3be00640 1 -- 172.21.15.83:0/3712277943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44105cf0 msgr2=0x7fef441a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef3be00640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44105cf0 0x7fef441a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef3be00640 1 -- 172.21.15.83:0/3712277943 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef441a2090 con 0x7fef44105330 2024-04-23T14:48:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.910+0000 7fef49000640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef44104140 0x7fef4419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.911+0000 7fef3be00640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef44105330 0x7fef4419a9b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fef3400e9f0 tx=0x7fef3400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.911+0000 7fef42000640 1 -- 172.21.15.83:0/3712277943 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3400cd80 con 0x7fef44105330 2024-04-23T14:48:14.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.911+0000 7fef4a400640 1 -- 172.21.15.83:0/3712277943 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef441a2380 con 0x7fef44105330 2024-04-23T14:48:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.911+0000 7fef42000640 1 -- 172.21.15.83:0/3712277943 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef34010910 con 0x7fef44105330 2024-04-23T14:48:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.911+0000 7fef42000640 1 -- 172.21.15.83:0/3712277943 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef340188f0 con 0x7fef44105330 2024-04-23T14:48:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.911+0000 7fef4a400640 1 -- 172.21.15.83:0/3712277943 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef441a28d0 con 0x7fef44105330 2024-04-23T14:48:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.912+0000 7fef4a400640 1 -- 172.21.15.83:0/3712277943 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4410b390 con 0x7fef44105330 2024-04-23T14:48:14.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.914+0000 7fef42000640 1 -- 172.21.15.83:0/3712277943 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fef34010ac0 con 0x7fef44105330 2024-04-23T14:48:14.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.916+0000 7fef42000640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fef240a3bb0 0x7fef240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:14.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.916+0000 7fef42000640 1 -- 172.21.15.83:0/3712277943 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fef340d0b30 con 0x7fef44105330 2024-04-23T14:48:14.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.916+0000 7fef49000640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fef240a3bb0 0x7fef240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:14.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.916+0000 7fef49000640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fef240a3bb0 0x7fef240a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef44105190 tx=0x7fef2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:14.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:14.918+0000 7fef42000640 1 -- 172.21.15.83:0/3712277943 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef3409ce80 con 0x7fef44105330 2024-04-23T14:48:15.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.117+0000 7fef4a400640 1 -- 172.21.15.83:0/3712277943 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef4402d870 con 0x7fef44105330 2024-04-23T14:48:15.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.118+0000 7fef42000640 1 -- 172.21.15.83:0/3712277943 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fef340a1180 con 0x7fef44105330 2024-04-23T14:48:15.119 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-23T14:48:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.120+0000 7fef4a400640 1 -- 172.21.15.83:0/3712277943 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fef240a3bb0 msgr2=0x7fef240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.120+0000 7fef4a400640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fef240a3bb0 0x7fef240a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef44105190 tx=0x7fef2c005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.120+0000 7fef4a400640 1 -- 172.21.15.83:0/3712277943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef44105330 msgr2=0x7fef4419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.120+0000 7fef4a400640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef44105330 0x7fef4419a9b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fef3400e9f0 tx=0x7fef3400eeb0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.120+0000 7fef4a400640 1 -- 172.21.15.83:0/3712277943 shutdown_connections 2024-04-23T14:48:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.120+0000 7fef4a400640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fef240a3bb0 0x7fef240a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.120+0000 7fef4a400640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44105cf0 0x7fef441a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.120+0000 7fef4a400640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef44105330 0x7fef4419a9b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.120+0000 7fef4a400640 1 --2- 172.21.15.83:0/3712277943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef44104140 0x7fef4419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.120+0000 7fef4a400640 1 -- 172.21.15.83:0/3712277943 >> 172.21.15.83:0/3712277943 conn(0x7fef440ff6b0 msgr2=0x7fef44108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:15.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.120+0000 7fef4a400640 1 -- 172.21.15.83:0/3712277943 shutdown_connections 2024-04-23T14:48:15.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.120+0000 7fef4a400640 1 -- 172.21.15.83:0/3712277943 wait complete. 2024-04-23T14:48:15.140 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:48:15.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.339+0000 7f2cae600640 1 -- 172.21.15.83:0/3351625651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca810a0f0 msgr2=0x7f2ca8110980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:15.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.339+0000 7f2cae600640 1 --2- 172.21.15.83:0/3351625651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca810a0f0 0x7f2ca8110980 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009ca0 tx=0x7f2ca002a8b0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.339+0000 7f2cae600640 1 -- 172.21.15.83:0/3351625651 shutdown_connections 2024-04-23T14:48:15.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.339+0000 7f2cae600640 1 --2- 172.21.15.83:0/3351625651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca810a0f0 0x7f2ca8110980 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.339+0000 7f2cae600640 1 --2- 172.21.15.83:0/3351625651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca8109730 0x7f2ca8109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.339+0000 7f2cae600640 1 --2- 172.21.15.83:0/3351625651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca8108540 0x7f2ca8108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.339+0000 7f2cae600640 1 -- 172.21.15.83:0/3351625651 >> 172.21.15.83:0/3351625651 conn(0x7f2ca80774d0 msgr2=0x7f2ca8079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:15.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.339+0000 7f2cae600640 1 -- 172.21.15.83:0/3351625651 shutdown_connections 2024-04-23T14:48:15.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.339+0000 7f2cae600640 1 -- 172.21.15.83:0/3351625651 wait complete. 2024-04-23T14:48:15.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.340+0000 7f2cae600640 1 Processor -- start 2024-04-23T14:48:15.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2cae600640 1 -- start start 2024-04-23T14:48:15.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2cae600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca8108540 0x7f2ca819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:15.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2cae600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca8109730 0x7f2ca819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:15.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2cae600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca810a0f0 0x7f2ca81a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:15.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2cae600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca81141d0 con 0x7f2ca810a0f0 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2cae600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8114050 con 0x7f2ca8109730 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2cae600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8114350 con 0x7f2ca8108540 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2cadc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca810a0f0 0x7f2ca81a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2ca7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca8109730 0x7f2ca819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2cad200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca8108540 0x7f2ca819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2ca7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca8109730 0x7f2ca819ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:57020/0 (socket says 172.21.15.83:57020) 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2cad200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca8108540 0x7f2ca819e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38282/0 (socket says 172.21.15.83:38282) 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2cadc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca810a0f0 0x7f2ca81a5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47900/0 (socket says 172.21.15.83:47900) 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2ca7e00640 1 -- 172.21.15.83:0/1265586090 learned_addr learned my addr 172.21.15.83:0/1265586090 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.341+0000 7f2cad200640 1 -- 172.21.15.83:0/1265586090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca8109730 msgr2=0x7f2ca819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.342+0000 7f2cad200640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca8109730 0x7f2ca819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.342+0000 7f2cad200640 1 -- 172.21.15.83:0/1265586090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca810a0f0 msgr2=0x7f2ca81a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.342+0000 7f2cad200640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca810a0f0 0x7f2ca81a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.342+0000 7f2cad200640 1 -- 172.21.15.83:0/1265586090 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca81a6390 con 0x7f2ca8108540 2024-04-23T14:48:15.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.342+0000 7f2ca7e00640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca8109730 0x7f2ca819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:15.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.342+0000 7f2cadc00640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca810a0f0 0x7f2ca81a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:15.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.342+0000 7f2cad200640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca8108540 0x7f2ca819e770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00e9c0 tx=0x7f2c9c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:15.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.342+0000 7f2ca5600640 1 -- 172.21.15.83:0/1265586090 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c00cd80 con 0x7f2ca8108540 2024-04-23T14:48:15.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.342+0000 7f2cae600640 1 -- 172.21.15.83:0/1265586090 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca81a6680 con 0x7f2ca8108540 2024-04-23T14:48:15.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.342+0000 7f2cae600640 1 -- 172.21.15.83:0/1265586090 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca81a6c20 con 0x7f2ca8108540 2024-04-23T14:48:15.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.343+0000 7f2ca5600640 1 -- 172.21.15.83:0/1265586090 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c9c009310 con 0x7f2ca8108540 2024-04-23T14:48:15.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.343+0000 7f2ca5600640 1 -- 172.21.15.83:0/1265586090 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c018680 con 0x7f2ca8108540 2024-04-23T14:48:15.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.343+0000 7f2cae600640 1 -- 172.21.15.83:0/1265586090 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c880052c0 con 0x7f2ca8108540 2024-04-23T14:48:15.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.346+0000 7f2ca5600640 1 -- 172.21.15.83:0/1265586090 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2c9c01f020 con 0x7f2ca8108540 2024-04-23T14:48:15.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.347+0000 7f2ca5600640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2c900a3bb0 0x7f2c900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:15.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.347+0000 7f2ca5600640 1 -- 172.21.15.83:0/1265586090 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2c9c0d04f0 con 0x7f2ca8108540 2024-04-23T14:48:15.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.347+0000 7f2ca7e00640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2c900a3bb0 0x7f2c900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:15.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.348+0000 7f2ca7e00640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2c900a3bb0 0x7f2c900a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2ca819fc80 tx=0x7f2c94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:15.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.349+0000 7f2ca5600640 1 -- 172.21.15.83:0/1265586090 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c9c09ce90 con 0x7f2ca8108540 2024-04-23T14:48:15.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:15 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2270384379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:15.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:15 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3712277943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:15.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.553+0000 7f2cae600640 1 -- 172.21.15.83:0/1265586090 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2c88005880 con 0x7f2ca8108540 2024-04-23T14:48:15.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.554+0000 7f2ca5600640 1 -- 172.21.15.83:0/1265586090 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f2c9c09c790 con 0x7f2ca8108540 2024-04-23T14:48:15.558 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-23T14:48:15.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.556+0000 7f2cae600640 1 -- 172.21.15.83:0/1265586090 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2c900a3bb0 msgr2=0x7f2c900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:15.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.556+0000 7f2cae600640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2c900a3bb0 0x7f2c900a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2ca819fc80 tx=0x7f2c94005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:15.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.556+0000 7f2cae600640 1 -- 172.21.15.83:0/1265586090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca8108540 msgr2=0x7f2ca819e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:15.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.556+0000 7f2cae600640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca8108540 0x7f2ca819e770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00e9c0 tx=0x7f2c9c00ee80 comp rx=0 tx=0).stop 2024-04-23T14:48:15.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.556+0000 7f2cae600640 1 -- 172.21.15.83:0/1265586090 shutdown_connections 2024-04-23T14:48:15.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.556+0000 7f2cae600640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f2c900a3bb0 0x7f2c900a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.556+0000 7f2cae600640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ca810a0f0 0x7f2ca81a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.556+0000 7f2cae600640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ca8109730 0x7f2ca819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.556+0000 7f2cae600640 1 --2- 172.21.15.83:0/1265586090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2ca8108540 0x7f2ca819e770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.556+0000 7f2cae600640 1 -- 172.21.15.83:0/1265586090 >> 172.21.15.83:0/1265586090 conn(0x7f2ca80774d0 msgr2=0x7f2ca810c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:15.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.557+0000 7f2cae600640 1 -- 172.21.15.83:0/1265586090 shutdown_connections 2024-04-23T14:48:15.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.557+0000 7f2cae600640 1 -- 172.21.15.83:0/1265586090 wait complete. 2024-04-23T14:48:15.577 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:48:15.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:15 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2270384379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:15.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:15 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3712277943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:15.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.780+0000 7f9fdbe00640 1 -- 172.21.15.83:0/3187101945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fdc108540 msgr2=0x7f9fdc108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:15.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.780+0000 7f9fdbe00640 1 --2- 172.21.15.83:0/3187101945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fdc108540 0x7f9fdc108960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009ca0 tx=0x7f9fcc02a8c0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.780+0000 7f9fdbe00640 1 -- 172.21.15.83:0/3187101945 shutdown_connections 2024-04-23T14:48:15.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.780+0000 7f9fdbe00640 1 --2- 172.21.15.83:0/3187101945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc10a0f0 0x7f9fdc110980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.780+0000 7f9fdbe00640 1 --2- 172.21.15.83:0/3187101945 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fdc109730 0x7f9fdc109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.780+0000 7f9fdbe00640 1 --2- 172.21.15.83:0/3187101945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fdc108540 0x7f9fdc108960 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.780+0000 7f9fdbe00640 1 -- 172.21.15.83:0/3187101945 >> 172.21.15.83:0/3187101945 conn(0x7f9fdc0774d0 msgr2=0x7f9fdc079910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:15.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.780+0000 7f9fdbe00640 1 -- 172.21.15.83:0/3187101945 shutdown_connections 2024-04-23T14:48:15.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.780+0000 7f9fdbe00640 1 -- 172.21.15.83:0/3187101945 wait complete. 2024-04-23T14:48:15.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.781+0000 7f9fdbe00640 1 Processor -- start 2024-04-23T14:48:15.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.781+0000 7f9fdbe00640 1 -- start start 2024-04-23T14:48:15.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fdc108540 0x7f9fdc19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:15.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdbe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc109730 0x7f9fdc19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:15.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fdc10a0f0 0x7f9fdc1a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:15.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdbe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc1141d0 con 0x7f9fdc108540 2024-04-23T14:48:15.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdbe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc114050 con 0x7f9fdc10a0f0 2024-04-23T14:48:15.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdbe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc114350 con 0x7f9fdc109730 2024-04-23T14:48:15.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fdc108540 0x7f9fdc19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fdc108540 0x7f9fdc19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47918/0 (socket says 172.21.15.83:47918) 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdb400640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fdc10a0f0 0x7f9fdc1a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdaa00640 1 -- 172.21.15.83:0/1521456792 learned_addr learned my addr 172.21.15.83:0/1521456792 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fd2000640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc109730 0x7f9fdc19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdaa00640 1 -- 172.21.15.83:0/1521456792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc109730 msgr2=0x7f9fdc19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdaa00640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc109730 0x7f9fdc19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdaa00640 1 -- 172.21.15.83:0/1521456792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fdc10a0f0 msgr2=0x7f9fdc1a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdaa00640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fdc10a0f0 0x7f9fdc1a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdaa00640 1 -- 172.21.15.83:0/1521456792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc1a6420 con 0x7f9fdc108540 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.782+0000 7f9fdb400640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fdc10a0f0 0x7f9fdc1a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.783+0000 7f9fdaa00640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fdc108540 0x7f9fdc19e800 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009bf0 tx=0x7f9fcc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.783+0000 7f9fe0a00640 1 -- 172.21.15.83:0/1521456792 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc02fa20 con 0x7f9fdc108540 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.783+0000 7f9fe0a00640 1 -- 172.21.15.83:0/1521456792 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fcc02fbc0 con 0x7f9fdc108540 2024-04-23T14:48:15.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.783+0000 7f9fdbe00640 1 -- 172.21.15.83:0/1521456792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fdc1a66b0 con 0x7f9fdc108540 2024-04-23T14:48:15.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.783+0000 7f9fe0a00640 1 -- 172.21.15.83:0/1521456792 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc046750 con 0x7f9fdc108540 2024-04-23T14:48:15.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.783+0000 7f9fdbe00640 1 -- 172.21.15.83:0/1521456792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fdc1a6c50 con 0x7f9fdc108540 2024-04-23T14:48:15.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.784+0000 7f9fdbe00640 1 -- 172.21.15.83:0/1521456792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fdc10f860 con 0x7f9fdc108540 2024-04-23T14:48:15.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.786+0000 7f9fe0a00640 1 -- 172.21.15.83:0/1521456792 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9fcc02fd70 con 0x7f9fdc108540 2024-04-23T14:48:15.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.787+0000 7f9fe0a00640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9fc00a3c30 0x7f9fc00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:15.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.788+0000 7f9fe0a00640 1 -- 172.21.15.83:0/1521456792 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f9fcc0f1740 con 0x7f9fdc108540 2024-04-23T14:48:15.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.788+0000 7f9fd2000640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9fc00a3c30 0x7f9fc00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:15.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.788+0000 7f9fd2000640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9fc00a3c30 0x7f9fc00a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc19fd10 tx=0x7f9fc4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:15.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.792+0000 7f9fe0a00640 1 -- 172.21.15.83:0/1521456792 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9fcc0be0e0 con 0x7f9fdc108540 2024-04-23T14:48:15.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:15 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2270384379' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:15.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:15 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3712277943' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:15.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.995+0000 7f9fdbe00640 1 -- 172.21.15.83:0/1521456792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9fdc062c00 con 0x7f9fdc108540 2024-04-23T14:48:15.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.996+0000 7f9fe0a00640 1 -- 172.21.15.83:0/1521456792 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f9fcc0bd9e0 con 0x7f9fdc108540 2024-04-23T14:48:15.998 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-23T14:48:16.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.999+0000 7f9fdbe00640 1 -- 172.21.15.83:0/1521456792 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9fc00a3c30 msgr2=0x7f9fc00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:16.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.999+0000 7f9fdbe00640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9fc00a3c30 0x7f9fc00a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc19fd10 tx=0x7f9fc4005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:16.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.999+0000 7f9fdbe00640 1 -- 172.21.15.83:0/1521456792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fdc108540 msgr2=0x7f9fdc19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:16.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.999+0000 7f9fdbe00640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fdc108540 0x7f9fdc19e800 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009bf0 tx=0x7f9fcc031ee0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.999+0000 7f9fdbe00640 1 -- 172.21.15.83:0/1521456792 shutdown_connections 2024-04-23T14:48:16.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.999+0000 7f9fdbe00640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f9fc00a3c30 0x7f9fc00a60f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.999+0000 7f9fdbe00640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fdc10a0f0 0x7f9fdc1a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.999+0000 7f9fdbe00640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fdc109730 0x7f9fdc19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.999+0000 7f9fdbe00640 1 --2- 172.21.15.83:0/1521456792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fdc108540 0x7f9fdc19e800 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.999+0000 7f9fdbe00640 1 -- 172.21.15.83:0/1521456792 >> 172.21.15.83:0/1521456792 conn(0x7f9fdc0774d0 msgr2=0x7f9fdc10ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.999+0000 7f9fdbe00640 1 -- 172.21.15.83:0/1521456792 shutdown_connections 2024-04-23T14:48:16.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:15.999+0000 7f9fdbe00640 1 -- 172.21.15.83:0/1521456792 wait complete. 2024-04-23T14:48:16.020 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:48:16.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.220+0000 7f7eef600640 1 -- 172.21.15.83:0/250617777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee8105450 msgr2=0x7f7ee81058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.220+0000 7f7eef600640 1 --2- 172.21.15.83:0/250617777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee8105450 0x7f7ee81058d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009900 tx=0x7f7ed002a410 comp rx=0 tx=0).stop 2024-04-23T14:48:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.220+0000 7f7eef600640 1 -- 172.21.15.83:0/250617777 shutdown_connections 2024-04-23T14:48:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.220+0000 7f7eef600640 1 --2- 172.21.15.83:0/250617777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ee8105e10 0x7f7ee810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.220+0000 7f7eef600640 1 --2- 172.21.15.83:0/250617777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee8105450 0x7f7ee81058d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.220+0000 7f7eef600640 1 --2- 172.21.15.83:0/250617777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8104260 0x7f7ee8104680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.220+0000 7f7eef600640 1 -- 172.21.15.83:0/250617777 >> 172.21.15.83:0/250617777 conn(0x7f7ee80ff7f0 msgr2=0x7f7ee8101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.220+0000 7f7eef600640 1 -- 172.21.15.83:0/250617777 shutdown_connections 2024-04-23T14:48:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.220+0000 7f7eef600640 1 -- 172.21.15.83:0/250617777 wait complete. 2024-04-23T14:48:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.221+0000 7f7eef600640 1 Processor -- start 2024-04-23T14:48:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.221+0000 7f7eef600640 1 -- start start 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.221+0000 7f7eef600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ee8104260 0x7f7ee819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.221+0000 7f7eef600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee8105450 0x7f7ee819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.221+0000 7f7eef600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8105e10 0x7f7ee81a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.221+0000 7f7eef600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee810ff40 con 0x7f7ee8105450 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.221+0000 7f7eef600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee810fdc0 con 0x7f7ee8105e10 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.221+0000 7f7eef600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee81100c0 con 0x7f7ee8104260 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.222+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee8105450 0x7f7ee819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.222+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee8105450 0x7f7ee819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47944/0 (socket says 172.21.15.83:47944) 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.222+0000 7f7edfe00640 1 -- 172.21.15.83:0/1659349884 learned_addr learned my addr 172.21.15.83:0/1659349884 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.222+0000 7f7eecc00640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ee8104260 0x7f7ee819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.222+0000 7f7eed600640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8105e10 0x7f7ee81a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.222+0000 7f7edfe00640 1 -- 172.21.15.83:0/1659349884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ee8104260 msgr2=0x7f7ee819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.222+0000 7f7edfe00640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ee8104260 0x7f7ee819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.222+0000 7f7edfe00640 1 -- 172.21.15.83:0/1659349884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8105e10 msgr2=0x7f7ee81a1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.222+0000 7f7edfe00640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8105e10 0x7f7ee81a1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.222+0000 7f7edfe00640 1 -- 172.21.15.83:0/1659349884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee81a2180 con 0x7f7ee8105450 2024-04-23T14:48:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.222+0000 7f7edfe00640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee8105450 0x7f7ee819aaa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0031440 tx=0x7f7ed0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.223+0000 7f7ee6000640 1 -- 172.21.15.83:0/1659349884 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed003f030 con 0x7f7ee8105450 2024-04-23T14:48:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.223+0000 7f7ee6000640 1 -- 172.21.15.83:0/1659349884 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ed0003ce0 con 0x7f7ee8105450 2024-04-23T14:48:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.223+0000 7f7eef600640 1 -- 172.21.15.83:0/1659349884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee81a2410 con 0x7f7ee8105450 2024-04-23T14:48:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.223+0000 7f7ee6000640 1 -- 172.21.15.83:0/1659349884 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0035070 con 0x7f7ee8105450 2024-04-23T14:48:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.223+0000 7f7eef600640 1 -- 172.21.15.83:0/1659349884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee81a2950 con 0x7f7ee8105450 2024-04-23T14:48:16.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.224+0000 7f7eef600640 1 -- 172.21.15.83:0/1659349884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee810b520 con 0x7f7ee8105450 2024-04-23T14:48:16.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.226+0000 7f7ee6000640 1 -- 172.21.15.83:0/1659349884 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7ed0042030 con 0x7f7ee8105450 2024-04-23T14:48:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.227+0000 7f7ee6000640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7ecc0a3c30 0x7f7ecc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.227+0000 7f7ee6000640 1 -- 172.21.15.83:0/1659349884 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f7ed00f0810 con 0x7f7ee8105450 2024-04-23T14:48:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.228+0000 7f7eecc00640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7ecc0a3c30 0x7f7ecc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.228+0000 7f7eecc00640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7ecc0a3c30 0x7f7ecc0a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80095f0 tx=0x7f7ed8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.232+0000 7f7ee6000640 1 -- 172.21.15.83:0/1659349884 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ed00bcba0 con 0x7f7ee8105450 2024-04-23T14:48:16.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.461+0000 7f7eef600640 1 -- 172.21.15.83:0/1659349884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ee8062c00 con 0x7f7ee8105450 2024-04-23T14:48:16.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.463+0000 7f7ee6000640 1 -- 172.21.15.83:0/1659349884 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11863 (secure 0 0 0) 0x7f7ed00c0ea0 con 0x7f7ee8105450 2024-04-23T14:48:16.464 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:16.465 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:46:05.301806+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:48:16.465 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 23 2024-04-23T14:48:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.465+0000 7f7eef600640 1 -- 172.21.15.83:0/1659349884 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7ecc0a3c30 msgr2=0x7f7ecc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.465+0000 7f7eef600640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7ecc0a3c30 0x7f7ecc0a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80095f0 tx=0x7f7ed8005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.465+0000 7f7eef600640 1 -- 172.21.15.83:0/1659349884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee8105450 msgr2=0x7f7ee819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.465+0000 7f7eef600640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee8105450 0x7f7ee819aaa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0031440 tx=0x7f7ed0032ce0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.465+0000 7f7eef600640 1 -- 172.21.15.83:0/1659349884 shutdown_connections 2024-04-23T14:48:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.465+0000 7f7eef600640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f7ecc0a3c30 0x7f7ecc0a60f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.465+0000 7f7eef600640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ee8105e10 0x7f7ee81a1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.465+0000 7f7eef600640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ee8105450 0x7f7ee819aaa0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.465+0000 7f7eef600640 1 --2- 172.21.15.83:0/1659349884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ee8104260 0x7f7ee819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.465+0000 7f7eef600640 1 -- 172.21.15.83:0/1659349884 >> 172.21.15.83:0/1659349884 conn(0x7f7ee80ff7f0 msgr2=0x7f7ee81086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.465+0000 7f7eef600640 1 -- 172.21.15.83:0/1659349884 shutdown_connections 2024-04-23T14:48:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.465+0000 7f7eef600640 1 -- 172.21.15.83:0/1659349884 wait complete. 2024-04-23T14:48:16.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:48:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.674+0000 7f22bbe00640 1 -- 172.21.15.83:0/869036393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc10a030 msgr2=0x7f22bc1108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.674+0000 7f22bbe00640 1 --2- 172.21.15.83:0/869036393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc10a030 0x7f22bc1108c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009900 tx=0x7f22b402a410 comp rx=0 tx=0).stop 2024-04-23T14:48:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.674+0000 7f22bbe00640 1 -- 172.21.15.83:0/869036393 shutdown_connections 2024-04-23T14:48:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.674+0000 7f22bbe00640 1 --2- 172.21.15.83:0/869036393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc10a030 0x7f22bc1108c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.674+0000 7f22bbe00640 1 --2- 172.21.15.83:0/869036393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22bc109670 0x7f22bc109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.674+0000 7f22bbe00640 1 --2- 172.21.15.83:0/869036393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22bc108480 0x7f22bc1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.674+0000 7f22bbe00640 1 -- 172.21.15.83:0/869036393 >> 172.21.15.83:0/869036393 conn(0x7f22bc077440 msgr2=0x7f22bc079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.674+0000 7f22bbe00640 1 -- 172.21.15.83:0/869036393 shutdown_connections 2024-04-23T14:48:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.674+0000 7f22bbe00640 1 -- 172.21.15.83:0/869036393 wait complete. 2024-04-23T14:48:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.675+0000 7f22bbe00640 1 Processor -- start 2024-04-23T14:48:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.675+0000 7f22bbe00640 1 -- start start 2024-04-23T14:48:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.675+0000 7f22bbe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22bc108480 0x7f22bc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.675+0000 7f22bbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc109670 0x7f22bc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.675+0000 7f22bbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22bc10a030 0x7f22bc1a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.675+0000 7f22bbe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc114160 con 0x7f22bc109670 2024-04-23T14:48:16.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.675+0000 7f22bbe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc113fe0 con 0x7f22bc10a030 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.675+0000 7f22bbe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc1142e0 con 0x7f22bc108480 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.675+0000 7f22ba000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc109670 0x7f22bc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22ba000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc109670 0x7f22bc19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47962/0 (socket says 172.21.15.83:47962) 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22ba000640 1 -- 172.21.15.83:0/3050105331 learned_addr learned my addr 172.21.15.83:0/3050105331 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22bb400640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22bc10a030 0x7f22bc1a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22baa00640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22bc108480 0x7f22bc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22ba000640 1 -- 172.21.15.83:0/3050105331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22bc108480 msgr2=0x7f22bc19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22ba000640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22bc108480 0x7f22bc19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22ba000640 1 -- 172.21.15.83:0/3050105331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22bc10a030 msgr2=0x7f22bc1a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22ba000640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22bc10a030 0x7f22bc1a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22ba000640 1 -- 172.21.15.83:0/3050105331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc1a63a0 con 0x7f22bc109670 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22ba000640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc109670 0x7f22bc19ecc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009d50 tx=0x7f22a400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22b3400640 1 -- 172.21.15.83:0/3050105331 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4003b70 con 0x7f22bc109670 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22b3400640 1 -- 172.21.15.83:0/3050105331 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a4003d10 con 0x7f22bc109670 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.676+0000 7f22b3400640 1 -- 172.21.15.83:0/3050105331 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a4010a20 con 0x7f22bc109670 2024-04-23T14:48:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.677+0000 7f22bbe00640 1 -- 172.21.15.83:0/3050105331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22bc1a6690 con 0x7f22bc109670 2024-04-23T14:48:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.677+0000 7f22bbe00640 1 -- 172.21.15.83:0/3050105331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22bc1a6be0 con 0x7f22bc109670 2024-04-23T14:48:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.678+0000 7f22bbe00640 1 -- 172.21.15.83:0/3050105331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22bc10f740 con 0x7f22bc109670 2024-04-23T14:48:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.680+0000 7f22b3400640 1 -- 172.21.15.83:0/3050105331 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f22a4003ec0 con 0x7f22bc109670 2024-04-23T14:48:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.683+0000 7f22b3400640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f22a00a3bb0 0x7f22a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.683+0000 7f22b3400640 1 -- 172.21.15.83:0/3050105331 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f22a40d0a70 con 0x7f22bc109670 2024-04-23T14:48:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.683+0000 7f22baa00640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f22a00a3bb0 0x7f22a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:16.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.684+0000 7f22baa00640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f22a00a3bb0 0x7f22a00a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f22bc1094d0 tx=0x7f22ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.687+0000 7f22b3400640 1 -- 172.21.15.83:0/3050105331 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22a409d450 con 0x7f22bc109670 2024-04-23T14:48:16.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:16 smithi203 ceph-mon[32800]: pgmap v387: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:16.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:16 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1265586090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:16.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:16 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1521456792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:16.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:16 smithi083 ceph-mon[27867]: pgmap v387: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:16.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:16 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1265586090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:16.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:16 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1521456792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:16.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:16 smithi173 ceph-mon[32443]: pgmap v387: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:16.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:16 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1265586090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:16.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:16 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1521456792' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:16.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.889+0000 7f22bbe00640 1 -- 172.21.15.83:0/3050105331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22bc062c00 con 0x7f22bc109670 2024-04-23T14:48:16.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.893+0000 7f22b3400640 1 -- 172.21.15.83:0/3050105331 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22699 (secure 0 0 0) 0x7f22a409cd50 con 0x7f22bc109670 2024-04-23T14:48:16.895 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:16.895 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":88,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:46:58.351541+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:48:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.896+0000 7f22bbe00640 1 -- 172.21.15.83:0/3050105331 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f22a00a3bb0 msgr2=0x7f22a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.896+0000 7f22bbe00640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f22a00a3bb0 0x7f22a00a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f22bc1094d0 tx=0x7f22ac005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.896+0000 7f22bbe00640 1 -- 172.21.15.83:0/3050105331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc109670 msgr2=0x7f22bc19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:16.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.896+0000 7f22bbe00640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc109670 0x7f22bc19ecc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f22a4009d50 tx=0x7f22a400c6a0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.896+0000 7f22bbe00640 1 -- 172.21.15.83:0/3050105331 shutdown_connections 2024-04-23T14:48:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.896+0000 7f22bbe00640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f22a00a3bb0 0x7f22a00a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.896+0000 7f22bbe00640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f22bc10a030 0x7f22bc1a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.896+0000 7f22bbe00640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22bc109670 0x7f22bc19ecc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.896+0000 7f22bbe00640 1 --2- 172.21.15.83:0/3050105331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22bc108480 0x7f22bc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:16.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.896+0000 7f22bbe00640 1 -- 172.21.15.83:0/3050105331 >> 172.21.15.83:0/3050105331 conn(0x7f22bc077440 msgr2=0x7f22bc10c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.897+0000 7f22bbe00640 1 -- 172.21.15.83:0/3050105331 shutdown_connections 2024-04-23T14:48:16.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:16.897+0000 7f22bbe00640 1 -- 172.21.15.83:0/3050105331 wait complete. 2024-04-23T14:48:16.917 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T14:45:59.723532+0000', 'modified': '2024-04-23T14:46:05.301806+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24599, 'mds_1': 24542, 'mds_2': 24532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6835/2384637189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6834', 'nonce': 2384637189}, {'type': 'v1', 'addr': '172.21.15.203:6835', 'nonce': 2384637189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6835/2567101723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2567101723}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2567101723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.203:6837/258138306', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6836', 'nonce': 258138306}, {'type': 'v1', 'addr': '172.21.15.203:6837', 'nonce': 258138306}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24532': {'gid': 24532, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.203:6839/2628587422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.203:6838', 'nonce': 2628587422}, {'type': 'v1', 'addr': '172.21.15.203:6839', 'nonce': 2628587422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.173:6837/3951862333', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 3951862333}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 3951862333}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24599': {'gid': 24599, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6839/2485025097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 2485025097}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 2485025097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24599, 'qdb_cluster': [24532, 24542, 24599]} 2024-04-23T14:48:16.918 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-23T14:48:16.918 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T14:48:17.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.117+0000 7f86d7200640 1 -- 172.21.15.83:0/2723076242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86d8105330 msgr2=0x7f86d81057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.117+0000 7f86d7200640 1 --2- 172.21.15.83:0/2723076242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86d8105330 0x7f86d81057b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009900 tx=0x7f86c002a3f0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.118+0000 7f86d7200640 1 -- 172.21.15.83:0/2723076242 shutdown_connections 2024-04-23T14:48:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.118+0000 7f86d7200640 1 --2- 172.21.15.83:0/2723076242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86d8105cf0 0x7f86d810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.118+0000 7f86d7200640 1 --2- 172.21.15.83:0/2723076242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86d8105330 0x7f86d81057b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.118+0000 7f86d7200640 1 --2- 172.21.15.83:0/2723076242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86d8104140 0x7f86d8104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.118+0000 7f86d7200640 1 -- 172.21.15.83:0/2723076242 >> 172.21.15.83:0/2723076242 conn(0x7f86d80ff6b0 msgr2=0x7f86d8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.118+0000 7f86d7200640 1 -- 172.21.15.83:0/2723076242 shutdown_connections 2024-04-23T14:48:17.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.118+0000 7f86d7200640 1 -- 172.21.15.83:0/2723076242 wait complete. 2024-04-23T14:48:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.119+0000 7f86d7200640 1 Processor -- start 2024-04-23T14:48:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.119+0000 7f86d7200640 1 -- start start 2024-04-23T14:48:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.119+0000 7f86d7200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86d8104140 0x7f86d819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.119+0000 7f86d7200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86d8105330 0x7f86d819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.119+0000 7f86d7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86d8105cf0 0x7f86d81a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.119+0000 7f86d7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86d810fe20 con 0x7f86d8105cf0 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.119+0000 7f86d7200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f86d810fca0 con 0x7f86d8105330 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.119+0000 7f86d7200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f86d810ffa0 con 0x7f86d8104140 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.120+0000 7f86d5e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86d8104140 0x7f86d819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.120+0000 7f86d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86d8105cf0 0x7f86d81a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.120+0000 7f86d5e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86d8104140 0x7f86d819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38342/0 (socket says 172.21.15.83:38342) 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.120+0000 7f86d5e00640 1 -- 172.21.15.83:0/4099144752 learned_addr learned my addr 172.21.15.83:0/4099144752 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.120+0000 7f86cd400640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86d8105330 0x7f86d819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.120+0000 7f86d5e00640 1 -- 172.21.15.83:0/4099144752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86d8105330 msgr2=0x7f86d819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.120+0000 7f86d5e00640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86d8105330 0x7f86d819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.120+0000 7f86d5e00640 1 -- 172.21.15.83:0/4099144752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86d8105cf0 msgr2=0x7f86d81a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.120+0000 7f86d5e00640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86d8105cf0 0x7f86d81a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.120+0000 7f86d5e00640 1 -- 172.21.15.83:0/4099144752 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d81a2010 con 0x7f86d8104140 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.120+0000 7f86d5e00640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86d8104140 0x7f86d819a3f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f86c800e9c0 tx=0x7f86c800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.121+0000 7f86cf400640 1 -- 172.21.15.83:0/4099144752 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c800cd80 con 0x7f86d8104140 2024-04-23T14:48:17.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.121+0000 7f86cf400640 1 -- 172.21.15.83:0/4099144752 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86c80035f0 con 0x7f86d8104140 2024-04-23T14:48:17.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.121+0000 7f86cf400640 1 -- 172.21.15.83:0/4099144752 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c8010690 con 0x7f86d8104140 2024-04-23T14:48:17.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.121+0000 7f86d7200640 1 -- 172.21.15.83:0/4099144752 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d81a2300 con 0x7f86d8104140 2024-04-23T14:48:17.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.121+0000 7f86d7200640 1 -- 172.21.15.83:0/4099144752 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d81a27d0 con 0x7f86d8104140 2024-04-23T14:48:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86dffd5740 1 Processor -- start 2024-04-23T14:48:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86dffd5740 1 -- start start 2024-04-23T14:48:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86dffd5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fae260 0x55d499fae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86dffd5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499faec50 0x55d499fb3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86cf400640 1 -- 172.21.15.83:0/4099144752 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f86c80037a0 con 0x7f86d8104140 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86dffd5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 0x55d499fb6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86dffd5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d499f13b10 con 0x55d499fb3dc0 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86dffd5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d499eba2b0 con 0x55d499faec50 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86dffd5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d499f0fad0 con 0x55d499fae260 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86cd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 0x55d499fb6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86d6800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fae260 0x55d499fae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86cd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 0x55d499fb6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48002/0 (socket says 172.21.15.83:48002) 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86cd400640 1 -- 172.21.15.83:0/4180732258 learned_addr learned my addr 172.21.15.83:0/4180732258 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.124+0000 7f86d5e00640 1 --2- 172.21.15.83:0/4180732258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499faec50 0x55d499fb3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.125+0000 7f86d6800640 1 -- 172.21.15.83:0/4180732258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499faec50 msgr2=0x55d499fb3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.125+0000 7f86d6800640 1 --2- 172.21.15.83:0/4180732258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499faec50 0x55d499fb3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.125+0000 7f86d6800640 1 -- 172.21.15.83:0/4180732258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 msgr2=0x55d499fb6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.125+0000 7f86d6800640 1 --2- 172.21.15.83:0/4180732258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 0x55d499fb6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.125+0000 7f86d6800640 1 -- 172.21.15.83:0/4180732258 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d499c262d0 con 0x55d499fae260 2024-04-23T14:48:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.125+0000 7f86cd400640 1 --2- 172.21.15.83:0/4180732258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 0x55d499fb6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:17.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.125+0000 7f86d6800640 1 --2- 172.21.15.83:0/4180732258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fae260 0x55d499fae680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009f30 tx=0x7f86d0010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdfe9e4ab69df95a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.125+0000 7f86c7400640 1 -- 172.21.15.83:0/4180732258 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0016070 con 0x55d499fae260 2024-04-23T14:48:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.126+0000 7f86c7400640 1 -- 172.21.15.83:0/4180732258 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d0004d20 con 0x55d499fae260 2024-04-23T14:48:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.126+0000 7f86c7400640 1 -- 172.21.15.83:0/4180732258 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0012870 con 0x55d499fae260 2024-04-23T14:48:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.126+0000 7f86cf400640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f86b80a3bb0 0x7f86b80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.126+0000 7f86d5e00640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f86b80a3bb0 0x7f86b80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.126+0000 7f86cf400640 1 -- 172.21.15.83:0/4099144752 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f86c80d0840 con 0x7f86d8104140 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.126+0000 7f86dffd5740 1 -- 172.21.15.83:0/4180732258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fae260 msgr2=0x55d499fae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.126+0000 7f86dffd5740 1 --2- 172.21.15.83:0/4180732258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fae260 0x55d499fae680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009f30 tx=0x7f86d0010800 comp rx=0 tx=0).stop 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.126+0000 7f86dffd5740 1 -- 172.21.15.83:0/4180732258 shutdown_connections 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.126+0000 7f86dffd5740 1 --2- 172.21.15.83:0/4180732258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 0x55d499fb6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.126+0000 7f86dffd5740 1 --2- 172.21.15.83:0/4180732258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499faec50 0x55d499fb3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.126+0000 7f86dffd5740 1 --2- 172.21.15.83:0/4180732258 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fae260 0x55d499fae680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.126+0000 7f86dffd5740 1 -- 172.21.15.83:0/4180732258 >> 172.21.15.83:0/4180732258 conn(0x55d499fa98d0 msgr2=0x55d499fa9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.127+0000 7f86dffd5740 1 -- 172.21.15.83:0/4180732258 shutdown_connections 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.127+0000 7f86d5e00640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f86b80a3bb0 0x7f86b80a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86c80994b0 tx=0x7f86c80d6530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.127+0000 7f86dffd5740 1 -- 172.21.15.83:0/4180732258 wait complete. 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.127+0000 7f86dffd5740 1 -- start start 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.127+0000 7f86dffd5740 1 Processor -- start 2024-04-23T14:48:17.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86dffd5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fae260 0x55d499fb8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86dffd5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499faec50 0x55d499fb8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86dffd5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499fb3dc0 0x55d499fbfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86dffd5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d499fb6d70 con 0x55d499faec50 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86dffd5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d499ecd590 con 0x55d499fb3dc0 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86dffd5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d499fb6ef0 con 0x55d499fae260 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86dffd5740 10 client.0 _ll_register_callbacks cb 0x55d499eb03a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86cd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499faec50 0x55d499fb8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86d6800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fae260 0x55d499fb8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86cd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499faec50 0x55d499fb8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48016/0 (socket says 172.21.15.83:48016) 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86d6800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fae260 0x55d499fb8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38348/0 (socket says 172.21.15.83:38348) 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86cd400640 1 -- 172.21.15.83:0/1658751175 learned_addr learned my addr 172.21.15.83:0/1658751175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86d6800640 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499fb3dc0 0x55d499fbfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86cd400640 1 -- 172.21.15.83:0/1658751175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fae260 msgr2=0x55d499fb8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86cd400640 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fae260 0x55d499fb8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86cd400640 1 -- 172.21.15.83:0/1658751175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499fb3dc0 msgr2=0x55d499fbfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86cd400640 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499fb3dc0 0x55d499fbfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86cd400640 1 -- 172.21.15.83:0/1658751175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d499cf06b0 con 0x55d499faec50 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.128+0000 7f86d6800640 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499fb3dc0 0x55d499fbfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.129+0000 7f86cd400640 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499faec50 0x55d499fb8c90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f86c0033000 tx=0x7f86c0004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c9dc4336140758 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.129+0000 7f86bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.129+0000 7f86bfe00640 1 -- 172.21.15.83:0/1658751175 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c0035040 con 0x55d499faec50 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.129+0000 7f86bfe00640 1 -- 172.21.15.83:0/1658751175 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f86c0005510 con 0x55d499faec50 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.129+0000 7f86dffd5740 1 -- 172.21.15.83:0/1658751175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d0004d20 con 0x55d499faec50 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.129+0000 7f86bfe00640 1 -- 172.21.15.83:0/1658751175 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f86c0004770 con 0x55d499faec50 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.129+0000 7f86dffd5740 10 client.15051 fetch_fsmap learned FSMap version 23 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.129+0000 7f86dffd5740 1 -- 172.21.15.83:0/1658751175 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d499fc30b0 con 0x55d499faec50 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.130+0000 7f86bfe00640 1 -- 172.21.15.83:0/1658751175 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d0004d20 con 0x55d499faec50 2024-04-23T14:48:17.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.130+0000 7f86bfe00640 1 -- 172.21.15.83:0/1658751175 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f86c00035f0 con 0x55d499faec50 2024-04-23T14:48:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.130+0000 7f86dffd5740 10 client.15051 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T14:48:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.130+0000 7f86dffd5740 10 client.15051 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T14:48:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.130+0000 7f86dffd5740 20 client.15051 populate_metadata read hostname 'smithi083' 2024-04-23T14:48:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.130+0000 7f86dffd5740 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d499fc56f0 0x55d499fc6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.130+0000 7f86dffd5740 4 client.15051 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:48:17.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.130+0000 7f86dffd5740 1 -- 172.21.15.83:0/1658751175 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d499f21530 con 0x55d499fc56f0 2024-04-23T14:48:17.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.130+0000 7f86d6800640 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d499fc56f0 0x55d499fc6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.131+0000 7f86d6800640 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d499fc56f0 0x55d499fc6b60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ba1f8173ec6cfd3 server_cookie=3d6d065f0bc7786a in_seq=0 out_seq=0 2024-04-23T14:48:17.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.131+0000 7f86bfe00640 10 client.15051 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:48:17.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86bfe00640 1 -- 172.21.15.83:0/1658751175 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d499f21530 con 0x55d499fc56f0 2024-04-23T14:48:17.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86bfe00640 10 client.15051 handle_command_reply: tid=0 2024-04-23T14:48:17.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86dffd5740 1 client.15051 shutdown 2024-04-23T14:48:17.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86dffd5740 1 -- 172.21.15.83:0/1658751175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499faec50 msgr2=0x55d499fb8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86dffd5740 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499faec50 0x55d499fb8c90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f86c0033000 tx=0x7f86c0004440 comp rx=0 tx=0).stop 2024-04-23T14:48:17.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86dffd5740 1 -- 172.21.15.83:0/1658751175 shutdown_connections 2024-04-23T14:48:17.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86dffd5740 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d499fc56f0 0x55d499fc6b60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86dffd5740 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499fb3dc0 0x55d499fbfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86dffd5740 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499faec50 0x55d499fb8c90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86bfe00640 0 client.15051 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:48:17.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86dffd5740 1 --2- 172.21.15.83:0/1658751175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fae260 0x55d499fb8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86dffd5740 1 -- 172.21.15.83:0/1658751175 >> 172.21.15.83:0/1658751175 conn(0x55d499fa98d0 msgr2=0x55d499fa8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86dffd5740 1 -- 172.21.15.83:0/1658751175 shutdown_connections 2024-04-23T14:48:17.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86dffd5740 1 -- 172.21.15.83:0/1658751175 wait complete. 2024-04-23T14:48:17.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.132+0000 7f86dffd5740 20 client.15051 trim_cache size 0 max 16384 2024-04-23T14:48:17.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86dffd5740 1 Processor -- start 2024-04-23T14:48:17.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86dffd5740 1 -- start start 2024-04-23T14:48:17.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86dffd5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499fae260 0x55d499fac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86dffd5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499faec50 0x55d499fbfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86dffd5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 0x55d499faccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86dffd5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d499fc4ea0 con 0x55d499fb3dc0 2024-04-23T14:48:17.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86dffd5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d499fc5570 con 0x55d499fae260 2024-04-23T14:48:17.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86dffd5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d499fc5c40 con 0x55d499faec50 2024-04-23T14:48:17.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 0x55d499faccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 0x55d499faccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48030/0 (socket says 172.21.15.83:48030) 2024-04-23T14:48:17.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86d6800640 1 -- 172.21.15.83:0/1536753053 learned_addr learned my addr 172.21.15.83:0/1536753053 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86d6800640 1 --2- 172.21.15.83:0/1536753053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499fae260 0x55d499fac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.152+0000 7f86cd400640 1 --2- 172.21.15.83:0/1536753053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499faec50 0x55d499fbfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.153+0000 7f86d6800640 1 -- 172.21.15.83:0/1536753053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499faec50 msgr2=0x55d499fbfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.153+0000 7f86d6800640 1 --2- 172.21.15.83:0/1536753053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499faec50 0x55d499fbfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.153+0000 7f86d6800640 1 -- 172.21.15.83:0/1536753053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499fae260 msgr2=0x55d499fac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.153+0000 7f86d6800640 1 --2- 172.21.15.83:0/1536753053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499fae260 0x55d499fac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.153+0000 7f86d6800640 1 -- 172.21.15.83:0/1536753053 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d499faf4e0 con 0x55d499fb3dc0 2024-04-23T14:48:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.153+0000 7f86cd400640 1 --2- 172.21.15.83:0/1536753053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499faec50 0x55d499fbfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.153+0000 7f86d6800640 1 --2- 172.21.15.83:0/1536753053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 0x55d499faccf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f86d0005ca0 tx=0x7f86d0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba181ce021e4a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.153+0000 7f86c6a00640 1 -- 172.21.15.83:0/1536753053 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0016040 con 0x55d499fb3dc0 2024-04-23T14:48:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.154+0000 7f86c6a00640 1 -- 172.21.15.83:0/1536753053 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d0009710 con 0x55d499fb3dc0 2024-04-23T14:48:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.154+0000 7f86c6a00640 1 -- 172.21.15.83:0/1536753053 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0023040 con 0x55d499fb3dc0 2024-04-23T14:48:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.154+0000 7f86dffd5740 1 -- 172.21.15.83:0/1536753053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 msgr2=0x55d499faccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.154+0000 7f86dffd5740 1 --2- 172.21.15.83:0/1536753053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 0x55d499faccf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f86d0005ca0 tx=0x7f86d0000f40 comp rx=0 tx=0).stop 2024-04-23T14:48:17.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.154+0000 7f86dffd5740 1 -- 172.21.15.83:0/1536753053 shutdown_connections 2024-04-23T14:48:17.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.154+0000 7f86dffd5740 1 --2- 172.21.15.83:0/1536753053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fb3dc0 0x55d499faccf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.154+0000 7f86dffd5740 1 --2- 172.21.15.83:0/1536753053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499faec50 0x55d499fbfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.154+0000 7f86dffd5740 1 --2- 172.21.15.83:0/1536753053 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499fae260 0x55d499fac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.154+0000 7f86dffd5740 1 -- 172.21.15.83:0/1536753053 >> 172.21.15.83:0/1536753053 conn(0x55d499fa98d0 msgr2=0x55d499fa9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.154+0000 7f86dffd5740 1 -- 172.21.15.83:0/1536753053 shutdown_connections 2024-04-23T14:48:17.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.154+0000 7f86dffd5740 1 -- 172.21.15.83:0/1536753053 wait complete. 2024-04-23T14:48:17.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.155+0000 7f86dffd5740 1 -- start start 2024-04-23T14:48:17.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.155+0000 7f86dffd5740 1 Processor -- start 2024-04-23T14:48:17.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.155+0000 7f86dffd5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fae260 0x55d499fbbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.155+0000 7f86dffd5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499faec50 0x55d499fbc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.155+0000 7f86dffd5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fb3dc0 0x55d499fbed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.155+0000 7f86dffd5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d499fc69e0 con 0x55d499fae260 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.155+0000 7f86dffd5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d499fc6310 con 0x55d499faec50 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.155+0000 7f86dffd5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d499fc70b0 con 0x55d499fb3dc0 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.155+0000 7f86dffd5740 10 client.0 _ll_register_callbacks cb 0x7f86c0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fae260 0x55d499fbbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fae260 0x55d499fbbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48038/0 (socket says 172.21.15.83:48038) 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86d6800640 1 -- 172.21.15.83:0/408478780 learned_addr learned my addr 172.21.15.83:0/408478780 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86cd400640 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499faec50 0x55d499fbc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86d6800640 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fb3dc0 0x55d499fbed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86d6800640 1 -- 172.21.15.83:0/408478780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fb3dc0 msgr2=0x55d499fbed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86d6800640 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fb3dc0 0x55d499fbed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86d6800640 1 -- 172.21.15.83:0/408478780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499faec50 msgr2=0x55d499fbc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86d6800640 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499faec50 0x55d499fbc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86d6800640 1 -- 172.21.15.83:0/408478780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d499fbf310 con 0x55d499fae260 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86cd400640 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499faec50 0x55d499fbc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86d6800640 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fae260 0x55d499fbbec0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f86d0010d00 tx=0x7f86d0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d4644db4bc5359 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.156+0000 7f86bfe00640 1 -- 172.21.15.83:0/408478780 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d003e060 con 0x55d499fae260 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86bfe00640 1 -- 172.21.15.83:0/408478780 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f86d00051f0 con 0x55d499fae260 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86dffd5740 1 -- 172.21.15.83:0/408478780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d0009710 con 0x55d499fae260 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86bfe00640 1 -- 172.21.15.83:0/408478780 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f86d0016070 con 0x55d499fae260 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86dffd5740 10 client.15057 fetch_fsmap learned FSMap version 23 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86dffd5740 1 -- 172.21.15.83:0/408478780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d49a0342a0 con 0x55d499fae260 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86bfe00640 1 -- 172.21.15.83:0/408478780 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d0009710 con 0x55d499fae260 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86bfe00640 1 -- 172.21.15.83:0/408478780 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f86d0029050 con 0x55d499fae260 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86dffd5740 10 client.15057 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86dffd5740 10 client.15057 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T14:48:17.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86dffd5740 20 client.15057 populate_metadata read hostname 'smithi083' 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86dffd5740 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d49a0387c0 0x55d49a0374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86dffd5740 4 client.15057 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.157+0000 7f86dffd5740 1 -- 172.21.15.83:0/408478780 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "status"}) -- 0x55d499f1b3a0 con 0x55d49a0387c0 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.158+0000 7f86cd400640 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d49a0387c0 0x55d49a0374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.158+0000 7f86cd400640 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d49a0387c0 0x55d49a0374d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d95554eefd244981 server_cookie=ff18a75f5aca17e8 in_seq=0 out_seq=0 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.158+0000 7f86bfe00640 10 client.15057 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.158+0000 7f86bfe00640 1 -- 172.21.15.83:0/408478780 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d499f1b3a0 con 0x55d49a0387c0 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.158+0000 7f86bfe00640 10 client.15057 handle_command_reply: tid=0 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.158+0000 7f86dffd5740 1 client.15057 shutdown 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.158+0000 7f86dffd5740 1 -- 172.21.15.83:0/408478780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fae260 msgr2=0x55d499fbbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.158+0000 7f86dffd5740 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fae260 0x55d499fbbec0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f86d0010d00 tx=0x7f86d0005b50 comp rx=0 tx=0).stop 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86dffd5740 1 -- 172.21.15.83:0/408478780 shutdown_connections 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86dffd5740 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d49a0387c0 0x55d49a0374d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86dffd5740 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d499fb3dc0 0x55d499fbed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86dffd5740 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d499faec50 0x55d499fbc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86bfe00640 0 client.15057 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:48:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86dffd5740 1 --2- 172.21.15.83:0/408478780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d499fae260 0x55d499fbbec0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86dffd5740 1 -- 172.21.15.83:0/408478780 >> 172.21.15.83:0/408478780 conn(0x55d499fa98d0 msgr2=0x55d499fac7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86dffd5740 1 -- 172.21.15.83:0/408478780 shutdown_connections 2024-04-23T14:48:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86dffd5740 1 -- 172.21.15.83:0/408478780 wait complete. 2024-04-23T14:48:17.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86dffd5740 20 client.15057 trim_cache size 0 max 16384 2024-04-23T14:48:17.165 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:48:17.165 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5f4fe708-017e-11ef-bc93-c7b262605968", 2024-04-23T14:48:17.165 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 2, 2024-04-23T14:48:17.165 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24532, 2024-04-23T14:48:17.165 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-23T14:48:17.166 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-23T14:48:17.166 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-23T14:48:17.166 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 132.865015661, 2024-04-23T14:48:17.166 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 23, 2024-04-23T14:48:17.166 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-23T14:48:17.166 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T14:48:17.166 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 154.46056497399999 2024-04-23T14:48:17.166 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:48:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86d7200640 1 -- 172.21.15.83:0/4099144752 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f86b80a3bb0 msgr2=0x7f86b80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86d7200640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f86b80a3bb0 0x7f86b80a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86c80994b0 tx=0x7f86c80d6530 comp rx=0 tx=0).stop 2024-04-23T14:48:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86d7200640 1 -- 172.21.15.83:0/4099144752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86d8104140 msgr2=0x7f86d819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.159+0000 7f86d7200640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86d8104140 0x7f86d819a3f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f86c800e9c0 tx=0x7f86c800ee80 comp rx=0 tx=0).stop 2024-04-23T14:48:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.160+0000 7f86d7200640 1 -- 172.21.15.83:0/4099144752 shutdown_connections 2024-04-23T14:48:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.160+0000 7f86d7200640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f86b80a3bb0 0x7f86b80a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.160+0000 7f86d7200640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86d8105cf0 0x7f86d81a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.160+0000 7f86d7200640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86d8105330 0x7f86d819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.160+0000 7f86d7200640 1 --2- 172.21.15.83:0/4099144752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f86d8104140 0x7f86d819a3f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.160+0000 7f86d7200640 1 -- 172.21.15.83:0/4099144752 >> 172.21.15.83:0/4099144752 conn(0x7f86d80ff6b0 msgr2=0x7f86d8108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.160+0000 7f86d7200640 1 -- 172.21.15.83:0/4099144752 shutdown_connections 2024-04-23T14:48:17.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.160+0000 7f86d7200640 1 -- 172.21.15.83:0/4099144752 wait complete. 2024-04-23T14:48:17.178 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T14:48:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.362+0000 7ff975400640 1 -- 172.21.15.83:0/609863839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff970076040 msgr2=0x7ff970074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.362+0000 7ff975400640 1 --2- 172.21.15.83:0/609863839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff970076040 0x7ff970074490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff960009900 tx=0x7ff96002a3f0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.362+0000 7ff975400640 1 -- 172.21.15.83:0/609863839 shutdown_connections 2024-04-23T14:48:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.362+0000 7ff975400640 1 --2- 172.21.15.83:0/609863839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff970109df0 0x7ff97010c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.362+0000 7ff975400640 1 --2- 172.21.15.83:0/609863839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff970074a60 0x7ff970074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.362+0000 7ff975400640 1 --2- 172.21.15.83:0/609863839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff970076040 0x7ff970074490 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.362+0000 7ff975400640 1 -- 172.21.15.83:0/609863839 >> 172.21.15.83:0/609863839 conn(0x7ff9700fd4a0 msgr2=0x7ff9700ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.362+0000 7ff975400640 1 -- 172.21.15.83:0/609863839 shutdown_connections 2024-04-23T14:48:17.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.362+0000 7ff975400640 1 -- 172.21.15.83:0/609863839 wait complete. 2024-04-23T14:48:17.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.363+0000 7ff975400640 1 Processor -- start 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.363+0000 7ff975400640 1 -- start start 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.363+0000 7ff975400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff970074a60 0x7ff97019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.363+0000 7ff975400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff970076040 0x7ff97019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.363+0000 7ff975400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff970109df0 0x7ff9701a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.363+0000 7ff975400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff97010fd80 con 0x7ff970076040 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.363+0000 7ff975400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff97010fc00 con 0x7ff970109df0 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.363+0000 7ff975400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff97010ff00 con 0x7ff970074a60 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.364+0000 7ff96f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff970076040 0x7ff97019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.364+0000 7ff96f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff970076040 0x7ff97019a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48068/0 (socket says 172.21.15.83:48068) 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.364+0000 7ff96f400640 1 -- 172.21.15.83:0/2258045191 learned_addr learned my addr 172.21.15.83:0/2258045191 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.364+0000 7ff96fe00640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff970074a60 0x7ff97019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.364+0000 7ff974a00640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff970109df0 0x7ff9701a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.364+0000 7ff96f400640 1 -- 172.21.15.83:0/2258045191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff970074a60 msgr2=0x7ff97019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.364+0000 7ff96f400640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff970074a60 0x7ff97019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.364+0000 7ff96f400640 1 -- 172.21.15.83:0/2258045191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff970109df0 msgr2=0x7ff9701a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.364+0000 7ff96f400640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff970109df0 0x7ff9701a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.364+0000 7ff96f400640 1 -- 172.21.15.83:0/2258045191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9701a2040 con 0x7ff970076040 2024-04-23T14:48:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.364+0000 7ff96f400640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff970076040 0x7ff97019a960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff95800e9f0 tx=0x7ff95800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.365+0000 7ff96cc00640 1 -- 172.21.15.83:0/2258045191 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95800cd80 con 0x7ff970076040 2024-04-23T14:48:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.365+0000 7ff96cc00640 1 -- 172.21.15.83:0/2258045191 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff958010910 con 0x7ff970076040 2024-04-23T14:48:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.365+0000 7ff96cc00640 1 -- 172.21.15.83:0/2258045191 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958018990 con 0x7ff970076040 2024-04-23T14:48:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.365+0000 7ff975400640 1 -- 172.21.15.83:0/2258045191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9701a2330 con 0x7ff970076040 2024-04-23T14:48:17.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.365+0000 7ff975400640 1 -- 172.21.15.83:0/2258045191 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9701a28d0 con 0x7ff970076040 2024-04-23T14:48:17.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.369+0000 7ff9797e8740 1 Processor -- start 2024-04-23T14:48:17.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.369+0000 7ff9797e8740 1 -- start start 2024-04-23T14:48:17.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.369+0000 7ff96cc00640 1 -- 172.21.15.83:0/2258045191 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff958010ac0 con 0x7ff970076040 2024-04-23T14:48:17.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.369+0000 7ff9797e8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0ea260 0x555e5d0ea680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.369+0000 7ff9797e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0eac50 0x555e5d0ef750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.369+0000 7ff9797e8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0efdc0 0x555e5d0f2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.369+0000 7ff9797e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555e5d04fb10 con 0x555e5d0eac50 2024-04-23T14:48:17.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.369+0000 7ff9797e8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555e5cff62b0 con 0x555e5d0ea260 2024-04-23T14:48:17.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.369+0000 7ff9797e8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555e5d04bad0 con 0x555e5d0efdc0 2024-04-23T14:48:17.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0eac50 0x555e5d0ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff974a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0ea260 0x555e5d0ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff974a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0ea260 0x555e5d0ea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:57166/0 (socket says 172.21.15.83:57166) 2024-04-23T14:48:17.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0eac50 0x555e5d0ef750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48074/0 (socket says 172.21.15.83:48074) 2024-04-23T14:48:17.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff974a00640 1 -- 172.21.15.83:0/2072298800 learned_addr learned my addr 172.21.15.83:0/2072298800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff96f400640 1 --2- 172.21.15.83:0/2072298800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0efdc0 0x555e5d0f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff96fe00640 1 -- 172.21.15.83:0/2072298800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0efdc0 msgr2=0x555e5d0f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff96fe00640 1 --2- 172.21.15.83:0/2072298800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0efdc0 0x555e5d0f2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff96fe00640 1 -- 172.21.15.83:0/2072298800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0ea260 msgr2=0x555e5d0ea680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff96fe00640 1 --2- 172.21.15.83:0/2072298800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0ea260 0x555e5d0ea680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff96fe00640 1 -- 172.21.15.83:0/2072298800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5cd622d0 con 0x555e5d0eac50 2024-04-23T14:48:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff974a00640 1 --2- 172.21.15.83:0/2072298800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0ea260 0x555e5d0ea680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.370+0000 7ff96fe00640 1 --2- 172.21.15.83:0/2072298800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0eac50 0x555e5d0ef750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff960031440 tx=0x7ff960005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a34ebaf07f3073 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.371+0000 7ff965200640 1 -- 172.21.15.83:0/2072298800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960040070 con 0x555e5d0eac50 2024-04-23T14:48:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.371+0000 7ff96cc00640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff9540a3c30 0x7ff9540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.371+0000 7ff965200640 1 -- 172.21.15.83:0/2072298800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff960004030 con 0x555e5d0eac50 2024-04-23T14:48:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.371+0000 7ff965200640 1 -- 172.21.15.83:0/2072298800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960035040 con 0x555e5d0eac50 2024-04-23T14:48:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.371+0000 7ff96cc00640 1 -- 172.21.15.83:0/2258045191 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff9580d0720 con 0x7ff970076040 2024-04-23T14:48:17.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.371+0000 7ff974a00640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff9540a3c30 0x7ff9540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.372+0000 7ff974a00640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff9540a3c30 0x7ff9540a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x555e5d0eb4b0 tx=0x7ff968009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.372+0000 7ff9797e8740 1 -- 172.21.15.83:0/2072298800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0eac50 msgr2=0x555e5d0ef750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.372+0000 7ff9797e8740 1 --2- 172.21.15.83:0/2072298800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0eac50 0x555e5d0ef750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff960031440 tx=0x7ff960005e00 comp rx=0 tx=0).stop 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.372+0000 7ff9797e8740 1 -- 172.21.15.83:0/2072298800 shutdown_connections 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.372+0000 7ff9797e8740 1 --2- 172.21.15.83:0/2072298800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0efdc0 0x555e5d0f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.372+0000 7ff9797e8740 1 --2- 172.21.15.83:0/2072298800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0eac50 0x555e5d0ef750 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.372+0000 7ff9797e8740 1 --2- 172.21.15.83:0/2072298800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0ea260 0x555e5d0ea680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.372+0000 7ff9797e8740 1 -- 172.21.15.83:0/2072298800 >> 172.21.15.83:0/2072298800 conn(0x555e5d0e58d0 msgr2=0x555e5d0e5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.372+0000 7ff9797e8740 1 -- 172.21.15.83:0/2072298800 shutdown_connections 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.372+0000 7ff9797e8740 1 -- 172.21.15.83:0/2072298800 wait complete. 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.372+0000 7ff9797e8740 1 -- start start 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.372+0000 7ff9797e8740 1 Processor -- start 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff9797e8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff9797e8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff9797e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff9797e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555e5d0f2d10 con 0x555e5d0efdc0 2024-04-23T14:48:17.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff9797e8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555e5d009590 con 0x555e5d0eac50 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff9797e8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555e5d0f2e90 con 0x555e5d0ea260 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff9797e8740 10 client.0 _ll_register_callbacks cb 0x555e5cfec3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff96f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff96f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0f4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:57182/0 (socket says 172.21.15.83:57182) 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff974a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff96f400640 1 -- 172.21.15.83:0/1985882909 learned_addr learned my addr 172.21.15.83:0/1985882909 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff96fe00640 1 --2- 172.21.15.83:0/1985882909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff96f400640 1 -- 172.21.15.83:0/1985882909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 msgr2=0x555e5d0f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff96f400640 1 --2- 172.21.15.83:0/1985882909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff96f400640 1 -- 172.21.15.83:0/1985882909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 msgr2=0x555e5d0fbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff96f400640 1 --2- 172.21.15.83:0/1985882909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0fbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.373+0000 7ff96f400640 1 -- 172.21.15.83:0/1985882909 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5ce2c6b0 con 0x555e5d0eac50 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.374+0000 7ff96f400640 1 --2- 172.21.15.83:0/1985882909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0f4b90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff958098e60 tx=0x7ff958098d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f8ae58731a032b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.374+0000 7ff95d600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.374+0000 7ff95d600640 1 -- 172.21.15.83:0/1985882909 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9580bbdc0 con 0x555e5d0eac50 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.374+0000 7ff95d600640 1 -- 172.21.15.83:0/1985882909 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff95801ab70 con 0x555e5d0eac50 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.374+0000 7ff9797e8740 1 -- 172.21.15.83:0/1985882909 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff960004030 con 0x555e5d0eac50 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.374+0000 7ff95d600640 1 -- 172.21.15.83:0/1985882909 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff95809d100 con 0x555e5d0eac50 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.374+0000 7ff9797e8740 10 client.25169 fetch_fsmap learned FSMap version 23 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.374+0000 7ff9797e8740 1 -- 172.21.15.83:0/1985882909 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555e5d0fefb0 con 0x555e5d0eac50 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.374+0000 7ff95d600640 1 -- 172.21.15.83:0/1985882909 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff960004030 con 0x555e5d0eac50 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.375+0000 7ff95d600640 1 -- 172.21.15.83:0/1985882909 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff95809be50 con 0x555e5d0eac50 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.375+0000 7ff9797e8740 10 client.25169 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.375+0000 7ff9797e8740 10 client.25169 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.375+0000 7ff9797e8740 20 client.25169 populate_metadata read hostname 'smithi083' 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.375+0000 7ff9797e8740 1 --2- 172.21.15.83:0/1985882909 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555e5d1015b0 0x555e5d102a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.375+0000 7ff9797e8740 4 client.25169 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:48:17.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.375+0000 7ff9797e8740 1 -- 172.21.15.83:0/1985882909 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e5d05d530 con 0x555e5d1015b0 2024-04-23T14:48:17.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.375+0000 7ff96fe00640 1 --2- 172.21.15.83:0/1985882909 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555e5d1015b0 0x555e5d102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.376+0000 7ff96fe00640 1 --2- 172.21.15.83:0/1985882909 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555e5d1015b0 0x555e5d102a20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85705c8b11d83599 server_cookie=9da895a85752e978 in_seq=0 out_seq=0 2024-04-23T14:48:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.376+0000 7ff95d600640 10 client.25169 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:48:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.376+0000 7ff95d600640 1 -- 172.21.15.83:0/1985882909 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e5d05d530 con 0x555e5d1015b0 2024-04-23T14:48:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff95d600640 10 client.25169 handle_command_reply: tid=0 2024-04-23T14:48:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff9797e8740 1 client.25169 shutdown 2024-04-23T14:48:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff9797e8740 1 -- 172.21.15.83:0/1985882909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 msgr2=0x555e5d0f4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff9797e8740 1 --2- 172.21.15.83:0/1985882909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0f4b90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff958098e60 tx=0x7ff958098d90 comp rx=0 tx=0).stop 2024-04-23T14:48:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff9797e8740 1 -- 172.21.15.83:0/1985882909 shutdown_connections 2024-04-23T14:48:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff9797e8740 1 --2- 172.21.15.83:0/1985882909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff9797e8740 1 --2- 172.21.15.83:0/1985882909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0f4b90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff9797e8740 1 --2- 172.21.15.83:0/1985882909 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555e5d1015b0 0x555e5d102a20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff9797e8740 1 --2- 172.21.15.83:0/1985882909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff95d600640 0 client.25169 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:48:17.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff9797e8740 1 -- 172.21.15.83:0/1985882909 >> 172.21.15.83:0/1985882909 conn(0x555e5d0e58d0 msgr2=0x555e5d0e4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff9797e8740 1 -- 172.21.15.83:0/1985882909 shutdown_connections 2024-04-23T14:48:17.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff9797e8740 1 -- 172.21.15.83:0/1985882909 wait complete. 2024-04-23T14:48:17.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.377+0000 7ff9797e8740 20 client.25169 trim_cache size 0 max 16384 2024-04-23T14:48:17.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.398+0000 7ff9797e8740 1 Processor -- start 2024-04-23T14:48:17.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.398+0000 7ff9797e8740 1 -- start start 2024-04-23T14:48:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff9797e8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff9797e8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff9797e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff9797e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555e5d100d60 con 0x555e5d0efdc0 2024-04-23T14:48:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff9797e8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555e5d101430 con 0x555e5d0eac50 2024-04-23T14:48:17.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff9797e8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555e5d101b00 con 0x555e5d0ea260 2024-04-23T14:48:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff974a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff974a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0f3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48098/0 (socket says 172.21.15.83:48098) 2024-04-23T14:48:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff974a00640 1 -- 172.21.15.83:0/2266387287 learned_addr learned my addr 172.21.15.83:0/2266387287 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff96f400640 1 --2- 172.21.15.83:0/2266387287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff96fe00640 1 -- 172.21.15.83:0/2266387287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 msgr2=0x555e5d0fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff96fe00640 1 --2- 172.21.15.83:0/2266387287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0fbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff96fe00640 1 -- 172.21.15.83:0/2266387287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 msgr2=0x555e5d0f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff96fe00640 1 --2- 172.21.15.83:0/2266387287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0f3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff96fe00640 1 -- 172.21.15.83:0/2266387287 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5d0e5700 con 0x555e5d0ea260 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.399+0000 7ff974a00640 1 --2- 172.21.15.83:0/2266387287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.400+0000 7ff96fe00640 1 --2- 172.21.15.83:0/2266387287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f3980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x555e5d0e67a0 tx=0x7ff960000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd2861e4f7c7de20 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.400+0000 7ff95fe00640 1 -- 172.21.15.83:0/2266387287 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960035040 con 0x555e5d0ea260 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.400+0000 7ff95fe00640 1 -- 172.21.15.83:0/2266387287 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff960004040 con 0x555e5d0ea260 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.400+0000 7ff95fe00640 1 -- 172.21.15.83:0/2266387287 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960040040 con 0x555e5d0ea260 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.401+0000 7ff9797e8740 1 -- 172.21.15.83:0/2266387287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 msgr2=0x555e5d0f3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.401+0000 7ff9797e8740 1 --2- 172.21.15.83:0/2266387287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f3980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x555e5d0e67a0 tx=0x7ff960000bc0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.401+0000 7ff9797e8740 1 -- 172.21.15.83:0/2266387287 shutdown_connections 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.401+0000 7ff9797e8740 1 --2- 172.21.15.83:0/2266387287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.401+0000 7ff9797e8740 1 --2- 172.21.15.83:0/2266387287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.401+0000 7ff9797e8740 1 --2- 172.21.15.83:0/2266387287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f3980 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.401+0000 7ff9797e8740 1 -- 172.21.15.83:0/2266387287 >> 172.21.15.83:0/2266387287 conn(0x555e5d0e58d0 msgr2=0x555e5d0f4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.401+0000 7ff9797e8740 1 -- 172.21.15.83:0/2266387287 shutdown_connections 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.401+0000 7ff9797e8740 1 -- 172.21.15.83:0/2266387287 wait complete. 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.401+0000 7ff9797e8740 1 -- start start 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff9797e8740 1 Processor -- start 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff9797e8740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff9797e8740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0f8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff9797e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff9797e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555e5d1028a0 con 0x555e5d0efdc0 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff9797e8740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555e5d1021d0 con 0x555e5d0eac50 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff9797e8740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555e5d102f70 con 0x555e5d0ea260 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff9797e8740 10 client.0 _ll_register_callbacks cb 0x7ff958009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff974a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0fb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff974a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0fb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48100/0 (socket says 172.21.15.83:48100) 2024-04-23T14:48:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff974a00640 1 -- 172.21.15.83:0/1457562250 learned_addr learned my addr 172.21.15.83:0/1457562250 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff96fe00640 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.402+0000 7ff96f400640 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff96f400640 1 -- 172.21.15.83:0/1457562250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 msgr2=0x555e5d0f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff96f400640 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff96f400640 1 -- 172.21.15.83:0/1457562250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 msgr2=0x555e5d0fb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff96f400640 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0fb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff96f400640 1 -- 172.21.15.83:0/1457562250 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e5d0fb6b0 con 0x555e5d0eac50 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff974a00640 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff96f400640 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0f8380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff95800e9c0 tx=0x7ff95800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb6bbb9e2e5e6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff95d600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff95d600640 1 -- 172.21.15.83:0/1457562250 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff958087430 con 0x555e5d0eac50 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff9797e8740 1 -- 172.21.15.83:0/1457562250 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff960004040 con 0x555e5d0eac50 2024-04-23T14:48:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff95d600640 1 -- 172.21.15.83:0/1457562250 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ff958018b30 con 0x555e5d0eac50 2024-04-23T14:48:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff95d600640 1 -- 172.21.15.83:0/1457562250 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff9580a0db0 con 0x555e5d0eac50 2024-04-23T14:48:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff9797e8740 10 client.25175 fetch_fsmap learned FSMap version 23 2024-04-23T14:48:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.403+0000 7ff9797e8740 1 -- 172.21.15.83:0/1457562250 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x555e5d167e70 con 0x555e5d0eac50 2024-04-23T14:48:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.404+0000 7ff95d600640 1 -- 172.21.15.83:0/1457562250 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff960004040 con 0x555e5d0eac50 2024-04-23T14:48:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.404+0000 7ff95d600640 1 -- 172.21.15.83:0/1457562250 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7ff9580987e0 con 0x555e5d0eac50 2024-04-23T14:48:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.404+0000 7ff9797e8740 10 client.25175 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T14:48:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.404+0000 7ff9797e8740 10 client.25175 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T14:48:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.404+0000 7ff9797e8740 20 client.25175 populate_metadata read hostname 'smithi083' 2024-04-23T14:48:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.404+0000 7ff9797e8740 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555e5d16ca10 0x555e5d16b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.404+0000 7ff9797e8740 4 client.25175 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:48:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.404+0000 7ff9797e8740 1 -- 172.21.15.83:0/1457562250 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "status"}) -- 0x555e5d0573a0 con 0x555e5d16ca10 2024-04-23T14:48:17.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.405+0000 7ff96fe00640 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555e5d16ca10 0x555e5d16b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.405+0000 7ff95d600640 10 client.25175 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.405+0000 7ff96fe00640 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555e5d16ca10 0x555e5d16b720 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb5594c3f1e01a01 server_cookie=5d8b1af6b90396f3 in_seq=0 out_seq=0 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff95d600640 1 -- 172.21.15.83:0/1457562250 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555e5d0573a0 con 0x555e5d16ca10 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff95d600640 10 client.25175 handle_command_reply: tid=0 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff9797e8740 1 client.25175 shutdown 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff9797e8740 1 -- 172.21.15.83:0/1457562250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 msgr2=0x555e5d0f8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff9797e8740 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0f8380 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff95800e9c0 tx=0x7ff95800cf90 comp rx=0 tx=0).stop 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff9797e8740 1 -- 172.21.15.83:0/1457562250 shutdown_connections 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff9797e8740 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e5d0efdc0 0x555e5d0fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff9797e8740 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555e5d0eac50 0x555e5d0f8380 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff9797e8740 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555e5d16ca10 0x555e5d16b720 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff9797e8740 1 --2- 172.21.15.83:0/1457562250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555e5d0ea260 0x555e5d0f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff95d600640 0 client.25175 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:48:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff9797e8740 1 -- 172.21.15.83:0/1457562250 >> 172.21.15.83:0/1457562250 conn(0x555e5d0e58d0 msgr2=0x555e5d0f3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff9797e8740 1 -- 172.21.15.83:0/1457562250 shutdown_connections 2024-04-23T14:48:17.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff9797e8740 1 -- 172.21.15.83:0/1457562250 wait complete. 2024-04-23T14:48:17.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.406+0000 7ff9797e8740 20 client.25175 trim_cache size 0 max 16384 2024-04-23T14:48:17.413 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:48:17.413 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5f4fe708-017e-11ef-bc93-c7b262605968", 2024-04-23T14:48:17.413 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 1, 2024-04-23T14:48:17.413 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24542, 2024-04-23T14:48:17.413 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-23T14:48:17.413 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-23T14:48:17.413 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-23T14:48:17.413 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 134.12656728100001, 2024-04-23T14:48:17.414 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 23, 2024-04-23T14:48:17.414 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-23T14:48:17.414 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T14:48:17.414 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 157.65046866200001 2024-04-23T14:48:17.414 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:48:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.407+0000 7ff975400640 1 -- 172.21.15.83:0/2258045191 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff9540a3c30 msgr2=0x7ff9540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.407+0000 7ff975400640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff9540a3c30 0x7ff9540a60f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x555e5d0eb4b0 tx=0x7ff968009500 comp rx=0 tx=0).stop 2024-04-23T14:48:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.407+0000 7ff975400640 1 -- 172.21.15.83:0/2258045191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff970076040 msgr2=0x7ff97019a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.407+0000 7ff975400640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff970076040 0x7ff97019a960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff95800e9f0 tx=0x7ff95800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.407+0000 7ff975400640 1 -- 172.21.15.83:0/2258045191 shutdown_connections 2024-04-23T14:48:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.407+0000 7ff975400640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7ff9540a3c30 0x7ff9540a60f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.407+0000 7ff975400640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff970109df0 0x7ff9701a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.407+0000 7ff975400640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff970076040 0x7ff97019a960 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.407+0000 7ff975400640 1 --2- 172.21.15.83:0/2258045191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff970074a60 0x7ff97019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.407+0000 7ff975400640 1 -- 172.21.15.83:0/2258045191 >> 172.21.15.83:0/2258045191 conn(0x7ff9700fd4a0 msgr2=0x7ff970105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.407+0000 7ff975400640 1 -- 172.21.15.83:0/2258045191 shutdown_connections 2024-04-23T14:48:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.407+0000 7ff975400640 1 -- 172.21.15.83:0/2258045191 wait complete. 2024-04-23T14:48:17.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T14:48:17.486 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:17 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1659349884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:48:17.486 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:17 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3050105331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:17.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.615+0000 7f1483e00640 1 -- 172.21.15.83:0/1070281101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1484105330 msgr2=0x7f14841057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.615+0000 7f1483e00640 1 --2- 172.21.15.83:0/1070281101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1484105330 0x7f14841057b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1470009900 tx=0x7f147002a3f0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.615+0000 7f1483e00640 1 -- 172.21.15.83:0/1070281101 shutdown_connections 2024-04-23T14:48:17.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.615+0000 7f1483e00640 1 --2- 172.21.15.83:0/1070281101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1484105cf0 0x7f148410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.615+0000 7f1483e00640 1 --2- 172.21.15.83:0/1070281101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1484105330 0x7f14841057b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.615+0000 7f1483e00640 1 --2- 172.21.15.83:0/1070281101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1484104140 0x7f1484104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.615+0000 7f1483e00640 1 -- 172.21.15.83:0/1070281101 >> 172.21.15.83:0/1070281101 conn(0x7f14840ff6b0 msgr2=0x7f1484101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.615+0000 7f1483e00640 1 -- 172.21.15.83:0/1070281101 shutdown_connections 2024-04-23T14:48:17.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.615+0000 7f1483e00640 1 -- 172.21.15.83:0/1070281101 wait complete. 2024-04-23T14:48:17.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.616+0000 7f1483e00640 1 Processor -- start 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.616+0000 7f1483e00640 1 -- start start 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1483e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1484104140 0x7f148419a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1483e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1484105330 0x7f148419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1483e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1484105cf0 0x7f14841a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1483e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f148410fe20 con 0x7f1484105330 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1483e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f148410fca0 con 0x7f1484105cf0 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1483e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f148410ffa0 con 0x7f1484104140 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1482000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1484105330 0x7f148419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1483400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1484105cf0 0x7f14841a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1482000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1484105330 0x7f148419a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48108/0 (socket says 172.21.15.83:48108) 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1482000640 1 -- 172.21.15.83:0/3397073005 learned_addr learned my addr 172.21.15.83:0/3397073005 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1482a00640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1484104140 0x7f148419a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1482000640 1 -- 172.21.15.83:0/3397073005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1484104140 msgr2=0x7f148419a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1482000640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1484104140 0x7f148419a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1482000640 1 -- 172.21.15.83:0/3397073005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1484105cf0 msgr2=0x7f14841a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1482000640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1484105cf0 0x7f14841a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.617+0000 7f1482000640 1 -- 172.21.15.83:0/3397073005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14841a2010 con 0x7f1484105330 2024-04-23T14:48:17.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.618+0000 7f1482000640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1484105330 0x7f148419a930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1470031440 tx=0x7f1470032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.618+0000 7f147b400640 1 -- 172.21.15.83:0/3397073005 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147003f030 con 0x7f1484105330 2024-04-23T14:48:17.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.618+0000 7f1483e00640 1 -- 172.21.15.83:0/3397073005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14841a22a0 con 0x7f1484105330 2024-04-23T14:48:17.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.618+0000 7f147b400640 1 -- 172.21.15.83:0/3397073005 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1470003ce0 con 0x7f1484105330 2024-04-23T14:48:17.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.618+0000 7f147b400640 1 -- 172.21.15.83:0/3397073005 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470035070 con 0x7f1484105330 2024-04-23T14:48:17.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.618+0000 7f1483e00640 1 -- 172.21.15.83:0/3397073005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14841a2790 con 0x7f1484105330 2024-04-23T14:48:17.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.621+0000 7f148cb65740 1 Processor -- start 2024-04-23T14:48:17.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.621+0000 7f148cb65740 1 -- start start 2024-04-23T14:48:17.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f148cb65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba7346b260 0x55ba7346b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f148cb65740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba73470750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f148cb65740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba73470dc0 0x55ba73473250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f148cb65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba733d0b10 con 0x55ba7346b260 2024-04-23T14:48:17.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f148cb65740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ba733772b0 con 0x55ba73470dc0 2024-04-23T14:48:17.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f148cb65740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ba733ccad0 con 0x55ba7346bc50 2024-04-23T14:48:17.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f147b400640 1 -- 172.21.15.83:0/3397073005 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1470042030 con 0x7f1484105330 2024-04-23T14:48:17.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f1482a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba73470750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f1483400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba7346b260 0x55ba7346b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f1482a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba73470750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38452/0 (socket says 172.21.15.83:38452) 2024-04-23T14:48:17.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f1482a00640 1 -- 172.21.15.83:0/260139649 learned_addr learned my addr 172.21.15.83:0/260139649 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f1482000640 1 --2- 172.21.15.83:0/260139649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba73470dc0 0x55ba73473250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f1482a00640 1 -- 172.21.15.83:0/260139649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba73470dc0 msgr2=0x55ba73473250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f1482a00640 1 --2- 172.21.15.83:0/260139649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba73470dc0 0x55ba73473250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f1482a00640 1 -- 172.21.15.83:0/260139649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba7346b260 msgr2=0x55ba7346b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f1482a00640 1 --2- 172.21.15.83:0/260139649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba7346b260 0x55ba7346b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.622+0000 7f1482a00640 1 -- 172.21.15.83:0/260139649 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba730e32d0 con 0x55ba7346bc50 2024-04-23T14:48:17.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.623+0000 7f1482a00640 1 --2- 172.21.15.83:0/260139649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba73470750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1474009ca0 tx=0x7f1474011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=660e0aacaf870c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.623+0000 7f146fe00640 1 -- 172.21.15.83:0/260139649 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474004220 con 0x55ba7346bc50 2024-04-23T14:48:17.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.623+0000 7f146fe00640 1 -- 172.21.15.83:0/260139649 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14740043c0 con 0x55ba7346bc50 2024-04-23T14:48:17.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.623+0000 7f146fe00640 1 -- 172.21.15.83:0/260139649 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474019930 con 0x55ba7346bc50 2024-04-23T14:48:17.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f147b400640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f14680a3bb0 0x7f14680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f148cb65740 1 -- 172.21.15.83:0/260139649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 msgr2=0x55ba73470750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f148cb65740 1 --2- 172.21.15.83:0/260139649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba73470750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1474009ca0 tx=0x7f1474011810 comp rx=0 tx=0).stop 2024-04-23T14:48:17.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f148cb65740 1 -- 172.21.15.83:0/260139649 shutdown_connections 2024-04-23T14:48:17.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f148cb65740 1 --2- 172.21.15.83:0/260139649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba73470dc0 0x55ba73473250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f148cb65740 1 --2- 172.21.15.83:0/260139649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba73470750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f148cb65740 1 --2- 172.21.15.83:0/260139649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba7346b260 0x55ba7346b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f148cb65740 1 -- 172.21.15.83:0/260139649 >> 172.21.15.83:0/260139649 conn(0x55ba734668d0 msgr2=0x55ba73466cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f147b400640 1 -- 172.21.15.83:0/3397073005 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f14700f0be0 con 0x7f1484105330 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f1483400640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f14680a3bb0 0x7f14680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f148cb65740 1 -- 172.21.15.83:0/260139649 shutdown_connections 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f148cb65740 1 -- 172.21.15.83:0/260139649 wait complete. 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.624+0000 7f148cb65740 1 -- start start 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.625+0000 7f1483400640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f14680a3bb0 0x7f14680a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55ba7346c4b0 tx=0x7f147c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.625+0000 7f148cb65740 1 Processor -- start 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.625+0000 7f148cb65740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346b260 0x55ba734756b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.625+0000 7f148cb65740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba73475bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.625+0000 7f148cb65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 0x55ba7347cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.625+0000 7f148cb65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba73473d70 con 0x55ba73470dc0 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.625+0000 7f148cb65740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ba7338a590 con 0x55ba7346b260 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.625+0000 7f148cb65740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ba73473ef0 con 0x55ba7346bc50 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.625+0000 7f148cb65740 10 client.0 _ll_register_callbacks cb 0x55ba7336d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.625+0000 7f1483400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 0x55ba7347cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.626+0000 7f1483400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 0x55ba7347cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48122/0 (socket says 172.21.15.83:48122) 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.626+0000 7f1483400640 1 -- 172.21.15.83:0/1690364298 learned_addr learned my addr 172.21.15.83:0/1690364298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.626+0000 7f1482000640 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba73475bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.626+0000 7f1482a00640 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346b260 0x55ba734756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.626+0000 7f1483400640 1 -- 172.21.15.83:0/1690364298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 msgr2=0x55ba73475bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.626+0000 7f1483400640 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba73475bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.626+0000 7f1483400640 1 -- 172.21.15.83:0/1690364298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346b260 msgr2=0x55ba734756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.626+0000 7f1483400640 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346b260 0x55ba734756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.626+0000 7f1483400640 1 -- 172.21.15.83:0/1690364298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba731ad6b0 con 0x55ba73470dc0 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.626+0000 7f1483400640 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 0x55ba7347cc30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f147c009f50 tx=0x7f147c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275b53a89c3613e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.626+0000 7f146cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.626+0000 7f146cc00640 1 -- 172.21.15.83:0/1690364298 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00f7d0 con 0x55ba73470dc0 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.627+0000 7f146cc00640 1 -- 172.21.15.83:0/1690364298 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f147c00f970 con 0x55ba73470dc0 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.627+0000 7f148cb65740 1 -- 172.21.15.83:0/1690364298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14740043c0 con 0x55ba73470dc0 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.627+0000 7f146cc00640 1 -- 172.21.15.83:0/1690364298 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f147c00fb70 con 0x55ba73470dc0 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.627+0000 7f148cb65740 10 client.15087 fetch_fsmap learned FSMap version 23 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.627+0000 7f148cb65740 1 -- 172.21.15.83:0/1690364298 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ba73480070 con 0x55ba73470dc0 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.627+0000 7f146cc00640 1 -- 172.21.15.83:0/1690364298 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14740043c0 con 0x55ba73470dc0 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.627+0000 7f146cc00640 1 -- 172.21.15.83:0/1690364298 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f147c026070 con 0x55ba73470dc0 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.627+0000 7f148cb65740 10 client.15087 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.627+0000 7f148cb65740 10 client.15087 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.627+0000 7f148cb65740 20 client.15087 populate_metadata read hostname 'smithi083' 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.627+0000 7f148cb65740 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ba73482740 0x55ba73483bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.628+0000 7f148cb65740 4 client.15087 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:48:17.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.628+0000 7f148cb65740 1 -- 172.21.15.83:0/1690364298 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba733de530 con 0x55ba73482740 2024-04-23T14:48:17.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.628+0000 7f1482a00640 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ba73482740 0x55ba73483bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.628+0000 7f1482a00640 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ba73482740 0x55ba73483bb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa83e82149ecb8a2 server_cookie=c33b902dea024e66 in_seq=0 out_seq=0 2024-04-23T14:48:17.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.628+0000 7f146cc00640 10 client.15087 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:48:17.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.629+0000 7f146cc00640 1 -- 172.21.15.83:0/1690364298 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba733de530 con 0x55ba73482740 2024-04-23T14:48:17.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.629+0000 7f146cc00640 10 client.15087 handle_command_reply: tid=0 2024-04-23T14:48:17.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.629+0000 7f148cb65740 1 client.15087 shutdown 2024-04-23T14:48:17.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.630+0000 7f148cb65740 1 -- 172.21.15.83:0/1690364298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 msgr2=0x55ba7347cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.630+0000 7f148cb65740 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 0x55ba7347cc30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f147c009f50 tx=0x7f147c004a40 comp rx=0 tx=0).stop 2024-04-23T14:48:17.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.630+0000 7f148cb65740 1 -- 172.21.15.83:0/1690364298 shutdown_connections 2024-04-23T14:48:17.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.630+0000 7f148cb65740 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ba73482740 0x55ba73483bb0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.630+0000 7f148cb65740 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 0x55ba7347cc30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.630+0000 7f148cb65740 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba73475bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.630+0000 7f148cb65740 1 --2- 172.21.15.83:0/1690364298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346b260 0x55ba734756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.630+0000 7f146cc00640 0 client.15087 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:48:17.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.630+0000 7f148cb65740 1 -- 172.21.15.83:0/1690364298 >> 172.21.15.83:0/1690364298 conn(0x55ba734668d0 msgr2=0x55ba73465970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.630+0000 7f148cb65740 1 -- 172.21.15.83:0/1690364298 shutdown_connections 2024-04-23T14:48:17.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.630+0000 7f148cb65740 1 -- 172.21.15.83:0/1690364298 wait complete. 2024-04-23T14:48:17.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.630+0000 7f148cb65740 20 client.15087 trim_cache size 0 max 16384 2024-04-23T14:48:17.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.649+0000 7f148cb65740 1 Processor -- start 2024-04-23T14:48:17.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.649+0000 7f148cb65740 1 -- start start 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f148cb65740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346b260 0x55ba7347d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f148cb65740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba7346c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f148cb65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 0x55ba7347cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f148cb65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba73481ef0 con 0x55ba73470dc0 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f148cb65740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ba734825c0 con 0x55ba7346b260 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f148cb65740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ba73482c90 con 0x55ba7346bc50 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f1483400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 0x55ba7347cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f1483400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 0x55ba7347cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48128/0 (socket says 172.21.15.83:48128) 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f1483400640 1 -- 172.21.15.83:0/942308643 learned_addr learned my addr 172.21.15.83:0/942308643 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f1482000640 1 --2- 172.21.15.83:0/942308643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba7346c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f1482a00640 1 --2- 172.21.15.83:0/942308643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346b260 0x55ba7347d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f1483400640 1 -- 172.21.15.83:0/942308643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 msgr2=0x55ba7346c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f1483400640 1 --2- 172.21.15.83:0/942308643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba7346c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f1483400640 1 -- 172.21.15.83:0/942308643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346b260 msgr2=0x55ba7347d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f1483400640 1 --2- 172.21.15.83:0/942308643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346b260 0x55ba7347d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f1483400640 1 -- 172.21.15.83:0/942308643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba734675a0 con 0x55ba73470dc0 2024-04-23T14:48:17.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.650+0000 7f1482a00640 1 --2- 172.21.15.83:0/942308643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346b260 0x55ba7347d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:48:17.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.651+0000 7f1483400640 1 --2- 172.21.15.83:0/942308643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 0x55ba7347cb50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f147c011f00 tx=0x7f147c00ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2ec941c0850dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.651+0000 7f146f400640 1 -- 172.21.15.83:0/942308643 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00f7d0 con 0x55ba73470dc0 2024-04-23T14:48:17.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.651+0000 7f146f400640 1 -- 172.21.15.83:0/942308643 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f147c00f970 con 0x55ba73470dc0 2024-04-23T14:48:17.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.651+0000 7f146f400640 1 -- 172.21.15.83:0/942308643 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c005280 con 0x55ba73470dc0 2024-04-23T14:48:17.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.651+0000 7f148cb65740 1 -- 172.21.15.83:0/942308643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 msgr2=0x55ba7347cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.651+0000 7f148cb65740 1 --2- 172.21.15.83:0/942308643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 0x55ba7347cb50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f147c011f00 tx=0x7f147c00ff20 comp rx=0 tx=0).stop 2024-04-23T14:48:17.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.651+0000 7f148cb65740 1 -- 172.21.15.83:0/942308643 shutdown_connections 2024-04-23T14:48:17.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.651+0000 7f148cb65740 1 --2- 172.21.15.83:0/942308643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba73470dc0 0x55ba7347cb50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.651+0000 7f148cb65740 1 --2- 172.21.15.83:0/942308643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba7346bc50 0x55ba7346c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.651+0000 7f148cb65740 1 --2- 172.21.15.83:0/942308643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346b260 0x55ba7347d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.651+0000 7f148cb65740 1 -- 172.21.15.83:0/942308643 >> 172.21.15.83:0/942308643 conn(0x55ba734668d0 msgr2=0x55ba73466cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.652+0000 7f148cb65740 1 -- 172.21.15.83:0/942308643 shutdown_connections 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.652+0000 7f148cb65740 1 -- 172.21.15.83:0/942308643 wait complete. 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.652+0000 7f148cb65740 1 -- start start 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.652+0000 7f148cb65740 1 Processor -- start 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f148cb65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba7346b260 0x55ba7347ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f148cb65740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346bc50 0x55ba7347b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f148cb65740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba73470dc0 0x55ba7347b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f148cb65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba73483a30 con 0x55ba7346b260 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f148cb65740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ba73483360 con 0x55ba7346bc50 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f148cb65740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ba73484100 con 0x55ba73470dc0 2024-04-23T14:48:17.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f148cb65740 10 client.0 _ll_register_callbacks cb 0x7f147c003f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f1482a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba7346b260 0x55ba7347ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f1482a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba7346b260 0x55ba7347ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48132/0 (socket says 172.21.15.83:48132) 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f1482a00640 1 -- 172.21.15.83:0/2625739065 learned_addr learned my addr 172.21.15.83:0/2625739065 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f1482000640 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346bc50 0x55ba7347b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f1483400640 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba73470dc0 0x55ba7347b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f1482a00640 1 -- 172.21.15.83:0/2625739065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba73470dc0 msgr2=0x55ba7347b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f1482a00640 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba73470dc0 0x55ba7347b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f1482a00640 1 -- 172.21.15.83:0/2625739065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346bc50 msgr2=0x55ba7347b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f1482a00640 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346bc50 0x55ba7347b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f1482a00640 1 -- 172.21.15.83:0/2625739065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba7347c110 con 0x55ba7346b260 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.653+0000 7f1482000640 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346bc50 0x55ba7347b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.654+0000 7f1482a00640 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba7346b260 0x55ba7347ad90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1474011d10 tx=0x7f1474003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636287ada5eaaed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.654+0000 7f146cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.654+0000 7f146cc00640 1 -- 172.21.15.83:0/2625739065 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147402bd60 con 0x55ba7346b260 2024-04-23T14:48:17.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.654+0000 7f146cc00640 1 -- 172.21.15.83:0/2625739065 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f147402b350 con 0x55ba7346b260 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.654+0000 7f148cb65740 1 -- 172.21.15.83:0/2625739065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147c00f970 con 0x55ba7346b260 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.654+0000 7f146cc00640 1 -- 172.21.15.83:0/2625739065 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f147403a050 con 0x55ba7346b260 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.654+0000 7f148cb65740 10 client.15093 fetch_fsmap learned FSMap version 23 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.654+0000 7f148cb65740 1 -- 172.21.15.83:0/2625739065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ba734e8960 con 0x55ba7346b260 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.654+0000 7f146cc00640 1 -- 172.21.15.83:0/2625739065 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f147c00f970 con 0x55ba7346b260 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.654+0000 7f146cc00640 1 -- 172.21.15.83:0/2625739065 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f1474005040 con 0x55ba7346b260 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.655+0000 7f148cb65740 10 client.15093 fetch_fsmap finished waiting for FSMap version 23 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.655+0000 7f148cb65740 10 client.15093 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.655+0000 7f148cb65740 20 client.15093 populate_metadata read hostname 'smithi083' 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.655+0000 7f148cb65740 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ba734ed250 0x55ba734e9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.655+0000 7f148cb65740 4 client.15093 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.655+0000 7f148cb65740 1 -- 172.21.15.83:0/2625739065 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "status"}) -- 0x55ba733d83a0 con 0x55ba734ed250 2024-04-23T14:48:17.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.655+0000 7f1482000640 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ba734ed250 0x55ba734e9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.655+0000 7f1482000640 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ba734ed250 0x55ba734e9d60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9564a7c1b0f328dd server_cookie=59fd7bbdb99961d4 in_seq=0 out_seq=0 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.655+0000 7f146cc00640 10 client.15093 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f146cc00640 1 -- 172.21.15.83:0/2625739065 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ba733d83a0 con 0x55ba734ed250 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f146cc00640 10 client.15093 handle_command_reply: tid=0 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f148cb65740 1 client.15093 shutdown 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f148cb65740 1 -- 172.21.15.83:0/2625739065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba7346b260 msgr2=0x55ba7347ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f148cb65740 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba7346b260 0x55ba7347ad90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1474011d10 tx=0x7f1474003dd0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f148cb65740 1 -- 172.21.15.83:0/2625739065 shutdown_connections 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f148cb65740 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ba734ed250 0x55ba734e9d60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f148cb65740 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ba73470dc0 0x55ba7347b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f148cb65740 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba7346bc50 0x55ba7347b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f148cb65740 1 --2- 172.21.15.83:0/2625739065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba7346b260 0x55ba7347ad90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f146cc00640 0 client.15093 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:48:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f148cb65740 1 -- 172.21.15.83:0/2625739065 >> 172.21.15.83:0/2625739065 conn(0x55ba734668d0 msgr2=0x55ba73466cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f148cb65740 1 -- 172.21.15.83:0/2625739065 shutdown_connections 2024-04-23T14:48:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f148cb65740 1 -- 172.21.15.83:0/2625739065 wait complete. 2024-04-23T14:48:17.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.656+0000 7f148cb65740 20 client.15093 trim_cache size 0 max 16384 2024-04-23T14:48:17.662 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:48:17.662 INFO:teuthology.orchestra.run.smithi083.stdout: "cluster_fsid": "5f4fe708-017e-11ef-bc93-c7b262605968", 2024-04-23T14:48:17.662 INFO:teuthology.orchestra.run.smithi083.stdout: "whoami": 0, 2024-04-23T14:48:17.662 INFO:teuthology.orchestra.run.smithi083.stdout: "id": 24599, 2024-04-23T14:48:17.662 INFO:teuthology.orchestra.run.smithi083.stdout: "want_state": "up:active", 2024-04-23T14:48:17.662 INFO:teuthology.orchestra.run.smithi083.stdout: "state": "up:active", 2024-04-23T14:48:17.662 INFO:teuthology.orchestra.run.smithi083.stdout: "fs_name": "cephfs", 2024-04-23T14:48:17.663 INFO:teuthology.orchestra.run.smithi083.stdout: "rank_uptime": 137.43940344800001, 2024-04-23T14:48:17.663 INFO:teuthology.orchestra.run.smithi083.stdout: "mdsmap_epoch": 23, 2024-04-23T14:48:17.663 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch": 88, 2024-04-23T14:48:17.663 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T14:48:17.663 INFO:teuthology.orchestra.run.smithi083.stdout: "uptime": 143.95370230500001 2024-04-23T14:48:17.663 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:48:17.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.657+0000 7f1483e00640 1 -- 172.21.15.83:0/3397073005 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f14680a3bb0 msgr2=0x7f14680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.657+0000 7f1483e00640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f14680a3bb0 0x7f14680a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55ba7346c4b0 tx=0x7f147c009500 comp rx=0 tx=0).stop 2024-04-23T14:48:17.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.657+0000 7f1483e00640 1 -- 172.21.15.83:0/3397073005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1484105330 msgr2=0x7f148419a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.657+0000 7f1483e00640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1484105330 0x7f148419a930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1470031440 tx=0x7f1470032ce0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.657+0000 7f1483e00640 1 -- 172.21.15.83:0/3397073005 shutdown_connections 2024-04-23T14:48:17.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.657+0000 7f1483e00640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f14680a3bb0 0x7f14680a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.657+0000 7f1483e00640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1484105cf0 0x7f14841a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.657+0000 7f1483e00640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1484105330 0x7f148419a930 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.657+0000 7f1483e00640 1 --2- 172.21.15.83:0/3397073005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1484104140 0x7f148419a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.657+0000 7f1483e00640 1 -- 172.21.15.83:0/3397073005 >> 172.21.15.83:0/3397073005 conn(0x7f14840ff6b0 msgr2=0x7f1484108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.657+0000 7f1483e00640 1 -- 172.21.15.83:0/3397073005 shutdown_connections 2024-04-23T14:48:17.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.657+0000 7f1483e00640 1 -- 172.21.15.83:0/3397073005 wait complete. 2024-04-23T14:48:17.676 INFO:teuthology.run_tasks:Running task sequential... 2024-04-23T14:48:17.685 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-23T14:48:17.685 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-23T14:48:17.686 INFO:teuthology.task.print:Enabling mgr modules 2024-04-23T14:48:17.686 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-23T14:48:17.687 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T14:48:17.687 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-23T14:48:17.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-23T14:48:17.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:17 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1659349884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:48:17.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:17 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3050105331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:17 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1659349884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:48:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:17 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3050105331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:17.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.875+0000 7faa53e00640 1 -- 172.21.15.83:0/3883040373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4c10a240 msgr2=0x7faa4c110ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.875+0000 7faa53e00640 1 --2- 172.21.15.83:0/3883040373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4c10a240 0x7faa4c110ad0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faa44009ca0 tx=0x7faa4402a8c0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.875+0000 7faa53e00640 1 -- 172.21.15.83:0/3883040373 shutdown_connections 2024-04-23T14:48:17.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.875+0000 7faa53e00640 1 --2- 172.21.15.83:0/3883040373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4c10a240 0x7faa4c110ad0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.875+0000 7faa53e00640 1 --2- 172.21.15.83:0/3883040373 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa4c109880 0x7faa4c109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.875+0000 7faa53e00640 1 --2- 172.21.15.83:0/3883040373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa4c108690 0x7faa4c108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.875+0000 7faa53e00640 1 -- 172.21.15.83:0/3883040373 >> 172.21.15.83:0/3883040373 conn(0x7faa4c077650 msgr2=0x7faa4c079a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:17.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.875+0000 7faa53e00640 1 -- 172.21.15.83:0/3883040373 shutdown_connections 2024-04-23T14:48:17.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.875+0000 7faa53e00640 1 -- 172.21.15.83:0/3883040373 wait complete. 2024-04-23T14:48:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.876+0000 7faa53e00640 1 Processor -- start 2024-04-23T14:48:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa53e00640 1 -- start start 2024-04-23T14:48:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa53e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa4c108690 0x7faa4c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4c109880 0x7faa4c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa53e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa4c10a240 0x7faa4c1a5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa53e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c114360 con 0x7faa4c109880 2024-04-23T14:48:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa53e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1141e0 con 0x7faa4c10a240 2024-04-23T14:48:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa53e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c1144e0 con 0x7faa4c108690 2024-04-23T14:48:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa51400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa4c108690 0x7faa4c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa50a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4c109880 0x7faa4c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa51400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa4c108690 0x7faa4c19e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38506/0 (socket says 172.21.15.83:38506) 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa51400640 1 -- 172.21.15.83:0/4016161730 learned_addr learned my addr 172.21.15.83:0/4016161730 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa51e00640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa4c10a240 0x7faa4c1a5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa51400640 1 -- 172.21.15.83:0/4016161730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa4c10a240 msgr2=0x7faa4c1a5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.877+0000 7faa51400640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa4c10a240 0x7faa4c1a5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.878+0000 7faa51400640 1 -- 172.21.15.83:0/4016161730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4c109880 msgr2=0x7faa4c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.878+0000 7faa51400640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4c109880 0x7faa4c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.878+0000 7faa51400640 1 -- 172.21.15.83:0/4016161730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c1a64b0 con 0x7faa4c108690 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.878+0000 7faa50a00640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4c109880 0x7faa4c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.878+0000 7faa51400640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa4c108690 0x7faa4c19e890 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faa4000e9c0 tx=0x7faa4000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.878+0000 7faa4a000640 1 -- 172.21.15.83:0/4016161730 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4000cd80 con 0x7faa4c108690 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.878+0000 7faa4a000640 1 -- 172.21.15.83:0/4016161730 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa400035f0 con 0x7faa4c108690 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.878+0000 7faa4a000640 1 -- 172.21.15.83:0/4016161730 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40010690 con 0x7faa4c108690 2024-04-23T14:48:17.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.878+0000 7faa53e00640 1 -- 172.21.15.83:0/4016161730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4c1a67a0 con 0x7faa4c108690 2024-04-23T14:48:17.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.878+0000 7faa53e00640 1 -- 172.21.15.83:0/4016161730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4c1a6d40 con 0x7faa4c108690 2024-04-23T14:48:17.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.879+0000 7faa53e00640 1 -- 172.21.15.83:0/4016161730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa4c10f8e0 con 0x7faa4c108690 2024-04-23T14:48:17.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.881+0000 7faa4a000640 1 -- 172.21.15.83:0/4016161730 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faa400037a0 con 0x7faa4c108690 2024-04-23T14:48:17.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.883+0000 7faa4a000640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7faa340a3cd0 0x7faa340a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:17.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.883+0000 7faa4a000640 1 -- 172.21.15.83:0/4016161730 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7faa400d07a0 con 0x7faa4c108690 2024-04-23T14:48:17.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.883+0000 7faa50a00640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7faa340a3cd0 0x7faa340a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:17.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.884+0000 7faa50a00640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7faa340a3cd0 0x7faa340a6190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa4c19fda0 tx=0x7faa38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:17.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:17.887+0000 7faa4a000640 1 -- 172.21.15.83:0/4016161730 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa4009d140 con 0x7faa4c108690 2024-04-23T14:48:18.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.143+0000 7faa53e00640 1 -- 172.21.15.83:0/4016161730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7faa4c062c00 con 0x7faa4c108690 2024-04-23T14:48:18.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.413+0000 7faa4a000640 1 -- 172.21.15.83:0/4016161730 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7faa400990b0 con 0x7faa4c108690 2024-04-23T14:48:18.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.416+0000 7faa4a000640 1 -- 172.21.15.83:0/4016161730 <== mon.1 v2:172.21.15.203:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7faa400795b0 con 0x7faa4c108690 2024-04-23T14:48:18.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.420+0000 7faa53e00640 1 -- 172.21.15.83:0/4016161730 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7faa340a3cd0 msgr2=0x7faa340a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:18.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.420+0000 7faa53e00640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7faa340a3cd0 0x7faa340a6190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa4c19fda0 tx=0x7faa38005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:18.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.420+0000 7faa53e00640 1 -- 172.21.15.83:0/4016161730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa4c108690 msgr2=0x7faa4c19e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:18.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.420+0000 7faa53e00640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa4c108690 0x7faa4c19e890 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faa4000e9c0 tx=0x7faa4000ee80 comp rx=0 tx=0).stop 2024-04-23T14:48:18.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.420+0000 7faa53e00640 1 -- 172.21.15.83:0/4016161730 shutdown_connections 2024-04-23T14:48:18.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.420+0000 7faa53e00640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7faa340a3cd0 0x7faa340a6190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.420+0000 7faa53e00640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa4c10a240 0x7faa4c1a5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.420+0000 7faa53e00640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa4c109880 0x7faa4c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.420+0000 7faa53e00640 1 --2- 172.21.15.83:0/4016161730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa4c108690 0x7faa4c19e890 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.420+0000 7faa53e00640 1 -- 172.21.15.83:0/4016161730 >> 172.21.15.83:0/4016161730 conn(0x7faa4c077650 msgr2=0x7faa4c10cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:18.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.420+0000 7faa53e00640 1 -- 172.21.15.83:0/4016161730 shutdown_connections 2024-04-23T14:48:18.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.420+0000 7faa53e00640 1 -- 172.21.15.83:0/4016161730 wait complete. 2024-04-23T14:48:18.443 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-23T14:48:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.621+0000 7f1ffba00640 1 -- 172.21.15.83:0/3138269388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ff4105e40 msgr2=0x7f1ff410c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.621+0000 7f1ffba00640 1 --2- 172.21.15.83:0/3138269388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ff4105e40 0x7f1ff410c6d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009ca0 tx=0x7f1fec02a8c0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.621+0000 7f1ffba00640 1 -- 172.21.15.83:0/3138269388 shutdown_connections 2024-04-23T14:48:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.621+0000 7f1ffba00640 1 --2- 172.21.15.83:0/3138269388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ff4105e40 0x7f1ff410c6d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.621+0000 7f1ffba00640 1 --2- 172.21.15.83:0/3138269388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff4105480 0x7f1ff4105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.621+0000 7f1ffba00640 1 --2- 172.21.15.83:0/3138269388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff4104290 0x7f1ff41046b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.621+0000 7f1ffba00640 1 -- 172.21.15.83:0/3138269388 >> 172.21.15.83:0/3138269388 conn(0x7f1ff40ff820 msgr2=0x7f1ff4101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:18.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.621+0000 7f1ffba00640 1 -- 172.21.15.83:0/3138269388 shutdown_connections 2024-04-23T14:48:18.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.621+0000 7f1ffba00640 1 -- 172.21.15.83:0/3138269388 wait complete. 2024-04-23T14:48:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.622+0000 7f1ffba00640 1 Processor -- start 2024-04-23T14:48:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.622+0000 7f1ffba00640 1 -- start start 2024-04-23T14:48:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ffba00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ff4104290 0x7f1ff419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ffba00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff4105480 0x7f1ff419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ffba00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff4105e40 0x7f1ff41a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ffba00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410ff20 con 0x7f1ff4105e40 2024-04-23T14:48:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ffba00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff410fda0 con 0x7f1ff4105480 2024-04-23T14:48:18.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ffba00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41100a0 con 0x7f1ff4104290 2024-04-23T14:48:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ff9000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ff4104290 0x7f1ff419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff4105480 0x7f1ff419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ff9000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ff4104290 0x7f1ff419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38538/0 (socket says 172.21.15.83:38538) 2024-04-23T14:48:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff4105480 0x7f1ff419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:57302/0 (socket says 172.21.15.83:57302) 2024-04-23T14:48:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ff9000640 1 -- 172.21.15.83:0/3839129056 learned_addr learned my addr 172.21.15.83:0/3839129056 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ff9a00640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff4105e40 0x7f1ff41a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ff9000640 1 -- 172.21.15.83:0/3839129056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff4105480 msgr2=0x7f1ff419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ff9000640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff4105480 0x7f1ff419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ff9000640 1 -- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff4105e40 msgr2=0x7f1ff41a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ff9000640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff4105e40 0x7f1ff41a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ff9000640 1 -- 172.21.15.83:0/3839129056 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff41a2080 con 0x7f1ff4104290 2024-04-23T14:48:18.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.623+0000 7f1ff9a00640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff4105e40 0x7f1ff41a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:48:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.624+0000 7f1ff9000640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ff4104290 0x7f1ff419a460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800e9c0 tx=0x7f1fe800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.624+0000 7f1ff1600640 1 -- 172.21.15.83:0/3839129056 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe800cd80 con 0x7f1ff4104290 2024-04-23T14:48:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.624+0000 7f1ff1600640 1 -- 172.21.15.83:0/3839129056 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe80035f0 con 0x7f1ff4104290 2024-04-23T14:48:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.624+0000 7f1ffba00640 1 -- 172.21.15.83:0/3839129056 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff41a2370 con 0x7f1ff4104290 2024-04-23T14:48:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.624+0000 7f1ff1600640 1 -- 172.21.15.83:0/3839129056 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8010690 con 0x7f1ff4104290 2024-04-23T14:48:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.624+0000 7f1ffba00640 1 -- 172.21.15.83:0/3839129056 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff41a2870 con 0x7f1ff4104290 2024-04-23T14:48:18.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.625+0000 7f1ffba00640 1 -- 172.21.15.83:0/3839129056 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff410b550 con 0x7f1ff4104290 2024-04-23T14:48:18.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.627+0000 7f1ff1600640 1 -- 172.21.15.83:0/3839129056 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f1fe80037a0 con 0x7f1ff4104290 2024-04-23T14:48:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.628+0000 7f1ff1600640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1fdc0a3ca0 0x7f1fdc0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.629+0000 7f1ff3e00640 1 -- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1fdc0a3ca0 msgr2=0x7f1fdc0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1964818199 2024-04-23T14:48:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.629+0000 7f1ff1600640 1 -- 172.21.15.83:0/3839129056 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1fe80d08b0 con 0x7f1ff4104290 2024-04-23T14:48:18.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.629+0000 7f1ff3e00640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1fdc0a3ca0 0x7f1fdc0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:48:18.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.632+0000 7f1ff1600640 1 -- 172.21.15.83:0/3839129056 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fe809d250 con 0x7f1ff4104290 2024-04-23T14:48:18.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:18 smithi203 ceph-mon[32800]: pgmap v388: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:48:18.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:18 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/4016161730' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T14:48:18.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:18 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T14:48:18.722 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:18 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: ignoring --setuser ceph since I am not root 2024-04-23T14:48:18.722 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:18 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: ignoring --setgroup ceph since I am not root 2024-04-23T14:48:18.722 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:18 smithi203 ceph-mgr[34662]: -- 172.21.15.203:0/3013028021 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574f2f7a5a0 con 0x5574f2f4ec00 2024-04-23T14:48:18.722 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:18 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:18.584+0000 7f3e25074200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T14:48:18.723 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:18 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: ignoring --setuser ceph since I am not root 2024-04-23T14:48:18.724 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:18 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: ignoring --setgroup ceph since I am not root 2024-04-23T14:48:18.724 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:18 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:18.721+0000 7f0b7dece200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T14:48:18.724 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:18 smithi083 ceph-mon[27867]: pgmap v388: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:48:18.724 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:18 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/4016161730' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T14:48:18.724 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:18 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T14:48:18.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:18 smithi173 ceph-mon[32443]: pgmap v388: 97 pgs: 97 active+clean; 639 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:48:18.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:18 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/4016161730' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T14:48:18.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:18 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T14:48:18.727 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:18 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: ignoring --setuser ceph since I am not root 2024-04-23T14:48:18.728 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:18 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: ignoring --setgroup ceph since I am not root 2024-04-23T14:48:18.728 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:18 smithi173 ceph-mgr[33821]: -- 172.21.15.173:0/2277110236 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5584ab4f85a0 con 0x5584ab4ccc00 2024-04-23T14:48:18.728 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:18 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:18.583+0000 7f7388758200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T14:48:18.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.830+0000 7f1ff3e00640 1 -- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1fdc0a3ca0 msgr2=0x7f1fdc0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1964818199 2024-04-23T14:48:18.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.830+0000 7f1ff3e00640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1fdc0a3ca0 0x7f1fdc0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T14:48:18.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.836+0000 7f1ffba00640 1 -- 172.21.15.83:0/3839129056 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f1ff402d620 con 0x7f1ff4104290 2024-04-23T14:48:18.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.842+0000 7f1ff1600640 1 -- 172.21.15.83:0/3839129056 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7f1fe809cb50 con 0x7f1ff4104290 2024-04-23T14:48:18.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.845+0000 7f1ffba00640 1 -- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1fdc0a3ca0 msgr2=0x7f1fdc0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:48:18.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.845+0000 7f1ffba00640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1fdc0a3ca0 0x7f1fdc0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.845+0000 7f1ffba00640 1 -- 172.21.15.83:0/3839129056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ff4104290 msgr2=0x7f1ff419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:18.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.845+0000 7f1ffba00640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ff4104290 0x7f1ff419a460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800e9c0 tx=0x7f1fe800ee80 comp rx=0 tx=0).stop 2024-04-23T14:48:18.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.845+0000 7f1ffba00640 1 -- 172.21.15.83:0/3839129056 shutdown_connections 2024-04-23T14:48:18.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.845+0000 7f1ffba00640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f1fdc0a3ca0 0x7f1fdc0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.845+0000 7f1ffba00640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ff4105e40 0x7f1ff41a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.845+0000 7f1ffba00640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1ff4105480 0x7f1ff419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.845+0000 7f1ffba00640 1 --2- 172.21.15.83:0/3839129056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ff4104290 0x7f1ff419a460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.845+0000 7f1ffba00640 1 -- 172.21.15.83:0/3839129056 >> 172.21.15.83:0/3839129056 conn(0x7f1ff40ff820 msgr2=0x7f1ff4108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.845+0000 7f1ffba00640 1 -- 172.21.15.83:0/3839129056 shutdown_connections 2024-04-23T14:48:18.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:18.845+0000 7f1ffba00640 1 -- 172.21.15.83:0/3839129056 wait complete. 2024-04-23T14:48:18.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-23T14:48:19.007 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:18 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:18.969+0000 7f0b7dece200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:48:19.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.050+0000 7effa4c00640 1 -- 172.21.15.83:0/3463354832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa0102900 msgr2=0x7effa0104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:19.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.050+0000 7effa4c00640 1 --2- 172.21.15.83:0/3463354832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa0102900 0x7effa0104cf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7eff90009900 tx=0x7eff9002a410 comp rx=0 tx=0).stop 2024-04-23T14:48:19.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.050+0000 7effa4c00640 1 -- 172.21.15.83:0/3463354832 shutdown_connections 2024-04-23T14:48:19.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.050+0000 7effa4c00640 1 --2- 172.21.15.83:0/3463354832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effa0107b60 0x7effa0109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:19.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.050+0000 7effa4c00640 1 --2- 172.21.15.83:0/3463354832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0105230 0x7effa0107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:19.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.050+0000 7effa4c00640 1 --2- 172.21.15.83:0/3463354832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa0102900 0x7effa0104cf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:19.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.050+0000 7effa4c00640 1 -- 172.21.15.83:0/3463354832 >> 172.21.15.83:0/3463354832 conn(0x7effa00fc2f0 msgr2=0x7effa00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:19.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.050+0000 7effa4c00640 1 -- 172.21.15.83:0/3463354832 shutdown_connections 2024-04-23T14:48:19.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.050+0000 7effa4c00640 1 -- 172.21.15.83:0/3463354832 wait complete. 2024-04-23T14:48:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.051+0000 7effa4c00640 1 Processor -- start 2024-04-23T14:48:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.051+0000 7effa4c00640 1 -- start start 2024-04-23T14:48:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.051+0000 7effa4c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effa0102900 0x7effa01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.051+0000 7effa4c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0105230 0x7effa0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:19.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.051+0000 7effa4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa0107b60 0x7effa019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.051+0000 7effa4c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7effa010db70 con 0x7effa0107b60 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.051+0000 7effa4c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7effa010d9f0 con 0x7effa0105230 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.051+0000 7effa4c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7effa010dcf0 con 0x7effa0102900 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.052+0000 7eff9ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0105230 0x7effa0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.052+0000 7eff9f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effa0102900 0x7effa01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.052+0000 7eff9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa0107b60 0x7effa019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.052+0000 7eff9ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0105230 0x7effa0198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:57318/0 (socket says 172.21.15.83:57318) 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.052+0000 7eff9f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effa0102900 0x7effa01981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38564/0 (socket says 172.21.15.83:38564) 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.052+0000 7eff9ea00640 1 -- 172.21.15.83:0/1330151417 learned_addr learned my addr 172.21.15.83:0/1330151417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.052+0000 7eff9f400640 1 -- 172.21.15.83:0/1330151417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0105230 msgr2=0x7effa0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.052+0000 7eff9f400640 1 --2- 172.21.15.83:0/1330151417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0105230 0x7effa0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.052+0000 7eff9f400640 1 -- 172.21.15.83:0/1330151417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa0107b60 msgr2=0x7effa019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.052+0000 7eff9f400640 1 --2- 172.21.15.83:0/1330151417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa0107b60 0x7effa019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.052+0000 7eff9f400640 1 -- 172.21.15.83:0/1330151417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa019fe00 con 0x7effa0102900 2024-04-23T14:48:19.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.052+0000 7eff9f400640 1 --2- 172.21.15.83:0/1330151417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effa0102900 0x7effa01981e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7eff9002a930 tx=0x7eff90032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.053+0000 7eff97e00640 1 -- 172.21.15.83:0/1330151417 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9003f030 con 0x7effa0102900 2024-04-23T14:48:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.053+0000 7effa4c00640 1 -- 172.21.15.83:0/1330151417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa01a0090 con 0x7effa0102900 2024-04-23T14:48:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.053+0000 7eff97e00640 1 -- 172.21.15.83:0/1330151417 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff90003ce0 con 0x7effa0102900 2024-04-23T14:48:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.053+0000 7eff97e00640 1 -- 172.21.15.83:0/1330151417 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff90035070 con 0x7effa0102900 2024-04-23T14:48:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.053+0000 7effa4c00640 1 -- 172.21.15.83:0/1330151417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa01a0550 con 0x7effa0102900 2024-04-23T14:48:19.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.054+0000 7effa4c00640 1 -- 172.21.15.83:0/1330151417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa00fdde0 con 0x7effa0102900 2024-04-23T14:48:19.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.056+0000 7eff97e00640 1 -- 172.21.15.83:0/1330151417 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7eff90042030 con 0x7effa0102900 2024-04-23T14:48:19.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.057+0000 7eff97e00640 1 --2- 172.21.15.83:0/1330151417 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7eff840a3bf0 0x7eff840a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:19.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.057+0000 7eff9ea00640 1 -- 172.21.15.83:0/1330151417 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7eff840a3bf0 msgr2=0x7eff840a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1964818199 2024-04-23T14:48:19.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.057+0000 7eff97e00640 1 -- 172.21.15.83:0/1330151417 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7eff900f0860 con 0x7effa0102900 2024-04-23T14:48:19.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.057+0000 7eff9ea00640 1 --2- 172.21.15.83:0/1330151417 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7eff840a3bf0 0x7eff840a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:48:19.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.061+0000 7eff97e00640 1 -- 172.21.15.83:0/1330151417 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff900bcbb0 con 0x7effa0102900 2024-04-23T14:48:19.076 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:18 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:18.768+0000 7f7388758200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:48:19.076 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:18 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:18.871+0000 7f7388758200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T14:48:19.181 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:18 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:18.769+0000 7f3e25074200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:48:19.181 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:18 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:18.873+0000 7f3e25074200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T14:48:19.181 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:19 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:19.180+0000 7f3e25074200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T14:48:19.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.258+0000 7eff9ea00640 1 -- 172.21.15.83:0/1330151417 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7eff840a3bf0 msgr2=0x7eff840a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1964818199 2024-04-23T14:48:19.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.258+0000 7eff9ea00640 1 --2- 172.21.15.83:0/1330151417 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7eff840a3bf0 0x7eff840a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T14:48:19.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.265+0000 7effa4c00640 1 -- 172.21.15.83:0/1330151417 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7effa002d620 con 0x7effa0102900 2024-04-23T14:48:19.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.271+0000 7eff97e00640 1 -- 172.21.15.83:0/1330151417 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) ==== 133+0+0 (secure 0 0 0) 0x7eff900c0eb0 con 0x7effa0102900 2024-04-23T14:48:19.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.273+0000 7effa4c00640 1 -- 172.21.15.83:0/1330151417 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7eff840a3bf0 msgr2=0x7eff840a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:48:19.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.273+0000 7effa4c00640 1 --2- 172.21.15.83:0/1330151417 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7eff840a3bf0 0x7eff840a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:19.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.273+0000 7effa4c00640 1 -- 172.21.15.83:0/1330151417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effa0102900 msgr2=0x7effa01981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:19.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.273+0000 7effa4c00640 1 --2- 172.21.15.83:0/1330151417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effa0102900 0x7effa01981e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7eff9002a930 tx=0x7eff90032ce0 comp rx=0 tx=0).stop 2024-04-23T14:48:19.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.274+0000 7effa4c00640 1 -- 172.21.15.83:0/1330151417 shutdown_connections 2024-04-23T14:48:19.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.274+0000 7effa4c00640 1 --2- 172.21.15.83:0/1330151417 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7eff840a3bf0 0x7eff840a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:19.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.274+0000 7effa4c00640 1 --2- 172.21.15.83:0/1330151417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7effa0107b60 0x7effa019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:19.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.274+0000 7effa4c00640 1 --2- 172.21.15.83:0/1330151417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7effa0105230 0x7effa0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:19.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.274+0000 7effa4c00640 1 --2- 172.21.15.83:0/1330151417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7effa0102900 0x7effa01981e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:19.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.274+0000 7effa4c00640 1 -- 172.21.15.83:0/1330151417 >> 172.21.15.83:0/1330151417 conn(0x7effa00fc2f0 msgr2=0x7effa01033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:19.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.274+0000 7effa4c00640 1 -- 172.21.15.83:0/1330151417 shutdown_connections 2024-04-23T14:48:19.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:19.274+0000 7effa4c00640 1 -- 172.21.15.83:0/1330151417 wait complete. 2024-04-23T14:48:19.292 INFO:teuthology.run_tasks:Running task kclient... 2024-04-23T14:48:19.315 INFO:tasks.kclient:Mounting kernel clients... 2024-04-23T14:48:19.315 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-23T14:48:19.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:48:19.316 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-23T14:48:19.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:48:19.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-23T14:48:19.424 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:19 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:19.105+0000 7f0b7dece200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T14:48:19.450 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-04-23T14:48:19.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:48:19.454 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-23T14:48:19.454 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T14:48:19.454 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:48:19.455 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-23T14:48:19.471 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:19 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:19.266+0000 7f3e25074200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T14:48:19.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:19 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T14:48:19.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:19 smithi203 ceph-mon[32800]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-23T14:48:19.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:19 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' 2024-04-23T14:48:19.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:19 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' 2024-04-23T14:48:19.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:19 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T14:48:19.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:19 smithi173 ceph-mon[32443]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-23T14:48:19.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:19 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' 2024-04-23T14:48:19.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:19 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' 2024-04-23T14:48:19.576 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:19 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:19.178+0000 7f7388758200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T14:48:19.576 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:19 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:19.263+0000 7f7388758200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T14:48:19.757 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:19 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:19.514+0000 7f0b7dece200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T14:48:19.757 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:19 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:19.628+0000 7f0b7dece200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T14:48:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:19 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T14:48:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:19 smithi083 ceph-mon[27867]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-23T14:48:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:19 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' 2024-04-23T14:48:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:19 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' 2024-04-23T14:48:19.888 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:48:20.721 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:20 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:20.444+0000 7f3e25074200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T14:48:20.721 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:20 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:20.614+0000 7f3e25074200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T14:48:20.794 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:20 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:20.449+0000 7f7388758200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T14:48:20.794 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:20 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:20.619+0000 7f7388758200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T14:48:20.794 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:20 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:20.793+0000 7f7388758200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T14:48:21.077 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:20 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:20.786+0000 7f3e25074200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T14:48:21.373 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:21 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:21.076+0000 7f3e25074200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T14:48:21.374 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:21 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:21.169+0000 7f3e25074200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T14:48:21.374 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:21 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:21.373+0000 7f3e25074200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T14:48:21.380 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:21 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:21.083+0000 7f7388758200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T14:48:21.380 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:21 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:21.175+0000 7f7388758200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T14:48:21.506 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:21 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:21.206+0000 7f0b7dece200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T14:48:21.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:21 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:21.438+0000 7f0b7dece200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T14:48:21.721 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:21 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:21.566+0000 7f3e25074200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:48:21.826 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:21 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:21.379+0000 7f7388758200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T14:48:21.826 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:21 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:21.574+0000 7f7388758200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:48:21.932 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:21 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:21.660+0000 7f0b7dece200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T14:48:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.090+0000 7f3c74a00700 1 -- 172.21.15.83:0/3765556773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c7010a040 msgr2=0x7f3c70111c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.090+0000 7f3c74a00700 1 --2- 172.21.15.83:0/3765556773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c7010a040 0x7f3c70111c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800b0d0 tx=0x7f3c6800b3e0 comp rx=0 tx=0).stop 2024-04-23T14:48:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.090+0000 7f3c74a00700 1 -- 172.21.15.83:0/3765556773 shutdown_connections 2024-04-23T14:48:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.090+0000 7f3c74a00700 1 --2- 172.21.15.83:0/3765556773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c7010a040 0x7f3c70111c10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:22.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.090+0000 7f3c74a00700 1 --2- 172.21.15.83:0/3765556773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70104f60 0x7f3c70105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.090+0000 7f3c74a00700 1 --2- 172.21.15.83:0/3765556773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c70105910 0x7f3c70109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.090+0000 7f3c74a00700 1 -- 172.21.15.83:0/3765556773 >> 172.21.15.83:0/3765556773 conn(0x7f3c701007d0 msgr2=0x7f3c70102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.090+0000 7f3c74a00700 1 -- 172.21.15.83:0/3765556773 shutdown_connections 2024-04-23T14:48:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.090+0000 7f3c74a00700 1 -- 172.21.15.83:0/3765556773 wait complete. 2024-04-23T14:48:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.091+0000 7f3c74a00700 1 Processor -- start 2024-04-23T14:48:22.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.092+0000 7f3c74a00700 1 -- start start 2024-04-23T14:48:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.092+0000 7f3c74a00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c70104f60 0x7f3c70111a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.092+0000 7f3c74a00700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70105910 0x7f3c7010eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.092+0000 7f3c74a00700 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c7010a040 0x7f3c7010f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.092+0000 7f3c74a00700 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70115080 con 0x7f3c7010a040 2024-04-23T14:48:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.092+0000 7f3c74a00700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70114f00 con 0x7f3c70105910 2024-04-23T14:48:22.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.092+0000 7f3c74a00700 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70115200 con 0x7f3c70104f60 2024-04-23T14:48:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.092+0000 7f3c6de00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c70104f60 0x7f3c70111a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.093+0000 7f3c6de00700 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c70104f60 0x7f3c70111a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38574/0 (socket says 172.21.15.83:38574) 2024-04-23T14:48:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.093+0000 7f3c6de00700 1 -- 172.21.15.83:0/2566552093 learned_addr learned my addr 172.21.15.83:0/2566552093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.093+0000 7f3c6d400700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70105910 0x7f3c7010eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.093+0000 7f3c6e800700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c7010a040 0x7f3c7010f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.093+0000 7f3c6de00700 1 -- 172.21.15.83:0/2566552093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70105910 msgr2=0x7f3c7010eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:22.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.093+0000 7f3c6de00700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70105910 0x7f3c7010eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:22.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.093+0000 7f3c6de00700 1 -- 172.21.15.83:0/2566552093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c7010a040 msgr2=0x7f3c7010f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:22.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.093+0000 7f3c6de00700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c7010a040 0x7f3c7010f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:22.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.093+0000 7f3c6de00700 1 -- 172.21.15.83:0/2566552093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7010f970 con 0x7f3c70104f60 2024-04-23T14:48:22.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.094+0000 7f3c6de00700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c70104f60 0x7f3c70111a60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000eb30 tx=0x7f3c6000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:22.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.094+0000 7f3c66a00700 1 -- 172.21.15.83:0/2566552093 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6000cbd0 con 0x7f3c70104f60 2024-04-23T14:48:22.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.094+0000 7f3c74a00700 1 -- 172.21.15.83:0/2566552093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7006c630 con 0x7f3c70104f60 2024-04-23T14:48:22.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.094+0000 7f3c66a00700 1 -- 172.21.15.83:0/2566552093 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c60004500 con 0x7f3c70104f60 2024-04-23T14:48:22.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.094+0000 7f3c66a00700 1 -- 172.21.15.83:0/2566552093 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c60010430 con 0x7f3c70104f60 2024-04-23T14:48:22.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.094+0000 7f3c74a00700 1 -- 172.21.15.83:0/2566552093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c7006cb90 con 0x7f3c70104f60 2024-04-23T14:48:22.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.095+0000 7f3c74a00700 1 -- 172.21.15.83:0/2566552093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c70106440 con 0x7f3c70104f60 2024-04-23T14:48:22.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.097+0000 7f3c66a00700 1 -- 172.21.15.83:0/2566552093 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f3c60003f00 con 0x7f3c70104f60 2024-04-23T14:48:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.098+0000 7f3c66a00700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f3c540a3c90 0x7f3c540a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.099+0000 7f3c6d400700 1 -- 172.21.15.83:0/2566552093 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f3c540a3c90 msgr2=0x7f3c540a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1964818199 2024-04-23T14:48:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.099+0000 7f3c66a00700 1 -- 172.21.15.83:0/2566552093 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f3c600d0820 con 0x7f3c70104f60 2024-04-23T14:48:22.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.099+0000 7f3c6d400700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f3c540a3c90 0x7f3c540a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:48:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.103+0000 7f3c66a00700 1 -- 172.21.15.83:0/2566552093 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c6009d1c0 con 0x7f3c70104f60 2024-04-23T14:48:22.183 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:22 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:22.055+0000 7f0b7dece200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T14:48:22.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.300+0000 7f3c6d400700 1 -- 172.21.15.83:0/2566552093 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f3c540a3c90 msgr2=0x7f3c540a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1964818199 2024-04-23T14:48:22.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.300+0000 7f3c6d400700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f3c540a3c90 0x7f3c540a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T14:48:22.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.373+0000 7f3c74a00700 1 -- 172.21.15.83:0/2566552093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3c7002dfe0 con 0x7f3c70104f60 2024-04-23T14:48:22.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.374+0000 7f3c66a00700 1 -- 172.21.15.83:0/2566552093 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f3c6009cac0 con 0x7f3c70104f60 2024-04-23T14:48:22.375 INFO:teuthology.orchestra.run.smithi083.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T14:48:22.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.376+0000 7f3c74a00700 1 -- 172.21.15.83:0/2566552093 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f3c540a3c90 msgr2=0x7f3c540a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:48:22.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.376+0000 7f3c74a00700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f3c540a3c90 0x7f3c540a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:22.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.376+0000 7f3c74a00700 1 -- 172.21.15.83:0/2566552093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c70104f60 msgr2=0x7f3c70111a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:22.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.376+0000 7f3c74a00700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c70104f60 0x7f3c70111a60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000eb30 tx=0x7f3c6000ee40 comp rx=0 tx=0).stop 2024-04-23T14:48:22.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.377+0000 7f3c74a00700 1 -- 172.21.15.83:0/2566552093 shutdown_connections 2024-04-23T14:48:22.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.377+0000 7f3c74a00700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7f3c540a3c90 0x7f3c540a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:22.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.377+0000 7f3c74a00700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c7010a040 0x7f3c7010f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:22.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.377+0000 7f3c74a00700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c70104f60 0x7f3c70111a60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:22.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.377+0000 7f3c74a00700 1 --2- 172.21.15.83:0/2566552093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c70105910 0x7f3c7010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:22.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.377+0000 7f3c74a00700 1 -- 172.21.15.83:0/2566552093 >> 172.21.15.83:0/2566552093 conn(0x7f3c701007d0 msgr2=0x7f3c70101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:22.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.377+0000 7f3c74a00700 1 -- 172.21.15.83:0/2566552093 shutdown_connections 2024-04-23T14:48:22.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:22.377+0000 7f3c74a00700 1 -- 172.21.15.83:0/2566552093 wait complete. 2024-04-23T14:48:22.427 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:22 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:22.174+0000 7f7388758200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T14:48:22.427 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:22 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:22.169+0000 7f3e25074200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T14:48:22.447 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:22 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:22.182+0000 7f0b7dece200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T14:48:22.448 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:22 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2566552093' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T14:48:22.688 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:22 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2566552093' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T14:48:22.688 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:22 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:22.591+0000 7f3e25074200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T14:48:22.689 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:22 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2566552093' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T14:48:22.690 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:22 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:22.594+0000 7f7388758200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T14:48:22.718 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:22 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:22.445+0000 7f0b7dece200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T14:48:22.971 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:22 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:22.687+0000 7f3e25074200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T14:48:22.971 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:22 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:22.887+0000 7f3e25074200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T14:48:23.007 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:22 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:22.716+0000 7f0b7dece200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T14:48:23.076 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:22 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:22.689+0000 7f7388758200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T14:48:23.076 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:22 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:22.890+0000 7f7388758200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T14:48:23.169 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T14:48:23.170 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T14:48:23.170 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-23T14:48:23.170 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T14:48:23.170 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T14:48:23.170 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T14:48:23.170 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T14:48:23.170 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T14:48:23.170 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:48:23.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:48:23.170 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-23T14:48:23.211 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T14:48:23.211 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T14:48:23.211 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T14:48:23.211 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:48:23.211 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-23T14:48:23.211 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:48:23.211 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 41790sec preferred_lft 41790sec 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::ec4:7aff:fe8f:ce91/64 scope link 2024-04-23T14:48:23.212 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-23T14:48:23.213 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T14:48:23.213 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:48:23.214 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-23T14:48:23.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2024-04-23T14:48:23.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2024-04-23T14:48:23.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2024-04-23T14:48:23.214 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T14:48:23.214 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-23T14:48:23.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:48:23.617 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:23 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:23.558+0000 7f0b7dece200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T14:48:23.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:48:23.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:48:23.707 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T14:48:23.721 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:23 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:23.415+0000 7f3e25074200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T14:48:23.721 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:23 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:23.508+0000 7f3e25074200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T14:48:23.750 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:23 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:23.417+0000 7f7388758200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T14:48:23.750 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:23 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:23.508+0000 7f7388758200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T14:48:23.771 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-23T14:48:23.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:48:23.775 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-23T14:48:23.841 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-23T14:48:23.841 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T14:48:23.841 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T14:48:23.841 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T14:48:23.841 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T14:48:23.842 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:48:23.842 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-23T14:48:23.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T14:48:23.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T14:48:23.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T14:48:23.842 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-23T14:48:23.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:48:24.076 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:23 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:23.749+0000 7f7388758200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T14:48:24.076 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:23 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:23.883+0000 7f7388758200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T14:48:24.076 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:23 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:23.979+0000 7f7388758200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T14:48:24.099 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:23 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:23.751+0000 7f3e25074200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T14:48:24.180 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:23 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:23.883+0000 7f3e25074200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T14:48:24.180 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:23 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:23.979+0000 7f3e25074200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T14:48:24.210 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:24 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:24.159+0000 7f0b7dece200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T14:48:24.403 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:24 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:24.098+0000 7f3e25074200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T14:48:24.404 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:24 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:24.099+0000 7f7388758200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T14:48:24.506 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:24 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:24.291+0000 7f0b7dece200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T14:48:24.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:48:24.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:48:24.582 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-23T14:48:24.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:48:24.631 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-04-23T14:48:24.675 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:48:24.675 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-23T14:48:24.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:48:24.675 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-23T14:48:24.675 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-23T14:48:24.721 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:24 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:24.402+0000 7f3e25074200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T14:48:24.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:48:24.826 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:24 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:24.403+0000 7f7388758200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T14:48:24.847 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:24 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:24.566+0000 7f0b7dece200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T14:48:24.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:48:24.900 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-23T14:48:24.901 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:48:24.901 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-23T14:48:24.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-23T14:48:24.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T14:48:24.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-23T14:48:24.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-23T14:48:24.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-23T14:48:24.901 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-23T14:48:24.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:48:25.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:48:25.182 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T14:48:25.182 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-23T14:48:25.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-23T14:48:25.182 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T14:48:25.182 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-23T14:48:25.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:48:25.289 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:25.001+0000 7f3e25074200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T14:48:25.289 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z[34658]: 2024-04-23T14:48:25.216+0000 7f3e25074200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T14:48:25.289 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:25.001+0000 7f7388758200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T14:48:25.289 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y[33817]: 2024-04-23T14:48:25.213+0000 7f7388758200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T14:48:25.357 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:25.274+0000 7f0b7dece200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T14:48:25.358 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-mon[27867]: Standby manager daemon y restarted 2024-04-23T14:48:25.358 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-mon[27867]: Standby manager daemon y started 2024-04-23T14:48:25.358 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.173:0/4274211714' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T14:48:25.358 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.173:0/4274211714' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T14:48:25.358 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-mon[27867]: Standby manager daemon z restarted 2024-04-23T14:48:25.358 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-mon[27867]: Standby manager daemon z started 2024-04-23T14:48:25.358 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.173:0/4274211714' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T14:48:25.358 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.173:0/4274211714' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T14:48:25.358 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.203:0/2996434778' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T14:48:25.358 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.203:0/2996434778' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T14:48:25.358 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.203:0/2996434778' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T14:48:25.359 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-mon[27867]: from='mgr.? 172.21.15.203:0/2996434778' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T14:48:25.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:48:25.381 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:48:25.381 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T14:48:25.421 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T14:48:25.421 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T14:48:25.421 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:48:25.508 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-23T14:48:25.508 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-23T14:48:25.508 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/d1dd1b37-f1a3-4aae-99d8-e2fe8203f9d8 2024-04-23T14:48:25.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:48:25.508 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/d1dd1b37-f1a3-4aae-99d8-e2fe8203f9d8 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-23T14:48:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-mon[32443]: Standby manager daemon y restarted 2024-04-23T14:48:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-mon[32443]: Standby manager daemon y started 2024-04-23T14:48:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.173:0/4274211714' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T14:48:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.173:0/4274211714' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T14:48:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-mon[32443]: Standby manager daemon z restarted 2024-04-23T14:48:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-mon[32443]: Standby manager daemon z started 2024-04-23T14:48:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.173:0/4274211714' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T14:48:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.173:0/4274211714' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T14:48:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.203:0/2996434778' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T14:48:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.203:0/2996434778' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T14:48:25.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.203:0/2996434778' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T14:48:25.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:25 smithi173 ceph-mon[32443]: from='mgr.? 172.21.15.203:0/2996434778' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T14:48:25.656 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-23T14:48:25.657 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-23T14:48:25.657 INFO:teuthology.orchestra.run.smithi083.stdout:invalid new device string format 2024-04-23T14:48:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-mon[32800]: Standby manager daemon y restarted 2024-04-23T14:48:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-mon[32800]: Standby manager daemon y started 2024-04-23T14:48:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.173:0/4274211714' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T14:48:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.173:0/4274211714' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T14:48:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-mon[32800]: Standby manager daemon z restarted 2024-04-23T14:48:25.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-mon[32800]: Standby manager daemon z started 2024-04-23T14:48:25.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.173:0/4274211714' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T14:48:25.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.173:0/4274211714' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T14:48:25.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.203:0/2996434778' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T14:48:25.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.203:0/2996434778' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T14:48:25.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.203:0/2996434778' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T14:48:25.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:25 smithi203 ceph-mon[32800]: from='mgr.? 172.21.15.203:0/2996434778' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T14:48:25.746 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:25.412+0000 7f0b7dece200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T14:48:26.007 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:25.743+0000 7f0b7dece200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T14:48:26.007 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:25 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:25.931+0000 7f0b7dece200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T14:48:26.397 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:26 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:26.062+0000 7f0b7dece200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T14:48:26.398 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:26 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:26.219+0000 7f0b7dece200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T14:48:26.398 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:26 smithi083 ceph-mon[27867]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-23T14:48:26.421 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-23T14:48:26.421 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-23T14:48:26.421 INFO:teuthology.orchestra.run.smithi083.stdout:invalid new device string format 2024-04-23T14:48:26.421 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.83:3300,172.21.15.173:3300,172.21.15.203:3300" 2024-04-23T14:48:26.421 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.83:3300,172.21.15.173:3300,172.21.15.203:3300:/volumes/qa/sv_1/d1dd1b37-f1a3-4aae-99d8-e2fe8203f9d8 2024-04-23T14:48:26.421 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=5f4fe708-017e-11ef-bc93-c7b262605968" will pass to kernel 2024-04-23T14:48:26.426 INFO:teuthology.orchestra.run.smithi083.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-23T14:48:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: You just mounted a file system that supports labels which does not 2024-04-23T14:48:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-23T14:48:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: applications will generate AVC messages and not be allowed access to 2024-04-23T14:48:26.426 INFO:teuthology.orchestra.run.smithi083.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-23T14:48:26.430 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-23T14:48:26.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:48:26.430 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T14:48:26.508 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T14:48:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:26 smithi173 ceph-mon[32443]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-23T14:48:26.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T14:48:26.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:48:26.617 INFO:teuthology.orchestra.run.smithi083.stdout:client25235 2024-04-23T14:48:26.617 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:48:26.618 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:48:26.668 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T14:48:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:26 smithi203 ceph-mon[32800]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-23T14:48:26.756 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:26 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:26.604+0000 7f0b7dece200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T14:48:26.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T14:48:26.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:48:26.818 INFO:teuthology.orchestra.run.smithi083.stdout:client25235 2024-04-23T14:48:26.818 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T14:48:26.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T14:48:26.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:48:26.930 INFO:teuthology.orchestra.run.smithi083.stdout:client25235 2024-04-23T14:48:26.930 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5f4fe708-017e-11ef-bc93-c7b262605968.client25235/status') 2024-04-23T14:48:27.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T14:48:27.054 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.25235 (3)172.21.15.83:0/2891100250 2024-04-23T14:48:27.054 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2024-04-23T14:48:27.054 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2024-04-23T14:48:27.054 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2024-04-23T14:48:27.055 INFO:teuthology.orchestra.run.smithi083.stderr:71 bytes copied, 0.000102046 s, 696 kB/s 2024-04-23T14:48:27.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T14:48:27.066 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T14:48:27.076 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T14:48:27.076 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-23T14:48:27.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-23T14:48:27.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.269+0000 7fb5fb200640 1 -- 172.21.15.83:0/4192751568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc103070 msgr2=0x7fb5fc1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.269+0000 7fb5fb200640 1 --2- 172.21.15.83:0/4192751568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc103070 0x7fb5fc1034f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009860 tx=0x7fb5e402a340 comp rx=0 tx=0).stop 2024-04-23T14:48:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.269+0000 7fb5fb200640 1 -- 172.21.15.83:0/4192751568 shutdown_connections 2024-04-23T14:48:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.269+0000 7fb5fb200640 1 --2- 172.21.15.83:0/4192751568 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5fc103a30 0x7fb5fc10a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.269+0000 7fb5fb200640 1 --2- 172.21.15.83:0/4192751568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc103070 0x7fb5fc1034f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.269+0000 7fb5fb200640 1 --2- 172.21.15.83:0/4192751568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc101f20 0x7fb5fc102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.269+0000 7fb5fb200640 1 -- 172.21.15.83:0/4192751568 >> 172.21.15.83:0/4192751568 conn(0x7fb5fc0fd490 msgr2=0x7fb5fc0ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.269+0000 7fb5fb200640 1 -- 172.21.15.83:0/4192751568 shutdown_connections 2024-04-23T14:48:27.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.269+0000 7fb5fb200640 1 -- 172.21.15.83:0/4192751568 wait complete. 2024-04-23T14:48:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.270+0000 7fb5fb200640 1 Processor -- start 2024-04-23T14:48:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.270+0000 7fb5fb200640 1 -- start start 2024-04-23T14:48:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.271+0000 7fb5fb200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5fc101f20 0x7fb5fc19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.271+0000 7fb5fb200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc103070 0x7fb5fc19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:27.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.271+0000 7fb5fb200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc103a30 0x7fb5fc1a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:27.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.271+0000 7fb5fb200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc10db40 con 0x7fb5fc103a30 2024-04-23T14:48:27.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.271+0000 7fb5fb200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc10d9c0 con 0x7fb5fc103070 2024-04-23T14:48:27.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.271+0000 7fb5fb200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc10dcc0 con 0x7fb5fc101f20 2024-04-23T14:48:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.271+0000 7fb5fa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc103a30 0x7fb5fc1a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.271+0000 7fb5f9e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5fc101f20 0x7fb5fc19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.271+0000 7fb5f3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc103070 0x7fb5fc19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.271+0000 7fb5f9e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5fc101f20 0x7fb5fc19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41392/0 (socket says 172.21.15.83:41392) 2024-04-23T14:48:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.271+0000 7fb5fa800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc103a30 0x7fb5fc1a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48702/0 (socket says 172.21.15.83:48702) 2024-04-23T14:48:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.271+0000 7fb5f9e00640 1 -- 172.21.15.83:0/2861920564 learned_addr learned my addr 172.21.15.83:0/2861920564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.272+0000 7fb5f3e00640 1 -- 172.21.15.83:0/2861920564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5fc101f20 msgr2=0x7fb5fc19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.272+0000 7fb5f3e00640 1 --2- 172.21.15.83:0/2861920564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5fc101f20 0x7fb5fc19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.272+0000 7fb5f3e00640 1 -- 172.21.15.83:0/2861920564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc103a30 msgr2=0x7fb5fc1a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.272+0000 7fb5f3e00640 1 --2- 172.21.15.83:0/2861920564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc103a30 0x7fb5fc1a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.272+0000 7fb5f3e00640 1 -- 172.21.15.83:0/2861920564 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5fc1a1fc0 con 0x7fb5fc103070 2024-04-23T14:48:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.272+0000 7fb5f3e00640 1 --2- 172.21.15.83:0/2861920564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc103070 0x7fb5fc19a8e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4035040 tx=0x7fb5e4033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.272+0000 7fb5f2a00640 1 -- 172.21.15.83:0/2861920564 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4037040 con 0x7fb5fc103070 2024-04-23T14:48:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.272+0000 7fb5f2a00640 1 -- 172.21.15.83:0/2861920564 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e4004040 con 0x7fb5fc103070 2024-04-23T14:48:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.272+0000 7fb5fb200640 1 -- 172.21.15.83:0/2861920564 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5fc1a2250 con 0x7fb5fc103070 2024-04-23T14:48:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.272+0000 7fb5f2a00640 1 -- 172.21.15.83:0/2861920564 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4005610 con 0x7fb5fc103070 2024-04-23T14:48:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.273+0000 7fb5fb200640 1 -- 172.21.15.83:0/2861920564 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5fc1a2740 con 0x7fb5fc103070 2024-04-23T14:48:27.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.274+0000 7fb5fb200640 1 -- 172.21.15.83:0/2861920564 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5fc1a29a0 con 0x7fb5fc103070 2024-04-23T14:48:27.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.276+0000 7fb5f2a00640 1 -- 172.21.15.83:0/2861920564 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7fb5e4042020 con 0x7fb5fc103070 2024-04-23T14:48:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.278+0000 7fb5f2a00640 1 --2- 172.21.15.83:0/2861920564 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5e00a3bf0 0x7fb5e00a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.278+0000 7fb5f9e00640 1 -- 172.21.15.83:0/2861920564 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5e00a3bf0 msgr2=0x7fb5e00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1964818199 2024-04-23T14:48:27.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.278+0000 7fb5f2a00640 1 -- 172.21.15.83:0/2861920564 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb5e40efd40 con 0x7fb5fc103070 2024-04-23T14:48:27.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.278+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/2861920564 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5e00a3bf0 0x7fb5e00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T14:48:27.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.280+0000 7fb5f2a00640 1 -- 172.21.15.83:0/2861920564 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5e4049020 con 0x7fb5fc103070 2024-04-23T14:48:27.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.479+0000 7fb5f9e00640 1 -- 172.21.15.83:0/2861920564 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5e00a3bf0 msgr2=0x7fb5e00a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.83:6800/1964818199 2024-04-23T14:48:27.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.479+0000 7fb5f9e00640 1 --2- 172.21.15.83:0/2861920564 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5e00a3bf0 0x7fb5e00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T14:48:27.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.522+0000 7fb5fb200640 1 -- 172.21.15.83:0/2861920564 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fb5fc1a2d90 con 0x7fb5fc103070 2024-04-23T14:48:27.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.581+0000 7fb5f2a00640 1 -- 172.21.15.83:0/2861920564 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7fb5e40bc6e0 con 0x7fb5fc103070 2024-04-23T14:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.584+0000 7fb5fb200640 1 -- 172.21.15.83:0/2861920564 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5e00a3bf0 msgr2=0x7fb5e00a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.584+0000 7fb5fb200640 1 --2- 172.21.15.83:0/2861920564 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5e00a3bf0 0x7fb5e00a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.584+0000 7fb5fb200640 1 -- 172.21.15.83:0/2861920564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc103070 msgr2=0x7fb5fc19a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.584+0000 7fb5fb200640 1 --2- 172.21.15.83:0/2861920564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc103070 0x7fb5fc19a8e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4035040 tx=0x7fb5e4033d90 comp rx=0 tx=0).stop 2024-04-23T14:48:27.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.584+0000 7fb5fb200640 1 -- 172.21.15.83:0/2861920564 shutdown_connections 2024-04-23T14:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.584+0000 7fb5fb200640 1 --2- 172.21.15.83:0/2861920564 >> [v2:172.21.15.83:6800/1964818199,v1:172.21.15.83:6801/1964818199] conn(0x7fb5e00a3bf0 0x7fb5e00a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.584+0000 7fb5fb200640 1 --2- 172.21.15.83:0/2861920564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb5fc103a30 0x7fb5fc1a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.585+0000 7fb5fb200640 1 --2- 172.21.15.83:0/2861920564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5fc103070 0x7fb5fc19a8e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.585+0000 7fb5fb200640 1 --2- 172.21.15.83:0/2861920564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5fc101f20 0x7fb5fc19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.585+0000 7fb5fb200640 1 -- 172.21.15.83:0/2861920564 >> 172.21.15.83:0/2861920564 conn(0x7fb5fc0fd490 msgr2=0x7fb5fc074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.585+0000 7fb5fb200640 1 -- 172.21.15.83:0/2861920564 shutdown_connections 2024-04-23T14:48:27.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.585+0000 7fb5fb200640 1 -- 172.21.15.83:0/2861920564 wait complete. 2024-04-23T14:48:27.603 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-23T14:48:27.713 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:27 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:27.412+0000 7f0b7dece200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T14:48:27.714 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:27 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2861920564' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T14:48:27.714 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:27 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T14:48:27.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.783+0000 7fd463200640 1 -- 172.21.15.83:0/1530554637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4641039a0 msgr2=0x7fd46410a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.783+0000 7fd463200640 1 --2- 172.21.15.83:0/1530554637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4641039a0 0x7fd46410a2c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009ca0 tx=0x7fd45c02a8c0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.783+0000 7fd463200640 1 -- 172.21.15.83:0/1530554637 shutdown_connections 2024-04-23T14:48:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.783+0000 7fd463200640 1 --2- 172.21.15.83:0/1530554637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4641039a0 0x7fd46410a2c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.783+0000 7fd463200640 1 --2- 172.21.15.83:0/1530554637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd464102fe0 0x7fd464103460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.783+0000 7fd463200640 1 --2- 172.21.15.83:0/1530554637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd464101e90 0x7fd4641022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.783+0000 7fd463200640 1 -- 172.21.15.83:0/1530554637 >> 172.21.15.83:0/1530554637 conn(0x7fd4640fd400 msgr2=0x7fd4640ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.783+0000 7fd463200640 1 -- 172.21.15.83:0/1530554637 shutdown_connections 2024-04-23T14:48:27.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.783+0000 7fd463200640 1 -- 172.21.15.83:0/1530554637 wait complete. 2024-04-23T14:48:27.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.784+0000 7fd463200640 1 Processor -- start 2024-04-23T14:48:27.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.784+0000 7fd463200640 1 -- start start 2024-04-23T14:48:27.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.784+0000 7fd463200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd464101e90 0x7fd46419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.784+0000 7fd463200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464102fe0 0x7fd46419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.784+0000 7fd463200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4641039a0 0x7fd4641a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.784+0000 7fd463200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd46410db70 con 0x7fd464102fe0 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.784+0000 7fd463200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd46410d9f0 con 0x7fd4641039a0 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.784+0000 7fd463200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd46410dcf0 con 0x7fd464101e90 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.784+0000 7fd461400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464102fe0 0x7fd46419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.784+0000 7fd461400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464102fe0 0x7fd46419a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48736/0 (socket says 172.21.15.83:48736) 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.784+0000 7fd461400640 1 -- 172.21.15.83:0/774535427 learned_addr learned my addr 172.21.15.83:0/774535427 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.785+0000 7fd461e00640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd464101e90 0x7fd46419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.785+0000 7fd462800640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4641039a0 0x7fd4641a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.785+0000 7fd461400640 1 -- 172.21.15.83:0/774535427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd464101e90 msgr2=0x7fd46419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.785+0000 7fd461400640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd464101e90 0x7fd46419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.785+0000 7fd461400640 1 -- 172.21.15.83:0/774535427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4641039a0 msgr2=0x7fd4641a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.785+0000 7fd461400640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4641039a0 0x7fd4641a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.785+0000 7fd461400640 1 -- 172.21.15.83:0/774535427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4641a1f50 con 0x7fd464102fe0 2024-04-23T14:48:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.785+0000 7fd461400640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464102fe0 0x7fd46419a870 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e9c0 tx=0x7fd45000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.785+0000 7fd45aa00640 1 -- 172.21.15.83:0/774535427 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45000cd80 con 0x7fd464102fe0 2024-04-23T14:48:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.785+0000 7fd45aa00640 1 -- 172.21.15.83:0/774535427 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4500035f0 con 0x7fd464102fe0 2024-04-23T14:48:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.785+0000 7fd463200640 1 -- 172.21.15.83:0/774535427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4641a2240 con 0x7fd464102fe0 2024-04-23T14:48:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.786+0000 7fd45aa00640 1 -- 172.21.15.83:0/774535427 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd450010690 con 0x7fd464102fe0 2024-04-23T14:48:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.786+0000 7fd463200640 1 -- 172.21.15.83:0/774535427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4641a27e0 con 0x7fd464102fe0 2024-04-23T14:48:27.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.787+0000 7fd463200640 1 -- 172.21.15.83:0/774535427 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4641a2aa0 con 0x7fd464102fe0 2024-04-23T14:48:27.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.789+0000 7fd45aa00640 1 -- 172.21.15.83:0/774535427 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fd4500037a0 con 0x7fd464102fe0 2024-04-23T14:48:27.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.791+0000 7fd45aa00640 1 -- 172.21.15.83:0/774535427 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd4500d0340 con 0x7fd464102fe0 2024-04-23T14:48:27.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:27.795+0000 7fd45aa00640 1 -- 172.21.15.83:0/774535427 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd45009cfb0 con 0x7fd464102fe0 2024-04-23T14:48:27.909 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:27 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2861920564' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T14:48:27.909 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:27 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T14:48:27.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:27 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2861920564' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T14:48:27.910 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:27 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T14:48:28.006 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:27 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:27.711+0000 7f0b7dece200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T14:48:28.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.040+0000 7f0ad2e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.040+0000 7f0ad2e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.040+0000 7f0ad2e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.508 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.041+0000 7f0ad2e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.508 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.041+0000 7f0ad2e00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.508 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.041+0000 7f0ad0600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.508 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.041+0000 7f0ad0600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.508 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.041+0000 7f0ad0600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.508 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.041+0000 7f0ad0600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.508 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.041+0000 7f0ad0600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.747+0000 7fd45aa00640 1 -- 172.21.15.83:0/774535427 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fd450098600 con 0x7fd464102fe0 2024-04-23T14:48:28.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.748+0000 7fd45aa00640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd4480aa9a0 0x7fd4480acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:28.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.748+0000 7fd45aa00640 1 -- 172.21.15.83:0/774535427 --> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fd44801dcc0 con 0x7fd4480aa9a0 2024-04-23T14:48:28.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.753+0000 7fd461e00640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd4480aa9a0 0x7fd4480acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:28.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.765+0000 7fd461e00640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd4480aa9a0 0x7fd4480acd90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd454009bd0 tx=0x7fd454005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:28.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.792+0000 7fd45aa00640 1 -- 172.21.15.83:0/774535427 <== mgr.15114 v2:172.21.15.83:6800/1667121447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fd44801dcc0 con 0x7fd4480aa9a0 2024-04-23T14:48:28.793 INFO:teuthology.orchestra.run.smithi083.stdout:{} 2024-04-23T14:48:28.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.794+0000 7fd463200640 1 -- 172.21.15.83:0/774535427 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd4480aa9a0 msgr2=0x7fd4480acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.794+0000 7fd463200640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd4480aa9a0 0x7fd4480acd90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd454009bd0 tx=0x7fd454005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.794+0000 7fd463200640 1 -- 172.21.15.83:0/774535427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464102fe0 msgr2=0x7fd46419a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.794+0000 7fd463200640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464102fe0 0x7fd46419a870 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e9c0 tx=0x7fd45000ee80 comp rx=0 tx=0).stop 2024-04-23T14:48:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.794+0000 7fd463200640 1 -- 172.21.15.83:0/774535427 shutdown_connections 2024-04-23T14:48:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.794+0000 7fd463200640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4641039a0 0x7fd4641a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.794+0000 7fd463200640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd4480aa9a0 0x7fd4480acd90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.794+0000 7fd463200640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd464102fe0 0x7fd46419a870 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.794+0000 7fd463200640 1 --2- 172.21.15.83:0/774535427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd464101e90 0x7fd46419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.794+0000 7fd463200640 1 -- 172.21.15.83:0/774535427 >> 172.21.15.83:0/774535427 conn(0x7fd4640fd400 msgr2=0x7fd464074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.794+0000 7fd463200640 1 -- 172.21.15.83:0/774535427 shutdown_connections 2024-04-23T14:48:28.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:28.794+0000 7fd463200640 1 -- 172.21.15.83:0/774535427 wait complete. 2024-04-23T14:48:28.813 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-23T14:48:28.823 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-23T14:48:28.833 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:48:28.888 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.778+0000 7f0acca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.888 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.778+0000 7f0acca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.888 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.778+0000 7f0acca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.888 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.778+0000 7f0acca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.888 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:48:28.778+0000 7f0acca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:48:28.888 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T14:48:28.888 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: Active manager daemon x restarted 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: Activating manager daemon x 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: mgrmap e21: x(active, starting, since 0.0152914s), standbys: z, y 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T14:48:28.889 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:48:28.890 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: Manager daemon x is now available 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T14:48:28.891 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:28 smithi083 ceph-mon[27867]: [23/Apr/2024:14:48:28] ENGINE Bus STARTING 2024-04-23T14:48:28.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T14:48:28.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:48:28.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: Active manager daemon x restarted 2024-04-23T14:48:28.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: Activating manager daemon x 2024-04-23T14:48:28.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T14:48:28.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: mgrmap e21: x(active, starting, since 0.0152914s), standbys: z, y 2024-04-23T14:48:28.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:48:28.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:48:28.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:48:28.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T14:48:28.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T14:48:28.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T14:48:28.975 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T14:48:28.975 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: Manager daemon x is now available 2024-04-23T14:48:28.975 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:48:28.975 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T14:48:28.975 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T14:48:28.975 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T14:48:28.975 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T14:48:28.975 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:28 smithi203 ceph-mon[32800]: [23/Apr/2024:14:48:28] ENGINE Bus STARTING 2024-04-23T14:48:29.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.013+0000 7fc1b2a00640 1 -- 172.21.15.83:0/3168241648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ac102900 msgr2=0x7fc1ac104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.013+0000 7fc1b2a00640 1 --2- 172.21.15.83:0/3168241648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ac102900 0x7fc1ac104cf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009900 tx=0x7fc1a002a3f0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.013+0000 7fc1b2a00640 1 -- 172.21.15.83:0/3168241648 shutdown_connections 2024-04-23T14:48:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.013+0000 7fc1b2a00640 1 --2- 172.21.15.83:0/3168241648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1ac107b60 0x7fc1ac109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.013+0000 7fc1b2a00640 1 --2- 172.21.15.83:0/3168241648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac105230 0x7fc1ac107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.013+0000 7fc1b2a00640 1 --2- 172.21.15.83:0/3168241648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ac102900 0x7fc1ac104cf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.013+0000 7fc1b2a00640 1 -- 172.21.15.83:0/3168241648 >> 172.21.15.83:0/3168241648 conn(0x7fc1ac0fc310 msgr2=0x7fc1ac0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.013+0000 7fc1b2a00640 1 -- 172.21.15.83:0/3168241648 shutdown_connections 2024-04-23T14:48:29.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.013+0000 7fc1b2a00640 1 -- 172.21.15.83:0/3168241648 wait complete. 2024-04-23T14:48:29.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.014+0000 7fc1b2a00640 1 Processor -- start 2024-04-23T14:48:29.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.014+0000 7fc1b2a00640 1 -- start start 2024-04-23T14:48:29.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.014+0000 7fc1b2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac102900 0x7fc1ac19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:29.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.014+0000 7fc1b2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ac105230 0x7fc1ac19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:29.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.014+0000 7fc1b2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1ac107b60 0x7fc1ac1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.014+0000 7fc1b2a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac10db70 con 0x7fc1ac102900 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.014+0000 7fc1b2a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac10d9f0 con 0x7fc1ac105230 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.014+0000 7fc1b2a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac10dcf0 con 0x7fc1ac107b60 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.014+0000 7fc1b0c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ac105230 0x7fc1ac19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.015+0000 7fc1b1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac102900 0x7fc1ac19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.015+0000 7fc1b0c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ac105230 0x7fc1ac19a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:43312/0 (socket says 172.21.15.83:43312) 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.015+0000 7fc1b0c00640 1 -- 172.21.15.83:0/1921317582 learned_addr learned my addr 172.21.15.83:0/1921317582 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.015+0000 7fc1b2000640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1ac107b60 0x7fc1ac1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.015+0000 7fc1b0c00640 1 -- 172.21.15.83:0/1921317582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1ac107b60 msgr2=0x7fc1ac1a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.015+0000 7fc1b0c00640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1ac107b60 0x7fc1ac1a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.015+0000 7fc1b0c00640 1 -- 172.21.15.83:0/1921317582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac102900 msgr2=0x7fc1ac19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.015+0000 7fc1b0c00640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac102900 0x7fc1ac19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.015+0000 7fc1b0c00640 1 -- 172.21.15.83:0/1921317582 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac1a1fa0 con 0x7fc1ac105230 2024-04-23T14:48:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.015+0000 7fc1b0c00640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ac105230 0x7fc1ac19a8c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc19800e9a0 tx=0x7fc19800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.016+0000 7fc1aa000640 1 -- 172.21.15.83:0/1921317582 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19800cd80 con 0x7fc1ac105230 2024-04-23T14:48:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.016+0000 7fc1aa000640 1 -- 172.21.15.83:0/1921317582 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1980035f0 con 0x7fc1ac105230 2024-04-23T14:48:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.016+0000 7fc1b2a00640 1 -- 172.21.15.83:0/1921317582 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac1a2290 con 0x7fc1ac105230 2024-04-23T14:48:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.016+0000 7fc1aa000640 1 -- 172.21.15.83:0/1921317582 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198010670 con 0x7fc1ac105230 2024-04-23T14:48:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.016+0000 7fc1b2a00640 1 -- 172.21.15.83:0/1921317582 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac1a2830 con 0x7fc1ac105230 2024-04-23T14:48:29.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.017+0000 7fc1b2a00640 1 -- 172.21.15.83:0/1921317582 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ac1096b0 con 0x7fc1ac105230 2024-04-23T14:48:29.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.019+0000 7fc1aa000640 1 -- 172.21.15.83:0/1921317582 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fc1980037a0 con 0x7fc1ac105230 2024-04-23T14:48:29.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.021+0000 7fc1aa000640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1940a3a30 0x7fc1940a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:29.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.021+0000 7fc1aa000640 1 -- 172.21.15.83:0/1921317582 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc1980d03d0 con 0x7fc1ac105230 2024-04-23T14:48:29.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.021+0000 7fc1b1600640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1940a3a30 0x7fc1940a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:29.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.022+0000 7fc1b1600640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1940a3a30 0x7fc1940a5ef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0031440 tx=0x7fc1a00315b0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:29.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.025+0000 7fc1aa000640 1 -- 172.21.15.83:0/1921317582 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc19809cf50 con 0x7fc1ac105230 2024-04-23T14:48:29.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T14:48:29.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:48:29.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: Active manager daemon x restarted 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: Activating manager daemon x 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: mgrmap e21: x(active, starting, since 0.0152914s), standbys: z, y 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T14:48:29.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T14:48:29.078 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: Manager daemon x is now available 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T14:48:29.079 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T14:48:29.080 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T14:48:29.080 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:28 smithi173 ceph-mon[32443]: [23/Apr/2024:14:48:28] ENGINE Bus STARTING 2024-04-23T14:48:29.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.240+0000 7fc1b2a00640 1 -- 172.21.15.83:0/1921317582 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1ac062c00 con 0x7fc1ac105230 2024-04-23T14:48:29.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.242+0000 7fc1aa000640 1 -- 172.21.15.83:0/1921317582 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23076 (secure 0 0 0) 0x7fc19809c850 con 0x7fc1ac105230 2024-04-23T14:48:29.244 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:29.244 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:48:27.715285+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/622708991":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/1081276647":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/4179157882":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6801/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/2255156114":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:48:29.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.245+0000 7fc1b2a00640 1 -- 172.21.15.83:0/1921317582 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1940a3a30 msgr2=0x7fc1940a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.245+0000 7fc1b2a00640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1940a3a30 0x7fc1940a5ef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0031440 tx=0x7fc1a00315b0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.245+0000 7fc1b2a00640 1 -- 172.21.15.83:0/1921317582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ac105230 msgr2=0x7fc1ac19a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.245+0000 7fc1b2a00640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ac105230 0x7fc1ac19a8c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc19800e9a0 tx=0x7fc19800ee60 comp rx=0 tx=0).stop 2024-04-23T14:48:29.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.245+0000 7fc1b2a00640 1 -- 172.21.15.83:0/1921317582 shutdown_connections 2024-04-23T14:48:29.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.245+0000 7fc1b2a00640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1ac107b60 0x7fc1ac1a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.245+0000 7fc1b2a00640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc1ac105230 0x7fc1ac19a8c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.245+0000 7fc1b2a00640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1940a3a30 0x7fc1940a5ef0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.245+0000 7fc1b2a00640 1 --2- 172.21.15.83:0/1921317582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1ac102900 0x7fc1ac19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.245+0000 7fc1b2a00640 1 -- 172.21.15.83:0/1921317582 >> 172.21.15.83:0/1921317582 conn(0x7fc1ac0fc310 msgr2=0x7fc1ac102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:29.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.245+0000 7fc1b2a00640 1 -- 172.21.15.83:0/1921317582 shutdown_connections 2024-04-23T14:48:29.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.245+0000 7fc1b2a00640 1 -- 172.21.15.83:0/1921317582 wait complete. 2024-04-23T14:48:29.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:40:41.241992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:45:57.341602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:45:58.654798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:48:29.265 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:48:29.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.455+0000 7fb80dc00640 1 -- 172.21.15.83:0/3580336445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb808102040 msgr2=0x7fb808102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.455+0000 7fb80dc00640 1 --2- 172.21.15.83:0/3580336445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb808102040 0x7fb808102460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009900 tx=0x7fb7f802a410 comp rx=0 tx=0).stop 2024-04-23T14:48:29.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.455+0000 7fb80dc00640 1 -- 172.21.15.83:0/3580336445 shutdown_connections 2024-04-23T14:48:29.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.455+0000 7fb80dc00640 1 --2- 172.21.15.83:0/3580336445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb808103bf0 0x7fb80810a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.455+0000 7fb80dc00640 1 --2- 172.21.15.83:0/3580336445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb808103230 0x7fb8081036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.455+0000 7fb80dc00640 1 --2- 172.21.15.83:0/3580336445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb808102040 0x7fb808102460 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.455+0000 7fb80dc00640 1 -- 172.21.15.83:0/3580336445 >> 172.21.15.83:0/3580336445 conn(0x7fb8080fd5b0 msgr2=0x7fb8080ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:29.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.455+0000 7fb80dc00640 1 -- 172.21.15.83:0/3580336445 shutdown_connections 2024-04-23T14:48:29.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.455+0000 7fb80dc00640 1 -- 172.21.15.83:0/3580336445 wait complete. 2024-04-23T14:48:29.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.456+0000 7fb80dc00640 1 Processor -- start 2024-04-23T14:48:29.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.456+0000 7fb80dc00640 1 -- start start 2024-04-23T14:48:29.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.456+0000 7fb80dc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb808102040 0x7fb80819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:29.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.456+0000 7fb80dc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb808103230 0x7fb80819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:29.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.456+0000 7fb80dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb808103bf0 0x7fb8081a1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.456+0000 7fb80dc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb80810dcd0 con 0x7fb808103bf0 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.456+0000 7fb80dc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb80810db50 con 0x7fb808102040 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.456+0000 7fb80dc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb80810de50 con 0x7fb808103230 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.456+0000 7fb806a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb808102040 0x7fb80819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.457+0000 7fb806a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb808102040 0x7fb80819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:43326/0 (socket says 172.21.15.83:43326) 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.457+0000 7fb806a00640 1 -- 172.21.15.83:0/2246272810 learned_addr learned my addr 172.21.15.83:0/2246272810 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.457+0000 7fb807400640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb808103bf0 0x7fb8081a1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.457+0000 7fb806000640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb808103230 0x7fb80819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.457+0000 7fb806a00640 1 -- 172.21.15.83:0/2246272810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb808103230 msgr2=0x7fb80819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.457+0000 7fb806a00640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb808103230 0x7fb80819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.457+0000 7fb806a00640 1 -- 172.21.15.83:0/2246272810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb808103bf0 msgr2=0x7fb8081a1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.457+0000 7fb806a00640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb808103bf0 0x7fb8081a1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.457+0000 7fb806a00640 1 -- 172.21.15.83:0/2246272810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8081a21e0 con 0x7fb808102040 2024-04-23T14:48:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.457+0000 7fb806a00640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb808102040 0x7fb80819a610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8031610 tx=0x7fb7f8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.457+0000 7fb7ff400640 1 -- 172.21.15.83:0/2246272810 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f803f030 con 0x7fb808102040 2024-04-23T14:48:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.458+0000 7fb7ff400640 1 -- 172.21.15.83:0/2246272810 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f8003ad0 con 0x7fb808102040 2024-04-23T14:48:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.458+0000 7fb80dc00640 1 -- 172.21.15.83:0/2246272810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8081a2470 con 0x7fb808102040 2024-04-23T14:48:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.458+0000 7fb7ff400640 1 -- 172.21.15.83:0/2246272810 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8035070 con 0x7fb808102040 2024-04-23T14:48:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.458+0000 7fb80dc00640 1 -- 172.21.15.83:0/2246272810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8081a2960 con 0x7fb808102040 2024-04-23T14:48:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.459+0000 7fb80dc00640 1 -- 172.21.15.83:0/2246272810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8081a2bc0 con 0x7fb808102040 2024-04-23T14:48:29.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.461+0000 7fb7ff400640 1 -- 172.21.15.83:0/2246272810 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fb7f8042030 con 0x7fb808102040 2024-04-23T14:48:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.462+0000 7fb7ff400640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb7ec0a39b0 0x7fb7ec0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.462+0000 7fb7ff400640 1 -- 172.21.15.83:0/2246272810 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb7f80f0700 con 0x7fb808102040 2024-04-23T14:48:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.462+0000 7fb806000640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb7ec0a39b0 0x7fb7ec0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.463+0000 7fb806000640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb7ec0a39b0 0x7fb7ec0a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40095f0 tx=0x7fb7f4005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.466+0000 7fb7ff400640 1 -- 172.21.15.83:0/2246272810 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7f80bd280 con 0x7fb808102040 2024-04-23T14:48:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.664+0000 7fb80dc00640 1 -- 172.21.15.83:0/2246272810 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8081a2d90 con 0x7fb808102040 2024-04-23T14:48:29.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.664+0000 7fb7ff400640 1 -- 172.21.15.83:0/2246272810 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fb7f80bcb80 con 0x7fb808102040 2024-04-23T14:48:29.666 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-23T14:48:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.667+0000 7fb80dc00640 1 -- 172.21.15.83:0/2246272810 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb7ec0a39b0 msgr2=0x7fb7ec0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.667+0000 7fb80dc00640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb7ec0a39b0 0x7fb7ec0a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40095f0 tx=0x7fb7f4005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:29.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.667+0000 7fb80dc00640 1 -- 172.21.15.83:0/2246272810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb808102040 msgr2=0x7fb80819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.667+0000 7fb80dc00640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb808102040 0x7fb80819a610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8031610 tx=0x7fb7f8004270 comp rx=0 tx=0).stop 2024-04-23T14:48:29.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.667+0000 7fb80dc00640 1 -- 172.21.15.83:0/2246272810 shutdown_connections 2024-04-23T14:48:29.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.667+0000 7fb80dc00640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb7ec0a39b0 0x7fb7ec0a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.667+0000 7fb80dc00640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb808103bf0 0x7fb8081a1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.667+0000 7fb80dc00640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb808103230 0x7fb80819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.667+0000 7fb80dc00640 1 --2- 172.21.15.83:0/2246272810 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb808102040 0x7fb80819a610 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.667+0000 7fb80dc00640 1 -- 172.21.15.83:0/2246272810 >> 172.21.15.83:0/2246272810 conn(0x7fb8080fd5b0 msgr2=0x7fb808108a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:29.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.667+0000 7fb80dc00640 1 -- 172.21.15.83:0/2246272810 shutdown_connections 2024-04-23T14:48:29.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.667+0000 7fb80dc00640 1 -- 172.21.15.83:0/2246272810 wait complete. 2024-04-23T14:48:29.685 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:48:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.862+0000 7fafa4000640 1 -- 172.21.15.83:0/2581681268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf9c103240 msgr2=0x7faf9c1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.862+0000 7fafa4000640 1 --2- 172.21.15.83:0/2581681268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf9c103240 0x7faf9c1036c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faf88009900 tx=0x7faf8802a410 comp rx=0 tx=0).stop 2024-04-23T14:48:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.862+0000 7fafa4000640 1 -- 172.21.15.83:0/2581681268 shutdown_connections 2024-04-23T14:48:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.862+0000 7fafa4000640 1 --2- 172.21.15.83:0/2581681268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf9c103c00 0x7faf9c10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.862+0000 7fafa4000640 1 --2- 172.21.15.83:0/2581681268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf9c103240 0x7faf9c1036c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.862+0000 7fafa4000640 1 --2- 172.21.15.83:0/2581681268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf9c102050 0x7faf9c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.862+0000 7fafa4000640 1 -- 172.21.15.83:0/2581681268 >> 172.21.15.83:0/2581681268 conn(0x7faf9c0fd5e0 msgr2=0x7faf9c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:29.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.862+0000 7fafa4000640 1 -- 172.21.15.83:0/2581681268 shutdown_connections 2024-04-23T14:48:29.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.863+0000 7fafa4000640 1 -- 172.21.15.83:0/2581681268 wait complete. 2024-04-23T14:48:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.863+0000 7fafa4000640 1 Processor -- start 2024-04-23T14:48:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.864+0000 7fafa4000640 1 -- start start 2024-04-23T14:48:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.864+0000 7fafa4000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf9c102050 0x7faf9c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.864+0000 7fafa4000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf9c103240 0x7faf9c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.864+0000 7fafa4000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf9c103c00 0x7faf9c19f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.864+0000 7fafa4000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c10dd30 con 0x7faf9c103c00 2024-04-23T14:48:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.864+0000 7fafa4000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c10dbb0 con 0x7faf9c102050 2024-04-23T14:48:29.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.864+0000 7fafa4000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c10deb0 con 0x7faf9c103240 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.864+0000 7fafa1600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf9c102050 0x7faf9c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.864+0000 7fafa0c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf9c103240 0x7faf9c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.864+0000 7fafa1600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf9c102050 0x7faf9c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:43364/0 (socket says 172.21.15.83:43364) 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.864+0000 7fafa1600640 1 -- 172.21.15.83:0/647971860 learned_addr learned my addr 172.21.15.83:0/647971860 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.864+0000 7fafa2000640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf9c103c00 0x7faf9c19f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.865+0000 7fafa1600640 1 -- 172.21.15.83:0/647971860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf9c103240 msgr2=0x7faf9c1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.865+0000 7fafa1600640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf9c103240 0x7faf9c1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.865+0000 7fafa1600640 1 -- 172.21.15.83:0/647971860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf9c103c00 msgr2=0x7faf9c19f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.865+0000 7fafa1600640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf9c103c00 0x7faf9c19f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.865+0000 7fafa1600640 1 -- 172.21.15.83:0/647971860 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9c19ffa0 con 0x7faf9c102050 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.865+0000 7fafa1600640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf9c102050 0x7faf9c198380 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf9000e9c0 tx=0x7faf9000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.865+0000 7faf9a000640 1 -- 172.21.15.83:0/647971860 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9000cd80 con 0x7faf9c102050 2024-04-23T14:48:29.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.865+0000 7faf9a000640 1 -- 172.21.15.83:0/647971860 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf900035f0 con 0x7faf9c102050 2024-04-23T14:48:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.865+0000 7fafa4000640 1 -- 172.21.15.83:0/647971860 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9c074290 con 0x7faf9c102050 2024-04-23T14:48:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.865+0000 7faf9a000640 1 -- 172.21.15.83:0/647971860 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf90010690 con 0x7faf9c102050 2024-04-23T14:48:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.865+0000 7fafa4000640 1 -- 172.21.15.83:0/647971860 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9c0747b0 con 0x7faf9c102050 2024-04-23T14:48:29.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.867+0000 7faf8f400640 1 -- 172.21.15.83:0/647971860 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf9c076180 con 0x7faf9c102050 2024-04-23T14:48:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.871+0000 7faf9a000640 1 -- 172.21.15.83:0/647971860 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7faf90043d10 con 0x7faf9c102050 2024-04-23T14:48:29.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.872+0000 7faf9a000640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faf840a3a30 0x7faf840a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:29.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.873+0000 7faf9a000640 1 -- 172.21.15.83:0/647971860 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7faf900d0450 con 0x7faf9c102050 2024-04-23T14:48:29.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.873+0000 7faf9a000640 1 -- 172.21.15.83:0/647971860 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf9009cfd0 con 0x7faf9c102050 2024-04-23T14:48:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.873+0000 7fafa0c00640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faf840a3a30 0x7faf840a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:29.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:29.874+0000 7fafa0c00640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faf840a3a30 0x7faf840a5ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf88031440 tx=0x7faf8803c040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:30.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:29 smithi083 ceph-mon[27867]: [23/Apr/2024:14:48:28] ENGINE Serving on https://172.21.15.83:7150 2024-04-23T14:48:30.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:29 smithi083 ceph-mon[27867]: [23/Apr/2024:14:48:28] ENGINE Serving on http://172.21.15.83:8765 2024-04-23T14:48:30.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:29 smithi083 ceph-mon[27867]: [23/Apr/2024:14:48:28] ENGINE Bus STARTED 2024-04-23T14:48:30.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:29 smithi083 ceph-mon[27867]: mgrmap e22: x(active, since 1.02424s), standbys: z, y 2024-04-23T14:48:30.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:29 smithi083 ceph-mon[27867]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:30.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:29 smithi083 ceph-mon[27867]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:48:30.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:29 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:30.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:29 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:30.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:29 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:30.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:29 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:30.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:29 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1921317582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:30.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:29 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2246272810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:30.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.038+0000 7faf9a000640 1 -- 172.21.15.83:0/647971860 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7faf90098f40 con 0x7faf9c102050 2024-04-23T14:48:30.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:29 smithi173 ceph-mon[32443]: [23/Apr/2024:14:48:28] ENGINE Serving on https://172.21.15.83:7150 2024-04-23T14:48:30.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:29 smithi173 ceph-mon[32443]: [23/Apr/2024:14:48:28] ENGINE Serving on http://172.21.15.83:8765 2024-04-23T14:48:30.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:29 smithi173 ceph-mon[32443]: [23/Apr/2024:14:48:28] ENGINE Bus STARTED 2024-04-23T14:48:30.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:29 smithi173 ceph-mon[32443]: mgrmap e22: x(active, since 1.02424s), standbys: z, y 2024-04-23T14:48:30.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:29 smithi173 ceph-mon[32443]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:30.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:29 smithi173 ceph-mon[32443]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:48:30.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:29 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:30.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:29 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:30.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:29 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:30.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:29 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:30.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:29 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1921317582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:30.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:29 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2246272810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:30.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.102+0000 7faf8f400640 1 -- 172.21.15.83:0/647971860 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faf9c0632f0 con 0x7faf9c102050 2024-04-23T14:48:30.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.103+0000 7faf9a000640 1 -- 172.21.15.83:0/647971860 <== mon.2 v2:172.21.15.173:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7faf90045080 con 0x7faf9c102050 2024-04-23T14:48:30.104 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-23T14:48:30.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.105+0000 7faf8f400640 1 -- 172.21.15.83:0/647971860 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faf840a3a30 msgr2=0x7faf840a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:30.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.105+0000 7faf8f400640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faf840a3a30 0x7faf840a5ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf88031440 tx=0x7faf8803c040 comp rx=0 tx=0).stop 2024-04-23T14:48:30.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.105+0000 7faf8f400640 1 -- 172.21.15.83:0/647971860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf9c102050 msgr2=0x7faf9c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:30.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.105+0000 7faf8f400640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf9c102050 0x7faf9c198380 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf9000e9c0 tx=0x7faf9000ee80 comp rx=0 tx=0).stop 2024-04-23T14:48:30.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.106+0000 7faf8f400640 1 -- 172.21.15.83:0/647971860 shutdown_connections 2024-04-23T14:48:30.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.106+0000 7faf8f400640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faf840a3a30 0x7faf840a5ef0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.106+0000 7faf8f400640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf9c103c00 0x7faf9c19f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.106+0000 7faf8f400640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf9c103240 0x7faf9c1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.106+0000 7faf8f400640 1 --2- 172.21.15.83:0/647971860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf9c102050 0x7faf9c198380 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.106+0000 7faf8f400640 1 -- 172.21.15.83:0/647971860 >> 172.21.15.83:0/647971860 conn(0x7faf9c0fd5e0 msgr2=0x7faf9c106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:30.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.106+0000 7faf8f400640 1 -- 172.21.15.83:0/647971860 shutdown_connections 2024-04-23T14:48:30.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.106+0000 7faf8f400640 1 -- 172.21.15.83:0/647971860 wait complete. 2024-04-23T14:48:30.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:48:30.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:29 smithi203 ceph-mon[32800]: [23/Apr/2024:14:48:28] ENGINE Serving on https://172.21.15.83:7150 2024-04-23T14:48:30.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:29 smithi203 ceph-mon[32800]: [23/Apr/2024:14:48:28] ENGINE Serving on http://172.21.15.83:8765 2024-04-23T14:48:30.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:29 smithi203 ceph-mon[32800]: [23/Apr/2024:14:48:28] ENGINE Bus STARTED 2024-04-23T14:48:30.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:29 smithi203 ceph-mon[32800]: mgrmap e22: x(active, since 1.02424s), standbys: z, y 2024-04-23T14:48:30.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:29 smithi203 ceph-mon[32800]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:30.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:29 smithi203 ceph-mon[32800]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:48:30.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:29 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:30.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:29 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:30.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:29 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:30.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:29 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:30.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:29 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1921317582' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:30.222 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:29 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2246272810' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:30.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.317+0000 7fe006e00640 1 -- 172.21.15.83:0/3225474098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe000103080 msgr2=0x7fe000103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.317+0000 7fe006e00640 1 --2- 172.21.15.83:0/3225474098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe000103080 0x7fe000103500 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009860 tx=0x7fdfec02a340 comp rx=0 tx=0).stop 2024-04-23T14:48:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.318+0000 7fe006e00640 1 -- 172.21.15.83:0/3225474098 shutdown_connections 2024-04-23T14:48:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.318+0000 7fe006e00640 1 --2- 172.21.15.83:0/3225474098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe000103a40 0x7fe00010a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.318+0000 7fe006e00640 1 --2- 172.21.15.83:0/3225474098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe000103080 0x7fe000103500 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.318+0000 7fe006e00640 1 --2- 172.21.15.83:0/3225474098 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe000101f30 0x7fe000102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.318+0000 7fe006e00640 1 -- 172.21.15.83:0/3225474098 >> 172.21.15.83:0/3225474098 conn(0x7fe0000fd4a0 msgr2=0x7fe0000ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.318+0000 7fe006e00640 1 -- 172.21.15.83:0/3225474098 shutdown_connections 2024-04-23T14:48:30.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.318+0000 7fe006e00640 1 -- 172.21.15.83:0/3225474098 wait complete. 2024-04-23T14:48:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe006e00640 1 Processor -- start 2024-04-23T14:48:30.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe006e00640 1 -- start start 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe006e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe000101f30 0x7fe00019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe006e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe000103080 0x7fe00019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe006e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe000103a40 0x7fe0001a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe006e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe00010dbc0 con 0x7fe000101f30 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe006e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe00010da40 con 0x7fe000103080 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe006e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe00010dd40 con 0x7fe000103a40 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe005a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe000101f30 0x7fe00019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe006400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe000103a40 0x7fe0001a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe005a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe000101f30 0x7fe00019a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48898/0 (socket says 172.21.15.83:48898) 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe005a00640 1 -- 172.21.15.83:0/1539626264 learned_addr learned my addr 172.21.15.83:0/1539626264 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.319+0000 7fe006400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe000103a40 0x7fe0001a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41660/0 (socket says 172.21.15.83:41660) 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.320+0000 7fe005000640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe000103080 0x7fe00019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.320+0000 7fe006400640 1 -- 172.21.15.83:0/1539626264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe000103080 msgr2=0x7fe00019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:30.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.320+0000 7fe006400640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe000103080 0x7fe00019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.320+0000 7fe006400640 1 -- 172.21.15.83:0/1539626264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe000101f30 msgr2=0x7fe00019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.320+0000 7fe006400640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe000101f30 0x7fe00019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.320+0000 7fe006400640 1 -- 172.21.15.83:0/1539626264 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0001a2020 con 0x7fe000103a40 2024-04-23T14:48:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.320+0000 7fe005a00640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe000101f30 0x7fe00019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.320+0000 7fe006400640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe000103a40 0x7fe0001a19b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009fd0 tx=0x7fdff800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.320+0000 7fdffe000640 1 -- 172.21.15.83:0/1539626264 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff800ce20 con 0x7fe000103a40 2024-04-23T14:48:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.320+0000 7fdffe000640 1 -- 172.21.15.83:0/1539626264 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff8005020 con 0x7fe000103a40 2024-04-23T14:48:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.320+0000 7fe006e00640 1 -- 172.21.15.83:0/1539626264 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0001a2310 con 0x7fe000103a40 2024-04-23T14:48:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.320+0000 7fdffe000640 1 -- 172.21.15.83:0/1539626264 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8010720 con 0x7fe000103a40 2024-04-23T14:48:30.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.321+0000 7fe006e00640 1 -- 172.21.15.83:0/1539626264 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0001a2830 con 0x7fe000103a40 2024-04-23T14:48:30.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.322+0000 7fe006e00640 1 -- 172.21.15.83:0/1539626264 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0001a2af0 con 0x7fe000103a40 2024-04-23T14:48:30.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.324+0000 7fdffe000640 1 -- 172.21.15.83:0/1539626264 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fdff8005540 con 0x7fe000103a40 2024-04-23T14:48:30.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.325+0000 7fdffe000640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdfe80a3e30 0x7fdfe80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:30.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.325+0000 7fdffe000640 1 -- 172.21.15.83:0/1539626264 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fdff80d1080 con 0x7fe000103a40 2024-04-23T14:48:30.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.325+0000 7fe005a00640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdfe80a3e30 0x7fdfe80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:30.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.326+0000 7fe005a00640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdfe80a3e30 0x7fdfe80a62f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdff40096c0 tx=0x7fdff4009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:30.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.329+0000 7fdffe000640 1 -- 172.21.15.83:0/1539626264 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdff809d560 con 0x7fe000103a40 2024-04-23T14:48:30.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.526+0000 7fe006e00640 1 -- 172.21.15.83:0/1539626264 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe0001a2d60 con 0x7fe000103a40 2024-04-23T14:48:30.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.527+0000 7fdffe000640 1 -- 172.21.15.83:0/1539626264 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fdff809ce60 con 0x7fe000103a40 2024-04-23T14:48:30.528 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-23T14:48:30.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.529+0000 7fe006e00640 1 -- 172.21.15.83:0/1539626264 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdfe80a3e30 msgr2=0x7fdfe80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:30.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.529+0000 7fe006e00640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdfe80a3e30 0x7fdfe80a62f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdff40096c0 tx=0x7fdff4009260 comp rx=0 tx=0).stop 2024-04-23T14:48:30.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.529+0000 7fe006e00640 1 -- 172.21.15.83:0/1539626264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe000103a40 msgr2=0x7fe0001a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:30.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.529+0000 7fe006e00640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe000103a40 0x7fe0001a19b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009fd0 tx=0x7fdff800ef10 comp rx=0 tx=0).stop 2024-04-23T14:48:30.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.530+0000 7fe006e00640 1 -- 172.21.15.83:0/1539626264 shutdown_connections 2024-04-23T14:48:30.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.530+0000 7fe006e00640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe000103a40 0x7fe0001a19b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.530+0000 7fe006e00640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe000103080 0x7fe00019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.530+0000 7fe006e00640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdfe80a3e30 0x7fdfe80a62f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.530+0000 7fe006e00640 1 --2- 172.21.15.83:0/1539626264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe000101f30 0x7fe00019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.530+0000 7fe006e00640 1 -- 172.21.15.83:0/1539626264 >> 172.21.15.83:0/1539626264 conn(0x7fe0000fd4a0 msgr2=0x7fe000074230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:30.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.530+0000 7fe006e00640 1 -- 172.21.15.83:0/1539626264 shutdown_connections 2024-04-23T14:48:30.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.530+0000 7fe006e00640 1 -- 172.21.15.83:0/1539626264 wait complete. 2024-04-23T14:48:30.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:48:30.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.735+0000 7fd895a00640 1 -- 172.21.15.83:0/2909980254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd890104260 msgr2=0x7fd890104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.735+0000 7fd895a00640 1 --2- 172.21.15.83:0/2909980254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd890104260 0x7fd890104680 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd880009860 tx=0x7fd88002a340 comp rx=0 tx=0).stop 2024-04-23T14:48:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.735+0000 7fd895a00640 1 -- 172.21.15.83:0/2909980254 shutdown_connections 2024-04-23T14:48:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.735+0000 7fd895a00640 1 --2- 172.21.15.83:0/2909980254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd890105e10 0x7fd89010c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.735+0000 7fd895a00640 1 --2- 172.21.15.83:0/2909980254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd890105450 0x7fd8901058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.735+0000 7fd895a00640 1 --2- 172.21.15.83:0/2909980254 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd890104260 0x7fd890104680 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.735+0000 7fd895a00640 1 -- 172.21.15.83:0/2909980254 >> 172.21.15.83:0/2909980254 conn(0x7fd8900ff7f0 msgr2=0x7fd890101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.735+0000 7fd895a00640 1 -- 172.21.15.83:0/2909980254 shutdown_connections 2024-04-23T14:48:30.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.735+0000 7fd895a00640 1 -- 172.21.15.83:0/2909980254 wait complete. 2024-04-23T14:48:30.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.737+0000 7fd895a00640 1 Processor -- start 2024-04-23T14:48:30.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.737+0000 7fd895a00640 1 -- start start 2024-04-23T14:48:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.737+0000 7fd895a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd890104260 0x7fd89019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.737+0000 7fd895a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd890105450 0x7fd89019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.737+0000 7fd895a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd890105e10 0x7fd8901a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.737+0000 7fd895a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd89010fef0 con 0x7fd890105450 2024-04-23T14:48:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.737+0000 7fd895a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd89010fd70 con 0x7fd890104260 2024-04-23T14:48:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.737+0000 7fd895a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd890110070 con 0x7fd890105e10 2024-04-23T14:48:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd88ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd890104260 0x7fd89019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:30.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd88f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd890105e10 0x7fd8901a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd88ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd890104260 0x7fd89019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:43404/0 (socket says 172.21.15.83:43404) 2024-04-23T14:48:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd88f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd890105e10 0x7fd8901a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41678/0 (socket says 172.21.15.83:41678) 2024-04-23T14:48:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd88ea00640 1 -- 172.21.15.83:0/248286426 learned_addr learned my addr 172.21.15.83:0/248286426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:30.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd886000640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd890105450 0x7fd89019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd886000640 1 -- 172.21.15.83:0/248286426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd890105e10 msgr2=0x7fd8901a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd886000640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd890105e10 0x7fd8901a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd886000640 1 -- 172.21.15.83:0/248286426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd890104260 msgr2=0x7fd89019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd886000640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd890104260 0x7fd89019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd886000640 1 -- 172.21.15.83:0/248286426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8901a21b0 con 0x7fd890105450 2024-04-23T14:48:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd88ea00640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd890104260 0x7fd89019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.738+0000 7fd886000640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd890105450 0x7fd89019aad0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd87800e9f0 tx=0x7fd87800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:30.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.739+0000 7fd887e00640 1 -- 172.21.15.83:0/248286426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87800cd80 con 0x7fd890105450 2024-04-23T14:48:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.739+0000 7fd887e00640 1 -- 172.21.15.83:0/248286426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8780035f0 con 0x7fd890105450 2024-04-23T14:48:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.739+0000 7fd895a00640 1 -- 172.21.15.83:0/248286426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8901a24a0 con 0x7fd890105450 2024-04-23T14:48:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.739+0000 7fd887e00640 1 -- 172.21.15.83:0/248286426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878010690 con 0x7fd890105450 2024-04-23T14:48:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.739+0000 7fd895a00640 1 -- 172.21.15.83:0/248286426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8901a2a40 con 0x7fd890105450 2024-04-23T14:48:30.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.740+0000 7fd895a00640 1 -- 172.21.15.83:0/248286426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd89010b600 con 0x7fd890105450 2024-04-23T14:48:30.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.742+0000 7fd887e00640 1 -- 172.21.15.83:0/248286426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fd878003b30 con 0x7fd890105450 2024-04-23T14:48:30.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.743+0000 7fd887e00640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd8700a3e30 0x7fd8700a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:30.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.743+0000 7fd887e00640 1 -- 172.21.15.83:0/248286426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd8780d0d00 con 0x7fd890105450 2024-04-23T14:48:30.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.744+0000 7fd88ea00640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd8700a3e30 0x7fd8700a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:30.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.744+0000 7fd88ea00640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd8700a3e30 0x7fd8700a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd880031000 tx=0x7fd880030550 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:30.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.748+0000 7fd887e00640 1 -- 172.21.15.83:0/248286426 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd87809d270 con 0x7fd890105450 2024-04-23T14:48:30.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.971+0000 7fd895a00640 1 -- 172.21.15.83:0/248286426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd89002d870 con 0x7fd890105450 2024-04-23T14:48:30.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.973+0000 7fd887e00640 1 -- 172.21.15.83:0/248286426 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23076 (secure 0 0 0) 0x7fd87809cb70 con 0x7fd890105450 2024-04-23T14:48:30.975 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:30.975 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:48:27.715285+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/622708991":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/1081276647":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/4179157882":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6801/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/2255156114":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:48:30.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.975+0000 7fd895a00640 1 -- 172.21.15.83:0/248286426 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd8700a3e30 msgr2=0x7fd8700a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:30.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.975+0000 7fd895a00640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd8700a3e30 0x7fd8700a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd880031000 tx=0x7fd880030550 comp rx=0 tx=0).stop 2024-04-23T14:48:30.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.975+0000 7fd895a00640 1 -- 172.21.15.83:0/248286426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd890105450 msgr2=0x7fd89019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:30.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.975+0000 7fd895a00640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd890105450 0x7fd89019aad0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd87800e9f0 tx=0x7fd87800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.975+0000 7fd895a00640 1 -- 172.21.15.83:0/248286426 shutdown_connections 2024-04-23T14:48:30.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.975+0000 7fd895a00640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd890105e10 0x7fd8901a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.975+0000 7fd895a00640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd8700a3e30 0x7fd8700a62f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.975+0000 7fd895a00640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd890105450 0x7fd89019aad0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.975+0000 7fd895a00640 1 --2- 172.21.15.83:0/248286426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd890104260 0x7fd89019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:30.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.975+0000 7fd895a00640 1 -- 172.21.15.83:0/248286426 >> 172.21.15.83:0/248286426 conn(0x7fd8900ff7f0 msgr2=0x7fd8901087d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:30.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.976+0000 7fd895a00640 1 -- 172.21.15.83:0/248286426 shutdown_connections 2024-04-23T14:48:30.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:30.976+0000 7fd895a00640 1 -- 172.21.15.83:0/248286426 wait complete. 2024-04-23T14:48:30.997 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:40:41.241992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:45:57.341602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:45:58.654798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:48:30.998 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:48:31.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.189+0000 7f9596000640 1 -- 172.21.15.83:0/1318736823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9590076040 msgr2=0x7f9590074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:31.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.189+0000 7f9596000640 1 --2- 172.21.15.83:0/1318736823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9590076040 0x7f9590074490 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9580009900 tx=0x7f958002a410 comp rx=0 tx=0).stop 2024-04-23T14:48:31.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.189+0000 7f9596000640 1 -- 172.21.15.83:0/1318736823 shutdown_connections 2024-04-23T14:48:31.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.189+0000 7f9596000640 1 --2- 172.21.15.83:0/1318736823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9590109f00 0x7f959010c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.189+0000 7f9596000640 1 --2- 172.21.15.83:0/1318736823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9590074a60 0x7f9590074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.189+0000 7f9596000640 1 --2- 172.21.15.83:0/1318736823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9590076040 0x7f9590074490 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.189+0000 7f9596000640 1 -- 172.21.15.83:0/1318736823 >> 172.21.15.83:0/1318736823 conn(0x7f95900fd5b0 msgr2=0x7f95900ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:31.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.189+0000 7f9596000640 1 -- 172.21.15.83:0/1318736823 shutdown_connections 2024-04-23T14:48:31.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.189+0000 7f9596000640 1 -- 172.21.15.83:0/1318736823 wait complete. 2024-04-23T14:48:31.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.191+0000 7f9596000640 1 Processor -- start 2024-04-23T14:48:31.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.191+0000 7f9596000640 1 -- start start 2024-04-23T14:48:31.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.191+0000 7f9596000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9590074a60 0x7f959019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:31.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.191+0000 7f9596000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9590076040 0x7f959019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.191+0000 7f9596000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9590109f00 0x7f95901a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.191+0000 7f9596000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f959010fed0 con 0x7f9590074a60 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.191+0000 7f9596000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f959010fd50 con 0x7f9590109f00 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.191+0000 7f9596000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9590110050 con 0x7f9590076040 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.192+0000 7f958f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9590074a60 0x7f959019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.192+0000 7f9587e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9590076040 0x7f959019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.192+0000 7f958f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9590074a60 0x7f959019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48936/0 (socket says 172.21.15.83:48936) 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.192+0000 7f958f400640 1 -- 172.21.15.83:0/998814118 learned_addr learned my addr 172.21.15.83:0/998814118 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.192+0000 7f958fe00640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9590109f00 0x7f95901a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.192+0000 7f9587e00640 1 -- 172.21.15.83:0/998814118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9590109f00 msgr2=0x7f95901a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.192+0000 7f9587e00640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9590109f00 0x7f95901a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.192+0000 7f9587e00640 1 -- 172.21.15.83:0/998814118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9590074a60 msgr2=0x7f959019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.192+0000 7f9587e00640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9590074a60 0x7f959019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.192+0000 7f9587e00640 1 -- 172.21.15.83:0/998814118 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95901a21b0 con 0x7f9590076040 2024-04-23T14:48:31.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.192+0000 7f9587e00640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9590076040 0x7f959019aad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f957800e9f0 tx=0x7f957800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:31.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.193+0000 7f958cc00640 1 -- 172.21.15.83:0/998814118 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957800cd80 con 0x7f9590076040 2024-04-23T14:48:31.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.193+0000 7f958cc00640 1 -- 172.21.15.83:0/998814118 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95780035f0 con 0x7f9590076040 2024-04-23T14:48:31.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.193+0000 7f9596000640 1 -- 172.21.15.83:0/998814118 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95901a24a0 con 0x7f9590076040 2024-04-23T14:48:31.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.193+0000 7f958cc00640 1 -- 172.21.15.83:0/998814118 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578010690 con 0x7f9590076040 2024-04-23T14:48:31.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.193+0000 7f9596000640 1 -- 172.21.15.83:0/998814118 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95901a2a40 con 0x7f9590076040 2024-04-23T14:48:31.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.195+0000 7f9596000640 1 -- 172.21.15.83:0/998814118 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9590108c30 con 0x7f9590076040 2024-04-23T14:48:31.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.196+0000 7f958cc00640 1 -- 172.21.15.83:0/998814118 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f9578003b30 con 0x7f9590076040 2024-04-23T14:48:31.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.198+0000 7f958cc00640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f95740a3e30 0x7f95740a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:31.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.198+0000 7f958cc00640 1 -- 172.21.15.83:0/998814118 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f95780d0c40 con 0x7f9590076040 2024-04-23T14:48:31.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.198+0000 7f958f400640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f95740a3e30 0x7f95740a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:31.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.199+0000 7f958f400640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f95740a3e30 0x7f95740a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95800096c0 tx=0x7f95800315b0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:31.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.201+0000 7f958cc00640 1 -- 172.21.15.83:0/998814118 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f957809d120 con 0x7f9590076040 2024-04-23T14:48:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:31 smithi083 ceph-mon[27867]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:48:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:31 smithi083 ceph-mon[27867]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-23T14:48:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:31 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:31 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:31 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/647971860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:31 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:31 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1539626264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:31 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:31.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:31 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/248286426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:31.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:31 smithi173 ceph-mon[32443]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:48:31.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:31 smithi173 ceph-mon[32443]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-23T14:48:31.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:31 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:31.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:31 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:31.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:31 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/647971860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:31.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:31 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:31.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:31 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1539626264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:31.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:31 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:31.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:31 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/248286426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.409+0000 7f9596000640 1 -- 172.21.15.83:0/998814118 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f959002d7d0 con 0x7f9590076040 2024-04-23T14:48:31.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.410+0000 7f958cc00640 1 -- 172.21.15.83:0/998814118 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f957809ca20 con 0x7f9590076040 2024-04-23T14:48:31.411 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-23T14:48:31.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.413+0000 7f9596000640 1 -- 172.21.15.83:0/998814118 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f95740a3e30 msgr2=0x7f95740a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:31.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.413+0000 7f9596000640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f95740a3e30 0x7f95740a62f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95800096c0 tx=0x7f95800315b0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.413+0000 7f9596000640 1 -- 172.21.15.83:0/998814118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9590076040 msgr2=0x7f959019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:31.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.413+0000 7f9596000640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9590076040 0x7f959019aad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f957800e9f0 tx=0x7f957800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.413+0000 7f9596000640 1 -- 172.21.15.83:0/998814118 shutdown_connections 2024-04-23T14:48:31.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.413+0000 7f9596000640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9590109f00 0x7f95901a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.413+0000 7f9596000640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9590076040 0x7f959019aad0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.413+0000 7f9596000640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f95740a3e30 0x7f95740a62f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.413+0000 7f9596000640 1 --2- 172.21.15.83:0/998814118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9590074a60 0x7f959019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.413+0000 7f9596000640 1 -- 172.21.15.83:0/998814118 >> 172.21.15.83:0/998814118 conn(0x7f95900fd5b0 msgr2=0x7f9590105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:31.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.413+0000 7f9596000640 1 -- 172.21.15.83:0/998814118 shutdown_connections 2024-04-23T14:48:31.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.413+0000 7f9596000640 1 -- 172.21.15.83:0/998814118 wait complete. 2024-04-23T14:48:31.434 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:48:31.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:31 smithi203 ceph-mon[32800]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:48:31.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:31 smithi203 ceph-mon[32800]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-23T14:48:31.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:31 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:31.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:31 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:31.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:31 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/647971860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:31.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:31 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:31.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:31 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1539626264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:31.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:31 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:31.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:31 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/248286426' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:31.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.623+0000 7fd804200640 1 -- 172.21.15.83:0/1607920726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7fc104280 msgr2=0x7fd7fc1046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:31.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.623+0000 7fd804200640 1 --2- 172.21.15.83:0/1607920726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7fc104280 0x7fd7fc1046a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009900 tx=0x7fd7f002a3f0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.623+0000 7fd804200640 1 -- 172.21.15.83:0/1607920726 shutdown_connections 2024-04-23T14:48:31.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.623+0000 7fd804200640 1 --2- 172.21.15.83:0/1607920726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7fc105e30 0x7fd7fc10c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.623+0000 7fd804200640 1 --2- 172.21.15.83:0/1607920726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7fc105470 0x7fd7fc1058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.623+0000 7fd804200640 1 --2- 172.21.15.83:0/1607920726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7fc104280 0x7fd7fc1046a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.623+0000 7fd804200640 1 -- 172.21.15.83:0/1607920726 >> 172.21.15.83:0/1607920726 conn(0x7fd7fc0ff7f0 msgr2=0x7fd7fc101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:31.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.623+0000 7fd804200640 1 -- 172.21.15.83:0/1607920726 shutdown_connections 2024-04-23T14:48:31.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.623+0000 7fd804200640 1 -- 172.21.15.83:0/1607920726 wait complete. 2024-04-23T14:48:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.624+0000 7fd804200640 1 Processor -- start 2024-04-23T14:48:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.624+0000 7fd804200640 1 -- start start 2024-04-23T14:48:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.625+0000 7fd804200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7fc104280 0x7fd7fc19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.625+0000 7fd804200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7fc105470 0x7fd7fc19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.625+0000 7fd804200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7fc105e30 0x7fd7fc1a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.625+0000 7fd804200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc10ff10 con 0x7fd7fc105e30 2024-04-23T14:48:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.625+0000 7fd804200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc10fd90 con 0x7fd7fc105470 2024-04-23T14:48:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.625+0000 7fd804200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc110090 con 0x7fd7fc104280 2024-04-23T14:48:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.625+0000 7fd802200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7fc105e30 0x7fd7fc1a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.625+0000 7fd801800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7fc104280 0x7fd7fc19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.625+0000 7fd800e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7fc105470 0x7fd7fc19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:31.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.625+0000 7fd801800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7fc104280 0x7fd7fc19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41730/0 (socket says 172.21.15.83:41730) 2024-04-23T14:48:31.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.625+0000 7fd801800640 1 -- 172.21.15.83:0/1406230769 learned_addr learned my addr 172.21.15.83:0/1406230769 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:31.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.626+0000 7fd801800640 1 -- 172.21.15.83:0/1406230769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7fc105470 msgr2=0x7fd7fc19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:31.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.626+0000 7fd801800640 1 --2- 172.21.15.83:0/1406230769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7fc105470 0x7fd7fc19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.626+0000 7fd801800640 1 -- 172.21.15.83:0/1406230769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7fc105e30 msgr2=0x7fd7fc1a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:31.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.626+0000 7fd801800640 1 --2- 172.21.15.83:0/1406230769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7fc105e30 0x7fd7fc1a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.626+0000 7fd801800640 1 -- 172.21.15.83:0/1406230769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7fc1a21b0 con 0x7fd7fc104280 2024-04-23T14:48:31.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.626+0000 7fd801800640 1 --2- 172.21.15.83:0/1406230769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7fc104280 0x7fd7fc19a590 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0031610 tx=0x7fd7f0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:31.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.626+0000 7fd7fa000640 1 -- 172.21.15.83:0/1406230769 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f003f030 con 0x7fd7fc104280 2024-04-23T14:48:31.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.626+0000 7fd804200640 1 -- 172.21.15.83:0/1406230769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7fc1a2440 con 0x7fd7fc104280 2024-04-23T14:48:31.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.626+0000 7fd7fa000640 1 -- 172.21.15.83:0/1406230769 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7f0003ad0 con 0x7fd7fc104280 2024-04-23T14:48:31.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.626+0000 7fd7fa000640 1 -- 172.21.15.83:0/1406230769 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0035070 con 0x7fd7fc104280 2024-04-23T14:48:31.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.627+0000 7fd804200640 1 -- 172.21.15.83:0/1406230769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7fc1a2930 con 0x7fd7fc104280 2024-04-23T14:48:31.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.628+0000 7fd804200640 1 -- 172.21.15.83:0/1406230769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7fc10b620 con 0x7fd7fc104280 2024-04-23T14:48:31.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.630+0000 7fd7fa000640 1 -- 172.21.15.83:0/1406230769 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fd7f0042030 con 0x7fd7fc104280 2024-04-23T14:48:31.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.631+0000 7fd7fa000640 1 --2- 172.21.15.83:0/1406230769 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd7e40a3db0 0x7fd7e40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:31.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.631+0000 7fd7fa000640 1 -- 172.21.15.83:0/1406230769 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd7f00f1190 con 0x7fd7fc104280 2024-04-23T14:48:31.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.632+0000 7fd800e00640 1 --2- 172.21.15.83:0/1406230769 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd7e40a3db0 0x7fd7e40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:31.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.633+0000 7fd800e00640 1 --2- 172.21.15.83:0/1406230769 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd7e40a3db0 0x7fd7e40a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc19baa0 tx=0x7fd7e8005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:31.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.633+0000 7fd7fa000640 1 -- 172.21.15.83:0/1406230769 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7f00bd080 con 0x7fd7fc104280 2024-04-23T14:48:31.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.853+0000 7fd804200640 1 -- 172.21.15.83:0/1406230769 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd7fc062c00 con 0x7fd7fc104280 2024-04-23T14:48:31.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.854+0000 7fd7fa000640 1 -- 172.21.15.83:0/1406230769 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fd7f00c1380 con 0x7fd7fc104280 2024-04-23T14:48:31.855 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-23T14:48:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.856+0000 7fd804200640 1 -- 172.21.15.83:0/1406230769 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd7e40a3db0 msgr2=0x7fd7e40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.856+0000 7fd804200640 1 --2- 172.21.15.83:0/1406230769 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd7e40a3db0 0x7fd7e40a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc19baa0 tx=0x7fd7e8005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.856+0000 7fd804200640 1 -- 172.21.15.83:0/1406230769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7fc104280 msgr2=0x7fd7fc19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.856+0000 7fd804200640 1 --2- 172.21.15.83:0/1406230769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7fc104280 0x7fd7fc19a590 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0031610 tx=0x7fd7f0004270 comp rx=0 tx=0).stop 2024-04-23T14:48:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.856+0000 7fd804200640 1 -- 172.21.15.83:0/1406230769 shutdown_connections 2024-04-23T14:48:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.856+0000 7fd804200640 1 --2- 172.21.15.83:0/1406230769 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd7e40a3db0 0x7fd7e40a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.856+0000 7fd804200640 1 --2- 172.21.15.83:0/1406230769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7fc105e30 0x7fd7fc1a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.856+0000 7fd804200640 1 --2- 172.21.15.83:0/1406230769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7fc105470 0x7fd7fc19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.856+0000 7fd804200640 1 --2- 172.21.15.83:0/1406230769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd7fc104280 0x7fd7fc19a590 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.856+0000 7fd804200640 1 -- 172.21.15.83:0/1406230769 >> 172.21.15.83:0/1406230769 conn(0x7fd7fc0ff7f0 msgr2=0x7fd7fc1087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:31.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.856+0000 7fd804200640 1 -- 172.21.15.83:0/1406230769 shutdown_connections 2024-04-23T14:48:31.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:31.856+0000 7fd804200640 1 -- 172.21.15.83:0/1406230769 wait complete. 2024-04-23T14:48:31.876 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:48:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.072+0000 7f4424400640 1 -- 172.21.15.83:0/4190552504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c0785a0 msgr2=0x7f441c078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.072+0000 7f4424400640 1 --2- 172.21.15.83:0/4190552504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c0785a0 0x7f441c078e40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4414009be0 tx=0x7f441402a6c0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.072+0000 7f4424400640 1 -- 172.21.15.83:0/4190552504 shutdown_connections 2024-04-23T14:48:32.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.072+0000 7f4424400640 1 --2- 172.21.15.83:0/4190552504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c0785a0 0x7f441c078e40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.072+0000 7f4424400640 1 --2- 172.21.15.83:0/4190552504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f441c077c00 0x7f441c078060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.072+0000 7f4424400640 1 --2- 172.21.15.83:0/4190552504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f441c0769c0 0x7f441c076de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.072+0000 7f4424400640 1 -- 172.21.15.83:0/4190552504 >> 172.21.15.83:0/4190552504 conn(0x7f441c0ff7b0 msgr2=0x7f441c101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:32.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.072+0000 7f4424400640 1 -- 172.21.15.83:0/4190552504 shutdown_connections 2024-04-23T14:48:32.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.072+0000 7f4424400640 1 -- 172.21.15.83:0/4190552504 wait complete. 2024-04-23T14:48:32.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.073+0000 7f4424400640 1 Processor -- start 2024-04-23T14:48:32.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.073+0000 7f4424400640 1 -- start start 2024-04-23T14:48:32.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.074+0000 7f4424400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f441c0769c0 0x7f441c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:32.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.074+0000 7f4424400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f441c077c00 0x7f441c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:32.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.074+0000 7f4424400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c0785a0 0x7f441c1a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:32.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.074+0000 7f4424400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f441c1142f0 con 0x7f441c077c00 2024-04-23T14:48:32.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.074+0000 7f4424400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f441c114170 con 0x7f441c0785a0 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.074+0000 7f4424400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f441c114470 con 0x7f441c0769c0 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.074+0000 7f4421a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f441c0769c0 0x7f441c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.074+0000 7f4421a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f441c0769c0 0x7f441c19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41740/0 (socket says 172.21.15.83:41740) 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.074+0000 7f4421a00640 1 -- 172.21.15.83:0/819584475 learned_addr learned my addr 172.21.15.83:0/819584475 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.074+0000 7f4422400640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c0785a0 0x7f441c1a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.074+0000 7f4421000640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f441c077c00 0x7f441c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.075+0000 7f4421a00640 1 -- 172.21.15.83:0/819584475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c0785a0 msgr2=0x7f441c1a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.075+0000 7f4421a00640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c0785a0 0x7f441c1a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.075+0000 7f4421a00640 1 -- 172.21.15.83:0/819584475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f441c077c00 msgr2=0x7f441c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.075+0000 7f4421a00640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f441c077c00 0x7f441c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.075+0000 7f4421a00640 1 -- 172.21.15.83:0/819584475 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441c1a6560 con 0x7f441c0769c0 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.075+0000 7f4422400640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c0785a0 0x7f441c1a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:32.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.075+0000 7f4421a00640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f441c0769c0 0x7f441c19e940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f441000e9f0 tx=0x7f441000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:32.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.075+0000 7f441a000640 1 -- 172.21.15.83:0/819584475 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441000ccc0 con 0x7f441c0769c0 2024-04-23T14:48:32.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.075+0000 7f4424400640 1 -- 172.21.15.83:0/819584475 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441c1a6850 con 0x7f441c0769c0 2024-04-23T14:48:32.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.075+0000 7f441a000640 1 -- 172.21.15.83:0/819584475 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f441000ce60 con 0x7f441c0769c0 2024-04-23T14:48:32.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.075+0000 7f441a000640 1 -- 172.21.15.83:0/819584475 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4410010690 con 0x7f441c0769c0 2024-04-23T14:48:32.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.076+0000 7f4424400640 1 -- 172.21.15.83:0/819584475 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f441c1a6da0 con 0x7f441c0769c0 2024-04-23T14:48:32.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.077+0000 7f4424400640 1 -- 172.21.15.83:0/819584475 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441c110d00 con 0x7f441c0769c0 2024-04-23T14:48:32.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.078+0000 7f441a000640 1 -- 172.21.15.83:0/819584475 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f44100046c0 con 0x7f441c0769c0 2024-04-23T14:48:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.080+0000 7f441a000640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f44040a3db0 0x7f44040a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.080+0000 7f441a000640 1 -- 172.21.15.83:0/819584475 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f44100d0dc0 con 0x7f441c0769c0 2024-04-23T14:48:32.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.080+0000 7f4421000640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f44040a3db0 0x7f44040a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:32.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.081+0000 7f4421000640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f44040a3db0 0x7f44040a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f441c19fe50 tx=0x7f4408009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:32.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.084+0000 7f441a000640 1 -- 172.21.15.83:0/819584475 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f441009d2a0 con 0x7f441c0769c0 2024-04-23T14:48:32.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.289+0000 7f4424400640 1 -- 172.21.15.83:0/819584475 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f441c02d870 con 0x7f441c0769c0 2024-04-23T14:48:32.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.290+0000 7f441a000640 1 -- 172.21.15.83:0/819584475 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f441009cba0 con 0x7f441c0769c0 2024-04-23T14:48:32.291 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-23T14:48:32.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.293+0000 7f4424400640 1 -- 172.21.15.83:0/819584475 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f44040a3db0 msgr2=0x7f44040a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.293+0000 7f4424400640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f44040a3db0 0x7f44040a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f441c19fe50 tx=0x7f4408009260 comp rx=0 tx=0).stop 2024-04-23T14:48:32.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.293+0000 7f4424400640 1 -- 172.21.15.83:0/819584475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f441c0769c0 msgr2=0x7f441c19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.293+0000 7f4424400640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f441c0769c0 0x7f441c19e940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f441000e9f0 tx=0x7f441000edf0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.294+0000 7f4424400640 1 -- 172.21.15.83:0/819584475 shutdown_connections 2024-04-23T14:48:32.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.294+0000 7f4424400640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f441c0785a0 0x7f441c1a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.294+0000 7f4424400640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f44040a3db0 0x7f44040a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.294+0000 7f4424400640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f441c077c00 0x7f441c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.294+0000 7f4424400640 1 --2- 172.21.15.83:0/819584475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f441c0769c0 0x7f441c19e940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.294+0000 7f4424400640 1 -- 172.21.15.83:0/819584475 >> 172.21.15.83:0/819584475 conn(0x7f441c0ff7b0 msgr2=0x7f441c100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:32.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.294+0000 7f4424400640 1 -- 172.21.15.83:0/819584475 shutdown_connections 2024-04-23T14:48:32.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.294+0000 7f4424400640 1 -- 172.21.15.83:0/819584475 wait complete. 2024-04-23T14:48:32.323 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-23T14:48:32.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:48:32.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/998814118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:32 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1406230769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:32.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.503+0000 7f6556600640 1 -- 172.21.15.83:0/3414490495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6550104280 msgr2=0x7f65501046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.503+0000 7f6556600640 1 --2- 172.21.15.83:0/3414490495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6550104280 0x7f65501046a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6540009900 tx=0x7f654002a400 comp rx=0 tx=0).stop 2024-04-23T14:48:32.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.503+0000 7f6556600640 1 -- 172.21.15.83:0/3414490495 shutdown_connections 2024-04-23T14:48:32.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.503+0000 7f6556600640 1 --2- 172.21.15.83:0/3414490495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6550105e30 0x7f655010c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.503+0000 7f6556600640 1 --2- 172.21.15.83:0/3414490495 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6550105470 0x7f65501058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.503+0000 7f6556600640 1 --2- 172.21.15.83:0/3414490495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6550104280 0x7f65501046a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.503+0000 7f6556600640 1 -- 172.21.15.83:0/3414490495 >> 172.21.15.83:0/3414490495 conn(0x7f65500ff7f0 msgr2=0x7f6550101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:32.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.503+0000 7f6556600640 1 -- 172.21.15.83:0/3414490495 shutdown_connections 2024-04-23T14:48:32.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.503+0000 7f6556600640 1 -- 172.21.15.83:0/3414490495 wait complete. 2024-04-23T14:48:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.504+0000 7f6556600640 1 Processor -- start 2024-04-23T14:48:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.504+0000 7f6556600640 1 -- start start 2024-04-23T14:48:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.504+0000 7f6556600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6550104280 0x7f655019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.504+0000 7f6556600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6550105470 0x7f655019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.504+0000 7f6556600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6550105e30 0x7f65501a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:32.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.504+0000 7f6556600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f655010ff30 con 0x7f6550104280 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.504+0000 7f6556600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f655010fdb0 con 0x7f6550105470 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.504+0000 7f6556600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f65501100b0 con 0x7f6550105e30 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.505+0000 7f654f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6550104280 0x7f655019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.505+0000 7f654f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6550104280 0x7f655019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48986/0 (socket says 172.21.15.83:48986) 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.505+0000 7f654f400640 1 -- 172.21.15.83:0/2732186884 learned_addr learned my addr 172.21.15.83:0/2732186884 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.505+0000 7f6547e00640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6550105470 0x7f655019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.505+0000 7f654fe00640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6550105e30 0x7f65501a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.505+0000 7f654f400640 1 -- 172.21.15.83:0/2732186884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6550105e30 msgr2=0x7f65501a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.505+0000 7f654f400640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6550105e30 0x7f65501a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.505+0000 7f654f400640 1 -- 172.21.15.83:0/2732186884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6550105470 msgr2=0x7f655019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.505+0000 7f654f400640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6550105470 0x7f655019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.505+0000 7f654f400640 1 -- 172.21.15.83:0/2732186884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65501a21f0 con 0x7f6550104280 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.505+0000 7f6547e00640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6550105470 0x7f655019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:32.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.506+0000 7f654f400640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6550104280 0x7f655019a5d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6540031440 tx=0x7f6540032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:32.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/998814118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.510 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.511 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.511 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.511 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:32 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1406230769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.506+0000 7f654cc00640 1 -- 172.21.15.83:0/2732186884 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654003f030 con 0x7f6550104280 2024-04-23T14:48:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.506+0000 7f654cc00640 1 -- 172.21.15.83:0/2732186884 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6540003ce0 con 0x7f6550104280 2024-04-23T14:48:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.506+0000 7f6556600640 1 -- 172.21.15.83:0/2732186884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65501a2480 con 0x7f6550104280 2024-04-23T14:48:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.506+0000 7f654cc00640 1 -- 172.21.15.83:0/2732186884 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540035070 con 0x7f6550104280 2024-04-23T14:48:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.506+0000 7f6556600640 1 -- 172.21.15.83:0/2732186884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65501a2970 con 0x7f6550104280 2024-04-23T14:48:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.508+0000 7f6556600640 1 -- 172.21.15.83:0/2732186884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f655010b5a0 con 0x7f6550104280 2024-04-23T14:48:32.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.510+0000 7f654cc00640 1 -- 172.21.15.83:0/2732186884 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f6540042030 con 0x7f6550104280 2024-04-23T14:48:32.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.511+0000 7f654cc00640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f65340a3db0 0x7f65340a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:32.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.511+0000 7f654cc00640 1 -- 172.21.15.83:0/2732186884 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f65400f0e80 con 0x7f6550104280 2024-04-23T14:48:32.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.511+0000 7f6547e00640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f65340a3db0 0x7f65340a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:32.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.512+0000 7f6547e00640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f65340a3db0 0x7f65340a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f65380095f0 tx=0x7f6538005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:32.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.516+0000 7f654cc00640 1 -- 172.21.15.83:0/2732186884 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65400bd360 con 0x7f6550104280 2024-04-23T14:48:32.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/998814118' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:32.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:32.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:32 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1406230769' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:32.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.713+0000 7f654cc00640 1 -- 172.21.15.83:0/2732186884 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f654003b460 con 0x7f6550104280 2024-04-23T14:48:32.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.766+0000 7f6556600640 1 -- 172.21.15.83:0/2732186884 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f655002d620 con 0x7f6550104280 2024-04-23T14:48:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.768+0000 7f654cc00640 1 -- 172.21.15.83:0/2732186884 <== mon.0 v2:172.21.15.83:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11863 (secure 0 0 0) 0x7f65400adca0 con 0x7f6550104280 2024-04-23T14:48:32.770 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:32.770 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:48:27.525285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24532,24542]},"id":1}]} 2024-04-23T14:48:32.770 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-04-23T14:48:32.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.771+0000 7f6556600640 1 -- 172.21.15.83:0/2732186884 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f65340a3db0 msgr2=0x7f65340a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.771+0000 7f6556600640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f65340a3db0 0x7f65340a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f65380095f0 tx=0x7f6538005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.771+0000 7f6556600640 1 -- 172.21.15.83:0/2732186884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6550104280 msgr2=0x7f655019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.771+0000 7f6556600640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6550104280 0x7f655019a5d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6540031440 tx=0x7f6540032ce0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.771+0000 7f6556600640 1 -- 172.21.15.83:0/2732186884 shutdown_connections 2024-04-23T14:48:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.771+0000 7f6556600640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6550105e30 0x7f65501a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.771+0000 7f6556600640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6550105470 0x7f655019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.771+0000 7f6556600640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f65340a3db0 0x7f65340a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.771+0000 7f6556600640 1 --2- 172.21.15.83:0/2732186884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6550104280 0x7f655019a5d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.771+0000 7f6556600640 1 -- 172.21.15.83:0/2732186884 >> 172.21.15.83:0/2732186884 conn(0x7f65500ff7f0 msgr2=0x7f6550108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.771+0000 7f6556600640 1 -- 172.21.15.83:0/2732186884 shutdown_connections 2024-04-23T14:48:32.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.771+0000 7f6556600640 1 -- 172.21.15.83:0/2732186884 wait complete. 2024-04-23T14:48:32.792 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-23T14:48:32.792 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-23T14:48:32.792 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:48:32.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.979+0000 7ff4f7200640 1 -- 172.21.15.83:0/4151197563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f8104080 msgr2=0x7ff4f81044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.979+0000 7ff4f7200640 1 --2- 172.21.15.83:0/4151197563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f8104080 0x7ff4f81044a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009900 tx=0x7ff4e802a410 comp rx=0 tx=0).stop 2024-04-23T14:48:32.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.979+0000 7ff4f7200640 1 -- 172.21.15.83:0/4151197563 shutdown_connections 2024-04-23T14:48:32.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.979+0000 7ff4f7200640 1 --2- 172.21.15.83:0/4151197563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f8105c30 0x7ff4f810c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.979+0000 7ff4f7200640 1 --2- 172.21.15.83:0/4151197563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4f8105270 0x7ff4f81056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.979+0000 7ff4f7200640 1 --2- 172.21.15.83:0/4151197563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f8104080 0x7ff4f81044a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.979+0000 7ff4f7200640 1 -- 172.21.15.83:0/4151197563 >> 172.21.15.83:0/4151197563 conn(0x7ff4f80ff610 msgr2=0x7ff4f8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:32.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.979+0000 7ff4f7200640 1 -- 172.21.15.83:0/4151197563 shutdown_connections 2024-04-23T14:48:32.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.979+0000 7ff4f7200640 1 -- 172.21.15.83:0/4151197563 wait complete. 2024-04-23T14:48:32.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.980+0000 7ff4f7200640 1 Processor -- start 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.980+0000 7ff4f7200640 1 -- start start 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4f7200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4f8104080 0x7ff4f819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4f7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f8105270 0x7ff4f819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4f7200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f8105c30 0x7ff4f81a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4f7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f810fd60 con 0x7ff4f8105270 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4f7200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f810fbe0 con 0x7ff4f8104080 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4f7200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f810fee0 con 0x7ff4f8105c30 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4f5e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4f8104080 0x7ff4f819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4f5e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4f8104080 0x7ff4f819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:43482/0 (socket says 172.21.15.83:43482) 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f8105270 0x7ff4f819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4f5e00640 1 -- 172.21.15.83:0/3136550947 learned_addr learned my addr 172.21.15.83:0/3136550947 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4f6800640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f8105c30 0x7ff4f81a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:32.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4efe00640 1 -- 172.21.15.83:0/3136550947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f8105c30 msgr2=0x7ff4f81a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4efe00640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f8105c30 0x7ff4f81a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.981+0000 7ff4efe00640 1 -- 172.21.15.83:0/3136550947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4f8104080 msgr2=0x7ff4f819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.982+0000 7ff4efe00640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4f8104080 0x7ff4f819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.982+0000 7ff4efe00640 1 -- 172.21.15.83:0/3136550947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f81a1fa0 con 0x7ff4f8105270 2024-04-23T14:48:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.982+0000 7ff4f5e00640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4f8104080 0x7ff4f819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.982+0000 7ff4efe00640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f8105270 0x7ff4f819a8c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000e9f0 tx=0x7ff4e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.982+0000 7ff4eea00640 1 -- 172.21.15.83:0/3136550947 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e000ccc0 con 0x7ff4f8105270 2024-04-23T14:48:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.982+0000 7ff4eea00640 1 -- 172.21.15.83:0/3136550947 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e0005020 con 0x7ff4f8105270 2024-04-23T14:48:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.982+0000 7ff4f7200640 1 -- 172.21.15.83:0/3136550947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f81a2290 con 0x7ff4f8105270 2024-04-23T14:48:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.982+0000 7ff4eea00640 1 -- 172.21.15.83:0/3136550947 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00049e0 con 0x7ff4f8105270 2024-04-23T14:48:32.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.982+0000 7ff4f7200640 1 -- 172.21.15.83:0/3136550947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f81a2790 con 0x7ff4f8105270 2024-04-23T14:48:32.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.984+0000 7ff4f7200640 1 -- 172.21.15.83:0/3136550947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f810b340 con 0x7ff4f8105270 2024-04-23T14:48:32.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.986+0000 7ff4eea00640 1 -- 172.21.15.83:0/3136550947 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff4e0005560 con 0x7ff4f8105270 2024-04-23T14:48:32.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.987+0000 7ff4eea00640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff4dc0a3e40 0x7ff4dc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:32.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.987+0000 7ff4eea00640 1 -- 172.21.15.83:0/3136550947 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ff4e00d0a90 con 0x7ff4f8105270 2024-04-23T14:48:32.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.987+0000 7ff4f5e00640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff4dc0a3e40 0x7ff4dc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:32.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.988+0000 7ff4f5e00640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff4dc0a3e40 0x7ff4dc0a6300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8000c00 tx=0x7ff4e80315b0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:32.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:32.992+0000 7ff4eea00640 1 -- 172.21.15.83:0/3136550947 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4e00051d0 con 0x7ff4f8105270 2024-04-23T14:48:33.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.193+0000 7ff4f7200640 1 -- 172.21.15.83:0/3136550947 --> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4f8062440 con 0x7ff4dc0a3e40 2024-04-23T14:48:33.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.220+0000 7ff4eea00640 1 -- 172.21.15.83:0/3136550947 <== mgr.15114 v2:172.21.15.83:6800/1667121447 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258061 (secure 0 0 0) 0x7ff4f8062440 con 0x7ff4dc0a3e40 2024-04-23T14:48:33.226 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:33.227 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-23T14:48:33.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.226+0000 7ff4f7200640 1 -- 172.21.15.83:0/3136550947 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff4dc0a3e40 msgr2=0x7ff4dc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:33.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.226+0000 7ff4f7200640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff4dc0a3e40 0x7ff4dc0a6300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8000c00 tx=0x7ff4e80315b0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.226+0000 7ff4f7200640 1 -- 172.21.15.83:0/3136550947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f8105270 msgr2=0x7ff4f819a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:33.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.226+0000 7ff4f7200640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f8105270 0x7ff4f819a8c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000e9f0 tx=0x7ff4e000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.226+0000 7ff4f7200640 1 -- 172.21.15.83:0/3136550947 shutdown_connections 2024-04-23T14:48:33.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.226+0000 7ff4f7200640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4f8105c30 0x7ff4f81a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.226+0000 7ff4f7200640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff4dc0a3e40 0x7ff4dc0a6300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.226+0000 7ff4f7200640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4f8105270 0x7ff4f819a8c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.226+0000 7ff4f7200640 1 --2- 172.21.15.83:0/3136550947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4f8104080 0x7ff4f819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.226+0000 7ff4f7200640 1 -- 172.21.15.83:0/3136550947 >> 172.21.15.83:0/3136550947 conn(0x7ff4f80ff610 msgr2=0x7ff4f8108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:33.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.226+0000 7ff4f7200640 1 -- 172.21.15.83:0/3136550947 shutdown_connections 2024-04-23T14:48:33.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.226+0000 7ff4f7200640 1 -- 172.21.15.83:0/3136550947 wait complete. 2024-04-23T14:48:33.247 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-23T14:48:31.744314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1186429,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1111,"num_read_kb":720,"num_write":471,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181771,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381976,"kb_used_data":11988,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559848,"statfs":{"total":1151940427776,"available":1151549284352,"internally_reserved":0,"allocated":12275712,"data_stored":9199575,"data_compressed":293193,"data_compressed_allocated":393216,"data_compressed_original":3637248,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.970653"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087755+0000","last_change":"2024-04-23T14:46:00.030934+0000","last_active":"2024-04-23T14:48:28.087755+0000","last_peered":"2024-04-23T14:48:28.087755+0000","last_clean":"2024-04-23T14:48:28.087755+0000","last_became_active":"2024-04-23T14:46:00.030595+0000","last_became_peered":"2024-04-23T14:46:00.030595+0000","last_unstale":"2024-04-23T14:48:28.087755+0000","last_undegraded":"2024-04-23T14:48:28.087755+0000","last_fullsized":"2024-04-23T14:48:28.087755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.814685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.736175+0000","last_change":"2024-04-23T14:46:00.023044+0000","last_active":"2024-04-23T14:48:27.736175+0000","last_peered":"2024-04-23T14:48:27.736175+0000","last_clean":"2024-04-23T14:48:27.736175+0000","last_became_active":"2024-04-23T14:46:00.022441+0000","last_became_peered":"2024-04-23T14:46:00.022441+0000","last_unstale":"2024-04-23T14:48:27.736175+0000","last_undegraded":"2024-04-23T14:48:27.736175+0000","last_fullsized":"2024-04-23T14:48:27.736175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:24:20.976312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735374+0000","last_change":"2024-04-23T14:46:00.025397+0000","last_active":"2024-04-23T14:48:27.735374+0000","last_peered":"2024-04-23T14:48:27.735374+0000","last_clean":"2024-04-23T14:48:27.735374+0000","last_became_active":"2024-04-23T14:46:00.024477+0000","last_became_peered":"2024-04-23T14:46:00.024477+0000","last_unstale":"2024-04-23T14:48:27.735374+0000","last_undegraded":"2024-04-23T14:48:27.735374+0000","last_fullsized":"2024-04-23T14:48:27.735374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:16.002964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049660+0000","last_change":"2024-04-23T14:46:00.013960+0000","last_active":"2024-04-23T14:48:28.049660+0000","last_peered":"2024-04-23T14:48:28.049660+0000","last_clean":"2024-04-23T14:48:28.049660+0000","last_became_active":"2024-04-23T14:46:00.013656+0000","last_became_peered":"2024-04-23T14:46:00.013656+0000","last_unstale":"2024-04-23T14:48:28.049660+0000","last_undegraded":"2024-04-23T14:48:28.049660+0000","last_fullsized":"2024-04-23T14:48:28.049660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:45:59.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860225+0000","last_change":"2024-04-23T14:46:00.032336+0000","last_active":"2024-04-23T14:48:27.860225+0000","last_peered":"2024-04-23T14:48:27.860225+0000","last_clean":"2024-04-23T14:48:27.860225+0000","last_became_active":"2024-04-23T14:46:00.032008+0000","last_became_peered":"2024-04-23T14:46:00.032008+0000","last_unstale":"2024-04-23T14:48:27.860225+0000","last_undegraded":"2024-04-23T14:48:27.860225+0000","last_fullsized":"2024-04-23T14:48:27.860225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:13:05.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869145+0000","last_change":"2024-04-23T14:46:00.015457+0000","last_active":"2024-04-23T14:48:27.869145+0000","last_peered":"2024-04-23T14:48:27.869145+0000","last_clean":"2024-04-23T14:48:27.869145+0000","last_became_active":"2024-04-23T14:46:00.015199+0000","last_became_peered":"2024-04-23T14:46:00.015199+0000","last_unstale":"2024-04-23T14:48:27.869145+0000","last_undegraded":"2024-04-23T14:48:27.869145+0000","last_fullsized":"2024-04-23T14:48:27.869145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:39.310993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726746+0000","last_change":"2024-04-23T14:46:00.016554+0000","last_active":"2024-04-23T14:48:27.726746+0000","last_peered":"2024-04-23T14:48:27.726746+0000","last_clean":"2024-04-23T14:48:27.726746+0000","last_became_active":"2024-04-23T14:46:00.016242+0000","last_became_peered":"2024-04-23T14:46:00.016242+0000","last_unstale":"2024-04-23T14:48:27.726746+0000","last_undegraded":"2024-04-23T14:48:27.726746+0000","last_fullsized":"2024-04-23T14:48:27.726746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.596637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.856174+0000","last_change":"2024-04-23T14:46:58.373050+0000","last_active":"2024-04-23T14:48:27.856174+0000","last_peered":"2024-04-23T14:48:27.856174+0000","last_clean":"2024-04-23T14:48:27.856174+0000","last_became_active":"2024-04-23T14:46:58.372507+0000","last_became_peered":"2024-04-23T14:46:58.372507+0000","last_unstale":"2024-04-23T14:48:27.856174+0000","last_undegraded":"2024-04-23T14:48:27.856174+0000","last_fullsized":"2024-04-23T14:48:27.856174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:47:08.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050180+0000","last_change":"2024-04-23T14:46:00.005588+0000","last_active":"2024-04-23T14:48:28.050180+0000","last_peered":"2024-04-23T14:48:28.050180+0000","last_clean":"2024-04-23T14:48:28.050180+0000","last_became_active":"2024-04-23T14:46:00.005238+0000","last_became_peered":"2024-04-23T14:46:00.005238+0000","last_unstale":"2024-04-23T14:48:28.050180+0000","last_undegraded":"2024-04-23T14:48:28.050180+0000","last_fullsized":"2024-04-23T14:48:28.050180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:53:40.522929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869539+0000","last_change":"2024-04-23T14:46:00.013773+0000","last_active":"2024-04-23T14:48:27.869539+0000","last_peered":"2024-04-23T14:48:27.869539+0000","last_clean":"2024-04-23T14:48:27.869539+0000","last_became_active":"2024-04-23T14:46:00.013390+0000","last_became_peered":"2024-04-23T14:46:00.013390+0000","last_unstale":"2024-04-23T14:48:27.869539+0000","last_undegraded":"2024-04-23T14:48:27.869539+0000","last_fullsized":"2024-04-23T14:48:27.869539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:44.352922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727104+0000","last_change":"2024-04-23T14:46:00.013190+0000","last_active":"2024-04-23T14:48:27.727104+0000","last_peered":"2024-04-23T14:48:27.727104+0000","last_clean":"2024-04-23T14:48:27.727104+0000","last_became_active":"2024-04-23T14:46:00.012880+0000","last_became_peered":"2024-04-23T14:46:00.012880+0000","last_unstale":"2024-04-23T14:48:27.727104+0000","last_undegraded":"2024-04-23T14:48:27.727104+0000","last_fullsized":"2024-04-23T14:48:27.727104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:57.611514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049546+0000","last_change":"2024-04-23T14:46:00.013459+0000","last_active":"2024-04-23T14:48:28.049546+0000","last_peered":"2024-04-23T14:48:28.049546+0000","last_clean":"2024-04-23T14:48:28.049546+0000","last_became_active":"2024-04-23T14:46:00.013106+0000","last_became_peered":"2024-04-23T14:46:00.013106+0000","last_unstale":"2024-04-23T14:48:28.049546+0000","last_undegraded":"2024-04-23T14:48:28.049546+0000","last_fullsized":"2024-04-23T14:48:28.049546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:08:38.849062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050075+0000","last_change":"2024-04-23T14:46:00.030491+0000","last_active":"2024-04-23T14:48:28.050075+0000","last_peered":"2024-04-23T14:48:28.050075+0000","last_clean":"2024-04-23T14:48:28.050075+0000","last_became_active":"2024-04-23T14:46:00.030024+0000","last_became_peered":"2024-04-23T14:46:00.030024+0000","last_unstale":"2024-04-23T14:48:28.050075+0000","last_undegraded":"2024-04-23T14:48:28.050075+0000","last_fullsized":"2024-04-23T14:48:28.050075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:31.289091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735829+0000","last_change":"2024-04-23T14:46:00.022606+0000","last_active":"2024-04-23T14:48:27.735829+0000","last_peered":"2024-04-23T14:48:27.735829+0000","last_clean":"2024-04-23T14:48:27.735829+0000","last_became_active":"2024-04-23T14:46:00.022155+0000","last_became_peered":"2024-04-23T14:46:00.022155+0000","last_unstale":"2024-04-23T14:48:27.735829+0000","last_undegraded":"2024-04-23T14:48:27.735829+0000","last_fullsized":"2024-04-23T14:48:27.735829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:17:20.250283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735894+0000","last_change":"2024-04-23T14:46:00.027193+0000","last_active":"2024-04-23T14:48:27.735894+0000","last_peered":"2024-04-23T14:48:27.735894+0000","last_clean":"2024-04-23T14:48:27.735894+0000","last_became_active":"2024-04-23T14:46:00.026211+0000","last_became_peered":"2024-04-23T14:46:00.026211+0000","last_unstale":"2024-04-23T14:48:27.735894+0000","last_undegraded":"2024-04-23T14:48:27.735894+0000","last_fullsized":"2024-04-23T14:48:27.735894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:52.722667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049048+0000","last_change":"2024-04-23T14:46:00.029917+0000","last_active":"2024-04-23T14:48:28.049048+0000","last_peered":"2024-04-23T14:48:28.049048+0000","last_clean":"2024-04-23T14:48:28.049048+0000","last_became_active":"2024-04-23T14:46:00.029563+0000","last_became_peered":"2024-04-23T14:46:00.029563+0000","last_unstale":"2024-04-23T14:48:28.049048+0000","last_undegraded":"2024-04-23T14:48:28.049048+0000","last_fullsized":"2024-04-23T14:48:28.049048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:01.787185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087901+0000","last_change":"2024-04-23T14:46:00.030910+0000","last_active":"2024-04-23T14:48:28.087901+0000","last_peered":"2024-04-23T14:48:28.087901+0000","last_clean":"2024-04-23T14:48:28.087901+0000","last_became_active":"2024-04-23T14:46:00.030555+0000","last_became_peered":"2024-04-23T14:46:00.030555+0000","last_unstale":"2024-04-23T14:48:28.087901+0000","last_undegraded":"2024-04-23T14:48:28.087901+0000","last_fullsized":"2024-04-23T14:48:28.087901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:43.701752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.857329+0000","last_change":"2024-04-23T14:46:00.017338+0000","last_active":"2024-04-23T14:48:27.857329+0000","last_peered":"2024-04-23T14:48:27.857329+0000","last_clean":"2024-04-23T14:48:27.857329+0000","last_became_active":"2024-04-23T14:46:00.015381+0000","last_became_peered":"2024-04-23T14:46:00.015381+0000","last_unstale":"2024-04-23T14:48:27.857329+0000","last_undegraded":"2024-04-23T14:48:27.857329+0000","last_fullsized":"2024-04-23T14:48:27.857329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:51:16.762588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049084+0000","last_change":"2024-04-23T14:46:00.033055+0000","last_active":"2024-04-23T14:48:28.049084+0000","last_peered":"2024-04-23T14:48:28.049084+0000","last_clean":"2024-04-23T14:48:28.049084+0000","last_became_active":"2024-04-23T14:46:00.032763+0000","last_became_peered":"2024-04-23T14:46:00.032763+0000","last_unstale":"2024-04-23T14:48:28.049084+0000","last_undegraded":"2024-04-23T14:48:28.049084+0000","last_fullsized":"2024-04-23T14:48:28.049084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:43.793644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.857697+0000","last_change":"2024-04-23T14:45:59.232611+0000","last_active":"2024-04-23T14:48:27.857697+0000","last_peered":"2024-04-23T14:48:27.857697+0000","last_clean":"2024-04-23T14:48:27.857697+0000","last_became_active":"2024-04-23T14:45:59.231066+0000","last_became_peered":"2024-04-23T14:45:59.231066+0000","last_unstale":"2024-04-23T14:48:27.857697+0000","last_undegraded":"2024-04-23T14:48:27.857697+0000","last_fullsized":"2024-04-23T14:48:27.857697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:24.204134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.732918+0000","last_change":"2024-04-23T14:46:00.021532+0000","last_active":"2024-04-23T14:48:27.732918+0000","last_peered":"2024-04-23T14:48:27.732918+0000","last_clean":"2024-04-23T14:48:27.732918+0000","last_became_active":"2024-04-23T14:46:00.021125+0000","last_became_peered":"2024-04-23T14:46:00.021125+0000","last_unstale":"2024-04-23T14:48:27.732918+0000","last_undegraded":"2024-04-23T14:48:27.732918+0000","last_fullsized":"2024-04-23T14:48:27.732918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:03.072779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.733092+0000","last_change":"2024-04-23T14:45:59.248925+0000","last_active":"2024-04-23T14:48:27.733092+0000","last_peered":"2024-04-23T14:48:27.733092+0000","last_clean":"2024-04-23T14:48:27.733092+0000","last_became_active":"2024-04-23T14:45:59.248579+0000","last_became_peered":"2024-04-23T14:45:59.248579+0000","last_unstale":"2024-04-23T14:48:27.733092+0000","last_undegraded":"2024-04-23T14:48:27.733092+0000","last_fullsized":"2024-04-23T14:48:27.733092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:17.012610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735767+0000","last_change":"2024-04-23T14:46:00.028419+0000","last_active":"2024-04-23T14:48:27.735767+0000","last_peered":"2024-04-23T14:48:27.735767+0000","last_clean":"2024-04-23T14:48:27.735767+0000","last_became_active":"2024-04-23T14:46:00.026300+0000","last_became_peered":"2024-04-23T14:46:00.026300+0000","last_unstale":"2024-04-23T14:48:27.735767+0000","last_undegraded":"2024-04-23T14:48:27.735767+0000","last_fullsized":"2024-04-23T14:48:27.735767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:21.784680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860625+0000","last_change":"2024-04-23T14:45:59.225701+0000","last_active":"2024-04-23T14:48:27.860625+0000","last_peered":"2024-04-23T14:48:27.860625+0000","last_clean":"2024-04-23T14:48:27.860625+0000","last_became_active":"2024-04-23T14:45:59.225290+0000","last_became_peered":"2024-04-23T14:45:59.225290+0000","last_unstale":"2024-04-23T14:48:27.860625+0000","last_undegraded":"2024-04-23T14:48:27.860625+0000","last_fullsized":"2024-04-23T14:48:27.860625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:26.877330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.048946+0000","last_change":"2024-04-23T14:46:00.035075+0000","last_active":"2024-04-23T14:48:28.048946+0000","last_peered":"2024-04-23T14:48:28.048946+0000","last_clean":"2024-04-23T14:48:28.048946+0000","last_became_active":"2024-04-23T14:46:00.034782+0000","last_became_peered":"2024-04-23T14:46:00.034782+0000","last_unstale":"2024-04-23T14:48:28.048946+0000","last_undegraded":"2024-04-23T14:48:28.048946+0000","last_fullsized":"2024-04-23T14:48:28.048946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:20.747643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087488+0000","last_change":"2024-04-23T14:46:58.380160+0000","last_active":"2024-04-23T14:48:28.087488+0000","last_peered":"2024-04-23T14:48:28.087488+0000","last_clean":"2024-04-23T14:48:28.087488+0000","last_became_active":"2024-04-23T14:46:58.371279+0000","last_became_peered":"2024-04-23T14:46:58.371279+0000","last_unstale":"2024-04-23T14:48:28.087488+0000","last_undegraded":"2024-04-23T14:48:28.087488+0000","last_fullsized":"2024-04-23T14:48:28.087488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:19:44.330147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833670+0000","last_change":"2024-04-23T14:46:58.372972+0000","last_active":"2024-04-23T14:48:27.833670+0000","last_peered":"2024-04-23T14:48:27.833670+0000","last_clean":"2024-04-23T14:48:27.833670+0000","last_became_active":"2024-04-23T14:46:58.372374+0000","last_became_peered":"2024-04-23T14:46:58.372374+0000","last_unstale":"2024-04-23T14:48:27.833670+0000","last_undegraded":"2024-04-23T14:48:27.833670+0000","last_fullsized":"2024-04-23T14:48:27.833670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:15.638664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":190,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.508063+0000","last_change":"2024-04-23T14:46:58.376256+0000","last_active":"2024-04-23T14:48:28.508063+0000","last_peered":"2024-04-23T14:48:28.508063+0000","last_clean":"2024-04-23T14:48:28.508063+0000","last_became_active":"2024-04-23T14:46:58.371767+0000","last_became_peered":"2024-04-23T14:46:58.371767+0000","last_unstale":"2024-04-23T14:48:28.508063+0000","last_undegraded":"2024-04-23T14:48:28.508063+0000","last_fullsized":"2024-04-23T14:48:28.508063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:01:47.773660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860463+0000","last_change":"2024-04-23T14:46:00.016700+0000","last_active":"2024-04-23T14:48:27.860463+0000","last_peered":"2024-04-23T14:48:27.860463+0000","last_clean":"2024-04-23T14:48:27.860463+0000","last_became_active":"2024-04-23T14:46:00.015779+0000","last_became_peered":"2024-04-23T14:46:00.015779+0000","last_unstale":"2024-04-23T14:48:27.860463+0000","last_undegraded":"2024-04-23T14:48:27.860463+0000","last_fullsized":"2024-04-23T14:48:27.860463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:19:59.431628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869082+0000","last_change":"2024-04-23T14:45:59.229791+0000","last_active":"2024-04-23T14:48:27.869082+0000","last_peered":"2024-04-23T14:48:27.869082+0000","last_clean":"2024-04-23T14:48:27.869082+0000","last_became_active":"2024-04-23T14:45:59.229054+0000","last_became_peered":"2024-04-23T14:45:59.229054+0000","last_unstale":"2024-04-23T14:48:27.869082+0000","last_undegraded":"2024-04-23T14:48:27.869082+0000","last_fullsized":"2024-04-23T14:48:27.869082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:19:59.385719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833649+0000","last_change":"2024-04-23T14:46:00.021858+0000","last_active":"2024-04-23T14:48:27.833649+0000","last_peered":"2024-04-23T14:48:27.833649+0000","last_clean":"2024-04-23T14:48:27.833649+0000","last_became_active":"2024-04-23T14:46:00.021450+0000","last_became_peered":"2024-04-23T14:46:00.021450+0000","last_unstale":"2024-04-23T14:48:27.833649+0000","last_undegraded":"2024-04-23T14:48:27.833649+0000","last_fullsized":"2024-04-23T14:48:27.833649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:00.270035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049632+0000","last_change":"2024-04-23T14:45:59.238181+0000","last_active":"2024-04-23T14:48:28.049632+0000","last_peered":"2024-04-23T14:48:28.049632+0000","last_clean":"2024-04-23T14:48:28.049632+0000","last_became_active":"2024-04-23T14:45:59.237784+0000","last_became_peered":"2024-04-23T14:45:59.237784+0000","last_unstale":"2024-04-23T14:48:28.049632+0000","last_undegraded":"2024-04-23T14:48:28.049632+0000","last_fullsized":"2024-04-23T14:48:28.049632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:35.016190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726896+0000","last_change":"2024-04-23T14:46:00.033814+0000","last_active":"2024-04-23T14:48:27.726896+0000","last_peered":"2024-04-23T14:48:27.726896+0000","last_clean":"2024-04-23T14:48:27.726896+0000","last_became_active":"2024-04-23T14:46:00.032128+0000","last_became_peered":"2024-04-23T14:46:00.032128+0000","last_unstale":"2024-04-23T14:48:27.726896+0000","last_undegraded":"2024-04-23T14:48:27.726896+0000","last_fullsized":"2024-04-23T14:48:27.726896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:32.067418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833724+0000","last_change":"2024-04-23T14:45:59.228054+0000","last_active":"2024-04-23T14:48:27.833724+0000","last_peered":"2024-04-23T14:48:27.833724+0000","last_clean":"2024-04-23T14:48:27.833724+0000","last_became_active":"2024-04-23T14:45:59.227759+0000","last_became_peered":"2024-04-23T14:45:59.227759+0000","last_unstale":"2024-04-23T14:48:27.833724+0000","last_undegraded":"2024-04-23T14:48:27.833724+0000","last_fullsized":"2024-04-23T14:48:27.833724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:52.185284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.857129+0000","last_change":"2024-04-23T14:46:00.028634+0000","last_active":"2024-04-23T14:48:27.857129+0000","last_peered":"2024-04-23T14:48:27.857129+0000","last_clean":"2024-04-23T14:48:27.857129+0000","last_became_active":"2024-04-23T14:46:00.027768+0000","last_became_peered":"2024-04-23T14:46:00.027768+0000","last_unstale":"2024-04-23T14:48:27.857129+0000","last_undegraded":"2024-04-23T14:48:27.857129+0000","last_fullsized":"2024-04-23T14:48:27.857129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:29.869845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":170,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.506886+0000","last_change":"2024-04-23T14:45:59.229480+0000","last_active":"2024-04-23T14:48:28.506886+0000","last_peered":"2024-04-23T14:48:28.506886+0000","last_clean":"2024-04-23T14:48:28.506886+0000","last_became_active":"2024-04-23T14:45:59.229124+0000","last_became_peered":"2024-04-23T14:45:59.229124+0000","last_unstale":"2024-04-23T14:48:28.506886+0000","last_undegraded":"2024-04-23T14:48:28.506886+0000","last_fullsized":"2024-04-23T14:48:28.506886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:33.837779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049919+0000","last_change":"2024-04-23T14:46:00.017753+0000","last_active":"2024-04-23T14:48:28.049919+0000","last_peered":"2024-04-23T14:48:28.049919+0000","last_clean":"2024-04-23T14:48:28.049919+0000","last_became_active":"2024-04-23T14:46:00.017370+0000","last_became_peered":"2024-04-23T14:46:00.017370+0000","last_unstale":"2024-04-23T14:48:28.049919+0000","last_undegraded":"2024-04-23T14:48:28.049919+0000","last_fullsized":"2024-04-23T14:48:28.049919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:30:35.460570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727402+0000","last_change":"2024-04-23T14:45:59.229361+0000","last_active":"2024-04-23T14:48:27.727402+0000","last_peered":"2024-04-23T14:48:27.727402+0000","last_clean":"2024-04-23T14:48:27.727402+0000","last_became_active":"2024-04-23T14:45:59.229043+0000","last_became_peered":"2024-04-23T14:45:59.229043+0000","last_unstale":"2024-04-23T14:48:27.727402+0000","last_undegraded":"2024-04-23T14:48:27.727402+0000","last_fullsized":"2024-04-23T14:48:27.727402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:09.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735632+0000","last_change":"2024-04-23T14:46:00.030181+0000","last_active":"2024-04-23T14:48:27.735632+0000","last_peered":"2024-04-23T14:48:27.735632+0000","last_clean":"2024-04-23T14:48:27.735632+0000","last_became_active":"2024-04-23T14:46:00.029303+0000","last_became_peered":"2024-04-23T14:46:00.029303+0000","last_unstale":"2024-04-23T14:48:27.735632+0000","last_undegraded":"2024-04-23T14:48:27.735632+0000","last_fullsized":"2024-04-23T14:48:27.735632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:24:53.904501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833914+0000","last_change":"2024-04-23T14:45:59.231270+0000","last_active":"2024-04-23T14:48:27.833914+0000","last_peered":"2024-04-23T14:48:27.833914+0000","last_clean":"2024-04-23T14:48:27.833914+0000","last_became_active":"2024-04-23T14:45:59.230914+0000","last_became_peered":"2024-04-23T14:45:59.230914+0000","last_unstale":"2024-04-23T14:48:27.833914+0000","last_undegraded":"2024-04-23T14:48:27.833914+0000","last_fullsized":"2024-04-23T14:48:27.833914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.422260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049242+0000","last_change":"2024-04-23T14:46:00.011358+0000","last_active":"2024-04-23T14:48:28.049242+0000","last_peered":"2024-04-23T14:48:28.049242+0000","last_clean":"2024-04-23T14:48:28.049242+0000","last_became_active":"2024-04-23T14:46:00.011040+0000","last_became_peered":"2024-04-23T14:46:00.011040+0000","last_unstale":"2024-04-23T14:48:28.049242+0000","last_undegraded":"2024-04-23T14:48:28.049242+0000","last_fullsized":"2024-04-23T14:48:28.049242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:27.248844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":351,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.508135+0000","last_change":"2024-04-23T14:45:59.227195+0000","last_active":"2024-04-23T14:48:28.508135+0000","last_peered":"2024-04-23T14:48:28.508135+0000","last_clean":"2024-04-23T14:48:28.508135+0000","last_became_active":"2024-04-23T14:45:59.226874+0000","last_became_peered":"2024-04-23T14:45:59.226874+0000","last_unstale":"2024-04-23T14:48:28.508135+0000","last_undegraded":"2024-04-23T14:48:28.508135+0000","last_fullsized":"2024-04-23T14:48:28.508135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:18:40.203839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65555,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":296,"num_read_kb":212,"num_write":34,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860491+0000","last_change":"2024-04-23T14:46:00.010215+0000","last_active":"2024-04-23T14:48:27.860491+0000","last_peered":"2024-04-23T14:48:27.860491+0000","last_clean":"2024-04-23T14:48:27.860491+0000","last_became_active":"2024-04-23T14:46:00.009756+0000","last_became_peered":"2024-04-23T14:46:00.009756+0000","last_unstale":"2024-04-23T14:48:27.860491+0000","last_undegraded":"2024-04-23T14:48:27.860491+0000","last_fullsized":"2024-04-23T14:48:27.860491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.797966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049180+0000","last_change":"2024-04-23T14:45:59.239258+0000","last_active":"2024-04-23T14:48:28.049180+0000","last_peered":"2024-04-23T14:48:28.049180+0000","last_clean":"2024-04-23T14:48:28.049180+0000","last_became_active":"2024-04-23T14:45:59.238899+0000","last_became_peered":"2024-04-23T14:45:59.238899+0000","last_unstale":"2024-04-23T14:48:28.049180+0000","last_undegraded":"2024-04-23T14:48:28.049180+0000","last_fullsized":"2024-04-23T14:48:28.049180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:45.866489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049683+0000","last_change":"2024-04-23T14:46:00.036627+0000","last_active":"2024-04-23T14:48:28.049683+0000","last_peered":"2024-04-23T14:48:28.049683+0000","last_clean":"2024-04-23T14:48:28.049683+0000","last_became_active":"2024-04-23T14:46:00.035397+0000","last_became_peered":"2024-04-23T14:46:00.035397+0000","last_unstale":"2024-04-23T14:48:28.049683+0000","last_undegraded":"2024-04-23T14:48:28.049683+0000","last_fullsized":"2024-04-23T14:48:28.049683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:24:02.038813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833573+0000","last_change":"2024-04-23T14:45:59.248241+0000","last_active":"2024-04-23T14:48:27.833573+0000","last_peered":"2024-04-23T14:48:27.833573+0000","last_clean":"2024-04-23T14:48:27.833573+0000","last_became_active":"2024-04-23T14:45:59.247895+0000","last_became_peered":"2024-04-23T14:45:59.247895+0000","last_unstale":"2024-04-23T14:48:27.833573+0000","last_undegraded":"2024-04-23T14:48:27.833573+0000","last_fullsized":"2024-04-23T14:48:27.833573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:01.964714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727084+0000","last_change":"2024-04-23T14:46:58.433781+0000","last_active":"2024-04-23T14:48:27.727084+0000","last_peered":"2024-04-23T14:48:27.727084+0000","last_clean":"2024-04-23T14:48:27.727084+0000","last_became_active":"2024-04-23T14:46:58.433460+0000","last_became_peered":"2024-04-23T14:46:58.433460+0000","last_unstale":"2024-04-23T14:48:27.727084+0000","last_undegraded":"2024-04-23T14:48:27.727084+0000","last_fullsized":"2024-04-23T14:48:27.727084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:44.149949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":177,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.630557+0000","last_change":"2024-04-23T14:45:59.227406+0000","last_active":"2024-04-23T14:48:28.630557+0000","last_peered":"2024-04-23T14:48:28.630557+0000","last_clean":"2024-04-23T14:48:28.630557+0000","last_became_active":"2024-04-23T14:45:59.227063+0000","last_became_peered":"2024-04-23T14:45:59.227063+0000","last_unstale":"2024-04-23T14:48:28.630557+0000","last_undegraded":"2024-04-23T14:48:28.630557+0000","last_fullsized":"2024-04-23T14:48:28.630557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:36.845854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726734+0000","last_change":"2024-04-23T14:46:00.006333+0000","last_active":"2024-04-23T14:48:27.726734+0000","last_peered":"2024-04-23T14:48:27.726734+0000","last_clean":"2024-04-23T14:48:27.726734+0000","last_became_active":"2024-04-23T14:46:00.006024+0000","last_became_peered":"2024-04-23T14:46:00.006024+0000","last_unstale":"2024-04-23T14:48:27.726734+0000","last_undegraded":"2024-04-23T14:48:27.726734+0000","last_fullsized":"2024-04-23T14:48:27.726734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:49.389445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860182+0000","last_change":"2024-04-23T14:45:59.226394+0000","last_active":"2024-04-23T14:48:27.860182+0000","last_peered":"2024-04-23T14:48:27.860182+0000","last_clean":"2024-04-23T14:48:27.860182+0000","last_became_active":"2024-04-23T14:45:59.226088+0000","last_became_peered":"2024-04-23T14:45:59.226088+0000","last_unstale":"2024-04-23T14:48:27.860182+0000","last_undegraded":"2024-04-23T14:48:27.860182+0000","last_fullsized":"2024-04-23T14:48:27.860182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:35.139520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833741+0000","last_change":"2024-04-23T14:46:00.009262+0000","last_active":"2024-04-23T14:48:27.833741+0000","last_peered":"2024-04-23T14:48:27.833741+0000","last_clean":"2024-04-23T14:48:27.833741+0000","last_became_active":"2024-04-23T14:46:00.008865+0000","last_became_peered":"2024-04-23T14:46:00.008865+0000","last_unstale":"2024-04-23T14:48:27.833741+0000","last_undegraded":"2024-04-23T14:48:27.833741+0000","last_fullsized":"2024-04-23T14:48:27.833741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:27.662886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860641+0000","last_change":"2024-04-23T14:45:59.223479+0000","last_active":"2024-04-23T14:48:27.860641+0000","last_peered":"2024-04-23T14:48:27.860641+0000","last_clean":"2024-04-23T14:48:27.860641+0000","last_became_active":"2024-04-23T14:45:59.222997+0000","last_became_peered":"2024-04-23T14:45:59.222997+0000","last_unstale":"2024-04-23T14:48:27.860641+0000","last_undegraded":"2024-04-23T14:48:27.860641+0000","last_fullsized":"2024-04-23T14:48:27.860641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:29:58.026918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087709+0000","last_change":"2024-04-23T14:46:58.434121+0000","last_active":"2024-04-23T14:48:28.087709+0000","last_peered":"2024-04-23T14:48:28.087709+0000","last_clean":"2024-04-23T14:48:28.087709+0000","last_became_active":"2024-04-23T14:46:58.433746+0000","last_became_peered":"2024-04-23T14:46:58.433746+0000","last_unstale":"2024-04-23T14:48:28.087709+0000","last_undegraded":"2024-04-23T14:48:28.087709+0000","last_fullsized":"2024-04-23T14:48:28.087709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:33:22.493604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.856389+0000","last_change":"2024-04-23T14:45:59.237813+0000","last_active":"2024-04-23T14:48:27.856389+0000","last_peered":"2024-04-23T14:48:27.856389+0000","last_clean":"2024-04-23T14:48:27.856389+0000","last_became_active":"2024-04-23T14:45:59.237354+0000","last_became_peered":"2024-04-23T14:45:59.237354+0000","last_unstale":"2024-04-23T14:48:27.856389+0000","last_undegraded":"2024-04-23T14:48:27.856389+0000","last_fullsized":"2024-04-23T14:48:27.856389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:35:38.323059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869336+0000","last_change":"2024-04-23T14:46:00.031956+0000","last_active":"2024-04-23T14:48:27.869336+0000","last_peered":"2024-04-23T14:48:27.869336+0000","last_clean":"2024-04-23T14:48:27.869336+0000","last_became_active":"2024-04-23T14:46:00.031666+0000","last_became_peered":"2024-04-23T14:46:00.031666+0000","last_unstale":"2024-04-23T14:48:27.869336+0000","last_undegraded":"2024-04-23T14:48:27.869336+0000","last_fullsized":"2024-04-23T14:48:27.869336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:17.346455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726835+0000","last_change":"2024-04-23T14:45:59.230633+0000","last_active":"2024-04-23T14:48:27.726835+0000","last_peered":"2024-04-23T14:48:27.726835+0000","last_clean":"2024-04-23T14:48:27.726835+0000","last_became_active":"2024-04-23T14:45:59.230300+0000","last_became_peered":"2024-04-23T14:45:59.230300+0000","last_unstale":"2024-04-23T14:48:27.726835+0000","last_undegraded":"2024-04-23T14:48:27.726835+0000","last_fullsized":"2024-04-23T14:48:27.726835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:41.170055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869090+0000","last_change":"2024-04-23T14:46:00.030891+0000","last_active":"2024-04-23T14:48:27.869090+0000","last_peered":"2024-04-23T14:48:27.869090+0000","last_clean":"2024-04-23T14:48:27.869090+0000","last_became_active":"2024-04-23T14:46:00.030438+0000","last_became_peered":"2024-04-23T14:46:00.030438+0000","last_unstale":"2024-04-23T14:48:27.869090+0000","last_undegraded":"2024-04-23T14:48:27.869090+0000","last_fullsized":"2024-04-23T14:48:27.869090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:43.038111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833930+0000","last_change":"2024-04-23T14:46:00.013683+0000","last_active":"2024-04-23T14:48:27.833930+0000","last_peered":"2024-04-23T14:48:27.833930+0000","last_clean":"2024-04-23T14:48:27.833930+0000","last_became_active":"2024-04-23T14:46:00.013356+0000","last_became_peered":"2024-04-23T14:46:00.013356+0000","last_unstale":"2024-04-23T14:48:27.833930+0000","last_undegraded":"2024-04-23T14:48:27.833930+0000","last_fullsized":"2024-04-23T14:48:27.833930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:32.866694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.053312+0000","last_change":"2024-04-23T14:45:59.252258+0000","last_active":"2024-04-23T14:48:28.053312+0000","last_peered":"2024-04-23T14:48:28.053312+0000","last_clean":"2024-04-23T14:48:28.053312+0000","last_became_active":"2024-04-23T14:45:59.251513+0000","last_became_peered":"2024-04-23T14:45:59.251513+0000","last_unstale":"2024-04-23T14:48:28.053312+0000","last_undegraded":"2024-04-23T14:48:28.053312+0000","last_fullsized":"2024-04-23T14:48:28.053312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:55:37.087807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.732380+0000","last_change":"2024-04-23T14:46:00.009468+0000","last_active":"2024-04-23T14:48:27.732380+0000","last_peered":"2024-04-23T14:48:27.732380+0000","last_clean":"2024-04-23T14:48:27.732380+0000","last_became_active":"2024-04-23T14:46:00.003118+0000","last_became_peered":"2024-04-23T14:46:00.003118+0000","last_unstale":"2024-04-23T14:48:27.732380+0000","last_undegraded":"2024-04-23T14:48:27.732380+0000","last_fullsized":"2024-04-23T14:48:27.732380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:20:24.488848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050208+0000","last_change":"2024-04-23T14:45:59.225758+0000","last_active":"2024-04-23T14:48:28.050208+0000","last_peered":"2024-04-23T14:48:28.050208+0000","last_clean":"2024-04-23T14:48:28.050208+0000","last_became_active":"2024-04-23T14:45:59.225427+0000","last_became_peered":"2024-04-23T14:45:59.225427+0000","last_unstale":"2024-04-23T14:48:28.050208+0000","last_undegraded":"2024-04-23T14:48:28.050208+0000","last_fullsized":"2024-04-23T14:48:28.050208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:59.495348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":354,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726734+0000","last_change":"2024-04-23T14:44:39.684486+0000","last_active":"2024-04-23T14:48:27.726734+0000","last_peered":"2024-04-23T14:48:27.726734+0000","last_clean":"2024-04-23T14:48:27.726734+0000","last_became_active":"2024-04-23T14:44:39.684074+0000","last_became_peered":"2024-04-23T14:44:39.684074+0000","last_unstale":"2024-04-23T14:48:27.726734+0000","last_undegraded":"2024-04-23T14:48:27.726734+0000","last_fullsized":"2024-04-23T14:48:27.726734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_clean_scrub_stamp":"2024-04-23T14:40:41.333258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:11:16.865449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049334+0000","last_change":"2024-04-23T14:46:00.025018+0000","last_active":"2024-04-23T14:48:28.049334+0000","last_peered":"2024-04-23T14:48:28.049334+0000","last_clean":"2024-04-23T14:48:28.049334+0000","last_became_active":"2024-04-23T14:46:00.024695+0000","last_became_peered":"2024-04-23T14:46:00.024695+0000","last_unstale":"2024-04-23T14:48:28.049334+0000","last_undegraded":"2024-04-23T14:48:28.049334+0000","last_fullsized":"2024-04-23T14:48:28.049334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:00:31.375963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726980+0000","last_change":"2024-04-23T14:45:59.225849+0000","last_active":"2024-04-23T14:48:27.726980+0000","last_peered":"2024-04-23T14:48:27.726980+0000","last_clean":"2024-04-23T14:48:27.726980+0000","last_became_active":"2024-04-23T14:45:59.225402+0000","last_became_peered":"2024-04-23T14:45:59.225402+0000","last_unstale":"2024-04-23T14:48:27.726980+0000","last_undegraded":"2024-04-23T14:48:27.726980+0000","last_fullsized":"2024-04-23T14:48:27.726980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:10:42.217717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087726+0000","last_change":"2024-04-23T14:46:58.377666+0000","last_active":"2024-04-23T14:48:28.087726+0000","last_peered":"2024-04-23T14:48:28.087726+0000","last_clean":"2024-04-23T14:48:28.087726+0000","last_became_active":"2024-04-23T14:46:58.370533+0000","last_became_peered":"2024-04-23T14:46:58.370533+0000","last_unstale":"2024-04-23T14:48:28.087726+0000","last_undegraded":"2024-04-23T14:48:28.087726+0000","last_fullsized":"2024-04-23T14:48:28.087726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:39:58.606732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049771+0000","last_change":"2024-04-23T14:46:00.039566+0000","last_active":"2024-04-23T14:48:28.049771+0000","last_peered":"2024-04-23T14:48:28.049771+0000","last_clean":"2024-04-23T14:48:28.049771+0000","last_became_active":"2024-04-23T14:46:00.039236+0000","last_became_peered":"2024-04-23T14:46:00.039236+0000","last_unstale":"2024-04-23T14:48:28.049771+0000","last_undegraded":"2024-04-23T14:48:28.049771+0000","last_fullsized":"2024-04-23T14:48:28.049771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:44.028367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'46","reported_seq":91,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.000361+0000","last_change":"2024-04-23T14:45:59.246192+0000","last_active":"2024-04-23T14:48:28.000361+0000","last_peered":"2024-04-23T14:48:28.000361+0000","last_clean":"2024-04-23T14:48:28.000361+0000","last_became_active":"2024-04-23T14:45:59.245873+0000","last_became_peered":"2024-04-23T14:45:59.245873+0000","last_unstale":"2024-04-23T14:48:28.000361+0000","last_undegraded":"2024-04-23T14:48:28.000361+0000","last_fullsized":"2024-04-23T14:48:28.000361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:22.915746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.733234+0000","last_change":"2024-04-23T14:46:00.029176+0000","last_active":"2024-04-23T14:48:27.733234+0000","last_peered":"2024-04-23T14:48:27.733234+0000","last_clean":"2024-04-23T14:48:27.733234+0000","last_became_active":"2024-04-23T14:46:00.028741+0000","last_became_peered":"2024-04-23T14:46:00.028741+0000","last_unstale":"2024-04-23T14:48:27.733234+0000","last_undegraded":"2024-04-23T14:48:27.733234+0000","last_fullsized":"2024-04-23T14:48:27.733234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:35:22.905348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049510+0000","last_change":"2024-04-23T14:45:59.231490+0000","last_active":"2024-04-23T14:48:28.049510+0000","last_peered":"2024-04-23T14:48:28.049510+0000","last_clean":"2024-04-23T14:48:28.049510+0000","last_became_active":"2024-04-23T14:45:59.231143+0000","last_became_peered":"2024-04-23T14:45:59.231143+0000","last_unstale":"2024-04-23T14:48:28.049510+0000","last_undegraded":"2024-04-23T14:48:28.049510+0000","last_fullsized":"2024-04-23T14:48:28.049510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:47:26.398150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.736394+0000","last_change":"2024-04-23T14:46:00.035838+0000","last_active":"2024-04-23T14:48:27.736394+0000","last_peered":"2024-04-23T14:48:27.736394+0000","last_clean":"2024-04-23T14:48:27.736394+0000","last_became_active":"2024-04-23T14:46:00.034498+0000","last_became_peered":"2024-04-23T14:46:00.034498+0000","last_unstale":"2024-04-23T14:48:27.736394+0000","last_undegraded":"2024-04-23T14:48:27.736394+0000","last_fullsized":"2024-04-23T14:48:27.736394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:28:15.748503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049161+0000","last_change":"2024-04-23T14:45:59.240817+0000","last_active":"2024-04-23T14:48:28.049161+0000","last_peered":"2024-04-23T14:48:28.049161+0000","last_clean":"2024-04-23T14:48:28.049161+0000","last_became_active":"2024-04-23T14:45:59.240557+0000","last_became_peered":"2024-04-23T14:45:59.240557+0000","last_unstale":"2024-04-23T14:48:28.049161+0000","last_undegraded":"2024-04-23T14:48:28.049161+0000","last_fullsized":"2024-04-23T14:48:28.049161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:03.099054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860155+0000","last_change":"2024-04-23T14:46:00.009094+0000","last_active":"2024-04-23T14:48:27.860155+0000","last_peered":"2024-04-23T14:48:27.860155+0000","last_clean":"2024-04-23T14:48:27.860155+0000","last_became_active":"2024-04-23T14:46:00.008751+0000","last_became_peered":"2024-04-23T14:46:00.008751+0000","last_unstale":"2024-04-23T14:48:27.860155+0000","last_undegraded":"2024-04-23T14:48:27.860155+0000","last_fullsized":"2024-04-23T14:48:27.860155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:12.480737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.728665+0000","last_change":"2024-04-23T14:45:59.249882+0000","last_active":"2024-04-23T14:48:27.728665+0000","last_peered":"2024-04-23T14:48:27.728665+0000","last_clean":"2024-04-23T14:48:27.728665+0000","last_became_active":"2024-04-23T14:45:59.249072+0000","last_became_peered":"2024-04-23T14:45:59.249072+0000","last_unstale":"2024-04-23T14:48:27.728665+0000","last_undegraded":"2024-04-23T14:48:27.728665+0000","last_fullsized":"2024-04-23T14:48:27.728665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:04:16.573005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087453+0000","last_change":"2024-04-23T14:46:00.030903+0000","last_active":"2024-04-23T14:48:28.087453+0000","last_peered":"2024-04-23T14:48:28.087453+0000","last_clean":"2024-04-23T14:48:28.087453+0000","last_became_active":"2024-04-23T14:46:00.030051+0000","last_became_peered":"2024-04-23T14:46:00.030051+0000","last_unstale":"2024-04-23T14:48:28.087453+0000","last_undegraded":"2024-04-23T14:48:28.087453+0000","last_fullsized":"2024-04-23T14:48:28.087453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:41:15.915795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049098+0000","last_change":"2024-04-23T14:45:59.236446+0000","last_active":"2024-04-23T14:48:28.049098+0000","last_peered":"2024-04-23T14:48:28.049098+0000","last_clean":"2024-04-23T14:48:28.049098+0000","last_became_active":"2024-04-23T14:45:59.236119+0000","last_became_peered":"2024-04-23T14:45:59.236119+0000","last_unstale":"2024-04-23T14:48:28.049098+0000","last_undegraded":"2024-04-23T14:48:28.049098+0000","last_fullsized":"2024-04-23T14:48:28.049098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:40.153626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.732671+0000","last_change":"2024-04-23T14:46:00.035577+0000","last_active":"2024-04-23T14:48:27.732671+0000","last_peered":"2024-04-23T14:48:27.732671+0000","last_clean":"2024-04-23T14:48:27.732671+0000","last_became_active":"2024-04-23T14:46:00.034929+0000","last_became_peered":"2024-04-23T14:46:00.034929+0000","last_unstale":"2024-04-23T14:48:27.732671+0000","last_undegraded":"2024-04-23T14:48:27.732671+0000","last_fullsized":"2024-04-23T14:48:27.732671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:22.844672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860296+0000","last_change":"2024-04-23T14:46:58.440882+0000","last_active":"2024-04-23T14:48:27.860296+0000","last_peered":"2024-04-23T14:48:27.860296+0000","last_clean":"2024-04-23T14:48:27.860296+0000","last_became_active":"2024-04-23T14:46:58.436914+0000","last_became_peered":"2024-04-23T14:46:58.436914+0000","last_unstale":"2024-04-23T14:48:27.860296+0000","last_undegraded":"2024-04-23T14:48:27.860296+0000","last_fullsized":"2024-04-23T14:48:27.860296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:36.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727242+0000","last_change":"2024-04-23T14:46:00.027934+0000","last_active":"2024-04-23T14:48:27.727242+0000","last_peered":"2024-04-23T14:48:27.727242+0000","last_clean":"2024-04-23T14:48:27.727242+0000","last_became_active":"2024-04-23T14:46:00.027571+0000","last_became_peered":"2024-04-23T14:46:00.027571+0000","last_unstale":"2024-04-23T14:48:27.727242+0000","last_undegraded":"2024-04-23T14:48:27.727242+0000","last_fullsized":"2024-04-23T14:48:27.727242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:54:27.890062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049081+0000","last_change":"2024-04-23T14:45:59.236967+0000","last_active":"2024-04-23T14:48:28.049081+0000","last_peered":"2024-04-23T14:48:28.049081+0000","last_clean":"2024-04-23T14:48:28.049081+0000","last_became_active":"2024-04-23T14:45:59.236612+0000","last_became_peered":"2024-04-23T14:45:59.236612+0000","last_unstale":"2024-04-23T14:48:28.049081+0000","last_undegraded":"2024-04-23T14:48:28.049081+0000","last_fullsized":"2024-04-23T14:48:28.049081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:58:09.422598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727288+0000","last_change":"2024-04-23T14:46:00.009289+0000","last_active":"2024-04-23T14:48:27.727288+0000","last_peered":"2024-04-23T14:48:27.727288+0000","last_clean":"2024-04-23T14:48:27.727288+0000","last_became_active":"2024-04-23T14:46:00.008911+0000","last_became_peered":"2024-04-23T14:46:00.008911+0000","last_unstale":"2024-04-23T14:48:27.727288+0000","last_undegraded":"2024-04-23T14:48:27.727288+0000","last_fullsized":"2024-04-23T14:48:27.727288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:47:55.772442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.994184+0000","last_change":"2024-04-23T14:45:59.239494+0000","last_active":"2024-04-23T14:48:27.994184+0000","last_peered":"2024-04-23T14:48:27.994184+0000","last_clean":"2024-04-23T14:48:27.994184+0000","last_became_active":"2024-04-23T14:45:59.239157+0000","last_became_peered":"2024-04-23T14:45:59.239157+0000","last_unstale":"2024-04-23T14:48:27.994184+0000","last_undegraded":"2024-04-23T14:48:27.994184+0000","last_fullsized":"2024-04-23T14:48:27.994184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:38.275320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726982+0000","last_change":"2024-04-23T14:46:00.029401+0000","last_active":"2024-04-23T14:48:27.726982+0000","last_peered":"2024-04-23T14:48:27.726982+0000","last_clean":"2024-04-23T14:48:27.726982+0000","last_became_active":"2024-04-23T14:46:00.029006+0000","last_became_peered":"2024-04-23T14:46:00.029006+0000","last_unstale":"2024-04-23T14:48:27.726982+0000","last_undegraded":"2024-04-23T14:48:27.726982+0000","last_fullsized":"2024-04-23T14:48:27.726982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:11.617083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050600+0000","last_change":"2024-04-23T14:45:59.247636+0000","last_active":"2024-04-23T14:48:28.050600+0000","last_peered":"2024-04-23T14:48:28.050600+0000","last_clean":"2024-04-23T14:48:28.050600+0000","last_became_active":"2024-04-23T14:45:59.247231+0000","last_became_peered":"2024-04-23T14:45:59.247231+0000","last_unstale":"2024-04-23T14:48:28.050600+0000","last_undegraded":"2024-04-23T14:48:28.050600+0000","last_fullsized":"2024-04-23T14:48:28.050600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:54.291463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727318+0000","last_change":"2024-04-23T14:46:00.010825+0000","last_active":"2024-04-23T14:48:27.727318+0000","last_peered":"2024-04-23T14:48:27.727318+0000","last_clean":"2024-04-23T14:48:27.727318+0000","last_became_active":"2024-04-23T14:46:00.010470+0000","last_became_peered":"2024-04-23T14:46:00.010470+0000","last_unstale":"2024-04-23T14:48:27.727318+0000","last_undegraded":"2024-04-23T14:48:27.727318+0000","last_fullsized":"2024-04-23T14:48:27.727318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:19.851825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087509+0000","last_change":"2024-04-23T14:46:00.011967+0000","last_active":"2024-04-23T14:48:28.087509+0000","last_peered":"2024-04-23T14:48:28.087509+0000","last_clean":"2024-04-23T14:48:28.087509+0000","last_became_active":"2024-04-23T14:46:00.011580+0000","last_became_peered":"2024-04-23T14:46:00.011580+0000","last_unstale":"2024-04-23T14:48:28.087509+0000","last_undegraded":"2024-04-23T14:48:28.087509+0000","last_fullsized":"2024-04-23T14:48:28.087509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:25:38.909994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.048960+0000","last_change":"2024-04-23T14:46:58.436615+0000","last_active":"2024-04-23T14:48:28.048960+0000","last_peered":"2024-04-23T14:48:28.048960+0000","last_clean":"2024-04-23T14:48:28.048960+0000","last_became_active":"2024-04-23T14:46:58.436288+0000","last_became_peered":"2024-04-23T14:46:58.436288+0000","last_unstale":"2024-04-23T14:48:28.048960+0000","last_undegraded":"2024-04-23T14:48:28.048960+0000","last_fullsized":"2024-04-23T14:48:28.048960+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:49.614004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.048911+0000","last_change":"2024-04-23T14:46:00.029874+0000","last_active":"2024-04-23T14:48:28.048911+0000","last_peered":"2024-04-23T14:48:28.048911+0000","last_clean":"2024-04-23T14:48:28.048911+0000","last_became_active":"2024-04-23T14:46:00.029520+0000","last_became_peered":"2024-04-23T14:46:00.029520+0000","last_unstale":"2024-04-23T14:48:28.048911+0000","last_undegraded":"2024-04-23T14:48:28.048911+0000","last_fullsized":"2024-04-23T14:48:28.048911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:30.711566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735746+0000","last_change":"2024-04-23T14:46:00.039545+0000","last_active":"2024-04-23T14:48:27.735746+0000","last_peered":"2024-04-23T14:48:27.735746+0000","last_clean":"2024-04-23T14:48:27.735746+0000","last_became_active":"2024-04-23T14:46:00.039193+0000","last_became_peered":"2024-04-23T14:46:00.039193+0000","last_unstale":"2024-04-23T14:48:27.735746+0000","last_undegraded":"2024-04-23T14:48:27.735746+0000","last_fullsized":"2024-04-23T14:48:27.735746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:18:13.528129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050273+0000","last_change":"2024-04-23T14:46:00.036816+0000","last_active":"2024-04-23T14:48:28.050273+0000","last_peered":"2024-04-23T14:48:28.050273+0000","last_clean":"2024-04-23T14:48:28.050273+0000","last_became_active":"2024-04-23T14:46:00.036372+0000","last_became_peered":"2024-04-23T14:46:00.036372+0000","last_unstale":"2024-04-23T14:48:28.050273+0000","last_undegraded":"2024-04-23T14:48:28.050273+0000","last_fullsized":"2024-04-23T14:48:28.050273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:56.179724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087795+0000","last_change":"2024-04-23T14:46:00.012058+0000","last_active":"2024-04-23T14:48:28.087795+0000","last_peered":"2024-04-23T14:48:28.087795+0000","last_clean":"2024-04-23T14:48:28.087795+0000","last_became_active":"2024-04-23T14:46:00.011595+0000","last_became_peered":"2024-04-23T14:46:00.011595+0000","last_unstale":"2024-04-23T14:48:28.087795+0000","last_undegraded":"2024-04-23T14:48:28.087795+0000","last_fullsized":"2024-04-23T14:48:28.087795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:28.341442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049273+0000","last_change":"2024-04-23T14:46:00.011419+0000","last_active":"2024-04-23T14:48:28.049273+0000","last_peered":"2024-04-23T14:48:28.049273+0000","last_clean":"2024-04-23T14:48:28.049273+0000","last_became_active":"2024-04-23T14:46:00.011004+0000","last_became_peered":"2024-04-23T14:46:00.011004+0000","last_unstale":"2024-04-23T14:48:28.049273+0000","last_undegraded":"2024-04-23T14:48:28.049273+0000","last_fullsized":"2024-04-23T14:48:28.049273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:46:57.451648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.733118+0000","last_change":"2024-04-23T14:46:00.023583+0000","last_active":"2024-04-23T14:48:27.733118+0000","last_peered":"2024-04-23T14:48:27.733118+0000","last_clean":"2024-04-23T14:48:27.733118+0000","last_became_active":"2024-04-23T14:46:00.023195+0000","last_became_peered":"2024-04-23T14:46:00.023195+0000","last_unstale":"2024-04-23T14:48:27.733118+0000","last_undegraded":"2024-04-23T14:48:27.733118+0000","last_fullsized":"2024-04-23T14:48:27.733118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:39.992959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833812+0000","last_change":"2024-04-23T14:46:00.014719+0000","last_active":"2024-04-23T14:48:27.833812+0000","last_peered":"2024-04-23T14:48:27.833812+0000","last_clean":"2024-04-23T14:48:27.833812+0000","last_became_active":"2024-04-23T14:46:00.014349+0000","last_became_peered":"2024-04-23T14:46:00.014349+0000","last_unstale":"2024-04-23T14:48:27.833812+0000","last_undegraded":"2024-04-23T14:48:27.833812+0000","last_fullsized":"2024-04-23T14:48:27.833812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:12.789316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.728187+0000","last_change":"2024-04-23T14:46:00.019162+0000","last_active":"2024-04-23T14:48:27.728187+0000","last_peered":"2024-04-23T14:48:27.728187+0000","last_clean":"2024-04-23T14:48:27.728187+0000","last_became_active":"2024-04-23T14:46:00.018814+0000","last_became_peered":"2024-04-23T14:46:00.018814+0000","last_unstale":"2024-04-23T14:48:27.728187+0000","last_undegraded":"2024-04-23T14:48:27.728187+0000","last_fullsized":"2024-04-23T14:48:27.728187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:10.741148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049189+0000","last_change":"2024-04-23T14:46:00.007254+0000","last_active":"2024-04-23T14:48:28.049189+0000","last_peered":"2024-04-23T14:48:28.049189+0000","last_clean":"2024-04-23T14:48:28.049189+0000","last_became_active":"2024-04-23T14:46:00.006826+0000","last_became_peered":"2024-04-23T14:46:00.006826+0000","last_unstale":"2024-04-23T14:48:28.049189+0000","last_undegraded":"2024-04-23T14:48:28.049189+0000","last_fullsized":"2024-04-23T14:48:28.049189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:29.155244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050003+0000","last_change":"2024-04-23T14:46:00.039265+0000","last_active":"2024-04-23T14:48:28.050003+0000","last_peered":"2024-04-23T14:48:28.050003+0000","last_clean":"2024-04-23T14:48:28.050003+0000","last_became_active":"2024-04-23T14:46:00.038848+0000","last_became_peered":"2024-04-23T14:46:00.038848+0000","last_unstale":"2024-04-23T14:48:28.050003+0000","last_undegraded":"2024-04-23T14:48:28.050003+0000","last_fullsized":"2024-04-23T14:48:28.050003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:10.218655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087627+0000","last_change":"2024-04-23T14:46:00.012134+0000","last_active":"2024-04-23T14:48:28.087627+0000","last_peered":"2024-04-23T14:48:28.087627+0000","last_clean":"2024-04-23T14:48:28.087627+0000","last_became_active":"2024-04-23T14:46:00.011796+0000","last_became_peered":"2024-04-23T14:46:00.011796+0000","last_unstale":"2024-04-23T14:48:28.087627+0000","last_undegraded":"2024-04-23T14:48:28.087627+0000","last_fullsized":"2024-04-23T14:48:28.087627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:41.671419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107307,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":595827,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":975,"num_read_kb":601,"num_write":288,"num_write_kb":1436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1035,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":1787481,"data_compressed":81462,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":470628,"data_compressed":8702,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":1260165,"data_compressed":52479,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":864058,"data_compressed":27848,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037821,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":863956,"data_compressed":27848,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":1064147,"data_compressed":44471,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495687,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":470231,"data_compressed":8702,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":537124,"data_compressed":10554,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":471502,"data_compressed":8702,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":536307,"data_compressed":10554,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":995358,"data_compressed":34004,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920851,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":471591,"data_compressed":8702,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1194508,"data_compressed":50627,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67100,"data_compressed":1852,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66283,"data_compressed":1852,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133999,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":199656,"data_compressed":8008,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:48:33.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:48:33.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:33 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-23T14:48:33.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:33 smithi203 ceph-mon[32800]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-23T14:48:33.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:33 smithi203 ceph-mon[32800]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:48:33.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:33 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/819584475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:33.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:33 smithi203 ceph-mon[32800]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-23T14:48:33.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:33 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2732186884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.488+0000 7f7bcf200640 1 -- 172.21.15.83:0/647280487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bd01006f0 msgr2=0x7f7bd0100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.488+0000 7f7bcf200640 1 --2- 172.21.15.83:0/647280487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bd01006f0 0x7f7bd0100b10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009900 tx=0x7f7bc002a400 comp rx=0 tx=0).stop 2024-04-23T14:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.489+0000 7f7bcf200640 1 -- 172.21.15.83:0/647280487 shutdown_connections 2024-04-23T14:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.489+0000 7f7bcf200640 1 --2- 172.21.15.83:0/647280487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd01019f0 0x7f7bd0101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.489+0000 7f7bcf200640 1 --2- 172.21.15.83:0/647280487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd0101050 0x7f7bd01014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.489+0000 7f7bcf200640 1 --2- 172.21.15.83:0/647280487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bd01006f0 0x7f7bd0100b10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.489+0000 7f7bcf200640 1 -- 172.21.15.83:0/647280487 >> 172.21.15.83:0/647280487 conn(0x7f7bd00fc2d0 msgr2=0x7f7bd00fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.489+0000 7f7bcf200640 1 -- 172.21.15.83:0/647280487 shutdown_connections 2024-04-23T14:48:33.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.489+0000 7f7bcf200640 1 -- 172.21.15.83:0/647280487 wait complete. 2024-04-23T14:48:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bcf200640 1 Processor -- start 2024-04-23T14:48:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bcf200640 1 -- start start 2024-04-23T14:48:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bcf200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd01006f0 0x7f7bd019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bcf200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bd0101050 0x7f7bd019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bcf200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd01019f0 0x7f7bd01a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bcf200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd010fce0 con 0x7f7bd01006f0 2024-04-23T14:48:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bcf200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd010fb60 con 0x7f7bd01019f0 2024-04-23T14:48:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bcf200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd010fe60 con 0x7f7bd0101050 2024-04-23T14:48:33.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd01006f0 0x7f7bd019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd01006f0 0x7f7bd019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49012/0 (socket says 172.21.15.83:49012) 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bcde00640 1 -- 172.21.15.83:0/230476178 learned_addr learned my addr 172.21.15.83:0/230476178 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bcd400640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bd0101050 0x7f7bd019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.490+0000 7f7bce800640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd01019f0 0x7f7bd01a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.491+0000 7f7bcde00640 1 -- 172.21.15.83:0/230476178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bd0101050 msgr2=0x7f7bd019a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.491+0000 7f7bcde00640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bd0101050 0x7f7bd019a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.491+0000 7f7bcde00640 1 -- 172.21.15.83:0/230476178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd01019f0 msgr2=0x7f7bd01a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.491+0000 7f7bcde00640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd01019f0 0x7f7bd01a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.491+0000 7f7bcde00640 1 -- 172.21.15.83:0/230476178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd01a1f80 con 0x7f7bd01006f0 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.491+0000 7f7bcde00640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd01006f0 0x7f7bd019a360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009230 tx=0x7f7bc00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.491+0000 7f7bc6a00640 1 -- 172.21.15.83:0/230476178 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc003f030 con 0x7f7bd01006f0 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.492+0000 7f7bcf200640 1 -- 172.21.15.83:0/230476178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd01a2210 con 0x7f7bd01006f0 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.492+0000 7f7bc6a00640 1 -- 172.21.15.83:0/230476178 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc0003ce0 con 0x7f7bd01006f0 2024-04-23T14:48:33.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.492+0000 7f7bcf200640 1 -- 172.21.15.83:0/230476178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd01a2700 con 0x7f7bd01006f0 2024-04-23T14:48:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.492+0000 7f7bc6a00640 1 -- 172.21.15.83:0/230476178 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0049060 con 0x7f7bd01006f0 2024-04-23T14:48:33.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.493+0000 7f7bcf200640 1 -- 172.21.15.83:0/230476178 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd010b440 con 0x7f7bd01006f0 2024-04-23T14:48:33.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.495+0000 7f7bc6a00640 1 -- 172.21.15.83:0/230476178 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f7bc0042030 con 0x7f7bd01006f0 2024-04-23T14:48:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.496+0000 7f7bc6a00640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7bb40a3e40 0x7f7bb40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:33.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.496+0000 7f7bc6a00640 1 -- 172.21.15.83:0/230476178 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7bc00f0dd0 con 0x7f7bd01006f0 2024-04-23T14:48:33.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.496+0000 7f7bcd400640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7bb40a3e40 0x7f7bb40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:33.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.497+0000 7f7bcd400640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7bb40a3e40 0x7f7bb40a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7bd019b870 tx=0x7f7bbc005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:33.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.505+0000 7f7bc6a00640 1 -- 172.21.15.83:0/230476178 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7bc00bd300 con 0x7f7bd01006f0 2024-04-23T14:48:33.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:33 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-23T14:48:33.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:33 smithi083 ceph-mon[27867]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-23T14:48:33.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:33 smithi083 ceph-mon[27867]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:48:33.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:33 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/819584475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:33.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:33 smithi083 ceph-mon[27867]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-23T14:48:33.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:33 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2732186884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:48:33.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:33 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-23T14:48:33.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:33 smithi173 ceph-mon[32443]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-23T14:48:33.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:33 smithi173 ceph-mon[32443]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:48:33.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:33 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/819584475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:33.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:33 smithi173 ceph-mon[32443]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-23T14:48:33.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:33 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2732186884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:48:33.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.712+0000 7f7bcf200640 1 -- 172.21.15.83:0/230476178 --> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7bd0062440 con 0x7f7bb40a3e40 2024-04-23T14:48:33.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.721+0000 7f7bc6a00640 1 -- 172.21.15.83:0/230476178 <== mgr.15114 v2:172.21.15.83:6800/1667121447 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258061 (secure 0 0 0) 0x7f7bd0062440 con 0x7f7bb40a3e40 2024-04-23T14:48:33.725 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:33.726 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-23T14:48:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.725+0000 7f7bcf200640 1 -- 172.21.15.83:0/230476178 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7bb40a3e40 msgr2=0x7f7bb40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.725+0000 7f7bcf200640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7bb40a3e40 0x7f7bb40a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7bd019b870 tx=0x7f7bbc005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.725+0000 7f7bcf200640 1 -- 172.21.15.83:0/230476178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd01006f0 msgr2=0x7f7bd019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.725+0000 7f7bcf200640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd01006f0 0x7f7bd019a360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009230 tx=0x7f7bc00042f0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.725+0000 7f7bcf200640 1 -- 172.21.15.83:0/230476178 shutdown_connections 2024-04-23T14:48:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.725+0000 7f7bcf200640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bd01019f0 0x7f7bd01a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.725+0000 7f7bcf200640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bd0101050 0x7f7bd019a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.725+0000 7f7bcf200640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7bb40a3e40 0x7f7bb40a6300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.725+0000 7f7bcf200640 1 --2- 172.21.15.83:0/230476178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bd01006f0 0x7f7bd019a360 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.725+0000 7f7bcf200640 1 -- 172.21.15.83:0/230476178 >> 172.21.15.83:0/230476178 conn(0x7f7bd00fc2d0 msgr2=0x7f7bd00ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.725+0000 7f7bcf200640 1 -- 172.21.15.83:0/230476178 shutdown_connections 2024-04-23T14:48:33.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.725+0000 7f7bcf200640 1 -- 172.21.15.83:0/230476178 wait complete. 2024-04-23T14:48:33.746 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-23T14:48:31.744314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1186429,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1111,"num_read_kb":720,"num_write":471,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181771,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381976,"kb_used_data":11988,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559848,"statfs":{"total":1151940427776,"available":1151549284352,"internally_reserved":0,"allocated":12275712,"data_stored":9199575,"data_compressed":293193,"data_compressed_allocated":393216,"data_compressed_original":3637248,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.970653"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087755+0000","last_change":"2024-04-23T14:46:00.030934+0000","last_active":"2024-04-23T14:48:28.087755+0000","last_peered":"2024-04-23T14:48:28.087755+0000","last_clean":"2024-04-23T14:48:28.087755+0000","last_became_active":"2024-04-23T14:46:00.030595+0000","last_became_peered":"2024-04-23T14:46:00.030595+0000","last_unstale":"2024-04-23T14:48:28.087755+0000","last_undegraded":"2024-04-23T14:48:28.087755+0000","last_fullsized":"2024-04-23T14:48:28.087755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.814685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.736175+0000","last_change":"2024-04-23T14:46:00.023044+0000","last_active":"2024-04-23T14:48:27.736175+0000","last_peered":"2024-04-23T14:48:27.736175+0000","last_clean":"2024-04-23T14:48:27.736175+0000","last_became_active":"2024-04-23T14:46:00.022441+0000","last_became_peered":"2024-04-23T14:46:00.022441+0000","last_unstale":"2024-04-23T14:48:27.736175+0000","last_undegraded":"2024-04-23T14:48:27.736175+0000","last_fullsized":"2024-04-23T14:48:27.736175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:24:20.976312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735374+0000","last_change":"2024-04-23T14:46:00.025397+0000","last_active":"2024-04-23T14:48:27.735374+0000","last_peered":"2024-04-23T14:48:27.735374+0000","last_clean":"2024-04-23T14:48:27.735374+0000","last_became_active":"2024-04-23T14:46:00.024477+0000","last_became_peered":"2024-04-23T14:46:00.024477+0000","last_unstale":"2024-04-23T14:48:27.735374+0000","last_undegraded":"2024-04-23T14:48:27.735374+0000","last_fullsized":"2024-04-23T14:48:27.735374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:16.002964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049660+0000","last_change":"2024-04-23T14:46:00.013960+0000","last_active":"2024-04-23T14:48:28.049660+0000","last_peered":"2024-04-23T14:48:28.049660+0000","last_clean":"2024-04-23T14:48:28.049660+0000","last_became_active":"2024-04-23T14:46:00.013656+0000","last_became_peered":"2024-04-23T14:46:00.013656+0000","last_unstale":"2024-04-23T14:48:28.049660+0000","last_undegraded":"2024-04-23T14:48:28.049660+0000","last_fullsized":"2024-04-23T14:48:28.049660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:45:59.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860225+0000","last_change":"2024-04-23T14:46:00.032336+0000","last_active":"2024-04-23T14:48:27.860225+0000","last_peered":"2024-04-23T14:48:27.860225+0000","last_clean":"2024-04-23T14:48:27.860225+0000","last_became_active":"2024-04-23T14:46:00.032008+0000","last_became_peered":"2024-04-23T14:46:00.032008+0000","last_unstale":"2024-04-23T14:48:27.860225+0000","last_undegraded":"2024-04-23T14:48:27.860225+0000","last_fullsized":"2024-04-23T14:48:27.860225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:13:05.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869145+0000","last_change":"2024-04-23T14:46:00.015457+0000","last_active":"2024-04-23T14:48:27.869145+0000","last_peered":"2024-04-23T14:48:27.869145+0000","last_clean":"2024-04-23T14:48:27.869145+0000","last_became_active":"2024-04-23T14:46:00.015199+0000","last_became_peered":"2024-04-23T14:46:00.015199+0000","last_unstale":"2024-04-23T14:48:27.869145+0000","last_undegraded":"2024-04-23T14:48:27.869145+0000","last_fullsized":"2024-04-23T14:48:27.869145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:39.310993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726746+0000","last_change":"2024-04-23T14:46:00.016554+0000","last_active":"2024-04-23T14:48:27.726746+0000","last_peered":"2024-04-23T14:48:27.726746+0000","last_clean":"2024-04-23T14:48:27.726746+0000","last_became_active":"2024-04-23T14:46:00.016242+0000","last_became_peered":"2024-04-23T14:46:00.016242+0000","last_unstale":"2024-04-23T14:48:27.726746+0000","last_undegraded":"2024-04-23T14:48:27.726746+0000","last_fullsized":"2024-04-23T14:48:27.726746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.596637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.856174+0000","last_change":"2024-04-23T14:46:58.373050+0000","last_active":"2024-04-23T14:48:27.856174+0000","last_peered":"2024-04-23T14:48:27.856174+0000","last_clean":"2024-04-23T14:48:27.856174+0000","last_became_active":"2024-04-23T14:46:58.372507+0000","last_became_peered":"2024-04-23T14:46:58.372507+0000","last_unstale":"2024-04-23T14:48:27.856174+0000","last_undegraded":"2024-04-23T14:48:27.856174+0000","last_fullsized":"2024-04-23T14:48:27.856174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:47:08.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050180+0000","last_change":"2024-04-23T14:46:00.005588+0000","last_active":"2024-04-23T14:48:28.050180+0000","last_peered":"2024-04-23T14:48:28.050180+0000","last_clean":"2024-04-23T14:48:28.050180+0000","last_became_active":"2024-04-23T14:46:00.005238+0000","last_became_peered":"2024-04-23T14:46:00.005238+0000","last_unstale":"2024-04-23T14:48:28.050180+0000","last_undegraded":"2024-04-23T14:48:28.050180+0000","last_fullsized":"2024-04-23T14:48:28.050180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:53:40.522929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869539+0000","last_change":"2024-04-23T14:46:00.013773+0000","last_active":"2024-04-23T14:48:27.869539+0000","last_peered":"2024-04-23T14:48:27.869539+0000","last_clean":"2024-04-23T14:48:27.869539+0000","last_became_active":"2024-04-23T14:46:00.013390+0000","last_became_peered":"2024-04-23T14:46:00.013390+0000","last_unstale":"2024-04-23T14:48:27.869539+0000","last_undegraded":"2024-04-23T14:48:27.869539+0000","last_fullsized":"2024-04-23T14:48:27.869539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:44.352922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727104+0000","last_change":"2024-04-23T14:46:00.013190+0000","last_active":"2024-04-23T14:48:27.727104+0000","last_peered":"2024-04-23T14:48:27.727104+0000","last_clean":"2024-04-23T14:48:27.727104+0000","last_became_active":"2024-04-23T14:46:00.012880+0000","last_became_peered":"2024-04-23T14:46:00.012880+0000","last_unstale":"2024-04-23T14:48:27.727104+0000","last_undegraded":"2024-04-23T14:48:27.727104+0000","last_fullsized":"2024-04-23T14:48:27.727104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:57.611514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049546+0000","last_change":"2024-04-23T14:46:00.013459+0000","last_active":"2024-04-23T14:48:28.049546+0000","last_peered":"2024-04-23T14:48:28.049546+0000","last_clean":"2024-04-23T14:48:28.049546+0000","last_became_active":"2024-04-23T14:46:00.013106+0000","last_became_peered":"2024-04-23T14:46:00.013106+0000","last_unstale":"2024-04-23T14:48:28.049546+0000","last_undegraded":"2024-04-23T14:48:28.049546+0000","last_fullsized":"2024-04-23T14:48:28.049546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:08:38.849062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050075+0000","last_change":"2024-04-23T14:46:00.030491+0000","last_active":"2024-04-23T14:48:28.050075+0000","last_peered":"2024-04-23T14:48:28.050075+0000","last_clean":"2024-04-23T14:48:28.050075+0000","last_became_active":"2024-04-23T14:46:00.030024+0000","last_became_peered":"2024-04-23T14:46:00.030024+0000","last_unstale":"2024-04-23T14:48:28.050075+0000","last_undegraded":"2024-04-23T14:48:28.050075+0000","last_fullsized":"2024-04-23T14:48:28.050075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:31.289091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735829+0000","last_change":"2024-04-23T14:46:00.022606+0000","last_active":"2024-04-23T14:48:27.735829+0000","last_peered":"2024-04-23T14:48:27.735829+0000","last_clean":"2024-04-23T14:48:27.735829+0000","last_became_active":"2024-04-23T14:46:00.022155+0000","last_became_peered":"2024-04-23T14:46:00.022155+0000","last_unstale":"2024-04-23T14:48:27.735829+0000","last_undegraded":"2024-04-23T14:48:27.735829+0000","last_fullsized":"2024-04-23T14:48:27.735829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:17:20.250283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735894+0000","last_change":"2024-04-23T14:46:00.027193+0000","last_active":"2024-04-23T14:48:27.735894+0000","last_peered":"2024-04-23T14:48:27.735894+0000","last_clean":"2024-04-23T14:48:27.735894+0000","last_became_active":"2024-04-23T14:46:00.026211+0000","last_became_peered":"2024-04-23T14:46:00.026211+0000","last_unstale":"2024-04-23T14:48:27.735894+0000","last_undegraded":"2024-04-23T14:48:27.735894+0000","last_fullsized":"2024-04-23T14:48:27.735894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:52.722667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049048+0000","last_change":"2024-04-23T14:46:00.029917+0000","last_active":"2024-04-23T14:48:28.049048+0000","last_peered":"2024-04-23T14:48:28.049048+0000","last_clean":"2024-04-23T14:48:28.049048+0000","last_became_active":"2024-04-23T14:46:00.029563+0000","last_became_peered":"2024-04-23T14:46:00.029563+0000","last_unstale":"2024-04-23T14:48:28.049048+0000","last_undegraded":"2024-04-23T14:48:28.049048+0000","last_fullsized":"2024-04-23T14:48:28.049048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:01.787185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087901+0000","last_change":"2024-04-23T14:46:00.030910+0000","last_active":"2024-04-23T14:48:28.087901+0000","last_peered":"2024-04-23T14:48:28.087901+0000","last_clean":"2024-04-23T14:48:28.087901+0000","last_became_active":"2024-04-23T14:46:00.030555+0000","last_became_peered":"2024-04-23T14:46:00.030555+0000","last_unstale":"2024-04-23T14:48:28.087901+0000","last_undegraded":"2024-04-23T14:48:28.087901+0000","last_fullsized":"2024-04-23T14:48:28.087901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:43.701752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.857329+0000","last_change":"2024-04-23T14:46:00.017338+0000","last_active":"2024-04-23T14:48:27.857329+0000","last_peered":"2024-04-23T14:48:27.857329+0000","last_clean":"2024-04-23T14:48:27.857329+0000","last_became_active":"2024-04-23T14:46:00.015381+0000","last_became_peered":"2024-04-23T14:46:00.015381+0000","last_unstale":"2024-04-23T14:48:27.857329+0000","last_undegraded":"2024-04-23T14:48:27.857329+0000","last_fullsized":"2024-04-23T14:48:27.857329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:51:16.762588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049084+0000","last_change":"2024-04-23T14:46:00.033055+0000","last_active":"2024-04-23T14:48:28.049084+0000","last_peered":"2024-04-23T14:48:28.049084+0000","last_clean":"2024-04-23T14:48:28.049084+0000","last_became_active":"2024-04-23T14:46:00.032763+0000","last_became_peered":"2024-04-23T14:46:00.032763+0000","last_unstale":"2024-04-23T14:48:28.049084+0000","last_undegraded":"2024-04-23T14:48:28.049084+0000","last_fullsized":"2024-04-23T14:48:28.049084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:43.793644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.857697+0000","last_change":"2024-04-23T14:45:59.232611+0000","last_active":"2024-04-23T14:48:27.857697+0000","last_peered":"2024-04-23T14:48:27.857697+0000","last_clean":"2024-04-23T14:48:27.857697+0000","last_became_active":"2024-04-23T14:45:59.231066+0000","last_became_peered":"2024-04-23T14:45:59.231066+0000","last_unstale":"2024-04-23T14:48:27.857697+0000","last_undegraded":"2024-04-23T14:48:27.857697+0000","last_fullsized":"2024-04-23T14:48:27.857697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:24.204134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.732918+0000","last_change":"2024-04-23T14:46:00.021532+0000","last_active":"2024-04-23T14:48:27.732918+0000","last_peered":"2024-04-23T14:48:27.732918+0000","last_clean":"2024-04-23T14:48:27.732918+0000","last_became_active":"2024-04-23T14:46:00.021125+0000","last_became_peered":"2024-04-23T14:46:00.021125+0000","last_unstale":"2024-04-23T14:48:27.732918+0000","last_undegraded":"2024-04-23T14:48:27.732918+0000","last_fullsized":"2024-04-23T14:48:27.732918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:03.072779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.733092+0000","last_change":"2024-04-23T14:45:59.248925+0000","last_active":"2024-04-23T14:48:27.733092+0000","last_peered":"2024-04-23T14:48:27.733092+0000","last_clean":"2024-04-23T14:48:27.733092+0000","last_became_active":"2024-04-23T14:45:59.248579+0000","last_became_peered":"2024-04-23T14:45:59.248579+0000","last_unstale":"2024-04-23T14:48:27.733092+0000","last_undegraded":"2024-04-23T14:48:27.733092+0000","last_fullsized":"2024-04-23T14:48:27.733092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:17.012610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735767+0000","last_change":"2024-04-23T14:46:00.028419+0000","last_active":"2024-04-23T14:48:27.735767+0000","last_peered":"2024-04-23T14:48:27.735767+0000","last_clean":"2024-04-23T14:48:27.735767+0000","last_became_active":"2024-04-23T14:46:00.026300+0000","last_became_peered":"2024-04-23T14:46:00.026300+0000","last_unstale":"2024-04-23T14:48:27.735767+0000","last_undegraded":"2024-04-23T14:48:27.735767+0000","last_fullsized":"2024-04-23T14:48:27.735767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:21.784680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860625+0000","last_change":"2024-04-23T14:45:59.225701+0000","last_active":"2024-04-23T14:48:27.860625+0000","last_peered":"2024-04-23T14:48:27.860625+0000","last_clean":"2024-04-23T14:48:27.860625+0000","last_became_active":"2024-04-23T14:45:59.225290+0000","last_became_peered":"2024-04-23T14:45:59.225290+0000","last_unstale":"2024-04-23T14:48:27.860625+0000","last_undegraded":"2024-04-23T14:48:27.860625+0000","last_fullsized":"2024-04-23T14:48:27.860625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:26.877330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.048946+0000","last_change":"2024-04-23T14:46:00.035075+0000","last_active":"2024-04-23T14:48:28.048946+0000","last_peered":"2024-04-23T14:48:28.048946+0000","last_clean":"2024-04-23T14:48:28.048946+0000","last_became_active":"2024-04-23T14:46:00.034782+0000","last_became_peered":"2024-04-23T14:46:00.034782+0000","last_unstale":"2024-04-23T14:48:28.048946+0000","last_undegraded":"2024-04-23T14:48:28.048946+0000","last_fullsized":"2024-04-23T14:48:28.048946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:20.747643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087488+0000","last_change":"2024-04-23T14:46:58.380160+0000","last_active":"2024-04-23T14:48:28.087488+0000","last_peered":"2024-04-23T14:48:28.087488+0000","last_clean":"2024-04-23T14:48:28.087488+0000","last_became_active":"2024-04-23T14:46:58.371279+0000","last_became_peered":"2024-04-23T14:46:58.371279+0000","last_unstale":"2024-04-23T14:48:28.087488+0000","last_undegraded":"2024-04-23T14:48:28.087488+0000","last_fullsized":"2024-04-23T14:48:28.087488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:19:44.330147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833670+0000","last_change":"2024-04-23T14:46:58.372972+0000","last_active":"2024-04-23T14:48:27.833670+0000","last_peered":"2024-04-23T14:48:27.833670+0000","last_clean":"2024-04-23T14:48:27.833670+0000","last_became_active":"2024-04-23T14:46:58.372374+0000","last_became_peered":"2024-04-23T14:46:58.372374+0000","last_unstale":"2024-04-23T14:48:27.833670+0000","last_undegraded":"2024-04-23T14:48:27.833670+0000","last_fullsized":"2024-04-23T14:48:27.833670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:15.638664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":190,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.508063+0000","last_change":"2024-04-23T14:46:58.376256+0000","last_active":"2024-04-23T14:48:28.508063+0000","last_peered":"2024-04-23T14:48:28.508063+0000","last_clean":"2024-04-23T14:48:28.508063+0000","last_became_active":"2024-04-23T14:46:58.371767+0000","last_became_peered":"2024-04-23T14:46:58.371767+0000","last_unstale":"2024-04-23T14:48:28.508063+0000","last_undegraded":"2024-04-23T14:48:28.508063+0000","last_fullsized":"2024-04-23T14:48:28.508063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:01:47.773660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860463+0000","last_change":"2024-04-23T14:46:00.016700+0000","last_active":"2024-04-23T14:48:27.860463+0000","last_peered":"2024-04-23T14:48:27.860463+0000","last_clean":"2024-04-23T14:48:27.860463+0000","last_became_active":"2024-04-23T14:46:00.015779+0000","last_became_peered":"2024-04-23T14:46:00.015779+0000","last_unstale":"2024-04-23T14:48:27.860463+0000","last_undegraded":"2024-04-23T14:48:27.860463+0000","last_fullsized":"2024-04-23T14:48:27.860463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:19:59.431628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869082+0000","last_change":"2024-04-23T14:45:59.229791+0000","last_active":"2024-04-23T14:48:27.869082+0000","last_peered":"2024-04-23T14:48:27.869082+0000","last_clean":"2024-04-23T14:48:27.869082+0000","last_became_active":"2024-04-23T14:45:59.229054+0000","last_became_peered":"2024-04-23T14:45:59.229054+0000","last_unstale":"2024-04-23T14:48:27.869082+0000","last_undegraded":"2024-04-23T14:48:27.869082+0000","last_fullsized":"2024-04-23T14:48:27.869082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:19:59.385719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833649+0000","last_change":"2024-04-23T14:46:00.021858+0000","last_active":"2024-04-23T14:48:27.833649+0000","last_peered":"2024-04-23T14:48:27.833649+0000","last_clean":"2024-04-23T14:48:27.833649+0000","last_became_active":"2024-04-23T14:46:00.021450+0000","last_became_peered":"2024-04-23T14:46:00.021450+0000","last_unstale":"2024-04-23T14:48:27.833649+0000","last_undegraded":"2024-04-23T14:48:27.833649+0000","last_fullsized":"2024-04-23T14:48:27.833649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:00.270035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049632+0000","last_change":"2024-04-23T14:45:59.238181+0000","last_active":"2024-04-23T14:48:28.049632+0000","last_peered":"2024-04-23T14:48:28.049632+0000","last_clean":"2024-04-23T14:48:28.049632+0000","last_became_active":"2024-04-23T14:45:59.237784+0000","last_became_peered":"2024-04-23T14:45:59.237784+0000","last_unstale":"2024-04-23T14:48:28.049632+0000","last_undegraded":"2024-04-23T14:48:28.049632+0000","last_fullsized":"2024-04-23T14:48:28.049632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:35.016190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726896+0000","last_change":"2024-04-23T14:46:00.033814+0000","last_active":"2024-04-23T14:48:27.726896+0000","last_peered":"2024-04-23T14:48:27.726896+0000","last_clean":"2024-04-23T14:48:27.726896+0000","last_became_active":"2024-04-23T14:46:00.032128+0000","last_became_peered":"2024-04-23T14:46:00.032128+0000","last_unstale":"2024-04-23T14:48:27.726896+0000","last_undegraded":"2024-04-23T14:48:27.726896+0000","last_fullsized":"2024-04-23T14:48:27.726896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:32.067418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833724+0000","last_change":"2024-04-23T14:45:59.228054+0000","last_active":"2024-04-23T14:48:27.833724+0000","last_peered":"2024-04-23T14:48:27.833724+0000","last_clean":"2024-04-23T14:48:27.833724+0000","last_became_active":"2024-04-23T14:45:59.227759+0000","last_became_peered":"2024-04-23T14:45:59.227759+0000","last_unstale":"2024-04-23T14:48:27.833724+0000","last_undegraded":"2024-04-23T14:48:27.833724+0000","last_fullsized":"2024-04-23T14:48:27.833724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:52.185284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.857129+0000","last_change":"2024-04-23T14:46:00.028634+0000","last_active":"2024-04-23T14:48:27.857129+0000","last_peered":"2024-04-23T14:48:27.857129+0000","last_clean":"2024-04-23T14:48:27.857129+0000","last_became_active":"2024-04-23T14:46:00.027768+0000","last_became_peered":"2024-04-23T14:46:00.027768+0000","last_unstale":"2024-04-23T14:48:27.857129+0000","last_undegraded":"2024-04-23T14:48:27.857129+0000","last_fullsized":"2024-04-23T14:48:27.857129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:29.869845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":170,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.506886+0000","last_change":"2024-04-23T14:45:59.229480+0000","last_active":"2024-04-23T14:48:28.506886+0000","last_peered":"2024-04-23T14:48:28.506886+0000","last_clean":"2024-04-23T14:48:28.506886+0000","last_became_active":"2024-04-23T14:45:59.229124+0000","last_became_peered":"2024-04-23T14:45:59.229124+0000","last_unstale":"2024-04-23T14:48:28.506886+0000","last_undegraded":"2024-04-23T14:48:28.506886+0000","last_fullsized":"2024-04-23T14:48:28.506886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:33.837779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049919+0000","last_change":"2024-04-23T14:46:00.017753+0000","last_active":"2024-04-23T14:48:28.049919+0000","last_peered":"2024-04-23T14:48:28.049919+0000","last_clean":"2024-04-23T14:48:28.049919+0000","last_became_active":"2024-04-23T14:46:00.017370+0000","last_became_peered":"2024-04-23T14:46:00.017370+0000","last_unstale":"2024-04-23T14:48:28.049919+0000","last_undegraded":"2024-04-23T14:48:28.049919+0000","last_fullsized":"2024-04-23T14:48:28.049919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:30:35.460570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727402+0000","last_change":"2024-04-23T14:45:59.229361+0000","last_active":"2024-04-23T14:48:27.727402+0000","last_peered":"2024-04-23T14:48:27.727402+0000","last_clean":"2024-04-23T14:48:27.727402+0000","last_became_active":"2024-04-23T14:45:59.229043+0000","last_became_peered":"2024-04-23T14:45:59.229043+0000","last_unstale":"2024-04-23T14:48:27.727402+0000","last_undegraded":"2024-04-23T14:48:27.727402+0000","last_fullsized":"2024-04-23T14:48:27.727402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:09.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735632+0000","last_change":"2024-04-23T14:46:00.030181+0000","last_active":"2024-04-23T14:48:27.735632+0000","last_peered":"2024-04-23T14:48:27.735632+0000","last_clean":"2024-04-23T14:48:27.735632+0000","last_became_active":"2024-04-23T14:46:00.029303+0000","last_became_peered":"2024-04-23T14:46:00.029303+0000","last_unstale":"2024-04-23T14:48:27.735632+0000","last_undegraded":"2024-04-23T14:48:27.735632+0000","last_fullsized":"2024-04-23T14:48:27.735632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:24:53.904501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833914+0000","last_change":"2024-04-23T14:45:59.231270+0000","last_active":"2024-04-23T14:48:27.833914+0000","last_peered":"2024-04-23T14:48:27.833914+0000","last_clean":"2024-04-23T14:48:27.833914+0000","last_became_active":"2024-04-23T14:45:59.230914+0000","last_became_peered":"2024-04-23T14:45:59.230914+0000","last_unstale":"2024-04-23T14:48:27.833914+0000","last_undegraded":"2024-04-23T14:48:27.833914+0000","last_fullsized":"2024-04-23T14:48:27.833914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.422260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049242+0000","last_change":"2024-04-23T14:46:00.011358+0000","last_active":"2024-04-23T14:48:28.049242+0000","last_peered":"2024-04-23T14:48:28.049242+0000","last_clean":"2024-04-23T14:48:28.049242+0000","last_became_active":"2024-04-23T14:46:00.011040+0000","last_became_peered":"2024-04-23T14:46:00.011040+0000","last_unstale":"2024-04-23T14:48:28.049242+0000","last_undegraded":"2024-04-23T14:48:28.049242+0000","last_fullsized":"2024-04-23T14:48:28.049242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:27.248844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":351,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.508135+0000","last_change":"2024-04-23T14:45:59.227195+0000","last_active":"2024-04-23T14:48:28.508135+0000","last_peered":"2024-04-23T14:48:28.508135+0000","last_clean":"2024-04-23T14:48:28.508135+0000","last_became_active":"2024-04-23T14:45:59.226874+0000","last_became_peered":"2024-04-23T14:45:59.226874+0000","last_unstale":"2024-04-23T14:48:28.508135+0000","last_undegraded":"2024-04-23T14:48:28.508135+0000","last_fullsized":"2024-04-23T14:48:28.508135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:18:40.203839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65555,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":296,"num_read_kb":212,"num_write":34,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860491+0000","last_change":"2024-04-23T14:46:00.010215+0000","last_active":"2024-04-23T14:48:27.860491+0000","last_peered":"2024-04-23T14:48:27.860491+0000","last_clean":"2024-04-23T14:48:27.860491+0000","last_became_active":"2024-04-23T14:46:00.009756+0000","last_became_peered":"2024-04-23T14:46:00.009756+0000","last_unstale":"2024-04-23T14:48:27.860491+0000","last_undegraded":"2024-04-23T14:48:27.860491+0000","last_fullsized":"2024-04-23T14:48:27.860491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.797966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049180+0000","last_change":"2024-04-23T14:45:59.239258+0000","last_active":"2024-04-23T14:48:28.049180+0000","last_peered":"2024-04-23T14:48:28.049180+0000","last_clean":"2024-04-23T14:48:28.049180+0000","last_became_active":"2024-04-23T14:45:59.238899+0000","last_became_peered":"2024-04-23T14:45:59.238899+0000","last_unstale":"2024-04-23T14:48:28.049180+0000","last_undegraded":"2024-04-23T14:48:28.049180+0000","last_fullsized":"2024-04-23T14:48:28.049180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:45.866489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049683+0000","last_change":"2024-04-23T14:46:00.036627+0000","last_active":"2024-04-23T14:48:28.049683+0000","last_peered":"2024-04-23T14:48:28.049683+0000","last_clean":"2024-04-23T14:48:28.049683+0000","last_became_active":"2024-04-23T14:46:00.035397+0000","last_became_peered":"2024-04-23T14:46:00.035397+0000","last_unstale":"2024-04-23T14:48:28.049683+0000","last_undegraded":"2024-04-23T14:48:28.049683+0000","last_fullsized":"2024-04-23T14:48:28.049683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:24:02.038813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833573+0000","last_change":"2024-04-23T14:45:59.248241+0000","last_active":"2024-04-23T14:48:27.833573+0000","last_peered":"2024-04-23T14:48:27.833573+0000","last_clean":"2024-04-23T14:48:27.833573+0000","last_became_active":"2024-04-23T14:45:59.247895+0000","last_became_peered":"2024-04-23T14:45:59.247895+0000","last_unstale":"2024-04-23T14:48:27.833573+0000","last_undegraded":"2024-04-23T14:48:27.833573+0000","last_fullsized":"2024-04-23T14:48:27.833573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:01.964714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727084+0000","last_change":"2024-04-23T14:46:58.433781+0000","last_active":"2024-04-23T14:48:27.727084+0000","last_peered":"2024-04-23T14:48:27.727084+0000","last_clean":"2024-04-23T14:48:27.727084+0000","last_became_active":"2024-04-23T14:46:58.433460+0000","last_became_peered":"2024-04-23T14:46:58.433460+0000","last_unstale":"2024-04-23T14:48:27.727084+0000","last_undegraded":"2024-04-23T14:48:27.727084+0000","last_fullsized":"2024-04-23T14:48:27.727084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:44.149949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":177,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.630557+0000","last_change":"2024-04-23T14:45:59.227406+0000","last_active":"2024-04-23T14:48:28.630557+0000","last_peered":"2024-04-23T14:48:28.630557+0000","last_clean":"2024-04-23T14:48:28.630557+0000","last_became_active":"2024-04-23T14:45:59.227063+0000","last_became_peered":"2024-04-23T14:45:59.227063+0000","last_unstale":"2024-04-23T14:48:28.630557+0000","last_undegraded":"2024-04-23T14:48:28.630557+0000","last_fullsized":"2024-04-23T14:48:28.630557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:36.845854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726734+0000","last_change":"2024-04-23T14:46:00.006333+0000","last_active":"2024-04-23T14:48:27.726734+0000","last_peered":"2024-04-23T14:48:27.726734+0000","last_clean":"2024-04-23T14:48:27.726734+0000","last_became_active":"2024-04-23T14:46:00.006024+0000","last_became_peered":"2024-04-23T14:46:00.006024+0000","last_unstale":"2024-04-23T14:48:27.726734+0000","last_undegraded":"2024-04-23T14:48:27.726734+0000","last_fullsized":"2024-04-23T14:48:27.726734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:49.389445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860182+0000","last_change":"2024-04-23T14:45:59.226394+0000","last_active":"2024-04-23T14:48:27.860182+0000","last_peered":"2024-04-23T14:48:27.860182+0000","last_clean":"2024-04-23T14:48:27.860182+0000","last_became_active":"2024-04-23T14:45:59.226088+0000","last_became_peered":"2024-04-23T14:45:59.226088+0000","last_unstale":"2024-04-23T14:48:27.860182+0000","last_undegraded":"2024-04-23T14:48:27.860182+0000","last_fullsized":"2024-04-23T14:48:27.860182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:35.139520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833741+0000","last_change":"2024-04-23T14:46:00.009262+0000","last_active":"2024-04-23T14:48:27.833741+0000","last_peered":"2024-04-23T14:48:27.833741+0000","last_clean":"2024-04-23T14:48:27.833741+0000","last_became_active":"2024-04-23T14:46:00.008865+0000","last_became_peered":"2024-04-23T14:46:00.008865+0000","last_unstale":"2024-04-23T14:48:27.833741+0000","last_undegraded":"2024-04-23T14:48:27.833741+0000","last_fullsized":"2024-04-23T14:48:27.833741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:27.662886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860641+0000","last_change":"2024-04-23T14:45:59.223479+0000","last_active":"2024-04-23T14:48:27.860641+0000","last_peered":"2024-04-23T14:48:27.860641+0000","last_clean":"2024-04-23T14:48:27.860641+0000","last_became_active":"2024-04-23T14:45:59.222997+0000","last_became_peered":"2024-04-23T14:45:59.222997+0000","last_unstale":"2024-04-23T14:48:27.860641+0000","last_undegraded":"2024-04-23T14:48:27.860641+0000","last_fullsized":"2024-04-23T14:48:27.860641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:29:58.026918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087709+0000","last_change":"2024-04-23T14:46:58.434121+0000","last_active":"2024-04-23T14:48:28.087709+0000","last_peered":"2024-04-23T14:48:28.087709+0000","last_clean":"2024-04-23T14:48:28.087709+0000","last_became_active":"2024-04-23T14:46:58.433746+0000","last_became_peered":"2024-04-23T14:46:58.433746+0000","last_unstale":"2024-04-23T14:48:28.087709+0000","last_undegraded":"2024-04-23T14:48:28.087709+0000","last_fullsized":"2024-04-23T14:48:28.087709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:33:22.493604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.856389+0000","last_change":"2024-04-23T14:45:59.237813+0000","last_active":"2024-04-23T14:48:27.856389+0000","last_peered":"2024-04-23T14:48:27.856389+0000","last_clean":"2024-04-23T14:48:27.856389+0000","last_became_active":"2024-04-23T14:45:59.237354+0000","last_became_peered":"2024-04-23T14:45:59.237354+0000","last_unstale":"2024-04-23T14:48:27.856389+0000","last_undegraded":"2024-04-23T14:48:27.856389+0000","last_fullsized":"2024-04-23T14:48:27.856389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:35:38.323059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869336+0000","last_change":"2024-04-23T14:46:00.031956+0000","last_active":"2024-04-23T14:48:27.869336+0000","last_peered":"2024-04-23T14:48:27.869336+0000","last_clean":"2024-04-23T14:48:27.869336+0000","last_became_active":"2024-04-23T14:46:00.031666+0000","last_became_peered":"2024-04-23T14:46:00.031666+0000","last_unstale":"2024-04-23T14:48:27.869336+0000","last_undegraded":"2024-04-23T14:48:27.869336+0000","last_fullsized":"2024-04-23T14:48:27.869336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:17.346455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726835+0000","last_change":"2024-04-23T14:45:59.230633+0000","last_active":"2024-04-23T14:48:27.726835+0000","last_peered":"2024-04-23T14:48:27.726835+0000","last_clean":"2024-04-23T14:48:27.726835+0000","last_became_active":"2024-04-23T14:45:59.230300+0000","last_became_peered":"2024-04-23T14:45:59.230300+0000","last_unstale":"2024-04-23T14:48:27.726835+0000","last_undegraded":"2024-04-23T14:48:27.726835+0000","last_fullsized":"2024-04-23T14:48:27.726835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:41.170055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869090+0000","last_change":"2024-04-23T14:46:00.030891+0000","last_active":"2024-04-23T14:48:27.869090+0000","last_peered":"2024-04-23T14:48:27.869090+0000","last_clean":"2024-04-23T14:48:27.869090+0000","last_became_active":"2024-04-23T14:46:00.030438+0000","last_became_peered":"2024-04-23T14:46:00.030438+0000","last_unstale":"2024-04-23T14:48:27.869090+0000","last_undegraded":"2024-04-23T14:48:27.869090+0000","last_fullsized":"2024-04-23T14:48:27.869090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:43.038111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833930+0000","last_change":"2024-04-23T14:46:00.013683+0000","last_active":"2024-04-23T14:48:27.833930+0000","last_peered":"2024-04-23T14:48:27.833930+0000","last_clean":"2024-04-23T14:48:27.833930+0000","last_became_active":"2024-04-23T14:46:00.013356+0000","last_became_peered":"2024-04-23T14:46:00.013356+0000","last_unstale":"2024-04-23T14:48:27.833930+0000","last_undegraded":"2024-04-23T14:48:27.833930+0000","last_fullsized":"2024-04-23T14:48:27.833930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:32.866694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.053312+0000","last_change":"2024-04-23T14:45:59.252258+0000","last_active":"2024-04-23T14:48:28.053312+0000","last_peered":"2024-04-23T14:48:28.053312+0000","last_clean":"2024-04-23T14:48:28.053312+0000","last_became_active":"2024-04-23T14:45:59.251513+0000","last_became_peered":"2024-04-23T14:45:59.251513+0000","last_unstale":"2024-04-23T14:48:28.053312+0000","last_undegraded":"2024-04-23T14:48:28.053312+0000","last_fullsized":"2024-04-23T14:48:28.053312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:55:37.087807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.732380+0000","last_change":"2024-04-23T14:46:00.009468+0000","last_active":"2024-04-23T14:48:27.732380+0000","last_peered":"2024-04-23T14:48:27.732380+0000","last_clean":"2024-04-23T14:48:27.732380+0000","last_became_active":"2024-04-23T14:46:00.003118+0000","last_became_peered":"2024-04-23T14:46:00.003118+0000","last_unstale":"2024-04-23T14:48:27.732380+0000","last_undegraded":"2024-04-23T14:48:27.732380+0000","last_fullsized":"2024-04-23T14:48:27.732380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:20:24.488848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050208+0000","last_change":"2024-04-23T14:45:59.225758+0000","last_active":"2024-04-23T14:48:28.050208+0000","last_peered":"2024-04-23T14:48:28.050208+0000","last_clean":"2024-04-23T14:48:28.050208+0000","last_became_active":"2024-04-23T14:45:59.225427+0000","last_became_peered":"2024-04-23T14:45:59.225427+0000","last_unstale":"2024-04-23T14:48:28.050208+0000","last_undegraded":"2024-04-23T14:48:28.050208+0000","last_fullsized":"2024-04-23T14:48:28.050208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:59.495348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":354,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726734+0000","last_change":"2024-04-23T14:44:39.684486+0000","last_active":"2024-04-23T14:48:27.726734+0000","last_peered":"2024-04-23T14:48:27.726734+0000","last_clean":"2024-04-23T14:48:27.726734+0000","last_became_active":"2024-04-23T14:44:39.684074+0000","last_became_peered":"2024-04-23T14:44:39.684074+0000","last_unstale":"2024-04-23T14:48:27.726734+0000","last_undegraded":"2024-04-23T14:48:27.726734+0000","last_fullsized":"2024-04-23T14:48:27.726734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_clean_scrub_stamp":"2024-04-23T14:40:41.333258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:11:16.865449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049334+0000","last_change":"2024-04-23T14:46:00.025018+0000","last_active":"2024-04-23T14:48:28.049334+0000","last_peered":"2024-04-23T14:48:28.049334+0000","last_clean":"2024-04-23T14:48:28.049334+0000","last_became_active":"2024-04-23T14:46:00.024695+0000","last_became_peered":"2024-04-23T14:46:00.024695+0000","last_unstale":"2024-04-23T14:48:28.049334+0000","last_undegraded":"2024-04-23T14:48:28.049334+0000","last_fullsized":"2024-04-23T14:48:28.049334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:00:31.375963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726980+0000","last_change":"2024-04-23T14:45:59.225849+0000","last_active":"2024-04-23T14:48:27.726980+0000","last_peered":"2024-04-23T14:48:27.726980+0000","last_clean":"2024-04-23T14:48:27.726980+0000","last_became_active":"2024-04-23T14:45:59.225402+0000","last_became_peered":"2024-04-23T14:45:59.225402+0000","last_unstale":"2024-04-23T14:48:27.726980+0000","last_undegraded":"2024-04-23T14:48:27.726980+0000","last_fullsized":"2024-04-23T14:48:27.726980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:10:42.217717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087726+0000","last_change":"2024-04-23T14:46:58.377666+0000","last_active":"2024-04-23T14:48:28.087726+0000","last_peered":"2024-04-23T14:48:28.087726+0000","last_clean":"2024-04-23T14:48:28.087726+0000","last_became_active":"2024-04-23T14:46:58.370533+0000","last_became_peered":"2024-04-23T14:46:58.370533+0000","last_unstale":"2024-04-23T14:48:28.087726+0000","last_undegraded":"2024-04-23T14:48:28.087726+0000","last_fullsized":"2024-04-23T14:48:28.087726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:39:58.606732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049771+0000","last_change":"2024-04-23T14:46:00.039566+0000","last_active":"2024-04-23T14:48:28.049771+0000","last_peered":"2024-04-23T14:48:28.049771+0000","last_clean":"2024-04-23T14:48:28.049771+0000","last_became_active":"2024-04-23T14:46:00.039236+0000","last_became_peered":"2024-04-23T14:46:00.039236+0000","last_unstale":"2024-04-23T14:48:28.049771+0000","last_undegraded":"2024-04-23T14:48:28.049771+0000","last_fullsized":"2024-04-23T14:48:28.049771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:44.028367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'46","reported_seq":91,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.000361+0000","last_change":"2024-04-23T14:45:59.246192+0000","last_active":"2024-04-23T14:48:28.000361+0000","last_peered":"2024-04-23T14:48:28.000361+0000","last_clean":"2024-04-23T14:48:28.000361+0000","last_became_active":"2024-04-23T14:45:59.245873+0000","last_became_peered":"2024-04-23T14:45:59.245873+0000","last_unstale":"2024-04-23T14:48:28.000361+0000","last_undegraded":"2024-04-23T14:48:28.000361+0000","last_fullsized":"2024-04-23T14:48:28.000361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:22.915746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.733234+0000","last_change":"2024-04-23T14:46:00.029176+0000","last_active":"2024-04-23T14:48:27.733234+0000","last_peered":"2024-04-23T14:48:27.733234+0000","last_clean":"2024-04-23T14:48:27.733234+0000","last_became_active":"2024-04-23T14:46:00.028741+0000","last_became_peered":"2024-04-23T14:46:00.028741+0000","last_unstale":"2024-04-23T14:48:27.733234+0000","last_undegraded":"2024-04-23T14:48:27.733234+0000","last_fullsized":"2024-04-23T14:48:27.733234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:35:22.905348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049510+0000","last_change":"2024-04-23T14:45:59.231490+0000","last_active":"2024-04-23T14:48:28.049510+0000","last_peered":"2024-04-23T14:48:28.049510+0000","last_clean":"2024-04-23T14:48:28.049510+0000","last_became_active":"2024-04-23T14:45:59.231143+0000","last_became_peered":"2024-04-23T14:45:59.231143+0000","last_unstale":"2024-04-23T14:48:28.049510+0000","last_undegraded":"2024-04-23T14:48:28.049510+0000","last_fullsized":"2024-04-23T14:48:28.049510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:47:26.398150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.736394+0000","last_change":"2024-04-23T14:46:00.035838+0000","last_active":"2024-04-23T14:48:27.736394+0000","last_peered":"2024-04-23T14:48:27.736394+0000","last_clean":"2024-04-23T14:48:27.736394+0000","last_became_active":"2024-04-23T14:46:00.034498+0000","last_became_peered":"2024-04-23T14:46:00.034498+0000","last_unstale":"2024-04-23T14:48:27.736394+0000","last_undegraded":"2024-04-23T14:48:27.736394+0000","last_fullsized":"2024-04-23T14:48:27.736394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:28:15.748503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049161+0000","last_change":"2024-04-23T14:45:59.240817+0000","last_active":"2024-04-23T14:48:28.049161+0000","last_peered":"2024-04-23T14:48:28.049161+0000","last_clean":"2024-04-23T14:48:28.049161+0000","last_became_active":"2024-04-23T14:45:59.240557+0000","last_became_peered":"2024-04-23T14:45:59.240557+0000","last_unstale":"2024-04-23T14:48:28.049161+0000","last_undegraded":"2024-04-23T14:48:28.049161+0000","last_fullsized":"2024-04-23T14:48:28.049161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:03.099054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860155+0000","last_change":"2024-04-23T14:46:00.009094+0000","last_active":"2024-04-23T14:48:27.860155+0000","last_peered":"2024-04-23T14:48:27.860155+0000","last_clean":"2024-04-23T14:48:27.860155+0000","last_became_active":"2024-04-23T14:46:00.008751+0000","last_became_peered":"2024-04-23T14:46:00.008751+0000","last_unstale":"2024-04-23T14:48:27.860155+0000","last_undegraded":"2024-04-23T14:48:27.860155+0000","last_fullsized":"2024-04-23T14:48:27.860155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:12.480737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.728665+0000","last_change":"2024-04-23T14:45:59.249882+0000","last_active":"2024-04-23T14:48:27.728665+0000","last_peered":"2024-04-23T14:48:27.728665+0000","last_clean":"2024-04-23T14:48:27.728665+0000","last_became_active":"2024-04-23T14:45:59.249072+0000","last_became_peered":"2024-04-23T14:45:59.249072+0000","last_unstale":"2024-04-23T14:48:27.728665+0000","last_undegraded":"2024-04-23T14:48:27.728665+0000","last_fullsized":"2024-04-23T14:48:27.728665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:04:16.573005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087453+0000","last_change":"2024-04-23T14:46:00.030903+0000","last_active":"2024-04-23T14:48:28.087453+0000","last_peered":"2024-04-23T14:48:28.087453+0000","last_clean":"2024-04-23T14:48:28.087453+0000","last_became_active":"2024-04-23T14:46:00.030051+0000","last_became_peered":"2024-04-23T14:46:00.030051+0000","last_unstale":"2024-04-23T14:48:28.087453+0000","last_undegraded":"2024-04-23T14:48:28.087453+0000","last_fullsized":"2024-04-23T14:48:28.087453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:41:15.915795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049098+0000","last_change":"2024-04-23T14:45:59.236446+0000","last_active":"2024-04-23T14:48:28.049098+0000","last_peered":"2024-04-23T14:48:28.049098+0000","last_clean":"2024-04-23T14:48:28.049098+0000","last_became_active":"2024-04-23T14:45:59.236119+0000","last_became_peered":"2024-04-23T14:45:59.236119+0000","last_unstale":"2024-04-23T14:48:28.049098+0000","last_undegraded":"2024-04-23T14:48:28.049098+0000","last_fullsized":"2024-04-23T14:48:28.049098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:40.153626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.732671+0000","last_change":"2024-04-23T14:46:00.035577+0000","last_active":"2024-04-23T14:48:27.732671+0000","last_peered":"2024-04-23T14:48:27.732671+0000","last_clean":"2024-04-23T14:48:27.732671+0000","last_became_active":"2024-04-23T14:46:00.034929+0000","last_became_peered":"2024-04-23T14:46:00.034929+0000","last_unstale":"2024-04-23T14:48:27.732671+0000","last_undegraded":"2024-04-23T14:48:27.732671+0000","last_fullsized":"2024-04-23T14:48:27.732671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:22.844672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860296+0000","last_change":"2024-04-23T14:46:58.440882+0000","last_active":"2024-04-23T14:48:27.860296+0000","last_peered":"2024-04-23T14:48:27.860296+0000","last_clean":"2024-04-23T14:48:27.860296+0000","last_became_active":"2024-04-23T14:46:58.436914+0000","last_became_peered":"2024-04-23T14:46:58.436914+0000","last_unstale":"2024-04-23T14:48:27.860296+0000","last_undegraded":"2024-04-23T14:48:27.860296+0000","last_fullsized":"2024-04-23T14:48:27.860296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:36.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727242+0000","last_change":"2024-04-23T14:46:00.027934+0000","last_active":"2024-04-23T14:48:27.727242+0000","last_peered":"2024-04-23T14:48:27.727242+0000","last_clean":"2024-04-23T14:48:27.727242+0000","last_became_active":"2024-04-23T14:46:00.027571+0000","last_became_peered":"2024-04-23T14:46:00.027571+0000","last_unstale":"2024-04-23T14:48:27.727242+0000","last_undegraded":"2024-04-23T14:48:27.727242+0000","last_fullsized":"2024-04-23T14:48:27.727242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:54:27.890062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049081+0000","last_change":"2024-04-23T14:45:59.236967+0000","last_active":"2024-04-23T14:48:28.049081+0000","last_peered":"2024-04-23T14:48:28.049081+0000","last_clean":"2024-04-23T14:48:28.049081+0000","last_became_active":"2024-04-23T14:45:59.236612+0000","last_became_peered":"2024-04-23T14:45:59.236612+0000","last_unstale":"2024-04-23T14:48:28.049081+0000","last_undegraded":"2024-04-23T14:48:28.049081+0000","last_fullsized":"2024-04-23T14:48:28.049081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:58:09.422598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727288+0000","last_change":"2024-04-23T14:46:00.009289+0000","last_active":"2024-04-23T14:48:27.727288+0000","last_peered":"2024-04-23T14:48:27.727288+0000","last_clean":"2024-04-23T14:48:27.727288+0000","last_became_active":"2024-04-23T14:46:00.008911+0000","last_became_peered":"2024-04-23T14:46:00.008911+0000","last_unstale":"2024-04-23T14:48:27.727288+0000","last_undegraded":"2024-04-23T14:48:27.727288+0000","last_fullsized":"2024-04-23T14:48:27.727288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:47:55.772442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.994184+0000","last_change":"2024-04-23T14:45:59.239494+0000","last_active":"2024-04-23T14:48:27.994184+0000","last_peered":"2024-04-23T14:48:27.994184+0000","last_clean":"2024-04-23T14:48:27.994184+0000","last_became_active":"2024-04-23T14:45:59.239157+0000","last_became_peered":"2024-04-23T14:45:59.239157+0000","last_unstale":"2024-04-23T14:48:27.994184+0000","last_undegraded":"2024-04-23T14:48:27.994184+0000","last_fullsized":"2024-04-23T14:48:27.994184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:38.275320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726982+0000","last_change":"2024-04-23T14:46:00.029401+0000","last_active":"2024-04-23T14:48:27.726982+0000","last_peered":"2024-04-23T14:48:27.726982+0000","last_clean":"2024-04-23T14:48:27.726982+0000","last_became_active":"2024-04-23T14:46:00.029006+0000","last_became_peered":"2024-04-23T14:46:00.029006+0000","last_unstale":"2024-04-23T14:48:27.726982+0000","last_undegraded":"2024-04-23T14:48:27.726982+0000","last_fullsized":"2024-04-23T14:48:27.726982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:11.617083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050600+0000","last_change":"2024-04-23T14:45:59.247636+0000","last_active":"2024-04-23T14:48:28.050600+0000","last_peered":"2024-04-23T14:48:28.050600+0000","last_clean":"2024-04-23T14:48:28.050600+0000","last_became_active":"2024-04-23T14:45:59.247231+0000","last_became_peered":"2024-04-23T14:45:59.247231+0000","last_unstale":"2024-04-23T14:48:28.050600+0000","last_undegraded":"2024-04-23T14:48:28.050600+0000","last_fullsized":"2024-04-23T14:48:28.050600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:54.291463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727318+0000","last_change":"2024-04-23T14:46:00.010825+0000","last_active":"2024-04-23T14:48:27.727318+0000","last_peered":"2024-04-23T14:48:27.727318+0000","last_clean":"2024-04-23T14:48:27.727318+0000","last_became_active":"2024-04-23T14:46:00.010470+0000","last_became_peered":"2024-04-23T14:46:00.010470+0000","last_unstale":"2024-04-23T14:48:27.727318+0000","last_undegraded":"2024-04-23T14:48:27.727318+0000","last_fullsized":"2024-04-23T14:48:27.727318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:19.851825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087509+0000","last_change":"2024-04-23T14:46:00.011967+0000","last_active":"2024-04-23T14:48:28.087509+0000","last_peered":"2024-04-23T14:48:28.087509+0000","last_clean":"2024-04-23T14:48:28.087509+0000","last_became_active":"2024-04-23T14:46:00.011580+0000","last_became_peered":"2024-04-23T14:46:00.011580+0000","last_unstale":"2024-04-23T14:48:28.087509+0000","last_undegraded":"2024-04-23T14:48:28.087509+0000","last_fullsized":"2024-04-23T14:48:28.087509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:25:38.909994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.048960+0000","last_change":"2024-04-23T14:46:58.436615+0000","last_active":"2024-04-23T14:48:28.048960+0000","last_peered":"2024-04-23T14:48:28.048960+0000","last_clean":"2024-04-23T14:48:28.048960+0000","last_became_active":"2024-04-23T14:46:58.436288+0000","last_became_peered":"2024-04-23T14:46:58.436288+0000","last_unstale":"2024-04-23T14:48:28.048960+0000","last_undegraded":"2024-04-23T14:48:28.048960+0000","last_fullsized":"2024-04-23T14:48:28.048960+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:49.614004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.048911+0000","last_change":"2024-04-23T14:46:00.029874+0000","last_active":"2024-04-23T14:48:28.048911+0000","last_peered":"2024-04-23T14:48:28.048911+0000","last_clean":"2024-04-23T14:48:28.048911+0000","last_became_active":"2024-04-23T14:46:00.029520+0000","last_became_peered":"2024-04-23T14:46:00.029520+0000","last_unstale":"2024-04-23T14:48:28.048911+0000","last_undegraded":"2024-04-23T14:48:28.048911+0000","last_fullsized":"2024-04-23T14:48:28.048911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:30.711566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735746+0000","last_change":"2024-04-23T14:46:00.039545+0000","last_active":"2024-04-23T14:48:27.735746+0000","last_peered":"2024-04-23T14:48:27.735746+0000","last_clean":"2024-04-23T14:48:27.735746+0000","last_became_active":"2024-04-23T14:46:00.039193+0000","last_became_peered":"2024-04-23T14:46:00.039193+0000","last_unstale":"2024-04-23T14:48:27.735746+0000","last_undegraded":"2024-04-23T14:48:27.735746+0000","last_fullsized":"2024-04-23T14:48:27.735746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:18:13.528129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050273+0000","last_change":"2024-04-23T14:46:00.036816+0000","last_active":"2024-04-23T14:48:28.050273+0000","last_peered":"2024-04-23T14:48:28.050273+0000","last_clean":"2024-04-23T14:48:28.050273+0000","last_became_active":"2024-04-23T14:46:00.036372+0000","last_became_peered":"2024-04-23T14:46:00.036372+0000","last_unstale":"2024-04-23T14:48:28.050273+0000","last_undegraded":"2024-04-23T14:48:28.050273+0000","last_fullsized":"2024-04-23T14:48:28.050273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:56.179724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087795+0000","last_change":"2024-04-23T14:46:00.012058+0000","last_active":"2024-04-23T14:48:28.087795+0000","last_peered":"2024-04-23T14:48:28.087795+0000","last_clean":"2024-04-23T14:48:28.087795+0000","last_became_active":"2024-04-23T14:46:00.011595+0000","last_became_peered":"2024-04-23T14:46:00.011595+0000","last_unstale":"2024-04-23T14:48:28.087795+0000","last_undegraded":"2024-04-23T14:48:28.087795+0000","last_fullsized":"2024-04-23T14:48:28.087795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:28.341442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049273+0000","last_change":"2024-04-23T14:46:00.011419+0000","last_active":"2024-04-23T14:48:28.049273+0000","last_peered":"2024-04-23T14:48:28.049273+0000","last_clean":"2024-04-23T14:48:28.049273+0000","last_became_active":"2024-04-23T14:46:00.011004+0000","last_became_peered":"2024-04-23T14:46:00.011004+0000","last_unstale":"2024-04-23T14:48:28.049273+0000","last_undegraded":"2024-04-23T14:48:28.049273+0000","last_fullsized":"2024-04-23T14:48:28.049273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:46:57.451648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.733118+0000","last_change":"2024-04-23T14:46:00.023583+0000","last_active":"2024-04-23T14:48:27.733118+0000","last_peered":"2024-04-23T14:48:27.733118+0000","last_clean":"2024-04-23T14:48:27.733118+0000","last_became_active":"2024-04-23T14:46:00.023195+0000","last_became_peered":"2024-04-23T14:46:00.023195+0000","last_unstale":"2024-04-23T14:48:27.733118+0000","last_undegraded":"2024-04-23T14:48:27.733118+0000","last_fullsized":"2024-04-23T14:48:27.733118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:39.992959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833812+0000","last_change":"2024-04-23T14:46:00.014719+0000","last_active":"2024-04-23T14:48:27.833812+0000","last_peered":"2024-04-23T14:48:27.833812+0000","last_clean":"2024-04-23T14:48:27.833812+0000","last_became_active":"2024-04-23T14:46:00.014349+0000","last_became_peered":"2024-04-23T14:46:00.014349+0000","last_unstale":"2024-04-23T14:48:27.833812+0000","last_undegraded":"2024-04-23T14:48:27.833812+0000","last_fullsized":"2024-04-23T14:48:27.833812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:12.789316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.728187+0000","last_change":"2024-04-23T14:46:00.019162+0000","last_active":"2024-04-23T14:48:27.728187+0000","last_peered":"2024-04-23T14:48:27.728187+0000","last_clean":"2024-04-23T14:48:27.728187+0000","last_became_active":"2024-04-23T14:46:00.018814+0000","last_became_peered":"2024-04-23T14:46:00.018814+0000","last_unstale":"2024-04-23T14:48:27.728187+0000","last_undegraded":"2024-04-23T14:48:27.728187+0000","last_fullsized":"2024-04-23T14:48:27.728187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:10.741148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049189+0000","last_change":"2024-04-23T14:46:00.007254+0000","last_active":"2024-04-23T14:48:28.049189+0000","last_peered":"2024-04-23T14:48:28.049189+0000","last_clean":"2024-04-23T14:48:28.049189+0000","last_became_active":"2024-04-23T14:46:00.006826+0000","last_became_peered":"2024-04-23T14:46:00.006826+0000","last_unstale":"2024-04-23T14:48:28.049189+0000","last_undegraded":"2024-04-23T14:48:28.049189+0000","last_fullsized":"2024-04-23T14:48:28.049189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:29.155244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050003+0000","last_change":"2024-04-23T14:46:00.039265+0000","last_active":"2024-04-23T14:48:28.050003+0000","last_peered":"2024-04-23T14:48:28.050003+0000","last_clean":"2024-04-23T14:48:28.050003+0000","last_became_active":"2024-04-23T14:46:00.038848+0000","last_became_peered":"2024-04-23T14:46:00.038848+0000","last_unstale":"2024-04-23T14:48:28.050003+0000","last_undegraded":"2024-04-23T14:48:28.050003+0000","last_fullsized":"2024-04-23T14:48:28.050003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:10.218655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087627+0000","last_change":"2024-04-23T14:46:00.012134+0000","last_active":"2024-04-23T14:48:28.087627+0000","last_peered":"2024-04-23T14:48:28.087627+0000","last_clean":"2024-04-23T14:48:28.087627+0000","last_became_active":"2024-04-23T14:46:00.011796+0000","last_became_peered":"2024-04-23T14:46:00.011796+0000","last_unstale":"2024-04-23T14:48:28.087627+0000","last_undegraded":"2024-04-23T14:48:28.087627+0000","last_fullsized":"2024-04-23T14:48:28.087627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:41.671419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107307,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":595827,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":975,"num_read_kb":601,"num_write":288,"num_write_kb":1436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1035,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":1787481,"data_compressed":81462,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":470628,"data_compressed":8702,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":1260165,"data_compressed":52479,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":864058,"data_compressed":27848,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037821,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":863956,"data_compressed":27848,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":1064147,"data_compressed":44471,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495687,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":470231,"data_compressed":8702,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":537124,"data_compressed":10554,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":471502,"data_compressed":8702,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":536307,"data_compressed":10554,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":995358,"data_compressed":34004,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920851,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":471591,"data_compressed":8702,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1194508,"data_compressed":50627,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67100,"data_compressed":1852,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66283,"data_compressed":1852,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133999,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":199656,"data_compressed":8008,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:48:33.750 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-23T14:48:33.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T14:48:33.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.955+0000 7f2d8f200640 1 -- 172.21.15.83:0/688624310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d88104260 msgr2=0x7f2d88104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.955+0000 7f2d8f200640 1 --2- 172.21.15.83:0/688624310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d88104260 0x7f2d88104680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009900 tx=0x7f2d7802a400 comp rx=0 tx=0).stop 2024-04-23T14:48:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.956+0000 7f2d8f200640 1 -- 172.21.15.83:0/688624310 shutdown_connections 2024-04-23T14:48:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.956+0000 7f2d8f200640 1 --2- 172.21.15.83:0/688624310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d88105e10 0x7f2d8810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.956+0000 7f2d8f200640 1 --2- 172.21.15.83:0/688624310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88105450 0x7f2d881058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.956+0000 7f2d8f200640 1 --2- 172.21.15.83:0/688624310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d88104260 0x7f2d88104680 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.956+0000 7f2d8f200640 1 -- 172.21.15.83:0/688624310 >> 172.21.15.83:0/688624310 conn(0x7f2d880ff7f0 msgr2=0x7f2d88101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.956+0000 7f2d8f200640 1 -- 172.21.15.83:0/688624310 shutdown_connections 2024-04-23T14:48:33.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.956+0000 7f2d8f200640 1 -- 172.21.15.83:0/688624310 wait complete. 2024-04-23T14:48:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d8f200640 1 Processor -- start 2024-04-23T14:48:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d8f200640 1 -- start start 2024-04-23T14:48:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d8f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d88104260 0x7f2d8819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d8f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d88105450 0x7f2d8819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d8f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88105e10 0x7f2d881a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d8f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8810ff40 con 0x7f2d88105e10 2024-04-23T14:48:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d8f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8810fdc0 con 0x7f2d88104260 2024-04-23T14:48:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d8f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d881100c0 con 0x7f2d88105450 2024-04-23T14:48:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d87e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d88104260 0x7f2d8819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d87e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d88104260 0x7f2d8819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:43518/0 (socket says 172.21.15.83:43518) 2024-04-23T14:48:33.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d87e00640 1 -- 172.21.15.83:0/3878383935 learned_addr learned my addr 172.21.15.83:0/3878383935 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d87400640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d88105450 0x7f2d8819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.957+0000 7f2d8d200640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88105e10 0x7f2d881a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.958+0000 7f2d87e00640 1 -- 172.21.15.83:0/3878383935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d88105450 msgr2=0x7f2d8819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.958+0000 7f2d87e00640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d88105450 0x7f2d8819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.958+0000 7f2d87e00640 1 -- 172.21.15.83:0/3878383935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88105e10 msgr2=0x7f2d881a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.958+0000 7f2d87e00640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88105e10 0x7f2d881a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.958+0000 7f2d87e00640 1 -- 172.21.15.83:0/3878383935 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d881a21f0 con 0x7f2d88104260 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.958+0000 7f2d87e00640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d88104260 0x7f2d8819a5d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2d78031440 tx=0x7f2d78032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.958+0000 7f2d84c00640 1 -- 172.21.15.83:0/3878383935 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7803f030 con 0x7f2d88104260 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.959+0000 7f2d84c00640 1 -- 172.21.15.83:0/3878383935 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d78003ce0 con 0x7f2d88104260 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.959+0000 7f2d84c00640 1 -- 172.21.15.83:0/3878383935 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d78035070 con 0x7f2d88104260 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.959+0000 7f2d8f200640 1 -- 172.21.15.83:0/3878383935 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d881a2480 con 0x7f2d88104260 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.959+0000 7f2d8f200640 1 -- 172.21.15.83:0/3878383935 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d881a2920 con 0x7f2d88104260 2024-04-23T14:48:33.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.960+0000 7f2d8f200640 1 -- 172.21.15.83:0/3878383935 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d8810b580 con 0x7f2d88104260 2024-04-23T14:48:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.962+0000 7f2d84c00640 1 -- 172.21.15.83:0/3878383935 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f2d78042030 con 0x7f2d88104260 2024-04-23T14:48:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.963+0000 7f2d84c00640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2d6c0a3ec0 0x7f2d6c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.963+0000 7f2d84c00640 1 -- 172.21.15.83:0/3878383935 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f2d780f0df0 con 0x7f2d88104260 2024-04-23T14:48:33.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.963+0000 7f2d87400640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2d6c0a3ec0 0x7f2d6c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:33.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.964+0000 7f2d87400640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2d6c0a3ec0 0x7f2d6c0a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2d8819bae0 tx=0x7f2d74005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:33.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:33.968+0000 7f2d84c00640 1 -- 172.21.15.83:0/3878383935 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d780bcc70 con 0x7f2d88104260 2024-04-23T14:48:34.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.170+0000 7f2d8f200640 1 -- 172.21.15.83:0/3878383935 --> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d88062440 con 0x7f2d6c0a3ec0 2024-04-23T14:48:34.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.178+0000 7f2d84c00640 1 -- 172.21.15.83:0/3878383935 <== mgr.15114 v2:172.21.15.83:6800/1667121447 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258061 (secure 0 0 0) 0x7f2d88062440 con 0x7f2d6c0a3ec0 2024-04-23T14:48:34.183 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:34.184 INFO:teuthology.orchestra.run.smithi083.stderr:dumped all 2024-04-23T14:48:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.184+0000 7f2d8f200640 1 -- 172.21.15.83:0/3878383935 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2d6c0a3ec0 msgr2=0x7f2d6c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.184+0000 7f2d8f200640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2d6c0a3ec0 0x7f2d6c0a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2d8819bae0 tx=0x7f2d74005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.184+0000 7f2d8f200640 1 -- 172.21.15.83:0/3878383935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d88104260 msgr2=0x7f2d8819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.184+0000 7f2d8f200640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d88104260 0x7f2d8819a5d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2d78031440 tx=0x7f2d78032ce0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.184+0000 7f2d8f200640 1 -- 172.21.15.83:0/3878383935 shutdown_connections 2024-04-23T14:48:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.184+0000 7f2d8f200640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2d6c0a3ec0 0x7f2d6c0a6380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.184+0000 7f2d8f200640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d88105e10 0x7f2d881a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.184+0000 7f2d8f200640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d88105450 0x7f2d8819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.184+0000 7f2d8f200640 1 --2- 172.21.15.83:0/3878383935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d88104260 0x7f2d8819a5d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.184+0000 7f2d8f200640 1 -- 172.21.15.83:0/3878383935 >> 172.21.15.83:0/3878383935 conn(0x7f2d880ff7f0 msgr2=0x7f2d88108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.184+0000 7f2d8f200640 1 -- 172.21.15.83:0/3878383935 shutdown_connections 2024-04-23T14:48:34.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.184+0000 7f2d8f200640 1 -- 172.21.15.83:0/3878383935 wait complete. 2024-04-23T14:48:34.202 INFO:teuthology.orchestra.run.smithi083.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-23T14:48:33.744728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1186429,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1111,"num_read_kb":720,"num_write":471,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181771,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381976,"kb_used_data":11988,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559848,"statfs":{"total":1151940427776,"available":1151549284352,"internally_reserved":0,"allocated":12275712,"data_stored":9199575,"data_compressed":293193,"data_compressed_allocated":393216,"data_compressed_original":3637248,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.971067"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087755+0000","last_change":"2024-04-23T14:46:00.030934+0000","last_active":"2024-04-23T14:48:28.087755+0000","last_peered":"2024-04-23T14:48:28.087755+0000","last_clean":"2024-04-23T14:48:28.087755+0000","last_became_active":"2024-04-23T14:46:00.030595+0000","last_became_peered":"2024-04-23T14:46:00.030595+0000","last_unstale":"2024-04-23T14:48:28.087755+0000","last_undegraded":"2024-04-23T14:48:28.087755+0000","last_fullsized":"2024-04-23T14:48:28.087755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:34:50.814685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.736175+0000","last_change":"2024-04-23T14:46:00.023044+0000","last_active":"2024-04-23T14:48:27.736175+0000","last_peered":"2024-04-23T14:48:27.736175+0000","last_clean":"2024-04-23T14:48:27.736175+0000","last_became_active":"2024-04-23T14:46:00.022441+0000","last_became_peered":"2024-04-23T14:46:00.022441+0000","last_unstale":"2024-04-23T14:48:27.736175+0000","last_undegraded":"2024-04-23T14:48:27.736175+0000","last_fullsized":"2024-04-23T14:48:27.736175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:24:20.976312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735374+0000","last_change":"2024-04-23T14:46:00.025397+0000","last_active":"2024-04-23T14:48:27.735374+0000","last_peered":"2024-04-23T14:48:27.735374+0000","last_clean":"2024-04-23T14:48:27.735374+0000","last_became_active":"2024-04-23T14:46:00.024477+0000","last_became_peered":"2024-04-23T14:46:00.024477+0000","last_unstale":"2024-04-23T14:48:27.735374+0000","last_undegraded":"2024-04-23T14:48:27.735374+0000","last_fullsized":"2024-04-23T14:48:27.735374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:16.002964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049660+0000","last_change":"2024-04-23T14:46:00.013960+0000","last_active":"2024-04-23T14:48:28.049660+0000","last_peered":"2024-04-23T14:48:28.049660+0000","last_clean":"2024-04-23T14:48:28.049660+0000","last_became_active":"2024-04-23T14:46:00.013656+0000","last_became_peered":"2024-04-23T14:46:00.013656+0000","last_unstale":"2024-04-23T14:48:28.049660+0000","last_undegraded":"2024-04-23T14:48:28.049660+0000","last_fullsized":"2024-04-23T14:48:28.049660+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:45:59.831155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860225+0000","last_change":"2024-04-23T14:46:00.032336+0000","last_active":"2024-04-23T14:48:27.860225+0000","last_peered":"2024-04-23T14:48:27.860225+0000","last_clean":"2024-04-23T14:48:27.860225+0000","last_became_active":"2024-04-23T14:46:00.032008+0000","last_became_peered":"2024-04-23T14:46:00.032008+0000","last_unstale":"2024-04-23T14:48:27.860225+0000","last_undegraded":"2024-04-23T14:48:27.860225+0000","last_fullsized":"2024-04-23T14:48:27.860225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:13:05.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869145+0000","last_change":"2024-04-23T14:46:00.015457+0000","last_active":"2024-04-23T14:48:27.869145+0000","last_peered":"2024-04-23T14:48:27.869145+0000","last_clean":"2024-04-23T14:48:27.869145+0000","last_became_active":"2024-04-23T14:46:00.015199+0000","last_became_peered":"2024-04-23T14:46:00.015199+0000","last_unstale":"2024-04-23T14:48:27.869145+0000","last_undegraded":"2024-04-23T14:48:27.869145+0000","last_fullsized":"2024-04-23T14:48:27.869145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:28:39.310993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726746+0000","last_change":"2024-04-23T14:46:00.016554+0000","last_active":"2024-04-23T14:48:27.726746+0000","last_peered":"2024-04-23T14:48:27.726746+0000","last_clean":"2024-04-23T14:48:27.726746+0000","last_became_active":"2024-04-23T14:46:00.016242+0000","last_became_peered":"2024-04-23T14:46:00.016242+0000","last_unstale":"2024-04-23T14:48:27.726746+0000","last_undegraded":"2024-04-23T14:48:27.726746+0000","last_fullsized":"2024-04-23T14:48:27.726746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:03:34.596637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.856174+0000","last_change":"2024-04-23T14:46:58.373050+0000","last_active":"2024-04-23T14:48:27.856174+0000","last_peered":"2024-04-23T14:48:27.856174+0000","last_clean":"2024-04-23T14:48:27.856174+0000","last_became_active":"2024-04-23T14:46:58.372507+0000","last_became_peered":"2024-04-23T14:46:58.372507+0000","last_unstale":"2024-04-23T14:48:27.856174+0000","last_undegraded":"2024-04-23T14:48:27.856174+0000","last_fullsized":"2024-04-23T14:48:27.856174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:47:08.093978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050180+0000","last_change":"2024-04-23T14:46:00.005588+0000","last_active":"2024-04-23T14:48:28.050180+0000","last_peered":"2024-04-23T14:48:28.050180+0000","last_clean":"2024-04-23T14:48:28.050180+0000","last_became_active":"2024-04-23T14:46:00.005238+0000","last_became_peered":"2024-04-23T14:46:00.005238+0000","last_unstale":"2024-04-23T14:48:28.050180+0000","last_undegraded":"2024-04-23T14:48:28.050180+0000","last_fullsized":"2024-04-23T14:48:28.050180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:53:40.522929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869539+0000","last_change":"2024-04-23T14:46:00.013773+0000","last_active":"2024-04-23T14:48:27.869539+0000","last_peered":"2024-04-23T14:48:27.869539+0000","last_clean":"2024-04-23T14:48:27.869539+0000","last_became_active":"2024-04-23T14:46:00.013390+0000","last_became_peered":"2024-04-23T14:46:00.013390+0000","last_unstale":"2024-04-23T14:48:27.869539+0000","last_undegraded":"2024-04-23T14:48:27.869539+0000","last_fullsized":"2024-04-23T14:48:27.869539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:50:44.352922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727104+0000","last_change":"2024-04-23T14:46:00.013190+0000","last_active":"2024-04-23T14:48:27.727104+0000","last_peered":"2024-04-23T14:48:27.727104+0000","last_clean":"2024-04-23T14:48:27.727104+0000","last_became_active":"2024-04-23T14:46:00.012880+0000","last_became_peered":"2024-04-23T14:46:00.012880+0000","last_unstale":"2024-04-23T14:48:27.727104+0000","last_undegraded":"2024-04-23T14:48:27.727104+0000","last_fullsized":"2024-04-23T14:48:27.727104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:57.611514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049546+0000","last_change":"2024-04-23T14:46:00.013459+0000","last_active":"2024-04-23T14:48:28.049546+0000","last_peered":"2024-04-23T14:48:28.049546+0000","last_clean":"2024-04-23T14:48:28.049546+0000","last_became_active":"2024-04-23T14:46:00.013106+0000","last_became_peered":"2024-04-23T14:46:00.013106+0000","last_unstale":"2024-04-23T14:48:28.049546+0000","last_undegraded":"2024-04-23T14:48:28.049546+0000","last_fullsized":"2024-04-23T14:48:28.049546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:08:38.849062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050075+0000","last_change":"2024-04-23T14:46:00.030491+0000","last_active":"2024-04-23T14:48:28.050075+0000","last_peered":"2024-04-23T14:48:28.050075+0000","last_clean":"2024-04-23T14:48:28.050075+0000","last_became_active":"2024-04-23T14:46:00.030024+0000","last_became_peered":"2024-04-23T14:46:00.030024+0000","last_unstale":"2024-04-23T14:48:28.050075+0000","last_undegraded":"2024-04-23T14:48:28.050075+0000","last_fullsized":"2024-04-23T14:48:28.050075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:31.289091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735829+0000","last_change":"2024-04-23T14:46:00.022606+0000","last_active":"2024-04-23T14:48:27.735829+0000","last_peered":"2024-04-23T14:48:27.735829+0000","last_clean":"2024-04-23T14:48:27.735829+0000","last_became_active":"2024-04-23T14:46:00.022155+0000","last_became_peered":"2024-04-23T14:46:00.022155+0000","last_unstale":"2024-04-23T14:48:27.735829+0000","last_undegraded":"2024-04-23T14:48:27.735829+0000","last_fullsized":"2024-04-23T14:48:27.735829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:17:20.250283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735894+0000","last_change":"2024-04-23T14:46:00.027193+0000","last_active":"2024-04-23T14:48:27.735894+0000","last_peered":"2024-04-23T14:48:27.735894+0000","last_clean":"2024-04-23T14:48:27.735894+0000","last_became_active":"2024-04-23T14:46:00.026211+0000","last_became_peered":"2024-04-23T14:46:00.026211+0000","last_unstale":"2024-04-23T14:48:27.735894+0000","last_undegraded":"2024-04-23T14:48:27.735894+0000","last_fullsized":"2024-04-23T14:48:27.735894+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:52.722667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049048+0000","last_change":"2024-04-23T14:46:00.029917+0000","last_active":"2024-04-23T14:48:28.049048+0000","last_peered":"2024-04-23T14:48:28.049048+0000","last_clean":"2024-04-23T14:48:28.049048+0000","last_became_active":"2024-04-23T14:46:00.029563+0000","last_became_peered":"2024-04-23T14:46:00.029563+0000","last_unstale":"2024-04-23T14:48:28.049048+0000","last_undegraded":"2024-04-23T14:48:28.049048+0000","last_fullsized":"2024-04-23T14:48:28.049048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:01.787185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087901+0000","last_change":"2024-04-23T14:46:00.030910+0000","last_active":"2024-04-23T14:48:28.087901+0000","last_peered":"2024-04-23T14:48:28.087901+0000","last_clean":"2024-04-23T14:48:28.087901+0000","last_became_active":"2024-04-23T14:46:00.030555+0000","last_became_peered":"2024-04-23T14:46:00.030555+0000","last_unstale":"2024-04-23T14:48:28.087901+0000","last_undegraded":"2024-04-23T14:48:28.087901+0000","last_fullsized":"2024-04-23T14:48:28.087901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:56:43.701752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.857329+0000","last_change":"2024-04-23T14:46:00.017338+0000","last_active":"2024-04-23T14:48:27.857329+0000","last_peered":"2024-04-23T14:48:27.857329+0000","last_clean":"2024-04-23T14:48:27.857329+0000","last_became_active":"2024-04-23T14:46:00.015381+0000","last_became_peered":"2024-04-23T14:46:00.015381+0000","last_unstale":"2024-04-23T14:48:27.857329+0000","last_undegraded":"2024-04-23T14:48:27.857329+0000","last_fullsized":"2024-04-23T14:48:27.857329+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:51:16.762588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049084+0000","last_change":"2024-04-23T14:46:00.033055+0000","last_active":"2024-04-23T14:48:28.049084+0000","last_peered":"2024-04-23T14:48:28.049084+0000","last_clean":"2024-04-23T14:48:28.049084+0000","last_became_active":"2024-04-23T14:46:00.032763+0000","last_became_peered":"2024-04-23T14:46:00.032763+0000","last_unstale":"2024-04-23T14:48:28.049084+0000","last_undegraded":"2024-04-23T14:48:28.049084+0000","last_fullsized":"2024-04-23T14:48:28.049084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:11:43.793644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.857697+0000","last_change":"2024-04-23T14:45:59.232611+0000","last_active":"2024-04-23T14:48:27.857697+0000","last_peered":"2024-04-23T14:48:27.857697+0000","last_clean":"2024-04-23T14:48:27.857697+0000","last_became_active":"2024-04-23T14:45:59.231066+0000","last_became_peered":"2024-04-23T14:45:59.231066+0000","last_unstale":"2024-04-23T14:48:27.857697+0000","last_undegraded":"2024-04-23T14:48:27.857697+0000","last_fullsized":"2024-04-23T14:48:27.857697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:24.204134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.732918+0000","last_change":"2024-04-23T14:46:00.021532+0000","last_active":"2024-04-23T14:48:27.732918+0000","last_peered":"2024-04-23T14:48:27.732918+0000","last_clean":"2024-04-23T14:48:27.732918+0000","last_became_active":"2024-04-23T14:46:00.021125+0000","last_became_peered":"2024-04-23T14:46:00.021125+0000","last_unstale":"2024-04-23T14:48:27.732918+0000","last_undegraded":"2024-04-23T14:48:27.732918+0000","last_fullsized":"2024-04-23T14:48:27.732918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:08:03.072779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.733092+0000","last_change":"2024-04-23T14:45:59.248925+0000","last_active":"2024-04-23T14:48:27.733092+0000","last_peered":"2024-04-23T14:48:27.733092+0000","last_clean":"2024-04-23T14:48:27.733092+0000","last_became_active":"2024-04-23T14:45:59.248579+0000","last_became_peered":"2024-04-23T14:45:59.248579+0000","last_unstale":"2024-04-23T14:48:27.733092+0000","last_undegraded":"2024-04-23T14:48:27.733092+0000","last_fullsized":"2024-04-23T14:48:27.733092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:15:17.012610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735767+0000","last_change":"2024-04-23T14:46:00.028419+0000","last_active":"2024-04-23T14:48:27.735767+0000","last_peered":"2024-04-23T14:48:27.735767+0000","last_clean":"2024-04-23T14:48:27.735767+0000","last_became_active":"2024-04-23T14:46:00.026300+0000","last_became_peered":"2024-04-23T14:46:00.026300+0000","last_unstale":"2024-04-23T14:48:27.735767+0000","last_undegraded":"2024-04-23T14:48:27.735767+0000","last_fullsized":"2024-04-23T14:48:27.735767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:21.784680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860625+0000","last_change":"2024-04-23T14:45:59.225701+0000","last_active":"2024-04-23T14:48:27.860625+0000","last_peered":"2024-04-23T14:48:27.860625+0000","last_clean":"2024-04-23T14:48:27.860625+0000","last_became_active":"2024-04-23T14:45:59.225290+0000","last_became_peered":"2024-04-23T14:45:59.225290+0000","last_unstale":"2024-04-23T14:48:27.860625+0000","last_undegraded":"2024-04-23T14:48:27.860625+0000","last_fullsized":"2024-04-23T14:48:27.860625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:26.877330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.048946+0000","last_change":"2024-04-23T14:46:00.035075+0000","last_active":"2024-04-23T14:48:28.048946+0000","last_peered":"2024-04-23T14:48:28.048946+0000","last_clean":"2024-04-23T14:48:28.048946+0000","last_became_active":"2024-04-23T14:46:00.034782+0000","last_became_peered":"2024-04-23T14:46:00.034782+0000","last_unstale":"2024-04-23T14:48:28.048946+0000","last_undegraded":"2024-04-23T14:48:28.048946+0000","last_fullsized":"2024-04-23T14:48:28.048946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:02:20.747643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087488+0000","last_change":"2024-04-23T14:46:58.380160+0000","last_active":"2024-04-23T14:48:28.087488+0000","last_peered":"2024-04-23T14:48:28.087488+0000","last_clean":"2024-04-23T14:48:28.087488+0000","last_became_active":"2024-04-23T14:46:58.371279+0000","last_became_peered":"2024-04-23T14:46:58.371279+0000","last_unstale":"2024-04-23T14:48:28.087488+0000","last_undegraded":"2024-04-23T14:48:28.087488+0000","last_fullsized":"2024-04-23T14:48:28.087488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:19:44.330147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833670+0000","last_change":"2024-04-23T14:46:58.372972+0000","last_active":"2024-04-23T14:48:27.833670+0000","last_peered":"2024-04-23T14:48:27.833670+0000","last_clean":"2024-04-23T14:48:27.833670+0000","last_became_active":"2024-04-23T14:46:58.372374+0000","last_became_peered":"2024-04-23T14:46:58.372374+0000","last_unstale":"2024-04-23T14:48:27.833670+0000","last_undegraded":"2024-04-23T14:48:27.833670+0000","last_fullsized":"2024-04-23T14:48:27.833670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:15.638664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'8","reported_seq":190,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.508063+0000","last_change":"2024-04-23T14:46:58.376256+0000","last_active":"2024-04-23T14:48:28.508063+0000","last_peered":"2024-04-23T14:48:28.508063+0000","last_clean":"2024-04-23T14:48:28.508063+0000","last_became_active":"2024-04-23T14:46:58.371767+0000","last_became_peered":"2024-04-23T14:46:58.371767+0000","last_unstale":"2024-04-23T14:48:28.508063+0000","last_undegraded":"2024-04-23T14:48:28.508063+0000","last_fullsized":"2024-04-23T14:48:28.508063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:01:47.773660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860463+0000","last_change":"2024-04-23T14:46:00.016700+0000","last_active":"2024-04-23T14:48:27.860463+0000","last_peered":"2024-04-23T14:48:27.860463+0000","last_clean":"2024-04-23T14:48:27.860463+0000","last_became_active":"2024-04-23T14:46:00.015779+0000","last_became_peered":"2024-04-23T14:46:00.015779+0000","last_unstale":"2024-04-23T14:48:27.860463+0000","last_undegraded":"2024-04-23T14:48:27.860463+0000","last_fullsized":"2024-04-23T14:48:27.860463+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:19:59.431628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869082+0000","last_change":"2024-04-23T14:45:59.229791+0000","last_active":"2024-04-23T14:48:27.869082+0000","last_peered":"2024-04-23T14:48:27.869082+0000","last_clean":"2024-04-23T14:48:27.869082+0000","last_became_active":"2024-04-23T14:45:59.229054+0000","last_became_peered":"2024-04-23T14:45:59.229054+0000","last_unstale":"2024-04-23T14:48:27.869082+0000","last_undegraded":"2024-04-23T14:48:27.869082+0000","last_fullsized":"2024-04-23T14:48:27.869082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:19:59.385719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833649+0000","last_change":"2024-04-23T14:46:00.021858+0000","last_active":"2024-04-23T14:48:27.833649+0000","last_peered":"2024-04-23T14:48:27.833649+0000","last_clean":"2024-04-23T14:48:27.833649+0000","last_became_active":"2024-04-23T14:46:00.021450+0000","last_became_peered":"2024-04-23T14:46:00.021450+0000","last_unstale":"2024-04-23T14:48:27.833649+0000","last_undegraded":"2024-04-23T14:48:27.833649+0000","last_fullsized":"2024-04-23T14:48:27.833649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:00.270035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049632+0000","last_change":"2024-04-23T14:45:59.238181+0000","last_active":"2024-04-23T14:48:28.049632+0000","last_peered":"2024-04-23T14:48:28.049632+0000","last_clean":"2024-04-23T14:48:28.049632+0000","last_became_active":"2024-04-23T14:45:59.237784+0000","last_became_peered":"2024-04-23T14:45:59.237784+0000","last_unstale":"2024-04-23T14:48:28.049632+0000","last_undegraded":"2024-04-23T14:48:28.049632+0000","last_fullsized":"2024-04-23T14:48:28.049632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:34:35.016190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726896+0000","last_change":"2024-04-23T14:46:00.033814+0000","last_active":"2024-04-23T14:48:27.726896+0000","last_peered":"2024-04-23T14:48:27.726896+0000","last_clean":"2024-04-23T14:48:27.726896+0000","last_became_active":"2024-04-23T14:46:00.032128+0000","last_became_peered":"2024-04-23T14:46:00.032128+0000","last_unstale":"2024-04-23T14:48:27.726896+0000","last_undegraded":"2024-04-23T14:48:27.726896+0000","last_fullsized":"2024-04-23T14:48:27.726896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:47:32.067418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833724+0000","last_change":"2024-04-23T14:45:59.228054+0000","last_active":"2024-04-23T14:48:27.833724+0000","last_peered":"2024-04-23T14:48:27.833724+0000","last_clean":"2024-04-23T14:48:27.833724+0000","last_became_active":"2024-04-23T14:45:59.227759+0000","last_became_peered":"2024-04-23T14:45:59.227759+0000","last_unstale":"2024-04-23T14:48:27.833724+0000","last_undegraded":"2024-04-23T14:48:27.833724+0000","last_fullsized":"2024-04-23T14:48:27.833724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:52.185284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.857129+0000","last_change":"2024-04-23T14:46:00.028634+0000","last_active":"2024-04-23T14:48:27.857129+0000","last_peered":"2024-04-23T14:48:27.857129+0000","last_clean":"2024-04-23T14:48:27.857129+0000","last_became_active":"2024-04-23T14:46:00.027768+0000","last_became_peered":"2024-04-23T14:46:00.027768+0000","last_unstale":"2024-04-23T14:48:27.857129+0000","last_undegraded":"2024-04-23T14:48:27.857129+0000","last_fullsized":"2024-04-23T14:48:27.857129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:21:29.869845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'5","reported_seq":170,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.506886+0000","last_change":"2024-04-23T14:45:59.229480+0000","last_active":"2024-04-23T14:48:28.506886+0000","last_peered":"2024-04-23T14:48:28.506886+0000","last_clean":"2024-04-23T14:48:28.506886+0000","last_became_active":"2024-04-23T14:45:59.229124+0000","last_became_peered":"2024-04-23T14:45:59.229124+0000","last_unstale":"2024-04-23T14:48:28.506886+0000","last_undegraded":"2024-04-23T14:48:28.506886+0000","last_fullsized":"2024-04-23T14:48:28.506886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:06:33.837779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":143,"num_read_kb":143,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049919+0000","last_change":"2024-04-23T14:46:00.017753+0000","last_active":"2024-04-23T14:48:28.049919+0000","last_peered":"2024-04-23T14:48:28.049919+0000","last_clean":"2024-04-23T14:48:28.049919+0000","last_became_active":"2024-04-23T14:46:00.017370+0000","last_became_peered":"2024-04-23T14:46:00.017370+0000","last_unstale":"2024-04-23T14:48:28.049919+0000","last_undegraded":"2024-04-23T14:48:28.049919+0000","last_fullsized":"2024-04-23T14:48:28.049919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:30:35.460570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727402+0000","last_change":"2024-04-23T14:45:59.229361+0000","last_active":"2024-04-23T14:48:27.727402+0000","last_peered":"2024-04-23T14:48:27.727402+0000","last_clean":"2024-04-23T14:48:27.727402+0000","last_became_active":"2024-04-23T14:45:59.229043+0000","last_became_peered":"2024-04-23T14:45:59.229043+0000","last_unstale":"2024-04-23T14:48:27.727402+0000","last_undegraded":"2024-04-23T14:48:27.727402+0000","last_fullsized":"2024-04-23T14:48:27.727402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:23:09.472877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735632+0000","last_change":"2024-04-23T14:46:00.030181+0000","last_active":"2024-04-23T14:48:27.735632+0000","last_peered":"2024-04-23T14:48:27.735632+0000","last_clean":"2024-04-23T14:48:27.735632+0000","last_became_active":"2024-04-23T14:46:00.029303+0000","last_became_peered":"2024-04-23T14:46:00.029303+0000","last_unstale":"2024-04-23T14:48:27.735632+0000","last_undegraded":"2024-04-23T14:48:27.735632+0000","last_fullsized":"2024-04-23T14:48:27.735632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:24:53.904501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833914+0000","last_change":"2024-04-23T14:45:59.231270+0000","last_active":"2024-04-23T14:48:27.833914+0000","last_peered":"2024-04-23T14:48:27.833914+0000","last_clean":"2024-04-23T14:48:27.833914+0000","last_became_active":"2024-04-23T14:45:59.230914+0000","last_became_peered":"2024-04-23T14:45:59.230914+0000","last_unstale":"2024-04-23T14:48:27.833914+0000","last_undegraded":"2024-04-23T14:48:27.833914+0000","last_fullsized":"2024-04-23T14:48:27.833914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:02:43.422260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049242+0000","last_change":"2024-04-23T14:46:00.011358+0000","last_active":"2024-04-23T14:48:28.049242+0000","last_peered":"2024-04-23T14:48:28.049242+0000","last_clean":"2024-04-23T14:48:28.049242+0000","last_became_active":"2024-04-23T14:46:00.011040+0000","last_became_peered":"2024-04-23T14:46:00.011040+0000","last_unstale":"2024-04-23T14:48:28.049242+0000","last_undegraded":"2024-04-23T14:48:28.049242+0000","last_fullsized":"2024-04-23T14:48:28.049242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:27.248844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":351,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.508135+0000","last_change":"2024-04-23T14:45:59.227195+0000","last_active":"2024-04-23T14:48:28.508135+0000","last_peered":"2024-04-23T14:48:28.508135+0000","last_clean":"2024-04-23T14:48:28.508135+0000","last_became_active":"2024-04-23T14:45:59.226874+0000","last_became_peered":"2024-04-23T14:45:59.226874+0000","last_unstale":"2024-04-23T14:48:28.508135+0000","last_undegraded":"2024-04-23T14:48:28.508135+0000","last_fullsized":"2024-04-23T14:48:28.508135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:18:40.203839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65555,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":296,"num_read_kb":212,"num_write":34,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860491+0000","last_change":"2024-04-23T14:46:00.010215+0000","last_active":"2024-04-23T14:48:27.860491+0000","last_peered":"2024-04-23T14:48:27.860491+0000","last_clean":"2024-04-23T14:48:27.860491+0000","last_became_active":"2024-04-23T14:46:00.009756+0000","last_became_peered":"2024-04-23T14:46:00.009756+0000","last_unstale":"2024-04-23T14:48:27.860491+0000","last_undegraded":"2024-04-23T14:48:27.860491+0000","last_fullsized":"2024-04-23T14:48:27.860491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.797966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049180+0000","last_change":"2024-04-23T14:45:59.239258+0000","last_active":"2024-04-23T14:48:28.049180+0000","last_peered":"2024-04-23T14:48:28.049180+0000","last_clean":"2024-04-23T14:48:28.049180+0000","last_became_active":"2024-04-23T14:45:59.238899+0000","last_became_peered":"2024-04-23T14:45:59.238899+0000","last_unstale":"2024-04-23T14:48:28.049180+0000","last_undegraded":"2024-04-23T14:48:28.049180+0000","last_fullsized":"2024-04-23T14:48:28.049180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:45.866489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049683+0000","last_change":"2024-04-23T14:46:00.036627+0000","last_active":"2024-04-23T14:48:28.049683+0000","last_peered":"2024-04-23T14:48:28.049683+0000","last_clean":"2024-04-23T14:48:28.049683+0000","last_became_active":"2024-04-23T14:46:00.035397+0000","last_became_peered":"2024-04-23T14:46:00.035397+0000","last_unstale":"2024-04-23T14:48:28.049683+0000","last_undegraded":"2024-04-23T14:48:28.049683+0000","last_fullsized":"2024-04-23T14:48:28.049683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:24:02.038813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833573+0000","last_change":"2024-04-23T14:45:59.248241+0000","last_active":"2024-04-23T14:48:27.833573+0000","last_peered":"2024-04-23T14:48:27.833573+0000","last_clean":"2024-04-23T14:48:27.833573+0000","last_became_active":"2024-04-23T14:45:59.247895+0000","last_became_peered":"2024-04-23T14:45:59.247895+0000","last_unstale":"2024-04-23T14:48:27.833573+0000","last_undegraded":"2024-04-23T14:48:27.833573+0000","last_fullsized":"2024-04-23T14:48:27.833573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:01.964714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727084+0000","last_change":"2024-04-23T14:46:58.433781+0000","last_active":"2024-04-23T14:48:27.727084+0000","last_peered":"2024-04-23T14:48:27.727084+0000","last_clean":"2024-04-23T14:48:27.727084+0000","last_became_active":"2024-04-23T14:46:58.433460+0000","last_became_peered":"2024-04-23T14:46:58.433460+0000","last_unstale":"2024-04-23T14:48:27.727084+0000","last_undegraded":"2024-04-23T14:48:27.727084+0000","last_fullsized":"2024-04-23T14:48:27.727084+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:31:44.149949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":177,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.630557+0000","last_change":"2024-04-23T14:45:59.227406+0000","last_active":"2024-04-23T14:48:28.630557+0000","last_peered":"2024-04-23T14:48:28.630557+0000","last_clean":"2024-04-23T14:48:28.630557+0000","last_became_active":"2024-04-23T14:45:59.227063+0000","last_became_peered":"2024-04-23T14:45:59.227063+0000","last_unstale":"2024-04-23T14:48:28.630557+0000","last_undegraded":"2024-04-23T14:48:28.630557+0000","last_fullsized":"2024-04-23T14:48:28.630557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:09:36.845854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726734+0000","last_change":"2024-04-23T14:46:00.006333+0000","last_active":"2024-04-23T14:48:27.726734+0000","last_peered":"2024-04-23T14:48:27.726734+0000","last_clean":"2024-04-23T14:48:27.726734+0000","last_became_active":"2024-04-23T14:46:00.006024+0000","last_became_peered":"2024-04-23T14:46:00.006024+0000","last_unstale":"2024-04-23T14:48:27.726734+0000","last_undegraded":"2024-04-23T14:48:27.726734+0000","last_fullsized":"2024-04-23T14:48:27.726734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:49.389445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860182+0000","last_change":"2024-04-23T14:45:59.226394+0000","last_active":"2024-04-23T14:48:27.860182+0000","last_peered":"2024-04-23T14:48:27.860182+0000","last_clean":"2024-04-23T14:48:27.860182+0000","last_became_active":"2024-04-23T14:45:59.226088+0000","last_became_peered":"2024-04-23T14:45:59.226088+0000","last_unstale":"2024-04-23T14:48:27.860182+0000","last_undegraded":"2024-04-23T14:48:27.860182+0000","last_fullsized":"2024-04-23T14:48:27.860182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:53:35.139520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833741+0000","last_change":"2024-04-23T14:46:00.009262+0000","last_active":"2024-04-23T14:48:27.833741+0000","last_peered":"2024-04-23T14:48:27.833741+0000","last_clean":"2024-04-23T14:48:27.833741+0000","last_became_active":"2024-04-23T14:46:00.008865+0000","last_became_peered":"2024-04-23T14:46:00.008865+0000","last_unstale":"2024-04-23T14:48:27.833741+0000","last_undegraded":"2024-04-23T14:48:27.833741+0000","last_fullsized":"2024-04-23T14:48:27.833741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:24:27.662886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860641+0000","last_change":"2024-04-23T14:45:59.223479+0000","last_active":"2024-04-23T14:48:27.860641+0000","last_peered":"2024-04-23T14:48:27.860641+0000","last_clean":"2024-04-23T14:48:27.860641+0000","last_became_active":"2024-04-23T14:45:59.222997+0000","last_became_peered":"2024-04-23T14:45:59.222997+0000","last_unstale":"2024-04-23T14:48:27.860641+0000","last_undegraded":"2024-04-23T14:48:27.860641+0000","last_fullsized":"2024-04-23T14:48:27.860641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:29:58.026918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087709+0000","last_change":"2024-04-23T14:46:58.434121+0000","last_active":"2024-04-23T14:48:28.087709+0000","last_peered":"2024-04-23T14:48:28.087709+0000","last_clean":"2024-04-23T14:48:28.087709+0000","last_became_active":"2024-04-23T14:46:58.433746+0000","last_became_peered":"2024-04-23T14:46:58.433746+0000","last_unstale":"2024-04-23T14:48:28.087709+0000","last_undegraded":"2024-04-23T14:48:28.087709+0000","last_fullsized":"2024-04-23T14:48:28.087709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:33:22.493604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.856389+0000","last_change":"2024-04-23T14:45:59.237813+0000","last_active":"2024-04-23T14:48:27.856389+0000","last_peered":"2024-04-23T14:48:27.856389+0000","last_clean":"2024-04-23T14:48:27.856389+0000","last_became_active":"2024-04-23T14:45:59.237354+0000","last_became_peered":"2024-04-23T14:45:59.237354+0000","last_unstale":"2024-04-23T14:48:27.856389+0000","last_undegraded":"2024-04-23T14:48:27.856389+0000","last_fullsized":"2024-04-23T14:48:27.856389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:35:38.323059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869336+0000","last_change":"2024-04-23T14:46:00.031956+0000","last_active":"2024-04-23T14:48:27.869336+0000","last_peered":"2024-04-23T14:48:27.869336+0000","last_clean":"2024-04-23T14:48:27.869336+0000","last_became_active":"2024-04-23T14:46:00.031666+0000","last_became_peered":"2024-04-23T14:46:00.031666+0000","last_unstale":"2024-04-23T14:48:27.869336+0000","last_undegraded":"2024-04-23T14:48:27.869336+0000","last_fullsized":"2024-04-23T14:48:27.869336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:58:17.346455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726835+0000","last_change":"2024-04-23T14:45:59.230633+0000","last_active":"2024-04-23T14:48:27.726835+0000","last_peered":"2024-04-23T14:48:27.726835+0000","last_clean":"2024-04-23T14:48:27.726835+0000","last_became_active":"2024-04-23T14:45:59.230300+0000","last_became_peered":"2024-04-23T14:45:59.230300+0000","last_unstale":"2024-04-23T14:48:27.726835+0000","last_undegraded":"2024-04-23T14:48:27.726835+0000","last_fullsized":"2024-04-23T14:48:27.726835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:07:41.170055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.869090+0000","last_change":"2024-04-23T14:46:00.030891+0000","last_active":"2024-04-23T14:48:27.869090+0000","last_peered":"2024-04-23T14:48:27.869090+0000","last_clean":"2024-04-23T14:48:27.869090+0000","last_became_active":"2024-04-23T14:46:00.030438+0000","last_became_peered":"2024-04-23T14:46:00.030438+0000","last_unstale":"2024-04-23T14:48:27.869090+0000","last_undegraded":"2024-04-23T14:48:27.869090+0000","last_fullsized":"2024-04-23T14:48:27.869090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:07:43.038111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833930+0000","last_change":"2024-04-23T14:46:00.013683+0000","last_active":"2024-04-23T14:48:27.833930+0000","last_peered":"2024-04-23T14:48:27.833930+0000","last_clean":"2024-04-23T14:48:27.833930+0000","last_became_active":"2024-04-23T14:46:00.013356+0000","last_became_peered":"2024-04-23T14:46:00.013356+0000","last_unstale":"2024-04-23T14:48:27.833930+0000","last_undegraded":"2024-04-23T14:48:27.833930+0000","last_fullsized":"2024-04-23T14:48:27.833930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:32.866694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":172,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.053312+0000","last_change":"2024-04-23T14:45:59.252258+0000","last_active":"2024-04-23T14:48:28.053312+0000","last_peered":"2024-04-23T14:48:28.053312+0000","last_clean":"2024-04-23T14:48:28.053312+0000","last_became_active":"2024-04-23T14:45:59.251513+0000","last_became_peered":"2024-04-23T14:45:59.251513+0000","last_unstale":"2024-04-23T14:48:28.053312+0000","last_undegraded":"2024-04-23T14:48:28.053312+0000","last_fullsized":"2024-04-23T14:48:28.053312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:55:37.087807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.732380+0000","last_change":"2024-04-23T14:46:00.009468+0000","last_active":"2024-04-23T14:48:27.732380+0000","last_peered":"2024-04-23T14:48:27.732380+0000","last_clean":"2024-04-23T14:48:27.732380+0000","last_became_active":"2024-04-23T14:46:00.003118+0000","last_became_peered":"2024-04-23T14:46:00.003118+0000","last_unstale":"2024-04-23T14:48:27.732380+0000","last_undegraded":"2024-04-23T14:48:27.732380+0000","last_fullsized":"2024-04-23T14:48:27.732380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:20:24.488848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050208+0000","last_change":"2024-04-23T14:45:59.225758+0000","last_active":"2024-04-23T14:48:28.050208+0000","last_peered":"2024-04-23T14:48:28.050208+0000","last_clean":"2024-04-23T14:48:28.050208+0000","last_became_active":"2024-04-23T14:45:59.225427+0000","last_became_peered":"2024-04-23T14:45:59.225427+0000","last_unstale":"2024-04-23T14:48:28.050208+0000","last_undegraded":"2024-04-23T14:48:28.050208+0000","last_fullsized":"2024-04-23T14:48:28.050208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:59.495348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":354,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726734+0000","last_change":"2024-04-23T14:44:39.684486+0000","last_active":"2024-04-23T14:48:27.726734+0000","last_peered":"2024-04-23T14:48:27.726734+0000","last_clean":"2024-04-23T14:48:27.726734+0000","last_became_active":"2024-04-23T14:44:39.684074+0000","last_became_peered":"2024-04-23T14:44:39.684074+0000","last_unstale":"2024-04-23T14:48:27.726734+0000","last_undegraded":"2024-04-23T14:48:27.726734+0000","last_fullsized":"2024-04-23T14:48:27.726734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:40:41.333258+0000","last_clean_scrub_stamp":"2024-04-23T14:40:41.333258+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:11:16.865449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049334+0000","last_change":"2024-04-23T14:46:00.025018+0000","last_active":"2024-04-23T14:48:28.049334+0000","last_peered":"2024-04-23T14:48:28.049334+0000","last_clean":"2024-04-23T14:48:28.049334+0000","last_became_active":"2024-04-23T14:46:00.024695+0000","last_became_peered":"2024-04-23T14:46:00.024695+0000","last_unstale":"2024-04-23T14:48:28.049334+0000","last_undegraded":"2024-04-23T14:48:28.049334+0000","last_fullsized":"2024-04-23T14:48:28.049334+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:00:31.375963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726980+0000","last_change":"2024-04-23T14:45:59.225849+0000","last_active":"2024-04-23T14:48:27.726980+0000","last_peered":"2024-04-23T14:48:27.726980+0000","last_clean":"2024-04-23T14:48:27.726980+0000","last_became_active":"2024-04-23T14:45:59.225402+0000","last_became_peered":"2024-04-23T14:45:59.225402+0000","last_unstale":"2024-04-23T14:48:27.726980+0000","last_undegraded":"2024-04-23T14:48:27.726980+0000","last_fullsized":"2024-04-23T14:48:27.726980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:10:42.217717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087726+0000","last_change":"2024-04-23T14:46:58.377666+0000","last_active":"2024-04-23T14:48:28.087726+0000","last_peered":"2024-04-23T14:48:28.087726+0000","last_clean":"2024-04-23T14:48:28.087726+0000","last_became_active":"2024-04-23T14:46:58.370533+0000","last_became_peered":"2024-04-23T14:46:58.370533+0000","last_unstale":"2024-04-23T14:48:28.087726+0000","last_undegraded":"2024-04-23T14:48:28.087726+0000","last_fullsized":"2024-04-23T14:48:28.087726+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:39:58.606732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049771+0000","last_change":"2024-04-23T14:46:00.039566+0000","last_active":"2024-04-23T14:48:28.049771+0000","last_peered":"2024-04-23T14:48:28.049771+0000","last_clean":"2024-04-23T14:48:28.049771+0000","last_became_active":"2024-04-23T14:46:00.039236+0000","last_became_peered":"2024-04-23T14:46:00.039236+0000","last_unstale":"2024-04-23T14:48:28.049771+0000","last_undegraded":"2024-04-23T14:48:28.049771+0000","last_fullsized":"2024-04-23T14:48:28.049771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:09:44.028367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'46","reported_seq":91,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.000361+0000","last_change":"2024-04-23T14:45:59.246192+0000","last_active":"2024-04-23T14:48:28.000361+0000","last_peered":"2024-04-23T14:48:28.000361+0000","last_clean":"2024-04-23T14:48:28.000361+0000","last_became_active":"2024-04-23T14:45:59.245873+0000","last_became_peered":"2024-04-23T14:45:59.245873+0000","last_unstale":"2024-04-23T14:48:28.000361+0000","last_undegraded":"2024-04-23T14:48:28.000361+0000","last_fullsized":"2024-04-23T14:48:28.000361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:30:22.915746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":82,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.733234+0000","last_change":"2024-04-23T14:46:00.029176+0000","last_active":"2024-04-23T14:48:27.733234+0000","last_peered":"2024-04-23T14:48:27.733234+0000","last_clean":"2024-04-23T14:48:27.733234+0000","last_became_active":"2024-04-23T14:46:00.028741+0000","last_became_peered":"2024-04-23T14:46:00.028741+0000","last_unstale":"2024-04-23T14:48:27.733234+0000","last_undegraded":"2024-04-23T14:48:27.733234+0000","last_fullsized":"2024-04-23T14:48:27.733234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:35:22.905348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049510+0000","last_change":"2024-04-23T14:45:59.231490+0000","last_active":"2024-04-23T14:48:28.049510+0000","last_peered":"2024-04-23T14:48:28.049510+0000","last_clean":"2024-04-23T14:48:28.049510+0000","last_became_active":"2024-04-23T14:45:59.231143+0000","last_became_peered":"2024-04-23T14:45:59.231143+0000","last_unstale":"2024-04-23T14:48:28.049510+0000","last_undegraded":"2024-04-23T14:48:28.049510+0000","last_fullsized":"2024-04-23T14:48:28.049510+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:47:26.398150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.736394+0000","last_change":"2024-04-23T14:46:00.035838+0000","last_active":"2024-04-23T14:48:27.736394+0000","last_peered":"2024-04-23T14:48:27.736394+0000","last_clean":"2024-04-23T14:48:27.736394+0000","last_became_active":"2024-04-23T14:46:00.034498+0000","last_became_peered":"2024-04-23T14:46:00.034498+0000","last_unstale":"2024-04-23T14:48:27.736394+0000","last_undegraded":"2024-04-23T14:48:27.736394+0000","last_fullsized":"2024-04-23T14:48:27.736394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:28:15.748503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049161+0000","last_change":"2024-04-23T14:45:59.240817+0000","last_active":"2024-04-23T14:48:28.049161+0000","last_peered":"2024-04-23T14:48:28.049161+0000","last_clean":"2024-04-23T14:48:28.049161+0000","last_became_active":"2024-04-23T14:45:59.240557+0000","last_became_peered":"2024-04-23T14:45:59.240557+0000","last_unstale":"2024-04-23T14:48:28.049161+0000","last_undegraded":"2024-04-23T14:48:28.049161+0000","last_fullsized":"2024-04-23T14:48:28.049161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:29:03.099054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860155+0000","last_change":"2024-04-23T14:46:00.009094+0000","last_active":"2024-04-23T14:48:27.860155+0000","last_peered":"2024-04-23T14:48:27.860155+0000","last_clean":"2024-04-23T14:48:27.860155+0000","last_became_active":"2024-04-23T14:46:00.008751+0000","last_became_peered":"2024-04-23T14:46:00.008751+0000","last_unstale":"2024-04-23T14:48:27.860155+0000","last_undegraded":"2024-04-23T14:48:27.860155+0000","last_fullsized":"2024-04-23T14:48:27.860155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:12.480737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.728665+0000","last_change":"2024-04-23T14:45:59.249882+0000","last_active":"2024-04-23T14:48:27.728665+0000","last_peered":"2024-04-23T14:48:27.728665+0000","last_clean":"2024-04-23T14:48:27.728665+0000","last_became_active":"2024-04-23T14:45:59.249072+0000","last_became_peered":"2024-04-23T14:45:59.249072+0000","last_unstale":"2024-04-23T14:48:27.728665+0000","last_undegraded":"2024-04-23T14:48:27.728665+0000","last_fullsized":"2024-04-23T14:48:27.728665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:04:16.573005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087453+0000","last_change":"2024-04-23T14:46:00.030903+0000","last_active":"2024-04-23T14:48:28.087453+0000","last_peered":"2024-04-23T14:48:28.087453+0000","last_clean":"2024-04-23T14:48:28.087453+0000","last_became_active":"2024-04-23T14:46:00.030051+0000","last_became_peered":"2024-04-23T14:46:00.030051+0000","last_unstale":"2024-04-23T14:48:28.087453+0000","last_undegraded":"2024-04-23T14:48:28.087453+0000","last_fullsized":"2024-04-23T14:48:28.087453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:41:15.915795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049098+0000","last_change":"2024-04-23T14:45:59.236446+0000","last_active":"2024-04-23T14:48:28.049098+0000","last_peered":"2024-04-23T14:48:28.049098+0000","last_clean":"2024-04-23T14:48:28.049098+0000","last_became_active":"2024-04-23T14:45:59.236119+0000","last_became_peered":"2024-04-23T14:45:59.236119+0000","last_unstale":"2024-04-23T14:48:28.049098+0000","last_undegraded":"2024-04-23T14:48:28.049098+0000","last_fullsized":"2024-04-23T14:48:28.049098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:54:40.153626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.732671+0000","last_change":"2024-04-23T14:46:00.035577+0000","last_active":"2024-04-23T14:48:27.732671+0000","last_peered":"2024-04-23T14:48:27.732671+0000","last_clean":"2024-04-23T14:48:27.732671+0000","last_became_active":"2024-04-23T14:46:00.034929+0000","last_became_peered":"2024-04-23T14:46:00.034929+0000","last_unstale":"2024-04-23T14:48:27.732671+0000","last_undegraded":"2024-04-23T14:48:27.732671+0000","last_fullsized":"2024-04-23T14:48:27.732671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:05:22.844672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.860296+0000","last_change":"2024-04-23T14:46:58.440882+0000","last_active":"2024-04-23T14:48:27.860296+0000","last_peered":"2024-04-23T14:48:27.860296+0000","last_clean":"2024-04-23T14:48:27.860296+0000","last_became_active":"2024-04-23T14:46:58.436914+0000","last_became_peered":"2024-04-23T14:46:58.436914+0000","last_unstale":"2024-04-23T14:48:27.860296+0000","last_undegraded":"2024-04-23T14:48:27.860296+0000","last_fullsized":"2024-04-23T14:48:27.860296+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:36.064173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727242+0000","last_change":"2024-04-23T14:46:00.027934+0000","last_active":"2024-04-23T14:48:27.727242+0000","last_peered":"2024-04-23T14:48:27.727242+0000","last_clean":"2024-04-23T14:48:27.727242+0000","last_became_active":"2024-04-23T14:46:00.027571+0000","last_became_peered":"2024-04-23T14:46:00.027571+0000","last_unstale":"2024-04-23T14:48:27.727242+0000","last_undegraded":"2024-04-23T14:48:27.727242+0000","last_fullsized":"2024-04-23T14:48:27.727242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:54:27.890062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049081+0000","last_change":"2024-04-23T14:45:59.236967+0000","last_active":"2024-04-23T14:48:28.049081+0000","last_peered":"2024-04-23T14:48:28.049081+0000","last_clean":"2024-04-23T14:48:28.049081+0000","last_became_active":"2024-04-23T14:45:59.236612+0000","last_became_peered":"2024-04-23T14:45:59.236612+0000","last_unstale":"2024-04-23T14:48:28.049081+0000","last_undegraded":"2024-04-23T14:48:28.049081+0000","last_fullsized":"2024-04-23T14:48:28.049081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:58:09.422598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727288+0000","last_change":"2024-04-23T14:46:00.009289+0000","last_active":"2024-04-23T14:48:27.727288+0000","last_peered":"2024-04-23T14:48:27.727288+0000","last_clean":"2024-04-23T14:48:27.727288+0000","last_became_active":"2024-04-23T14:46:00.008911+0000","last_became_peered":"2024-04-23T14:46:00.008911+0000","last_unstale":"2024-04-23T14:48:27.727288+0000","last_undegraded":"2024-04-23T14:48:27.727288+0000","last_fullsized":"2024-04-23T14:48:27.727288+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:47:55.772442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.994184+0000","last_change":"2024-04-23T14:45:59.239494+0000","last_active":"2024-04-23T14:48:27.994184+0000","last_peered":"2024-04-23T14:48:27.994184+0000","last_clean":"2024-04-23T14:48:27.994184+0000","last_became_active":"2024-04-23T14:45:59.239157+0000","last_became_peered":"2024-04-23T14:45:59.239157+0000","last_unstale":"2024-04-23T14:48:27.994184+0000","last_undegraded":"2024-04-23T14:48:27.994184+0000","last_fullsized":"2024-04-23T14:48:27.994184+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:38.275320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.726982+0000","last_change":"2024-04-23T14:46:00.029401+0000","last_active":"2024-04-23T14:48:27.726982+0000","last_peered":"2024-04-23T14:48:27.726982+0000","last_clean":"2024-04-23T14:48:27.726982+0000","last_became_active":"2024-04-23T14:46:00.029006+0000","last_became_peered":"2024-04-23T14:46:00.029006+0000","last_unstale":"2024-04-23T14:48:27.726982+0000","last_undegraded":"2024-04-23T14:48:27.726982+0000","last_fullsized":"2024-04-23T14:48:27.726982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:56:11.617083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050600+0000","last_change":"2024-04-23T14:45:59.247636+0000","last_active":"2024-04-23T14:48:28.050600+0000","last_peered":"2024-04-23T14:48:28.050600+0000","last_clean":"2024-04-23T14:48:28.050600+0000","last_became_active":"2024-04-23T14:45:59.247231+0000","last_became_peered":"2024-04-23T14:45:59.247231+0000","last_unstale":"2024-04-23T14:48:28.050600+0000","last_undegraded":"2024-04-23T14:48:28.050600+0000","last_fullsized":"2024-04-23T14:48:28.050600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:58.197123+0000","last_clean_scrub_stamp":"2024-04-23T14:45:58.197123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:51:54.291463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.727318+0000","last_change":"2024-04-23T14:46:00.010825+0000","last_active":"2024-04-23T14:48:27.727318+0000","last_peered":"2024-04-23T14:48:27.727318+0000","last_clean":"2024-04-23T14:48:27.727318+0000","last_became_active":"2024-04-23T14:46:00.010470+0000","last_became_peered":"2024-04-23T14:46:00.010470+0000","last_unstale":"2024-04-23T14:48:27.727318+0000","last_undegraded":"2024-04-23T14:48:27.727318+0000","last_fullsized":"2024-04-23T14:48:27.727318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:55:19.851825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087509+0000","last_change":"2024-04-23T14:46:00.011967+0000","last_active":"2024-04-23T14:48:28.087509+0000","last_peered":"2024-04-23T14:48:28.087509+0000","last_clean":"2024-04-23T14:48:28.087509+0000","last_became_active":"2024-04-23T14:46:00.011580+0000","last_became_peered":"2024-04-23T14:46:00.011580+0000","last_unstale":"2024-04-23T14:48:28.087509+0000","last_undegraded":"2024-04-23T14:48:28.087509+0000","last_fullsized":"2024-04-23T14:48:28.087509+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:25:38.909994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.048960+0000","last_change":"2024-04-23T14:46:58.436615+0000","last_active":"2024-04-23T14:48:28.048960+0000","last_peered":"2024-04-23T14:48:28.048960+0000","last_clean":"2024-04-23T14:48:28.048960+0000","last_became_active":"2024-04-23T14:46:58.436288+0000","last_became_peered":"2024-04-23T14:46:58.436288+0000","last_unstale":"2024-04-23T14:48:28.048960+0000","last_undegraded":"2024-04-23T14:48:28.048960+0000","last_fullsized":"2024-04-23T14:48:28.048960+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:00:49.614004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.048911+0000","last_change":"2024-04-23T14:46:00.029874+0000","last_active":"2024-04-23T14:48:28.048911+0000","last_peered":"2024-04-23T14:48:28.048911+0000","last_clean":"2024-04-23T14:48:28.048911+0000","last_became_active":"2024-04-23T14:46:00.029520+0000","last_became_peered":"2024-04-23T14:46:00.029520+0000","last_unstale":"2024-04-23T14:48:28.048911+0000","last_undegraded":"2024-04-23T14:48:28.048911+0000","last_fullsized":"2024-04-23T14:48:28.048911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:45:30.711566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.735746+0000","last_change":"2024-04-23T14:46:00.039545+0000","last_active":"2024-04-23T14:48:27.735746+0000","last_peered":"2024-04-23T14:48:27.735746+0000","last_clean":"2024-04-23T14:48:27.735746+0000","last_became_active":"2024-04-23T14:46:00.039193+0000","last_became_peered":"2024-04-23T14:46:00.039193+0000","last_unstale":"2024-04-23T14:48:27.735746+0000","last_undegraded":"2024-04-23T14:48:27.735746+0000","last_fullsized":"2024-04-23T14:48:27.735746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:18:13.528129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050273+0000","last_change":"2024-04-23T14:46:00.036816+0000","last_active":"2024-04-23T14:48:28.050273+0000","last_peered":"2024-04-23T14:48:28.050273+0000","last_clean":"2024-04-23T14:48:28.050273+0000","last_became_active":"2024-04-23T14:46:00.036372+0000","last_became_peered":"2024-04-23T14:46:00.036372+0000","last_unstale":"2024-04-23T14:48:28.050273+0000","last_undegraded":"2024-04-23T14:48:28.050273+0000","last_fullsized":"2024-04-23T14:48:28.050273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:14:56.179724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087795+0000","last_change":"2024-04-23T14:46:00.012058+0000","last_active":"2024-04-23T14:48:28.087795+0000","last_peered":"2024-04-23T14:48:28.087795+0000","last_clean":"2024-04-23T14:48:28.087795+0000","last_became_active":"2024-04-23T14:46:00.011595+0000","last_became_peered":"2024-04-23T14:46:00.011595+0000","last_unstale":"2024-04-23T14:48:28.087795+0000","last_undegraded":"2024-04-23T14:48:28.087795+0000","last_fullsized":"2024-04-23T14:48:28.087795+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:28.341442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049273+0000","last_change":"2024-04-23T14:46:00.011419+0000","last_active":"2024-04-23T14:48:28.049273+0000","last_peered":"2024-04-23T14:48:28.049273+0000","last_clean":"2024-04-23T14:48:28.049273+0000","last_became_active":"2024-04-23T14:46:00.011004+0000","last_became_peered":"2024-04-23T14:46:00.011004+0000","last_unstale":"2024-04-23T14:48:28.049273+0000","last_undegraded":"2024-04-23T14:48:28.049273+0000","last_fullsized":"2024-04-23T14:48:28.049273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:46:57.451648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.733118+0000","last_change":"2024-04-23T14:46:00.023583+0000","last_active":"2024-04-23T14:48:27.733118+0000","last_peered":"2024-04-23T14:48:27.733118+0000","last_clean":"2024-04-23T14:48:27.733118+0000","last_became_active":"2024-04-23T14:46:00.023195+0000","last_became_peered":"2024-04-23T14:46:00.023195+0000","last_unstale":"2024-04-23T14:48:27.733118+0000","last_undegraded":"2024-04-23T14:48:27.733118+0000","last_fullsized":"2024-04-23T14:48:27.733118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:16:39.992959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.833812+0000","last_change":"2024-04-23T14:46:00.014719+0000","last_active":"2024-04-23T14:48:27.833812+0000","last_peered":"2024-04-23T14:48:27.833812+0000","last_clean":"2024-04-23T14:48:27.833812+0000","last_became_active":"2024-04-23T14:46:00.014349+0000","last_became_peered":"2024-04-23T14:46:00.014349+0000","last_unstale":"2024-04-23T14:48:27.833812+0000","last_undegraded":"2024-04-23T14:48:27.833812+0000","last_fullsized":"2024-04-23T14:48:27.833812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:12.789316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:27.728187+0000","last_change":"2024-04-23T14:46:00.019162+0000","last_active":"2024-04-23T14:48:27.728187+0000","last_peered":"2024-04-23T14:48:27.728187+0000","last_clean":"2024-04-23T14:48:27.728187+0000","last_became_active":"2024-04-23T14:46:00.018814+0000","last_became_peered":"2024-04-23T14:46:00.018814+0000","last_unstale":"2024-04-23T14:48:27.728187+0000","last_undegraded":"2024-04-23T14:48:27.728187+0000","last_fullsized":"2024-04-23T14:48:27.728187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:35:10.741148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.049189+0000","last_change":"2024-04-23T14:46:00.007254+0000","last_active":"2024-04-23T14:48:28.049189+0000","last_peered":"2024-04-23T14:48:28.049189+0000","last_clean":"2024-04-23T14:48:28.049189+0000","last_became_active":"2024-04-23T14:46:00.006826+0000","last_became_peered":"2024-04-23T14:46:00.006826+0000","last_unstale":"2024-04-23T14:48:28.049189+0000","last_undegraded":"2024-04-23T14:48:28.049189+0000","last_fullsized":"2024-04-23T14:48:28.049189+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:29.155244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.050003+0000","last_change":"2024-04-23T14:46:00.039265+0000","last_active":"2024-04-23T14:48:28.050003+0000","last_peered":"2024-04-23T14:48:28.050003+0000","last_clean":"2024-04-23T14:48:28.050003+0000","last_became_active":"2024-04-23T14:46:00.038848+0000","last_became_peered":"2024-04-23T14:46:00.038848+0000","last_unstale":"2024-04-23T14:48:28.050003+0000","last_undegraded":"2024-04-23T14:48:28.050003+0000","last_fullsized":"2024-04-23T14:48:28.050003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:01:10.218655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T14:48:28.087627+0000","last_change":"2024-04-23T14:46:00.012134+0000","last_active":"2024-04-23T14:48:28.087627+0000","last_peered":"2024-04-23T14:48:28.087627+0000","last_clean":"2024-04-23T14:48:28.087627+0000","last_became_active":"2024-04-23T14:46:00.011796+0000","last_became_peered":"2024-04-23T14:46:00.011796+0000","last_unstale":"2024-04-23T14:48:28.087627+0000","last_undegraded":"2024-04-23T14:48:28.087627+0000","last_fullsized":"2024-04-23T14:48:28.087627+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T14:45:59.206156+0000","last_clean_scrub_stamp":"2024-04-23T14:45:59.206156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:15:41.671419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107307,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":595827,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":975,"num_read_kb":601,"num_write":288,"num_write_kb":1436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1035,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":1787481,"data_compressed":81462,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":470628,"data_compressed":8702,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1138688,"data_stored":1260165,"data_compressed":52479,"data_compressed_allocated":69632,"data_compressed_original":745472,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1007616,"data_stored":864058,"data_compressed":27848,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037821,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":863956,"data_compressed":27848,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":1064147,"data_compressed":44471,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495687,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":470231,"data_compressed":8702,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":537124,"data_compressed":10554,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":471502,"data_compressed":8702,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":536307,"data_compressed":10554,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":995358,"data_compressed":34004,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920851,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":471591,"data_compressed":8702,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757325,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":1194508,"data_compressed":50627,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35769,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67100,"data_compressed":1852,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":66283,"data_compressed":1852,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133999,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":199656,"data_compressed":8008,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T14:48:34.207 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:48:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.416+0000 7f5b7a400640 1 -- 172.21.15.83:0/1631593420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b74105470 msgr2=0x7f5b741058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.416+0000 7f5b7a400640 1 --2- 172.21.15.83:0/1631593420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b74105470 0x7f5b741058f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009900 tx=0x7f5b6002a410 comp rx=0 tx=0).stop 2024-04-23T14:48:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.416+0000 7f5b7a400640 1 -- 172.21.15.83:0/1631593420 shutdown_connections 2024-04-23T14:48:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.416+0000 7f5b7a400640 1 --2- 172.21.15.83:0/1631593420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b74105e30 0x7f5b7410c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.416+0000 7f5b7a400640 1 --2- 172.21.15.83:0/1631593420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b74105470 0x7f5b741058f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.416+0000 7f5b7a400640 1 --2- 172.21.15.83:0/1631593420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b74104280 0x7f5b741046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.416+0000 7f5b7a400640 1 -- 172.21.15.83:0/1631593420 >> 172.21.15.83:0/1631593420 conn(0x7f5b740ff7f0 msgr2=0x7f5b74101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.416+0000 7f5b7a400640 1 -- 172.21.15.83:0/1631593420 shutdown_connections 2024-04-23T14:48:34.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.416+0000 7f5b7a400640 1 -- 172.21.15.83:0/1631593420 wait complete. 2024-04-23T14:48:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.417+0000 7f5b7a400640 1 Processor -- start 2024-04-23T14:48:34.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.417+0000 7f5b7a400640 1 -- start start 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b7a400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b74104280 0x7f5b7419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b7a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b74105470 0x7f5b7419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b7a400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b74105e30 0x7f5b741a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b7a400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7410ff60 con 0x7f5b74105470 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b7a400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7410fde0 con 0x7f5b74105e30 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b7a400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5b741100e0 con 0x7f5b74104280 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b73400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b74104280 0x7f5b7419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b73e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b74105e30 0x7f5b741a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b73400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b74104280 0x7f5b7419a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41816/0 (socket says 172.21.15.83:41816) 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b73400640 1 -- 172.21.15.83:0/1167329607 learned_addr learned my addr 172.21.15.83:0/1167329607 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b72a00640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b74105470 0x7f5b7419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b73e00640 1 -- 172.21.15.83:0/1167329607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b74104280 msgr2=0x7f5b7419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b73e00640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b74104280 0x7f5b7419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b73e00640 1 -- 172.21.15.83:0/1167329607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b74105470 msgr2=0x7f5b7419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b73e00640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b74105470 0x7f5b7419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b73e00640 1 -- 172.21.15.83:0/1167329607 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b741a21f0 con 0x7f5b74105e30 2024-04-23T14:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.418+0000 7f5b73400640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b74104280 0x7f5b7419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.419+0000 7f5b73e00640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b74105e30 0x7f5b741a1b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009fd0 tx=0x7f5b6c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.419+0000 7f5b6be00640 1 -- 172.21.15.83:0/1167329607 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c00ce20 con 0x7f5b74105e30 2024-04-23T14:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.419+0000 7f5b7a400640 1 -- 172.21.15.83:0/1167329607 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b741a24e0 con 0x7f5b74105e30 2024-04-23T14:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.419+0000 7f5b6be00640 1 -- 172.21.15.83:0/1167329607 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b6c005020 con 0x7f5b74105e30 2024-04-23T14:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.419+0000 7f5b6be00640 1 -- 172.21.15.83:0/1167329607 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c010720 con 0x7f5b74105e30 2024-04-23T14:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.419+0000 7f5b7a400640 1 -- 172.21.15.83:0/1167329607 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b741a2a00 con 0x7f5b74105e30 2024-04-23T14:48:34.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.421+0000 7f5b7a400640 1 -- 172.21.15.83:0/1167329607 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7410b540 con 0x7f5b74105e30 2024-04-23T14:48:34.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.422+0000 7f5b6be00640 1 -- 172.21.15.83:0/1167329607 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5b6c005540 con 0x7f5b74105e30 2024-04-23T14:48:34.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.423+0000 7f5b6be00640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5b580a3e40 0x7f5b580a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:34.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.424+0000 7f5b6be00640 1 -- 172.21.15.83:0/1167329607 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f5b6c0d1160 con 0x7f5b74105e30 2024-04-23T14:48:34.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.424+0000 7f5b73400640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5b580a3e40 0x7f5b580a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:34.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.424+0000 7f5b73400640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5b580a3e40 0x7f5b580a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5b640058c0 tx=0x7f5b64005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:34.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.427+0000 7f5b6be00640 1 -- 172.21.15.83:0/1167329607 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b6c09d5c0 con 0x7f5b74105e30 2024-04-23T14:48:34.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:48:34.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:34.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:34.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:48:34.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:34 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:48:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:48:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:48:34.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:34 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:48:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:48:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:48:34.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:48:34.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:34 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:48:34.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.692+0000 7f5b7a400640 1 -- 172.21.15.83:0/1167329607 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b74062c00 con 0x7f5b74105e30 2024-04-23T14:48:34.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.693+0000 7f5b6be00640 1 -- 172.21.15.83:0/1167329607 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23076 (secure 0 0 0) 0x7f5b6c09cec0 con 0x7f5b74105e30 2024-04-23T14:48:34.695 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:34.695 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:48:27.715285+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/622708991":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/1081276647":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/4179157882":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6801/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/2255156114":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:48:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.696+0000 7f5b7a400640 1 -- 172.21.15.83:0/1167329607 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5b580a3e40 msgr2=0x7f5b580a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.696+0000 7f5b7a400640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5b580a3e40 0x7f5b580a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5b640058c0 tx=0x7f5b64005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.696+0000 7f5b7a400640 1 -- 172.21.15.83:0/1167329607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b74105e30 msgr2=0x7f5b741a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.696+0000 7f5b7a400640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b74105e30 0x7f5b741a1b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009fd0 tx=0x7f5b6c00ef10 comp rx=0 tx=0).stop 2024-04-23T14:48:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.696+0000 7f5b7a400640 1 -- 172.21.15.83:0/1167329607 shutdown_connections 2024-04-23T14:48:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.696+0000 7f5b7a400640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b74105e30 0x7f5b741a1b80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.696+0000 7f5b7a400640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5b580a3e40 0x7f5b580a6300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.696+0000 7f5b7a400640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b74105470 0x7f5b7419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.696+0000 7f5b7a400640 1 --2- 172.21.15.83:0/1167329607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b74104280 0x7f5b7419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.696+0000 7f5b7a400640 1 -- 172.21.15.83:0/1167329607 >> 172.21.15.83:0/1167329607 conn(0x7f5b740ff7f0 msgr2=0x7f5b74108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.696+0000 7f5b7a400640 1 -- 172.21.15.83:0/1167329607 shutdown_connections 2024-04-23T14:48:34.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.696+0000 7f5b7a400640 1 -- 172.21.15.83:0/1167329607 wait complete. 2024-04-23T14:48:34.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:40:41.241992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:45:57.341602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:45:58.654798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:48:34.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:48:34.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.892+0000 7fc19ea00640 1 -- 172.21.15.83:0/4062268805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc198109e50 msgr2=0x7fc19810c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:34.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.892+0000 7fc19ea00640 1 --2- 172.21.15.83:0/4062268805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc198109e50 0x7fc19810c2e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc190009ca0 tx=0x7fc19002a8c0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.892+0000 7fc19ea00640 1 -- 172.21.15.83:0/4062268805 shutdown_connections 2024-04-23T14:48:34.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.892+0000 7fc19ea00640 1 --2- 172.21.15.83:0/4062268805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc198109e50 0x7fc19810c2e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.893+0000 7fc19ea00640 1 --2- 172.21.15.83:0/4062268805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc198074a60 0x7fc198074ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.893+0000 7fc19ea00640 1 --2- 172.21.15.83:0/4062268805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc198076040 0x7fc198074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.893+0000 7fc19ea00640 1 -- 172.21.15.83:0/4062268805 >> 172.21.15.83:0/4062268805 conn(0x7fc1980fd500 msgr2=0x7fc1980ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:34.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.893+0000 7fc19ea00640 1 -- 172.21.15.83:0/4062268805 shutdown_connections 2024-04-23T14:48:34.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.893+0000 7fc19ea00640 1 -- 172.21.15.83:0/4062268805 wait complete. 2024-04-23T14:48:34.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.894+0000 7fc19ea00640 1 Processor -- start 2024-04-23T14:48:34.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.894+0000 7fc19ea00640 1 -- start start 2024-04-23T14:48:34.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.894+0000 7fc19ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc198074a60 0x7fc19819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:34.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.894+0000 7fc19ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc198076040 0x7fc19819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:34.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.894+0000 7fc19ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc198109e50 0x7fc1981a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.894+0000 7fc19ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc19810fdc0 con 0x7fc198076040 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.894+0000 7fc19ea00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc19810fc40 con 0x7fc198109e50 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.894+0000 7fc19ea00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc19810ff40 con 0x7fc198074a60 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.894+0000 7fc19d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc198074a60 0x7fc19819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.894+0000 7fc19d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc198074a60 0x7fc19819a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49908/0 (socket says 172.21.15.83:49908) 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.894+0000 7fc19d600640 1 -- 172.21.15.83:0/4199662727 learned_addr learned my addr 172.21.15.83:0/4199662727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.894+0000 7fc19cc00640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc198076040 0x7fc19819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.895+0000 7fc19e000640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc198109e50 0x7fc1981a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.895+0000 7fc19d600640 1 -- 172.21.15.83:0/4199662727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc198109e50 msgr2=0x7fc1981a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.895+0000 7fc19d600640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc198109e50 0x7fc1981a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.895+0000 7fc19d600640 1 -- 172.21.15.83:0/4199662727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc198076040 msgr2=0x7fc19819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.895+0000 7fc19d600640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc198076040 0x7fc19819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.895+0000 7fc19d600640 1 -- 172.21.15.83:0/4199662727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1981a1fc0 con 0x7fc198074a60 2024-04-23T14:48:34.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.895+0000 7fc19cc00640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc198076040 0x7fc19819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:48:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.895+0000 7fc19d600640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc198074a60 0x7fc19819a3a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00e9f0 tx=0x7fc18c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.895+0000 7fc196000640 1 -- 172.21.15.83:0/4199662727 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00cd80 con 0x7fc198074a60 2024-04-23T14:48:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.895+0000 7fc19ea00640 1 -- 172.21.15.83:0/4199662727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1981a22b0 con 0x7fc198074a60 2024-04-23T14:48:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.895+0000 7fc196000640 1 -- 172.21.15.83:0/4199662727 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc18c010910 con 0x7fc198074a60 2024-04-23T14:48:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.895+0000 7fc196000640 1 -- 172.21.15.83:0/4199662727 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0188f0 con 0x7fc198074a60 2024-04-23T14:48:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.896+0000 7fc19ea00640 1 -- 172.21.15.83:0/4199662727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1981a2800 con 0x7fc198074a60 2024-04-23T14:48:34.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.897+0000 7fc19ea00640 1 -- 172.21.15.83:0/4199662727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc198108b80 con 0x7fc198074a60 2024-04-23T14:48:34.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.899+0000 7fc196000640 1 -- 172.21.15.83:0/4199662727 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc18c010ac0 con 0x7fc198074a60 2024-04-23T14:48:34.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.900+0000 7fc196000640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1800a3e40 0x7fc1800a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:34.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.900+0000 7fc196000640 1 -- 172.21.15.83:0/4199662727 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc18c0d0ef0 con 0x7fc198074a60 2024-04-23T14:48:34.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.900+0000 7fc19cc00640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1800a3e40 0x7fc1800a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:34.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.901+0000 7fc19cc00640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1800a3e40 0x7fc1800a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc19819b8b0 tx=0x7fc184005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:34.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:34.904+0000 7fc196000640 1 -- 172.21.15.83:0/4199662727 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc18c09d350 con 0x7fc198074a60 2024-04-23T14:48:35.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.106+0000 7fc19ea00640 1 -- 172.21.15.83:0/4199662727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc198062c00 con 0x7fc198074a60 2024-04-23T14:48:35.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.106+0000 7fc196000640 1 -- 172.21.15.83:0/4199662727 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fc18c09cc50 con 0x7fc198074a60 2024-04-23T14:48:35.108 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-23T14:48:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.108+0000 7fc19ea00640 1 -- 172.21.15.83:0/4199662727 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1800a3e40 msgr2=0x7fc1800a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.108+0000 7fc19ea00640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1800a3e40 0x7fc1800a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc19819b8b0 tx=0x7fc184005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.108+0000 7fc19ea00640 1 -- 172.21.15.83:0/4199662727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc198074a60 msgr2=0x7fc19819a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.108+0000 7fc19ea00640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc198074a60 0x7fc19819a3a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00e9f0 tx=0x7fc18c00eeb0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.108+0000 7fc19ea00640 1 -- 172.21.15.83:0/4199662727 shutdown_connections 2024-04-23T14:48:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.108+0000 7fc19ea00640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc198109e50 0x7fc1981a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.108+0000 7fc19ea00640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1800a3e40 0x7fc1800a6300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.108+0000 7fc19ea00640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc198076040 0x7fc19819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.108+0000 7fc19ea00640 1 --2- 172.21.15.83:0/4199662727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc198074a60 0x7fc19819a3a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.108+0000 7fc19ea00640 1 -- 172.21.15.83:0/4199662727 >> 172.21.15.83:0/4199662727 conn(0x7fc1980fd500 msgr2=0x7fc198105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.108+0000 7fc19ea00640 1 -- 172.21.15.83:0/4199662727 shutdown_connections 2024-04-23T14:48:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.109+0000 7fc19ea00640 1 -- 172.21.15.83:0/4199662727 wait complete. 2024-04-23T14:48:35.127 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:48:35.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.300+0000 7f4a0f200640 1 -- 172.21.15.83:0/2450192463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10109dc0 msgr2=0x7f4a1010c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:35.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.300+0000 7f4a0f200640 1 --2- 172.21.15.83:0/2450192463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10109dc0 0x7f4a1010c260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009ca0 tx=0x7f4a0802a950 comp rx=0 tx=0).stop 2024-04-23T14:48:35.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.300+0000 7f4a0f200640 1 -- 172.21.15.83:0/2450192463 shutdown_connections 2024-04-23T14:48:35.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.300+0000 7f4a0f200640 1 --2- 172.21.15.83:0/2450192463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10109dc0 0x7f4a1010c260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.300+0000 7f4a0f200640 1 --2- 172.21.15.83:0/2450192463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a10074a00 0x7f4a10074e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.300+0000 7f4a0f200640 1 --2- 172.21.15.83:0/2450192463 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a10075fe0 0x7f4a10074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.300+0000 7f4a0f200640 1 -- 172.21.15.83:0/2450192463 >> 172.21.15.83:0/2450192463 conn(0x7f4a100fd4a0 msgr2=0x7f4a100ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:35.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.300+0000 7f4a0f200640 1 -- 172.21.15.83:0/2450192463 shutdown_connections 2024-04-23T14:48:35.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.301+0000 7f4a0f200640 1 -- 172.21.15.83:0/2450192463 wait complete. 2024-04-23T14:48:35.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.302+0000 7f4a0f200640 1 Processor -- start 2024-04-23T14:48:35.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.302+0000 7f4a0f200640 1 -- start start 2024-04-23T14:48:35.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.302+0000 7f4a0f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a10074a00 0x7f4a1019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:35.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.302+0000 7f4a0f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10075fe0 0x7f4a1019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:35.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.302+0000 7f4a0f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a10109dc0 0x7f4a101a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.302+0000 7f4a0f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1010fe10 con 0x7f4a10109dc0 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.302+0000 7f4a0f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1010fc90 con 0x7f4a10074a00 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.302+0000 7f4a0f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1010ff90 con 0x7f4a10075fe0 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.302+0000 7f4a0de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a10074a00 0x7f4a1019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a0de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a10074a00 0x7f4a1019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:44338/0 (socket says 172.21.15.83:44338) 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a0de00640 1 -- 172.21.15.83:0/1908906401 learned_addr learned my addr 172.21.15.83:0/1908906401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a0d400640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10075fe0 0x7f4a1019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a0e800640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a10109dc0 0x7f4a101a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a0de00640 1 -- 172.21.15.83:0/1908906401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10075fe0 msgr2=0x7f4a1019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a0de00640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10075fe0 0x7f4a1019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a0de00640 1 -- 172.21.15.83:0/1908906401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a10109dc0 msgr2=0x7f4a101a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a0de00640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a10109dc0 0x7f4a101a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a0de00640 1 -- 172.21.15.83:0/1908906401 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a101a2020 con 0x7f4a10074a00 2024-04-23T14:48:35.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a0de00640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a10074a00 0x7f4a1019a400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000e9f0 tx=0x7f4a0000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:35.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a06a00640 1 -- 172.21.15.83:0/1908906401 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0000cd80 con 0x7f4a10074a00 2024-04-23T14:48:35.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a06a00640 1 -- 172.21.15.83:0/1908906401 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a00005500 con 0x7f4a10074a00 2024-04-23T14:48:35.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a0f200640 1 -- 172.21.15.83:0/1908906401 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a101a2310 con 0x7f4a10074a00 2024-04-23T14:48:35.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.303+0000 7f4a06a00640 1 -- 172.21.15.83:0/1908906401 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00010690 con 0x7f4a10074a00 2024-04-23T14:48:35.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.304+0000 7f4a0f200640 1 -- 172.21.15.83:0/1908906401 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a101a2810 con 0x7f4a10074a00 2024-04-23T14:48:35.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.305+0000 7f4a0f200640 1 -- 172.21.15.83:0/1908906401 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a10108b80 con 0x7f4a10074a00 2024-04-23T14:48:35.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.306+0000 7f4a06a00640 1 -- 172.21.15.83:0/1908906401 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4a000046c0 con 0x7f4a10074a00 2024-04-23T14:48:35.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.308+0000 7f4a06a00640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f49f40a3e40 0x7f49f40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:35.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.308+0000 7f4a06a00640 1 -- 172.21.15.83:0/1908906401 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4a000d0fc0 con 0x7f4a10074a00 2024-04-23T14:48:35.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.308+0000 7f4a0d400640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f49f40a3e40 0x7f49f40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:35.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.309+0000 7f4a0d400640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f49f40a3e40 0x7f49f40a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4a1019b910 tx=0x7f49fc005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:35.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.312+0000 7f4a06a00640 1 -- 172.21.15.83:0/1908906401 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a0009d3c0 con 0x7f4a10074a00 2024-04-23T14:48:35.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-23T14:48:35.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-23T14:48:35.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-23T14:48:35.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T14:48:35.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: Updating smithi203:/etc/ceph/ceph.conf 2024-04-23T14:48:35.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:48:35.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:48:35.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:48:35.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:48:35.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: from='client.25364 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:48:35.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:48:35.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:48:35.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: Updating smithi083:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:48:35.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:48:35.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:48:35.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1167329607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:35.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:35.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:35.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:35.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:35.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:35 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/4199662727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-23T14:48:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-23T14:48:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-23T14:48:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T14:48:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: Updating smithi203:/etc/ceph/ceph.conf 2024-04-23T14:48:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:48:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:48:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:48:35.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:48:35.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: from='client.25364 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:48:35.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:48:35.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:48:35.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: Updating smithi083:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:48:35.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:48:35.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:48:35.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1167329607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:35.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:35.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:35.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:35.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:35.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:35 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/4199662727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:35.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.513+0000 7f4a0f200640 1 -- 172.21.15.83:0/1908906401 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4a10062c00 con 0x7f4a10074a00 2024-04-23T14:48:35.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.513+0000 7f4a06a00640 1 -- 172.21.15.83:0/1908906401 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f4a0009ccc0 con 0x7f4a10074a00 2024-04-23T14:48:35.515 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-23T14:48:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.516+0000 7f4a0f200640 1 -- 172.21.15.83:0/1908906401 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f49f40a3e40 msgr2=0x7f49f40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.516+0000 7f4a0f200640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f49f40a3e40 0x7f49f40a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4a1019b910 tx=0x7f49fc005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:35.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.516+0000 7f4a0f200640 1 -- 172.21.15.83:0/1908906401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a10074a00 msgr2=0x7f4a1019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.516+0000 7f4a0f200640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a10074a00 0x7f4a1019a400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000e9f0 tx=0x7f4a0000edf0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.516+0000 7f4a0f200640 1 -- 172.21.15.83:0/1908906401 shutdown_connections 2024-04-23T14:48:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.516+0000 7f4a0f200640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f49f40a3e40 0x7f49f40a6300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.516+0000 7f4a0f200640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a10109dc0 0x7f4a101a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.516+0000 7f4a0f200640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a10075fe0 0x7f4a1019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.516+0000 7f4a0f200640 1 --2- 172.21.15.83:0/1908906401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a10074a00 0x7f4a1019a400 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.516+0000 7f4a0f200640 1 -- 172.21.15.83:0/1908906401 >> 172.21.15.83:0/1908906401 conn(0x7f4a100fd4a0 msgr2=0x7f4a10105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.516+0000 7f4a0f200640 1 -- 172.21.15.83:0/1908906401 shutdown_connections 2024-04-23T14:48:35.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.516+0000 7f4a0f200640 1 -- 172.21.15.83:0/1908906401 wait complete. 2024-04-23T14:48:35.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:48:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-23T14:48:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-23T14:48:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: Updating smithi083:/etc/ceph/ceph.conf 2024-04-23T14:48:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: Updating smithi173:/etc/ceph/ceph.conf 2024-04-23T14:48:35.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: Updating smithi203:/etc/ceph/ceph.conf 2024-04-23T14:48:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:48:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T14:48:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:48:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:48:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: from='client.25364 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T14:48:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:48:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: Updating smithi203:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:48:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: Updating smithi083:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T14:48:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: Updating smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:48:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: Updating smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:48:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1167329607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:35.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:35.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:35.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:35.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:35.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:35 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/4199662727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:48:35.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.710+0000 7f445be00640 1 -- 172.21.15.83:0/3975149525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f445c103060 msgr2=0x7f445c103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:35.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.710+0000 7f445be00640 1 --2- 172.21.15.83:0/3975149525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f445c103060 0x7f445c103440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f444c009900 tx=0x7f444c02a480 comp rx=0 tx=0).stop 2024-04-23T14:48:35.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.710+0000 7f445be00640 1 -- 172.21.15.83:0/3975149525 shutdown_connections 2024-04-23T14:48:35.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.710+0000 7f445be00640 1 --2- 172.21.15.83:0/3975149525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c10c480 0x7f445c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.710+0000 7f445be00640 1 --2- 172.21.15.83:0/3975149525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445c103980 0x7f445c10be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.710+0000 7f445be00640 1 --2- 172.21.15.83:0/3975149525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f445c103060 0x7f445c103440 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.710+0000 7f445be00640 1 -- 172.21.15.83:0/3975149525 >> 172.21.15.83:0/3975149525 conn(0x7f445c0fc880 msgr2=0x7f445c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:35.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.710+0000 7f445be00640 1 -- 172.21.15.83:0/3975149525 shutdown_connections 2024-04-23T14:48:35.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.710+0000 7f445be00640 1 -- 172.21.15.83:0/3975149525 wait complete. 2024-04-23T14:48:35.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.711+0000 7f445be00640 1 Processor -- start 2024-04-23T14:48:35.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.711+0000 7f445be00640 1 -- start start 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.711+0000 7f445be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c103060 0x7f445c100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.711+0000 7f445be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445c103980 0x7f445c1011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f445c10c480 0x7f445c101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f445c112240 con 0x7f445c10c480 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f445c1120c0 con 0x7f445c103060 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445be00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f445c1123c0 con 0x7f445c103980 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f445c10c480 0x7f445c101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c103060 0x7f445c100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c103060 0x7f445c100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:44352/0 (socket says 172.21.15.83:44352) 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f445c10c480 0x7f445c101730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46044/0 (socket says 172.21.15.83:46044) 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445aa00640 1 -- 172.21.15.83:0/3363422639 learned_addr learned my addr 172.21.15.83:0/3363422639 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445a000640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445c103980 0x7f445c1011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:35.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445b400640 1 -- 172.21.15.83:0/3363422639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445c103980 msgr2=0x7f445c1011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445b400640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445c103980 0x7f445c1011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445b400640 1 -- 172.21.15.83:0/3363422639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c103060 msgr2=0x7f445c100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445b400640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c103060 0x7f445c100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.712+0000 7f445b400640 1 -- 172.21.15.83:0/3363422639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445c1a6c40 con 0x7f445c10c480 2024-04-23T14:48:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.713+0000 7f445b400640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f445c10c480 0x7f445c101730 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f445400e940 tx=0x7f445400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.713+0000 7f4460a00640 1 -- 172.21.15.83:0/3363422639 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454004d20 con 0x7f445c10c480 2024-04-23T14:48:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.713+0000 7f4460a00640 1 -- 172.21.15.83:0/3363422639 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4454005020 con 0x7f445c10c480 2024-04-23T14:48:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.713+0000 7f445be00640 1 -- 172.21.15.83:0/3363422639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445c1a6f30 con 0x7f445c10c480 2024-04-23T14:48:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.713+0000 7f4460a00640 1 -- 172.21.15.83:0/3363422639 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44540116b0 con 0x7f445c10c480 2024-04-23T14:48:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.713+0000 7f445be00640 1 -- 172.21.15.83:0/3363422639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445c1a7450 con 0x7f445c10c480 2024-04-23T14:48:35.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.714+0000 7f445be00640 1 -- 172.21.15.83:0/3363422639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f445c068d40 con 0x7f445c10c480 2024-04-23T14:48:35.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.717+0000 7f4460a00640 1 -- 172.21.15.83:0/3363422639 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f44540054d0 con 0x7f445c10c480 2024-04-23T14:48:35.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.718+0000 7f4460a00640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f44400a3ec0 0x7f44400a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:35.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.718+0000 7f4460a00640 1 -- 172.21.15.83:0/3363422639 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f44540d0ab0 con 0x7f445c10c480 2024-04-23T14:48:35.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.718+0000 7f445aa00640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f44400a3ec0 0x7f44400a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:35.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.719+0000 7f445aa00640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f44400a3ec0 0x7f44400a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f444c0096c0 tx=0x7f444c005a20 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:35.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.723+0000 7f4460a00640 1 -- 172.21.15.83:0/3363422639 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4454016030 con 0x7f445c10c480 2024-04-23T14:48:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.922+0000 7f445be00640 1 -- 172.21.15.83:0/3363422639 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f445c107520 con 0x7f445c10c480 2024-04-23T14:48:35.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.923+0000 7f4460a00640 1 -- 172.21.15.83:0/3363422639 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f445409c860 con 0x7f445c10c480 2024-04-23T14:48:35.925 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-23T14:48:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.925+0000 7f445be00640 1 -- 172.21.15.83:0/3363422639 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f44400a3ec0 msgr2=0x7f44400a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.925+0000 7f445be00640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f44400a3ec0 0x7f44400a6380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f444c0096c0 tx=0x7f444c005a20 comp rx=0 tx=0).stop 2024-04-23T14:48:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.925+0000 7f445be00640 1 -- 172.21.15.83:0/3363422639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f445c10c480 msgr2=0x7f445c101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.925+0000 7f445be00640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f445c10c480 0x7f445c101730 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f445400e940 tx=0x7f445400ee00 comp rx=0 tx=0).stop 2024-04-23T14:48:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.925+0000 7f445be00640 1 -- 172.21.15.83:0/3363422639 shutdown_connections 2024-04-23T14:48:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.925+0000 7f445be00640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f44400a3ec0 0x7f44400a6380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.925+0000 7f445be00640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f445c10c480 0x7f445c101730 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.925+0000 7f445be00640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445c103980 0x7f445c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.925+0000 7f445be00640 1 --2- 172.21.15.83:0/3363422639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f445c103060 0x7f445c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.925+0000 7f445be00640 1 -- 172.21.15.83:0/3363422639 >> 172.21.15.83:0/3363422639 conn(0x7f445c0fc880 msgr2=0x7f445c0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.925+0000 7f445be00640 1 -- 172.21.15.83:0/3363422639 shutdown_connections 2024-04-23T14:48:35.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:35.925+0000 7f445be00640 1 -- 172.21.15.83:0/3363422639 wait complete. 2024-04-23T14:48:35.945 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:48:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.137+0000 7f9e0d000640 1 -- 172.21.15.83:0/1407613389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e080743a0 msgr2=0x7f9e08074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.137+0000 7f9e0d000640 1 --2- 172.21.15.83:0/1407613389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e080743a0 0x7f9e08074780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009860 tx=0x7f9df802a3d0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.137+0000 7f9e0d000640 1 -- 172.21.15.83:0/1407613389 shutdown_connections 2024-04-23T14:48:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.137+0000 7f9e0d000640 1 --2- 172.21.15.83:0/1407613389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e08112fe0 0x7f9e081153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.137+0000 7f9e0d000640 1 --2- 172.21.15.83:0/1407613389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e08074cc0 0x7f9e08112aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.137+0000 7f9e0d000640 1 --2- 172.21.15.83:0/1407613389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e080743a0 0x7f9e08074780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.137+0000 7f9e0d000640 1 -- 172.21.15.83:0/1407613389 >> 172.21.15.83:0/1407613389 conn(0x7f9e080793c0 msgr2=0x7f9e080797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.137+0000 7f9e0d000640 1 -- 172.21.15.83:0/1407613389 shutdown_connections 2024-04-23T14:48:36.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.137+0000 7f9e0d000640 1 -- 172.21.15.83:0/1407613389 wait complete. 2024-04-23T14:48:36.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.138+0000 7f9e0d000640 1 Processor -- start 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.138+0000 7f9e0d000640 1 -- start start 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9e0d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e080743a0 0x7f9e0819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9e0d000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e08074cc0 0x7f9e0819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9e0d000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08112fe0 0x7f9e081a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9e0d000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9e081189e0 con 0x7f9e080743a0 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9e0d000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08118860 con 0x7f9e08074cc0 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9e0d000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08118b60 con 0x7f9e08112fe0 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9e05e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e080743a0 0x7f9e0819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9dffe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e08074cc0 0x7f9e0819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9e05e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e080743a0 0x7f9e0819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46054/0 (socket says 172.21.15.83:46054) 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9e06800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08112fe0 0x7f9e081a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9e05e00640 1 -- 172.21.15.83:0/3887325504 learned_addr learned my addr 172.21.15.83:0/3887325504 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:36.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9dffe00640 1 -- 172.21.15.83:0/3887325504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08112fe0 msgr2=0x7f9e081a3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9dffe00640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08112fe0 0x7f9e081a3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9dffe00640 1 -- 172.21.15.83:0/3887325504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e080743a0 msgr2=0x7f9e0819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9dffe00640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e080743a0 0x7f9e0819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9dffe00640 1 -- 172.21.15.83:0/3887325504 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e081a3f70 con 0x7f9e08074cc0 2024-04-23T14:48:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9e05e00640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e080743a0 0x7f9e0819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.139+0000 7f9e06800640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08112fe0 0x7f9e081a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.140+0000 7f9dffe00640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e08074cc0 0x7f9e0819f4c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9df0005fd0 tx=0x7f9df000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.140+0000 7f9dfea00640 1 -- 172.21.15.83:0/3887325504 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df000cd00 con 0x7f9e08074cc0 2024-04-23T14:48:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.140+0000 7f9e0d000640 1 -- 172.21.15.83:0/3887325504 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e081a4260 con 0x7f9e08074cc0 2024-04-23T14:48:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.140+0000 7f9dfea00640 1 -- 172.21.15.83:0/3887325504 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9df00035f0 con 0x7f9e08074cc0 2024-04-23T14:48:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.140+0000 7f9dfea00640 1 -- 172.21.15.83:0/3887325504 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df0010690 con 0x7f9e08074cc0 2024-04-23T14:48:36.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.140+0000 7f9e0d000640 1 -- 172.21.15.83:0/3887325504 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e081abaf0 con 0x7f9e08074cc0 2024-04-23T14:48:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.141+0000 7f9e0d000640 1 -- 172.21.15.83:0/3887325504 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e08069270 con 0x7f9e08074cc0 2024-04-23T14:48:36.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.143+0000 7f9dfea00640 1 -- 172.21.15.83:0/3887325504 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f9df00037a0 con 0x7f9e08074cc0 2024-04-23T14:48:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.145+0000 7f9dfea00640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9dec0a3e40 0x7f9dec0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.145+0000 7f9dfea00640 1 -- 172.21.15.83:0/3887325504 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9df00d0e00 con 0x7f9e08074cc0 2024-04-23T14:48:36.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.145+0000 7f9e05e00640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9dec0a3e40 0x7f9dec0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:36.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.146+0000 7f9e05e00640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9dec0a3e40 0x7f9dec0a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9df80095f0 tx=0x7f9df80058b0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:36.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.147+0000 7f9dfea00640 1 -- 172.21.15.83:0/3887325504 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9df009d290 con 0x7f9e08074cc0 2024-04-23T14:48:36.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.378+0000 7f9e0d000640 1 -- 172.21.15.83:0/3887325504 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e081abdb0 con 0x7f9e08074cc0 2024-04-23T14:48:36.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.380+0000 7f9dfea00640 1 -- 172.21.15.83:0/3887325504 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11863 (secure 0 0 0) 0x7f9df009cb90 con 0x7f9e08074cc0 2024-04-23T14:48:36.382 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:36.382 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:48:27.525285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24532,24542]},"id":1}]} 2024-04-23T14:48:36.382 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-04-23T14:48:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.382+0000 7f9e0d000640 1 -- 172.21.15.83:0/3887325504 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9dec0a3e40 msgr2=0x7f9dec0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.382+0000 7f9e0d000640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9dec0a3e40 0x7f9dec0a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9df80095f0 tx=0x7f9df80058b0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.382+0000 7f9e0d000640 1 -- 172.21.15.83:0/3887325504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e08074cc0 msgr2=0x7f9e0819f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.382+0000 7f9e0d000640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e08074cc0 0x7f9e0819f4c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9df0005fd0 tx=0x7f9df000ee50 comp rx=0 tx=0).stop 2024-04-23T14:48:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.382+0000 7f9e0d000640 1 -- 172.21.15.83:0/3887325504 shutdown_connections 2024-04-23T14:48:36.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.382+0000 7f9e0d000640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e08112fe0 0x7f9e081a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.382+0000 7f9e0d000640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e08074cc0 0x7f9e0819f4c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.382+0000 7f9e0d000640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9dec0a3e40 0x7f9dec0a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.382+0000 7f9e0d000640 1 --2- 172.21.15.83:0/3887325504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e080743a0 0x7f9e0819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.382+0000 7f9e0d000640 1 -- 172.21.15.83:0/3887325504 >> 172.21.15.83:0/3887325504 conn(0x7f9e080793c0 msgr2=0x7f9e080ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:36.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.382+0000 7f9e0d000640 1 -- 172.21.15.83:0/3887325504 shutdown_connections 2024-04-23T14:48:36.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.382+0000 7f9e0d000640 1 -- 172.21.15.83:0/3887325504 wait complete. 2024-04-23T14:48:36.401 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:48:36.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:36 smithi083 ceph-mon[27867]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:48:36.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:36 smithi083 ceph-mon[27867]: Updating smithi083:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:48:36.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:36 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1908906401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:36.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:36 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3363422639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:36.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:36 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:36.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:36 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:36.469 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:36 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:36.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:36 smithi203 ceph-mon[32800]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:48:36.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:36 smithi203 ceph-mon[32800]: Updating smithi083:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:48:36.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:36 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1908906401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:36.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:36 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3363422639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:36.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:36 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:36.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:36 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:36.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:36 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:36 smithi173 ceph-mon[32443]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-04-23T14:48:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:36 smithi173 ceph-mon[32443]: Updating smithi083:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T14:48:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:36 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1908906401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:48:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:36 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3363422639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:48:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:36 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:36 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:36.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:36 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:36.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.595+0000 7fef5c200640 1 -- 172.21.15.83:0/3147651313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54108fd0 msgr2=0x7fef541154e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:36.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.595+0000 7fef5c200640 1 --2- 172.21.15.83:0/3147651313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54108fd0 0x7fef541154e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009cd0 tx=0x7fef4c02a8f0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.595+0000 7fef5c200640 1 -- 172.21.15.83:0/3147651313 shutdown_connections 2024-04-23T14:48:36.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.595+0000 7fef5c200640 1 --2- 172.21.15.83:0/3147651313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54108fd0 0x7fef541154e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.595+0000 7fef5c200640 1 --2- 172.21.15.83:0/3147651313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54108610 0x7fef54108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.595+0000 7fef5c200640 1 --2- 172.21.15.83:0/3147651313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5410e6b0 0x7fef5410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.595+0000 7fef5c200640 1 -- 172.21.15.83:0/3147651313 >> 172.21.15.83:0/3147651313 conn(0x7fef54075800 msgr2=0x7fef54106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:36.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.595+0000 7fef5c200640 1 -- 172.21.15.83:0/3147651313 shutdown_connections 2024-04-23T14:48:36.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.595+0000 7fef5c200640 1 -- 172.21.15.83:0/3147651313 wait complete. 2024-04-23T14:48:36.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.596+0000 7fef5c200640 1 Processor -- start 2024-04-23T14:48:36.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.596+0000 7fef5c200640 1 -- start start 2024-04-23T14:48:36.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.596+0000 7fef5c200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54108610 0x7fef5419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:36.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef5c200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54108fd0 0x7fef5419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:36.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef5c200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5410e6b0 0x7fef541a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:36.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef5c200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef54118a30 con 0x7fef5410e6b0 2024-04-23T14:48:36.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef5c200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef541188b0 con 0x7fef54108fd0 2024-04-23T14:48:36.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef5c200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef54118bb0 con 0x7fef54108610 2024-04-23T14:48:36.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef59800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54108610 0x7fef5419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:36.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef59800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54108610 0x7fef5419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49962/0 (socket says 172.21.15.83:49962) 2024-04-23T14:48:36.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef59800640 1 -- 172.21.15.83:0/4259216960 learned_addr learned my addr 172.21.15.83:0/4259216960 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef5a200640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5410e6b0 0x7fef541a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef58e00640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54108fd0 0x7fef5419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef59800640 1 -- 172.21.15.83:0/4259216960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54108fd0 msgr2=0x7fef5419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef59800640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54108fd0 0x7fef5419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef59800640 1 -- 172.21.15.83:0/4259216960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5410e6b0 msgr2=0x7fef541a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef59800640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5410e6b0 0x7fef541a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef59800640 1 -- 172.21.15.83:0/4259216960 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef541a3ea0 con 0x7fef54108610 2024-04-23T14:48:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef5a200640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5410e6b0 0x7fef541a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:48:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.597+0000 7fef59800640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54108610 0x7fef5419eeb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef4800d950 tx=0x7fef4800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.598+0000 7fef52000640 1 -- 172.21.15.83:0/4259216960 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48017020 con 0x7fef54108610 2024-04-23T14:48:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.598+0000 7fef5c200640 1 -- 172.21.15.83:0/4259216960 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef541a4190 con 0x7fef54108610 2024-04-23T14:48:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.598+0000 7fef52000640 1 -- 172.21.15.83:0/4259216960 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef48009c30 con 0x7fef54108610 2024-04-23T14:48:36.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.598+0000 7fef52000640 1 -- 172.21.15.83:0/4259216960 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4800f6b0 con 0x7fef54108610 2024-04-23T14:48:36.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.598+0000 7fef5c200640 1 -- 172.21.15.83:0/4259216960 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef541aba20 con 0x7fef54108610 2024-04-23T14:48:36.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.599+0000 7fef5c200640 1 -- 172.21.15.83:0/4259216960 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef54069270 con 0x7fef54108610 2024-04-23T14:48:36.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.601+0000 7fef52000640 1 -- 172.21.15.83:0/4259216960 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fef4801a020 con 0x7fef54108610 2024-04-23T14:48:36.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.602+0000 7fef52000640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fef3c0a3e40 0x7fef3c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:36.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.602+0000 7fef52000640 1 -- 172.21.15.83:0/4259216960 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fef480d0a70 con 0x7fef54108610 2024-04-23T14:48:36.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.602+0000 7fef58e00640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fef3c0a3e40 0x7fef3c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:36.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.603+0000 7fef58e00640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fef3c0a3e40 0x7fef3c0a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fef54075e40 tx=0x7fef40005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:36.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.606+0000 7fef52000640 1 -- 172.21.15.83:0/4259216960 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef48021020 con 0x7fef54108610 2024-04-23T14:48:36.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.819+0000 7fef5c200640 1 -- 172.21.15.83:0/4259216960 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef541a0200 con 0x7fef54108610 2024-04-23T14:48:36.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.821+0000 7fef52000640 1 -- 172.21.15.83:0/4259216960 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23076 (secure 0 0 0) 0x7fef4809ce70 con 0x7fef54108610 2024-04-23T14:48:36.823 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:48:36.823 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":89,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:48:27.715285+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/622708991":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/1081276647":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/4179157882":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6801/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/2255156114":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:48:36.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.823+0000 7fef5c200640 1 -- 172.21.15.83:0/4259216960 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fef3c0a3e40 msgr2=0x7fef3c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:36.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.824+0000 7fef5c200640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fef3c0a3e40 0x7fef3c0a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fef54075e40 tx=0x7fef40005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:36.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.824+0000 7fef5c200640 1 -- 172.21.15.83:0/4259216960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54108610 msgr2=0x7fef5419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:36.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.824+0000 7fef5c200640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54108610 0x7fef5419eeb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef4800d950 tx=0x7fef4800de10 comp rx=0 tx=0).stop 2024-04-23T14:48:36.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.824+0000 7fef5c200640 1 -- 172.21.15.83:0/4259216960 shutdown_connections 2024-04-23T14:48:36.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.824+0000 7fef5c200640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fef3c0a3e40 0x7fef3c0a6300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.824+0000 7fef5c200640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef5410e6b0 0x7fef541a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.824+0000 7fef5c200640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef54108fd0 0x7fef5419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.824+0000 7fef5c200640 1 --2- 172.21.15.83:0/4259216960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef54108610 0x7fef5419eeb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:36.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.824+0000 7fef5c200640 1 -- 172.21.15.83:0/4259216960 >> 172.21.15.83:0/4259216960 conn(0x7fef54075800 msgr2=0x7fef54076730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:36.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.824+0000 7fef5c200640 1 -- 172.21.15.83:0/4259216960 shutdown_connections 2024-04-23T14:48:36.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:36.824+0000 7fef5c200640 1 -- 172.21.15.83:0/4259216960 wait complete. 2024-04-23T14:48:36.845 DEBUG:tasks.fwd_scrub:Yielding 2024-04-23T14:48:36.845 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T14:48:36.855 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T14:48:36.855 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-23T14:48:36.855 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-23T14:48:36.855 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-23T14:48:36.855 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:48:36.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:48:36.856 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:48:37.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.026+0000 7f0413e00640 1 -- 172.21.15.83:0/904433738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f041410c400 msgr2=0x7f041410e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.026+0000 7f0413e00640 1 --2- 172.21.15.83:0/904433738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f041410c400 0x7f041410e8a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f040c009900 tx=0x7f040c02a480 comp rx=0 tx=0).stop 2024-04-23T14:48:37.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.026+0000 7f0413e00640 1 -- 172.21.15.83:0/904433738 shutdown_connections 2024-04-23T14:48:37.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.026+0000 7f0413e00640 1 --2- 172.21.15.83:0/904433738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f041410c400 0x7f041410e8a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.028 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.026+0000 7f0413e00640 1 --2- 172.21.15.83:0/904433738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0414103900 0x7f041410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.026+0000 7f0413e00640 1 --2- 172.21.15.83:0/904433738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0414102fe0 0x7f04141033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.026+0000 7f0413e00640 1 -- 172.21.15.83:0/904433738 >> 172.21.15.83:0/904433738 conn(0x7f04140fc7e0 msgr2=0x7f04140fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.026+0000 7f0413e00640 1 -- 172.21.15.83:0/904433738 shutdown_connections 2024-04-23T14:48:37.029 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.026+0000 7f0413e00640 1 -- 172.21.15.83:0/904433738 wait complete. 2024-04-23T14:48:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.027+0000 7f0413e00640 1 Processor -- start 2024-04-23T14:48:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.028+0000 7f0413e00640 1 -- start start 2024-04-23T14:48:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.028+0000 7f0413e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0414102fe0 0x7f04141a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.028+0000 7f0413e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0414103900 0x7f04141a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.028+0000 7f0413e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f041410c400 0x7f041419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.028+0000 7f0413e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04141121e0 con 0x7f041410c400 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.028+0000 7f0413e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0414112060 con 0x7f0414102fe0 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.028+0000 7f0413e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0414112360 con 0x7f0414103900 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.028+0000 7f040a000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0414103900 0x7f04141a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.028+0000 7f0413400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f041410c400 0x7f041419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.028+0000 7f0412a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0414102fe0 0x7f04141a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.028+0000 7f040a000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0414103900 0x7f04141a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49998/0 (socket says 172.21.15.83:49998) 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.028+0000 7f040a000640 1 -- 172.21.15.83:0/2539783991 learned_addr learned my addr 172.21.15.83:0/2539783991 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.029+0000 7f040a000640 1 -- 172.21.15.83:0/2539783991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0414102fe0 msgr2=0x7f04141a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.029+0000 7f040a000640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0414102fe0 0x7f04141a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.029+0000 7f040a000640 1 -- 172.21.15.83:0/2539783991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f041410c400 msgr2=0x7f041419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.029+0000 7f040a000640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f041410c400 0x7f041419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.029+0000 7f040a000640 1 -- 172.21.15.83:0/2539783991 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041419afb0 con 0x7f0414103900 2024-04-23T14:48:37.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.029+0000 7f0412a00640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0414102fe0 0x7f04141a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.029+0000 7f0413400640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f041410c400 0x7f041419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.029+0000 7f040a000640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0414103900 0x7f04141a0b70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00e9f0 tx=0x7f03fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.029+0000 7f040be00640 1 -- 172.21.15.83:0/2539783991 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00ccc0 con 0x7f0414103900 2024-04-23T14:48:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.029+0000 7f040be00640 1 -- 172.21.15.83:0/2539783991 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03fc00ce60 con 0x7f0414103900 2024-04-23T14:48:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.030+0000 7f0413e00640 1 -- 172.21.15.83:0/2539783991 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041419b2a0 con 0x7f0414103900 2024-04-23T14:48:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.030+0000 7f040be00640 1 -- 172.21.15.83:0/2539783991 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc010690 con 0x7f0414103900 2024-04-23T14:48:37.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.030+0000 7f0413e00640 1 -- 172.21.15.83:0/2539783991 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04141a75b0 con 0x7f0414103900 2024-04-23T14:48:37.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.030+0000 7f4660600640 1 -- 172.21.15.83:0/368606023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4658104e60 msgr2=0x7f4658108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.030+0000 7f4660600640 1 --2- 172.21.15.83:0/368606023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4658104e60 0x7f4658108ea0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4644009860 tx=0x7f464402a340 comp rx=0 tx=0).stop 2024-04-23T14:48:37.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.030+0000 7f4660600640 1 -- 172.21.15.83:0/368606023 shutdown_connections 2024-04-23T14:48:37.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.030+0000 7f4660600640 1 --2- 172.21.15.83:0/368606023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4658109580 0x7f4658111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.030+0000 7f4660600640 1 --2- 172.21.15.83:0/368606023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4658104e60 0x7f4658108ea0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.030+0000 7f4660600640 1 --2- 172.21.15.83:0/368606023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46581044b0 0x7f4658104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.030+0000 7f4660600640 1 -- 172.21.15.83:0/368606023 >> 172.21.15.83:0/368606023 conn(0x7f46580ffd90 msgr2=0x7f46581021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.031+0000 7f4660600640 1 -- 172.21.15.83:0/368606023 shutdown_connections 2024-04-23T14:48:37.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.031+0000 7f4660600640 1 -- 172.21.15.83:0/368606023 wait complete. 2024-04-23T14:48:37.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.031+0000 7f0413e00640 1 -- 172.21.15.83:0/2539783991 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041406a940 con 0x7f0414103900 2024-04-23T14:48:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.032+0000 7f4660600640 1 Processor -- start 2024-04-23T14:48:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.032+0000 7f4660600640 1 -- start start 2024-04-23T14:48:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.032+0000 7f4660600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46581044b0 0x7f46581a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.032+0000 7f4660600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4658109580 0x7f46581a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.032+0000 7f4660600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465819aa10 0x7f465819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.032+0000 7f4660600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4658114610 con 0x7f46581044b0 2024-04-23T14:48:37.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.032+0000 7f4660600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4658114490 con 0x7f4658109580 2024-04-23T14:48:37.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.032+0000 7f4660600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4658114790 con 0x7f465819aa10 2024-04-23T14:48:37.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.033+0000 7f465dc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46581044b0 0x7f46581a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.033+0000 7f465e600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465819aa10 0x7f465819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.033+0000 7f465e600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465819aa10 0x7f465819aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50000/0 (socket says 172.21.15.83:50000) 2024-04-23T14:48:37.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.033+0000 7f465e600640 1 -- 172.21.15.83:0/2038124430 learned_addr learned my addr 172.21.15.83:0/2038124430 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:37.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.033+0000 7f465d200640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4658109580 0x7f46581a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.033+0000 7f040be00640 1 -- 172.21.15.83:0/2539783991 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f03fc0046c0 con 0x7f0414103900 2024-04-23T14:48:37.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.033+0000 7f465e600640 1 -- 172.21.15.83:0/2038124430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4658109580 msgr2=0x7f46581a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.033+0000 7f465e600640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4658109580 0x7f46581a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.033+0000 7f465e600640 1 -- 172.21.15.83:0/2038124430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46581044b0 msgr2=0x7f46581a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.033+0000 7f465e600640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46581044b0 0x7f46581a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.033+0000 7f465e600640 1 -- 172.21.15.83:0/2038124430 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465819b530 con 0x7f465819aa10 2024-04-23T14:48:37.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.033+0000 7f465e600640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465819aa10 0x7f465819aec0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f465000efd0 tx=0x7f465000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.034+0000 7f040be00640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f03f80a3eb0 0x7f03f80a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.035+0000 7f040be00640 1 -- 172.21.15.83:0/2539783991 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f03fc0d0d00 con 0x7f0414103900 2024-04-23T14:48:37.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.035+0000 7f0412a00640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f03f80a3eb0 0x7f03f80a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.035+0000 7f0412a00640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f03f80a3eb0 0x7f03f80a6370 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f04040099a0 tx=0x7f0404009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.036+0000 7f4656000640 1 -- 172.21.15.83:0/2038124430 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650003b70 con 0x7f465819aa10 2024-04-23T14:48:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.036+0000 7f4656000640 1 -- 172.21.15.83:0/2038124430 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4650003d10 con 0x7f465819aa10 2024-04-23T14:48:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.036+0000 7f4660600640 1 -- 172.21.15.83:0/2038124430 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465819b820 con 0x7f465819aa10 2024-04-23T14:48:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.036+0000 7f4656000640 1 -- 172.21.15.83:0/2038124430 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46500108d0 con 0x7f465819aa10 2024-04-23T14:48:37.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.036+0000 7f4660600640 1 -- 172.21.15.83:0/2038124430 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46581a7a70 con 0x7f465819aa10 2024-04-23T14:48:37.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.036+0000 7f040be00640 1 -- 172.21.15.83:0/2539783991 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03fc09d100 con 0x7f0414103900 2024-04-23T14:48:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.039+0000 7f4656000640 1 -- 172.21.15.83:0/2038124430 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4650004020 con 0x7f465819aa10 2024-04-23T14:48:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.040+0000 7f466357f740 1 Processor -- start 2024-04-23T14:48:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.040+0000 7f466357f740 1 -- start start 2024-04-23T14:48:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.040+0000 7f466357f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223333470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.040+0000 7f466357f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c22333db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.040+0000 7f466357f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c223340470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.040+0000 7f466357f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c223298e70 con 0x55c22333e080 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.040+0000 7f466357f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c22323f380 con 0x55c2233339b0 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.040+0000 7f466357f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c223294e30 con 0x55c223333090 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.040+0000 7f465d200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223333470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f465dc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c22333db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f465d200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223333470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50012/0 (socket says 172.21.15.83:50012) 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f465d200640 1 -- 172.21.15.83:0/2066564732 learned_addr learned my addr 172.21.15.83:0/2066564732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f465d200640 1 --2- 172.21.15.83:0/2066564732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c223340470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f4656000640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f46400a3ec0 0x7f46400a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f465dc00640 1 -- 172.21.15.83:0/2066564732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 msgr2=0x55c223333470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f465dc00640 1 --2- 172.21.15.83:0/2066564732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223333470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f465dc00640 1 -- 172.21.15.83:0/2066564732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 msgr2=0x55c223340470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f465dc00640 1 --2- 172.21.15.83:0/2066564732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c223340470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f465dc00640 1 -- 172.21.15.83:0/2066564732 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2230737e0 con 0x55c2233339b0 2024-04-23T14:48:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f465d200640 1 --2- 172.21.15.83:0/2066564732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c223340470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:48:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f4656000640 1 -- 172.21.15.83:0/2038124430 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f46500d0e00 con 0x7f465819aa10 2024-04-23T14:48:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f465e600640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f46400a3ec0 0x7f46400a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.041+0000 7f465dc00640 1 --2- 172.21.15.83:0/2066564732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c22333db40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f464c005fd0 tx=0x7f464c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65bee8771a25c802 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.042+0000 7f464aa00640 1 -- 172.21.15.83:0/2066564732 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c004230 con 0x55c2233339b0 2024-04-23T14:48:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.042+0000 7f464aa00640 1 -- 172.21.15.83:0/2066564732 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f464c009e50 con 0x55c2233339b0 2024-04-23T14:48:37.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.042+0000 7f464aa00640 1 -- 172.21.15.83:0/2066564732 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c019750 con 0x55c2233339b0 2024-04-23T14:48:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.042+0000 7f466357f740 1 -- 172.21.15.83:0/2066564732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 msgr2=0x55c22333db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.042+0000 7f465e600640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f46400a3ec0 0x7f46400a6380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f46500ce3e0 tx=0x7f4650087040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.042+0000 7f466357f740 1 --2- 172.21.15.83:0/2066564732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c22333db40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f464c005fd0 tx=0x7f464c011810 comp rx=0 tx=0).stop 2024-04-23T14:48:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.042+0000 7f466357f740 1 -- 172.21.15.83:0/2066564732 shutdown_connections 2024-04-23T14:48:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.042+0000 7f466357f740 1 --2- 172.21.15.83:0/2066564732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c223340470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.042+0000 7f466357f740 1 --2- 172.21.15.83:0/2066564732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c22333db40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.042+0000 7f466357f740 1 --2- 172.21.15.83:0/2066564732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223333470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.042+0000 7f466357f740 1 -- 172.21.15.83:0/2066564732 >> 172.21.15.83:0/2066564732 conn(0x55c22332f3b0 msgr2=0x55c22332f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.043+0000 7f466357f740 1 -- 172.21.15.83:0/2066564732 shutdown_connections 2024-04-23T14:48:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.043+0000 7f466357f740 1 -- 172.21.15.83:0/2066564732 wait complete. 2024-04-23T14:48:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.043+0000 7f466357f740 1 -- start start 2024-04-23T14:48:37.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.043+0000 7f466357f740 1 Processor -- start 2024-04-23T14:48:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.044+0000 7f466357f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c223333090 0x55c22333af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.044+0000 7f466357f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2233339b0 0x55c22333b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.044+0000 7f466357f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c22333b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.044+0000 7f466357f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c223340e70 con 0x55c22333e080 2024-04-23T14:48:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.044+0000 7f466357f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c223252660 con 0x55c223333090 2024-04-23T14:48:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.044+0000 7f466357f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c223340ff0 con 0x55c2233339b0 2024-04-23T14:48:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.044+0000 7f466357f740 10 client.0 _ll_register_callbacks cb 0x55c223240aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:48:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.044+0000 7f465dc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2233339b0 0x55c22333b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.044+0000 7f465dc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2233339b0 0x55c22333b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50024/0 (socket says 172.21.15.83:50024) 2024-04-23T14:48:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.044+0000 7f465dc00640 1 -- 172.21.15.83:0/1296434327 learned_addr learned my addr 172.21.15.83:0/1296434327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.044+0000 7f465d200640 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c22333b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.044+0000 7f465d200640 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c223333090 0x55c22333af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.045+0000 7f465dc00640 1 -- 172.21.15.83:0/1296434327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c223333090 msgr2=0x55c22333af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.045+0000 7f465dc00640 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c223333090 0x55c22333af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.045+0000 7f465dc00640 1 -- 172.21.15.83:0/1296434327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 msgr2=0x55c22333b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.045+0000 7f465dc00640 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c22333b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.045+0000 7f465dc00640 1 -- 172.21.15.83:0/1296434327 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2230d3f70 con 0x55c2233339b0 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.045+0000 7f465dc00640 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2233339b0 0x55c22333b490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f464c009640 tx=0x7f464c005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a23d4b9cc36ba458 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.045+0000 7f463f400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.045+0000 7f463f400640 1 -- 172.21.15.83:0/1296434327 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c028020 con 0x55c2233339b0 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.045+0000 7f463f400640 1 -- 172.21.15.83:0/1296434327 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f464c0035f0 con 0x55c2233339b0 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.045+0000 7f466357f740 1 -- 172.21.15.83:0/1296434327 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f464c009e50 con 0x55c2233339b0 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.045+0000 7f463f400640 1 -- 172.21.15.83:0/1296434327 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f464c033050 con 0x55c2233339b0 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.046+0000 7f466357f740 10 client.25399 fetch_fsmap learned FSMap version 24 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.046+0000 7f466357f740 1 -- 172.21.15.83:0/1296434327 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c222f6b170 con 0x55c2233339b0 2024-04-23T14:48:37.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.046+0000 7f463f400640 1 -- 172.21.15.83:0/1296434327 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f464c009e50 con 0x55c2233339b0 2024-04-23T14:48:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.046+0000 7f463f400640 1 -- 172.21.15.83:0/1296434327 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f464c018070 con 0x55c2233339b0 2024-04-23T14:48:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.046+0000 7f466357f740 10 client.25399 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:48:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.046+0000 7f466357f740 10 client.25399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:48:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.046+0000 7f466357f740 20 client.25399 populate_metadata read hostname 'smithi083' 2024-04-23T14:48:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.046+0000 7f466357f740 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c22334e2d0 0x55c2233506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.046+0000 7f466357f740 4 client.25399 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:48:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.046+0000 7f466357f740 1 -- 172.21.15.83:0/1296434327 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2232a6890 con 0x55c22334e2d0 2024-04-23T14:48:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.047+0000 7f465d200640 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c22334e2d0 0x55c2233506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.047+0000 7f463f400640 10 client.25399 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:48:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.047+0000 7f465d200640 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c22334e2d0 0x55c2233506c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f039216eacab32c server_cookie=1ce8606f44854e09 in_seq=0 out_seq=0 2024-04-23T14:48:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.048+0000 7f463f400640 1 -- 172.21.15.83:0/1296434327 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2232a6890 con 0x55c22334e2d0 2024-04-23T14:48:37.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.048+0000 7f463f400640 10 client.25399 handle_command_reply: tid=0 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.048+0000 7f466357f740 1 client.25399 shutdown 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.048+0000 7f466357f740 1 -- 172.21.15.83:0/1296434327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2233339b0 msgr2=0x55c22333b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.048+0000 7f466357f740 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2233339b0 0x55c22333b490 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f464c009640 tx=0x7f464c005270 comp rx=0 tx=0).stop 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.049+0000 7f466357f740 1 -- 172.21.15.83:0/1296434327 shutdown_connections 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.049+0000 7f466357f740 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c22334e2d0 0x55c2233506c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.049+0000 7f466357f740 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c22333b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.049+0000 7f463f400640 0 client.25399 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.049+0000 7f466357f740 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c2233339b0 0x55c22333b490 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.049+0000 7f466357f740 1 --2- 172.21.15.83:0/1296434327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c223333090 0x55c22333af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.049+0000 7f466357f740 1 -- 172.21.15.83:0/1296434327 >> 172.21.15.83:0/1296434327 conn(0x55c22332f3b0 msgr2=0x55c2233351b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.049+0000 7f466357f740 1 -- 172.21.15.83:0/1296434327 shutdown_connections 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.049+0000 7f466357f740 1 -- 172.21.15.83:0/1296434327 wait complete. 2024-04-23T14:48:37.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.049+0000 7f466357f740 20 client.25399 trim_cache size 0 max 16384 2024-04-23T14:48:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.068+0000 7f466357f740 1 Processor -- start 2024-04-23T14:48:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.069+0000 7f466357f740 1 -- start start 2024-04-23T14:48:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.069+0000 7f466357f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223349d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.069+0000 7f466357f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c223355ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.069+0000 7f466357f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c223356400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.069+0000 7f466357f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46581b27c0 con 0x55c22333e080 2024-04-23T14:48:37.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.069+0000 7f466357f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f46581b0090 con 0x55c2233339b0 2024-04-23T14:48:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.069+0000 7f466357f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f46581adee0 con 0x55c223333090 2024-04-23T14:48:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.069+0000 7f465d200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223349d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.069+0000 7f465d200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223349d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50036/0 (socket says 172.21.15.83:50036) 2024-04-23T14:48:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.069+0000 7f465dc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c223355ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.069+0000 7f465d200640 1 -- 172.21.15.83:0/1912519453 learned_addr learned my addr 172.21.15.83:0/1912519453 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.069+0000 7f465d200640 1 --2- 172.21.15.83:0/1912519453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c223356400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.070+0000 7f465dc00640 1 -- 172.21.15.83:0/1912519453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 msgr2=0x55c223349d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.070+0000 7f465dc00640 1 --2- 172.21.15.83:0/1912519453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223349d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.070+0000 7f465dc00640 1 -- 172.21.15.83:0/1912519453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 msgr2=0x55c223356400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.070+0000 7f465dc00640 1 --2- 172.21.15.83:0/1912519453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c223356400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.070+0000 7f465dc00640 1 -- 172.21.15.83:0/1912519453 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2233337e0 con 0x55c2233339b0 2024-04-23T14:48:37.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.070+0000 7f465dc00640 1 --2- 172.21.15.83:0/1912519453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c223355ec0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f464c009670 tx=0x7f464c012f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e02faadc3239afb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.070+0000 7f464a000640 1 -- 172.21.15.83:0/1912519453 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c028030 con 0x55c2233339b0 2024-04-23T14:48:37.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.070+0000 7f464a000640 1 -- 172.21.15.83:0/1912519453 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f464c009e50 con 0x55c2233339b0 2024-04-23T14:48:37.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.070+0000 7f464a000640 1 -- 172.21.15.83:0/1912519453 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c01f040 con 0x55c2233339b0 2024-04-23T14:48:37.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.071+0000 7f466357f740 1 -- 172.21.15.83:0/1912519453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 msgr2=0x55c223355ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.071+0000 7f466357f740 1 --2- 172.21.15.83:0/1912519453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c223355ec0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f464c009670 tx=0x7f464c012f90 comp rx=0 tx=0).stop 2024-04-23T14:48:37.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.071+0000 7f466357f740 1 -- 172.21.15.83:0/1912519453 shutdown_connections 2024-04-23T14:48:37.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.071+0000 7f466357f740 1 --2- 172.21.15.83:0/1912519453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c223356400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.071+0000 7f466357f740 1 --2- 172.21.15.83:0/1912519453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c223355ec0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.071+0000 7f466357f740 1 --2- 172.21.15.83:0/1912519453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223349d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.071+0000 7f466357f740 1 -- 172.21.15.83:0/1912519453 >> 172.21.15.83:0/1912519453 conn(0x55c22332f3b0 msgr2=0x55c22333b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.071+0000 7f466357f740 1 -- 172.21.15.83:0/1912519453 shutdown_connections 2024-04-23T14:48:37.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.071+0000 7f466357f740 1 -- 172.21.15.83:0/1912519453 wait complete. 2024-04-23T14:48:37.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.071+0000 7f466357f740 1 -- start start 2024-04-23T14:48:37.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.072+0000 7f466357f740 1 Processor -- start 2024-04-23T14:48:37.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.072+0000 7f466357f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223342340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.072+0000 7f466357f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c223342880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.072+0000 7f466357f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c2233b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.072+0000 7f466357f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46581ae550 con 0x55c22333e080 2024-04-23T14:48:37.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.072+0000 7f466357f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f46581af9c0 con 0x55c2233339b0 2024-04-23T14:48:37.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.072+0000 7f466357f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f46581aec20 con 0x55c223333090 2024-04-23T14:48:37.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.072+0000 7f466357f740 10 client.0 _ll_register_callbacks cb 0x7f46500093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:48:37.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.072+0000 7f465dc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c223342880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.072+0000 7f465d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c2233b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.072+0000 7f465d200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c2233b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46150/0 (socket says 172.21.15.83:46150) 2024-04-23T14:48:37.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.072+0000 7f465d200640 1 -- 172.21.15.83:0/3178994042 learned_addr learned my addr 172.21.15.83:0/3178994042 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:37.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.073+0000 7f465d200640 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223342340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.073+0000 7f465dc00640 1 -- 172.21.15.83:0/3178994042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 msgr2=0x55c223342340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.073+0000 7f465dc00640 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223342340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.073+0000 7f465dc00640 1 -- 172.21.15.83:0/3178994042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 msgr2=0x55c2233b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.073+0000 7f465dc00640 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c2233b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.073+0000 7f465dc00640 1 -- 172.21.15.83:0/3178994042 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2233b3c70 con 0x55c2233339b0 2024-04-23T14:48:37.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.073+0000 7f465d200640 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c2233b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.073+0000 7f465dc00640 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c223342880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f464c004e90 tx=0x7f464c004450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=113626d52f868be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.073+0000 7f463f400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.074+0000 7f463f400640 1 -- 172.21.15.83:0/3178994042 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c028070 con 0x55c2233339b0 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.074+0000 7f466357f740 1 -- 172.21.15.83:0/3178994042 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f464c009e50 con 0x55c2233339b0 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.074+0000 7f463f400640 1 -- 172.21.15.83:0/3178994042 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f464c004b10 con 0x55c2233339b0 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.074+0000 7f463f400640 1 -- 172.21.15.83:0/3178994042 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f464c005250 con 0x55c2233339b0 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.074+0000 7f466357f740 10 client.25421 fetch_fsmap learned FSMap version 24 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.074+0000 7f466357f740 1 -- 172.21.15.83:0/3178994042 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c2233b4f60 con 0x55c2233339b0 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.074+0000 7f463f400640 1 -- 172.21.15.83:0/3178994042 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f464c009e50 con 0x55c2233339b0 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.074+0000 7f463f400640 1 -- 172.21.15.83:0/3178994042 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f464c018040 con 0x55c2233339b0 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.074+0000 7f466357f740 10 client.25421 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.074+0000 7f466357f740 10 client.25421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:48:37.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.074+0000 7f466357f740 20 client.25421 populate_metadata read hostname 'smithi083' 2024-04-23T14:48:37.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.075+0000 7f466357f740 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c2233b55c0 0x55c2233b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.075+0000 7f466357f740 4 client.25421 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:48:37.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.075+0000 7f466357f740 1 -- 172.21.15.83:0/3178994042 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c2232a0700 con 0x55c2233b55c0 2024-04-23T14:48:37.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.075+0000 7f465d200640 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c2233b55c0 0x55c2233b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.075+0000 7f463f400640 10 client.25421 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:48:37.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.075+0000 7f465d200640 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c2233b55c0 0x55c2233b79b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2272296a6065aa5 server_cookie=d1ba45aef186267 in_seq=0 out_seq=0 2024-04-23T14:48:37.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.077+0000 7f463f400640 1 -- 172.21.15.83:0/3178994042 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c2232a0700 con 0x55c2233b55c0 2024-04-23T14:48:37.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.077+0000 7f463f400640 10 client.25421 handle_command_reply: tid=0 2024-04-23T14:48:37.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.077+0000 7f466357f740 1 client.25421 shutdown 2024-04-23T14:48:37.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.077+0000 7f466357f740 1 -- 172.21.15.83:0/3178994042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 msgr2=0x55c223342880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.077+0000 7f466357f740 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c223342880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f464c004e90 tx=0x7f464c004450 comp rx=0 tx=0).stop 2024-04-23T14:48:37.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.077+0000 7f466357f740 1 -- 172.21.15.83:0/3178994042 shutdown_connections 2024-04-23T14:48:37.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.077+0000 7f466357f740 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c2233b55c0 0x55c2233b79b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.077+0000 7f466357f740 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c22333e080 0x55c2233b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.077+0000 7f466357f740 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c2233339b0 0x55c223342880 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.077+0000 7f463f400640 0 client.25421 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:48:37.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.077+0000 7f466357f740 1 --2- 172.21.15.83:0/3178994042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c223333090 0x55c223342340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.077+0000 7f466357f740 1 -- 172.21.15.83:0/3178994042 >> 172.21.15.83:0/3178994042 conn(0x55c22332f3b0 msgr2=0x55c2233347e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.078+0000 7f466357f740 1 -- 172.21.15.83:0/3178994042 shutdown_connections 2024-04-23T14:48:37.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.078+0000 7f466357f740 1 -- 172.21.15.83:0/3178994042 wait complete. 2024-04-23T14:48:37.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.078+0000 7f466357f740 20 client.25421 trim_cache size 0 max 16384 2024-04-23T14:48:37.084 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:48:37.084 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:48:37.084 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "a36634b2-6a33-46c8-a66a-7c86f57ac9fb", 2024-04-23T14:48:37.084 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:48:37.084 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:48:37.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.078+0000 7f4660600640 1 -- 172.21.15.83:0/2038124430 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f46400a3ec0 msgr2=0x7f46400a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.078+0000 7f4660600640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f46400a3ec0 0x7f46400a6380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f46500ce3e0 tx=0x7f4650087040 comp rx=0 tx=0).stop 2024-04-23T14:48:37.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.078+0000 7f4660600640 1 -- 172.21.15.83:0/2038124430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465819aa10 msgr2=0x7f465819aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.078+0000 7f4660600640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465819aa10 0x7f465819aec0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f465000efd0 tx=0x7f465000c6a0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.079+0000 7f4660600640 1 -- 172.21.15.83:0/2038124430 shutdown_connections 2024-04-23T14:48:37.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.079+0000 7f4660600640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f465819aa10 0x7f465819aec0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.079+0000 7f4660600640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4658109580 0x7f46581a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.079+0000 7f4660600640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f46400a3ec0 0x7f46400a6380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.079+0000 7f4660600640 1 --2- 172.21.15.83:0/2038124430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46581044b0 0x7f46581a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.079+0000 7f4660600640 1 -- 172.21.15.83:0/2038124430 >> 172.21.15.83:0/2038124430 conn(0x7f46580ffd90 msgr2=0x7f4658101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.079+0000 7f4660600640 1 -- 172.21.15.83:0/2038124430 shutdown_connections 2024-04-23T14:48:37.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.079+0000 7f4660600640 1 -- 172.21.15.83:0/2038124430 wait complete. 2024-04-23T14:48:37.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:48:37.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:48:37.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.275+0000 7f0413e00640 1 -- 172.21.15.83:0/2539783991 --> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f0414062440 con 0x7f03f80a3eb0 2024-04-23T14:48:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.290+0000 7f0883e00640 1 -- 172.21.15.83:0/130449063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088410c480 msgr2=0x7f088410e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.290+0000 7f0883e00640 1 --2- 172.21.15.83:0/130449063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088410c480 0x7f088410e920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f087c009ca0 tx=0x7f087c02a940 comp rx=0 tx=0).stop 2024-04-23T14:48:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.290+0000 7f0883e00640 1 -- 172.21.15.83:0/130449063 shutdown_connections 2024-04-23T14:48:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.290+0000 7f0883e00640 1 --2- 172.21.15.83:0/130449063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088410c480 0x7f088410e920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.290+0000 7f0883e00640 1 --2- 172.21.15.83:0/130449063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0884103980 0x7f088410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.290+0000 7f0883e00640 1 --2- 172.21.15.83:0/130449063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0884103060 0x7f0884103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.290+0000 7f0883e00640 1 -- 172.21.15.83:0/130449063 >> 172.21.15.83:0/130449063 conn(0x7f08840fc880 msgr2=0x7f08840feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.291+0000 7f0883e00640 1 -- 172.21.15.83:0/130449063 shutdown_connections 2024-04-23T14:48:37.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.291+0000 7f0883e00640 1 -- 172.21.15.83:0/130449063 wait complete. 2024-04-23T14:48:37.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.292+0000 7f0883e00640 1 Processor -- start 2024-04-23T14:48:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.292+0000 7f0883e00640 1 -- start start 2024-04-23T14:48:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.293+0000 7f0883e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0884103060 0x7f08841a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.293+0000 7f0883e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0884103980 0x7f08841a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.293+0000 7f0883e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088410c480 0x7f088419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.293+0000 7f0883e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08841122b0 con 0x7f0884103980 2024-04-23T14:48:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.293+0000 7f0883e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0884112130 con 0x7f0884103060 2024-04-23T14:48:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.293+0000 7f0883e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0884112430 con 0x7f088410c480 2024-04-23T14:48:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.293+0000 7f0883400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088410c480 0x7f088419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.293+0000 7f0882000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0884103980 0x7f08841a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.293+0000 7f0882a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0884103060 0x7f08841a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.293+0000 7f0882a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0884103060 0x7f08841a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:44482/0 (socket says 172.21.15.83:44482) 2024-04-23T14:48:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.293+0000 7f0883400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088410c480 0x7f088419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50064/0 (socket says 172.21.15.83:50064) 2024-04-23T14:48:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.293+0000 7f0882a00640 1 -- 172.21.15.83:0/3794484527 learned_addr learned my addr 172.21.15.83:0/3794484527 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.294+0000 7f0883400640 1 -- 172.21.15.83:0/3794484527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0884103060 msgr2=0x7f08841a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.294+0000 7f0883400640 1 --2- 172.21.15.83:0/3794484527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0884103060 0x7f08841a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.294+0000 7f0883400640 1 -- 172.21.15.83:0/3794484527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0884103980 msgr2=0x7f08841a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.294+0000 7f0883400640 1 --2- 172.21.15.83:0/3794484527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0884103980 0x7f08841a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.294+0000 7f0883400640 1 -- 172.21.15.83:0/3794484527 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088419b020 con 0x7f088410c480 2024-04-23T14:48:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.294+0000 7f0883400640 1 --2- 172.21.15.83:0/3794484527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088410c480 0x7f088419a760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f087c005e30 tx=0x7f087c02f700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.295+0000 7f087b400640 1 -- 172.21.15.83:0/3794484527 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c034040 con 0x7f088410c480 2024-04-23T14:48:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.295+0000 7f087b400640 1 -- 172.21.15.83:0/3794484527 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f087c02fe10 con 0x7f088410c480 2024-04-23T14:48:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.295+0000 7f0883e00640 1 -- 172.21.15.83:0/3794484527 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088419b2b0 con 0x7f088410c480 2024-04-23T14:48:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.295+0000 7f087b400640 1 -- 172.21.15.83:0/3794484527 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c004730 con 0x7f088410c480 2024-04-23T14:48:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.295+0000 7f0883e00640 1 -- 172.21.15.83:0/3794484527 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08841a7620 con 0x7f088410c480 2024-04-23T14:48:37.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.299+0000 7f087b400640 1 -- 172.21.15.83:0/3794484527 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f087c037020 con 0x7f088410c480 2024-04-23T14:48:37.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.299+0000 7f088c43c740 1 Processor -- start 2024-04-23T14:48:37.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.300+0000 7f088c43c740 1 -- start start 2024-04-23T14:48:37.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.300+0000 7f088c43c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34ea30 0x56165c34ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.300+0000 7f088c43c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c34f350 0x56165c3594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.300+0000 7f088c43c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359a20 0x56165c35be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.300+0000 7f088c43c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56165c2b41f0 con 0x56165c34ea30 2024-04-23T14:48:37.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.300+0000 7f088c43c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56165c25a350 con 0x56165c34f350 2024-04-23T14:48:37.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.300+0000 7f088c43c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56165c2b01b0 con 0x56165c359a20 2024-04-23T14:48:37.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.300+0000 7f0882000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34ea30 0x56165c34ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.300+0000 7f0882000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34ea30 0x56165c34ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46168/0 (socket says 172.21.15.83:46168) 2024-04-23T14:48:37.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.300+0000 7f0882000640 1 -- 172.21.15.83:0/2118213889 learned_addr learned my addr 172.21.15.83:0/2118213889 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:37.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.301+0000 7f0882000640 1 --2- 172.21.15.83:0/2118213889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359a20 0x56165c35be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.301+0000 7f0882a00640 1 --2- 172.21.15.83:0/2118213889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c34f350 0x56165c3594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.301+0000 7f0882000640 1 -- 172.21.15.83:0/2118213889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359a20 msgr2=0x56165c35be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.301+0000 7f0882000640 1 --2- 172.21.15.83:0/2118213889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359a20 0x56165c35be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.301+0000 7f0882000640 1 -- 172.21.15.83:0/2118213889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c34f350 msgr2=0x56165c3594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.301+0000 7f0882000640 1 --2- 172.21.15.83:0/2118213889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c34f350 0x56165c3594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.301+0000 7f0882000640 1 -- 172.21.15.83:0/2118213889 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56165c2aef50 con 0x56165c34ea30 2024-04-23T14:48:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.301+0000 7f087b400640 1 --2- 172.21.15.83:0/3794484527 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f08680a3ec0 0x7f08680a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.301+0000 7f0882000640 1 --2- 172.21.15.83:0/2118213889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34ea30 0x56165c34ee10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f087000bd90 tx=0x7f08700040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5d9e43790fbf9e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.302+0000 7f087b400640 1 -- 172.21.15.83:0/3794484527 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f087c0f17e0 con 0x7f088410c480 2024-04-23T14:48:37.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.302+0000 7f0882a00640 1 --2- 172.21.15.83:0/3794484527 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f08680a3ec0 0x7f08680a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.302+0000 7f086f400640 1 -- 172.21.15.83:0/2118213889 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087000be80 con 0x56165c34ea30 2024-04-23T14:48:37.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.302+0000 7f086f400640 1 -- 172.21.15.83:0/2118213889 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870003e80 con 0x56165c34ea30 2024-04-23T14:48:37.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.302+0000 7f086f400640 1 -- 172.21.15.83:0/2118213889 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870012450 con 0x56165c34ea30 2024-04-23T14:48:37.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.303+0000 7f088c43c740 1 -- 172.21.15.83:0/2118213889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34ea30 msgr2=0x56165c34ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.303+0000 7f088c43c740 1 --2- 172.21.15.83:0/2118213889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34ea30 0x56165c34ee10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f087000bd90 tx=0x7f08700040e0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.303+0000 7f0882a00640 1 --2- 172.21.15.83:0/3794484527 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f08680a3ec0 0x7f08680a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x56165c34fff0 tx=0x7f087400b480 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.303+0000 7f088c43c740 1 -- 172.21.15.83:0/2118213889 shutdown_connections 2024-04-23T14:48:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.303+0000 7f088c43c740 1 --2- 172.21.15.83:0/2118213889 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359a20 0x56165c35be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.303+0000 7f088c43c740 1 --2- 172.21.15.83:0/2118213889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c34f350 0x56165c3594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.303+0000 7f088c43c740 1 --2- 172.21.15.83:0/2118213889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34ea30 0x56165c34ee10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.303+0000 7f088c43c740 1 -- 172.21.15.83:0/2118213889 >> 172.21.15.83:0/2118213889 conn(0x56165c34ad50 msgr2=0x56165c34b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.303+0000 7f088c43c740 1 -- 172.21.15.83:0/2118213889 shutdown_connections 2024-04-23T14:48:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.303+0000 7f088c43c740 1 -- 172.21.15.83:0/2118213889 wait complete. 2024-04-23T14:48:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.304+0000 7f088c43c740 1 -- start start 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.304+0000 7f088c43c740 1 Processor -- start 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.304+0000 7f088c43c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c34ea30 0x56165c3569f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f088c43c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34f350 0x56165c356f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f088c43c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c359a20 0x56165c357470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f088c43c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56165c35c7b0 con 0x56165c34f350 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f088c43c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56165c26d690 con 0x56165c359a20 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f088c43c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56165c35c930 con 0x56165c34ea30 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f088c43c740 10 client.0 _ll_register_callbacks cb 0x56165c2a9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f0882000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34f350 0x56165c356f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f0882a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c34ea30 0x56165c3569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f0883400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c359a20 0x56165c357470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f0882a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c34ea30 0x56165c3569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50078/0 (socket says 172.21.15.83:50078) 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f0882a00640 1 -- 172.21.15.83:0/3360014567 learned_addr learned my addr 172.21.15.83:0/3360014567 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f0882000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34f350 0x56165c356f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46182/0 (socket says 172.21.15.83:46182) 2024-04-23T14:48:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f0882a00640 1 -- 172.21.15.83:0/3360014567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c359a20 msgr2=0x56165c357470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f0882a00640 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c359a20 0x56165c357470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f0882a00640 1 -- 172.21.15.83:0/3360014567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34f350 msgr2=0x56165c356f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f0882a00640 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34f350 0x56165c356f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.305+0000 7f0882a00640 1 -- 172.21.15.83:0/3360014567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56165c26c920 con 0x56165c34ea30 2024-04-23T14:48:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.306+0000 7f0882000640 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34f350 0x56165c356f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.306+0000 7f0883400640 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c359a20 0x56165c357470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.306+0000 7f0882a00640 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c34ea30 0x56165c3569f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0874003e20 tx=0x7f0874004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=886cf2a92facf755 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.306+0000 7f0863e00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:48:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.306+0000 7f0863e00640 1 -- 172.21.15.83:0/3360014567 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874005020 con 0x56165c34ea30 2024-04-23T14:48:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.306+0000 7f088c43c740 1 -- 172.21.15.83:0/3360014567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0870003e80 con 0x56165c34ea30 2024-04-23T14:48:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.306+0000 7f0863e00640 1 -- 172.21.15.83:0/3360014567 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f08740051c0 con 0x56165c34ea30 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.307+0000 7f0863e00640 1 -- 172.21.15.83:0/3360014567 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f08740053c0 con 0x56165c34ea30 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.307+0000 7f088c43c740 10 client.25417 fetch_fsmap learned FSMap version 24 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.307+0000 7f088c43c740 1 -- 172.21.15.83:0/3360014567 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56165c0907d0 con 0x56165c34ea30 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.307+0000 7f0863e00640 1 -- 172.21.15.83:0/3360014567 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870003e80 con 0x56165c34ea30 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.307+0000 7f0863e00640 1 -- 172.21.15.83:0/3360014567 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0874026070 con 0x56165c34ea30 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.307+0000 7f088c43c740 10 client.25417 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.307+0000 7f088c43c740 10 client.25417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.307+0000 7f088c43c740 20 client.25417 populate_metadata read hostname 'smithi083' 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.307+0000 7f088c43c740 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56165c3725f0 0x56165c374a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.308+0000 7f088c43c740 4 client.25417 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.308+0000 7f088c43c740 1 -- 172.21.15.83:0/3360014567 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56165c2c1c10 con 0x56165c3725f0 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.308+0000 7f0882000640 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56165c3725f0 0x56165c374a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.309+0000 7f0863e00640 10 client.25417 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:48:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.309+0000 7f0882000640 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56165c3725f0 0x56165c374a60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e88d66bb9495b52 server_cookie=f506259ff90b8db8 in_seq=0 out_seq=0 2024-04-23T14:48:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.309+0000 7f0863e00640 1 -- 172.21.15.83:0/3360014567 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56165c2c1c10 con 0x56165c3725f0 2024-04-23T14:48:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.309+0000 7f0863e00640 10 client.25417 handle_command_reply: tid=0 2024-04-23T14:48:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.310+0000 7f088c43c740 1 client.25417 shutdown 2024-04-23T14:48:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.310+0000 7f088c43c740 1 -- 172.21.15.83:0/3360014567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c34ea30 msgr2=0x56165c3569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.310+0000 7f088c43c740 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c34ea30 0x56165c3569f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0874003e20 tx=0x7f0874004730 comp rx=0 tx=0).stop 2024-04-23T14:48:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.310+0000 7f088c43c740 1 -- 172.21.15.83:0/3360014567 shutdown_connections 2024-04-23T14:48:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.310+0000 7f088c43c740 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56165c3725f0 0x56165c374a60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.310+0000 7f088c43c740 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c359a20 0x56165c357470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.310+0000 7f088c43c740 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34f350 0x56165c356f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.310+0000 7f088c43c740 1 --2- 172.21.15.83:0/3360014567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c34ea30 0x56165c3569f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.310+0000 7f0863e00640 0 client.25417 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:48:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.310+0000 7f088c43c740 1 -- 172.21.15.83:0/3360014567 >> 172.21.15.83:0/3360014567 conn(0x56165c34ad50 msgr2=0x56165c349fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.311+0000 7f088c43c740 1 -- 172.21.15.83:0/3360014567 shutdown_connections 2024-04-23T14:48:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.311+0000 7f088c43c740 1 -- 172.21.15.83:0/3360014567 wait complete. 2024-04-23T14:48:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.311+0000 7f088c43c740 20 client.25417 trim_cache size 0 max 16384 2024-04-23T14:48:37.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.333+0000 7f088c43c740 1 Processor -- start 2024-04-23T14:48:37.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.333+0000 7f088c43c740 1 -- start start 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f088c43c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34f350 0x56165c36df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f088c43c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359a20 0x56165c350890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f088c43c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c359230 0x56165c37a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f088c43c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08841ae190 con 0x56165c34f350 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f088c43c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f08841aba60 con 0x56165c359230 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f088c43c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f08841a98d0 con 0x56165c359a20 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f0882000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359a20 0x56165c350890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f0882000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359a20 0x56165c350890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50092/0 (socket says 172.21.15.83:50092) 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f0882000640 1 -- 172.21.15.83:0/2504785239 learned_addr learned my addr 172.21.15.83:0/2504785239 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f0882a00640 1 --2- 172.21.15.83:0/2504785239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34f350 0x56165c36df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f0883400640 1 --2- 172.21.15.83:0/2504785239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c359230 0x56165c37a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f0882000640 1 -- 172.21.15.83:0/2504785239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c359230 msgr2=0x56165c37a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f0882000640 1 --2- 172.21.15.83:0/2504785239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c359230 0x56165c37a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f0882000640 1 -- 172.21.15.83:0/2504785239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34f350 msgr2=0x56165c36df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f0882000640 1 --2- 172.21.15.83:0/2504785239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34f350 0x56165c36df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f0882000640 1 -- 172.21.15.83:0/2504785239 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56165c34f180 con 0x56165c359a20 2024-04-23T14:48:37.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.334+0000 7f0883400640 1 --2- 172.21.15.83:0/2504785239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c359230 0x56165c37a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:48:37.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.335+0000 7f0882000640 1 --2- 172.21.15.83:0/2504785239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359a20 0x56165c350890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0870004650 tx=0x7f087001b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b519da94945bd95 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.335+0000 7f086ea00640 1 -- 172.21.15.83:0/2504785239 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087002c020 con 0x56165c359a20 2024-04-23T14:48:37.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.335+0000 7f086ea00640 1 -- 172.21.15.83:0/2504785239 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870028dc0 con 0x56165c359a20 2024-04-23T14:48:37.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.335+0000 7f086ea00640 1 -- 172.21.15.83:0/2504785239 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870004da0 con 0x56165c359a20 2024-04-23T14:48:37.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.336+0000 7f088c43c740 1 -- 172.21.15.83:0/2504785239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359a20 msgr2=0x56165c350890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.336+0000 7f088c43c740 1 --2- 172.21.15.83:0/2504785239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359a20 0x56165c350890 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0870004650 tx=0x7f087001b040 comp rx=0 tx=0).stop 2024-04-23T14:48:37.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.336+0000 7f088c43c740 1 -- 172.21.15.83:0/2504785239 shutdown_connections 2024-04-23T14:48:37.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.336+0000 7f088c43c740 1 --2- 172.21.15.83:0/2504785239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c359230 0x56165c37a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.336+0000 7f088c43c740 1 --2- 172.21.15.83:0/2504785239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359a20 0x56165c350890 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.336+0000 7f088c43c740 1 --2- 172.21.15.83:0/2504785239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c34f350 0x56165c36df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.336+0000 7f088c43c740 1 -- 172.21.15.83:0/2504785239 >> 172.21.15.83:0/2504785239 conn(0x56165c34ad50 msgr2=0x56165c350180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.336+0000 7f088c43c740 1 -- 172.21.15.83:0/2504785239 shutdown_connections 2024-04-23T14:48:37.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.336+0000 7f088c43c740 1 -- 172.21.15.83:0/2504785239 wait complete. 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.337+0000 7f088c43c740 1 -- start start 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.337+0000 7f088c43c740 1 Processor -- start 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.337+0000 7f088c43c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c34f350 0x56165c365f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.337+0000 7f088c43c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359230 0x56165c366440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.337+0000 7f088c43c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c359a20 0x56165c366980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.337+0000 7f088c43c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f08841a9f20 con 0x56165c359a20 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.337+0000 7f088c43c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f08841ab390 con 0x56165c34f350 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.337+0000 7f088c43c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f08841aa5f0 con 0x56165c359230 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.337+0000 7f088c43c740 10 client.0 _ll_register_callbacks cb 0x7f08740042a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0882000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359230 0x56165c366440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0882000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359230 0x56165c366440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50102/0 (socket says 172.21.15.83:50102) 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0882000640 1 -- 172.21.15.83:0/498859946 learned_addr learned my addr 172.21.15.83:0/498859946 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0882a00640 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c34f350 0x56165c365f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0883400640 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c359a20 0x56165c366980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0882000640 1 -- 172.21.15.83:0/498859946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c34f350 msgr2=0x56165c365f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0882000640 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c34f350 0x56165c365f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0882000640 1 -- 172.21.15.83:0/498859946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c359a20 msgr2=0x56165c366980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:48:37.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0882000640 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c359a20 0x56165c366980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0882000640 1 -- 172.21.15.83:0/498859946 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56165bfc63f0 con 0x56165c359230 2024-04-23T14:48:37.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0883400640 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c359a20 0x56165c366980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:48:37.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0882000640 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359230 0x56165c366440 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f087000b540 tx=0x7f0870003e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4be55237495d3d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.338+0000 7f0863e00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:48:37.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.339+0000 7f0863e00640 1 -- 172.21.15.83:0/498859946 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087002c030 con 0x56165c359230 2024-04-23T14:48:37.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.339+0000 7f088c43c740 1 -- 172.21.15.83:0/498859946 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0870028dc0 con 0x56165c359230 2024-04-23T14:48:37.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.339+0000 7f0863e00640 1 -- 172.21.15.83:0/498859946 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f08700285a0 con 0x56165c359230 2024-04-23T14:48:37.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.339+0000 7f0863e00640 1 -- 172.21.15.83:0/498859946 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f087003f050 con 0x56165c359230 2024-04-23T14:48:37.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.339+0000 7f088c43c740 10 client.25423 fetch_fsmap learned FSMap version 24 2024-04-23T14:48:37.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.339+0000 7f088c43c740 1 -- 172.21.15.83:0/498859946 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56165bf860f0 con 0x56165c359230 2024-04-23T14:48:37.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.340+0000 7f0863e00640 1 -- 172.21.15.83:0/498859946 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870028dc0 con 0x56165c359230 2024-04-23T14:48:37.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.340+0000 7f0863e00640 1 -- 172.21.15.83:0/498859946 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f0870005250 con 0x56165c359230 2024-04-23T14:48:37.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.340+0000 7f088c43c740 10 client.25423 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.340+0000 7f088c43c740 10 client.25423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.340+0000 7f088c43c740 20 client.25423 populate_metadata read hostname 'smithi083' 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.340+0000 7f088c43c740 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56165c3d9160 0x56165c3db580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.340+0000 7f088c43c740 4 client.25423 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.340+0000 7f088c43c740 1 -- 172.21.15.83:0/498859946 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56165c2bba80 con 0x56165c3d9160 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.340+0000 7f0882a00640 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56165c3d9160 0x56165c3db580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.341+0000 7f0882a00640 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56165c3d9160 0x56165c3db580 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846b4cab2f41fdcd server_cookie=3f405c2c37c62e8b in_seq=0 out_seq=0 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.341+0000 7f0863e00640 10 client.25423 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.341+0000 7f0863e00640 1 -- 172.21.15.83:0/498859946 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56165c2bba80 con 0x56165c3d9160 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.341+0000 7f0863e00640 10 client.25423 handle_command_reply: tid=0 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.341+0000 7f088c43c740 1 client.25423 shutdown 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.342+0000 7f088c43c740 1 -- 172.21.15.83:0/498859946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359230 msgr2=0x56165c366440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.342+0000 7f088c43c740 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359230 0x56165c366440 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f087000b540 tx=0x7f0870003e50 comp rx=0 tx=0).stop 2024-04-23T14:48:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.342+0000 7f088c43c740 1 -- 172.21.15.83:0/498859946 shutdown_connections 2024-04-23T14:48:37.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.342+0000 7f088c43c740 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56165c3d9160 0x56165c3db580 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.342+0000 7f088c43c740 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56165c359a20 0x56165c366980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.342+0000 7f088c43c740 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56165c359230 0x56165c366440 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.342+0000 7f088c43c740 1 --2- 172.21.15.83:0/498859946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56165c34f350 0x56165c365f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.342+0000 7f088c43c740 1 -- 172.21.15.83:0/498859946 >> 172.21.15.83:0/498859946 conn(0x56165c34ad50 msgr2=0x56165c358ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.342+0000 7f0863e00640 0 client.25423 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:48:37.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.342+0000 7f088c43c740 1 -- 172.21.15.83:0/498859946 shutdown_connections 2024-04-23T14:48:37.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.342+0000 7f088c43c740 1 -- 172.21.15.83:0/498859946 wait complete. 2024-04-23T14:48:37.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.342+0000 7f088c43c740 20 client.25423 trim_cache size 0 max 16384 2024-04-23T14:48:37.346 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:48:37.347 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:48:37.347 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:48:37.347 INFO:teuthology.orchestra.run.smithi083.stdout: "a36634b2-6a33-46c8-a66a-7c86f57ac9fb": { 2024-04-23T14:48:37.347 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:48:37.347 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "a36634b2-6a33-46c8-a66a-7c86f57ac9fb", 2024-04-23T14:48:37.347 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:48:37.347 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:48:37.347 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:48:37.347 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:48:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.343+0000 7f0883e00640 1 -- 172.21.15.83:0/3794484527 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f08680a3ec0 msgr2=0x7f08680a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.343+0000 7f0883e00640 1 --2- 172.21.15.83:0/3794484527 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f08680a3ec0 0x7f08680a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x56165c34fff0 tx=0x7f087400b480 comp rx=0 tx=0).stop 2024-04-23T14:48:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.343+0000 7f0883e00640 1 -- 172.21.15.83:0/3794484527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088410c480 msgr2=0x7f088419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.343+0000 7f0883e00640 1 --2- 172.21.15.83:0/3794484527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088410c480 0x7f088419a760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f087c005e30 tx=0x7f087c02f700 comp rx=0 tx=0).stop 2024-04-23T14:48:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.343+0000 7f0883e00640 1 -- 172.21.15.83:0/3794484527 shutdown_connections 2024-04-23T14:48:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.343+0000 7f0883e00640 1 --2- 172.21.15.83:0/3794484527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f088410c480 0x7f088419a760 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.343+0000 7f0883e00640 1 --2- 172.21.15.83:0/3794484527 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f08680a3ec0 0x7f08680a6380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.343+0000 7f0883e00640 1 --2- 172.21.15.83:0/3794484527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0884103980 0x7f08841a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.343+0000 7f0883e00640 1 --2- 172.21.15.83:0/3794484527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0884103060 0x7f08841a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.343+0000 7f0883e00640 1 -- 172.21.15.83:0/3794484527 >> 172.21.15.83:0/3794484527 conn(0x7f08840fc880 msgr2=0x7f0884100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.343+0000 7f0883e00640 1 -- 172.21.15.83:0/3794484527 shutdown_connections 2024-04-23T14:48:37.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.343+0000 7f0883e00640 1 -- 172.21.15.83:0/3794484527 wait complete. 2024-04-23T14:48:37.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.353+0000 7f040be00640 1 -- 172.21.15.83:0/2539783991 <== mgr.15114 v2:172.21.15.83:6800/1667121447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0414062440 con 0x7f03f80a3eb0 2024-04-23T14:48:37.355 INFO:teuthology.orchestra.run.smithi083.stdout:Schedule set for path / 2024-04-23T14:48:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.356+0000 7f0413e00640 1 -- 172.21.15.83:0/2539783991 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f03f80a3eb0 msgr2=0x7f03f80a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.356+0000 7f0413e00640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f03f80a3eb0 0x7f03f80a6370 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f04040099a0 tx=0x7f0404009500 comp rx=0 tx=0).stop 2024-04-23T14:48:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.356+0000 7f0413e00640 1 -- 172.21.15.83:0/2539783991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0414103900 msgr2=0x7f04141a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.356+0000 7f0413e00640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0414103900 0x7f04141a0b70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00e9f0 tx=0x7f03fc00eeb0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.356+0000 7f0413e00640 1 -- 172.21.15.83:0/2539783991 shutdown_connections 2024-04-23T14:48:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.356+0000 7f0413e00640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f03f80a3eb0 0x7f03f80a6370 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.356+0000 7f0413e00640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f041410c400 0x7f041419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.356+0000 7f0413e00640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0414103900 0x7f04141a0b70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.356+0000 7f0413e00640 1 --2- 172.21.15.83:0/2539783991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0414102fe0 0x7f04141a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.356+0000 7f0413e00640 1 -- 172.21.15.83:0/2539783991 >> 172.21.15.83:0/2539783991 conn(0x7f04140fc7e0 msgr2=0x7f04140fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.356+0000 7f0413e00640 1 -- 172.21.15.83:0/2539783991 shutdown_connections 2024-04-23T14:48:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.356+0000 7f0413e00640 1 -- 172.21.15.83:0/2539783991 wait complete. 2024-04-23T14:48:37.360 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:37 smithi083 ceph-mon[27867]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 22 op/s 2024-04-23T14:48:37.360 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:37 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:48:37.360 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:37 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3887325504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:48:37.360 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:37 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:37.360 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:37 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:37.360 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:37 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:48:37.360 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:37 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:48:37.360 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:37 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:37.360 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:37 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/4259216960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:37.363 INFO:tasks.cephfs.filesystem:scrub status for tag:a36634b2-6a33-46c8-a66a-7c86f57ac9fb - {'path': '/', 'tag': 'a36634b2-6a33-46c8-a66a-7c86f57ac9fb', 'options': 'recursive,force'} 2024-04-23T14:48:37.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-23T14:48:37.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:37 smithi203 ceph-mon[32800]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 22 op/s 2024-04-23T14:48:37.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:37 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:48:37.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:37 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3887325504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:48:37.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:37 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:37.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:37 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:37.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:37 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:48:37.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:37 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:48:37.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:37 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:37.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:37 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/4259216960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:37.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:37 smithi173 ceph-mon[32443]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 22 op/s 2024-04-23T14:48:37.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:37 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:48:37.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:37 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3887325504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:48:37.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:37 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:37.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:37 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:37.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:37 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:48:37.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:37 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:48:37.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:37 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:48:37.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:37 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/4259216960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:48:37.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.592+0000 7f7811200640 1 -- 172.21.15.83:0/2997031857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780c074430 msgr2=0x7f780c074810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.592+0000 7f7811200640 1 --2- 172.21.15.83:0/2997031857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780c074430 0x7f780c074810 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009900 tx=0x7f77fc02a480 comp rx=0 tx=0).stop 2024-04-23T14:48:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.593+0000 7f7811200640 1 -- 172.21.15.83:0/2997031857 shutdown_connections 2024-04-23T14:48:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.593+0000 7f7811200640 1 --2- 172.21.15.83:0/2997031857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f780c10e640 0x7f780c110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.593+0000 7f7811200640 1 --2- 172.21.15.83:0/2997031857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f780c074d50 0x7f780c10e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.593+0000 7f7811200640 1 --2- 172.21.15.83:0/2997031857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780c074430 0x7f780c074810 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.593+0000 7f7811200640 1 -- 172.21.15.83:0/2997031857 >> 172.21.15.83:0/2997031857 conn(0x7f780c0fd950 msgr2=0x7f780c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.593+0000 7f7811200640 1 -- 172.21.15.83:0/2997031857 shutdown_connections 2024-04-23T14:48:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.593+0000 7f7811200640 1 -- 172.21.15.83:0/2997031857 wait complete. 2024-04-23T14:48:37.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.594+0000 7f7811200640 1 Processor -- start 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.594+0000 7f7811200640 1 -- start start 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.594+0000 7f7811200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780c074430 0x7f780c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.594+0000 7f7811200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f780c074d50 0x7f780c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.594+0000 7f7811200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f780c10e640 0x7f780c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.594+0000 7f7811200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f780c114420 con 0x7f780c074430 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.594+0000 7f7811200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1142a0 con 0x7f780c074d50 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.594+0000 7f7811200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f780c1145a0 con 0x7f780c10e640 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.594+0000 7f780b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780c074430 0x7f780c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.595+0000 7f780b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780c074430 0x7f780c1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46234/0 (socket says 172.21.15.83:46234) 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.595+0000 7f780b400640 1 -- 172.21.15.83:0/3587259299 learned_addr learned my addr 172.21.15.83:0/3587259299 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.595+0000 7f7802a00640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f780c074d50 0x7f780c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.595+0000 7f780be00640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f780c10e640 0x7f780c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.595+0000 7f780b400640 1 -- 172.21.15.83:0/3587259299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f780c10e640 msgr2=0x7f780c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.595+0000 7f780b400640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f780c10e640 0x7f780c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.595+0000 7f780b400640 1 -- 172.21.15.83:0/3587259299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f780c074d50 msgr2=0x7f780c1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.595+0000 7f780b400640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f780c074d50 0x7f780c1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.595+0000 7f780b400640 1 -- 172.21.15.83:0/3587259299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780c19afe0 con 0x7f780c074430 2024-04-23T14:48:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.595+0000 7f780be00640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f780c10e640 0x7f780c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:48:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.595+0000 7f780b400640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780c074430 0x7f780c1a0660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f77fc004770 tx=0x7f77fc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.596+0000 7f7808c00640 1 -- 172.21.15.83:0/3587259299 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc03f070 con 0x7f780c074430 2024-04-23T14:48:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.596+0000 7f7808c00640 1 -- 172.21.15.83:0/3587259299 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77fc031ca0 con 0x7f780c074430 2024-04-23T14:48:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.596+0000 7f7808c00640 1 -- 172.21.15.83:0/3587259299 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc035030 con 0x7f780c074430 2024-04-23T14:48:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.596+0000 7f7811200640 1 -- 172.21.15.83:0/3587259299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780c19b270 con 0x7f780c074430 2024-04-23T14:48:37.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.596+0000 7f7811200640 1 -- 172.21.15.83:0/3587259299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780c1a7400 con 0x7f780c074430 2024-04-23T14:48:37.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.597+0000 7f7811200640 1 -- 172.21.15.83:0/3587259299 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780c075ae0 con 0x7f780c074430 2024-04-23T14:48:37.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.599+0000 7f7808c00640 1 -- 172.21.15.83:0/3587259299 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f77fc042070 con 0x7f780c074430 2024-04-23T14:48:37.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.601+0000 7f7808c00640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f77f00a3e30 0x7f77f00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:37.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.601+0000 7f7808c00640 1 -- 172.21.15.83:0/3587259299 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f77fc0f1000 con 0x7f780c074430 2024-04-23T14:48:37.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.601+0000 7f7802a00640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f77f00a3e30 0x7f77f00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:37.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.602+0000 7f7802a00640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f77f00a3e30 0x7f77f00a62f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f780c19bf70 tx=0x7f77f4005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:37.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.606+0000 7f7808c00640 1 -- 172.21.15.83:0/3587259299 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77fc0bd5f0 con 0x7f780c074430 2024-04-23T14:48:37.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.833+0000 7f7811200640 1 -- 172.21.15.83:0/3587259299 --> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f780c062440 con 0x7f77f00a3e30 2024-04-23T14:48:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.985+0000 7f7808c00640 1 -- 172.21.15.83:0/3587259299 <== mgr.15114 v2:172.21.15.83:6800/1667121447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f780c062440 con 0x7f77f00a3e30 2024-04-23T14:48:37.986 INFO:teuthology.orchestra.run.smithi083.stdout:Retention added to path / 2024-04-23T14:48:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.988+0000 7f7811200640 1 -- 172.21.15.83:0/3587259299 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f77f00a3e30 msgr2=0x7f77f00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.988+0000 7f7811200640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f77f00a3e30 0x7f77f00a62f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f780c19bf70 tx=0x7f77f4005f90 comp rx=0 tx=0).stop 2024-04-23T14:48:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.988+0000 7f7811200640 1 -- 172.21.15.83:0/3587259299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780c074430 msgr2=0x7f780c1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.988+0000 7f7811200640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780c074430 0x7f780c1a0660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f77fc004770 tx=0x7f77fc003c10 comp rx=0 tx=0).stop 2024-04-23T14:48:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.988+0000 7f7811200640 1 -- 172.21.15.83:0/3587259299 shutdown_connections 2024-04-23T14:48:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.988+0000 7f7811200640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f780c10e640 0x7f780c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.988+0000 7f7811200640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f780c074d50 0x7f780c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.988+0000 7f7811200640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f77f00a3e30 0x7f77f00a62f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.988+0000 7f7811200640 1 --2- 172.21.15.83:0/3587259299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f780c074430 0x7f780c1a0660 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.988+0000 7f7811200640 1 -- 172.21.15.83:0/3587259299 >> 172.21.15.83:0/3587259299 conn(0x7f780c0fd950 msgr2=0x7f780c0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.989+0000 7f7811200640 1 -- 172.21.15.83:0/3587259299 shutdown_connections 2024-04-23T14:48:37.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:37.989+0000 7f7811200640 1 -- 172.21.15.83:0/3587259299 wait complete. 2024-04-23T14:48:38.009 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-23T14:48:38.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.200+0000 7f6867000640 1 -- 172.21.15.83:0/2493781213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6860100e10 msgr2=0x7f68601011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:38.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.200+0000 7f6867000640 1 --2- 172.21.15.83:0/2493781213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6860100e10 0x7f68601011f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6850009860 tx=0x7f685002a340 comp rx=0 tx=0).stop 2024-04-23T14:48:38.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.200+0000 7f6867000640 1 -- 172.21.15.83:0/2493781213 shutdown_connections 2024-04-23T14:48:38.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.200+0000 7f6867000640 1 --2- 172.21.15.83:0/2493781213 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f686010e9c0 0x7f6860110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.200+0000 7f6867000640 1 --2- 172.21.15.83:0/2493781213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6860101730 0x7f686010e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.200+0000 7f6867000640 1 --2- 172.21.15.83:0/2493781213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6860100e10 0x7f68601011f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.200+0000 7f6867000640 1 -- 172.21.15.83:0/2493781213 >> 172.21.15.83:0/2493781213 conn(0x7f6860078010 msgr2=0x7f6860078420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:38.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.201+0000 7f6867000640 1 -- 172.21.15.83:0/2493781213 shutdown_connections 2024-04-23T14:48:38.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.201+0000 7f6867000640 1 -- 172.21.15.83:0/2493781213 wait complete. 2024-04-23T14:48:38.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.202+0000 7f6867000640 1 Processor -- start 2024-04-23T14:48:38.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.202+0000 7f6867000640 1 -- start start 2024-04-23T14:48:38.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.202+0000 7f6867000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6860100e10 0x7f68601a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:38.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.202+0000 7f6867000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6860101730 0x7f68601a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:38.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.202+0000 7f6867000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f686010e9c0 0x7f686019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:38.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.202+0000 7f6867000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6860114630 con 0x7f6860101730 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.202+0000 7f6867000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f68601144b0 con 0x7f686010e9c0 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.202+0000 7f6867000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f68601147b0 con 0x7f6860100e10 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.202+0000 7f685fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6860100e10 0x7f68601a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.202+0000 7f685f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6860101730 0x7f68601a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.202+0000 7f685fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6860100e10 0x7f68601a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50128/0 (socket says 172.21.15.83:50128) 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.203+0000 7f685fe00640 1 -- 172.21.15.83:0/3052809381 learned_addr learned my addr 172.21.15.83:0/3052809381 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.203+0000 7f6865000640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f686010e9c0 0x7f686019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.203+0000 7f685fe00640 1 -- 172.21.15.83:0/3052809381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f686010e9c0 msgr2=0x7f686019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.203+0000 7f685fe00640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f686010e9c0 0x7f686019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.203+0000 7f685fe00640 1 -- 172.21.15.83:0/3052809381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6860101730 msgr2=0x7f68601a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.203+0000 7f685fe00640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6860101730 0x7f68601a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.203+0000 7f685fe00640 1 -- 172.21.15.83:0/3052809381 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686019b170 con 0x7f6860100e10 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.203+0000 7f685fe00640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6860100e10 0x7f68601a07f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6850035040 tx=0x7f6850033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.204+0000 7f685cc00640 1 -- 172.21.15.83:0/3052809381 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850037040 con 0x7f6860100e10 2024-04-23T14:48:38.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.204+0000 7f685cc00640 1 -- 172.21.15.83:0/3052809381 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6850040070 con 0x7f6860100e10 2024-04-23T14:48:38.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.204+0000 7f6867000640 1 -- 172.21.15.83:0/3052809381 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686019b400 con 0x7f6860100e10 2024-04-23T14:48:38.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.204+0000 7f685cc00640 1 -- 172.21.15.83:0/3052809381 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68500039b0 con 0x7f6860100e10 2024-04-23T14:48:38.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.204+0000 7f6867000640 1 -- 172.21.15.83:0/3052809381 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68601a76b0 con 0x7f6860100e10 2024-04-23T14:48:38.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.205+0000 7f6867000640 1 -- 172.21.15.83:0/3052809381 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f686004fd90 con 0x7f6860100e10 2024-04-23T14:48:38.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.207+0000 7f685cc00640 1 -- 172.21.15.83:0/3052809381 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6850003b50 con 0x7f6860100e10 2024-04-23T14:48:38.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.208+0000 7f685cc00640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f68440a3ee0 0x7f68440a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:38.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.208+0000 7f685cc00640 1 -- 172.21.15.83:0/3052809381 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f68500f0910 con 0x7f6860100e10 2024-04-23T14:48:38.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.208+0000 7f685f400640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f68440a3ee0 0x7f68440a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:38.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.209+0000 7f685f400640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f68440a3ee0 0x7f68440a63a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6860102850 tx=0x7f684c009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:38.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.212+0000 7f685cc00640 1 -- 172.21.15.83:0/3052809381 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68500bce20 con 0x7f6860100e10 2024-04-23T14:48:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:38 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:48:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:38 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:48:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:38 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:48:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:38 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:48:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:38 smithi083 ceph-mon[27867]: from='client.25390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:38.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.439+0000 7f6867000640 1 -- 172.21.15.83:0/3052809381 --> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f6860062440 con 0x7f68440a3ee0 2024-04-23T14:48:38.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.450+0000 7f685cc00640 1 -- 172.21.15.83:0/3052809381 <== mgr.15114 v2:172.21.15.83:6800/1667121447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f6860062440 con 0x7f68440a3ee0 2024-04-23T14:48:38.451 INFO:teuthology.orchestra.run.smithi083.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-23T00:00:00", "created": "2024-04-23T14:48:37", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-23T14:48:38.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.453+0000 7f6867000640 1 -- 172.21.15.83:0/3052809381 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f68440a3ee0 msgr2=0x7f68440a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.453+0000 7f6867000640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f68440a3ee0 0x7f68440a63a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6860102850 tx=0x7f684c009260 comp rx=0 tx=0).stop 2024-04-23T14:48:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.453+0000 7f6867000640 1 -- 172.21.15.83:0/3052809381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6860100e10 msgr2=0x7f68601a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.453+0000 7f6867000640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6860100e10 0x7f68601a07f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6850035040 tx=0x7f6850033b60 comp rx=0 tx=0).stop 2024-04-23T14:48:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.453+0000 7f6867000640 1 -- 172.21.15.83:0/3052809381 shutdown_connections 2024-04-23T14:48:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.453+0000 7f6867000640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f686010e9c0 0x7f686019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.453+0000 7f6867000640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f68440a3ee0 0x7f68440a63a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.453+0000 7f6867000640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6860101730 0x7f68601a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.453+0000 7f6867000640 1 --2- 172.21.15.83:0/3052809381 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6860100e10 0x7f68601a07f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.453+0000 7f6867000640 1 -- 172.21.15.83:0/3052809381 >> 172.21.15.83:0/3052809381 conn(0x7f6860078010 msgr2=0x7f686010feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.453+0000 7f6867000640 1 -- 172.21.15.83:0/3052809381 shutdown_connections 2024-04-23T14:48:38.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.453+0000 7f6867000640 1 -- 172.21.15.83:0/3052809381 wait complete. 2024-04-23T14:48:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:38 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:48:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:38 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:48:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:38 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:48:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:38 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:48:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:38 smithi203 ceph-mon[32800]: from='client.25390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:38.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-23T14:48:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:38 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:48:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:38 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:48:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:38 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:48:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:38 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:48:38.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:38 smithi173 ceph-mon[32443]: from='client.25390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:38.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.671+0000 7fa886000640 1 -- 172.21.15.83:0/1413276777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880102280 msgr2=0x7fa880102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:38.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.671+0000 7fa886000640 1 --2- 172.21.15.83:0/1413276777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880102280 0x7fa880102660 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa870009ca0 tx=0x7fa87002a8c0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.672+0000 7fa886000640 1 -- 172.21.15.83:0/1413276777 shutdown_connections 2024-04-23T14:48:38.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.672+0000 7fa886000640 1 --2- 172.21.15.83:0/1413276777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa880107350 0x7fa88010ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.672+0000 7fa886000640 1 --2- 172.21.15.83:0/1413276777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa880102c30 0x7fa880106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.672+0000 7fa886000640 1 --2- 172.21.15.83:0/1413276777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880102280 0x7fa880102660 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.672+0000 7fa886000640 1 -- 172.21.15.83:0/1413276777 >> 172.21.15.83:0/1413276777 conn(0x7fa880079490 msgr2=0x7fa8800798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:38.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.672+0000 7fa886000640 1 -- 172.21.15.83:0/1413276777 shutdown_connections 2024-04-23T14:48:38.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.672+0000 7fa886000640 1 -- 172.21.15.83:0/1413276777 wait complete. 2024-04-23T14:48:38.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.673+0000 7fa886000640 1 Processor -- start 2024-04-23T14:48:38.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.673+0000 7fa886000640 1 -- start start 2024-04-23T14:48:38.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.673+0000 7fa886000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa880102280 0x7fa88019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:38.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.673+0000 7fa886000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880107350 0x7fa88019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:38.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.673+0000 7fa886000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa880198680 0x7fa880198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:38.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.673+0000 7fa886000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801123e0 con 0x7fa880107350 2024-04-23T14:48:38.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.673+0000 7fa886000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa880112260 con 0x7fa880198680 2024-04-23T14:48:38.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.673+0000 7fa886000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa880112560 con 0x7fa880102280 2024-04-23T14:48:38.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa876a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880107350 0x7fa88019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:38.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa87fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa880198680 0x7fa880198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:38.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa87fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa880198680 0x7fa880198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:44590/0 (socket says 172.21.15.83:44590) 2024-04-23T14:48:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa876a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880107350 0x7fa88019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46274/0 (socket says 172.21.15.83:46274) 2024-04-23T14:48:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa87fe00640 1 -- 172.21.15.83:0/1889334443 learned_addr learned my addr 172.21.15.83:0/1889334443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:48:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa87f400640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa880102280 0x7fa88019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa87fe00640 1 -- 172.21.15.83:0/1889334443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa880102280 msgr2=0x7fa88019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa87fe00640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa880102280 0x7fa88019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa87fe00640 1 -- 172.21.15.83:0/1889334443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880107350 msgr2=0x7fa88019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa87fe00640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880107350 0x7fa88019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa87fe00640 1 -- 172.21.15.83:0/1889334443 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa880199280 con 0x7fa880198680 2024-04-23T14:48:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa876a00640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880107350 0x7fa88019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:48:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.674+0000 7fa87fe00640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa880198680 0x7fa880198b00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa87800e940 tx=0x7fa87800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.675+0000 7fa87cc00640 1 -- 172.21.15.83:0/1889334443 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87800cd00 con 0x7fa880198680 2024-04-23T14:48:38.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.675+0000 7fa886000640 1 -- 172.21.15.83:0/1889334443 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8801a5310 con 0x7fa880198680 2024-04-23T14:48:38.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.675+0000 7fa87cc00640 1 -- 172.21.15.83:0/1889334443 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8780035f0 con 0x7fa880198680 2024-04-23T14:48:38.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.675+0000 7fa87cc00640 1 -- 172.21.15.83:0/1889334443 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87800cea0 con 0x7fa880198680 2024-04-23T14:48:38.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.675+0000 7fa886000640 1 -- 172.21.15.83:0/1889334443 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8801a5710 con 0x7fa880198680 2024-04-23T14:48:38.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.676+0000 7fa886000640 1 -- 172.21.15.83:0/1889334443 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa880103750 con 0x7fa880198680 2024-04-23T14:48:38.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.678+0000 7fa87cc00640 1 -- 172.21.15.83:0/1889334443 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa8780037a0 con 0x7fa880198680 2024-04-23T14:48:38.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.680+0000 7fa87cc00640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa8640a3ee0 0x7fa8640a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:48:38.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.680+0000 7fa87cc00640 1 -- 172.21.15.83:0/1889334443 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa8780d0cb0 con 0x7fa880198680 2024-04-23T14:48:38.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.680+0000 7fa87f400640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa8640a3ee0 0x7fa8640a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:48:38.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.680+0000 7fa87f400640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa8640a3ee0 0x7fa8640a63a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa870006010 tx=0x7fa870009360 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:48:38.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.684+0000 7fa87cc00640 1 -- 172.21.15.83:0/1889334443 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa87809d0b0 con 0x7fa880198680 2024-04-23T14:48:38.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.914+0000 7fa886000640 1 -- 172.21.15.83:0/1889334443 --> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa880102c30 con 0x7fa8640a3ee0 2024-04-23T14:48:38.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.928+0000 7fa87cc00640 1 -- 172.21.15.83:0/1889334443 <== mgr.15114 v2:172.21.15.83:6800/1667121447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa880102c30 con 0x7fa8640a3ee0 2024-04-23T14:48:38.929 INFO:teuthology.orchestra.run.smithi083.stdout:/ 1m 6m3h 2024-04-23T14:48:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.931+0000 7fa886000640 1 -- 172.21.15.83:0/1889334443 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa8640a3ee0 msgr2=0x7fa8640a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.931+0000 7fa886000640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa8640a3ee0 0x7fa8640a63a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa870006010 tx=0x7fa870009360 comp rx=0 tx=0).stop 2024-04-23T14:48:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.931+0000 7fa886000640 1 -- 172.21.15.83:0/1889334443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa880198680 msgr2=0x7fa880198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:48:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.931+0000 7fa886000640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa880198680 0x7fa880198b00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa87800e940 tx=0x7fa87800ee00 comp rx=0 tx=0).stop 2024-04-23T14:48:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.931+0000 7fa886000640 1 -- 172.21.15.83:0/1889334443 shutdown_connections 2024-04-23T14:48:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.931+0000 7fa886000640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa880198680 0x7fa880198b00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.931+0000 7fa886000640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa8640a3ee0 0x7fa8640a63a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.931+0000 7fa886000640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa880107350 0x7fa88019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.931+0000 7fa886000640 1 --2- 172.21.15.83:0/1889334443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa880102280 0x7fa88019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:48:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.931+0000 7fa886000640 1 -- 172.21.15.83:0/1889334443 >> 172.21.15.83:0/1889334443 conn(0x7fa880079490 msgr2=0x7fa8800ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:48:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.931+0000 7fa886000640 1 -- 172.21.15.83:0/1889334443 shutdown_connections 2024-04-23T14:48:38.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:48:38.931+0000 7fa886000640 1 -- 172.21.15.83:0/1889334443 wait complete. 2024-04-23T14:48:38.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-23T14:48:39.035 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-23T14:48:39.046 INFO:teuthology.run_tasks:Running task workunit... 2024-04-23T14:48:39.058 INFO:tasks.workunit:Pulling workunits from ref d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:48:39.058 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-23T14:48:39.058 INFO:tasks.workunit:timeout=3h 2024-04-23T14:48:39.058 INFO:tasks.workunit:cleanup=True 2024-04-23T14:48:39.059 DEBUG:teuthology.orchestra.run.smithi083:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-23T14:48:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-23T14:48:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-23T14:48:39.117 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-23T14:48:39.117 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T14:48:39.117 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-23T14:48:39.117 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-23 14:46:07.270119965 +0000 2024-04-23T14:48:39.117 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-23 14:46:07.270119965 +0000 2024-04-23T14:48:39.117 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-23 14:48:26.500703997 +0000 2024-04-23T14:48:39.117 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-23 14:46:07.270119965 +0000 2024-04-23T14:48:39.118 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-23T14:48:39.118 DEBUG:teuthology.orchestra.run.smithi083:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-23T14:48:39.220 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d44fddd910b5e94da0f2357af4be37d7bd495ae0 2024-04-23T14:48:39.280 INFO:tasks.workunit.client.0.smithi083.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-23T14:48:39.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:39 smithi203 ceph-mon[32800]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 798 B/s wr, 17 op/s 2024-04-23T14:48:39.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:39 smithi203 ceph-mon[32800]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:39.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:39 smithi203 ceph-mon[32800]: from='client.25435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:39.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:39 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:48:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:39 smithi083 ceph-mon[27867]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 798 B/s wr, 17 op/s 2024-04-23T14:48:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:39 smithi083 ceph-mon[27867]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:39 smithi083 ceph-mon[27867]: from='client.25435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:39.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:39 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:48:39.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:39 smithi173 ceph-mon[32443]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 798 B/s wr, 17 op/s 2024-04-23T14:48:39.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:39 smithi173 ceph-mon[32443]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:39.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:39 smithi173 ceph-mon[32443]: from='client.25435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:39.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:39 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:48:40.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:40 smithi203 ceph-mon[32800]: from='client.25457 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:40.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:40 smithi083 ceph-mon[27867]: from='client.25457 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:40.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:40 smithi173 ceph-mon[32443]: from='client.25457 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:48:41.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:41 smithi203 ceph-mon[32800]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 50 op/s 2024-04-23T14:48:41.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:41 smithi083 ceph-mon[27867]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 50 op/s 2024-04-23T14:48:41.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:41 smithi173 ceph-mon[32443]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 50 op/s 2024-04-23T14:48:43.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:43 smithi203 ceph-mon[32800]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-04-23T14:48:43.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:43 smithi083 ceph-mon[27867]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-04-23T14:48:43.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:43 smithi173 ceph-mon[32443]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-04-23T14:48:45.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:45 smithi203 ceph-mon[32800]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-04-23T14:48:45.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:45 smithi083 ceph-mon[27867]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-04-23T14:48:45.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:45 smithi173 ceph-mon[32443]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-04-23T14:48:47.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:47 smithi083 ceph-mon[27867]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 50 op/s 2024-04-23T14:48:47.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:47 smithi173 ceph-mon[32443]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 50 op/s 2024-04-23T14:48:47.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:47 smithi203 ceph-mon[32800]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 50 op/s 2024-04-23T14:48:49.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:49 smithi083 ceph-mon[27867]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-23T14:48:49.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:49 smithi173 ceph-mon[32443]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-23T14:48:49.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:49 smithi203 ceph-mon[32800]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-23T14:48:51.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:51 smithi083 ceph-mon[27867]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-23T14:48:51.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:51 smithi173 ceph-mon[32443]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-23T14:48:51.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:51 smithi203 ceph-mon[32800]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-23T14:48:53.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:53 smithi083 ceph-mon[27867]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-23T14:48:53.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:53 smithi173 ceph-mon[32443]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-23T14:48:53.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:53 smithi203 ceph-mon[32800]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-23T14:48:55.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:55 smithi083 ceph-mon[27867]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-23T14:48:55.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:55 smithi173 ceph-mon[32443]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-23T14:48:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:55 smithi203 ceph-mon[32800]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-23T14:48:57.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:57 smithi173 ceph-mon[32443]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-23T14:48:57.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:57 smithi203 ceph-mon[32800]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-23T14:48:57.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:57 smithi083 ceph-mon[27867]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-23T14:48:59.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:48:59 smithi173 ceph-mon[32443]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:48:59 smithi203 ceph-mon[32800]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:48:59.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:48:59 smithi083 ceph-mon[27867]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:49:01.268 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:49:01 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:49:01.008+0000 7f0a9f000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:49:01.268 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:49:01 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:49:01.008+0000 7f0a9f000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:49:01.268 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:49:01 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:49:01.008+0000 7f0a9f000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:49:01.268 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:49:01 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:49:01.008+0000 7f0a9f000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:49:01.268 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 14:49:01 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T14:49:01.008+0000 7f0a9f000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T14:49:01.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:01 smithi173 ceph-mon[32443]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:49:01.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:01 smithi203 ceph-mon[32800]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:49:01.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:01 smithi083 ceph-mon[27867]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:49:03.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:03 smithi173 ceph-mon[32443]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:49:03.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:03 smithi173 ceph-mon[32443]: mgrmap e25: x(active, since 34s), standbys: z, y 2024-04-23T14:49:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:03 smithi203 ceph-mon[32800]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:49:03.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:03 smithi203 ceph-mon[32800]: mgrmap e25: x(active, since 34s), standbys: z, y 2024-04-23T14:49:03.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:03 smithi083 ceph-mon[27867]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:49:03.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:03 smithi083 ceph-mon[27867]: mgrmap e25: x(active, since 34s), standbys: z, y 2024-04-23T14:49:05.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:05 smithi173 ceph-mon[32443]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:49:05.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:05 smithi203 ceph-mon[32800]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:49:05.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:05 smithi083 ceph-mon[27867]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:49:07.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:49:07.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:49:07.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:07 smithi173 ceph-mon[32443]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-23T14:49:07.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.582+0000 7f898a400640 1 -- 172.21.15.83:0/1963935493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984108f40 msgr2=0x7f89841153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:07.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.582+0000 7f898a400640 1 --2- 172.21.15.83:0/1963935493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984108f40 0x7f89841153b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f897c009cd0 tx=0x7f897c02a8f0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.582+0000 7f898a400640 1 -- 172.21.15.83:0/1963935493 shutdown_connections 2024-04-23T14:49:07.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.582+0000 7f898a400640 1 --2- 172.21.15.83:0/1963935493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8984108f40 0x7f89841153b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.582+0000 7f898a400640 1 --2- 172.21.15.83:0/1963935493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8984108580 0x7f8984108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.582+0000 7f898a400640 1 --2- 172.21.15.83:0/1963935493 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f898410d500 0x7f898410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.582+0000 7f898a400640 1 -- 172.21.15.83:0/1963935493 >> 172.21.15.83:0/1963935493 conn(0x7f8984077e20 msgr2=0x7f8984106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:07.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.582+0000 7f898a400640 1 -- 172.21.15.83:0/1963935493 shutdown_connections 2024-04-23T14:49:07.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.582+0000 7f898a400640 1 -- 172.21.15.83:0/1963935493 wait complete. 2024-04-23T14:49:07.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.583+0000 7f898a400640 1 Processor -- start 2024-04-23T14:49:07.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f898a400640 1 -- start start 2024-04-23T14:49:07.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f898a400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8984108580 0x7f89841a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f898a400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8984108f40 0x7f89841a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f898a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898410d500 0x7f898419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f898a400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8984118900 con 0x7f898410d500 2024-04-23T14:49:07.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f898a400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8984118780 con 0x7f8984108580 2024-04-23T14:49:07.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f898a400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8984118a80 con 0x7f8984108f40 2024-04-23T14:49:07.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f8989a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898410d500 0x7f898419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f8989000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8984108580 0x7f89841a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f8983e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8984108f40 0x7f89841a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f8989a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898410d500 0x7f898419eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52140/0 (socket says 172.21.15.83:52140) 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f8989000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8984108580 0x7f89841a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:51976/0 (socket says 172.21.15.83:51976) 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.584+0000 7f8989a00640 1 -- 172.21.15.83:0/994316918 learned_addr learned my addr 172.21.15.83:0/994316918 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.585+0000 7f8983e00640 1 -- 172.21.15.83:0/994316918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8984108580 msgr2=0x7f89841a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.585+0000 7f8983e00640 1 --2- 172.21.15.83:0/994316918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8984108580 0x7f89841a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.585+0000 7f8983e00640 1 -- 172.21.15.83:0/994316918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898410d500 msgr2=0x7f898419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.585+0000 7f8983e00640 1 --2- 172.21.15.83:0/994316918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898410d500 0x7f898419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.585+0000 7f8983e00640 1 -- 172.21.15.83:0/994316918 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898419f2a0 con 0x7f8984108f40 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.585+0000 7f8983e00640 1 --2- 172.21.15.83:0/994316918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8984108f40 0x7f89841a4e60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f897000e9f0 tx=0x7f897000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.585+0000 7f8981600640 1 -- 172.21.15.83:0/994316918 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897000cd80 con 0x7f8984108f40 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.585+0000 7f898a400640 1 -- 172.21.15.83:0/994316918 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898419f590 con 0x7f8984108f40 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.585+0000 7f8981600640 1 -- 172.21.15.83:0/994316918 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89700035f0 con 0x7f8984108f40 2024-04-23T14:49:07.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.585+0000 7f8981600640 1 -- 172.21.15.83:0/994316918 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970010670 con 0x7f8984108f40 2024-04-23T14:49:07.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.585+0000 7f898a400640 1 -- 172.21.15.83:0/994316918 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89841ab7e0 con 0x7f8984108f40 2024-04-23T14:49:07.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.589+0000 7f898e7b5740 1 Processor -- start 2024-04-23T14:49:07.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.590+0000 7f898e7b5740 1 -- start start 2024-04-23T14:49:07.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.590+0000 7f898e7b5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b24a30 0x5604d7b24e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.590+0000 7f898e7b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b25350 0x5604d7b2f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.590+0000 7f8981600640 1 -- 172.21.15.83:0/994316918 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f89700037a0 con 0x7f8984108f40 2024-04-23T14:49:07.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.590+0000 7f898e7b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b2fa20 0x5604d7b31e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.590+0000 7f898e7b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5604d7a8a1f0 con 0x5604d7b25350 2024-04-23T14:49:07.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.590+0000 7f898e7b5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5604d7a30350 con 0x5604d7b24a30 2024-04-23T14:49:07.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.590+0000 7f898e7b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5604d7a861b0 con 0x5604d7b2fa20 2024-04-23T14:49:07.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.591+0000 7f8989000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b25350 0x5604d7b2f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.591+0000 7f8989000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b25350 0x5604d7b2f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52146/0 (socket says 172.21.15.83:52146) 2024-04-23T14:49:07.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.591+0000 7f8989000640 1 -- 172.21.15.83:0/1313573598 learned_addr learned my addr 172.21.15.83:0/1313573598 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:07.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.591+0000 7f8983e00640 1 --2- 172.21.15.83:0/1313573598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b2fa20 0x5604d7b31e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.591+0000 7f8989a00640 1 --2- 172.21.15.83:0/1313573598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b24a30 0x5604d7b24e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.591+0000 7f8989000640 1 -- 172.21.15.83:0/1313573598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b2fa20 msgr2=0x5604d7b31e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:07.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.591+0000 7f8989000640 1 --2- 172.21.15.83:0/1313573598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b2fa20 0x5604d7b31e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.591+0000 7f8989000640 1 -- 172.21.15.83:0/1313573598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b24a30 msgr2=0x5604d7b24e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:07.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.591+0000 7f8989000640 1 --2- 172.21.15.83:0/1313573598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b24a30 0x5604d7b24e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.591+0000 7f8989000640 1 -- 172.21.15.83:0/1313573598 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d7a84f50 con 0x5604d7b25350 2024-04-23T14:49:07.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.591+0000 7f8989a00640 1 --2- 172.21.15.83:0/1313573598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b24a30 0x5604d7b24e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:07.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.591+0000 7f8989000640 1 --2- 172.21.15.83:0/1313573598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b25350 0x5604d7b2f4e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8974009c80 tx=0x7f8974011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5325de7ee9cea999 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:07.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.592+0000 7f897a000640 1 -- 172.21.15.83:0/1313573598 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897401e070 con 0x5604d7b25350 2024-04-23T14:49:07.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.592+0000 7f897a000640 1 -- 172.21.15.83:0/1313573598 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8974009e30 con 0x5604d7b25350 2024-04-23T14:49:07.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.592+0000 7f897a000640 1 -- 172.21.15.83:0/1313573598 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897400ea20 con 0x5604d7b25350 2024-04-23T14:49:07.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.592+0000 7f8981600640 1 --2- 172.21.15.83:0/994316918 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f89680a3ed0 0x7f89680a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.592+0000 7f8981600640 1 -- 172.21.15.83:0/994316918 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f89700d0d70 con 0x7f8984108f40 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.592+0000 7f8989a00640 1 --2- 172.21.15.83:0/994316918 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f89680a3ed0 0x7f89680a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.592+0000 7f898e7b5740 1 -- 172.21.15.83:0/1313573598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b25350 msgr2=0x5604d7b2f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.592+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1313573598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b25350 0x5604d7b2f4e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8974009c80 tx=0x7f8974011810 comp rx=0 tx=0).stop 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.593+0000 7f898e7b5740 1 -- 172.21.15.83:0/1313573598 shutdown_connections 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.593+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1313573598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b2fa20 0x5604d7b31e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.593+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1313573598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b25350 0x5604d7b2f4e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.593+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1313573598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b24a30 0x5604d7b24e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.593+0000 7f898e7b5740 1 -- 172.21.15.83:0/1313573598 >> 172.21.15.83:0/1313573598 conn(0x5604d7b20d50 msgr2=0x5604d7b21130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.593+0000 7f898e7b5740 1 -- 172.21.15.83:0/1313573598 shutdown_connections 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.593+0000 7f898e7b5740 1 -- 172.21.15.83:0/1313573598 wait complete. 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.593+0000 7f8989a00640 1 --2- 172.21.15.83:0/994316918 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f89680a3ed0 0x7f89680a6390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f897c006010 tx=0x7f897c005d80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.593+0000 7f898e7b5740 1 -- start start 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f898e7b5740 1 Processor -- start 2024-04-23T14:49:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f898e7b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b24a30 0x5604d7b2c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f898e7b5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b25350 0x5604d7b2cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f898e7b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b2d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f898e7b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5604d7b32810 con 0x5604d7b2fa20 2024-04-23T14:49:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f898e7b5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5604d7a43690 con 0x5604d7b25350 2024-04-23T14:49:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f898e7b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5604d7b32990 con 0x5604d7b24a30 2024-04-23T14:49:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f898e7b5740 10 client.0 _ll_register_callbacks cb 0x5604d7a7f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f8989a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b2d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f8989a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b2d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52160/0 (socket says 172.21.15.83:52160) 2024-04-23T14:49:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f8989a00640 1 -- 172.21.15.83:0/3027091280 learned_addr learned my addr 172.21.15.83:0/3027091280 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f8983e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b25350 0x5604d7b2cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.594+0000 7f8989000640 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b24a30 0x5604d7b2c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.595+0000 7f8989a00640 1 -- 172.21.15.83:0/3027091280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b24a30 msgr2=0x5604d7b2c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.595+0000 7f8989a00640 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b24a30 0x5604d7b2c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.595+0000 7f8989a00640 1 -- 172.21.15.83:0/3027091280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b25350 msgr2=0x5604d7b2cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.595+0000 7f8989a00640 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b25350 0x5604d7b2cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.595+0000 7f8989a00640 1 -- 172.21.15.83:0/3027091280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d7a42920 con 0x5604d7b2fa20 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.595+0000 7f8983e00640 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b25350 0x5604d7b2cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.595+0000 7f8989a00640 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b2d400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f897c003ae0 tx=0x7f897c02a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24af1003afcd1470 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.595+0000 7f896ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.595+0000 7f896ea00640 1 -- 172.21.15.83:0/3027091280 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c030930 con 0x5604d7b2fa20 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f898e7b5740 1 -- 172.21.15.83:0/3027091280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8974009e30 con 0x5604d7b2fa20 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f896ea00640 1 -- 172.21.15.83:0/3027091280 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f897c030ad0 con 0x5604d7b2fa20 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f896ea00640 1 -- 172.21.15.83:0/3027091280 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f897c030d10 con 0x5604d7b2fa20 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f898e7b5740 10 client.15372 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:07.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f898e7b5740 1 -- 172.21.15.83:0/3027091280 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5604d78667d0 con 0x5604d7b2fa20 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f896ea00640 1 -- 172.21.15.83:0/3027091280 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8974009e30 con 0x5604d7b2fa20 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f896ea00640 1 -- 172.21.15.83:0/3027091280 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f897c051070 con 0x5604d7b2fa20 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f898e7b5740 10 client.15372 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f898e7b5740 10 client.15372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f898e7b5740 20 client.15372 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f898e7b5740 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5604d7b3fd00 0x5604d7b42130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f898e7b5740 4 client.15372 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.596+0000 7f898e7b5740 1 -- 172.21.15.83:0/3027091280 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604d7a97c10 con 0x5604d7b3fd00 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.597+0000 7f8989000640 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5604d7b3fd00 0x5604d7b42130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.597+0000 7f8989000640 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5604d7b3fd00 0x5604d7b42130 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3c86b9c2708ecb server_cookie=8b70ead416d0b7db in_seq=0 out_seq=0 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.597+0000 7f896ea00640 10 client.15372 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.598+0000 7f896ea00640 1 -- 172.21.15.83:0/3027091280 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5604d7a97c10 con 0x5604d7b3fd00 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.598+0000 7f896ea00640 10 client.15372 handle_command_reply: tid=0 2024-04-23T14:49:07.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.598+0000 7f898e7b5740 1 client.15372 shutdown 2024-04-23T14:49:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.598+0000 7f898e7b5740 1 -- 172.21.15.83:0/3027091280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 msgr2=0x5604d7b2d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.598+0000 7f898e7b5740 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b2d400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f897c003ae0 tx=0x7f897c02a8f0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.599+0000 7f898e7b5740 1 -- 172.21.15.83:0/3027091280 shutdown_connections 2024-04-23T14:49:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.599+0000 7f898e7b5740 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5604d7b3fd00 0x5604d7b42130 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.599+0000 7f898e7b5740 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b2d400 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.599+0000 7f898e7b5740 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b25350 0x5604d7b2cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.599+0000 7f898e7b5740 1 --2- 172.21.15.83:0/3027091280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b24a30 0x5604d7b2c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.599+0000 7f898e7b5740 1 -- 172.21.15.83:0/3027091280 >> 172.21.15.83:0/3027091280 conn(0x5604d7b20d50 msgr2=0x5604d7b1ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.599+0000 7f896ea00640 0 client.15372 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.599+0000 7f898e7b5740 1 -- 172.21.15.83:0/3027091280 shutdown_connections 2024-04-23T14:49:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.599+0000 7f898e7b5740 1 -- 172.21.15.83:0/3027091280 wait complete. 2024-04-23T14:49:07.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.599+0000 7f898e7b5740 20 client.15372 trim_cache size 0 max 16384 2024-04-23T14:49:07.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.618+0000 7f898e7b5740 1 Processor -- start 2024-04-23T14:49:07.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.618+0000 7f898e7b5740 1 -- start start 2024-04-23T14:49:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.618+0000 7f898e7b5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b24a30 0x5604d7b3b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.618+0000 7f898e7b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b25350 0x5604d7b271d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.618+0000 7f898e7b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b47a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.618+0000 7f898e7b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89841b6980 con 0x5604d7b2fa20 2024-04-23T14:49:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.618+0000 7f898e7b5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f89841b4250 con 0x5604d7b24a30 2024-04-23T14:49:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.618+0000 7f898e7b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f89841b20a0 con 0x5604d7b25350 2024-04-23T14:49:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.619+0000 7f8989a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b47a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.619+0000 7f8989a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b47a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52176/0 (socket says 172.21.15.83:52176) 2024-04-23T14:49:07.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.619+0000 7f8989a00640 1 -- 172.21.15.83:0/1966567296 learned_addr learned my addr 172.21.15.83:0/1966567296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.619+0000 7f8989000640 1 --2- 172.21.15.83:0/1966567296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b24a30 0x5604d7b3b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.619+0000 7f8983e00640 1 --2- 172.21.15.83:0/1966567296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b25350 0x5604d7b271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.619+0000 7f8989a00640 1 -- 172.21.15.83:0/1966567296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b25350 msgr2=0x5604d7b271d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.619+0000 7f8989a00640 1 --2- 172.21.15.83:0/1966567296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b25350 0x5604d7b271d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.619+0000 7f8989a00640 1 -- 172.21.15.83:0/1966567296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b24a30 msgr2=0x5604d7b3b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.619+0000 7f8989a00640 1 --2- 172.21.15.83:0/1966567296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b24a30 0x5604d7b3b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.619+0000 7f8989a00640 1 -- 172.21.15.83:0/1966567296 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d779c3f0 con 0x5604d7b2fa20 2024-04-23T14:49:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.619+0000 7f8989a00640 1 --2- 172.21.15.83:0/1966567296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b47a00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f897c005e30 tx=0x7f897c003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28b9ae86ca232fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.620+0000 7f8979600640 1 -- 172.21.15.83:0/1966567296 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c030eb0 con 0x5604d7b2fa20 2024-04-23T14:49:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.620+0000 7f8979600640 1 -- 172.21.15.83:0/1966567296 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897c036050 con 0x5604d7b2fa20 2024-04-23T14:49:07.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.620+0000 7f8979600640 1 -- 172.21.15.83:0/1966567296 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c0043f0 con 0x5604d7b2fa20 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.620+0000 7f898e7b5740 1 -- 172.21.15.83:0/1966567296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 msgr2=0x5604d7b47a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.620+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1966567296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b47a00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f897c005e30 tx=0x7f897c003b10 comp rx=0 tx=0).stop 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.620+0000 7f898e7b5740 1 -- 172.21.15.83:0/1966567296 shutdown_connections 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.620+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1966567296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b47a00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.620+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1966567296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b25350 0x5604d7b271d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.620+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1966567296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b24a30 0x5604d7b3b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.620+0000 7f898e7b5740 1 -- 172.21.15.83:0/1966567296 >> 172.21.15.83:0/1966567296 conn(0x5604d7b20d50 msgr2=0x5604d7b2ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.621+0000 7f898e7b5740 1 -- 172.21.15.83:0/1966567296 shutdown_connections 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.621+0000 7f898e7b5740 1 -- 172.21.15.83:0/1966567296 wait complete. 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.621+0000 7f898e7b5740 1 -- start start 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.621+0000 7f898e7b5740 1 Processor -- start 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f898e7b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b24a30 0x5604d7b38060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f898e7b5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b25350 0x5604d7b333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f898e7b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b33970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f898e7b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f89841b2710 con 0x5604d7b2fa20 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f898e7b5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f89841b3b80 con 0x5604d7b25350 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f898e7b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f89841b2de0 con 0x5604d7b24a30 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f898e7b5740 10 client.0 _ll_register_callbacks cb 0x7f8974005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f8989000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b24a30 0x5604d7b38060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f8989a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b33970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f8989000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b24a30 0x5604d7b38060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42180/0 (socket says 172.21.15.83:42180) 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f8983e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b25350 0x5604d7b333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f8989000640 1 -- 172.21.15.83:0/1575868113 learned_addr learned my addr 172.21.15.83:0/1575868113 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f8989000640 1 -- 172.21.15.83:0/1575868113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b25350 msgr2=0x5604d7b333d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f8989000640 1 --2- 172.21.15.83:0/1575868113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b25350 0x5604d7b333d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f8989000640 1 -- 172.21.15.83:0/1575868113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 msgr2=0x5604d7b33970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:07.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f8989000640 1 --2- 172.21.15.83:0/1575868113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b33970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.622+0000 7f8989000640 1 -- 172.21.15.83:0/1575868113 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d775c0f0 con 0x5604d7b24a30 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.623+0000 7f8989000640 1 --2- 172.21.15.83:0/1575868113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b24a30 0x5604d7b38060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8974009c80 tx=0x7f897400e660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a7b60f72a5f084 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.623+0000 7f896ea00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.623+0000 7f896ea00640 1 -- 172.21.15.83:0/1575868113 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897401e040 con 0x5604d7b24a30 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.623+0000 7f896ea00640 1 -- 172.21.15.83:0/1575868113 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f897402cb50 con 0x5604d7b24a30 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.623+0000 7f898e7b5740 1 -- 172.21.15.83:0/1575868113 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f897c036050 con 0x5604d7b24a30 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.623+0000 7f896ea00640 1 -- 172.21.15.83:0/1575868113 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f897403b050 con 0x5604d7b24a30 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.623+0000 7f898e7b5740 10 client.25465 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.623+0000 7f898e7b5740 1 -- 172.21.15.83:0/1575868113 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5604d7a4f810 con 0x5604d7b24a30 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.624+0000 7f896ea00640 1 -- 172.21.15.83:0/1575868113 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897c036050 con 0x5604d7b24a30 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.624+0000 7f896ea00640 1 -- 172.21.15.83:0/1575868113 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f897402c730 con 0x5604d7b24a30 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.624+0000 7f898e7b5740 10 client.25465 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.624+0000 7f898e7b5740 10 client.25465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:07.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.624+0000 7f898e7b5740 20 client.25465 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.624+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1575868113 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5604d7ba6580 0x5604d7ba8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.624+0000 7f898e7b5740 4 client.25465 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:49:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.624+0000 7f898e7b5740 1 -- 172.21.15.83:0/1575868113 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5604d7a91a80 con 0x5604d7ba6580 2024-04-23T14:49:07.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.624+0000 7f8983e00640 1 --2- 172.21.15.83:0/1575868113 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5604d7ba6580 0x5604d7ba8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f8983e00640 1 --2- 172.21.15.83:0/1575868113 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5604d7ba6580 0x5604d7ba8970 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f5df4612536ea83 server_cookie=2c0f3d4888c7d67 in_seq=0 out_seq=0 2024-04-23T14:49:07.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f896ea00640 10 client.25465 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:07.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f896ea00640 1 -- 172.21.15.83:0/1575868113 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5604d7a91a80 con 0x5604d7ba6580 2024-04-23T14:49:07.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f896ea00640 10 client.25465 handle_command_reply: tid=0 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f898e7b5740 1 client.25465 shutdown 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f898e7b5740 1 -- 172.21.15.83:0/1575868113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b24a30 msgr2=0x5604d7b38060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1575868113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b24a30 0x5604d7b38060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8974009c80 tx=0x7f897400e660 comp rx=0 tx=0).stop 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f898e7b5740 1 -- 172.21.15.83:0/1575868113 shutdown_connections 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1575868113 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5604d7ba6580 0x5604d7ba8970 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1575868113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5604d7b2fa20 0x5604d7b33970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1575868113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5604d7b25350 0x5604d7b333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f896ea00640 0 client.25465 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f898e7b5740 1 --2- 172.21.15.83:0/1575868113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604d7b24a30 0x5604d7b38060 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f898e7b5740 1 -- 172.21.15.83:0/1575868113 >> 172.21.15.83:0/1575868113 conn(0x5604d7b20d50 msgr2=0x5604d7b2c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f898e7b5740 1 -- 172.21.15.83:0/1575868113 shutdown_connections 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.625+0000 7f898e7b5740 1 -- 172.21.15.83:0/1575868113 wait complete. 2024-04-23T14:49:07.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898e7b5740 20 client.25465 trim_cache size 0 max 16384 2024-04-23T14:49:07.632 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:49:07.632 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:49:07.632 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:49:07.632 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:49:07.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898a400640 1 -- 172.21.15.83:0/994316918 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f89680a3ed0 msgr2=0x7f89680a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:07.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898a400640 1 --2- 172.21.15.83:0/994316918 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f89680a3ed0 0x7f89680a6390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f897c006010 tx=0x7f897c005d80 comp rx=0 tx=0).stop 2024-04-23T14:49:07.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898a400640 1 -- 172.21.15.83:0/994316918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8984108f40 msgr2=0x7f89841a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:07.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898a400640 1 --2- 172.21.15.83:0/994316918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8984108f40 0x7f89841a4e60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f897000e9f0 tx=0x7f897000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898a400640 1 -- 172.21.15.83:0/994316918 shutdown_connections 2024-04-23T14:49:07.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898a400640 1 --2- 172.21.15.83:0/994316918 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f89680a3ed0 0x7f89680a6390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898a400640 1 --2- 172.21.15.83:0/994316918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f898410d500 0x7f898419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898a400640 1 --2- 172.21.15.83:0/994316918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8984108f40 0x7f89841a4e60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898a400640 1 --2- 172.21.15.83:0/994316918 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8984108580 0x7f89841a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898a400640 1 -- 172.21.15.83:0/994316918 >> 172.21.15.83:0/994316918 conn(0x7f8984077e20 msgr2=0x7f8984078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:07.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898a400640 1 -- 172.21.15.83:0/994316918 shutdown_connections 2024-04-23T14:49:07.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.626+0000 7f898a400640 1 -- 172.21.15.83:0/994316918 wait complete. 2024-04-23T14:49:07.643 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:07 smithi083 ceph-mon[27867]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-23T14:49:07.645 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:49:07.645 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:49:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:07 smithi203 ceph-mon[32800]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-23T14:49:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.836+0000 7ffbe5000640 1 -- 172.21.15.83:0/3886365706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbe010af50 msgr2=0x7ffbe010d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:07.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.836+0000 7ffbe5000640 1 --2- 172.21.15.83:0/3886365706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbe010af50 0x7ffbe010d340 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009ca0 tx=0x7ffbd802a8c0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.836+0000 7ffbe5000640 1 -- 172.21.15.83:0/3886365706 shutdown_connections 2024-04-23T14:49:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.836+0000 7ffbe5000640 1 --2- 172.21.15.83:0/3886365706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbe010af50 0x7ffbe010d340 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.836+0000 7ffbe5000640 1 --2- 172.21.15.83:0/3886365706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe0100e80 0x7ffbe0105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.836+0000 7ffbe5000640 1 --2- 172.21.15.83:0/3886365706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbe00684b0 0x7ffbe0104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.836+0000 7ffbe5000640 1 -- 172.21.15.83:0/3886365706 >> 172.21.15.83:0/3886365706 conn(0x7ffbe0076c80 msgr2=0x7ffbe0077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.837+0000 7ffbe5000640 1 -- 172.21.15.83:0/3886365706 shutdown_connections 2024-04-23T14:49:07.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.837+0000 7ffbe5000640 1 -- 172.21.15.83:0/3886365706 wait complete. 2024-04-23T14:49:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.838+0000 7ffbe5000640 1 Processor -- start 2024-04-23T14:49:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.838+0000 7ffbe5000640 1 -- start start 2024-04-23T14:49:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.838+0000 7ffbe5000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbe00684b0 0x7ffbe01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.838+0000 7ffbe5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbe0100e80 0x7ffbe01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.838+0000 7ffbe5000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe010af50 0x7ffbe0109b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.838+0000 7ffbe5000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe01121f0 con 0x7ffbe0100e80 2024-04-23T14:49:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.838+0000 7ffbe5000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe0112070 con 0x7ffbe010af50 2024-04-23T14:49:07.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.838+0000 7ffbe5000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe0112370 con 0x7ffbe00684b0 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.838+0000 7ffbdf400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbe00684b0 0x7ffbe01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.838+0000 7ffbdf400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbe00684b0 0x7ffbe01a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42198/0 (socket says 172.21.15.83:42198) 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.838+0000 7ffbdf400640 1 -- 172.21.15.83:0/3214518350 learned_addr learned my addr 172.21.15.83:0/3214518350 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.838+0000 7ffbdfe00640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe010af50 0x7ffbe0109b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.839+0000 7ffbdea00640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbe0100e80 0x7ffbe01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.839+0000 7ffbdf400640 1 -- 172.21.15.83:0/3214518350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe010af50 msgr2=0x7ffbe0109b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.839+0000 7ffbdf400640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe010af50 0x7ffbe0109b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.839+0000 7ffbdf400640 1 -- 172.21.15.83:0/3214518350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbe0100e80 msgr2=0x7ffbe01a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.839+0000 7ffbdf400640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbe0100e80 0x7ffbe01a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.839+0000 7ffbdf400640 1 -- 172.21.15.83:0/3214518350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe010a410 con 0x7ffbe00684b0 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.839+0000 7ffbdf400640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbe00684b0 0x7ffbe01a0610 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000e9f0 tx=0x7ffbd000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.840+0000 7ffbd7e00640 1 -- 172.21.15.83:0/3214518350 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd000cd80 con 0x7ffbe00684b0 2024-04-23T14:49:07.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.840+0000 7ffbd7e00640 1 -- 172.21.15.83:0/3214518350 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd0010910 con 0x7ffbe00684b0 2024-04-23T14:49:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.840+0000 7ffbe5000640 1 -- 172.21.15.83:0/3214518350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe010a700 con 0x7ffbe00684b0 2024-04-23T14:49:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.840+0000 7ffbd7e00640 1 -- 172.21.15.83:0/3214518350 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0018990 con 0x7ffbe00684b0 2024-04-23T14:49:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.840+0000 7ffbe5000640 1 -- 172.21.15.83:0/3214518350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe01a7590 con 0x7ffbe00684b0 2024-04-23T14:49:07.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.841+0000 7ffbe5000640 1 -- 172.21.15.83:0/3214518350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe0101760 con 0x7ffbe00684b0 2024-04-23T14:49:07.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.843+0000 7ffbd7e00640 1 -- 172.21.15.83:0/3214518350 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7ffbd0010ac0 con 0x7ffbe00684b0 2024-04-23T14:49:07.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.844+0000 7ffbd7e00640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ffbc40a3ed0 0x7ffbc40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:07.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.845+0000 7ffbd7e00640 1 -- 172.21.15.83:0/3214518350 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ffbd00d1070 con 0x7ffbe00684b0 2024-04-23T14:49:07.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.845+0000 7ffbdea00640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ffbc40a3ed0 0x7ffbc40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:07.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.845+0000 7ffbdea00640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ffbc40a3ed0 0x7ffbc40a6390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbe019bf20 tx=0x7ffbcc005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:07.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:07.847+0000 7ffbd7e00640 1 -- 172.21.15.83:0/3214518350 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffbd009ce90 con 0x7ffbe00684b0 2024-04-23T14:49:08.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.087+0000 7ffbe5000640 1 -- 172.21.15.83:0/3214518350 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbe01a7760 con 0x7ffbe00684b0 2024-04-23T14:49:08.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.088+0000 7ffbd7e00640 1 -- 172.21.15.83:0/3214518350 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11863 (secure 0 0 0) 0x7ffbd00a1190 con 0x7ffbe00684b0 2024-04-23T14:49:08.090 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:49:08.090 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:48:27.525285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24532,24542]},"id":1}]} 2024-04-23T14:49:08.090 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-04-23T14:49:08.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.090+0000 7ffbe5000640 1 -- 172.21.15.83:0/3214518350 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ffbc40a3ed0 msgr2=0x7ffbc40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.090+0000 7ffbe5000640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ffbc40a3ed0 0x7ffbc40a6390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbe019bf20 tx=0x7ffbcc005f90 comp rx=0 tx=0).stop 2024-04-23T14:49:08.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.090+0000 7ffbe5000640 1 -- 172.21.15.83:0/3214518350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbe00684b0 msgr2=0x7ffbe01a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.090+0000 7ffbe5000640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbe00684b0 0x7ffbe01a0610 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000e9f0 tx=0x7ffbd000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.090+0000 7ffbe5000640 1 -- 172.21.15.83:0/3214518350 shutdown_connections 2024-04-23T14:49:08.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.090+0000 7ffbe5000640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbe010af50 0x7ffbe0109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.090+0000 7ffbe5000640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ffbc40a3ed0 0x7ffbc40a6390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.090+0000 7ffbe5000640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffbe0100e80 0x7ffbe01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.090+0000 7ffbe5000640 1 --2- 172.21.15.83:0/3214518350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbe00684b0 0x7ffbe01a0610 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.090+0000 7ffbe5000640 1 -- 172.21.15.83:0/3214518350 >> 172.21.15.83:0/3214518350 conn(0x7ffbe0076c80 msgr2=0x7ffbe00ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.091+0000 7ffbe5000640 1 -- 172.21.15.83:0/3214518350 shutdown_connections 2024-04-23T14:49:08.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.091+0000 7ffbe5000640 1 -- 172.21.15.83:0/3214518350 wait complete. 2024-04-23T14:49:08.112 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:49:08.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.319+0000 7fd099e00640 1 -- 172.21.15.83:0/2161537407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd094102110 msgr2=0x7fd0941024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.319+0000 7fd099e00640 1 --2- 172.21.15.83:0/2161537407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd094102110 0x7fd0941024f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd084009900 tx=0x7fd08402a410 comp rx=0 tx=0).stop 2024-04-23T14:49:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.319+0000 7fd099e00640 1 -- 172.21.15.83:0/2161537407 shutdown_connections 2024-04-23T14:49:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.319+0000 7fd099e00640 1 --2- 172.21.15.83:0/2161537407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd0941071e0 0x7fd09410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.319+0000 7fd099e00640 1 --2- 172.21.15.83:0/2161537407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd094102ac0 0x7fd094106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.319+0000 7fd099e00640 1 --2- 172.21.15.83:0/2161537407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd094102110 0x7fd0941024f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.319+0000 7fd099e00640 1 -- 172.21.15.83:0/2161537407 >> 172.21.15.83:0/2161537407 conn(0x7fd0940fd9d0 msgr2=0x7fd0940ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.319+0000 7fd099e00640 1 -- 172.21.15.83:0/2161537407 shutdown_connections 2024-04-23T14:49:08.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.319+0000 7fd099e00640 1 -- 172.21.15.83:0/2161537407 wait complete. 2024-04-23T14:49:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.320+0000 7fd099e00640 1 Processor -- start 2024-04-23T14:49:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.320+0000 7fd099e00640 1 -- start start 2024-04-23T14:49:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd099e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd094102110 0x7fd094074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd099e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0941071e0 0x7fd094074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd099e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd094074d80 0x7fd0941a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd099e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd0941122c0 con 0x7fd094074d80 2024-04-23T14:49:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd099e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd094112140 con 0x7fd094102110 2024-04-23T14:49:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd099e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd094112440 con 0x7fd0941071e0 2024-04-23T14:49:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd098a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd094102110 0x7fd094074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd098a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd094102110 0x7fd094074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:52060/0 (socket says 172.21.15.83:52060) 2024-04-23T14:49:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd098a00640 1 -- 172.21.15.83:0/733668518 learned_addr learned my addr 172.21.15.83:0/733668518 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd08be00640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0941071e0 0x7fd094074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd099400640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd094074d80 0x7fd0941a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd098a00640 1 -- 172.21.15.83:0/733668518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0941071e0 msgr2=0x7fd094074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd098a00640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0941071e0 0x7fd094074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd098a00640 1 -- 172.21.15.83:0/733668518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd094074d80 msgr2=0x7fd0941a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd098a00640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd094074d80 0x7fd0941a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.321+0000 7fd098a00640 1 -- 172.21.15.83:0/733668518 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0941a70b0 con 0x7fd094102110 2024-04-23T14:49:08.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.322+0000 7fd098a00640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd094102110 0x7fd094074300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd084004770 tx=0x7fd084003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.322+0000 7fd092000640 1 -- 172.21.15.83:0/733668518 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08403f070 con 0x7fd094102110 2024-04-23T14:49:08.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.322+0000 7fd092000640 1 -- 172.21.15.83:0/733668518 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd084031ca0 con 0x7fd094102110 2024-04-23T14:49:08.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.322+0000 7fd092000640 1 -- 172.21.15.83:0/733668518 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084035030 con 0x7fd094102110 2024-04-23T14:49:08.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.322+0000 7fd099e00640 1 -- 172.21.15.83:0/733668518 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0941a7340 con 0x7fd094102110 2024-04-23T14:49:08.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.322+0000 7fd099e00640 1 -- 172.21.15.83:0/733668518 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0941a7880 con 0x7fd094102110 2024-04-23T14:49:08.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.325+0000 7fd09e22a740 1 Processor -- start 2024-04-23T14:49:08.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.325+0000 7fd092000640 1 -- 172.21.15.83:0/733668518 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fd084042070 con 0x7fd094102110 2024-04-23T14:49:08.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.325+0000 7fd09e22a740 1 -- start start 2024-04-23T14:49:08.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.326+0000 7fd09e22a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae03b40 0x55eccae03f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.326+0000 7fd09e22a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae04460 0x55eccae0e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.326+0000 7fd09e22a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae0eb30 0x55eccae10f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.326+0000 7fd09e22a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eccad691d0 con 0x55eccae04460 2024-04-23T14:49:08.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.326+0000 7fd09e22a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eccad0f450 con 0x55eccae0eb30 2024-04-23T14:49:08.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.326+0000 7fd09e22a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55eccad65190 con 0x55eccae03b40 2024-04-23T14:49:08.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.326+0000 7fd099400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae03b40 0x55eccae03f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.326+0000 7fd098a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae04460 0x55eccae0e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.326+0000 7fd099400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae03b40 0x55eccae03f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42216/0 (socket says 172.21.15.83:42216) 2024-04-23T14:49:08.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.326+0000 7fd099400640 1 -- 172.21.15.83:0/1083452263 learned_addr learned my addr 172.21.15.83:0/1083452263 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.326+0000 7fd08be00640 1 --2- 172.21.15.83:0/1083452263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae0eb30 0x55eccae10f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd099400640 1 -- 172.21.15.83:0/1083452263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae0eb30 msgr2=0x55eccae10f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd099400640 1 --2- 172.21.15.83:0/1083452263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae0eb30 0x55eccae10f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd099400640 1 -- 172.21.15.83:0/1083452263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae04460 msgr2=0x55eccae0e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd099400640 1 --2- 172.21.15.83:0/1083452263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae04460 0x55eccae0e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd099400640 1 -- 172.21.15.83:0/1083452263 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eccadcc310 con 0x55eccae03b40 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd098a00640 1 --2- 172.21.15.83:0/1083452263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae04460 0x55eccae0e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd092000640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0780a3f50 0x7fd0780a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd092000640 1 -- 172.21.15.83:0/733668518 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd0840f1000 con 0x7fd094102110 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd099400640 1 --2- 172.21.15.83:0/1083452263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae03b40 0x55eccae03f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009f50 tx=0x7fd08c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df278c923e610f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd098a00640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0780a3f50 0x7fd0780a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd089c00640 1 -- 172.21.15.83:0/1083452263 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c004d20 con 0x55eccae03b40 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd089c00640 1 -- 172.21.15.83:0/1083452263 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08c004050 con 0x55eccae03b40 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.327+0000 7fd089c00640 1 -- 172.21.15.83:0/1083452263 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c005280 con 0x55eccae03b40 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.328+0000 7fd09e22a740 1 -- 172.21.15.83:0/1083452263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae03b40 msgr2=0x55eccae03f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.328+0000 7fd09e22a740 1 --2- 172.21.15.83:0/1083452263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae03b40 0x55eccae03f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009f50 tx=0x7fd08c0118d0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.328+0000 7fd098a00640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0780a3f50 0x7fd0780a6410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd0840ee580 tx=0x7fd0840f4ab0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.328+0000 7fd09e22a740 1 -- 172.21.15.83:0/1083452263 shutdown_connections 2024-04-23T14:49:08.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.328+0000 7fd09e22a740 1 --2- 172.21.15.83:0/1083452263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae0eb30 0x55eccae10f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.328+0000 7fd09e22a740 1 --2- 172.21.15.83:0/1083452263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae04460 0x55eccae0e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.328+0000 7fd09e22a740 1 --2- 172.21.15.83:0/1083452263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae03b40 0x55eccae03f20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.328+0000 7fd09e22a740 1 -- 172.21.15.83:0/1083452263 >> 172.21.15.83:0/1083452263 conn(0x55eccadffe60 msgr2=0x55eccae00240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.328+0000 7fd09e22a740 1 -- 172.21.15.83:0/1083452263 shutdown_connections 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.328+0000 7fd09e22a740 1 -- 172.21.15.83:0/1083452263 wait complete. 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.329+0000 7fd09e22a740 1 -- start start 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.329+0000 7fd09e22a740 1 Processor -- start 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.329+0000 7fd09e22a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 0x55eccae0e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.329+0000 7fd09e22a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae0b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.329+0000 7fd09e22a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae0eb30 0x55eccae0bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.329+0000 7fd09e22a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eccae119c0 con 0x55eccae03b40 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.329+0000 7fd09e22a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55eccad22810 con 0x55eccae04460 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.329+0000 7fd09e22a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55eccae11b40 con 0x55eccae0eb30 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.330+0000 7fd09e22a740 10 client.0 _ll_register_callbacks cb 0x55eccacf0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:08.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.330+0000 7fd099400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 0x55eccae0e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.330+0000 7fd099400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 0x55eccae0e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52234/0 (socket says 172.21.15.83:52234) 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.330+0000 7fd099400640 1 -- 172.21.15.83:0/3648146583 learned_addr learned my addr 172.21.15.83:0/3648146583 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.330+0000 7fd08be00640 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae0b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.330+0000 7fd099400640 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae0eb30 0x55eccae0bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.330+0000 7fd099400640 1 -- 172.21.15.83:0/3648146583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae0eb30 msgr2=0x55eccae0bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.330+0000 7fd099400640 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae0eb30 0x55eccae0bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.330+0000 7fd099400640 1 -- 172.21.15.83:0/3648146583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 msgr2=0x55eccae0b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.330+0000 7fd099400640 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae0b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.330+0000 7fd099400640 1 -- 172.21.15.83:0/3648146583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eccad333f0 con 0x55eccae03b40 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.330+0000 7fd08be00640 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae0b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.331+0000 7fd099400640 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 0x55eccae0e5a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009250 tx=0x7fd08c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d32327280548261 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.331+0000 7fd082000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.331+0000 7fd082000640 1 -- 172.21.15.83:0/3648146583 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c02b070 con 0x55eccae03b40 2024-04-23T14:49:08.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.331+0000 7fd082000640 1 -- 172.21.15.83:0/3648146583 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd08c025450 con 0x55eccae03b40 2024-04-23T14:49:08.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.331+0000 7fd09e22a740 1 -- 172.21.15.83:0/3648146583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd08c004050 con 0x55eccae03b40 2024-04-23T14:49:08.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.331+0000 7fd082000640 1 -- 172.21.15.83:0/3648146583 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd08c025650 con 0x55eccae03b40 2024-04-23T14:49:08.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.331+0000 7fd09e22a740 10 client.15396 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:08.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.331+0000 7fd09e22a740 1 -- 172.21.15.83:0/3648146583 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55eccaa7b3f0 con 0x55eccae03b40 2024-04-23T14:49:08.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.331+0000 7fd082000640 1 -- 172.21.15.83:0/3648146583 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08c004050 con 0x55eccae03b40 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.332+0000 7fd082000640 1 -- 172.21.15.83:0/3648146583 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd08c00f580 con 0x55eccae03b40 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.332+0000 7fd09e22a740 10 client.15396 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.332+0000 7fd09e22a740 10 client.15396 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.332+0000 7fd09e22a740 20 client.15396 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.332+0000 7fd09e22a740 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55eccae1ed40 0x55eccae21130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.332+0000 7fd09e22a740 4 client.15396 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.332+0000 7fd09e22a740 1 -- 172.21.15.83:0/3648146583 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eccad76bf0 con 0x55eccae1ed40 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.332+0000 7fd08be00640 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55eccae1ed40 0x55eccae21130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.333+0000 7fd08be00640 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55eccae1ed40 0x55eccae21130 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85fd77616f5dbbab server_cookie=bb37cba8ab5b2715 in_seq=0 out_seq=0 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.333+0000 7fd082000640 10 client.15396 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd082000640 1 -- 172.21.15.83:0/3648146583 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eccad76bf0 con 0x55eccae1ed40 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd082000640 10 client.15396 handle_command_reply: tid=0 2024-04-23T14:49:08.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd09e22a740 1 client.15396 shutdown 2024-04-23T14:49:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd09e22a740 1 -- 172.21.15.83:0/3648146583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 msgr2=0x55eccae0e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd09e22a740 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 0x55eccae0e5a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009250 tx=0x7fd08c0096c0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd09e22a740 1 -- 172.21.15.83:0/3648146583 shutdown_connections 2024-04-23T14:49:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd09e22a740 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55eccae1ed40 0x55eccae21130 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd09e22a740 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae0eb30 0x55eccae0bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd09e22a740 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae0b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd09e22a740 1 --2- 172.21.15.83:0/3648146583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 0x55eccae0e5a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd082000640 0 client.15396 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:49:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd09e22a740 1 -- 172.21.15.83:0/3648146583 >> 172.21.15.83:0/3648146583 conn(0x55eccadffe60 msgr2=0x55eccae05c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd09e22a740 1 -- 172.21.15.83:0/3648146583 shutdown_connections 2024-04-23T14:49:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd09e22a740 1 -- 172.21.15.83:0/3648146583 wait complete. 2024-04-23T14:49:08.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.334+0000 7fd09e22a740 20 client.15396 trim_cache size 0 max 16384 2024-04-23T14:49:08.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.349+0000 7fd09e22a740 1 Processor -- start 2024-04-23T14:49:08.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.349+0000 7fd09e22a740 1 -- start start 2024-04-23T14:49:08.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.349+0000 7fd09e22a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccadffe60 0x55eccadfedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.349+0000 7fd09e22a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae03b40 0x55eccae0e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.349+0000 7fd09e22a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae26900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.349+0000 7fd09e22a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd0941b2670 con 0x55eccadffe60 2024-04-23T14:49:08.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.349+0000 7fd09e22a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd0941aff40 con 0x55eccae04460 2024-04-23T14:49:08.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.349+0000 7fd09e22a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd0941add90 con 0x55eccae03b40 2024-04-23T14:49:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.350+0000 7fd099400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccadffe60 0x55eccadfedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.350+0000 7fd099400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccadffe60 0x55eccadfedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52250/0 (socket says 172.21.15.83:52250) 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.350+0000 7fd099400640 1 -- 172.21.15.83:0/2089903759 learned_addr learned my addr 172.21.15.83:0/2089903759 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.350+0000 7fd08be00640 1 --2- 172.21.15.83:0/2089903759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae03b40 0x55eccae0e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.350+0000 7fd099400640 1 --2- 172.21.15.83:0/2089903759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae26900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.350+0000 7fd099400640 1 -- 172.21.15.83:0/2089903759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae03b40 msgr2=0x55eccae0e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.350+0000 7fd099400640 1 --2- 172.21.15.83:0/2089903759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae03b40 0x55eccae0e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.350+0000 7fd099400640 1 -- 172.21.15.83:0/2089903759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 msgr2=0x55eccae26900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.350+0000 7fd099400640 1 --2- 172.21.15.83:0/2089903759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae26900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.350+0000 7fd099400640 1 -- 172.21.15.83:0/2089903759 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eccab457d0 con 0x55eccadffe60 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.350+0000 7fd099400640 1 --2- 172.21.15.83:0/2089903759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccadffe60 0x55eccadfedd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009d60 tx=0x7fd08c0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd0bd57bb3bd628 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.351+0000 7fd089200640 1 -- 172.21.15.83:0/2089903759 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c02b070 con 0x55eccadffe60 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.351+0000 7fd089200640 1 -- 172.21.15.83:0/2089903759 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08c00fbe0 con 0x55eccadffe60 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.351+0000 7fd089200640 1 -- 172.21.15.83:0/2089903759 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c025a50 con 0x55eccadffe60 2024-04-23T14:49:08.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.351+0000 7fd09e22a740 1 -- 172.21.15.83:0/2089903759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccadffe60 msgr2=0x55eccadfedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.351+0000 7fd09e22a740 1 --2- 172.21.15.83:0/2089903759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccadffe60 0x55eccadfedd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009d60 tx=0x7fd08c0021f0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.351+0000 7fd09e22a740 1 -- 172.21.15.83:0/2089903759 shutdown_connections 2024-04-23T14:49:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.351+0000 7fd09e22a740 1 --2- 172.21.15.83:0/2089903759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae26900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.351+0000 7fd09e22a740 1 --2- 172.21.15.83:0/2089903759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccae03b40 0x55eccae0e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.351+0000 7fd09e22a740 1 --2- 172.21.15.83:0/2089903759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccadffe60 0x55eccadfedd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.351+0000 7fd09e22a740 1 -- 172.21.15.83:0/2089903759 >> 172.21.15.83:0/2089903759 conn(0x55eccae11240 msgr2=0x55eccadfe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.351+0000 7fd09e22a740 1 -- 172.21.15.83:0/2089903759 shutdown_connections 2024-04-23T14:49:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.352+0000 7fd09e22a740 1 -- 172.21.15.83:0/2089903759 wait complete. 2024-04-23T14:49:08.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.352+0000 7fd09e22a740 1 -- start start 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.352+0000 7fd09e22a740 1 Processor -- start 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.352+0000 7fd09e22a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccadffe60 0x55eccae17150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.352+0000 7fd09e22a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 0x55eccae0f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.352+0000 7fd09e22a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae0fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.352+0000 7fd09e22a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd0941ae400 con 0x55eccae03b40 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.352+0000 7fd09e22a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd0941af870 con 0x55eccae04460 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.352+0000 7fd09e22a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd0941aead0 con 0x55eccadffe60 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.353+0000 7fd09e22a740 10 client.0 _ll_register_callbacks cb 0x7fd0840f4980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.353+0000 7fd08be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 0x55eccae0f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.353+0000 7fd099400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccadffe60 0x55eccae17150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.353+0000 7fd099400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccadffe60 0x55eccae17150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42246/0 (socket says 172.21.15.83:42246) 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.353+0000 7fd099400640 1 -- 172.21.15.83:0/1903892051 learned_addr learned my addr 172.21.15.83:0/1903892051 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.353+0000 7fd099400640 1 --2- 172.21.15.83:0/1903892051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae0fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.353+0000 7fd08be00640 1 -- 172.21.15.83:0/1903892051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccadffe60 msgr2=0x55eccae17150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.353+0000 7fd08be00640 1 --2- 172.21.15.83:0/1903892051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccadffe60 0x55eccae17150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.353+0000 7fd08be00640 1 -- 172.21.15.83:0/1903892051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 msgr2=0x55eccae0fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.353+0000 7fd08be00640 1 --2- 172.21.15.83:0/1903892051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae0fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.353+0000 7fd08be00640 1 -- 172.21.15.83:0/1903892051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eccae10380 con 0x55eccae03b40 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.353+0000 7fd08be00640 1 --2- 172.21.15.83:0/1903892051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 0x55eccae0f630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55eccae0ce60 tx=0x7fd07c012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f825a749ff810a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.354+0000 7fd082000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.354+0000 7fd082000640 1 -- 172.21.15.83:0/1903892051 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c004250 con 0x55eccae03b40 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.354+0000 7fd082000640 1 -- 172.21.15.83:0/1903892051 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd07c0043f0 con 0x55eccae03b40 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.354+0000 7fd09e22a740 1 -- 172.21.15.83:0/1903892051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd08c00fbe0 con 0x55eccae03b40 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.354+0000 7fd082000640 1 -- 172.21.15.83:0/1903892051 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd07c031050 con 0x55eccae03b40 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.354+0000 7fd09e22a740 10 client.15402 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.354+0000 7fd09e22a740 1 -- 172.21.15.83:0/1903892051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55eccae8d980 con 0x55eccae03b40 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.354+0000 7fd082000640 1 -- 172.21.15.83:0/1903892051 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd08c00fbe0 con 0x55eccae03b40 2024-04-23T14:49:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.355+0000 7fd082000640 1 -- 172.21.15.83:0/1903892051 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd07c0047a0 con 0x55eccae03b40 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.355+0000 7fd09e22a740 10 client.15402 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.355+0000 7fd09e22a740 10 client.15402 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.355+0000 7fd09e22a740 20 client.15402 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.355+0000 7fd09e22a740 1 --2- 172.21.15.83:0/1903892051 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55eccae0eb30 0x55eccae900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.355+0000 7fd09e22a740 4 client.15402 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.355+0000 7fd09e22a740 1 -- 172.21.15.83:0/1903892051 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eccad70a60 con 0x55eccae0eb30 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.355+0000 7fd099400640 1 --2- 172.21.15.83:0/1903892051 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55eccae0eb30 0x55eccae900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.355+0000 7fd099400640 1 --2- 172.21.15.83:0/1903892051 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55eccae0eb30 0x55eccae900c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3afb5564a95a1dff server_cookie=754560f9e6afb6cd in_seq=0 out_seq=0 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.355+0000 7fd082000640 10 client.15402 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd082000640 1 -- 172.21.15.83:0/1903892051 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eccad70a60 con 0x55eccae0eb30 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd082000640 10 client.15402 handle_command_reply: tid=0 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd09e22a740 1 client.15402 shutdown 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd09e22a740 1 -- 172.21.15.83:0/1903892051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 msgr2=0x55eccae0f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd09e22a740 1 --2- 172.21.15.83:0/1903892051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 0x55eccae0f630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55eccae0ce60 tx=0x7fd07c012b40 comp rx=0 tx=0).stop 2024-04-23T14:49:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd09e22a740 1 -- 172.21.15.83:0/1903892051 shutdown_connections 2024-04-23T14:49:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd09e22a740 1 --2- 172.21.15.83:0/1903892051 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55eccae0eb30 0x55eccae900c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd09e22a740 1 --2- 172.21.15.83:0/1903892051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55eccae04460 0x55eccae0fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd09e22a740 1 --2- 172.21.15.83:0/1903892051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eccae03b40 0x55eccae0f630 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd082000640 0 client.15402 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:49:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd09e22a740 1 --2- 172.21.15.83:0/1903892051 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55eccadffe60 0x55eccae17150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd09e22a740 1 -- 172.21.15.83:0/1903892051 >> 172.21.15.83:0/1903892051 conn(0x55eccae11240 msgr2=0x55eccadfef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd09e22a740 1 -- 172.21.15.83:0/1903892051 shutdown_connections 2024-04-23T14:49:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd09e22a740 1 -- 172.21.15.83:0/1903892051 wait complete. 2024-04-23T14:49:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.356+0000 7fd09e22a740 20 client.15402 trim_cache size 0 max 16384 2024-04-23T14:49:08.361 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:49:08.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.357+0000 7fd099e00640 1 -- 172.21.15.83:0/733668518 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0780a3f50 msgr2=0x7fd0780a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.357+0000 7fd099e00640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0780a3f50 0x7fd0780a6410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd0840ee580 tx=0x7fd0840f4ab0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.357+0000 7fd099e00640 1 -- 172.21.15.83:0/733668518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd094102110 msgr2=0x7fd094074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.357+0000 7fd099e00640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd094102110 0x7fd094074300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd084004770 tx=0x7fd084003c10 comp rx=0 tx=0).stop 2024-04-23T14:49:08.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.357+0000 7fd099e00640 1 -- 172.21.15.83:0/733668518 shutdown_connections 2024-04-23T14:49:08.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.357+0000 7fd099e00640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0780a3f50 0x7fd0780a6410 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.357+0000 7fd099e00640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd094074d80 0x7fd0941a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.357+0000 7fd099e00640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0941071e0 0x7fd094074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.357+0000 7fd099e00640 1 --2- 172.21.15.83:0/733668518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd094102110 0x7fd094074300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.357+0000 7fd099e00640 1 -- 172.21.15.83:0/733668518 >> 172.21.15.83:0/733668518 conn(0x7fd0940fd9d0 msgr2=0x7fd0940ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.357+0000 7fd099e00640 1 -- 172.21.15.83:0/733668518 shutdown_connections 2024-04-23T14:49:08.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.357+0000 7fd099e00640 1 -- 172.21.15.83:0/733668518 wait complete. 2024-04-23T14:49:08.372 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:08 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3214518350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:49:08.375 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:49:08.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.564+0000 7f9eea800640 1 -- 172.21.15.83:0/763577393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ee41044b0 msgr2=0x7f9ee4104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.564+0000 7f9eea800640 1 --2- 172.21.15.83:0/763577393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ee41044b0 0x7f9ee4104890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009900 tx=0x7f9ed402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.565+0000 7f9eea800640 1 -- 172.21.15.83:0/763577393 shutdown_connections 2024-04-23T14:49:08.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.565+0000 7f9eea800640 1 --2- 172.21.15.83:0/763577393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee4109580 0x7f9ee4111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.565+0000 7f9eea800640 1 --2- 172.21.15.83:0/763577393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ee4104e60 0x7f9ee4108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.565+0000 7f9eea800640 1 --2- 172.21.15.83:0/763577393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ee41044b0 0x7f9ee4104890 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.565+0000 7f9eea800640 1 -- 172.21.15.83:0/763577393 >> 172.21.15.83:0/763577393 conn(0x7f9ee40ffd90 msgr2=0x7f9ee41021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.565+0000 7f9eea800640 1 -- 172.21.15.83:0/763577393 shutdown_connections 2024-04-23T14:49:08.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.565+0000 7f9eea800640 1 -- 172.21.15.83:0/763577393 wait complete. 2024-04-23T14:49:08.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.566+0000 7f9eea800640 1 Processor -- start 2024-04-23T14:49:08.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.566+0000 7f9eea800640 1 -- start start 2024-04-23T14:49:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.566+0000 7f9eea800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ee41044b0 0x7f9ee41a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.566+0000 7f9eea800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee4109580 0x7f9ee41a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.566+0000 7f9eea800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ee419aa80 0x7f9ee419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.566+0000 7f9eea800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4114660 con 0x7f9ee4109580 2024-04-23T14:49:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.566+0000 7f9eea800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41144e0 con 0x7f9ee41044b0 2024-04-23T14:49:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.566+0000 7f9eea800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41147e0 con 0x7f9ee419aa80 2024-04-23T14:49:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.566+0000 7f9edaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee4109580 0x7f9ee41a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.566+0000 7f9edaa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee4109580 0x7f9ee41a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52272/0 (socket says 172.21.15.83:52272) 2024-04-23T14:49:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.566+0000 7f9edaa00640 1 -- 172.21.15.83:0/650694403 learned_addr learned my addr 172.21.15.83:0/650694403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.566+0000 7f9ee3400640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ee41044b0 0x7f9ee41a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.567+0000 7f9ee3e00640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ee419aa80 0x7f9ee419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.567+0000 7f9ee3400640 1 -- 172.21.15.83:0/650694403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ee419aa80 msgr2=0x7f9ee419af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.567+0000 7f9ee3400640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ee419aa80 0x7f9ee419af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.567+0000 7f9ee3400640 1 -- 172.21.15.83:0/650694403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee4109580 msgr2=0x7f9ee41a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.567+0000 7f9ee3400640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee4109580 0x7f9ee41a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.567+0000 7f9ee3400640 1 -- 172.21.15.83:0/650694403 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee419b5a0 con 0x7f9ee41044b0 2024-04-23T14:49:08.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.567+0000 7f9edaa00640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee4109580 0x7f9ee41a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:08.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.567+0000 7f9ee3400640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ee41044b0 0x7f9ee41a0880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9ed402a910 tx=0x7f9ed4003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.568+0000 7f9ee0c00640 1 -- 172.21.15.83:0/650694403 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed403f070 con 0x7f9ee41044b0 2024-04-23T14:49:08.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.568+0000 7f9ee0c00640 1 -- 172.21.15.83:0/650694403 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed4031c90 con 0x7f9ee41044b0 2024-04-23T14:49:08.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.568+0000 7f9ee0c00640 1 -- 172.21.15.83:0/650694403 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4035030 con 0x7f9ee41044b0 2024-04-23T14:49:08.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.568+0000 7f9eea800640 1 -- 172.21.15.83:0/650694403 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee419b830 con 0x7f9ee41044b0 2024-04-23T14:49:08.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.568+0000 7f9eea800640 1 -- 172.21.15.83:0/650694403 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee41a7980 con 0x7f9ee41044b0 2024-04-23T14:49:08.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.571+0000 7f9eed63f740 1 Processor -- start 2024-04-23T14:49:08.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.571+0000 7f9eed63f740 1 -- start start 2024-04-23T14:49:08.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.571+0000 7f9ee0c00640 1 -- 172.21.15.83:0/650694403 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f9ed4042070 con 0x7f9ee41044b0 2024-04-23T14:49:08.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.571+0000 7f9eed63f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fa950 0x55cb8f1fad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.571+0000 7f9eed63f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f1fb270 0x55cb8f205400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.571+0000 7f9eed63f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f205940 0x55cb8f207d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.571+0000 7f9eed63f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cb8f1601d0 con 0x55cb8f1fb270 2024-04-23T14:49:08.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.571+0000 7f9eed63f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cb8f106450 con 0x55cb8f1fa950 2024-04-23T14:49:08.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.571+0000 7f9eed63f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cb8f15c190 con 0x55cb8f205940 2024-04-23T14:49:08.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.572+0000 7f9ee3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f1fb270 0x55cb8f205400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.572+0000 7f9edaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f205940 0x55cb8f207d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.572+0000 7f9ee3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fa950 0x55cb8f1fad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.572+0000 7f9edaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f205940 0x55cb8f207d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42256/0 (socket says 172.21.15.83:42256) 2024-04-23T14:49:08.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.572+0000 7f9edaa00640 1 -- 172.21.15.83:0/3950561501 learned_addr learned my addr 172.21.15.83:0/3950561501 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.572+0000 7f9edaa00640 1 -- 172.21.15.83:0/3950561501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fa950 msgr2=0x55cb8f1fad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.572+0000 7f9edaa00640 1 --2- 172.21.15.83:0/3950561501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fa950 0x55cb8f1fad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.572+0000 7f9edaa00640 1 -- 172.21.15.83:0/3950561501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f1fb270 msgr2=0x55cb8f205400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.572+0000 7f9edaa00640 1 --2- 172.21.15.83:0/3950561501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f1fb270 0x55cb8f205400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.572+0000 7f9edaa00640 1 -- 172.21.15.83:0/3950561501 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8f1c3310 con 0x55cb8f205940 2024-04-23T14:49:08.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.572+0000 7f9edaa00640 1 --2- 172.21.15.83:0/3950561501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f205940 0x55cb8f207d30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc0118b0 tx=0x7f9ecc011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7770bf393875832 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.573+0000 7f9ed8c00640 1 -- 172.21.15.83:0/3950561501 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc011330 con 0x55cb8f205940 2024-04-23T14:49:08.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.573+0000 7f9ed8c00640 1 -- 172.21.15.83:0/3950561501 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ecc003e50 con 0x55cb8f205940 2024-04-23T14:49:08.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.573+0000 7f9ed8c00640 1 -- 172.21.15.83:0/3950561501 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc004550 con 0x55cb8f205940 2024-04-23T14:49:08.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.573+0000 7f9ee0c00640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9ec80a3f50 0x7f9ec80a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.573+0000 7f9ee0c00640 1 -- 172.21.15.83:0/650694403 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9ed40f12c0 con 0x7f9ee41044b0 2024-04-23T14:49:08.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.573+0000 7f9ee3e00640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9ec80a3f50 0x7f9ec80a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.573+0000 7f9eed63f740 1 -- 172.21.15.83:0/3950561501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f205940 msgr2=0x55cb8f207d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.573+0000 7f9eed63f740 1 --2- 172.21.15.83:0/3950561501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f205940 0x55cb8f207d30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc0118b0 tx=0x7f9ecc011d70 comp rx=0 tx=0).stop 2024-04-23T14:49:08.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:08 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3214518350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:49:08.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.573+0000 7f9eed63f740 1 -- 172.21.15.83:0/3950561501 shutdown_connections 2024-04-23T14:49:08.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.574+0000 7f9eed63f740 1 --2- 172.21.15.83:0/3950561501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f205940 0x55cb8f207d30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.574+0000 7f9eed63f740 1 --2- 172.21.15.83:0/3950561501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f1fb270 0x55cb8f205400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.574+0000 7f9eed63f740 1 --2- 172.21.15.83:0/3950561501 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fa950 0x55cb8f1fad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.574+0000 7f9eed63f740 1 -- 172.21.15.83:0/3950561501 >> 172.21.15.83:0/3950561501 conn(0x55cb8f1f6c70 msgr2=0x55cb8f1f7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.574+0000 7f9eed63f740 1 -- 172.21.15.83:0/3950561501 shutdown_connections 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.574+0000 7f9ee3e00640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9ec80a3f50 0x7f9ec80a6410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009d30 tx=0x7f9edc005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.574+0000 7f9eed63f740 1 -- 172.21.15.83:0/3950561501 wait complete. 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.574+0000 7f9eed63f740 1 -- start start 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.574+0000 7f9eed63f740 1 Processor -- start 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9eed63f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f1fa950 0x55cb8f205170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9eed63f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fb270 0x55cb8f2024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9eed63f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f205940 0x55cb8f202a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9eed63f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cb8f2087d0 con 0x55cb8f1fa950 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9eed63f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cb8f119810 con 0x55cb8f205940 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9eed63f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cb8f208950 con 0x55cb8f1fb270 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9eed63f740 10 client.0 _ll_register_callbacks cb 0x55cb8f0e7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:08.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9ee3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f1fa950 0x55cb8f205170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9edaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fb270 0x55cb8f2024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9ee3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f1fa950 0x55cb8f205170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52298/0 (socket says 172.21.15.83:52298) 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9ee3400640 1 -- 172.21.15.83:0/1127776433 learned_addr learned my addr 172.21.15.83:0/1127776433 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9ee3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f205940 0x55cb8f202a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9ee3400640 1 -- 172.21.15.83:0/1127776433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fb270 msgr2=0x55cb8f2024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9ee3400640 1 --2- 172.21.15.83:0/1127776433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fb270 0x55cb8f2024d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9ee3400640 1 -- 172.21.15.83:0/1127776433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f205940 msgr2=0x55cb8f202a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9ee3400640 1 --2- 172.21.15.83:0/1127776433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f205940 0x55cb8f202a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.575+0000 7f9ee3400640 1 -- 172.21.15.83:0/1127776433 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb8f12a3f0 con 0x55cb8f1fa950 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.576+0000 7f9ee3400640 1 --2- 172.21.15.83:0/1127776433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f1fa950 0x55cb8f205170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40bcfc0 tx=0x7f9ed4031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fdf3571aef287d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.576+0000 7f9ed1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.576+0000 7f9ed1600640 1 -- 172.21.15.83:0/1127776433 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4049070 con 0x55cb8f1fa950 2024-04-23T14:49:08.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.576+0000 7f9ed1600640 1 -- 172.21.15.83:0/1127776433 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9ed40b96a0 con 0x55cb8f1fa950 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.576+0000 7f9eed63f740 1 -- 172.21.15.83:0/1127776433 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ecc003e50 con 0x55cb8f1fa950 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.576+0000 7f9ed1600640 1 -- 172.21.15.83:0/1127776433 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9ed40b92d0 con 0x55cb8f1fa950 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.576+0000 7f9eed63f740 10 client.15414 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.576+0000 7f9eed63f740 1 -- 172.21.15.83:0/1127776433 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cb8ee723f0 con 0x55cb8f1fa950 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.576+0000 7f9ed1600640 1 -- 172.21.15.83:0/1127776433 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ecc003e50 con 0x55cb8f1fa950 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.576+0000 7f9ed1600640 1 -- 172.21.15.83:0/1127776433 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9ed403bbe0 con 0x55cb8f1fa950 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.576+0000 7f9eed63f740 10 client.15414 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.576+0000 7f9eed63f740 10 client.15414 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.577+0000 7f9eed63f740 20 client.15414 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.577+0000 7f9eed63f740 1 --2- 172.21.15.83:0/1127776433 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55cb8f215860 0x55cb8f217cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.577+0000 7f9eed63f740 4 client.15414 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.577+0000 7f9eed63f740 1 -- 172.21.15.83:0/1127776433 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb8f16dbf0 con 0x55cb8f215860 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.577+0000 7f9edaa00640 1 --2- 172.21.15.83:0/1127776433 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55cb8f215860 0x55cb8f217cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.577+0000 7f9edaa00640 1 --2- 172.21.15.83:0/1127776433 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55cb8f215860 0x55cb8f217cb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5dec7b03295108c server_cookie=fce0e63baed716df in_seq=0 out_seq=0 2024-04-23T14:49:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.577+0000 7f9ed1600640 10 client.15414 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:49:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.578+0000 7f9ed1600640 1 -- 172.21.15.83:0/1127776433 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb8f16dbf0 con 0x55cb8f215860 2024-04-23T14:49:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.578+0000 7f9ed1600640 10 client.15414 handle_command_reply: tid=0 2024-04-23T14:49:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.578+0000 7f9eed63f740 1 client.15414 shutdown 2024-04-23T14:49:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.578+0000 7f9eed63f740 1 -- 172.21.15.83:0/1127776433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f1fa950 msgr2=0x55cb8f205170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.578+0000 7f9eed63f740 1 --2- 172.21.15.83:0/1127776433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f1fa950 0x55cb8f205170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40bcfc0 tx=0x7f9ed4031490 comp rx=0 tx=0).stop 2024-04-23T14:49:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.579+0000 7f9eed63f740 1 -- 172.21.15.83:0/1127776433 shutdown_connections 2024-04-23T14:49:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.579+0000 7f9eed63f740 1 --2- 172.21.15.83:0/1127776433 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f205940 0x55cb8f202a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.579+0000 7f9eed63f740 1 --2- 172.21.15.83:0/1127776433 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55cb8f215860 0x55cb8f217cb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.579+0000 7f9eed63f740 1 --2- 172.21.15.83:0/1127776433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fb270 0x55cb8f2024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.579+0000 7f9eed63f740 1 --2- 172.21.15.83:0/1127776433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f1fa950 0x55cb8f205170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.579+0000 7f9ed1600640 0 client.15414 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:49:08.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.579+0000 7f9eed63f740 1 -- 172.21.15.83:0/1127776433 >> 172.21.15.83:0/1127776433 conn(0x55cb8f1f6c70 msgr2=0x55cb8f1f5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.579+0000 7f9eed63f740 1 -- 172.21.15.83:0/1127776433 shutdown_connections 2024-04-23T14:49:08.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.579+0000 7f9eed63f740 1 -- 172.21.15.83:0/1127776433 wait complete. 2024-04-23T14:49:08.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.579+0000 7f9eed63f740 20 client.15414 trim_cache size 0 max 16384 2024-04-23T14:49:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.593+0000 7f9eed63f740 1 Processor -- start 2024-04-23T14:49:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.593+0000 7f9eed63f740 1 -- start start 2024-04-23T14:49:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.594+0000 7f9eed63f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fa950 0x55cb8f1f6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.594+0000 7f9eed63f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fb270 0x55cb8f204d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.594+0000 7f9eed63f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 0x55cb8f21d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.594+0000 7f9eed63f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41b2790 con 0x55cb8f205940 2024-04-23T14:49:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.594+0000 7f9eed63f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41b0060 con 0x55cb8f1fb270 2024-04-23T14:49:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.594+0000 7f9eed63f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41adeb0 con 0x55cb8f1fa950 2024-04-23T14:49:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.594+0000 7f9ee3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 0x55cb8f21d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.594+0000 7f9ee3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 0x55cb8f21d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52312/0 (socket says 172.21.15.83:52312) 2024-04-23T14:49:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.594+0000 7f9ee3e00640 1 -- 172.21.15.83:0/569781072 learned_addr learned my addr 172.21.15.83:0/569781072 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.594+0000 7f9edaa00640 1 --2- 172.21.15.83:0/569781072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fb270 0x55cb8f204d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.594+0000 7f9ee3400640 1 --2- 172.21.15.83:0/569781072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fa950 0x55cb8f1f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.595+0000 7f9ee3e00640 1 -- 172.21.15.83:0/569781072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fa950 msgr2=0x55cb8f1f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.595+0000 7f9ee3e00640 1 --2- 172.21.15.83:0/569781072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fa950 0x55cb8f1f6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.595+0000 7f9ee3e00640 1 -- 172.21.15.83:0/569781072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fb270 msgr2=0x55cb8f204d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.595+0000 7f9ee3e00640 1 --2- 172.21.15.83:0/569781072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fb270 0x55cb8f204d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.595+0000 7f9ee3e00640 1 -- 172.21.15.83:0/569781072 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8ef3c7d0 con 0x55cb8f205940 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.595+0000 7f9ee3e00640 1 --2- 172.21.15.83:0/569781072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 0x55cb8f21d520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009b10 tx=0x7f9edc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d0e96453d3a8b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.595+0000 7f9ed3e00640 1 -- 172.21.15.83:0/569781072 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc01dcf0 con 0x55cb8f205940 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.595+0000 7f9ed3e00640 1 -- 172.21.15.83:0/569781072 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9edc02e070 con 0x55cb8f205940 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.595+0000 7f9ed3e00640 1 -- 172.21.15.83:0/569781072 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00f760 con 0x55cb8f205940 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.596+0000 7f9eed63f740 1 -- 172.21.15.83:0/569781072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 msgr2=0x55cb8f21d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.596+0000 7f9eed63f740 1 --2- 172.21.15.83:0/569781072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 0x55cb8f21d520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009b10 tx=0x7f9edc017000 comp rx=0 tx=0).stop 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.596+0000 7f9eed63f740 1 -- 172.21.15.83:0/569781072 shutdown_connections 2024-04-23T14:49:08.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.596+0000 7f9eed63f740 1 --2- 172.21.15.83:0/569781072 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 0x55cb8f21d520 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.596+0000 7f9eed63f740 1 --2- 172.21.15.83:0/569781072 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fb270 0x55cb8f204d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.596+0000 7f9eed63f740 1 --2- 172.21.15.83:0/569781072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fa950 0x55cb8f1f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.596+0000 7f9eed63f740 1 -- 172.21.15.83:0/569781072 >> 172.21.15.83:0/569781072 conn(0x55cb8f1f6c70 msgr2=0x55cb8f200da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.596+0000 7f9eed63f740 1 -- 172.21.15.83:0/569781072 shutdown_connections 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.596+0000 7f9eed63f740 1 -- 172.21.15.83:0/569781072 wait complete. 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.596+0000 7f9eed63f740 1 -- start start 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.597+0000 7f9eed63f740 1 Processor -- start 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.597+0000 7f9eed63f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fa950 0x55cb8f20dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.597+0000 7f9eed63f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fb270 0x55cb8f20e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.597+0000 7f9eed63f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 0x55cb8f209390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.597+0000 7f9eed63f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41ae520 con 0x55cb8f205940 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.597+0000 7f9eed63f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41af990 con 0x55cb8f1fa950 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.597+0000 7f9eed63f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41aebf0 con 0x55cb8f1fb270 2024-04-23T14:49:08.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.597+0000 7f9eed63f740 10 client.0 _ll_register_callbacks cb 0x7f9ed40f4f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.597+0000 7f9edaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fb270 0x55cb8f20e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.597+0000 7f9ee3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 0x55cb8f209390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.597+0000 7f9edaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fb270 0x55cb8f20e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42294/0 (socket says 172.21.15.83:42294) 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.597+0000 7f9edaa00640 1 -- 172.21.15.83:0/2577931633 learned_addr learned my addr 172.21.15.83:0/2577931633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9ee3400640 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fa950 0x55cb8f20dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9edaa00640 1 -- 172.21.15.83:0/2577931633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fa950 msgr2=0x55cb8f20dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9edaa00640 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fa950 0x55cb8f20dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9edaa00640 1 -- 172.21.15.83:0/2577931633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 msgr2=0x55cb8f209390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9edaa00640 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 0x55cb8f209390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9edaa00640 1 -- 172.21.15.83:0/2577931633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb8f209b60 con 0x55cb8f1fb270 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9ee3400640 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fa950 0x55cb8f20dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9ee3e00640 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 0x55cb8f209390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:08.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9edaa00640 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fb270 0x55cb8f20e130 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc004160 tx=0x7f9ecc011840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4dedf9474d8f8db server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9ed1600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9ed1600640 1 -- 172.21.15.83:0/2577931633 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc01f040 con 0x55cb8f1fb270 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9ed1600640 1 -- 172.21.15.83:0/2577931633 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9ecc011130 con 0x55cb8f1fb270 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.598+0000 7f9eed63f740 1 -- 172.21.15.83:0/2577931633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9edc02e070 con 0x55cb8f1fb270 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.599+0000 7f9ed1600640 1 -- 172.21.15.83:0/2577931633 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9ecc039050 con 0x55cb8f1fb270 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.599+0000 7f9eed63f740 10 client.25507 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.599+0000 7f9eed63f740 1 -- 172.21.15.83:0/2577931633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55cb8f27ba60 con 0x55cb8f1fb270 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.599+0000 7f9ed1600640 1 -- 172.21.15.83:0/2577931633 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9edc02e070 con 0x55cb8f1fb270 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.599+0000 7f9ed1600640 1 -- 172.21.15.83:0/2577931633 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f9ecc018070 con 0x55cb8f1fb270 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.599+0000 7f9eed63f740 10 client.25507 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.599+0000 7f9eed63f740 10 client.25507 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.599+0000 7f9eed63f740 20 client.25507 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:08.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.599+0000 7f9eed63f740 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55cb8f27c070 0x55cb8f27e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.599+0000 7f9eed63f740 4 client.25507 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.599+0000 7f9eed63f740 1 -- 172.21.15.83:0/2577931633 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb8f167a60 con 0x55cb8f27c070 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.599+0000 7f9ee3400640 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55cb8f27c070 0x55cb8f27e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9ee3400640 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55cb8f27c070 0x55cb8f27e490 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49c219d425544589 server_cookie=ae10f2b3fd1b03c5 in_seq=0 out_seq=0 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9ed1600640 10 client.25507 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9ed1600640 1 -- 172.21.15.83:0/2577931633 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb8f167a60 con 0x55cb8f27c070 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9ed1600640 10 client.25507 handle_command_reply: tid=0 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9eed63f740 1 client.25507 shutdown 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9eed63f740 1 -- 172.21.15.83:0/2577931633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fb270 msgr2=0x55cb8f20e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9eed63f740 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fb270 0x55cb8f20e130 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc004160 tx=0x7f9ecc011840 comp rx=0 tx=0).stop 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9eed63f740 1 -- 172.21.15.83:0/2577931633 shutdown_connections 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9eed63f740 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cb8f205940 0x55cb8f209390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9eed63f740 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55cb8f27c070 0x55cb8f27e490 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9eed63f740 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb8f1fb270 0x55cb8f20e130 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9ed1600640 0 client.25507 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:49:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9eed63f740 1 --2- 172.21.15.83:0/2577931633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cb8f1fa950 0x55cb8f20dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.600+0000 7f9eed63f740 1 -- 172.21.15.83:0/2577931633 >> 172.21.15.83:0/2577931633 conn(0x55cb8f1f6c70 msgr2=0x55cb8f201200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eed63f740 1 -- 172.21.15.83:0/2577931633 shutdown_connections 2024-04-23T14:49:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eed63f740 1 -- 172.21.15.83:0/2577931633 wait complete. 2024-04-23T14:49:08.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eed63f740 20 client.25507 trim_cache size 0 max 16384 2024-04-23T14:49:08.610 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:49:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eea800640 1 -- 172.21.15.83:0/650694403 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9ec80a3f50 msgr2=0x7f9ec80a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eea800640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9ec80a3f50 0x7f9ec80a6410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009d30 tx=0x7f9edc005f90 comp rx=0 tx=0).stop 2024-04-23T14:49:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eea800640 1 -- 172.21.15.83:0/650694403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ee41044b0 msgr2=0x7f9ee41a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eea800640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ee41044b0 0x7f9ee41a0880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9ed402a910 tx=0x7f9ed4003b20 comp rx=0 tx=0).stop 2024-04-23T14:49:08.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eea800640 1 -- 172.21.15.83:0/650694403 shutdown_connections 2024-04-23T14:49:08.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eea800640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ee419aa80 0x7f9ee419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eea800640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9ec80a3f50 0x7f9ec80a6410 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eea800640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ee4109580 0x7f9ee41a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eea800640 1 --2- 172.21.15.83:0/650694403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ee41044b0 0x7f9ee41a0880 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eea800640 1 -- 172.21.15.83:0/650694403 >> 172.21.15.83:0/650694403 conn(0x7f9ee40ffd90 msgr2=0x7f9ee41010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.601+0000 7f9eea800640 1 -- 172.21.15.83:0/650694403 shutdown_connections 2024-04-23T14:49:08.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.602+0000 7f9eea800640 1 -- 172.21.15.83:0/650694403 wait complete. 2024-04-23T14:49:08.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:49:08.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:08 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3214518350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:49:08.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.813+0000 7fded6600640 1 -- 172.21.15.83:0/1153657306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded0104e00 msgr2=0x7fded0108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.813+0000 7fded6600640 1 --2- 172.21.15.83:0/1153657306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded0104e00 0x7fded0108e40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009900 tx=0x7fdeb802a410 comp rx=0 tx=0).stop 2024-04-23T14:49:08.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.813+0000 7fded6600640 1 -- 172.21.15.83:0/1153657306 shutdown_connections 2024-04-23T14:49:08.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.813+0000 7fded6600640 1 --2- 172.21.15.83:0/1153657306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fded0109520 0x7fded01110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.813+0000 7fded6600640 1 --2- 172.21.15.83:0/1153657306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded0104e00 0x7fded0108e40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.813+0000 7fded6600640 1 --2- 172.21.15.83:0/1153657306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fded0104450 0x7fded0104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.813+0000 7fded6600640 1 -- 172.21.15.83:0/1153657306 >> 172.21.15.83:0/1153657306 conn(0x7fded00ffc70 msgr2=0x7fded0102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.814+0000 7fded6600640 1 -- 172.21.15.83:0/1153657306 shutdown_connections 2024-04-23T14:49:08.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.814+0000 7fded6600640 1 -- 172.21.15.83:0/1153657306 wait complete. 2024-04-23T14:49:08.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.814+0000 7fded6600640 1 Processor -- start 2024-04-23T14:49:08.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.815+0000 7fded6600640 1 -- start start 2024-04-23T14:49:08.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.815+0000 7fded6600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fded0104450 0x7fded01a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.815+0000 7fded6600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fded0109520 0x7fded01a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.815+0000 7fded6600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded019a920 0x7fded019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.815+0000 7fded6600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fded0114600 con 0x7fded019a920 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.815+0000 7fded6600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fded0114480 con 0x7fded0109520 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.815+0000 7fded6600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fded0114780 con 0x7fded0104450 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.815+0000 7fdecfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded019a920 0x7fded019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.815+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fded0109520 0x7fded01a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.815+0000 7fdecf400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fded0104450 0x7fded01a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.815+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fded0109520 0x7fded01a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:52162/0 (socket says 172.21.15.83:52162) 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.815+0000 7fdec6a00640 1 -- 172.21.15.83:0/1866489938 learned_addr learned my addr 172.21.15.83:0/1866489938 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.816+0000 7fdecf400640 1 -- 172.21.15.83:0/1866489938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fded0109520 msgr2=0x7fded01a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.816+0000 7fdecf400640 1 --2- 172.21.15.83:0/1866489938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fded0109520 0x7fded01a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.816+0000 7fdecf400640 1 -- 172.21.15.83:0/1866489938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded019a920 msgr2=0x7fded019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.816+0000 7fdecf400640 1 --2- 172.21.15.83:0/1866489938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded019a920 0x7fded019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.816+0000 7fdecf400640 1 -- 172.21.15.83:0/1866489938 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded019b440 con 0x7fded0104450 2024-04-23T14:49:08.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.816+0000 7fdecf400640 1 --2- 172.21.15.83:0/1866489938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fded0104450 0x7fded01a0770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdec000eb00 tx=0x7fdec000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.816+0000 7fdeccc00640 1 -- 172.21.15.83:0/1866489938 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0010430 con 0x7fded0104450 2024-04-23T14:49:08.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.816+0000 7fdeccc00640 1 -- 172.21.15.83:0/1866489938 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdec0010ab0 con 0x7fded0104450 2024-04-23T14:49:08.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.816+0000 7fded6600640 1 -- 172.21.15.83:0/1866489938 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded019b730 con 0x7fded0104450 2024-04-23T14:49:08.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.817+0000 7fdeccc00640 1 -- 172.21.15.83:0/1866489938 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0017bb0 con 0x7fded0104450 2024-04-23T14:49:08.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.817+0000 7fded6600640 1 -- 172.21.15.83:0/1866489938 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded01a7980 con 0x7fded0104450 2024-04-23T14:49:08.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.820+0000 7fded93fc740 1 Processor -- start 2024-04-23T14:49:08.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.820+0000 7fded93fc740 1 -- start start 2024-04-23T14:49:08.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.820+0000 7fdeccc00640 1 -- 172.21.15.83:0/1866489938 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fdec0010c60 con 0x7fded0104450 2024-04-23T14:49:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.820+0000 7fded93fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 0x556ce0bded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.820+0000 7fded93fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0be9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.820+0000 7fded93fc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0be9940 0x556ce0bebd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.820+0000 7fded93fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ce0b441d0 con 0x556ce0bde950 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.820+0000 7fded93fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556ce0aea450 con 0x556ce0bdf270 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.820+0000 7fded93fc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556ce0b40190 con 0x556ce0be9940 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.821+0000 7fdecfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 0x556ce0bded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.821+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0be9940 0x556ce0bebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.821+0000 7fdecf400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0be9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.821+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0be9940 0x556ce0bebd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42308/0 (socket says 172.21.15.83:42308) 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.821+0000 7fdec6a00640 1 -- 172.21.15.83:0/666481882 learned_addr learned my addr 172.21.15.83:0/666481882 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.821+0000 7fdec6a00640 1 -- 172.21.15.83:0/666481882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 msgr2=0x556ce0be9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.821+0000 7fdec6a00640 1 --2- 172.21.15.83:0/666481882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0be9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.821+0000 7fdec6a00640 1 -- 172.21.15.83:0/666481882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 msgr2=0x556ce0bded30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.821+0000 7fdec6a00640 1 --2- 172.21.15.83:0/666481882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 0x556ce0bded30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.821+0000 7fdec6a00640 1 -- 172.21.15.83:0/666481882 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ce0ba7310 con 0x556ce0be9940 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.821+0000 7fdecfe00640 1 --2- 172.21.15.83:0/666481882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 0x556ce0bded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:08.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.822+0000 7fdec6a00640 1 --2- 172.21.15.83:0/666481882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0be9940 0x556ce0bebd30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdeb802a930 tx=0x7fdeb8005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a43e414e556c8035 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.822+0000 7fdec4c00640 1 -- 172.21.15.83:0/666481882 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8035040 con 0x556ce0be9940 2024-04-23T14:49:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.822+0000 7fdec4c00640 1 -- 172.21.15.83:0/666481882 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb8004480 con 0x556ce0be9940 2024-04-23T14:49:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.822+0000 7fdec4c00640 1 -- 172.21.15.83:0/666481882 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb80035f0 con 0x556ce0be9940 2024-04-23T14:49:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.822+0000 7fdeccc00640 1 --2- 172.21.15.83:0/1866489938 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdeb40a3ed0 0x7fdeb40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.822+0000 7fdeccc00640 1 -- 172.21.15.83:0/1866489938 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fdec00d1480 con 0x7fded0104450 2024-04-23T14:49:08.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.822+0000 7fdecfe00640 1 --2- 172.21.15.83:0/1866489938 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdeb40a3ed0 0x7fdeb40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.823+0000 7fded93fc740 1 -- 172.21.15.83:0/666481882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0be9940 msgr2=0x556ce0bebd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.823+0000 7fded93fc740 1 --2- 172.21.15.83:0/666481882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0be9940 0x556ce0bebd30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdeb802a930 tx=0x7fdeb8005de0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.823+0000 7fded93fc740 1 -- 172.21.15.83:0/666481882 shutdown_connections 2024-04-23T14:49:08.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.823+0000 7fded93fc740 1 --2- 172.21.15.83:0/666481882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0be9940 0x556ce0bebd30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.823+0000 7fded93fc740 1 --2- 172.21.15.83:0/666481882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0be9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.823+0000 7fded93fc740 1 --2- 172.21.15.83:0/666481882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 0x556ce0bded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.823+0000 7fded93fc740 1 -- 172.21.15.83:0/666481882 >> 172.21.15.83:0/666481882 conn(0x556ce0bdac70 msgr2=0x556ce0bdb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.823+0000 7fded93fc740 1 -- 172.21.15.83:0/666481882 shutdown_connections 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.823+0000 7fded93fc740 1 -- 172.21.15.83:0/666481882 wait complete. 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.823+0000 7fdecfe00640 1 --2- 172.21.15.83:0/1866489938 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdeb40a3ed0 0x7fdeb40a6390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009ef0 tx=0x7fdec8005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.823+0000 7fded93fc740 1 -- start start 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fded93fc740 1 Processor -- start 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fded93fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 0x556ce0be93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fded93fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0be66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fded93fc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0be9940 0x556ce0be6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fded93fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ce0bec770 con 0x556ce0bde950 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fded93fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556ce0afd810 con 0x556ce0bdf270 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fded93fc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556ce0bec8f0 con 0x556ce0be9940 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fded93fc740 10 client.0 _ll_register_callbacks cb 0x556ce0acb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:08.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fdecf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 0x556ce0be93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fdecf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 0x556ce0be93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52366/0 (socket says 172.21.15.83:52366) 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fdecf400640 1 -- 172.21.15.83:0/3071238331 learned_addr learned my addr 172.21.15.83:0/3071238331 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fdec6a00640 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0be66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.824+0000 7fdecfe00640 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0be9940 0x556ce0be6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.825+0000 7fdecf400640 1 -- 172.21.15.83:0/3071238331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0be9940 msgr2=0x556ce0be6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.825+0000 7fdecf400640 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0be9940 0x556ce0be6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.825+0000 7fdecf400640 1 -- 172.21.15.83:0/3071238331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 msgr2=0x556ce0be66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.825+0000 7fdecf400640 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0be66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.825+0000 7fdecf400640 1 -- 172.21.15.83:0/3071238331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ce0b0e3f0 con 0x556ce0bde950 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.825+0000 7fdecf400640 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 0x556ce0be93d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdec0099650 tx=0x7fdec0099490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225434bf8355e45e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.825+0000 7fdebd600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.825+0000 7fdebd600640 1 -- 172.21.15.83:0/3071238331 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec009cb50 con 0x556ce0bde950 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.825+0000 7fdebd600640 1 -- 172.21.15.83:0/3071238331 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fdec00a1120 con 0x556ce0bde950 2024-04-23T14:49:08.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.825+0000 7fded93fc740 1 -- 172.21.15.83:0/3071238331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb8004480 con 0x556ce0bde950 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.826+0000 7fdebd600640 1 -- 172.21.15.83:0/3071238331 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdec009d8f0 con 0x556ce0bde950 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.826+0000 7fded93fc740 10 client.15432 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.826+0000 7fded93fc740 1 -- 172.21.15.83:0/3071238331 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556ce08563f0 con 0x556ce0bde950 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.826+0000 7fdebd600640 1 -- 172.21.15.83:0/3071238331 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb8004480 con 0x556ce0bde950 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.826+0000 7fdebd600640 1 -- 172.21.15.83:0/3071238331 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdec009c640 con 0x556ce0bde950 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.826+0000 7fded93fc740 10 client.15432 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.826+0000 7fded93fc740 10 client.15432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.826+0000 7fded93fc740 20 client.15432 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.826+0000 7fded93fc740 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ce0bf9ba0 0x556ce0bfbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.826+0000 7fded93fc740 4 client.15432 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.826+0000 7fded93fc740 1 -- 172.21.15.83:0/3071238331 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ce0b51bf0 con 0x556ce0bf9ba0 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.827+0000 7fdec6a00640 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ce0bf9ba0 0x556ce0bfbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.827+0000 7fdec6a00640 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ce0bf9ba0 0x556ce0bfbf90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=382545fe4d9833f5 server_cookie=d585a87c9dddd4ff in_seq=0 out_seq=0 2024-04-23T14:49:08.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.827+0000 7fdebd600640 10 client.15432 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:08.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fdebd600640 1 -- 172.21.15.83:0/3071238331 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ce0b51bf0 con 0x556ce0bf9ba0 2024-04-23T14:49:08.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fdebd600640 10 client.15432 handle_command_reply: tid=0 2024-04-23T14:49:08.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fded93fc740 1 client.15432 shutdown 2024-04-23T14:49:08.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fded93fc740 1 -- 172.21.15.83:0/3071238331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 msgr2=0x556ce0be93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fded93fc740 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 0x556ce0be93d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdec0099650 tx=0x7fdec0099490 comp rx=0 tx=0).stop 2024-04-23T14:49:08.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fded93fc740 1 -- 172.21.15.83:0/3071238331 shutdown_connections 2024-04-23T14:49:08.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fded93fc740 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ce0bf9ba0 0x556ce0bfbf90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fded93fc740 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0be9940 0x556ce0be6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fded93fc740 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0be66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fdebd600640 0 client.15432 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fded93fc740 1 --2- 172.21.15.83:0/3071238331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0bde950 0x556ce0be93d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fded93fc740 1 -- 172.21.15.83:0/3071238331 >> 172.21.15.83:0/3071238331 conn(0x556ce0bdac70 msgr2=0x556ce0be0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fded93fc740 1 -- 172.21.15.83:0/3071238331 shutdown_connections 2024-04-23T14:49:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.828+0000 7fded93fc740 1 -- 172.21.15.83:0/3071238331 wait complete. 2024-04-23T14:49:08.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.829+0000 7fded93fc740 20 client.15432 trim_cache size 0 max 16384 2024-04-23T14:49:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.843+0000 7fded93fc740 1 Processor -- start 2024-04-23T14:49:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.843+0000 7fded93fc740 1 -- start start 2024-04-23T14:49:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fded93fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bde950 0x556ce0bf5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fded93fc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0bdf270 0x556ce0be8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fded93fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0be9940 0x556ce0c017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fded93fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fded01b2820 con 0x556ce0be9940 2024-04-23T14:49:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fded93fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fded01b00f0 con 0x556ce0bde950 2024-04-23T14:49:08.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fded93fc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fded01adf40 con 0x556ce0bdf270 2024-04-23T14:49:08.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fdecfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0be9940 0x556ce0c017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fdecfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0be9940 0x556ce0c017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52370/0 (socket says 172.21.15.83:52370) 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fdecfe00640 1 -- 172.21.15.83:0/2933838780 learned_addr learned my addr 172.21.15.83:0/2933838780 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fdecf400640 1 --2- 172.21.15.83:0/2933838780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bde950 0x556ce0bf5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fdec6a00640 1 --2- 172.21.15.83:0/2933838780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0bdf270 0x556ce0be8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fdec6a00640 1 -- 172.21.15.83:0/2933838780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bde950 msgr2=0x556ce0bf5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fdec6a00640 1 --2- 172.21.15.83:0/2933838780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bde950 0x556ce0bf5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fdec6a00640 1 -- 172.21.15.83:0/2933838780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0be9940 msgr2=0x556ce0c017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fdec6a00640 1 --2- 172.21.15.83:0/2933838780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0be9940 0x556ce0c017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fdec6a00640 1 -- 172.21.15.83:0/2933838780 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ce0be7120 con 0x556ce0bdf270 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.844+0000 7fdecf400640 1 --2- 172.21.15.83:0/2933838780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bde950 0x556ce0bf5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.845+0000 7fdec6a00640 1 --2- 172.21.15.83:0/2933838780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0bdf270 0x556ce0be8f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8004650 tx=0x7fdeb8004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d026ba0dfc7416b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.845+0000 7fdebfe00640 1 -- 172.21.15.83:0/2933838780 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8059070 con 0x556ce0bdf270 2024-04-23T14:49:08.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.845+0000 7fdebfe00640 1 -- 172.21.15.83:0/2933838780 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb80046c0 con 0x556ce0bdf270 2024-04-23T14:49:08.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.845+0000 7fdebfe00640 1 -- 172.21.15.83:0/2933838780 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8035040 con 0x556ce0bdf270 2024-04-23T14:49:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.846+0000 7fded93fc740 1 -- 172.21.15.83:0/2933838780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0bdf270 msgr2=0x556ce0be8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.846+0000 7fded93fc740 1 --2- 172.21.15.83:0/2933838780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0bdf270 0x556ce0be8f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8004650 tx=0x7fdeb8004af0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.846+0000 7fded93fc740 1 -- 172.21.15.83:0/2933838780 shutdown_connections 2024-04-23T14:49:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.846+0000 7fded93fc740 1 --2- 172.21.15.83:0/2933838780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0be9940 0x556ce0c017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.846+0000 7fded93fc740 1 --2- 172.21.15.83:0/2933838780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0bdf270 0x556ce0be8f00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.846+0000 7fded93fc740 1 --2- 172.21.15.83:0/2933838780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bde950 0x556ce0bf5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.846+0000 7fded93fc740 1 -- 172.21.15.83:0/2933838780 >> 172.21.15.83:0/2933838780 conn(0x556ce0bdac70 msgr2=0x556ce0be1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.846+0000 7fded93fc740 1 -- 172.21.15.83:0/2933838780 shutdown_connections 2024-04-23T14:49:08.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.846+0000 7fded93fc740 1 -- 172.21.15.83:0/2933838780 wait complete. 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.846+0000 7fded93fc740 1 -- start start 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fded93fc740 1 Processor -- start 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fded93fc740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0bde950 0x556ce0bea9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fded93fc740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0beaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fded93fc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0be9940 0x556ce0beb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fded93fc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fded01ae5b0 con 0x556ce0be9940 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fded93fc740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fded01afa20 con 0x556ce0bdf270 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fded93fc740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fded01aec80 con 0x556ce0bde950 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fded93fc740 10 client.0 _ll_register_callbacks cb 0x7fdec00174d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fdecfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0be9940 0x556ce0beb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0beaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fdec6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0beaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:52206/0 (socket says 172.21.15.83:52206) 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fdec6a00640 1 -- 172.21.15.83:0/4129150249 learned_addr learned my addr 172.21.15.83:0/4129150249 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:08.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.847+0000 7fdecf400640 1 --2- 172.21.15.83:0/4129150249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0bde950 0x556ce0bea9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.848+0000 7fdec6a00640 1 -- 172.21.15.83:0/4129150249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0bde950 msgr2=0x556ce0bea9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.848+0000 7fdec6a00640 1 --2- 172.21.15.83:0/4129150249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0bde950 0x556ce0bea9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.848+0000 7fdec6a00640 1 -- 172.21.15.83:0/4129150249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0be9940 msgr2=0x556ce0beb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.848+0000 7fdec6a00640 1 --2- 172.21.15.83:0/4129150249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0be9940 0x556ce0beb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.848+0000 7fdec6a00640 1 -- 172.21.15.83:0/4129150249 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ce09207d0 con 0x556ce0bdf270 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.848+0000 7fdec6a00640 1 --2- 172.21.15.83:0/4129150249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0beaef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80092a0 tx=0x7fdeb8009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de3754ea8c76415c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.848+0000 7fdebd600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.848+0000 7fdebd600640 1 -- 172.21.15.83:0/4129150249 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8035050 con 0x556ce0bdf270 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.848+0000 7fded93fc740 1 -- 172.21.15.83:0/4129150249 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb80046c0 con 0x556ce0bdf270 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.848+0000 7fdebd600640 1 -- 172.21.15.83:0/4129150249 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fdeb8004380 con 0x556ce0bdf270 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.849+0000 7fdebd600640 1 -- 172.21.15.83:0/4129150249 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdeb8059040 con 0x556ce0bdf270 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.849+0000 7fded93fc740 10 client.25541 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.849+0000 7fded93fc740 1 -- 172.21.15.83:0/4129150249 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556ce0c603f0 con 0x556ce0bdf270 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.849+0000 7fdebd600640 1 -- 172.21.15.83:0/4129150249 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb80046c0 con 0x556ce0bdf270 2024-04-23T14:49:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.849+0000 7fdebd600640 1 -- 172.21.15.83:0/4129150249 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fdeb805c070 con 0x556ce0bdf270 2024-04-23T14:49:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.849+0000 7fded93fc740 10 client.25541 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.849+0000 7fded93fc740 10 client.25541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.849+0000 7fded93fc740 20 client.25541 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.849+0000 7fded93fc740 1 --2- 172.21.15.83:0/4129150249 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ce0c606f0 0x556ce0c62ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.849+0000 7fded93fc740 4 client.25541 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:49:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.849+0000 7fded93fc740 1 -- 172.21.15.83:0/4129150249 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ce0b4ba60 con 0x556ce0c606f0 2024-04-23T14:49:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.850+0000 7fdecf400640 1 --2- 172.21.15.83:0/4129150249 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ce0c606f0 0x556ce0c62ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:08.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.850+0000 7fdebd600640 10 client.25541 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:08.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.850+0000 7fdecf400640 1 --2- 172.21.15.83:0/4129150249 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ce0c606f0 0x556ce0c62ae0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f86e9e306af1632 server_cookie=b16bfb8f0af3a784 in_seq=0 out_seq=0 2024-04-23T14:49:08.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.850+0000 7fdebd600640 1 -- 172.21.15.83:0/4129150249 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ce0b4ba60 con 0x556ce0c606f0 2024-04-23T14:49:08.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.850+0000 7fdebd600640 10 client.25541 handle_command_reply: tid=0 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.850+0000 7fded93fc740 1 client.25541 shutdown 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.850+0000 7fded93fc740 1 -- 172.21.15.83:0/4129150249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 msgr2=0x556ce0beaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.850+0000 7fded93fc740 1 --2- 172.21.15.83:0/4129150249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0beaef0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80092a0 tx=0x7fdeb8009520 comp rx=0 tx=0).stop 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded93fc740 1 -- 172.21.15.83:0/4129150249 shutdown_connections 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded93fc740 1 --2- 172.21.15.83:0/4129150249 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ce0c606f0 0x556ce0c62ae0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded93fc740 1 --2- 172.21.15.83:0/4129150249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ce0be9940 0x556ce0beb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded93fc740 1 --2- 172.21.15.83:0/4129150249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ce0bdf270 0x556ce0beaef0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fdebd600640 0 client.25541 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded93fc740 1 --2- 172.21.15.83:0/4129150249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ce0bde950 0x556ce0bea9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded93fc740 1 -- 172.21.15.83:0/4129150249 >> 172.21.15.83:0/4129150249 conn(0x556ce0bdac70 msgr2=0x556ce0bd9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded93fc740 1 -- 172.21.15.83:0/4129150249 shutdown_connections 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded93fc740 1 -- 172.21.15.83:0/4129150249 wait complete. 2024-04-23T14:49:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded93fc740 20 client.25541 trim_cache size 0 max 16384 2024-04-23T14:49:08.857 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:49:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded6600640 1 -- 172.21.15.83:0/1866489938 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdeb40a3ed0 msgr2=0x7fdeb40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded6600640 1 --2- 172.21.15.83:0/1866489938 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdeb40a3ed0 0x7fdeb40a6390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009ef0 tx=0x7fdec8005f90 comp rx=0 tx=0).stop 2024-04-23T14:49:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded6600640 1 -- 172.21.15.83:0/1866489938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fded0104450 msgr2=0x7fded01a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.851+0000 7fded6600640 1 --2- 172.21.15.83:0/1866489938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fded0104450 0x7fded01a0770 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdec000eb00 tx=0x7fdec000c2d0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.852+0000 7fded6600640 1 -- 172.21.15.83:0/1866489938 shutdown_connections 2024-04-23T14:49:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.852+0000 7fded6600640 1 --2- 172.21.15.83:0/1866489938 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdeb40a3ed0 0x7fdeb40a6390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.852+0000 7fded6600640 1 --2- 172.21.15.83:0/1866489938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fded019a920 0x7fded019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.852+0000 7fded6600640 1 --2- 172.21.15.83:0/1866489938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fded0109520 0x7fded01a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.852+0000 7fded6600640 1 --2- 172.21.15.83:0/1866489938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fded0104450 0x7fded01a0770 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.852+0000 7fded6600640 1 -- 172.21.15.83:0/1866489938 >> 172.21.15.83:0/1866489938 conn(0x7fded00ffc70 msgr2=0x7fded0101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.852+0000 7fded6600640 1 -- 172.21.15.83:0/1866489938 shutdown_connections 2024-04-23T14:49:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:08.852+0000 7fded6600640 1 -- 172.21.15.83:0/1866489938 wait complete. 2024-04-23T14:49:09.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:09 smithi173 ceph-mon[32443]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-23T14:49:09.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:09 smithi203 ceph-mon[32800]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-23T14:49:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:09 smithi083 ceph-mon[27867]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-23T14:49:09.871 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:49:09.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:49:09.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:49:10.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.086+0000 7fef04400640 1 -- 172.21.15.83:0/4026636202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feefc104df0 msgr2=0x7feefc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.086+0000 7fef04400640 1 --2- 172.21.15.83:0/4026636202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feefc104df0 0x7feefc108e30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feee8009900 tx=0x7feee802a3f0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.086+0000 7fef04400640 1 -- 172.21.15.83:0/4026636202 shutdown_connections 2024-04-23T14:49:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.086+0000 7fef04400640 1 --2- 172.21.15.83:0/4026636202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc109510 0x7feefc1110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.086+0000 7fef04400640 1 --2- 172.21.15.83:0/4026636202 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feefc104df0 0x7feefc108e30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.086+0000 7fef04400640 1 --2- 172.21.15.83:0/4026636202 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feefc104440 0x7feefc104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.086+0000 7fef04400640 1 -- 172.21.15.83:0/4026636202 >> 172.21.15.83:0/4026636202 conn(0x7feefc0ffc80 msgr2=0x7feefc1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.086+0000 7fef04400640 1 -- 172.21.15.83:0/4026636202 shutdown_connections 2024-04-23T14:49:10.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.086+0000 7fef04400640 1 -- 172.21.15.83:0/4026636202 wait complete. 2024-04-23T14:49:10.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.087+0000 7fef04400640 1 Processor -- start 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.087+0000 7fef04400640 1 -- start start 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef04400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feefc104440 0x7feefc1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef04400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feefc109510 0x7feefc1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef04400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc19a9d0 0x7feefc19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef04400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feefc114580 con 0x7feefc19a9d0 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef04400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feefc114400 con 0x7feefc104440 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef04400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feefc114700 con 0x7feefc109510 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef01a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feefc104440 0x7feefc1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef02400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc19a9d0 0x7feefc19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef01a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feefc104440 0x7feefc1a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:52222/0 (socket says 172.21.15.83:52222) 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef01a00640 1 -- 172.21.15.83:0/2616368839 learned_addr learned my addr 172.21.15.83:0/2616368839 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef01000640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feefc109510 0x7feefc1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef01a00640 1 -- 172.21.15.83:0/2616368839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feefc109510 msgr2=0x7feefc1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef01a00640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feefc109510 0x7feefc1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef01a00640 1 -- 172.21.15.83:0/2616368839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc19a9d0 msgr2=0x7feefc19ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef01a00640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc19a9d0 0x7feefc19ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef01a00640 1 -- 172.21.15.83:0/2616368839 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc19b4f0 con 0x7feefc104440 2024-04-23T14:49:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.088+0000 7fef02400640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc19a9d0 0x7feefc19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.089+0000 7fef01a00640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feefc104440 0x7feefc1a0780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feef000ea60 tx=0x7feef000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.089+0000 7feefa000640 1 -- 172.21.15.83:0/2616368839 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef000cd20 con 0x7feefc104440 2024-04-23T14:49:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.089+0000 7feefa000640 1 -- 172.21.15.83:0/2616368839 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef00035f0 con 0x7feefc104440 2024-04-23T14:49:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.089+0000 7feefa000640 1 -- 172.21.15.83:0/2616368839 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0010690 con 0x7feefc104440 2024-04-23T14:49:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.089+0000 7fef04400640 1 -- 172.21.15.83:0/2616368839 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc19b7e0 con 0x7feefc104440 2024-04-23T14:49:10.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.089+0000 7fef04400640 1 -- 172.21.15.83:0/2616368839 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc1a7940 con 0x7feefc104440 2024-04-23T14:49:10.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.092+0000 7fef0725a740 1 Processor -- start 2024-04-23T14:49:10.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7fef0725a740 1 -- start start 2024-04-23T14:49:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7feefa000640 1 -- 172.21.15.83:0/2616368839 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7feef00037a0 con 0x7feefc104440 2024-04-23T14:49:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7fef0725a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f63090 0x55c8c7f63470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7fef0725a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f6db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7fef0725a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f6e080 0x55c8c7f70470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7fef0725a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c8c7ec8e70 con 0x55c8c7f639b0 2024-04-23T14:49:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7fef0725a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c8c7e6f380 con 0x55c8c7f6e080 2024-04-23T14:49:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7fef0725a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c8c7ec4e30 con 0x55c8c7f63090 2024-04-23T14:49:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7fef01a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f6db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7fef01a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f6db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52416/0 (socket says 172.21.15.83:52416) 2024-04-23T14:49:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7fef02400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f63090 0x55c8c7f63470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7fef01a00640 1 -- 172.21.15.83:0/2321551059 learned_addr learned my addr 172.21.15.83:0/2321551059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:10.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.093+0000 7fef01000640 1 --2- 172.21.15.83:0/2321551059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f6e080 0x55c8c7f70470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.094+0000 7fef01000640 1 -- 172.21.15.83:0/2321551059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f63090 msgr2=0x55c8c7f63470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.094+0000 7fef01000640 1 --2- 172.21.15.83:0/2321551059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f63090 0x55c8c7f63470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.094+0000 7fef01000640 1 -- 172.21.15.83:0/2321551059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 msgr2=0x55c8c7f6db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.094+0000 7fef01000640 1 --2- 172.21.15.83:0/2321551059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f6db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.094+0000 7fef01000640 1 -- 172.21.15.83:0/2321551059 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c7ca37e0 con 0x55c8c7f6e080 2024-04-23T14:49:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.094+0000 7fef01000640 1 --2- 172.21.15.83:0/2321551059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f6e080 0x55c8c7f70470 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feee8006010 tx=0x7feee8005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac2e8490934b0113 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:10.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.094+0000 7feeeea00640 1 -- 172.21.15.83:0/2321551059 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8040070 con 0x55c8c7f6e080 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.094+0000 7feeeea00640 1 -- 172.21.15.83:0/2321551059 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee8005500 con 0x55c8c7f6e080 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.094+0000 7feeeea00640 1 -- 172.21.15.83:0/2321551059 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8035040 con 0x55c8c7f6e080 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.094+0000 7feefa000640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feee40a3f50 0x7feee40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.095+0000 7feefa000640 1 -- 172.21.15.83:0/2616368839 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7feef00d0d40 con 0x7feefc104440 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.095+0000 7fef02400640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feee40a3f50 0x7feee40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.095+0000 7fef0725a740 1 -- 172.21.15.83:0/2321551059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f6e080 msgr2=0x55c8c7f70470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.095+0000 7fef0725a740 1 --2- 172.21.15.83:0/2321551059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f6e080 0x55c8c7f70470 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feee8006010 tx=0x7feee8005bd0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.095+0000 7fef0725a740 1 -- 172.21.15.83:0/2321551059 shutdown_connections 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.095+0000 7fef0725a740 1 --2- 172.21.15.83:0/2321551059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f6e080 0x55c8c7f70470 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.095+0000 7fef0725a740 1 --2- 172.21.15.83:0/2321551059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f6db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.095+0000 7fef0725a740 1 --2- 172.21.15.83:0/2321551059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f63090 0x55c8c7f63470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.095+0000 7fef0725a740 1 -- 172.21.15.83:0/2321551059 >> 172.21.15.83:0/2321551059 conn(0x55c8c7f5f3b0 msgr2=0x55c8c7f5f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.095+0000 7fef0725a740 1 -- 172.21.15.83:0/2321551059 shutdown_connections 2024-04-23T14:49:10.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.095+0000 7fef02400640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feee40a3f50 0x7feee40a6410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feef4005fd0 tx=0x7feef4009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.096+0000 7fef0725a740 1 -- 172.21.15.83:0/2321551059 wait complete. 2024-04-23T14:49:10.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.096+0000 7fef0725a740 1 -- start start 2024-04-23T14:49:10.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.096+0000 7fef0725a740 1 Processor -- start 2024-04-23T14:49:10.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef0725a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f63090 0x55c8c7f6aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef0725a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f6b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef0725a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f6e080 0x55c8c7f6ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef0725a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c8c7f70e10 con 0x55c8c7f639b0 2024-04-23T14:49:10.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef0725a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c8c7e82660 con 0x55c8c7f63090 2024-04-23T14:49:10.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef0725a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c8c7f70f90 con 0x55c8c7f6e080 2024-04-23T14:49:10.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef0725a740 10 client.0 _ll_register_callbacks cb 0x55c8c7e70aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef01000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f6b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef01000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f6b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52432/0 (socket says 172.21.15.83:52432) 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef01000640 1 -- 172.21.15.83:0/2467869747 learned_addr learned my addr 172.21.15.83:0/2467869747 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef02400640 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f6e080 0x55c8c7f6ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef01a00640 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f63090 0x55c8c7f6aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef01000640 1 -- 172.21.15.83:0/2467869747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f6e080 msgr2=0x55c8c7f6ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef01000640 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f6e080 0x55c8c7f6ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef01000640 1 -- 172.21.15.83:0/2467869747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f63090 msgr2=0x55c8c7f6aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef01000640 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f63090 0x55c8c7f6aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.097+0000 7fef01000640 1 -- 172.21.15.83:0/2467869747 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c7d03f70 con 0x55c8c7f639b0 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.098+0000 7fef01a00640 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f63090 0x55c8c7f6aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.098+0000 7fef01000640 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f6b530 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feee8000c00 tx=0x7feee8004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06c6e7a5be30f02 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.098+0000 7feee3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:49:10.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.098+0000 7feee3400640 1 -- 172.21.15.83:0/2467869747 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8040070 con 0x55c8c7f639b0 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.098+0000 7feee3400640 1 -- 172.21.15.83:0/2467869747 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7feee8031870 con 0x55c8c7f639b0 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.098+0000 7fef0725a740 1 -- 172.21.15.83:0/2467869747 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee8005500 con 0x55c8c7f639b0 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.098+0000 7feee3400640 1 -- 172.21.15.83:0/2467869747 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feee8056050 con 0x55c8c7f639b0 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.098+0000 7fef0725a740 10 client.15450 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.098+0000 7fef0725a740 1 -- 172.21.15.83:0/2467869747 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c8c7b9b170 con 0x55c8c7f639b0 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.099+0000 7feee3400640 1 -- 172.21.15.83:0/2467869747 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee8005500 con 0x55c8c7f639b0 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.099+0000 7feee3400640 1 -- 172.21.15.83:0/2467869747 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7feee8003680 con 0x55c8c7f639b0 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.099+0000 7fef0725a740 10 client.15450 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.099+0000 7fef0725a740 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.099+0000 7fef0725a740 20 client.15450 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.099+0000 7fef0725a740 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c8c7f7e210 0x55c8c7f80600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.099+0000 7fef0725a740 4 client.15450 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.099+0000 7fef0725a740 1 -- 172.21.15.83:0/2467869747 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8c7ed6890 con 0x55c8c7f7e210 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.099+0000 7fef01a00640 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c8c7f7e210 0x55c8c7f80600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.100+0000 7feee3400640 10 client.15450 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.100+0000 7fef01a00640 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c8c7f7e210 0x55c8c7f80600 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a42745eb8c74b3 server_cookie=6db3983e9cd3b656 in_seq=0 out_seq=0 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.101+0000 7feee3400640 1 -- 172.21.15.83:0/2467869747 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8c7ed6890 con 0x55c8c7f7e210 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.101+0000 7feee3400640 10 client.15450 handle_command_reply: tid=0 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.101+0000 7fef0725a740 1 client.15450 shutdown 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.101+0000 7fef0725a740 1 -- 172.21.15.83:0/2467869747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 msgr2=0x55c8c7f6b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.101+0000 7fef0725a740 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f6b530 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feee8000c00 tx=0x7feee8004980 comp rx=0 tx=0).stop 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.101+0000 7fef0725a740 1 -- 172.21.15.83:0/2467869747 shutdown_connections 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.101+0000 7fef0725a740 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c8c7f7e210 0x55c8c7f80600 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.101+0000 7fef0725a740 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f6e080 0x55c8c7f6ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.101+0000 7feee3400640 0 client.15450 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.101+0000 7fef0725a740 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f6b530 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.101+0000 7fef0725a740 1 --2- 172.21.15.83:0/2467869747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f63090 0x55c8c7f6aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.101+0000 7fef0725a740 1 -- 172.21.15.83:0/2467869747 >> 172.21.15.83:0/2467869747 conn(0x55c8c7f5f3b0 msgr2=0x55c8c7f651b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:10.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.102+0000 7fef0725a740 1 -- 172.21.15.83:0/2467869747 shutdown_connections 2024-04-23T14:49:10.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.102+0000 7fef0725a740 1 -- 172.21.15.83:0/2467869747 wait complete. 2024-04-23T14:49:10.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.102+0000 7fef0725a740 20 client.15450 trim_cache size 0 max 16384 2024-04-23T14:49:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.120+0000 7fef0725a740 1 Processor -- start 2024-04-23T14:49:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.121+0000 7fef0725a740 1 -- start start 2024-04-23T14:49:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.121+0000 7fef0725a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f63090 0x55c8c7f69f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.121+0000 7fef0725a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f639b0 0x55c8c7f79cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.121+0000 7fef0725a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f6e080 0x55c8c7f85c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.121+0000 7fef0725a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1b27c0 con 0x55c8c7f63090 2024-04-23T14:49:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.121+0000 7fef0725a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1b0090 con 0x55c8c7f6e080 2024-04-23T14:49:10.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.121+0000 7fef0725a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1adee0 con 0x55c8c7f639b0 2024-04-23T14:49:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.121+0000 7fef01a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f63090 0x55c8c7f69f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.121+0000 7fef01a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f63090 0x55c8c7f69f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52442/0 (socket says 172.21.15.83:52442) 2024-04-23T14:49:10.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.121+0000 7fef01000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f639b0 0x55c8c7f79cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.121+0000 7fef01a00640 1 -- 172.21.15.83:0/12136704 learned_addr learned my addr 172.21.15.83:0/12136704 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.121+0000 7fef02400640 1 --2- 172.21.15.83:0/12136704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f6e080 0x55c8c7f85c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.122+0000 7fef01a00640 1 -- 172.21.15.83:0/12136704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f639b0 msgr2=0x55c8c7f79cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.122+0000 7fef01a00640 1 --2- 172.21.15.83:0/12136704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f639b0 0x55c8c7f79cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.122+0000 7fef01a00640 1 -- 172.21.15.83:0/12136704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f6e080 msgr2=0x55c8c7f85c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.122+0000 7fef01a00640 1 --2- 172.21.15.83:0/12136704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f6e080 0x55c8c7f85c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.122+0000 7fef01a00640 1 -- 172.21.15.83:0/12136704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c7f69540 con 0x55c8c7f63090 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.122+0000 7fef01a00640 1 --2- 172.21.15.83:0/12136704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f63090 0x55c8c7f69f30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feef001f000 tx=0x7feef00d47f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d31f05efe8c6bc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.122+0000 7feeee000640 1 -- 172.21.15.83:0/12136704 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef009c5f0 con 0x55c8c7f63090 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.122+0000 7feeee000640 1 -- 172.21.15.83:0/12136704 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef009d1b0 con 0x55c8c7f63090 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.122+0000 7feeee000640 1 -- 172.21.15.83:0/12136704 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef009c410 con 0x55c8c7f63090 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.123+0000 7fef0725a740 1 -- 172.21.15.83:0/12136704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f63090 msgr2=0x55c8c7f69f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.123+0000 7fef0725a740 1 --2- 172.21.15.83:0/12136704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f63090 0x55c8c7f69f30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feef001f000 tx=0x7feef00d47f0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.123+0000 7fef0725a740 1 -- 172.21.15.83:0/12136704 shutdown_connections 2024-04-23T14:49:10.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.123+0000 7fef0725a740 1 --2- 172.21.15.83:0/12136704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f6e080 0x55c8c7f85c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.123+0000 7fef0725a740 1 --2- 172.21.15.83:0/12136704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f639b0 0x55c8c7f79cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.123+0000 7fef0725a740 1 --2- 172.21.15.83:0/12136704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f63090 0x55c8c7f69f30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.123+0000 7fef0725a740 1 -- 172.21.15.83:0/12136704 >> 172.21.15.83:0/12136704 conn(0x55c8c7f5f3b0 msgr2=0x55c8c7f6b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.123+0000 7fef0725a740 1 -- 172.21.15.83:0/12136704 shutdown_connections 2024-04-23T14:49:10.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.123+0000 7fef0725a740 1 -- 172.21.15.83:0/12136704 wait complete. 2024-04-23T14:49:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.124+0000 7fef0725a740 1 -- start start 2024-04-23T14:49:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.124+0000 7fef0725a740 1 Processor -- start 2024-04-23T14:49:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.124+0000 7fef0725a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f63090 0x55c8c7f68070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.124+0000 7fef0725a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.124+0000 7fef0725a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f6e080 0x55c8c7fe34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.124+0000 7fef0725a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1ae550 con 0x55c8c7f639b0 2024-04-23T14:49:10.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.124+0000 7fef0725a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1af9c0 con 0x55c8c7f63090 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.124+0000 7fef0725a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feefc1aec20 con 0x55c8c7f6e080 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.124+0000 7fef0725a740 10 client.0 _ll_register_callbacks cb 0x7feef00d46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.124+0000 7fef01000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7fef01000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f685b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52456/0 (socket says 172.21.15.83:52456) 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7fef01000640 1 -- 172.21.15.83:0/1283192440 learned_addr learned my addr 172.21.15.83:0/1283192440 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7fef02400640 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f6e080 0x55c8c7fe34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7fef01a00640 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f63090 0x55c8c7f68070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7fef01000640 1 -- 172.21.15.83:0/1283192440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f6e080 msgr2=0x55c8c7fe34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7fef01000640 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f6e080 0x55c8c7fe34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7fef01000640 1 -- 172.21.15.83:0/1283192440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f63090 msgr2=0x55c8c7f68070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7fef01000640 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f63090 0x55c8c7f68070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7fef01000640 1 -- 172.21.15.83:0/1283192440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c7fe39e0 con 0x55c8c7f639b0 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7fef01a00640 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f63090 0x55c8c7f68070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:49:10.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7fef01000640 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f685b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55c8c7f6a850 tx=0x7feee8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473db04aa90c64b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7feee3400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7feee3400640 1 -- 172.21.15.83:0/1283192440 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee805e020 con 0x55c8c7f639b0 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.125+0000 7feee3400640 1 -- 172.21.15.83:0/1283192440 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7feee8004e10 con 0x55c8c7f639b0 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.126+0000 7fef0725a740 1 -- 172.21.15.83:0/1283192440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feef009d1b0 con 0x55c8c7f639b0 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.126+0000 7feee3400640 1 -- 172.21.15.83:0/1283192440 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feee8069050 con 0x55c8c7f639b0 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.126+0000 7fef0725a740 10 client.15456 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.126+0000 7fef0725a740 1 -- 172.21.15.83:0/1283192440 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55c8c7fe4ba0 con 0x55c8c7f639b0 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.126+0000 7feee3400640 1 -- 172.21.15.83:0/1283192440 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef009d1b0 con 0x55c8c7f639b0 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.126+0000 7feee3400640 1 -- 172.21.15.83:0/1283192440 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7feee8003e80 con 0x55c8c7f639b0 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.126+0000 7fef0725a740 10 client.15456 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.126+0000 7fef0725a740 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.126+0000 7fef0725a740 20 client.15456 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.126+0000 7fef0725a740 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c8c7fe5310 0x55c8c7fe7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.126+0000 7fef0725a740 4 client.15456 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:49:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.126+0000 7fef0725a740 1 -- 172.21.15.83:0/1283192440 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c8c7ed0700 con 0x55c8c7fe5310 2024-04-23T14:49:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.127+0000 7fef01a00640 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c8c7fe5310 0x55c8c7fe7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.127+0000 7fef01a00640 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c8c7fe5310 0x55c8c7fe7730 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e43106210c8988 server_cookie=b0e2d82c3633c359 in_seq=0 out_seq=0 2024-04-23T14:49:10.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.127+0000 7feee3400640 10 client.15456 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.128+0000 7feee3400640 1 -- 172.21.15.83:0/1283192440 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c8c7ed0700 con 0x55c8c7fe5310 2024-04-23T14:49:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.128+0000 7feee3400640 10 client.15456 handle_command_reply: tid=0 2024-04-23T14:49:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.128+0000 7fef0725a740 1 client.15456 shutdown 2024-04-23T14:49:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.129+0000 7fef0725a740 1 -- 172.21.15.83:0/1283192440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 msgr2=0x55c8c7f685b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.129+0000 7fef0725a740 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f685b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55c8c7f6a850 tx=0x7feee8003820 comp rx=0 tx=0).stop 2024-04-23T14:49:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.129+0000 7fef0725a740 1 -- 172.21.15.83:0/1283192440 shutdown_connections 2024-04-23T14:49:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.129+0000 7fef0725a740 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c8c7fe5310 0x55c8c7fe7730 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.129+0000 7fef0725a740 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c8c7f6e080 0x55c8c7fe34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.129+0000 7fef0725a740 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c8c7f639b0 0x55c8c7f685b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.129+0000 7fef0725a740 1 --2- 172.21.15.83:0/1283192440 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c8c7f63090 0x55c8c7f68070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.129+0000 7fef0725a740 1 -- 172.21.15.83:0/1283192440 >> 172.21.15.83:0/1283192440 conn(0x55c8c7f5f3b0 msgr2=0x55c8c7f647e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.129+0000 7feee3400640 0 client.15456 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.129+0000 7fef0725a740 1 -- 172.21.15.83:0/1283192440 shutdown_connections 2024-04-23T14:49:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.129+0000 7fef0725a740 1 -- 172.21.15.83:0/1283192440 wait complete. 2024-04-23T14:49:10.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.129+0000 7fef0725a740 20 client.15456 trim_cache size 0 max 16384 2024-04-23T14:49:10.134 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:49:10.134 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:49:10.135 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "c77f7352-4590-4578-8a95-181025a91108", 2024-04-23T14:49:10.135 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:49:10.135 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:49:10.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.130+0000 7fef04400640 1 -- 172.21.15.83:0/2616368839 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feee40a3f50 msgr2=0x7feee40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.130+0000 7fef04400640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feee40a3f50 0x7feee40a6410 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feef4005fd0 tx=0x7feef4009500 comp rx=0 tx=0).stop 2024-04-23T14:49:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.130+0000 7fef04400640 1 -- 172.21.15.83:0/2616368839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feefc104440 msgr2=0x7feefc1a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.130+0000 7fef04400640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feefc104440 0x7feefc1a0780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7feef000ea60 tx=0x7feef000ee60 comp rx=0 tx=0).stop 2024-04-23T14:49:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.130+0000 7fef04400640 1 -- 172.21.15.83:0/2616368839 shutdown_connections 2024-04-23T14:49:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.130+0000 7fef04400640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feee40a3f50 0x7feee40a6410 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.130+0000 7fef04400640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feefc19a9d0 0x7feefc19ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.130+0000 7fef04400640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feefc109510 0x7feefc1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.130+0000 7fef04400640 1 --2- 172.21.15.83:0/2616368839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feefc104440 0x7feefc1a0780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.130+0000 7fef04400640 1 -- 172.21.15.83:0/2616368839 >> 172.21.15.83:0/2616368839 conn(0x7feefc0ffc80 msgr2=0x7feefc1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.130+0000 7fef04400640 1 -- 172.21.15.83:0/2616368839 shutdown_connections 2024-04-23T14:49:10.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.130+0000 7fef04400640 1 -- 172.21.15.83:0/2616368839 wait complete. 2024-04-23T14:49:10.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:49:10.149 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:49:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.354+0000 7f7999200640 1 -- 172.21.15.83:0/3108616794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7994076530 msgr2=0x7f7994076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.354+0000 7f7999200640 1 --2- 172.21.15.83:0/3108616794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7994076530 0x7f7994076910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7984009930 tx=0x7f798402a440 comp rx=0 tx=0).stop 2024-04-23T14:49:10.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.354+0000 7f7999200640 1 -- 172.21.15.83:0/3108616794 shutdown_connections 2024-04-23T14:49:10.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.354+0000 7f7999200640 1 --2- 172.21.15.83:0/3108616794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7994112d40 0x7f7994115130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.354+0000 7f7999200640 1 --2- 172.21.15.83:0/3108616794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7994076e50 0x7f7994112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.354+0000 7f7999200640 1 --2- 172.21.15.83:0/3108616794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7994076530 0x7f7994076910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.354+0000 7f7999200640 1 -- 172.21.15.83:0/3108616794 >> 172.21.15.83:0/3108616794 conn(0x7f79940ffb70 msgr2=0x7f7994101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:10.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.354+0000 7f7999200640 1 -- 172.21.15.83:0/3108616794 shutdown_connections 2024-04-23T14:49:10.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.354+0000 7f7999200640 1 -- 172.21.15.83:0/3108616794 wait complete. 2024-04-23T14:49:10.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.355+0000 7f7999200640 1 Processor -- start 2024-04-23T14:49:10.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.355+0000 7f7999200640 1 -- start start 2024-04-23T14:49:10.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7999200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7994076530 0x7f799419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7999200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7994076e50 0x7f799419f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7999200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7994112d40 0x7f79941a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7999200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7994118900 con 0x7f7994076e50 2024-04-23T14:49:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7999200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7994118780 con 0x7f7994076530 2024-04-23T14:49:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7999200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7994118a80 con 0x7f7994112d40 2024-04-23T14:49:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7992a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7994076e50 0x7f799419f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7992a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7994076e50 0x7f799419f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52474/0 (socket says 172.21.15.83:52474) 2024-04-23T14:49:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7992a00640 1 -- 172.21.15.83:0/1455197856 learned_addr learned my addr 172.21.15.83:0/1455197856 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7993400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7994076530 0x7f799419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7993e00640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7994112d40 0x7f79941a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7992a00640 1 -- 172.21.15.83:0/1455197856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7994112d40 msgr2=0x7f79941a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7992a00640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7994112d40 0x7f79941a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7992a00640 1 -- 172.21.15.83:0/1455197856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7994076530 msgr2=0x7f799419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7992a00640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7994076530 0x7f799419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.356+0000 7f7992a00640 1 -- 172.21.15.83:0/1455197856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79941a3df0 con 0x7f7994076e50 2024-04-23T14:49:10.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.357+0000 7f7992a00640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7994076e50 0x7f799419f340 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f798000e9f0 tx=0x7f798000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.357+0000 7f798be00640 1 -- 172.21.15.83:0/1455197856 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798000cd80 con 0x7f7994076e50 2024-04-23T14:49:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.357+0000 7f798be00640 1 -- 172.21.15.83:0/1455197856 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7980005500 con 0x7f7994076e50 2024-04-23T14:49:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.357+0000 7f7999200640 1 -- 172.21.15.83:0/1455197856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79941a40e0 con 0x7f7994076e50 2024-04-23T14:49:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.357+0000 7f798be00640 1 -- 172.21.15.83:0/1455197856 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980010690 con 0x7f7994076e50 2024-04-23T14:49:10.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.357+0000 7f7999200640 1 -- 172.21.15.83:0/1455197856 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79941ab970 con 0x7f7994076e50 2024-04-23T14:49:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.360+0000 7f799d5b5740 1 Processor -- start 2024-04-23T14:49:10.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.361+0000 7f799d5b5740 1 -- start start 2024-04-23T14:49:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.361+0000 7f799d5b5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd317a30 0x5646dd317e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.361+0000 7f798be00640 1 -- 172.21.15.83:0/1455197856 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f79800046c0 con 0x7f7994076e50 2024-04-23T14:49:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.361+0000 7f799d5b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd318350 0x5646dd3224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.361+0000 7f799d5b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 0x5646dd324e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.361+0000 7f799d5b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646dd27d1f0 con 0x5646dd322a20 2024-04-23T14:49:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.361+0000 7f799d5b5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5646dd223350 con 0x5646dd317a30 2024-04-23T14:49:10.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.361+0000 7f799d5b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5646dd2791b0 con 0x5646dd318350 2024-04-23T14:49:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.361+0000 7f7992a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 0x5646dd324e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.361+0000 7f7992a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 0x5646dd324e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52482/0 (socket says 172.21.15.83:52482) 2024-04-23T14:49:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.361+0000 7f7992a00640 1 -- 172.21.15.83:0/1429645025 learned_addr learned my addr 172.21.15.83:0/1429645025 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.361+0000 7f7993e00640 1 --2- 172.21.15.83:0/1429645025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd317a30 0x5646dd317e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.362+0000 7f7992a00640 1 -- 172.21.15.83:0/1429645025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd318350 msgr2=0x5646dd3224e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.362+0000 7f7992a00640 1 --2- 172.21.15.83:0/1429645025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd318350 0x5646dd3224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.362+0000 7f7992a00640 1 -- 172.21.15.83:0/1429645025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd317a30 msgr2=0x5646dd317e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.362+0000 7f7992a00640 1 --2- 172.21.15.83:0/1429645025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd317a30 0x5646dd317e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.362+0000 7f7992a00640 1 -- 172.21.15.83:0/1429645025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646dd277f50 con 0x5646dd322a20 2024-04-23T14:49:10.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.362+0000 7f7992a00640 1 --2- 172.21.15.83:0/1429645025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 0x5646dd324e10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f79800ce350 tx=0x7f79800d4ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d7382ea5f6f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:10.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.362+0000 7f7988c00640 1 -- 172.21.15.83:0/1429645025 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79800e4070 con 0x5646dd322a20 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.362+0000 7f7988c00640 1 -- 172.21.15.83:0/1429645025 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79800d9bc0 con 0x5646dd322a20 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.362+0000 7f7988c00640 1 -- 172.21.15.83:0/1429645025 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79800de5a0 con 0x5646dd322a20 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.362+0000 7f798be00640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79780a3ed0 0x7f79780a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.363+0000 7f798be00640 1 -- 172.21.15.83:0/1455197856 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f79800d0ec0 con 0x7f7994076e50 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.363+0000 7f7993e00640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79780a3ed0 0x7f79780a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.363+0000 7f799d5b5740 1 -- 172.21.15.83:0/1429645025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 msgr2=0x5646dd324e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.363+0000 7f799d5b5740 1 --2- 172.21.15.83:0/1429645025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 0x5646dd324e10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f79800ce350 tx=0x7f79800d4ed0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.364+0000 7f799d5b5740 1 -- 172.21.15.83:0/1429645025 shutdown_connections 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.364+0000 7f799d5b5740 1 --2- 172.21.15.83:0/1429645025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 0x5646dd324e10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.364+0000 7f799d5b5740 1 --2- 172.21.15.83:0/1429645025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd318350 0x5646dd3224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.364+0000 7f799d5b5740 1 --2- 172.21.15.83:0/1429645025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd317a30 0x5646dd317e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.364+0000 7f799d5b5740 1 -- 172.21.15.83:0/1429645025 >> 172.21.15.83:0/1429645025 conn(0x5646dd313d50 msgr2=0x5646dd314130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.364+0000 7f7993e00640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79780a3ed0 0x7f79780a6390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f799406b460 tx=0x7f798c009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:10.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.364+0000 7f799d5b5740 1 -- 172.21.15.83:0/1429645025 shutdown_connections 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.364+0000 7f799d5b5740 1 -- 172.21.15.83:0/1429645025 wait complete. 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.364+0000 7f799d5b5740 1 -- start start 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.364+0000 7f799d5b5740 1 Processor -- start 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.365+0000 7f799d5b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd317a30 0x5646dd31f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.365+0000 7f799d5b5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd318350 0x5646dd31fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.365+0000 7f799d5b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 0x5646dd320420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.365+0000 7f799d5b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646dd325810 con 0x5646dd322a20 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.365+0000 7f799d5b5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5646dd236690 con 0x5646dd318350 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.365+0000 7f799d5b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5646dd325990 con 0x5646dd317a30 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.365+0000 7f799d5b5740 10 client.0 _ll_register_callbacks cb 0x5646dd272650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.365+0000 7f7993e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 0x5646dd320420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.365+0000 7f7993400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd317a30 0x5646dd31f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.365+0000 7f7992a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd318350 0x5646dd31fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.365+0000 7f7993e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 0x5646dd320420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52490/0 (socket says 172.21.15.83:52490) 2024-04-23T14:49:10.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.365+0000 7f7993e00640 1 -- 172.21.15.83:0/394273947 learned_addr learned my addr 172.21.15.83:0/394273947 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f7993e00640 1 -- 172.21.15.83:0/394273947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd317a30 msgr2=0x5646dd31f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f7993e00640 1 --2- 172.21.15.83:0/394273947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd317a30 0x5646dd31f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f7993e00640 1 -- 172.21.15.83:0/394273947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd318350 msgr2=0x5646dd31fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f7993e00640 1 --2- 172.21.15.83:0/394273947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd318350 0x5646dd31fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f7993e00640 1 -- 172.21.15.83:0/394273947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646dd235920 con 0x5646dd322a20 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f7993e00640 1 --2- 172.21.15.83:0/394273947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 0x5646dd320420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f798c009c30 tx=0x7f798c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fc38eca40bcabd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f797d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f797d600640 1 -- 172.21.15.83:0/394273947 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c004b20 con 0x5646dd322a20 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f797d600640 1 -- 172.21.15.83:0/394273947 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f798c004cc0 con 0x5646dd322a20 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f799d5b5740 1 -- 172.21.15.83:0/394273947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79800d9bc0 con 0x5646dd322a20 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f797d600640 1 -- 172.21.15.83:0/394273947 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f798c00f7d0 con 0x5646dd322a20 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f799d5b5740 10 client.15468 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:10.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.366+0000 7f799d5b5740 1 -- 172.21.15.83:0/394273947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5646dd0597d0 con 0x5646dd322a20 2024-04-23T14:49:10.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.367+0000 7f797d600640 1 -- 172.21.15.83:0/394273947 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79800d9bc0 con 0x5646dd322a20 2024-04-23T14:49:10.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.367+0000 7f797d600640 1 -- 172.21.15.83:0/394273947 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f798c026070 con 0x5646dd322a20 2024-04-23T14:49:10.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.367+0000 7f799d5b5740 10 client.15468 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:10.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.367+0000 7f799d5b5740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:10.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.367+0000 7f799d5b5740 20 client.15468 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.367+0000 7f799d5b5740 1 --2- 172.21.15.83:0/394273947 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5646dd332e60 0x5646dd335270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.367+0000 7f799d5b5740 4 client.15468 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.367+0000 7f799d5b5740 1 -- 172.21.15.83:0/394273947 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646dd28ac10 con 0x5646dd332e60 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.367+0000 7f7993400640 1 --2- 172.21.15.83:0/394273947 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5646dd332e60 0x5646dd335270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.368+0000 7f7993400640 1 --2- 172.21.15.83:0/394273947 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5646dd332e60 0x5646dd335270 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e363d98b5332be8f server_cookie=fb294fa13db47cf3 in_seq=0 out_seq=0 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.368+0000 7f797d600640 10 client.15468 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f797d600640 1 -- 172.21.15.83:0/394273947 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646dd28ac10 con 0x5646dd332e60 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f797d600640 10 client.15468 handle_command_reply: tid=0 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f799d5b5740 1 client.15468 shutdown 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f799d5b5740 1 -- 172.21.15.83:0/394273947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 msgr2=0x5646dd320420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f799d5b5740 1 --2- 172.21.15.83:0/394273947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 0x5646dd320420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f798c009c30 tx=0x7f798c004ab0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f799d5b5740 1 -- 172.21.15.83:0/394273947 shutdown_connections 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f799d5b5740 1 --2- 172.21.15.83:0/394273947 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5646dd332e60 0x5646dd335270 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f799d5b5740 1 --2- 172.21.15.83:0/394273947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd322a20 0x5646dd320420 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f799d5b5740 1 --2- 172.21.15.83:0/394273947 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd318350 0x5646dd31fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f799d5b5740 1 --2- 172.21.15.83:0/394273947 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd317a30 0x5646dd31f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f797d600640 0 client.15468 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f799d5b5740 1 -- 172.21.15.83:0/394273947 >> 172.21.15.83:0/394273947 conn(0x5646dd313d50 msgr2=0x5646dd312fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f799d5b5740 1 -- 172.21.15.83:0/394273947 shutdown_connections 2024-04-23T14:49:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f799d5b5740 1 -- 172.21.15.83:0/394273947 wait complete. 2024-04-23T14:49:10.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.369+0000 7f799d5b5740 20 client.15468 trim_cache size 0 max 16384 2024-04-23T14:49:10.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.388+0000 7f799d5b5740 1 Processor -- start 2024-04-23T14:49:10.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.388+0000 7f799d5b5740 1 -- start start 2024-04-23T14:49:10.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.388+0000 7f799d5b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd317a30 0x5646dd3201f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.388+0000 7f799d5b5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd318350 0x5646dd31a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.388+0000 7f799d5b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd322a20 0x5646dd33ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.388+0000 7f799d5b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79941b6b10 con 0x5646dd317a30 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.388+0000 7f799d5b5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f79941b43e0 con 0x5646dd318350 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.388+0000 7f799d5b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79941b2230 con 0x5646dd322a20 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.389+0000 7f7993400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd317a30 0x5646dd3201f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.389+0000 7f7993400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd317a30 0x5646dd3201f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52496/0 (socket says 172.21.15.83:52496) 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.389+0000 7f7993400640 1 -- 172.21.15.83:0/2625106611 learned_addr learned my addr 172.21.15.83:0/2625106611 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.389+0000 7f7992a00640 1 --2- 172.21.15.83:0/2625106611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd318350 0x5646dd31a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.389+0000 7f7993e00640 1 --2- 172.21.15.83:0/2625106611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd322a20 0x5646dd33ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.389+0000 7f7993400640 1 -- 172.21.15.83:0/2625106611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd322a20 msgr2=0x5646dd33ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.389+0000 7f7993400640 1 --2- 172.21.15.83:0/2625106611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd322a20 0x5646dd33ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.389+0000 7f7993400640 1 -- 172.21.15.83:0/2625106611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd318350 msgr2=0x5646dd31a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.389+0000 7f7993400640 1 --2- 172.21.15.83:0/2625106611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd318350 0x5646dd31a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.389+0000 7f7993400640 1 -- 172.21.15.83:0/2625106611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646dd325180 con 0x5646dd317a30 2024-04-23T14:49:10.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.389+0000 7f7992a00640 1 --2- 172.21.15.83:0/2625106611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd318350 0x5646dd31a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:49:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.389+0000 7f7993400640 1 --2- 172.21.15.83:0/2625106611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd317a30 0x5646dd3201f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7984031550 tx=0x7f7984009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150ae64ea901156d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.390+0000 7f797fe00640 1 -- 172.21.15.83:0/2625106611 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798404b070 con 0x5646dd317a30 2024-04-23T14:49:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.390+0000 7f797fe00640 1 -- 172.21.15.83:0/2625106611 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7984005520 con 0x5646dd317a30 2024-04-23T14:49:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.390+0000 7f797fe00640 1 -- 172.21.15.83:0/2625106611 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984035040 con 0x5646dd317a30 2024-04-23T14:49:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.390+0000 7f799d5b5740 1 -- 172.21.15.83:0/2625106611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd317a30 msgr2=0x5646dd3201f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.390+0000 7f799d5b5740 1 --2- 172.21.15.83:0/2625106611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd317a30 0x5646dd3201f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7984031550 tx=0x7f7984009260 comp rx=0 tx=0).stop 2024-04-23T14:49:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.390+0000 7f799d5b5740 1 -- 172.21.15.83:0/2625106611 shutdown_connections 2024-04-23T14:49:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.390+0000 7f799d5b5740 1 --2- 172.21.15.83:0/2625106611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd322a20 0x5646dd33ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.390+0000 7f799d5b5740 1 --2- 172.21.15.83:0/2625106611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd318350 0x5646dd31a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.390+0000 7f799d5b5740 1 --2- 172.21.15.83:0/2625106611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd317a30 0x5646dd3201f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.390+0000 7f799d5b5740 1 -- 172.21.15.83:0/2625106611 >> 172.21.15.83:0/2625106611 conn(0x5646dd313d50 msgr2=0x5646dd312740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.391+0000 7f799d5b5740 1 -- 172.21.15.83:0/2625106611 shutdown_connections 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.391+0000 7f799d5b5740 1 -- 172.21.15.83:0/2625106611 wait complete. 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.391+0000 7f799d5b5740 1 -- start start 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.391+0000 7f799d5b5740 1 Processor -- start 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f799d5b5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd317a30 0x5646dd31c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f799d5b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd318350 0x5646dd31cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f799d5b5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd322a20 0x5646dd3983b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f799d5b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79941b28a0 con 0x5646dd318350 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f799d5b5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f79941b3d10 con 0x5646dd322a20 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f799d5b5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79941b2f70 con 0x5646dd317a30 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f799d5b5740 10 client.0 _ll_register_callbacks cb 0x7f798c009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f7992a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd318350 0x5646dd31cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f7992a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd318350 0x5646dd31cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52508/0 (socket says 172.21.15.83:52508) 2024-04-23T14:49:10.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f7992a00640 1 -- 172.21.15.83:0/669434007 learned_addr learned my addr 172.21.15.83:0/669434007 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f7993e00640 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd322a20 0x5646dd3983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f7993400640 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd317a30 0x5646dd31c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f7992a00640 1 -- 172.21.15.83:0/669434007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd317a30 msgr2=0x5646dd31c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f7992a00640 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd317a30 0x5646dd31c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f7992a00640 1 -- 172.21.15.83:0/669434007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd322a20 msgr2=0x5646dd3983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f7992a00640 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd322a20 0x5646dd3983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.392+0000 7f7992a00640 1 -- 172.21.15.83:0/669434007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646dcf8f3f0 con 0x5646dd318350 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f7992a00640 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd318350 0x5646dd31cee0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7980099090 tx=0x7f798000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b848d48289ceff71 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f797d600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f797d600640 1 -- 172.21.15.83:0/669434007 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980087040 con 0x5646dd318350 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f797d600640 1 -- 172.21.15.83:0/669434007 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f79800bd0b0 con 0x5646dd318350 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f799d5b5740 1 -- 172.21.15.83:0/669434007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7984005520 con 0x5646dd318350 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f797d600640 1 -- 172.21.15.83:0/669434007 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7980098ed0 con 0x5646dd318350 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f799d5b5740 10 client.15474 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:10.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f799d5b5740 1 -- 172.21.15.83:0/669434007 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5646dcf4f0f0 con 0x5646dd318350 2024-04-23T14:49:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f797d600640 1 -- 172.21.15.83:0/669434007 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7984005520 con 0x5646dd318350 2024-04-23T14:49:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f797d600640 1 -- 172.21.15.83:0/669434007 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f798009ce90 con 0x5646dd318350 2024-04-23T14:49:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f799d5b5740 10 client.15474 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f799d5b5740 10 client.15474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f799d5b5740 20 client.15474 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f799d5b5740 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5646dd399990 0x5646dd39bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f799d5b5740 4 client.15474 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:49:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.393+0000 7f799d5b5740 1 -- 172.21.15.83:0/669434007 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646dd284a80 con 0x5646dd399990 2024-04-23T14:49:10.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.394+0000 7f7993400640 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5646dd399990 0x5646dd39bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:10.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.394+0000 7f7993400640 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5646dd399990 0x5646dd39bdb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4718b69132924870 server_cookie=f191705cc08ddbcf in_seq=0 out_seq=0 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.394+0000 7f797d600640 10 client.15474 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f797d600640 1 -- 172.21.15.83:0/669434007 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5646dd284a80 con 0x5646dd399990 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f797d600640 10 client.15474 handle_command_reply: tid=0 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f799d5b5740 1 client.15474 shutdown 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f799d5b5740 1 -- 172.21.15.83:0/669434007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd318350 msgr2=0x5646dd31cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f799d5b5740 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd318350 0x5646dd31cee0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7980099090 tx=0x7f798000c7d0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f799d5b5740 1 -- 172.21.15.83:0/669434007 shutdown_connections 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f799d5b5740 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5646dd399990 0x5646dd39bdb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f799d5b5740 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5646dd322a20 0x5646dd3983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f799d5b5740 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646dd318350 0x5646dd31cee0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f799d5b5740 1 --2- 172.21.15.83:0/669434007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5646dd317a30 0x5646dd31c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f799d5b5740 1 -- 172.21.15.83:0/669434007 >> 172.21.15.83:0/669434007 conn(0x5646dd313d50 msgr2=0x5646dd320070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f797d600640 0 client.15474 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:10.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f799d5b5740 1 -- 172.21.15.83:0/669434007 shutdown_connections 2024-04-23T14:49:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f799d5b5740 1 -- 172.21.15.83:0/669434007 wait complete. 2024-04-23T14:49:10.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.395+0000 7f799d5b5740 20 client.15474 trim_cache size 0 max 16384 2024-04-23T14:49:10.401 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:49:10.401 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:49:10.401 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:49:10.401 INFO:teuthology.orchestra.run.smithi083.stdout: "c77f7352-4590-4578-8a95-181025a91108": { 2024-04-23T14:49:10.401 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:49:10.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "c77f7352-4590-4578-8a95-181025a91108", 2024-04-23T14:49:10.401 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:49:10.401 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:49:10.401 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:49:10.402 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:49:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.396+0000 7f7999200640 1 -- 172.21.15.83:0/1455197856 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79780a3ed0 msgr2=0x7f79780a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.396+0000 7f7999200640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79780a3ed0 0x7f79780a6390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f799406b460 tx=0x7f798c009500 comp rx=0 tx=0).stop 2024-04-23T14:49:10.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.396+0000 7f7999200640 1 -- 172.21.15.83:0/1455197856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7994076e50 msgr2=0x7f799419f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:10.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.396+0000 7f7999200640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7994076e50 0x7f799419f340 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f798000e9f0 tx=0x7f798000edf0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.396+0000 7f7999200640 1 -- 172.21.15.83:0/1455197856 shutdown_connections 2024-04-23T14:49:10.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.396+0000 7f7999200640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7994112d40 0x7f79941a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.396+0000 7f7999200640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79780a3ed0 0x7f79780a6390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.396+0000 7f7999200640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7994076e50 0x7f799419f340 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.396+0000 7f7999200640 1 --2- 172.21.15.83:0/1455197856 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7994076530 0x7f799419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:10.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.396+0000 7f7999200640 1 -- 172.21.15.83:0/1455197856 >> 172.21.15.83:0/1455197856 conn(0x7f79940ffb70 msgr2=0x7f79941139b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:10.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.396+0000 7f7999200640 1 -- 172.21.15.83:0/1455197856 shutdown_connections 2024-04-23T14:49:10.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:10.396+0000 7f7999200640 1 -- 172.21.15.83:0/1455197856 wait complete. 2024-04-23T14:49:10.415 INFO:tasks.cephfs.filesystem:scrub status for tag:c77f7352-4590-4578-8a95-181025a91108 - {'path': '/', 'tag': 'c77f7352-4590-4578-8a95-181025a91108', 'options': 'recursive,force'} 2024-04-23T14:49:11.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:11 smithi173 ceph-mon[32443]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-23T14:49:11.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:11 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:49:11.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:11 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:49:11.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:11 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:49:11.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:11 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:49:11.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:11 smithi203 ceph-mon[32800]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-23T14:49:11.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:11 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:49:11.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:11 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:49:11.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:11 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:49:11.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:11 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:49:11.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:11 smithi083 ceph-mon[27867]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-23T14:49:11.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:11 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:49:11.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:11 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:49:11.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:11 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:49:11.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:11 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:49:13.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:13 smithi173 ceph-mon[32443]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-23T14:49:13.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:13 smithi203 ceph-mon[32800]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-23T14:49:13.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:13 smithi083 ceph-mon[27867]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-23T14:49:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:14 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:49:14.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:14 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:49:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:14 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:49:15.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:15 smithi203 ceph-mon[32800]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-23T14:49:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:15 smithi083 ceph-mon[27867]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-23T14:49:15.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:15 smithi173 ceph-mon[32443]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-23T14:49:17.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:17 smithi203 ceph-mon[32800]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-23T14:49:17.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:17 smithi083 ceph-mon[27867]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-23T14:49:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:17 smithi173 ceph-mon[32443]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-23T14:49:19.627 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:19 smithi173 ceph-mon[32443]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-23T14:49:19.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:19 smithi203 ceph-mon[32800]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-23T14:49:19.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:19 smithi083 ceph-mon[27867]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-23T14:49:21.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:21 smithi203 ceph-mon[32800]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-23T14:49:21.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:21 smithi083 ceph-mon[27867]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-23T14:49:21.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:21 smithi173 ceph-mon[32443]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-23T14:49:23.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:23 smithi203 ceph-mon[32800]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-23T14:49:23.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:23 smithi083 ceph-mon[27867]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-23T14:49:23.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:23 smithi173 ceph-mon[32443]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-23T14:49:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:25 smithi203 ceph-mon[32800]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-23T14:49:25.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:25 smithi083 ceph-mon[27867]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-23T14:49:25.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:25 smithi173 ceph-mon[32443]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-23T14:49:27.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:27 smithi203 ceph-mon[32800]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 6 op/s 2024-04-23T14:49:27.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:27 smithi083 ceph-mon[27867]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 6 op/s 2024-04-23T14:49:27.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:27 smithi173 ceph-mon[32443]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 852 B/s wr, 6 op/s 2024-04-23T14:49:29.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:29 smithi203 ceph-mon[32800]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-23T14:49:29.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:29 smithi083 ceph-mon[27867]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-23T14:49:29.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:29 smithi173 ceph-mon[32443]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 852 B/s wr, 5 op/s 2024-04-23T14:49:31.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:31 smithi203 ceph-mon[32800]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-23T14:49:31.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:31 smithi083 ceph-mon[27867]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-23T14:49:31.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:31 smithi173 ceph-mon[32443]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 852 B/s wr, 7 op/s 2024-04-23T14:49:33.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:33 smithi203 ceph-mon[32800]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:49:33.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:33 smithi083 ceph-mon[27867]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:49:33.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:33 smithi173 ceph-mon[32443]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:49:35.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:35 smithi203 ceph-mon[32800]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:49:35.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:35 smithi083 ceph-mon[27867]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:49:35.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:35 smithi173 ceph-mon[32443]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:49:37.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:37 smithi203 ceph-mon[32800]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T14:49:37.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:37 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:49:37.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:37 smithi083 ceph-mon[27867]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T14:49:37.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:37 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:49:37.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:37 smithi173 ceph-mon[32443]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T14:49:37.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:37 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:49:39.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:39 smithi203 ceph-mon[32800]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-23T14:49:39.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:39 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:49:39.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:39 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:49:39.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:39 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:49:39.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:39 smithi083 ceph-mon[27867]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-23T14:49:39.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:39 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:49:39.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:39 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:49:39.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:39 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:49:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:39 smithi173 ceph-mon[32443]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-23T14:49:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:39 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:49:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:39 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:49:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:39 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:49:40.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:49:40.416 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:49:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.595+0000 7fa3cfe00640 1 -- 172.21.15.83:0/4016887269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3d01093f0 msgr2=0x7fa3d0110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.595+0000 7fa3cfe00640 1 --2- 172.21.15.83:0/4016887269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3d01093f0 0x7fa3d0110fc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009ca0 tx=0x7fa3c802a8c0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.595+0000 7fa3cfe00640 1 -- 172.21.15.83:0/4016887269 shutdown_connections 2024-04-23T14:49:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.595+0000 7fa3cfe00640 1 --2- 172.21.15.83:0/4016887269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3d01093f0 0x7fa3d0110fc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.595+0000 7fa3cfe00640 1 --2- 172.21.15.83:0/4016887269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3d0104cd0 0x7fa3d0108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.595+0000 7fa3cfe00640 1 --2- 172.21.15.83:0/4016887269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d0104320 0x7fa3d0104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.595+0000 7fa3cfe00640 1 -- 172.21.15.83:0/4016887269 >> 172.21.15.83:0/4016887269 conn(0x7fa3d00ffb40 msgr2=0x7fa3d0101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:40.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.596+0000 7fa3cfe00640 1 -- 172.21.15.83:0/4016887269 shutdown_connections 2024-04-23T14:49:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.596+0000 7fa3cfe00640 1 -- 172.21.15.83:0/4016887269 wait complete. 2024-04-23T14:49:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.597+0000 7fa3cfe00640 1 Processor -- start 2024-04-23T14:49:40.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.597+0000 7fa3cfe00640 1 -- start start 2024-04-23T14:49:40.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.598+0000 7fa3cfe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3d0104320 0x7fa3d01a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.598+0000 7fa3cfe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d01093f0 0x7fa3d01a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.598+0000 7fa3cfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3d019a790 0x7fa3d019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.598+0000 7fa3cfe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0114490 con 0x7fa3d019a790 2024-04-23T14:49:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.598+0000 7fa3cfe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0114310 con 0x7fa3d0104320 2024-04-23T14:49:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.598+0000 7fa3cfe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0114610 con 0x7fa3d01093f0 2024-04-23T14:49:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.598+0000 7fa3cea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3d0104320 0x7fa3d01a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.598+0000 7fa3cea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3d0104320 0x7fa3d01a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37168/0 (socket says 172.21.15.83:37168) 2024-04-23T14:49:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.598+0000 7fa3cea00640 1 -- 172.21.15.83:0/148239784 learned_addr learned my addr 172.21.15.83:0/148239784 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.598+0000 7fa3cf400640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3d019a790 0x7fa3d019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.598+0000 7fa3ce000640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d01093f0 0x7fa3d01a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.599+0000 7fa3cea00640 1 -- 172.21.15.83:0/148239784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d01093f0 msgr2=0x7fa3d01a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.599+0000 7fa3cea00640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d01093f0 0x7fa3d01a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.599+0000 7fa3cea00640 1 -- 172.21.15.83:0/148239784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3d019a790 msgr2=0x7fa3d019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.599+0000 7fa3cea00640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3d019a790 0x7fa3d019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.599+0000 7fa3cea00640 1 -- 172.21.15.83:0/148239784 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d019b320 con 0x7fa3d0104320 2024-04-23T14:49:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.599+0000 7fa3cea00640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3d0104320 0x7fa3d01a0620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000ea60 tx=0x7fa3c000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.599+0000 7fa3d4a00640 1 -- 172.21.15.83:0/148239784 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000cde0 con 0x7fa3d0104320 2024-04-23T14:49:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.599+0000 7fa3cfe00640 1 -- 172.21.15.83:0/148239784 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d019b610 con 0x7fa3d0104320 2024-04-23T14:49:40.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.599+0000 7fa3d4a00640 1 -- 172.21.15.83:0/148239784 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c00035f0 con 0x7fa3d0104320 2024-04-23T14:49:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.599+0000 7fa3cfe00640 1 -- 172.21.15.83:0/148239784 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d01a7840 con 0x7fa3d0104320 2024-04-23T14:49:40.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.600+0000 7fa3d4a00640 1 -- 172.21.15.83:0/148239784 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0010670 con 0x7fa3d0104320 2024-04-23T14:49:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.603+0000 7fa3d8b73740 1 Processor -- start 2024-04-23T14:49:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.603+0000 7fa3d8b73740 1 -- start start 2024-04-23T14:49:40.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.603+0000 7fa3d8b73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a229e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.603+0000 7fa3d8b73740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 0x56503a2344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.603+0000 7fa3d8b73740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a234a20 0x56503a236e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.603+0000 7fa3d8b73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56503a18f1f0 con 0x56503a229a30 2024-04-23T14:49:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.603+0000 7fa3d8b73740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56503a135350 con 0x56503a234a20 2024-04-23T14:49:40.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.603+0000 7fa3d8b73740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56503a18b1b0 con 0x56503a22a350 2024-04-23T14:49:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.604+0000 7fa3cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a229e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.604+0000 7fa3ce000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a234a20 0x56503a236e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.604+0000 7fa3cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a229e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50604/0 (socket says 172.21.15.83:50604) 2024-04-23T14:49:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.604+0000 7fa3cf400640 1 -- 172.21.15.83:0/3320650102 learned_addr learned my addr 172.21.15.83:0/3320650102 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.604+0000 7fa3d4a00640 1 -- 172.21.15.83:0/148239784 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fa3c0010810 con 0x7fa3d0104320 2024-04-23T14:49:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.604+0000 7fa3ce000640 1 -- 172.21.15.83:0/3320650102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 msgr2=0x56503a2344e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T14:49:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.604+0000 7fa3ce000640 1 --2- 172.21.15.83:0/3320650102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 0x56503a2344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.604+0000 7fa3ce000640 1 -- 172.21.15.83:0/3320650102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 msgr2=0x56503a229e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:40.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.604+0000 7fa3ce000640 1 --2- 172.21.15.83:0/3320650102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a229e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.604+0000 7fa3ce000640 1 -- 172.21.15.83:0/3320650102 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503a189f50 con 0x56503a234a20 2024-04-23T14:49:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.605+0000 7fa3ce000640 1 --2- 172.21.15.83:0/3320650102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a234a20 0x56503a236e10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa3d019bae0 tx=0x7fa3bc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=273a0e110bfb6f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.605+0000 7fa3bbe00640 1 -- 172.21.15.83:0/3320650102 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc004050 con 0x56503a234a20 2024-04-23T14:49:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.605+0000 7fa3bbe00640 1 -- 172.21.15.83:0/3320650102 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3bc005500 con 0x56503a234a20 2024-04-23T14:49:40.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.605+0000 7fa3bbe00640 1 -- 172.21.15.83:0/3320650102 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc004d40 con 0x56503a234a20 2024-04-23T14:49:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.606+0000 7fa3d8b73740 1 -- 172.21.15.83:0/3320650102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a234a20 msgr2=0x56503a236e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.606+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/3320650102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a234a20 0x56503a236e10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa3d019bae0 tx=0x7fa3bc011810 comp rx=0 tx=0).stop 2024-04-23T14:49:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.606+0000 7fa3d8b73740 1 -- 172.21.15.83:0/3320650102 shutdown_connections 2024-04-23T14:49:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.606+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/3320650102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a234a20 0x56503a236e10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.606+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/3320650102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 0x56503a2344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.606+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/3320650102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a229e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.606+0000 7fa3d8b73740 1 -- 172.21.15.83:0/3320650102 >> 172.21.15.83:0/3320650102 conn(0x56503a225d50 msgr2=0x56503a226130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.606+0000 7fa3d8b73740 1 -- 172.21.15.83:0/3320650102 shutdown_connections 2024-04-23T14:49:40.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.606+0000 7fa3d8b73740 1 -- 172.21.15.83:0/3320650102 wait complete. 2024-04-23T14:49:40.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.606+0000 7fa3d4a00640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa3b40a3e60 0x7fa3b40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.607+0000 7fa3d4a00640 1 -- 172.21.15.83:0/148239784 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa3c00d0c80 con 0x7fa3d0104320 2024-04-23T14:49:40.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.607+0000 7fa3d8b73740 1 -- start start 2024-04-23T14:49:40.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.607+0000 7fa3ce000640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa3b40a3e60 0x7fa3b40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.607+0000 7fa3d8b73740 1 Processor -- start 2024-04-23T14:49:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.607+0000 7fa3d8b73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a2319c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.607+0000 7fa3d8b73740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a22a350 0x56503a231f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.607+0000 7fa3d8b73740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a234a20 0x56503a232440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3d8b73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56503a237810 con 0x56503a229a30 2024-04-23T14:49:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3d8b73740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56503a148690 con 0x56503a22a350 2024-04-23T14:49:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3d8b73740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56503a237990 con 0x56503a234a20 2024-04-23T14:49:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3d8b73740 10 client.0 _ll_register_callbacks cb 0x56503a184650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3ce000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a22a350 0x56503a231f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3ce000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a22a350 0x56503a231f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37184/0 (socket says 172.21.15.83:37184) 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3ce000640 1 -- 172.21.15.83:0/1899664296 learned_addr learned my addr 172.21.15.83:0/1899664296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3ce000640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa3b40a3e60 0x7fa3b40a6320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc01b040 tx=0x7fa3bc0048e0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3cea00640 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a2319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3cf400640 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a234a20 0x56503a232440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3ce000640 1 -- 172.21.15.83:0/1899664296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a234a20 msgr2=0x56503a232440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3ce000640 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a234a20 0x56503a232440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3ce000640 1 -- 172.21.15.83:0/1899664296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 msgr2=0x56503a2319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3ce000640 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a2319c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3ce000640 1 -- 172.21.15.83:0/1899664296 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56503a147920 con 0x56503a22a350 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3cf400640 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a234a20 0x56503a232440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.608+0000 7fa3cea00640 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a2319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.609+0000 7fa3ce000640 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a22a350 0x56503a231f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc01a000 tx=0x7fa3bc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32d3ca3c11b5e0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.609+0000 7fa3b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.609+0000 7fa3b8c00640 1 -- 172.21.15.83:0/1899664296 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc034020 con 0x56503a22a350 2024-04-23T14:49:40.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.609+0000 7fa3d8b73740 1 -- 172.21.15.83:0/1899664296 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3bc005500 con 0x56503a22a350 2024-04-23T14:49:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.609+0000 7fa3b8c00640 1 -- 172.21.15.83:0/1899664296 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa3bc003d70 con 0x56503a22a350 2024-04-23T14:49:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.610+0000 7fa3b8c00640 1 -- 172.21.15.83:0/1899664296 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa3bc03f050 con 0x56503a22a350 2024-04-23T14:49:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.610+0000 7fa3d8b73740 10 client.25589 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.610+0000 7fa3d8b73740 1 -- 172.21.15.83:0/1899664296 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x565039f6b7d0 con 0x56503a22a350 2024-04-23T14:49:40.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.610+0000 7fa3b8c00640 1 -- 172.21.15.83:0/1899664296 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3bc005500 con 0x56503a22a350 2024-04-23T14:49:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.611+0000 7fa3b8c00640 1 -- 172.21.15.83:0/1899664296 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa3bc0306f0 con 0x56503a22a350 2024-04-23T14:49:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.611+0000 7fa3d8b73740 10 client.25589 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.611+0000 7fa3d8b73740 10 client.25589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.611+0000 7fa3d8b73740 20 client.25589 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.611+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56503a244d50 0x56503a247140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.611+0000 7fa3d8b73740 4 client.25589 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:49:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.611+0000 7fa3d8b73740 1 -- 172.21.15.83:0/1899664296 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56503a19cc10 con 0x56503a244d50 2024-04-23T14:49:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.612+0000 7fa3cea00640 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56503a244d50 0x56503a247140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.612+0000 7fa3cea00640 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56503a244d50 0x56503a247140 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4218cde64c8980e1 server_cookie=78bba9e9ee339fba in_seq=0 out_seq=0 2024-04-23T14:49:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.612+0000 7fa3b8c00640 10 client.25589 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.613+0000 7fa3b8c00640 1 -- 172.21.15.83:0/1899664296 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56503a19cc10 con 0x56503a244d50 2024-04-23T14:49:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.613+0000 7fa3b8c00640 10 client.25589 handle_command_reply: tid=0 2024-04-23T14:49:40.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.613+0000 7fa3d8b73740 1 client.25589 shutdown 2024-04-23T14:49:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.613+0000 7fa3d8b73740 1 -- 172.21.15.83:0/1899664296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a22a350 msgr2=0x56503a231f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.613+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a22a350 0x56503a231f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc01a000 tx=0x7fa3bc011810 comp rx=0 tx=0).stop 2024-04-23T14:49:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.613+0000 7fa3d8b73740 1 -- 172.21.15.83:0/1899664296 shutdown_connections 2024-04-23T14:49:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.614+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56503a244d50 0x56503a247140 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.614+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a234a20 0x56503a232440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.614+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a22a350 0x56503a231f00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.614+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/1899664296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a2319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.614+0000 7fa3d8b73740 1 -- 172.21.15.83:0/1899664296 >> 172.21.15.83:0/1899664296 conn(0x56503a225d50 msgr2=0x56503a224fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.614+0000 7fa3b8c00640 0 client.25589 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:40.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.614+0000 7fa3d8b73740 1 -- 172.21.15.83:0/1899664296 shutdown_connections 2024-04-23T14:49:40.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.614+0000 7fa3d8b73740 1 -- 172.21.15.83:0/1899664296 wait complete. 2024-04-23T14:49:40.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.614+0000 7fa3d8b73740 20 client.25589 trim_cache size 0 max 16384 2024-04-23T14:49:40.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.650+0000 7fa3d8b73740 1 Processor -- start 2024-04-23T14:49:40.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.650+0000 7fa3d8b73740 1 -- start start 2024-04-23T14:49:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.650+0000 7fa3d8b73740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a229a30 0x56503a2473d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.650+0000 7fa3d8b73740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 0x56503a240730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.650+0000 7fa3d8b73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a234a20 0x56503a2302d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.650+0000 7fa3d8b73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b0011010 con 0x56503a234a20 2024-04-23T14:49:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.650+0000 7fa3d8b73740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b000e8e0 con 0x56503a229a30 2024-04-23T14:49:40.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.650+0000 7fa3d8b73740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b000c730 con 0x56503a22a350 2024-04-23T14:49:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.651+0000 7fa3cea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a229a30 0x56503a2473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.651+0000 7fa3cea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a229a30 0x56503a2473d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37192/0 (socket says 172.21.15.83:37192) 2024-04-23T14:49:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.651+0000 7fa3cea00640 1 -- 172.21.15.83:0/278791538 learned_addr learned my addr 172.21.15.83:0/278791538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.651+0000 7fa3ce000640 1 --2- 172.21.15.83:0/278791538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 0x56503a240730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.651+0000 7fa3cea00640 1 -- 172.21.15.83:0/278791538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 msgr2=0x56503a240730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.651+0000 7fa3cea00640 1 --2- 172.21.15.83:0/278791538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 0x56503a240730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.651+0000 7fa3cea00640 1 -- 172.21.15.83:0/278791538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a234a20 msgr2=0x56503a2302d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.651+0000 7fa3cf400640 1 --2- 172.21.15.83:0/278791538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a234a20 0x56503a2302d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.651+0000 7fa3cea00640 1 --2- 172.21.15.83:0/278791538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a234a20 0x56503a2302d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.651+0000 7fa3cea00640 1 -- 172.21.15.83:0/278791538 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565039ea13f0 con 0x56503a229a30 2024-04-23T14:49:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.651+0000 7fa3cf400640 1 --2- 172.21.15.83:0/278791538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a234a20 0x56503a2302d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:49:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.652+0000 7fa3cea00640 1 --2- 172.21.15.83:0/278791538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a229a30 0x56503a2473d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa3c009c980 tx=0x7fa3c0009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c31550304c351678 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.652+0000 7fa3bb400640 1 -- 172.21.15.83:0/278791538 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00b7070 con 0x56503a229a30 2024-04-23T14:49:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.652+0000 7fa3bb400640 1 -- 172.21.15.83:0/278791538 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c00a5660 con 0x56503a229a30 2024-04-23T14:49:40.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.652+0000 7fa3bb400640 1 -- 172.21.15.83:0/278791538 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00b5020 con 0x56503a229a30 2024-04-23T14:49:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.653+0000 7fa3d8b73740 1 -- 172.21.15.83:0/278791538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a229a30 msgr2=0x56503a2473d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.653+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/278791538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a229a30 0x56503a2473d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa3c009c980 tx=0x7fa3c0009500 comp rx=0 tx=0).stop 2024-04-23T14:49:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.653+0000 7fa3d8b73740 1 -- 172.21.15.83:0/278791538 shutdown_connections 2024-04-23T14:49:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.653+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/278791538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a234a20 0x56503a2302d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.653+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/278791538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 0x56503a240730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.653+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/278791538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a229a30 0x56503a2473d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.653+0000 7fa3d8b73740 1 -- 172.21.15.83:0/278791538 >> 172.21.15.83:0/278791538 conn(0x56503a225d50 msgr2=0x56503a224fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.653+0000 7fa3d8b73740 1 -- 172.21.15.83:0/278791538 shutdown_connections 2024-04-23T14:49:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.653+0000 7fa3d8b73740 1 -- 172.21.15.83:0/278791538 wait complete. 2024-04-23T14:49:40.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.654+0000 7fa3d8b73740 1 -- start start 2024-04-23T14:49:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.654+0000 7fa3d8b73740 1 Processor -- start 2024-04-23T14:49:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3d8b73740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a22e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3d8b73740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 0x56503a22eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3d8b73740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a234a20 0x56503a2aa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3d8b73740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b000cda0 con 0x56503a229a30 2024-04-23T14:49:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3d8b73740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b000e210 con 0x56503a234a20 2024-04-23T14:49:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3d8b73740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b000d470 con 0x56503a22a350 2024-04-23T14:49:40.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3d8b73740 10 client.0 _ll_register_callbacks cb 0x7fa3bc009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3cea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a22e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3ce000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 0x56503a22eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3cea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a22e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50626/0 (socket says 172.21.15.83:50626) 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3cea00640 1 -- 172.21.15.83:0/4254986648 learned_addr learned my addr 172.21.15.83:0/4254986648 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3cf400640 1 --2- 172.21.15.83:0/4254986648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a234a20 0x56503a2aa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.655+0000 7fa3ce000640 1 -- 172.21.15.83:0/4254986648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a234a20 msgr2=0x56503a2aa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.656+0000 7fa3ce000640 1 --2- 172.21.15.83:0/4254986648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a234a20 0x56503a2aa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.656+0000 7fa3ce000640 1 -- 172.21.15.83:0/4254986648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 msgr2=0x56503a22e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.656+0000 7fa3ce000640 1 --2- 172.21.15.83:0/4254986648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a22e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.656+0000 7fa3ce000640 1 -- 172.21.15.83:0/4254986648 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565039e610f0 con 0x56503a22a350 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.656+0000 7fa3ce000640 1 --2- 172.21.15.83:0/4254986648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 0x56503a22eea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc004970 tx=0x7fa3bc011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf90589b5d1494d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.656+0000 7fa3b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:49:40.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.656+0000 7fa3b8c00640 1 -- 172.21.15.83:0/4254986648 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc034040 con 0x56503a22a350 2024-04-23T14:49:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.656+0000 7fa3b8c00640 1 -- 172.21.15.83:0/4254986648 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa3bc0039c0 con 0x56503a22a350 2024-04-23T14:49:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.656+0000 7fa3d8b73740 1 -- 172.21.15.83:0/4254986648 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3c00a5660 con 0x56503a22a350 2024-04-23T14:49:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.657+0000 7fa3b8c00640 1 -- 172.21.15.83:0/4254986648 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa3bc01d040 con 0x56503a22a350 2024-04-23T14:49:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.657+0000 7fa3d8b73740 10 client.25570 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:40.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.657+0000 7fa3d8b73740 1 -- 172.21.15.83:0/4254986648 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56503a154810 con 0x56503a22a350 2024-04-23T14:49:40.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.657+0000 7fa3b8c00640 1 -- 172.21.15.83:0/4254986648 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c00a5660 con 0x56503a22a350 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.657+0000 7fa3b8c00640 1 -- 172.21.15.83:0/4254986648 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fa3bc043b10 con 0x56503a22a350 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.657+0000 7fa3d8b73740 10 client.25570 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.657+0000 7fa3d8b73740 10 client.25570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.657+0000 7fa3d8b73740 20 client.25570 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.657+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/4254986648 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56503a2aba10 0x56503a2ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.657+0000 7fa3d8b73740 4 client.25570 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.657+0000 7fa3d8b73740 1 -- 172.21.15.83:0/4254986648 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56503a196a80 con 0x56503a2aba10 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.658+0000 7fa3cea00640 1 --2- 172.21.15.83:0/4254986648 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56503a2aba10 0x56503a2ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.658+0000 7fa3cea00640 1 --2- 172.21.15.83:0/4254986648 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56503a2aba10 0x56503a2ade30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6e8e75f6dbb7881 server_cookie=70876e3cd490d04e in_seq=0 out_seq=0 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.658+0000 7fa3b8c00640 10 client.25570 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3b8c00640 1 -- 172.21.15.83:0/4254986648 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56503a196a80 con 0x56503a2aba10 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3b8c00640 10 client.25570 handle_command_reply: tid=0 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3d8b73740 1 client.25570 shutdown 2024-04-23T14:49:40.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3d8b73740 1 -- 172.21.15.83:0/4254986648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 msgr2=0x56503a22eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/4254986648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 0x56503a22eea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc004970 tx=0x7fa3bc011d90 comp rx=0 tx=0).stop 2024-04-23T14:49:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3d8b73740 1 -- 172.21.15.83:0/4254986648 shutdown_connections 2024-04-23T14:49:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/4254986648 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56503a2aba10 0x56503a2ade30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/4254986648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56503a234a20 0x56503a2aa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/4254986648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56503a22a350 0x56503a22eea0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3b8c00640 0 client.25570 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3d8b73740 1 --2- 172.21.15.83:0/4254986648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56503a229a30 0x56503a22e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3d8b73740 1 -- 172.21.15.83:0/4254986648 >> 172.21.15.83:0/4254986648 conn(0x56503a225d50 msgr2=0x56503a224fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3d8b73740 1 -- 172.21.15.83:0/4254986648 shutdown_connections 2024-04-23T14:49:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3d8b73740 1 -- 172.21.15.83:0/4254986648 wait complete. 2024-04-23T14:49:40.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.659+0000 7fa3d8b73740 20 client.25570 trim_cache size 0 max 16384 2024-04-23T14:49:40.663 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:49:40.663 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:49:40.663 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:49:40.663 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:49:40.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.660+0000 7fa3cfe00640 1 -- 172.21.15.83:0/148239784 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa3b40a3e60 msgr2=0x7fa3b40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.660+0000 7fa3cfe00640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa3b40a3e60 0x7fa3b40a6320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc01b040 tx=0x7fa3bc0048e0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.660+0000 7fa3cfe00640 1 -- 172.21.15.83:0/148239784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3d0104320 msgr2=0x7fa3d01a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.660+0000 7fa3cfe00640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3d0104320 0x7fa3d01a0620 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000ea60 tx=0x7fa3c000ef20 comp rx=0 tx=0).stop 2024-04-23T14:49:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.660+0000 7fa3cfe00640 1 -- 172.21.15.83:0/148239784 shutdown_connections 2024-04-23T14:49:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.660+0000 7fa3cfe00640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa3b40a3e60 0x7fa3b40a6320 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.660+0000 7fa3cfe00640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3d019a790 0x7fa3d019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.660+0000 7fa3cfe00640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa3d01093f0 0x7fa3d01a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.660+0000 7fa3cfe00640 1 --2- 172.21.15.83:0/148239784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3d0104320 0x7fa3d01a0620 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.660+0000 7fa3cfe00640 1 -- 172.21.15.83:0/148239784 >> 172.21.15.83:0/148239784 conn(0x7fa3d00ffb40 msgr2=0x7fa3d0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.660+0000 7fa3cfe00640 1 -- 172.21.15.83:0/148239784 shutdown_connections 2024-04-23T14:49:40.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.660+0000 7fa3cfe00640 1 -- 172.21.15.83:0/148239784 wait complete. 2024-04-23T14:49:40.678 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:49:40.678 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:49:40.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.850+0000 7f3b21c00640 1 -- 172.21.15.83:0/3513162943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c104440 msgr2=0x7f3b1c104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:40.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.850+0000 7f3b21c00640 1 --2- 172.21.15.83:0/3513162943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c104440 0x7f3b1c104820 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009900 tx=0x7f3b0c02a3f0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.851+0000 7f3b21c00640 1 -- 172.21.15.83:0/3513162943 shutdown_connections 2024-04-23T14:49:40.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.851+0000 7f3b21c00640 1 --2- 172.21.15.83:0/3513162943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1c109510 0x7f3b1c1110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.851+0000 7f3b21c00640 1 --2- 172.21.15.83:0/3513162943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b1c104df0 0x7f3b1c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.851+0000 7f3b21c00640 1 --2- 172.21.15.83:0/3513162943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c104440 0x7f3b1c104820 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.851+0000 7f3b21c00640 1 -- 172.21.15.83:0/3513162943 >> 172.21.15.83:0/3513162943 conn(0x7f3b1c0ffd00 msgr2=0x7f3b1c102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:40.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.851+0000 7f3b21c00640 1 -- 172.21.15.83:0/3513162943 shutdown_connections 2024-04-23T14:49:40.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.851+0000 7f3b21c00640 1 -- 172.21.15.83:0/3513162943 wait complete. 2024-04-23T14:49:40.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.852+0000 7f3b21c00640 1 Processor -- start 2024-04-23T14:49:40.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.852+0000 7f3b21c00640 1 -- start start 2024-04-23T14:49:40.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.853+0000 7f3b21c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c104440 0x7f3b1c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.853+0000 7f3b21c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b1c109510 0x7f3b1c1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.853+0000 7f3b21c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1c19a930 0x7f3b1c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.853+0000 7f3b21c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c1145f0 con 0x7f3b1c104440 2024-04-23T14:49:40.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.853+0000 7f3b21c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c114470 con 0x7f3b1c19a930 2024-04-23T14:49:40.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.853+0000 7f3b21c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c114770 con 0x7f3b1c109510 2024-04-23T14:49:40.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.853+0000 7f3b1aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c104440 0x7f3b1c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.853+0000 7f3b1aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c104440 0x7f3b1c1a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50636/0 (socket says 172.21.15.83:50636) 2024-04-23T14:49:40.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.853+0000 7f3b1aa00640 1 -- 172.21.15.83:0/2582027376 learned_addr learned my addr 172.21.15.83:0/2582027376 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:40.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.853+0000 7f3b1a000640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b1c109510 0x7f3b1c1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.854+0000 7f3b1b400640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1c19a930 0x7f3b1c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.854+0000 7f3b1aa00640 1 -- 172.21.15.83:0/2582027376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b1c109510 msgr2=0x7f3b1c1a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:40.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.854+0000 7f3b1aa00640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b1c109510 0x7f3b1c1a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.854+0000 7f3b1aa00640 1 -- 172.21.15.83:0/2582027376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1c19a930 msgr2=0x7f3b1c19ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.854+0000 7f3b1aa00640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1c19a930 0x7f3b1c19ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.854+0000 7f3b1aa00640 1 -- 172.21.15.83:0/2582027376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1c19b450 con 0x7f3b1c104440 2024-04-23T14:49:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.854+0000 7f3b1b400640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1c19a930 0x7f3b1c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:49:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.854+0000 7f3b1aa00640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c104440 0x7f3b1c1a0730 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c004770 tx=0x7f3b0c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.855+0000 7f3b13400640 1 -- 172.21.15.83:0/2582027376 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c03f070 con 0x7f3b1c104440 2024-04-23T14:49:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.855+0000 7f3b13400640 1 -- 172.21.15.83:0/2582027376 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b0c031ca0 con 0x7f3b1c104440 2024-04-23T14:49:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.855+0000 7f3b13400640 1 -- 172.21.15.83:0/2582027376 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c035030 con 0x7f3b1c104440 2024-04-23T14:49:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.855+0000 7f3b21c00640 1 -- 172.21.15.83:0/2582027376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1c19b6e0 con 0x7f3b1c104440 2024-04-23T14:49:40.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.855+0000 7f3b21c00640 1 -- 172.21.15.83:0/2582027376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1c1a7830 con 0x7f3b1c104440 2024-04-23T14:49:40.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.857+0000 7f3b21c00640 1 -- 172.21.15.83:0/2582027376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b1c19be10 con 0x7f3b1c104440 2024-04-23T14:49:40.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.860+0000 7f3b13400640 1 -- 172.21.15.83:0/2582027376 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f3b0c042070 con 0x7f3b1c104440 2024-04-23T14:49:40.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.861+0000 7f3b13400640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f3b000a3f50 0x7f3b000a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:40.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.862+0000 7f3b13400640 1 -- 172.21.15.83:0/2582027376 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3b0c0f10b0 con 0x7f3b1c104440 2024-04-23T14:49:40.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.864+0000 7f3b1a000640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f3b000a3f50 0x7f3b000a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:40.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.864+0000 7f3b1a000640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f3b000a3f50 0x7f3b000a6410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c103430 tx=0x7f3b08005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:40.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:40.871+0000 7f3b13400640 1 -- 172.21.15.83:0/2582027376 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b0c0bd5e0 con 0x7f3b1c104440 2024-04-23T14:49:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.162+0000 7f3b21c00640 1 -- 172.21.15.83:0/2582027376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b1c069250 con 0x7f3b1c104440 2024-04-23T14:49:41.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.163+0000 7f3b13400640 1 -- 172.21.15.83:0/2582027376 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11863 (secure 0 0 0) 0x7f3b0c0bcee0 con 0x7f3b1c104440 2024-04-23T14:49:41.165 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:49:41.165 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:48:27.525285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24532,24542]},"id":1}]} 2024-04-23T14:49:41.166 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-04-23T14:49:41.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.166+0000 7f3b21c00640 1 -- 172.21.15.83:0/2582027376 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f3b000a3f50 msgr2=0x7f3b000a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.166+0000 7f3b21c00640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f3b000a3f50 0x7f3b000a6410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c103430 tx=0x7f3b08005f90 comp rx=0 tx=0).stop 2024-04-23T14:49:41.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.166+0000 7f3b21c00640 1 -- 172.21.15.83:0/2582027376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c104440 msgr2=0x7f3b1c1a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.166+0000 7f3b21c00640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c104440 0x7f3b1c1a0730 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c004770 tx=0x7f3b0c003c10 comp rx=0 tx=0).stop 2024-04-23T14:49:41.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.166+0000 7f3b21c00640 1 -- 172.21.15.83:0/2582027376 shutdown_connections 2024-04-23T14:49:41.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.166+0000 7f3b21c00640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b1c19a930 0x7f3b1c19ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.166+0000 7f3b21c00640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b1c109510 0x7f3b1c1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.166+0000 7f3b21c00640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f3b000a3f50 0x7f3b000a6410 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.166+0000 7f3b21c00640 1 --2- 172.21.15.83:0/2582027376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1c104440 0x7f3b1c1a0730 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.166+0000 7f3b21c00640 1 -- 172.21.15.83:0/2582027376 >> 172.21.15.83:0/2582027376 conn(0x7f3b1c0ffd00 msgr2=0x7f3b1c1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.167+0000 7f3b21c00640 1 -- 172.21.15.83:0/2582027376 shutdown_connections 2024-04-23T14:49:41.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.167+0000 7f3b21c00640 1 -- 172.21.15.83:0/2582027376 wait complete. 2024-04-23T14:49:41.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:49:41.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.364+0000 7fef2a000640 1 -- 172.21.15.83:0/1618769468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24100ef0 msgr2=0x7fef241012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.364+0000 7fef2a000640 1 --2- 172.21.15.83:0/1618769468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24100ef0 0x7fef241012d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fef14009900 tx=0x7fef1402a480 comp rx=0 tx=0).stop 2024-04-23T14:49:41.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.364+0000 7fef2a000640 1 -- 172.21.15.83:0/1618769468 shutdown_connections 2024-04-23T14:49:41.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.364+0000 7fef2a000640 1 --2- 172.21.15.83:0/1618769468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef241063f0 0x7fef241068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.364+0000 7fef2a000640 1 --2- 172.21.15.83:0/1618769468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef24101810 0x7fef24105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.364+0000 7fef2a000640 1 --2- 172.21.15.83:0/1618769468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24100ef0 0x7fef241012d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.364+0000 7fef2a000640 1 -- 172.21.15.83:0/1618769468 >> 172.21.15.83:0/1618769468 conn(0x7fef24078060 msgr2=0x7fef24078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.364+0000 7fef2a000640 1 -- 172.21.15.83:0/1618769468 shutdown_connections 2024-04-23T14:49:41.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.365+0000 7fef2a000640 1 -- 172.21.15.83:0/1618769468 wait complete. 2024-04-23T14:49:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.366+0000 7fef2a000640 1 Processor -- start 2024-04-23T14:49:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.366+0000 7fef2a000640 1 -- start start 2024-04-23T14:49:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.367+0000 7fef2a000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24100ef0 0x7fef2419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.367+0000 7fef2a000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef24101810 0x7fef2419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.367+0000 7fef2a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef241063f0 0x7fef241a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.367+0000 7fef2a000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef24118980 con 0x7fef241063f0 2024-04-23T14:49:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.367+0000 7fef2a000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef24118800 con 0x7fef24101810 2024-04-23T14:49:41.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.367+0000 7fef2a000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef24118b00 con 0x7fef24100ef0 2024-04-23T14:49:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.367+0000 7fef23400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24100ef0 0x7fef2419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.367+0000 7fef23400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24100ef0 0x7fef2419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40204/0 (socket says 172.21.15.83:40204) 2024-04-23T14:49:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.367+0000 7fef23400640 1 -- 172.21.15.83:0/1394996615 learned_addr learned my addr 172.21.15.83:0/1394996615 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.367+0000 7fef23e00640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef241063f0 0x7fef241a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.367+0000 7fef22a00640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef24101810 0x7fef2419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.368+0000 7fef23400640 1 -- 172.21.15.83:0/1394996615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef24101810 msgr2=0x7fef2419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.368+0000 7fef23400640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef24101810 0x7fef2419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.368+0000 7fef23400640 1 -- 172.21.15.83:0/1394996615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef241063f0 msgr2=0x7fef241a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.368+0000 7fef23400640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef241063f0 0x7fef241a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.368+0000 7fef23400640 1 -- 172.21.15.83:0/1394996615 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef241a3e90 con 0x7fef24100ef0 2024-04-23T14:49:41.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.368+0000 7fef23e00640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef241063f0 0x7fef241a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:41.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.368+0000 7fef23400640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24100ef0 0x7fef2419eea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef14006010 tx=0x7fef14004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.369+0000 7fef1be00640 1 -- 172.21.15.83:0/1394996615 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1403f030 con 0x7fef24100ef0 2024-04-23T14:49:41.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.369+0000 7fef2a000640 1 -- 172.21.15.83:0/1394996615 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef241a4120 con 0x7fef24100ef0 2024-04-23T14:49:41.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.369+0000 7fef2a000640 1 -- 172.21.15.83:0/1394996615 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef241aba10 con 0x7fef24100ef0 2024-04-23T14:49:41.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.369+0000 7fef1be00640 1 -- 172.21.15.83:0/1394996615 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef14004440 con 0x7fef24100ef0 2024-04-23T14:49:41.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.369+0000 7fef1be00640 1 -- 172.21.15.83:0/1394996615 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14035070 con 0x7fef24100ef0 2024-04-23T14:49:41.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.372+0000 7fef2ce27740 1 Processor -- start 2024-04-23T14:49:41.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.372+0000 7fef2ce27740 1 -- start start 2024-04-23T14:49:41.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.373+0000 7fef2ce27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f6950 0x55a7729f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.373+0000 7fef2ce27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7729f7270 0x55a772a01400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.373+0000 7fef2ce27740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a772a01940 0x55a772a03d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.373+0000 7fef2ce27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a77295c1d0 con 0x55a7729f7270 2024-04-23T14:49:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.373+0000 7fef2ce27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a772902450 con 0x55a7729f6950 2024-04-23T14:49:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.373+0000 7fef2ce27740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a772958190 con 0x55a772a01940 2024-04-23T14:49:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.373+0000 7fef22a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a772a01940 0x55a772a03d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.373+0000 7fef22a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a772a01940 0x55a772a03d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40214/0 (socket says 172.21.15.83:40214) 2024-04-23T14:49:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.373+0000 7fef22a00640 1 -- 172.21.15.83:0/2540431630 learned_addr learned my addr 172.21.15.83:0/2540431630 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.375 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.373+0000 7fef23e00640 1 --2- 172.21.15.83:0/2540431630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f6950 0x55a7729f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.374+0000 7fef22a00640 1 -- 172.21.15.83:0/2540431630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f6950 msgr2=0x55a7729f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.374+0000 7fef22a00640 1 --2- 172.21.15.83:0/2540431630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f6950 0x55a7729f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.374+0000 7fef22a00640 1 -- 172.21.15.83:0/2540431630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7729f7270 msgr2=0x55a772a01400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:49:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.374+0000 7fef22a00640 1 --2- 172.21.15.83:0/2540431630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7729f7270 0x55a772a01400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.374+0000 7fef22a00640 1 -- 172.21.15.83:0/2540431630 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7729bf310 con 0x55a772a01940 2024-04-23T14:49:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.374+0000 7fef22a00640 1 --2- 172.21.15.83:0/2540431630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a772a01940 0x55a772a03d30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fef10009c80 tx=0x7fef10011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb4d99d43a29b6c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.374+0000 7fef1be00640 1 -- 172.21.15.83:0/1394996615 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fef14042030 con 0x7fef24100ef0 2024-04-23T14:49:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.374+0000 7fef0fe00640 1 -- 172.21.15.83:0/2540431630 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1001e070 con 0x55a772a01940 2024-04-23T14:49:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.374+0000 7fef0fe00640 1 -- 172.21.15.83:0/2540431630 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef10019b10 con 0x55a772a01940 2024-04-23T14:49:41.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.374+0000 7fef0fe00640 1 -- 172.21.15.83:0/2540431630 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1000eb20 con 0x55a772a01940 2024-04-23T14:49:41.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.375+0000 7fef2ce27740 1 -- 172.21.15.83:0/2540431630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a772a01940 msgr2=0x55a772a03d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.375+0000 7fef2ce27740 1 --2- 172.21.15.83:0/2540431630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a772a01940 0x55a772a03d30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fef10009c80 tx=0x7fef10011810 comp rx=0 tx=0).stop 2024-04-23T14:49:41.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.376+0000 7fef2ce27740 1 -- 172.21.15.83:0/2540431630 shutdown_connections 2024-04-23T14:49:41.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.376+0000 7fef2ce27740 1 --2- 172.21.15.83:0/2540431630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a772a01940 0x55a772a03d30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.376+0000 7fef2ce27740 1 --2- 172.21.15.83:0/2540431630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a7729f7270 0x55a772a01400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.376+0000 7fef2ce27740 1 --2- 172.21.15.83:0/2540431630 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f6950 0x55a7729f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.376+0000 7fef2ce27740 1 -- 172.21.15.83:0/2540431630 >> 172.21.15.83:0/2540431630 conn(0x55a7729f2c70 msgr2=0x55a7729f3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.376+0000 7fef2ce27740 1 -- 172.21.15.83:0/2540431630 shutdown_connections 2024-04-23T14:49:41.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.376+0000 7fef2ce27740 1 -- 172.21.15.83:0/2540431630 wait complete. 2024-04-23T14:49:41.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.377+0000 7fef1be00640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fef080a3ef0 0x7fef080a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.377+0000 7fef2ce27740 1 -- start start 2024-04-23T14:49:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.377+0000 7fef1be00640 1 -- 172.21.15.83:0/1394996615 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fef140f0c90 con 0x7fef24100ef0 2024-04-23T14:49:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.377+0000 7fef2ce27740 1 Processor -- start 2024-04-23T14:49:41.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.378+0000 7fef23e00640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fef080a3ef0 0x7fef080a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.378+0000 7fef2ce27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f6950 0x55a772a01350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.378+0000 7fef2ce27740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f7270 0x55a7729fe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.378+0000 7fef2ce27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a7729fec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.378+0000 7fef2ce27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a772a047d0 con 0x55a772a01940 2024-04-23T14:49:41.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.378+0000 7fef2ce27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a772915810 con 0x55a7729f6950 2024-04-23T14:49:41.380 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.378+0000 7fef2ce27740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a772a04950 con 0x55a7729f7270 2024-04-23T14:49:41.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.378+0000 7fef2ce27740 10 client.0 _ll_register_callbacks cb 0x55a7728e3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:41.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.378+0000 7fef22a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f7270 0x55a7729fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.378+0000 7fef22a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f7270 0x55a7729fe640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40228/0 (socket says 172.21.15.83:40228) 2024-04-23T14:49:41.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.378+0000 7fef22a00640 1 -- 172.21.15.83:0/3698243727 learned_addr learned my addr 172.21.15.83:0/3698243727 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.378+0000 7fef23400640 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f6950 0x55a772a01350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.379+0000 7fef22a00640 1 -- 172.21.15.83:0/3698243727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f6950 msgr2=0x55a772a01350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.379+0000 7fef22a00640 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f6950 0x55a772a01350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.379+0000 7fef22a00640 1 -- 172.21.15.83:0/3698243727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 msgr2=0x55a7729fec50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:49:41.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.379+0000 7fef23e00640 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a7729fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.379+0000 7fef22a00640 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a7729fec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.379+0000 7fef22a00640 1 -- 172.21.15.83:0/3698243727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7729263f0 con 0x55a7729f7270 2024-04-23T14:49:41.381 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.379+0000 7fef23e00640 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a7729fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:49:41.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.379+0000 7fef23e00640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fef080a3ef0 0x7fef080a63b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fef1c005fd0 tx=0x7fef1c009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.380+0000 7fef22a00640 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f7270 0x55a7729fe640 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef1000e6c0 tx=0x7fef1000e5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c241fc60eb3bad24 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.380+0000 7fef0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:49:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.380+0000 7fef0cc00640 1 -- 172.21.15.83:0/3698243727 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1001e070 con 0x55a7729f7270 2024-04-23T14:49:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.380+0000 7fef0cc00640 1 -- 172.21.15.83:0/3698243727 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fef10005030 con 0x55a7729f7270 2024-04-23T14:49:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.380+0000 7fef2ce27740 1 -- 172.21.15.83:0/3698243727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef10019b10 con 0x55a7729f7270 2024-04-23T14:49:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.381+0000 7fef0cc00640 1 -- 172.21.15.83:0/3698243727 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fef10032050 con 0x55a7729f7270 2024-04-23T14:49:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.381+0000 7fef2ce27740 10 client.25588 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.381+0000 7fef2ce27740 1 -- 172.21.15.83:0/3698243727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a77266e3f0 con 0x55a7729f7270 2024-04-23T14:49:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.381+0000 7fef0cc00640 1 -- 172.21.15.83:0/3698243727 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef10029070 con 0x55a7729f7270 2024-04-23T14:49:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.381+0000 7fef0cc00640 1 -- 172.21.15.83:0/3698243727 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fef100046c0 con 0x55a7729f7270 2024-04-23T14:49:41.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.381+0000 7fef2ce27740 10 client.25588 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.381+0000 7fef2ce27740 10 client.25588 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:49:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.381+0000 7fef2ce27740 20 client.25588 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.381+0000 7fef2ce27740 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a772a11ae0 0x55a772a13ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.381+0000 7fef2ce27740 4 client.25588 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:49:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.381+0000 7fef2ce27740 1 -- 172.21.15.83:0/3698243727 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fef10019b10 con 0x55a772a11ae0 2024-04-23T14:49:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.382+0000 7fef23400640 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a772a11ae0 0x55a772a13ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.382+0000 7fef23400640 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a772a11ae0 0x55a772a13ed0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57d8d50459b04182 server_cookie=2214ac526e286bf5 in_seq=0 out_seq=0 2024-04-23T14:49:41.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.382+0000 7fef0cc00640 10 client.25588 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:49:41.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.383+0000 7fef0cc00640 1 -- 172.21.15.83:0/3698243727 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fef10019b10 con 0x55a772a11ae0 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.383+0000 7fef0cc00640 10 client.25588 handle_command_reply: tid=0 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.383+0000 7fef2ce27740 1 client.25588 shutdown 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.383+0000 7fef2ce27740 1 -- 172.21.15.83:0/3698243727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f7270 msgr2=0x55a7729fe640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.383+0000 7fef2ce27740 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f7270 0x55a7729fe640 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fef1000e6c0 tx=0x7fef1000e5e0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.383+0000 7fef2ce27740 1 -- 172.21.15.83:0/3698243727 shutdown_connections 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.383+0000 7fef2ce27740 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a772a11ae0 0x55a772a13ed0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.384+0000 7fef2ce27740 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a7729fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.384+0000 7fef2ce27740 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f7270 0x55a7729fe640 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.384+0000 7fef2ce27740 1 --2- 172.21.15.83:0/3698243727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f6950 0x55a772a01350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.384+0000 7fef2ce27740 1 -- 172.21.15.83:0/3698243727 >> 172.21.15.83:0/3698243727 conn(0x55a7729f2c70 msgr2=0x55a7729f8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.384+0000 7fef0cc00640 0 client.25588 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.384+0000 7fef2ce27740 1 -- 172.21.15.83:0/3698243727 shutdown_connections 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.384+0000 7fef2ce27740 1 -- 172.21.15.83:0/3698243727 wait complete. 2024-04-23T14:49:41.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.384+0000 7fef2ce27740 20 client.25588 trim_cache size 0 max 16384 2024-04-23T14:49:41.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.412+0000 7fef2ce27740 1 Processor -- start 2024-04-23T14:49:41.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.413+0000 7fef2ce27740 1 -- start start 2024-04-23T14:49:41.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.413+0000 7fef2ce27740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 0x55a772a0d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.413+0000 7fef2ce27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f7270 0x55a772a00e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.413+0000 7fef2ce27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a772a196e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.413+0000 7fef2ce27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef04011000 con 0x55a772a01940 2024-04-23T14:49:41.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.413+0000 7fef2ce27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef0400e8d0 con 0x55a7729f7270 2024-04-23T14:49:41.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.413+0000 7fef2ce27740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef0400c720 con 0x55a7729f6950 2024-04-23T14:49:41.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.414+0000 7fef23400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 0x55a772a0d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.414+0000 7fef23400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 0x55a772a0d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40244/0 (socket says 172.21.15.83:40244) 2024-04-23T14:49:41.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.414+0000 7fef23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a772a196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.414+0000 7fef23400640 1 -- 172.21.15.83:0/1205845725 learned_addr learned my addr 172.21.15.83:0/1205845725 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.414+0000 7fef22a00640 1 --2- 172.21.15.83:0/1205845725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f7270 0x55a772a00e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.414+0000 7fef23400640 1 -- 172.21.15.83:0/1205845725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f7270 msgr2=0x55a772a00e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.414+0000 7fef23400640 1 --2- 172.21.15.83:0/1205845725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f7270 0x55a772a00e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.414+0000 7fef23400640 1 -- 172.21.15.83:0/1205845725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 msgr2=0x55a772a196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.414+0000 7fef23400640 1 --2- 172.21.15.83:0/1205845725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a772a196e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.414+0000 7fef23400640 1 -- 172.21.15.83:0/1205845725 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7729fefc0 con 0x55a7729f6950 2024-04-23T14:49:41.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.414+0000 7fef23e00640 1 --2- 172.21.15.83:0/1205845725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a772a196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:49:41.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.415+0000 7fef23400640 1 --2- 172.21.15.83:0/1205845725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 0x55a772a0d6f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef140b93d0 tx=0x7fef1402a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3118fdb9fb41fc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.415+0000 7fef0f400640 1 -- 172.21.15.83:0/1205845725 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef140bca00 con 0x55a7729f6950 2024-04-23T14:49:41.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.415+0000 7fef0f400640 1 -- 172.21.15.83:0/1205845725 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef140c0db0 con 0x55a7729f6950 2024-04-23T14:49:41.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.415+0000 7fef0f400640 1 -- 172.21.15.83:0/1205845725 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef140c51e0 con 0x55a7729f6950 2024-04-23T14:49:41.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.416+0000 7fef2ce27740 1 -- 172.21.15.83:0/1205845725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 msgr2=0x55a772a0d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.416+0000 7fef2ce27740 1 --2- 172.21.15.83:0/1205845725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 0x55a772a0d6f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef140b93d0 tx=0x7fef1402a480 comp rx=0 tx=0).stop 2024-04-23T14:49:41.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.416+0000 7fef2ce27740 1 -- 172.21.15.83:0/1205845725 shutdown_connections 2024-04-23T14:49:41.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.416+0000 7fef2ce27740 1 --2- 172.21.15.83:0/1205845725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a772a196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.416+0000 7fef2ce27740 1 --2- 172.21.15.83:0/1205845725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f7270 0x55a772a00e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.416+0000 7fef2ce27740 1 --2- 172.21.15.83:0/1205845725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 0x55a772a0d6f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.416+0000 7fef2ce27740 1 -- 172.21.15.83:0/1205845725 >> 172.21.15.83:0/1205845725 conn(0x55a7729f2c70 msgr2=0x55a7729f3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.416+0000 7fef2ce27740 1 -- 172.21.15.83:0/1205845725 shutdown_connections 2024-04-23T14:49:41.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.416+0000 7fef2ce27740 1 -- 172.21.15.83:0/1205845725 wait complete. 2024-04-23T14:49:41.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.416+0000 7fef2ce27740 1 -- start start 2024-04-23T14:49:41.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.417+0000 7fef2ce27740 1 Processor -- start 2024-04-23T14:49:41.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.417+0000 7fef2ce27740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 0x55a772a01d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.417+0000 7fef2ce27740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f7270 0x55a772a02260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.417+0000 7fef2ce27740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a772a027a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.417+0000 7fef2ce27740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef0400cd90 con 0x55a772a01940 2024-04-23T14:49:41.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.417+0000 7fef2ce27740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fef0400e200 con 0x55a7729f7270 2024-04-23T14:49:41.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.417+0000 7fef2ce27740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fef0400d460 con 0x55a7729f6950 2024-04-23T14:49:41.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.417+0000 7fef2ce27740 10 client.0 _ll_register_callbacks cb 0x7fef1001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:41.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.418+0000 7fef23e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a772a027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.418+0000 7fef23400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 0x55a772a01d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.418+0000 7fef23400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 0x55a772a01d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40254/0 (socket says 172.21.15.83:40254) 2024-04-23T14:49:41.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.418+0000 7fef23400640 1 -- 172.21.15.83:0/48310876 learned_addr learned my addr 172.21.15.83:0/48310876 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.418+0000 7fef22a00640 1 --2- 172.21.15.83:0/48310876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f7270 0x55a772a02260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.418+0000 7fef23400640 1 -- 172.21.15.83:0/48310876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f7270 msgr2=0x55a772a02260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.418+0000 7fef23400640 1 --2- 172.21.15.83:0/48310876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f7270 0x55a772a02260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.418+0000 7fef23400640 1 -- 172.21.15.83:0/48310876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 msgr2=0x55a772a027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.418+0000 7fef23400640 1 --2- 172.21.15.83:0/48310876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a772a027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.418+0000 7fef23400640 1 -- 172.21.15.83:0/48310876 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7727387d0 con 0x55a7729f6950 2024-04-23T14:49:41.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.418+0000 7fef23400640 1 --2- 172.21.15.83:0/48310876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 0x55a772a01d20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fef140c5140 tx=0x7fef140b8670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46b35274330880a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.418+0000 7fef0cc00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:49:41.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.419+0000 7fef0cc00640 1 -- 172.21.15.83:0/48310876 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14031bd0 con 0x55a7729f6950 2024-04-23T14:49:41.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.419+0000 7fef0cc00640 1 -- 172.21.15.83:0/48310876 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fef14031d70 con 0x55a7729f6950 2024-04-23T14:49:41.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.419+0000 7fef2ce27740 1 -- 172.21.15.83:0/48310876 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef140c0db0 con 0x55a7729f6950 2024-04-23T14:49:41.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.419+0000 7fef0cc00640 1 -- 172.21.15.83:0/48310876 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fef140c51e0 con 0x55a7729f6950 2024-04-23T14:49:41.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.419+0000 7fef2ce27740 10 client.25594 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.419+0000 7fef2ce27740 1 -- 172.21.15.83:0/48310876 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a772a09e40 con 0x55a7729f6950 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.419+0000 7fef0cc00640 1 -- 172.21.15.83:0/48310876 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef140c0db0 con 0x55a7729f6950 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.420+0000 7fef0cc00640 1 -- 172.21.15.83:0/48310876 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fef140eeb40 con 0x55a7729f6950 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.420+0000 7fef2ce27740 10 client.25594 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.420+0000 7fef2ce27740 10 client.25594 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.420+0000 7fef2ce27740 20 client.25594 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.420+0000 7fef2ce27740 1 --2- 172.21.15.83:0/48310876 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a772a77f50 0x55a772a7a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.420+0000 7fef2ce27740 4 client.25594 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.420+0000 7fef2ce27740 1 -- 172.21.15.83:0/48310876 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a772969bf0 con 0x55a772a77f50 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.420+0000 7fef22a00640 1 --2- 172.21.15.83:0/48310876 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a772a77f50 0x55a772a7a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef22a00640 1 --2- 172.21.15.83:0/48310876 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a772a77f50 0x55a772a7a340 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3036cfc7fc57b4d server_cookie=30633f2bc6047ba7 in_seq=0 out_seq=0 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef0cc00640 10 client.25594 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef0cc00640 1 -- 172.21.15.83:0/48310876 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a772969bf0 con 0x55a772a77f50 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef0cc00640 10 client.25594 handle_command_reply: tid=0 2024-04-23T14:49:41.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef2ce27740 1 client.25594 shutdown 2024-04-23T14:49:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef2ce27740 1 -- 172.21.15.83:0/48310876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 msgr2=0x55a772a01d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef2ce27740 1 --2- 172.21.15.83:0/48310876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 0x55a772a01d20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fef140c5140 tx=0x7fef140b8670 comp rx=0 tx=0).stop 2024-04-23T14:49:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef2ce27740 1 -- 172.21.15.83:0/48310876 shutdown_connections 2024-04-23T14:49:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef2ce27740 1 --2- 172.21.15.83:0/48310876 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a772a77f50 0x55a772a7a340 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef2ce27740 1 --2- 172.21.15.83:0/48310876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a772a01940 0x55a772a027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef2ce27740 1 --2- 172.21.15.83:0/48310876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a7729f7270 0x55a772a02260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef2ce27740 1 --2- 172.21.15.83:0/48310876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7729f6950 0x55a772a01d20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef2ce27740 1 -- 172.21.15.83:0/48310876 >> 172.21.15.83:0/48310876 conn(0x55a7729f2c70 msgr2=0x55a7729f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef0cc00640 0 client.25594 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:49:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef2ce27740 1 -- 172.21.15.83:0/48310876 shutdown_connections 2024-04-23T14:49:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.421+0000 7fef2ce27740 1 -- 172.21.15.83:0/48310876 wait complete. 2024-04-23T14:49:41.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.422+0000 7fef2ce27740 20 client.25594 trim_cache size 0 max 16384 2024-04-23T14:49:41.426 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:49:41.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.422+0000 7fef2a000640 1 -- 172.21.15.83:0/1394996615 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fef080a3ef0 msgr2=0x7fef080a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.422+0000 7fef2a000640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fef080a3ef0 0x7fef080a63b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fef1c005fd0 tx=0x7fef1c009500 comp rx=0 tx=0).stop 2024-04-23T14:49:41.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.422+0000 7fef2a000640 1 -- 172.21.15.83:0/1394996615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24100ef0 msgr2=0x7fef2419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.422+0000 7fef2a000640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24100ef0 0x7fef2419eea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef14006010 tx=0x7fef14004040 comp rx=0 tx=0).stop 2024-04-23T14:49:41.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.422+0000 7fef2a000640 1 -- 172.21.15.83:0/1394996615 shutdown_connections 2024-04-23T14:49:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.422+0000 7fef2a000640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fef080a3ef0 0x7fef080a63b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.422+0000 7fef2a000640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef241063f0 0x7fef241a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.422+0000 7fef2a000640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fef24101810 0x7fef2419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.422+0000 7fef2a000640 1 --2- 172.21.15.83:0/1394996615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef24100ef0 0x7fef2419eea0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.422+0000 7fef2a000640 1 -- 172.21.15.83:0/1394996615 >> 172.21.15.83:0/1394996615 conn(0x7fef24078060 msgr2=0x7fef24103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.423+0000 7fef2a000640 1 -- 172.21.15.83:0/1394996615 shutdown_connections 2024-04-23T14:49:41.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.423+0000 7fef2a000640 1 -- 172.21.15.83:0/1394996615 wait complete. 2024-04-23T14:49:41.439 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:41 smithi083 ceph-mon[27867]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:49:41.439 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:41 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2582027376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:49:41.441 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:49:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.612+0000 7f72eca00640 1 -- 172.21.15.83:0/1849006967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e8101770 msgr2=0x7f72e8105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.612+0000 7f72eca00640 1 --2- 172.21.15.83:0/1849006967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e8101770 0x7f72e8105c30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009900 tx=0x7f72d402a480 comp rx=0 tx=0).stop 2024-04-23T14:49:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.612+0000 7f72eca00640 1 -- 172.21.15.83:0/1849006967 shutdown_connections 2024-04-23T14:49:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.612+0000 7f72eca00640 1 --2- 172.21.15.83:0/1849006967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e8106250 0x7f72e81066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.612+0000 7f72eca00640 1 --2- 172.21.15.83:0/1849006967 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e8101770 0x7f72e8105c30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.612+0000 7f72eca00640 1 --2- 172.21.15.83:0/1849006967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72e8100e50 0x7f72e8101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.612+0000 7f72eca00640 1 -- 172.21.15.83:0/1849006967 >> 172.21.15.83:0/1849006967 conn(0x7f72e80fc880 msgr2=0x7f72e80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.612+0000 7f72eca00640 1 -- 172.21.15.83:0/1849006967 shutdown_connections 2024-04-23T14:49:41.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.612+0000 7f72eca00640 1 -- 172.21.15.83:0/1849006967 wait complete. 2024-04-23T14:49:41.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.613+0000 7f72eca00640 1 Processor -- start 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.613+0000 7f72eca00640 1 -- start start 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72eca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e8100e50 0x7f72e819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72eca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e8101770 0x7f72e819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72eca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72e8106250 0x7f72e81a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72eca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8118810 con 0x7f72e8101770 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72eca00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8118690 con 0x7f72e8100e50 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72eca00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8118990 con 0x7f72e8106250 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72e7200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72e8106250 0x7f72e81a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72e7200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72e8106250 0x7f72e81a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40268/0 (socket says 172.21.15.83:40268) 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72e7200640 1 -- 172.21.15.83:0/3188675960 learned_addr learned my addr 172.21.15.83:0/3188675960 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72e6800640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e8100e50 0x7f72e819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72e7200640 1 -- 172.21.15.83:0/3188675960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e8100e50 msgr2=0x7f72e819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72e7200640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e8100e50 0x7f72e819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72e7200640 1 -- 172.21.15.83:0/3188675960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e8101770 msgr2=0x7f72e819f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.614+0000 7f72e5e00640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e8101770 0x7f72e819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.615+0000 7f72e7200640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e8101770 0x7f72e819f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.615+0000 7f72e7200640 1 -- 172.21.15.83:0/3188675960 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e81a3da0 con 0x7f72e8106250 2024-04-23T14:49:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.615+0000 7f72e5e00640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e8101770 0x7f72e819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:49:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.615+0000 7f72e7200640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72e8106250 0x7f72e81a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f72e000eaa0 tx=0x7f72e000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.615+0000 7f72df400640 1 -- 172.21.15.83:0/3188675960 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e000cdb0 con 0x7f72e8106250 2024-04-23T14:49:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.615+0000 7f72eca00640 1 -- 172.21.15.83:0/3188675960 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e81a4090 con 0x7f72e8106250 2024-04-23T14:49:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.615+0000 7f72eca00640 1 -- 172.21.15.83:0/3188675960 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e81ab920 con 0x7f72e8106250 2024-04-23T14:49:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.616+0000 7f72df400640 1 -- 172.21.15.83:0/3188675960 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72e0005020 con 0x7f72e8106250 2024-04-23T14:49:41.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.616+0000 7f72df400640 1 -- 172.21.15.83:0/3188675960 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e00116b0 con 0x7f72e8106250 2024-04-23T14:49:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.619+0000 7f72df400640 1 -- 172.21.15.83:0/3188675960 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f72e0011850 con 0x7f72e8106250 2024-04-23T14:49:41.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.619+0000 7f72f0391740 1 Processor -- start 2024-04-23T14:49:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.620+0000 7f72f0391740 1 -- start start 2024-04-23T14:49:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.620+0000 7f72f0391740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab83b40 0x55fa7ab83f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.620+0000 7f72f0391740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 0x55fa7ab8e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.620+0000 7f72f0391740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab90f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.620+0000 7f72f0391740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fa7aae91d0 con 0x55fa7ab83b40 2024-04-23T14:49:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.620+0000 7f72f0391740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fa7aa8f450 con 0x55fa7ab84460 2024-04-23T14:49:41.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.620+0000 7f72f0391740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fa7aae5190 con 0x55fa7ab8eb30 2024-04-23T14:49:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.621+0000 7f72e6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 0x55fa7ab8e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.621+0000 7f72e6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 0x55fa7ab8e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37308/0 (socket says 172.21.15.83:37308) 2024-04-23T14:49:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.621+0000 7f72e6800640 1 -- 172.21.15.83:0/789407154 learned_addr learned my addr 172.21.15.83:0/789407154 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.621+0000 7f72e5e00640 1 --2- 172.21.15.83:0/789407154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab83b40 0x55fa7ab83f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.621+0000 7f72df400640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f72cc0a3f50 0x7f72cc0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.621+0000 7f72e6800640 1 -- 172.21.15.83:0/789407154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab8eb30 msgr2=0x55fa7ab90f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.621+0000 7f72e6800640 1 --2- 172.21.15.83:0/789407154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab90f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.621+0000 7f72e6800640 1 -- 172.21.15.83:0/789407154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab83b40 msgr2=0x55fa7ab83f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.621+0000 7f72df400640 1 -- 172.21.15.83:0/3188675960 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f72e00d0b30 con 0x7f72e8106250 2024-04-23T14:49:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.621+0000 7f72e6800640 1 --2- 172.21.15.83:0/789407154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab83b40 0x55fa7ab83f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.621+0000 7f72e6800640 1 -- 172.21.15.83:0/789407154 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7ab4c310 con 0x55fa7ab84460 2024-04-23T14:49:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.622+0000 7f72e7200640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f72cc0a3f50 0x7f72cc0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.622+0000 7f72e6800640 1 --2- 172.21.15.83:0/789407154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 0x55fa7ab8e5f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009d20 tx=0x7f72d8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec8c87cc4a2069be server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.622+0000 7f72d3400640 1 -- 172.21.15.83:0/789407154 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d801e070 con 0x55fa7ab84460 2024-04-23T14:49:41.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.622+0000 7f72d3400640 1 -- 172.21.15.83:0/789407154 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d8019b10 con 0x55fa7ab84460 2024-04-23T14:49:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.622+0000 7f72d3400640 1 -- 172.21.15.83:0/789407154 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d800ebf0 con 0x55fa7ab84460 2024-04-23T14:49:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.623+0000 7f72f0391740 1 -- 172.21.15.83:0/789407154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 msgr2=0x55fa7ab8e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.623+0000 7f72e7200640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f72cc0a3f50 0x7f72cc0a6410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72e00ce0b0 tx=0x7f72e0098b40 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.623+0000 7f72f0391740 1 --2- 172.21.15.83:0/789407154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 0x55fa7ab8e5f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009d20 tx=0x7f72d8011810 comp rx=0 tx=0).stop 2024-04-23T14:49:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.623+0000 7f72f0391740 1 -- 172.21.15.83:0/789407154 shutdown_connections 2024-04-23T14:49:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.624+0000 7f72f0391740 1 --2- 172.21.15.83:0/789407154 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab90f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.624+0000 7f72f0391740 1 --2- 172.21.15.83:0/789407154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 0x55fa7ab8e5f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.624+0000 7f72f0391740 1 --2- 172.21.15.83:0/789407154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab83b40 0x55fa7ab83f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.624+0000 7f72f0391740 1 -- 172.21.15.83:0/789407154 >> 172.21.15.83:0/789407154 conn(0x55fa7ab7fe60 msgr2=0x55fa7ab80240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.624+0000 7f72f0391740 1 -- 172.21.15.83:0/789407154 shutdown_connections 2024-04-23T14:49:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.625+0000 7f72f0391740 1 -- 172.21.15.83:0/789407154 wait complete. 2024-04-23T14:49:41.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.625+0000 7f72f0391740 1 -- start start 2024-04-23T14:49:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.625+0000 7f72f0391740 1 Processor -- start 2024-04-23T14:49:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.626+0000 7f72f0391740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab83b40 0x55fa7ab8e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.626+0000 7f72f0391740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 0x55fa7ab8b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.626+0000 7f72f0391740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab8bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.626+0000 7f72f0391740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fa7ab919c0 con 0x55fa7ab84460 2024-04-23T14:49:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.626+0000 7f72f0391740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fa7aaa2810 con 0x55fa7ab8eb30 2024-04-23T14:49:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.626+0000 7f72f0391740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fa7ab91b40 con 0x55fa7ab83b40 2024-04-23T14:49:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.626+0000 7f72f0391740 10 client.0 _ll_register_callbacks cb 0x55fa7aa70140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.627+0000 7f72e6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 0x55fa7ab8b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.627+0000 7f72e5e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab8bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.627+0000 7f72e6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 0x55fa7ab8b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50714/0 (socket says 172.21.15.83:50714) 2024-04-23T14:49:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.627+0000 7f72e6800640 1 -- 172.21.15.83:0/3022802892 learned_addr learned my addr 172.21.15.83:0/3022802892 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.627+0000 7f72e5e00640 1 --2- 172.21.15.83:0/3022802892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab83b40 0x55fa7ab8e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.627+0000 7f72e6800640 1 -- 172.21.15.83:0/3022802892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab83b40 msgr2=0x55fa7ab8e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.627+0000 7f72e6800640 1 --2- 172.21.15.83:0/3022802892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab83b40 0x55fa7ab8e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.627+0000 7f72e6800640 1 -- 172.21.15.83:0/3022802892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab8eb30 msgr2=0x55fa7ab8bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.627+0000 7f72e6800640 1 --2- 172.21.15.83:0/3022802892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab8bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.627+0000 7f72e6800640 1 -- 172.21.15.83:0/3022802892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7aab33f0 con 0x55fa7ab84460 2024-04-23T14:49:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.629+0000 7f72e6800640 1 --2- 172.21.15.83:0/3022802892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 0x55fa7ab8b850 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f72d8019660 tx=0x7f72d8019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a432507c35bd6d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.629+0000 7f72c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:49:41.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.629+0000 7f72c7e00640 1 -- 172.21.15.83:0/3022802892 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8026030 con 0x55fa7ab84460 2024-04-23T14:49:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.629+0000 7f72c7e00640 1 -- 172.21.15.83:0/3022802892 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f72d8004020 con 0x55fa7ab84460 2024-04-23T14:49:41.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.629+0000 7f72f0391740 1 -- 172.21.15.83:0/3022802892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72d8019b10 con 0x55fa7ab84460 2024-04-23T14:49:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.629+0000 7f72c7e00640 1 -- 172.21.15.83:0/3022802892 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f72d8030050 con 0x55fa7ab84460 2024-04-23T14:49:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.630+0000 7f72f0391740 10 client.15516 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.630+0000 7f72f0391740 1 -- 172.21.15.83:0/3022802892 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fa7a7fb3f0 con 0x55fa7ab84460 2024-04-23T14:49:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.630+0000 7f72c7e00640 1 -- 172.21.15.83:0/3022802892 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d8019b10 con 0x55fa7ab84460 2024-04-23T14:49:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.630+0000 7f72c7e00640 1 -- 172.21.15.83:0/3022802892 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f72d800e810 con 0x55fa7ab84460 2024-04-23T14:49:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.630+0000 7f72f0391740 10 client.15516 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.630+0000 7f72f0391740 10 client.15516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:49:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.630+0000 7f72f0391740 20 client.15516 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:41.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.630+0000 7f72f0391740 1 --2- 172.21.15.83:0/3022802892 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa7ab9eb20 0x55fa7aba0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.630+0000 7f72f0391740 4 client.15516 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:49:41.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.630+0000 7f72f0391740 1 -- 172.21.15.83:0/3022802892 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa7aaf6bf0 con 0x55fa7ab9eb20 2024-04-23T14:49:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.631+0000 7f72e5e00640 1 --2- 172.21.15.83:0/3022802892 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa7ab9eb20 0x55fa7aba0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.631+0000 7f72e5e00640 1 --2- 172.21.15.83:0/3022802892 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa7ab9eb20 0x55fa7aba0f10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d2bf8b60efcbed6 server_cookie=f47fdb449a2d3c3b in_seq=0 out_seq=0 2024-04-23T14:49:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.631+0000 7f72c7e00640 10 client.15516 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:49:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.632+0000 7f72c7e00640 1 -- 172.21.15.83:0/3022802892 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa7aaf6bf0 con 0x55fa7ab9eb20 2024-04-23T14:49:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.632+0000 7f72c7e00640 10 client.15516 handle_command_reply: tid=0 2024-04-23T14:49:41.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.632+0000 7f72f0391740 1 client.15516 shutdown 2024-04-23T14:49:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.632+0000 7f72f0391740 1 -- 172.21.15.83:0/3022802892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 msgr2=0x55fa7ab8b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.632+0000 7f72f0391740 1 --2- 172.21.15.83:0/3022802892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 0x55fa7ab8b850 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f72d8019660 tx=0x7f72d8019690 comp rx=0 tx=0).stop 2024-04-23T14:49:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.633+0000 7f72f0391740 1 -- 172.21.15.83:0/3022802892 shutdown_connections 2024-04-23T14:49:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.633+0000 7f72f0391740 1 --2- 172.21.15.83:0/3022802892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab8bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.633+0000 7f72f0391740 1 --2- 172.21.15.83:0/3022802892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 0x55fa7ab8b850 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.633+0000 7f72f0391740 1 --2- 172.21.15.83:0/3022802892 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa7ab9eb20 0x55fa7aba0f10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.633+0000 7f72f0391740 1 --2- 172.21.15.83:0/3022802892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab83b40 0x55fa7ab8e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.633+0000 7f72f0391740 1 -- 172.21.15.83:0/3022802892 >> 172.21.15.83:0/3022802892 conn(0x55fa7ab7fe60 msgr2=0x55fa7ab85b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.633+0000 7f72c7e00640 0 client.15516 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:49:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.633+0000 7f72f0391740 1 -- 172.21.15.83:0/3022802892 shutdown_connections 2024-04-23T14:49:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.633+0000 7f72f0391740 1 -- 172.21.15.83:0/3022802892 wait complete. 2024-04-23T14:49:41.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.633+0000 7f72f0391740 20 client.15516 trim_cache size 0 max 16384 2024-04-23T14:49:41.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.649+0000 7f72f0391740 1 Processor -- start 2024-04-23T14:49:41.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.649+0000 7f72f0391740 1 -- start start 2024-04-23T14:49:41.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.649+0000 7f72f0391740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab83b40 0x55fa7ab9a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.649+0000 7f72f0391740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 0x55fa7ab90e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.649+0000 7f72f0391740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab8e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.649+0000 7f72f0391740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8010fb0 con 0x55fa7ab84460 2024-04-23T14:49:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.649+0000 7f72f0391740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f72c800e880 con 0x55fa7ab8eb30 2024-04-23T14:49:41.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.649+0000 7f72f0391740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f72c800c6d0 con 0x55fa7ab83b40 2024-04-23T14:49:41.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.650+0000 7f72e5e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab83b40 0x55fa7ab9a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.650+0000 7f72e5e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab83b40 0x55fa7ab9a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40312/0 (socket says 172.21.15.83:40312) 2024-04-23T14:49:41.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.650+0000 7f72e5e00640 1 -- 172.21.15.83:0/1734433200 learned_addr learned my addr 172.21.15.83:0/1734433200 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.650+0000 7f72e5e00640 1 --2- 172.21.15.83:0/1734433200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab8e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.650+0000 7f72e6800640 1 --2- 172.21.15.83:0/1734433200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 0x55fa7ab90e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.650+0000 7f72e5e00640 1 -- 172.21.15.83:0/1734433200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab8eb30 msgr2=0x55fa7ab8e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.650+0000 7f72e5e00640 1 --2- 172.21.15.83:0/1734433200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab8e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.650+0000 7f72e5e00640 1 -- 172.21.15.83:0/1734433200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 msgr2=0x55fa7ab90e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.650+0000 7f72e5e00640 1 --2- 172.21.15.83:0/1734433200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 0x55fa7ab90e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.650+0000 7f72e5e00640 1 -- 172.21.15.83:0/1734433200 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7a8c57d0 con 0x55fa7ab83b40 2024-04-23T14:49:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.650+0000 7f72e6800640 1 --2- 172.21.15.83:0/1734433200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 0x55fa7ab90e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:49:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.651+0000 7f72e5e00640 1 --2- 172.21.15.83:0/1734433200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab83b40 0x55fa7ab9a910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f72d4006010 tx=0x7f72d402a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=664749a08b4a1b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.651+0000 7f72d2a00640 1 -- 172.21.15.83:0/1734433200 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4035040 con 0x55fa7ab83b40 2024-04-23T14:49:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.651+0000 7f72d2a00640 1 -- 172.21.15.83:0/1734433200 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d4005550 con 0x55fa7ab83b40 2024-04-23T14:49:41.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.651+0000 7f72d2a00640 1 -- 172.21.15.83:0/1734433200 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4049790 con 0x55fa7ab83b40 2024-04-23T14:49:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.652+0000 7f72f0391740 1 -- 172.21.15.83:0/1734433200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab83b40 msgr2=0x55fa7ab9a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.652+0000 7f72f0391740 1 --2- 172.21.15.83:0/1734433200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab83b40 0x55fa7ab9a910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f72d4006010 tx=0x7f72d402a480 comp rx=0 tx=0).stop 2024-04-23T14:49:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.652+0000 7f72f0391740 1 -- 172.21.15.83:0/1734433200 shutdown_connections 2024-04-23T14:49:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.652+0000 7f72f0391740 1 --2- 172.21.15.83:0/1734433200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab8e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.652+0000 7f72f0391740 1 --2- 172.21.15.83:0/1734433200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab84460 0x55fa7ab90e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.652+0000 7f72f0391740 1 --2- 172.21.15.83:0/1734433200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab83b40 0x55fa7ab9a910 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.652+0000 7f72f0391740 1 -- 172.21.15.83:0/1734433200 >> 172.21.15.83:0/1734433200 conn(0x55fa7ab7fe60 msgr2=0x55fa7ab8a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.652+0000 7f72f0391740 1 -- 172.21.15.83:0/1734433200 shutdown_connections 2024-04-23T14:49:41.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.652+0000 7f72f0391740 1 -- 172.21.15.83:0/1734433200 wait complete. 2024-04-23T14:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.652+0000 7f72f0391740 1 -- start start 2024-04-23T14:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.653+0000 7f72f0391740 1 Processor -- start 2024-04-23T14:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.653+0000 7f72f0391740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab83b40 0x55fa7ab8fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.653+0000 7f72f0391740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 0x55fa7ab900e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.653+0000 7f72f0391740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab95e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.653+0000 7f72f0391740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72c800cd40 con 0x55fa7ab83b40 2024-04-23T14:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.653+0000 7f72f0391740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f72c800e1b0 con 0x55fa7ab84460 2024-04-23T14:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.653+0000 7f72f0391740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f72c800d410 con 0x55fa7ab8eb30 2024-04-23T14:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72f0391740 10 client.0 _ll_register_callbacks cb 0x7f72d801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72e5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab83b40 0x55fa7ab8fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72e5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab83b40 0x55fa7ab8fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50736/0 (socket says 172.21.15.83:50736) 2024-04-23T14:49:41.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72e5e00640 1 -- 172.21.15.83:0/812029231 learned_addr learned my addr 172.21.15.83:0/812029231 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72e6800640 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 0x55fa7ab900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72e5e00640 1 -- 172.21.15.83:0/812029231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab8eb30 msgr2=0x55fa7ab95e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72e5e00640 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab95e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72e5e00640 1 -- 172.21.15.83:0/812029231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 msgr2=0x55fa7ab900e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72e5e00640 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 0x55fa7ab900e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72e5e00640 1 -- 172.21.15.83:0/812029231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7ab96460 con 0x55fa7ab83b40 2024-04-23T14:49:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72e6800640 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 0x55fa7ab900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72e5e00640 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab83b40 0x55fa7ab8fba0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f72d4005ec0 tx=0x7f72d4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28fb3b48682039d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.654+0000 7f72c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:49:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.655+0000 7f72c7e00640 1 -- 172.21.15.83:0/812029231 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4057060 con 0x55fa7ab83b40 2024-04-23T14:49:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.655+0000 7f72c7e00640 1 -- 172.21.15.83:0/812029231 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f72d4004230 con 0x55fa7ab83b40 2024-04-23T14:49:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.655+0000 7f72f0391740 1 -- 172.21.15.83:0/812029231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72d4005550 con 0x55fa7ab83b40 2024-04-23T14:49:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.655+0000 7f72c7e00640 1 -- 172.21.15.83:0/812029231 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f72d4035070 con 0x55fa7ab83b40 2024-04-23T14:49:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.655+0000 7f72f0391740 10 client.15522 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.655+0000 7f72f0391740 1 -- 172.21.15.83:0/812029231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55fa7ac04ff0 con 0x55fa7ab83b40 2024-04-23T14:49:41.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.655+0000 7f72c7e00640 1 -- 172.21.15.83:0/812029231 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d4005550 con 0x55fa7ab83b40 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.656+0000 7f72c7e00640 1 -- 172.21.15.83:0/812029231 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f72d403a040 con 0x55fa7ab83b40 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.656+0000 7f72f0391740 10 client.15522 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.656+0000 7f72f0391740 10 client.15522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.656+0000 7f72f0391740 20 client.15522 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.656+0000 7f72f0391740 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa7ac05500 0x55fa7ac07920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.656+0000 7f72f0391740 4 client.15522 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.656+0000 7f72f0391740 1 -- 172.21.15.83:0/812029231 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa7ab9b5c0 con 0x55fa7ac05500 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.656+0000 7f72e6800640 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa7ac05500 0x55fa7ac07920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72e6800640 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa7ac05500 0x55fa7ac07920 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=66cfb09aff6a0f82 server_cookie=b21c2587e35a3533 in_seq=0 out_seq=0 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72c7e00640 10 client.15522 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72c7e00640 1 -- 172.21.15.83:0/812029231 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa7ab9b5c0 con 0x55fa7ac05500 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72c7e00640 10 client.15522 handle_command_reply: tid=0 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72f0391740 1 client.15522 shutdown 2024-04-23T14:49:41.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72f0391740 1 -- 172.21.15.83:0/812029231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab83b40 msgr2=0x55fa7ab8fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72f0391740 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab83b40 0x55fa7ab8fba0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f72d4005ec0 tx=0x7f72d4005b50 comp rx=0 tx=0).stop 2024-04-23T14:49:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72f0391740 1 -- 172.21.15.83:0/812029231 shutdown_connections 2024-04-23T14:49:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72f0391740 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa7ac05500 0x55fa7ac07920 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72f0391740 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa7ab8eb30 0x55fa7ab95e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72f0391740 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa7ab84460 0x55fa7ab900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72f0391740 1 --2- 172.21.15.83:0/812029231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa7ab83b40 0x55fa7ab8fba0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72f0391740 1 -- 172.21.15.83:0/812029231 >> 172.21.15.83:0/812029231 conn(0x55fa7ab7fe60 msgr2=0x55fa7ab7ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.657+0000 7f72c7e00640 0 client.15522 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:49:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.658+0000 7f72f0391740 1 -- 172.21.15.83:0/812029231 shutdown_connections 2024-04-23T14:49:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.658+0000 7f72f0391740 1 -- 172.21.15.83:0/812029231 wait complete. 2024-04-23T14:49:41.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.658+0000 7f72f0391740 20 client.15522 trim_cache size 0 max 16384 2024-04-23T14:49:41.663 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:49:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.658+0000 7f72eca00640 1 -- 172.21.15.83:0/3188675960 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f72cc0a3f50 msgr2=0x7f72cc0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.658+0000 7f72eca00640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f72cc0a3f50 0x7f72cc0a6410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72e00ce0b0 tx=0x7f72e0098b40 comp rx=0 tx=0).stop 2024-04-23T14:49:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.658+0000 7f72eca00640 1 -- 172.21.15.83:0/3188675960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72e8106250 msgr2=0x7f72e81a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.658+0000 7f72eca00640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72e8106250 0x7f72e81a3670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f72e000eaa0 tx=0x7f72e000ef60 comp rx=0 tx=0).stop 2024-04-23T14:49:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.659+0000 7f72eca00640 1 -- 172.21.15.83:0/3188675960 shutdown_connections 2024-04-23T14:49:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.659+0000 7f72eca00640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72e8106250 0x7f72e81a3670 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.659+0000 7f72eca00640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f72cc0a3f50 0x7f72cc0a6410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.659+0000 7f72eca00640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72e8101770 0x7f72e819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.659+0000 7f72eca00640 1 --2- 172.21.15.83:0/3188675960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e8100e50 0x7f72e819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.659+0000 7f72eca00640 1 -- 172.21.15.83:0/3188675960 >> 172.21.15.83:0/3188675960 conn(0x7f72e80fc880 msgr2=0x7f72e80fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.659+0000 7f72eca00640 1 -- 172.21.15.83:0/3188675960 shutdown_connections 2024-04-23T14:49:41.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.659+0000 7f72eca00640 1 -- 172.21.15.83:0/3188675960 wait complete. 2024-04-23T14:49:41.676 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:49:41.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:41 smithi203 ceph-mon[32800]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:49:41.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:41 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2582027376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:49:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:41 smithi173 ceph-mon[32443]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T14:49:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:41 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2582027376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:49:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.845+0000 7f1c8fe00640 1 -- 172.21.15.83:0/2455301675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c90101770 msgr2=0x7f1c90105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.845+0000 7f1c8fe00640 1 --2- 172.21.15.83:0/2455301675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c90101770 0x7f1c90105c30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009900 tx=0x7f1c7c02a480 comp rx=0 tx=0).stop 2024-04-23T14:49:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.845+0000 7f1c8fe00640 1 -- 172.21.15.83:0/2455301675 shutdown_connections 2024-04-23T14:49:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.845+0000 7f1c8fe00640 1 --2- 172.21.15.83:0/2455301675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c90106250 0x7f1c901066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.845+0000 7f1c8fe00640 1 --2- 172.21.15.83:0/2455301675 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c90101770 0x7f1c90105c30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.845+0000 7f1c8fe00640 1 --2- 172.21.15.83:0/2455301675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c90100e50 0x7f1c90101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.845+0000 7f1c8fe00640 1 -- 172.21.15.83:0/2455301675 >> 172.21.15.83:0/2455301675 conn(0x7f1c900fc880 msgr2=0x7f1c900feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.845+0000 7f1c8fe00640 1 -- 172.21.15.83:0/2455301675 shutdown_connections 2024-04-23T14:49:41.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.845+0000 7f1c8fe00640 1 -- 172.21.15.83:0/2455301675 wait complete. 2024-04-23T14:49:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.847+0000 7f1c8fe00640 1 Processor -- start 2024-04-23T14:49:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.847+0000 7f1c8fe00640 1 -- start start 2024-04-23T14:49:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.847+0000 7f1c8fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c90100e50 0x7f1c9019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.847+0000 7f1c8fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c90101770 0x7f1c9019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.847+0000 7f1c8fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c90106250 0x7f1c901a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.847+0000 7f1c8fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90118860 con 0x7f1c90106250 2024-04-23T14:49:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.847+0000 7f1c8fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901186e0 con 0x7f1c90100e50 2024-04-23T14:49:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.847+0000 7f1c8fe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901189e0 con 0x7f1c90101770 2024-04-23T14:49:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.848+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c90100e50 0x7f1c9019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.848+0000 7f1c8f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c90106250 0x7f1c901a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.848+0000 7f1c8f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c90106250 0x7f1c901a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50762/0 (socket says 172.21.15.83:50762) 2024-04-23T14:49:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.848+0000 7f1c8f400640 1 -- 172.21.15.83:0/1435220320 learned_addr learned my addr 172.21.15.83:0/1435220320 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.848+0000 7f1c8e000640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c90101770 0x7f1c9019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.848+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c90100e50 0x7f1c9019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37354/0 (socket says 172.21.15.83:37354) 2024-04-23T14:49:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.848+0000 7f1c8e000640 1 -- 172.21.15.83:0/1435220320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c90100e50 msgr2=0x7f1c9019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.848+0000 7f1c8e000640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c90100e50 0x7f1c9019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.848+0000 7f1c8e000640 1 -- 172.21.15.83:0/1435220320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c90106250 msgr2=0x7f1c901a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.848+0000 7f1c8e000640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c90106250 0x7f1c901a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.848+0000 7f1c8e000640 1 -- 172.21.15.83:0/1435220320 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c901a3d70 con 0x7f1c90101770 2024-04-23T14:49:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.848+0000 7f1c8f400640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c90106250 0x7f1c901a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.849+0000 7f1c8e000640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c90101770 0x7f1c9019f2c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c005ab0 tx=0x7f1c7c032c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.849+0000 7f1c87400640 1 -- 172.21.15.83:0/1435220320 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c03f030 con 0x7f1c90101770 2024-04-23T14:49:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.849+0000 7f1c87400640 1 -- 172.21.15.83:0/1435220320 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7c032e50 con 0x7f1c90101770 2024-04-23T14:49:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.849+0000 7f1c87400640 1 -- 172.21.15.83:0/1435220320 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c035070 con 0x7f1c90101770 2024-04-23T14:49:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.849+0000 7f1c8fe00640 1 -- 172.21.15.83:0/1435220320 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c901a4000 con 0x7f1c90101770 2024-04-23T14:49:41.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.849+0000 7f1c8fe00640 1 -- 172.21.15.83:0/1435220320 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c901ab8f0 con 0x7f1c90101770 2024-04-23T14:49:41.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.853+0000 7f1c98b82740 1 Processor -- start 2024-04-23T14:49:41.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.853+0000 7f1c98b82740 1 -- start start 2024-04-23T14:49:41.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.854+0000 7f1c98b82740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 0x556f2fcc8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.854+0000 7f1c98b82740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc9460 0x556f2fcd35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.854+0000 7f1c98b82740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcd3b30 0x556f2fcd5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.854+0000 7f1c98b82740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f2fc2e1d0 con 0x556f2fcd3b30 2024-04-23T14:49:41.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.854+0000 7f1c98b82740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556f2fbd4450 con 0x556f2fcc9460 2024-04-23T14:49:41.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.854+0000 7f1c98b82740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556f2fc2a190 con 0x556f2fcc8b40 2024-04-23T14:49:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.854+0000 7f1c8f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 0x556f2fcc8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.854+0000 7f1c8f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 0x556f2fcc8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40348/0 (socket says 172.21.15.83:40348) 2024-04-23T14:49:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.854+0000 7f1c8f400640 1 -- 172.21.15.83:0/699758677 learned_addr learned my addr 172.21.15.83:0/699758677 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.854+0000 7f1c8ea00640 1 --2- 172.21.15.83:0/699758677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc9460 0x556f2fcd35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.855+0000 7f1c8f400640 1 -- 172.21.15.83:0/699758677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc9460 msgr2=0x556f2fcd35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.855+0000 7f1c8f400640 1 --2- 172.21.15.83:0/699758677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc9460 0x556f2fcd35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.855+0000 7f1c8f400640 1 -- 172.21.15.83:0/699758677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcd3b30 msgr2=0x556f2fcd5f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:49:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.855+0000 7f1c8f400640 1 --2- 172.21.15.83:0/699758677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcd3b30 0x556f2fcd5f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.855+0000 7f1c8f400640 1 -- 172.21.15.83:0/699758677 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f2fc91310 con 0x556f2fcc8b40 2024-04-23T14:49:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.855+0000 7f1c87400640 1 -- 172.21.15.83:0/1435220320 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f1c7c042030 con 0x7f1c90101770 2024-04-23T14:49:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.855+0000 7f1c8f400640 1 --2- 172.21.15.83:0/699758677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 0x556f2fcc8f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1c88011850 tx=0x7f1c88011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e622d6ca59ff3296 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.855+0000 7f1c7be00640 1 -- 172.21.15.83:0/699758677 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88011330 con 0x556f2fcc8b40 2024-04-23T14:49:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.856+0000 7f1c7be00640 1 -- 172.21.15.83:0/699758677 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c8800fe80 con 0x556f2fcc8b40 2024-04-23T14:49:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.856+0000 7f1c7be00640 1 -- 172.21.15.83:0/699758677 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c880193e0 con 0x556f2fcc8b40 2024-04-23T14:49:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.856+0000 7f1c87400640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1c740a3fd0 0x7f1c740a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.856+0000 7f1c98b82740 1 -- 172.21.15.83:0/699758677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 msgr2=0x556f2fcc8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.856+0000 7f1c98b82740 1 --2- 172.21.15.83:0/699758677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 0x556f2fcc8f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1c88011850 tx=0x7f1c88011d10 comp rx=0 tx=0).stop 2024-04-23T14:49:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.857+0000 7f1c87400640 1 -- 172.21.15.83:0/1435220320 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1c7c0f0ba0 con 0x7f1c90101770 2024-04-23T14:49:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.857+0000 7f1c98b82740 1 -- 172.21.15.83:0/699758677 shutdown_connections 2024-04-23T14:49:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.857+0000 7f1c98b82740 1 --2- 172.21.15.83:0/699758677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcd3b30 0x556f2fcd5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.857+0000 7f1c98b82740 1 --2- 172.21.15.83:0/699758677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc9460 0x556f2fcd35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.857+0000 7f1c98b82740 1 --2- 172.21.15.83:0/699758677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 0x556f2fcc8f20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.857+0000 7f1c98b82740 1 -- 172.21.15.83:0/699758677 >> 172.21.15.83:0/699758677 conn(0x556f2fcc4e60 msgr2=0x556f2fcc5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.857+0000 7f1c98b82740 1 -- 172.21.15.83:0/699758677 shutdown_connections 2024-04-23T14:49:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.857+0000 7f1c8ea00640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1c740a3fd0 0x7f1c740a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.857+0000 7f1c98b82740 1 -- 172.21.15.83:0/699758677 wait complete. 2024-04-23T14:49:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.858+0000 7f1c98b82740 1 -- start start 2024-04-23T14:49:41.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.858+0000 7f1c8ea00640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1c740a3fd0 0x7f1c740a6490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x556f2fcca100 tx=0x7f1c80005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.858+0000 7f1c98b82740 1 Processor -- start 2024-04-23T14:49:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.859+0000 7f1c98b82740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 0x556f2fcd3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.859+0000 7f1c98b82740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc9460 0x556f2fcd0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.859+0000 7f1c98b82740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcd3b30 0x556f2fcd0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.859+0000 7f1c98b82740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556f2fcd69c0 con 0x556f2fcd3b30 2024-04-23T14:49:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.859+0000 7f1c98b82740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556f2fbe7810 con 0x556f2fcc8b40 2024-04-23T14:49:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.859+0000 7f1c98b82740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556f2fcd6b40 con 0x556f2fcc9460 2024-04-23T14:49:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.859+0000 7f1c98b82740 10 client.0 _ll_register_callbacks cb 0x556f2fbb5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.859+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 0x556f2fcd3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.859+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 0x556f2fcd3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37380/0 (socket says 172.21.15.83:37380) 2024-04-23T14:49:41.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.859+0000 7f1c8ea00640 1 -- 172.21.15.83:0/2323887715 learned_addr learned my addr 172.21.15.83:0/2323887715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.859+0000 7f1c8e000640 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc9460 0x556f2fcd0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.859+0000 7f1c8f400640 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcd3b30 0x556f2fcd0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.860+0000 7f1c8ea00640 1 -- 172.21.15.83:0/2323887715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc9460 msgr2=0x556f2fcd0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.860+0000 7f1c8ea00640 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc9460 0x556f2fcd0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.860+0000 7f1c8ea00640 1 -- 172.21.15.83:0/2323887715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcd3b30 msgr2=0x556f2fcd0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.860+0000 7f1c8ea00640 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcd3b30 0x556f2fcd0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.860+0000 7f1c8ea00640 1 -- 172.21.15.83:0/2323887715 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f2fbf83f0 con 0x556f2fcc8b40 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.860+0000 7f1c8ea00640 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 0x556f2fcd3550 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1c800042a0 tx=0x7f1c80004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b957b6ae75d8e53f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.860+0000 7f1c78c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.860+0000 7f1c78c00640 1 -- 172.21.15.83:0/2323887715 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80029440 con 0x556f2fcc8b40 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.860+0000 7f1c98b82740 1 -- 172.21.15.83:0/2323887715 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556f2fc3bbf0 con 0x556f2fcc8b40 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.860+0000 7f1c78c00640 1 -- 172.21.15.83:0/2323887715 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1c8001f420 con 0x556f2fcc8b40 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.861+0000 7f1c78c00640 1 -- 172.21.15.83:0/2323887715 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1c8001f620 con 0x556f2fcc8b40 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.861+0000 7f1c98b82740 10 client.25649 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.861+0000 7f1c98b82740 1 -- 172.21.15.83:0/2323887715 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556f2f9403f0 con 0x556f2fcc8b40 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.861+0000 7f1c78c00640 1 -- 172.21.15.83:0/2323887715 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556f2fc3bbf0 con 0x556f2fcc8b40 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.861+0000 7f1c78c00640 1 -- 172.21.15.83:0/2323887715 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1c80027070 con 0x556f2fcc8b40 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.862+0000 7f1c98b82740 10 client.25649 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.862+0000 7f1c98b82740 10 client.25649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:41.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.862+0000 7f1c98b82740 20 client.25649 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:41.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.862+0000 7f1c98b82740 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556f2fce3ce0 0x556f2fce6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.862+0000 7f1c98b82740 4 client.25649 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:49:41.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.862+0000 7f1c98b82740 1 -- 172.21.15.83:0/2323887715 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f2fc35a60 con 0x556f2fce3ce0 2024-04-23T14:49:41.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.862+0000 7f1c8e000640 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556f2fce3ce0 0x556f2fce6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.862+0000 7f1c8e000640 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556f2fce3ce0 0x556f2fce6110 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d91316c0842c844c server_cookie=554aaa155c9f88cd in_seq=0 out_seq=0 2024-04-23T14:49:41.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.862+0000 7f1c78c00640 10 client.25649 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:41.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.863+0000 7f1c78c00640 1 -- 172.21.15.83:0/2323887715 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f2fc35a60 con 0x556f2fce3ce0 2024-04-23T14:49:41.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.863+0000 7f1c78c00640 10 client.25649 handle_command_reply: tid=0 2024-04-23T14:49:41.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.863+0000 7f1c98b82740 1 client.25649 shutdown 2024-04-23T14:49:41.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.864+0000 7f1c98b82740 1 -- 172.21.15.83:0/2323887715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 msgr2=0x556f2fcd3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.864+0000 7f1c98b82740 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 0x556f2fcd3550 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1c800042a0 tx=0x7f1c80004730 comp rx=0 tx=0).stop 2024-04-23T14:49:41.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.864+0000 7f1c98b82740 1 -- 172.21.15.83:0/2323887715 shutdown_connections 2024-04-23T14:49:41.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.864+0000 7f1c98b82740 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556f2fce3ce0 0x556f2fce6110 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.864+0000 7f1c98b82740 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcd3b30 0x556f2fcd0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.864+0000 7f1c98b82740 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc9460 0x556f2fcd0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.864+0000 7f1c98b82740 1 --2- 172.21.15.83:0/2323887715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 0x556f2fcd3550 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.864+0000 7f1c98b82740 1 -- 172.21.15.83:0/2323887715 >> 172.21.15.83:0/2323887715 conn(0x556f2fcc4e60 msgr2=0x556f2fccac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.864+0000 7f1c78c00640 0 client.25649 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:41.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.864+0000 7f1c98b82740 1 -- 172.21.15.83:0/2323887715 shutdown_connections 2024-04-23T14:49:41.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.864+0000 7f1c98b82740 1 -- 172.21.15.83:0/2323887715 wait complete. 2024-04-23T14:49:41.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.864+0000 7f1c98b82740 20 client.25649 trim_cache size 0 max 16384 2024-04-23T14:49:41.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.878+0000 7f1c98b82740 1 Processor -- start 2024-04-23T14:49:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.878+0000 7f1c98b82740 1 -- start start 2024-04-23T14:49:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.879+0000 7f1c98b82740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 0x556f2fcc3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.879+0000 7f1c98b82740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcc9460 0x556f2fcd3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.879+0000 7f1c98b82740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcd3b30 0x556f2fceb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.879+0000 7f1c98b82740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901b6a40 con 0x556f2fcc9460 2024-04-23T14:49:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.879+0000 7f1c98b82740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901b4310 con 0x556f2fcd3b30 2024-04-23T14:49:41.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.879+0000 7f1c98b82740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901b2160 con 0x556f2fcc8b40 2024-04-23T14:49:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.879+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 0x556f2fcc3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.879+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 0x556f2fcc3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40358/0 (socket says 172.21.15.83:40358) 2024-04-23T14:49:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.879+0000 7f1c8ea00640 1 -- 172.21.15.83:0/626160921 learned_addr learned my addr 172.21.15.83:0/626160921 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.879+0000 7f1c8f400640 1 --2- 172.21.15.83:0/626160921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcd3b30 0x556f2fceb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.879+0000 7f1c8e000640 1 --2- 172.21.15.83:0/626160921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcc9460 0x556f2fcd3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.880+0000 7f1c8ea00640 1 -- 172.21.15.83:0/626160921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcd3b30 msgr2=0x556f2fceb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.880+0000 7f1c8ea00640 1 --2- 172.21.15.83:0/626160921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcd3b30 0x556f2fceb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.880+0000 7f1c8ea00640 1 -- 172.21.15.83:0/626160921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcc9460 msgr2=0x556f2fcd3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.880+0000 7f1c8ea00640 1 --2- 172.21.15.83:0/626160921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcc9460 0x556f2fcd3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.880+0000 7f1c8ea00640 1 -- 172.21.15.83:0/626160921 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f2fa0a7d0 con 0x556f2fcc8b40 2024-04-23T14:49:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.880+0000 7f1c8e000640 1 --2- 172.21.15.83:0/626160921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcc9460 0x556f2fcd3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:49:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.880+0000 7f1c8ea00640 1 --2- 172.21.15.83:0/626160921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 0x556f2fcc3dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1c800128e0 tx=0x7f1c80004620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ded94f51a03f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.880+0000 7f1c7b400640 1 -- 172.21.15.83:0/626160921 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80029440 con 0x556f2fcc8b40 2024-04-23T14:49:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.880+0000 7f1c7b400640 1 -- 172.21.15.83:0/626160921 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c8001f7c0 con 0x556f2fcc8b40 2024-04-23T14:49:41.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.880+0000 7f1c7b400640 1 -- 172.21.15.83:0/626160921 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80031900 con 0x556f2fcc8b40 2024-04-23T14:49:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.881+0000 7f1c98b82740 1 -- 172.21.15.83:0/626160921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 msgr2=0x556f2fcc3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.881+0000 7f1c98b82740 1 --2- 172.21.15.83:0/626160921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 0x556f2fcc3dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1c800128e0 tx=0x7f1c80004620 comp rx=0 tx=0).stop 2024-04-23T14:49:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.881+0000 7f1c98b82740 1 -- 172.21.15.83:0/626160921 shutdown_connections 2024-04-23T14:49:41.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.881+0000 7f1c98b82740 1 --2- 172.21.15.83:0/626160921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcd3b30 0x556f2fceb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.881+0000 7f1c98b82740 1 --2- 172.21.15.83:0/626160921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcc9460 0x556f2fcd3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.881+0000 7f1c98b82740 1 --2- 172.21.15.83:0/626160921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcc8b40 0x556f2fcc3dd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.881+0000 7f1c98b82740 1 -- 172.21.15.83:0/626160921 >> 172.21.15.83:0/626160921 conn(0x556f2fcc4e60 msgr2=0x556f2fcc36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.881+0000 7f1c98b82740 1 -- 172.21.15.83:0/626160921 shutdown_connections 2024-04-23T14:49:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.881+0000 7f1c98b82740 1 -- 172.21.15.83:0/626160921 wait complete. 2024-04-23T14:49:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.882+0000 7f1c98b82740 1 -- start start 2024-04-23T14:49:41.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.882+0000 7f1c98b82740 1 Processor -- start 2024-04-23T14:49:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c98b82740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 0x556f2fcdc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c98b82740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcc9460 0x556f2fccdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c98b82740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcd3b30 0x556f2fcce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c98b82740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901b27d0 con 0x556f2fcc9460 2024-04-23T14:49:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c98b82740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901b3c40 con 0x556f2fcc8b40 2024-04-23T14:49:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c98b82740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1c901b2ea0 con 0x556f2fcd3b30 2024-04-23T14:49:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c98b82740 10 client.0 _ll_register_callbacks cb 0x7f1c800041b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:41.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c8f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcd3b30 0x556f2fcce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 0x556f2fcdc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c8f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcd3b30 0x556f2fcce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40370/0 (socket says 172.21.15.83:40370) 2024-04-23T14:49:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c8f400640 1 -- 172.21.15.83:0/4084708306 learned_addr learned my addr 172.21.15.83:0/4084708306 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c8e000640 1 --2- 172.21.15.83:0/4084708306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcc9460 0x556f2fccdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c8ea00640 1 -- 172.21.15.83:0/4084708306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcd3b30 msgr2=0x556f2fcce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c8ea00640 1 --2- 172.21.15.83:0/4084708306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcd3b30 0x556f2fcce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c8ea00640 1 -- 172.21.15.83:0/4084708306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcc9460 msgr2=0x556f2fccdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c8ea00640 1 --2- 172.21.15.83:0/4084708306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcc9460 0x556f2fccdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.883+0000 7f1c8ea00640 1 -- 172.21.15.83:0/4084708306 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f2fcd4710 con 0x556f2fcc8b40 2024-04-23T14:49:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.884+0000 7f1c8ea00640 1 --2- 172.21.15.83:0/4084708306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 0x556f2fcdc260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1c80021040 tx=0x7f1c800057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cac9dccdea645cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:41.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.884+0000 7f1c78c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.884+0000 7f1c78c00640 1 -- 172.21.15.83:0/4084708306 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80005450 con 0x556f2fcc8b40 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.884+0000 7f1c78c00640 1 -- 172.21.15.83:0/4084708306 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1c80029440 con 0x556f2fcc8b40 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.884+0000 7f1c98b82740 1 -- 172.21.15.83:0/4084708306 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c8001f7c0 con 0x556f2fcc8b40 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.884+0000 7f1c78c00640 1 -- 172.21.15.83:0/4084708306 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1c8004c050 con 0x556f2fcc8b40 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.885+0000 7f1c98b82740 10 client.25655 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.885+0000 7f1c98b82740 1 -- 172.21.15.83:0/4084708306 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x556f2fd4a3d0 con 0x556f2fcc8b40 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.885+0000 7f1c78c00640 1 -- 172.21.15.83:0/4084708306 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c8001f7c0 con 0x556f2fcc8b40 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.885+0000 7f1c78c00640 1 -- 172.21.15.83:0/4084708306 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1c80043070 con 0x556f2fcc8b40 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.885+0000 7f1c98b82740 10 client.25655 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.885+0000 7f1c98b82740 10 client.25655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.885+0000 7f1c98b82740 20 client.25655 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.885+0000 7f1c98b82740 1 --2- 172.21.15.83:0/4084708306 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556f2fd4a9e0 0x556f2fd4ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:41.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.885+0000 7f1c98b82740 4 client.25655 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.885+0000 7f1c98b82740 1 -- 172.21.15.83:0/4084708306 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f2fbb14d0 con 0x556f2fd4a9e0 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.885+0000 7f1c8e000640 1 --2- 172.21.15.83:0/4084708306 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556f2fd4a9e0 0x556f2fd4ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.886+0000 7f1c8e000640 1 --2- 172.21.15.83:0/4084708306 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556f2fd4a9e0 0x556f2fd4ce00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea76c62d0dc212a5 server_cookie=ddd0db62fea687a2 in_seq=0 out_seq=0 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.886+0000 7f1c78c00640 10 client.25655 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.886+0000 7f1c78c00640 1 -- 172.21.15.83:0/4084708306 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f2fbb14d0 con 0x556f2fd4a9e0 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.886+0000 7f1c78c00640 10 client.25655 handle_command_reply: tid=0 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.886+0000 7f1c98b82740 1 client.25655 shutdown 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.887+0000 7f1c98b82740 1 -- 172.21.15.83:0/4084708306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 msgr2=0x556f2fcdc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.887+0000 7f1c98b82740 1 --2- 172.21.15.83:0/4084708306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 0x556f2fcdc260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1c80021040 tx=0x7f1c800057f0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.887+0000 7f1c98b82740 1 -- 172.21.15.83:0/4084708306 shutdown_connections 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.887+0000 7f1c98b82740 1 --2- 172.21.15.83:0/4084708306 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556f2fd4a9e0 0x556f2fd4ce00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.887+0000 7f1c98b82740 1 --2- 172.21.15.83:0/4084708306 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f2fcd3b30 0x556f2fcce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.887+0000 7f1c98b82740 1 --2- 172.21.15.83:0/4084708306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556f2fcc9460 0x556f2fccdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.887+0000 7f1c98b82740 1 --2- 172.21.15.83:0/4084708306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556f2fcc8b40 0x556f2fcdc260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.887+0000 7f1c98b82740 1 -- 172.21.15.83:0/4084708306 >> 172.21.15.83:0/4084708306 conn(0x556f2fcc4e60 msgr2=0x556f2fcc3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.887+0000 7f1c78c00640 0 client.25655 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.887+0000 7f1c98b82740 1 -- 172.21.15.83:0/4084708306 shutdown_connections 2024-04-23T14:49:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.887+0000 7f1c98b82740 1 -- 172.21.15.83:0/4084708306 wait complete. 2024-04-23T14:49:41.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.887+0000 7f1c98b82740 20 client.25655 trim_cache size 0 max 16384 2024-04-23T14:49:41.891 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:49:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.888+0000 7f1c8fe00640 1 -- 172.21.15.83:0/1435220320 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1c740a3fd0 msgr2=0x7f1c740a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.888+0000 7f1c8fe00640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1c740a3fd0 0x7f1c740a6490 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x556f2fcca100 tx=0x7f1c80005f90 comp rx=0 tx=0).stop 2024-04-23T14:49:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.888+0000 7f1c8fe00640 1 -- 172.21.15.83:0/1435220320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c90101770 msgr2=0x7f1c9019f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.888+0000 7f1c8fe00640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c90101770 0x7f1c9019f2c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c005ab0 tx=0x7f1c7c032c60 comp rx=0 tx=0).stop 2024-04-23T14:49:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.888+0000 7f1c8fe00640 1 -- 172.21.15.83:0/1435220320 shutdown_connections 2024-04-23T14:49:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.888+0000 7f1c8fe00640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1c740a3fd0 0x7f1c740a6490 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.888+0000 7f1c8fe00640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c90106250 0x7f1c901a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.888+0000 7f1c8fe00640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c90101770 0x7f1c9019f2c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.888+0000 7f1c8fe00640 1 --2- 172.21.15.83:0/1435220320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c90100e50 0x7f1c9019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.888+0000 7f1c8fe00640 1 -- 172.21.15.83:0/1435220320 >> 172.21.15.83:0/1435220320 conn(0x7f1c900fc880 msgr2=0x7f1c900fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:41.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.888+0000 7f1c8fe00640 1 -- 172.21.15.83:0/1435220320 shutdown_connections 2024-04-23T14:49:41.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:41.888+0000 7f1c8fe00640 1 -- 172.21.15.83:0/1435220320 wait complete. 2024-04-23T14:49:42.906 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:49:42.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:49:42.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:49:43.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.172+0000 7f1267e00640 1 -- 172.21.15.83:0/3056556284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1268101770 msgr2=0x7f1268105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.172+0000 7f1267e00640 1 --2- 172.21.15.83:0/3056556284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1268101770 0x7f1268105c30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1254009900 tx=0x7f125402a470 comp rx=0 tx=0).stop 2024-04-23T14:49:43.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.172+0000 7f1267e00640 1 -- 172.21.15.83:0/3056556284 shutdown_connections 2024-04-23T14:49:43.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.172+0000 7f1267e00640 1 --2- 172.21.15.83:0/3056556284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268106250 0x7f12681066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.172+0000 7f1267e00640 1 --2- 172.21.15.83:0/3056556284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1268101770 0x7f1268105c30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.172+0000 7f1267e00640 1 --2- 172.21.15.83:0/3056556284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1268100e50 0x7f1268101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.172+0000 7f1267e00640 1 -- 172.21.15.83:0/3056556284 >> 172.21.15.83:0/3056556284 conn(0x7f12680fc880 msgr2=0x7f12680feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.172+0000 7f1267e00640 1 -- 172.21.15.83:0/3056556284 shutdown_connections 2024-04-23T14:49:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.172+0000 7f1267e00640 1 -- 172.21.15.83:0/3056556284 wait complete. 2024-04-23T14:49:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.173+0000 7f1267e00640 1 Processor -- start 2024-04-23T14:49:43.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.173+0000 7f1267e00640 1 -- start start 2024-04-23T14:49:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1268100e50 0x7f126819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1268101770 0x7f126819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268106250 0x7f12681a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12681188a0 con 0x7f1268101770 2024-04-23T14:49:43.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1268118720 con 0x7f1268106250 2024-04-23T14:49:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1268118a20 con 0x7f1268100e50 2024-04-23T14:49:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268106250 0x7f12681a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268106250 0x7f12681a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37424/0 (socket says 172.21.15.83:37424) 2024-04-23T14:49:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267400640 1 -- 172.21.15.83:0/3151249598 learned_addr learned my addr 172.21.15.83:0/3151249598 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1266a00640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1268100e50 0x7f126819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1266000640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1268101770 0x7f126819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267400640 1 -- 172.21.15.83:0/3151249598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1268100e50 msgr2=0x7f126819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267400640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1268100e50 0x7f126819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267400640 1 -- 172.21.15.83:0/3151249598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1268101770 msgr2=0x7f126819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267400640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1268101770 0x7f126819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.174+0000 7f1267400640 1 -- 172.21.15.83:0/3151249598 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12681a3d70 con 0x7f1268106250 2024-04-23T14:49:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.175+0000 7f1266000640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1268101770 0x7f126819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:49:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.175+0000 7f1267400640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268106250 0x7f12681a3640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f126000eb10 tx=0x7f126000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:43.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.175+0000 7f125f400640 1 -- 172.21.15.83:0/3151249598 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260004b20 con 0x7f1268106250 2024-04-23T14:49:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.175+0000 7f125f400640 1 -- 172.21.15.83:0/3151249598 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1260004cc0 con 0x7f1268106250 2024-04-23T14:49:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.175+0000 7f1267e00640 1 -- 172.21.15.83:0/3151249598 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12681a4060 con 0x7f1268106250 2024-04-23T14:49:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.175+0000 7f1267e00640 1 -- 172.21.15.83:0/3151249598 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12681ab8f0 con 0x7f1268106250 2024-04-23T14:49:43.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.175+0000 7f125f400640 1 -- 172.21.15.83:0/3151249598 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260010760 con 0x7f1268106250 2024-04-23T14:49:43.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.179+0000 7f125f400640 1 -- 172.21.15.83:0/3151249598 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f1260005020 con 0x7f1268106250 2024-04-23T14:49:43.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.180+0000 7f127041d740 1 Processor -- start 2024-04-23T14:49:43.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.180+0000 7f127041d740 1 -- start start 2024-04-23T14:49:43.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.180+0000 7f127041d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 0x562521c2d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.180+0000 7f127041d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c2db90 0x562521c37d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.180+0000 7f127041d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c38260 0x562521c3a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.180+0000 7f127041d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562521b92e70 con 0x562521c2d270 2024-04-23T14:49:43.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.180+0000 7f127041d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562521b39380 con 0x562521c38260 2024-04-23T14:49:43.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.180+0000 7f127041d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562521b8ee30 con 0x562521c2db90 2024-04-23T14:49:43.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.180+0000 7f125f400640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f124c0a3ed0 0x7f124c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.180+0000 7f1266000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 0x562521c2d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.181+0000 7f1266000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 0x562521c2d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50812/0 (socket says 172.21.15.83:50812) 2024-04-23T14:49:43.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.181+0000 7f1266000640 1 -- 172.21.15.83:0/3828362658 learned_addr learned my addr 172.21.15.83:0/3828362658 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:43.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.181+0000 7f1266a00640 1 --2- 172.21.15.83:0/3828362658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c2db90 0x562521c37d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.181+0000 7f125f400640 1 -- 172.21.15.83:0/3151249598 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f12600d0f60 con 0x7f1268106250 2024-04-23T14:49:43.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.181+0000 7f1266a00640 1 -- 172.21.15.83:0/3828362658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c38260 msgr2=0x562521c3a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.181+0000 7f1266a00640 1 --2- 172.21.15.83:0/3828362658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c38260 0x562521c3a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.181+0000 7f1266a00640 1 -- 172.21.15.83:0/3828362658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 msgr2=0x562521c2d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.181+0000 7f1266a00640 1 --2- 172.21.15.83:0/3828362658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 0x562521c2d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.181+0000 7f1266a00640 1 -- 172.21.15.83:0/3828362658 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252196d7e0 con 0x562521c2db90 2024-04-23T14:49:43.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.181+0000 7f1267400640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f124c0a3ed0 0x7f124c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.182+0000 7f1266a00640 1 --2- 172.21.15.83:0/3828362658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c2db90 0x562521c37d20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1258009c20 tx=0x7f1258011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35a534c40bc246fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:43.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.182+0000 7f1267400640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f124c0a3ed0 0x7f124c0a6390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f12600ce4e0 tx=0x7f1260009a70 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:43.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.182+0000 7f1253400640 1 -- 172.21.15.83:0/3828362658 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258009dd0 con 0x562521c2db90 2024-04-23T14:49:43.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.182+0000 7f1253400640 1 -- 172.21.15.83:0/3828362658 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1258005020 con 0x562521c2db90 2024-04-23T14:49:43.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.182+0000 7f1253400640 1 -- 172.21.15.83:0/3828362658 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258004b90 con 0x562521c2db90 2024-04-23T14:49:43.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.183+0000 7f127041d740 1 -- 172.21.15.83:0/3828362658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c2db90 msgr2=0x562521c37d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.183+0000 7f127041d740 1 --2- 172.21.15.83:0/3828362658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c2db90 0x562521c37d20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1258009c20 tx=0x7f1258011810 comp rx=0 tx=0).stop 2024-04-23T14:49:43.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.183+0000 7f127041d740 1 -- 172.21.15.83:0/3828362658 shutdown_connections 2024-04-23T14:49:43.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.183+0000 7f127041d740 1 --2- 172.21.15.83:0/3828362658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c38260 0x562521c3a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.183+0000 7f127041d740 1 --2- 172.21.15.83:0/3828362658 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c2db90 0x562521c37d20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.183+0000 7f127041d740 1 --2- 172.21.15.83:0/3828362658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 0x562521c2d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.183+0000 7f127041d740 1 -- 172.21.15.83:0/3828362658 >> 172.21.15.83:0/3828362658 conn(0x562521c29590 msgr2=0x562521c29970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:43.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.183+0000 7f127041d740 1 -- 172.21.15.83:0/3828362658 shutdown_connections 2024-04-23T14:49:43.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.184+0000 7f127041d740 1 -- 172.21.15.83:0/3828362658 wait complete. 2024-04-23T14:49:43.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.185+0000 7f127041d740 1 -- start start 2024-04-23T14:49:43.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.185+0000 7f127041d740 1 Processor -- start 2024-04-23T14:49:43.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f127041d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 0x562521c35160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f127041d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2db90 0x562521c356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f127041d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c38260 0x562521c35be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f127041d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562521c3b050 con 0x562521c2d270 2024-04-23T14:49:43.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f127041d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562521b4c660 con 0x562521c2db90 2024-04-23T14:49:43.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f127041d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562521c3b1d0 con 0x562521c38260 2024-04-23T14:49:43.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f127041d740 10 client.0 _ll_register_callbacks cb 0x562521b3aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:43.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f1266000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 0x562521c35160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f1266000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 0x562521c35160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50818/0 (socket says 172.21.15.83:50818) 2024-04-23T14:49:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f1266000640 1 -- 172.21.15.83:0/1549113865 learned_addr learned my addr 172.21.15.83:0/1549113865 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f1266a00640 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2db90 0x562521c356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f1266000640 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c38260 0x562521c35be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f1266a00640 1 -- 172.21.15.83:0/1549113865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c38260 msgr2=0x562521c35be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f1266a00640 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c38260 0x562521c35be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f1266a00640 1 -- 172.21.15.83:0/1549113865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 msgr2=0x562521c35160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f1266a00640 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 0x562521c35160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f1266a00640 1 -- 172.21.15.83:0/1549113865 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625219cdf70 con 0x562521c2db90 2024-04-23T14:49:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.186+0000 7f1266000640 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 0x562521c35160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:49:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.187+0000 7f1266a00640 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2db90 0x562521c356a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1258000c00 tx=0x7f1258009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac5dfedd6c99c286 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:43.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.187+0000 7f1247e00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.187+0000 7f1247e00640 1 -- 172.21.15.83:0/1549113865 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258029020 con 0x562521c2db90 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.187+0000 7f127041d740 1 -- 172.21.15.83:0/1549113865 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1258005020 con 0x562521c2db90 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.187+0000 7f1247e00640 1 -- 172.21.15.83:0/1549113865 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1258004b20 con 0x562521c2db90 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.188+0000 7f1247e00640 1 -- 172.21.15.83:0/1549113865 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1258034050 con 0x562521c2db90 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.188+0000 7f127041d740 10 client.25664 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.188+0000 7f127041d740 1 -- 172.21.15.83:0/1549113865 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562521865170 con 0x562521c2db90 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.188+0000 7f1247e00640 1 -- 172.21.15.83:0/1549113865 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1258005020 con 0x562521c2db90 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.188+0000 7f1247e00640 1 -- 172.21.15.83:0/1549113865 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f12580304a0 con 0x562521c2db90 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.188+0000 7f127041d740 10 client.25664 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.188+0000 7f127041d740 10 client.25664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.189+0000 7f127041d740 20 client.25664 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.189+0000 7f127041d740 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562521c485d0 0x562521c4a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.189+0000 7f127041d740 4 client.25664 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:49:43.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.189+0000 7f127041d740 1 -- 172.21.15.83:0/1549113865 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562521ba0890 con 0x562521c485d0 2024-04-23T14:49:43.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.189+0000 7f1266000640 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562521c485d0 0x562521c4a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.189+0000 7f1247e00640 10 client.25664 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:43.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.189+0000 7f1266000640 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562521c485d0 0x562521c4a9c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3af837170ff1716f server_cookie=d866cbd2a8075275 in_seq=0 out_seq=0 2024-04-23T14:49:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.190+0000 7f1247e00640 1 -- 172.21.15.83:0/1549113865 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562521ba0890 con 0x562521c485d0 2024-04-23T14:49:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.190+0000 7f1247e00640 10 client.25664 handle_command_reply: tid=0 2024-04-23T14:49:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.190+0000 7f127041d740 1 client.25664 shutdown 2024-04-23T14:49:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.191+0000 7f127041d740 1 -- 172.21.15.83:0/1549113865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2db90 msgr2=0x562521c356a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.191+0000 7f127041d740 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2db90 0x562521c356a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1258000c00 tx=0x7f1258009890 comp rx=0 tx=0).stop 2024-04-23T14:49:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.191+0000 7f127041d740 1 -- 172.21.15.83:0/1549113865 shutdown_connections 2024-04-23T14:49:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.191+0000 7f127041d740 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562521c485d0 0x562521c4a9c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.191+0000 7f1247e00640 0 client.25664 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:43.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.191+0000 7f127041d740 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c38260 0x562521c35be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.191+0000 7f127041d740 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2db90 0x562521c356a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.191+0000 7f127041d740 1 --2- 172.21.15.83:0/1549113865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2d270 0x562521c35160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.191+0000 7f127041d740 1 -- 172.21.15.83:0/1549113865 >> 172.21.15.83:0/1549113865 conn(0x562521c29590 msgr2=0x562521c2f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:43.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.191+0000 7f127041d740 1 -- 172.21.15.83:0/1549113865 shutdown_connections 2024-04-23T14:49:43.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.191+0000 7f127041d740 1 -- 172.21.15.83:0/1549113865 wait complete. 2024-04-23T14:49:43.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.191+0000 7f127041d740 20 client.25664 trim_cache size 0 max 16384 2024-04-23T14:49:43.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.209+0000 7f127041d740 1 Processor -- start 2024-04-23T14:49:43.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.210+0000 7f127041d740 1 -- start start 2024-04-23T14:49:43.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.210+0000 7f127041d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2d270 0x562521c43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.210+0000 7f127041d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2db90 0x562521c2f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.210+0000 7f127041d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c38260 0x562521c50150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.210+0000 7f127041d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1248011010 con 0x562521c2db90 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.210+0000 7f127041d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f124800e8e0 con 0x562521c2d270 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.210+0000 7f127041d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f124800c730 con 0x562521c38260 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.210+0000 7f1266a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2db90 0x562521c2f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.210+0000 7f1266a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2db90 0x562521c2f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50822/0 (socket says 172.21.15.83:50822) 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.210+0000 7f1266a00640 1 -- 172.21.15.83:0/3749588857 learned_addr learned my addr 172.21.15.83:0/3749588857 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.210+0000 7f1266000640 1 --2- 172.21.15.83:0/3749588857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2d270 0x562521c43fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.211+0000 7f1266a00640 1 -- 172.21.15.83:0/3749588857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c38260 msgr2=0x562521c50150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.211+0000 7f1266a00640 1 --2- 172.21.15.83:0/3749588857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c38260 0x562521c50150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.211+0000 7f1266a00640 1 -- 172.21.15.83:0/3749588857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2d270 msgr2=0x562521c43fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.211+0000 7f1266a00640 1 --2- 172.21.15.83:0/3749588857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2d270 0x562521c43fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.211+0000 7f1266a00640 1 -- 172.21.15.83:0/3749588857 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562521c293c0 con 0x562521c2db90 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.211+0000 7f1266000640 1 --2- 172.21.15.83:0/3749588857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2d270 0x562521c43fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.211+0000 7f1266a00640 1 --2- 172.21.15.83:0/3749588857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2db90 0x562521c2f510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1258004790 tx=0x7f12580097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d81f1f1465f46b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.211+0000 7f1252a00640 1 -- 172.21.15.83:0/3749588857 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258029070 con 0x562521c2db90 2024-04-23T14:49:43.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.211+0000 7f1252a00640 1 -- 172.21.15.83:0/3749588857 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1258005000 con 0x562521c2db90 2024-04-23T14:49:43.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.211+0000 7f1252a00640 1 -- 172.21.15.83:0/3749588857 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12580304a0 con 0x562521c2db90 2024-04-23T14:49:43.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.212+0000 7f127041d740 1 -- 172.21.15.83:0/3749588857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2db90 msgr2=0x562521c2f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.212+0000 7f127041d740 1 --2- 172.21.15.83:0/3749588857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2db90 0x562521c2f510 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1258004790 tx=0x7f12580097a0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.212+0000 7f127041d740 1 -- 172.21.15.83:0/3749588857 shutdown_connections 2024-04-23T14:49:43.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.212+0000 7f127041d740 1 --2- 172.21.15.83:0/3749588857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c38260 0x562521c50150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.215 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.212+0000 7f127041d740 1 --2- 172.21.15.83:0/3749588857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c2db90 0x562521c2f510 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.212+0000 7f127041d740 1 --2- 172.21.15.83:0/3749588857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2d270 0x562521c43fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.212+0000 7f127041d740 1 -- 172.21.15.83:0/3749588857 >> 172.21.15.83:0/3749588857 conn(0x562521c29590 msgr2=0x562521c286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.212+0000 7f127041d740 1 -- 172.21.15.83:0/3749588857 shutdown_connections 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.212+0000 7f127041d740 1 -- 172.21.15.83:0/3749588857 wait complete. 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.213+0000 7f127041d740 1 -- start start 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.213+0000 7f127041d740 1 Processor -- start 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.213+0000 7f127041d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2d270 0x562521c31ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.213+0000 7f127041d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c2db90 0x562521c32400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.213+0000 7f127041d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c38260 0x562521c32940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.213+0000 7f127041d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f124800cda0 con 0x562521c38260 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.213+0000 7f127041d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f124800e210 con 0x562521c2d270 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.213+0000 7f127041d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f124800d470 con 0x562521c2db90 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.213+0000 7f127041d740 10 client.0 _ll_register_callbacks cb 0x7f126806a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:43.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.214+0000 7f1266000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c38260 0x562521c32940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.214+0000 7f1266000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c38260 0x562521c32940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50834/0 (socket says 172.21.15.83:50834) 2024-04-23T14:49:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.214+0000 7f1266000640 1 -- 172.21.15.83:0/380193760 learned_addr learned my addr 172.21.15.83:0/380193760 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.214+0000 7f1266000640 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2d270 0x562521c31ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.214+0000 7f1266a00640 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c2db90 0x562521c32400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.214+0000 7f1266000640 1 -- 172.21.15.83:0/380193760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c2db90 msgr2=0x562521c32400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.214+0000 7f1266000640 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c2db90 0x562521c32400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.214+0000 7f1266000640 1 -- 172.21.15.83:0/380193760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2d270 msgr2=0x562521c31ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.214+0000 7f1266000640 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2d270 0x562521c31ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.214+0000 7f1266000640 1 -- 172.21.15.83:0/380193760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562521cada40 con 0x562521c38260 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.215+0000 7f1266000640 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c38260 0x562521c32940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f125402a4b0 tx=0x7f1254031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466c13a1a94aa004 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.215+0000 7f1247e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.215+0000 7f1247e00640 1 -- 172.21.15.83:0/380193760 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254046040 con 0x562521c38260 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.215+0000 7f1247e00640 1 -- 172.21.15.83:0/380193760 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1254005040 con 0x562521c38260 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.215+0000 7f127041d740 1 -- 172.21.15.83:0/380193760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1258005000 con 0x562521c38260 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.215+0000 7f1247e00640 1 -- 172.21.15.83:0/380193760 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1254059050 con 0x562521c38260 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.216+0000 7f127041d740 10 client.15552 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.216+0000 7f127041d740 1 -- 172.21.15.83:0/380193760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562521caed60 con 0x562521c38260 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.216+0000 7f1247e00640 1 -- 172.21.15.83:0/380193760 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1258005000 con 0x562521c38260 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.216+0000 7f1247e00640 1 -- 172.21.15.83:0/380193760 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f12540046c0 con 0x562521c38260 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.216+0000 7f127041d740 10 client.15552 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.216+0000 7f127041d740 10 client.15552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.216+0000 7f127041d740 20 client.15552 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:43.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.216+0000 7f127041d740 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562521c485d0 0x562521cb14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.216+0000 7f127041d740 4 client.15552 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:49:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.216+0000 7f127041d740 1 -- 172.21.15.83:0/380193760 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562521b9a700 con 0x562521c485d0 2024-04-23T14:49:43.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.218+0000 7f1266a00640 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562521c485d0 0x562521cb14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.221+0000 7f1266a00640 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562521c485d0 0x562521cb14a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a71d72ed6055381d server_cookie=1b038a2a9eba4ecb in_seq=0 out_seq=0 2024-04-23T14:49:43.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.221+0000 7f1247e00640 10 client.15552 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.223+0000 7f1247e00640 1 -- 172.21.15.83:0/380193760 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562521b9a700 con 0x562521c485d0 2024-04-23T14:49:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.223+0000 7f1247e00640 10 client.15552 handle_command_reply: tid=0 2024-04-23T14:49:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.223+0000 7f127041d740 1 client.15552 shutdown 2024-04-23T14:49:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.223+0000 7f127041d740 1 -- 172.21.15.83:0/380193760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c38260 msgr2=0x562521c32940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.223+0000 7f127041d740 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c38260 0x562521c32940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f125402a4b0 tx=0x7f1254031400 comp rx=0 tx=0).stop 2024-04-23T14:49:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.223+0000 7f127041d740 1 -- 172.21.15.83:0/380193760 shutdown_connections 2024-04-23T14:49:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.223+0000 7f127041d740 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562521c485d0 0x562521cb14a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.223+0000 7f127041d740 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562521c38260 0x562521c32940 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.223+0000 7f127041d740 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562521c2db90 0x562521c32400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.223+0000 7f127041d740 1 --2- 172.21.15.83:0/380193760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562521c2d270 0x562521c31ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.223+0000 7f127041d740 1 -- 172.21.15.83:0/380193760 >> 172.21.15.83:0/380193760 conn(0x562521c29590 msgr2=0x562521c35090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.223+0000 7f1247e00640 0 client.15552 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.224+0000 7f127041d740 1 -- 172.21.15.83:0/380193760 shutdown_connections 2024-04-23T14:49:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.224+0000 7f127041d740 1 -- 172.21.15.83:0/380193760 wait complete. 2024-04-23T14:49:43.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.224+0000 7f127041d740 20 client.15552 trim_cache size 0 max 16384 2024-04-23T14:49:43.226 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:49:43.226 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:49:43.226 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "bf153c12-d4be-4afb-b1f1-f4b2e29a89e7", 2024-04-23T14:49:43.227 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:49:43.227 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:49:43.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.224+0000 7f1267e00640 1 -- 172.21.15.83:0/3151249598 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f124c0a3ed0 msgr2=0x7f124c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.224+0000 7f1267e00640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f124c0a3ed0 0x7f124c0a6390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f12600ce4e0 tx=0x7f1260009a70 comp rx=0 tx=0).stop 2024-04-23T14:49:43.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.225+0000 7f1267e00640 1 -- 172.21.15.83:0/3151249598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268106250 msgr2=0x7f12681a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.225+0000 7f1267e00640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268106250 0x7f12681a3640 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f126000eb10 tx=0x7f126000c2d0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.225+0000 7f1267e00640 1 -- 172.21.15.83:0/3151249598 shutdown_connections 2024-04-23T14:49:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.225+0000 7f1267e00640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1268106250 0x7f12681a3640 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.225+0000 7f1267e00640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f124c0a3ed0 0x7f124c0a6390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.225+0000 7f1267e00640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1268101770 0x7f126819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.225+0000 7f1267e00640 1 --2- 172.21.15.83:0/3151249598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1268100e50 0x7f126819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.225+0000 7f1267e00640 1 -- 172.21.15.83:0/3151249598 >> 172.21.15.83:0/3151249598 conn(0x7f12680fc880 msgr2=0x7f12680fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.225+0000 7f1267e00640 1 -- 172.21.15.83:0/3151249598 shutdown_connections 2024-04-23T14:49:43.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.225+0000 7f1267e00640 1 -- 172.21.15.83:0/3151249598 wait complete. 2024-04-23T14:49:43.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:49:43.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:49:43.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.410+0000 7f8036e00640 1 -- 172.21.15.83:0/2384045444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8030101770 msgr2=0x7f8030105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.410+0000 7f8036e00640 1 --2- 172.21.15.83:0/2384045444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8030101770 0x7f8030105c30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f801c009900 tx=0x7f801c02a4b0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.411+0000 7f8036e00640 1 -- 172.21.15.83:0/2384045444 shutdown_connections 2024-04-23T14:49:43.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.411+0000 7f8036e00640 1 --2- 172.21.15.83:0/2384045444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8030106250 0x7f80301066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.411+0000 7f8036e00640 1 --2- 172.21.15.83:0/2384045444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8030101770 0x7f8030105c30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.411+0000 7f8036e00640 1 --2- 172.21.15.83:0/2384045444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8030100e50 0x7f8030101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.411+0000 7f8036e00640 1 -- 172.21.15.83:0/2384045444 >> 172.21.15.83:0/2384045444 conn(0x7f80300fc880 msgr2=0x7f80300feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:43.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.411+0000 7f8036e00640 1 -- 172.21.15.83:0/2384045444 shutdown_connections 2024-04-23T14:49:43.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.411+0000 7f8036e00640 1 -- 172.21.15.83:0/2384045444 wait complete. 2024-04-23T14:49:43.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.412+0000 7f8036e00640 1 Processor -- start 2024-04-23T14:49:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.413+0000 7f8036e00640 1 -- start start 2024-04-23T14:49:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.413+0000 7f8036e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8030100e50 0x7f803019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.413+0000 7f8036e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8030101770 0x7f803019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.413+0000 7f8036e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8030106250 0x7f80301a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.413+0000 7f8036e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80301189e0 con 0x7f8030100e50 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.413+0000 7f8036e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8030118860 con 0x7f8030106250 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.413+0000 7f8036e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8030118b60 con 0x7f8030101770 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.413+0000 7f8035a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8030100e50 0x7f803019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.414+0000 7f8035a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8030100e50 0x7f803019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50860/0 (socket says 172.21.15.83:50860) 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.414+0000 7f8035a00640 1 -- 172.21.15.83:0/1233525519 learned_addr learned my addr 172.21.15.83:0/1233525519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.414+0000 7f8036400640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8030106250 0x7f80301a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.414+0000 7f8035000640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8030101770 0x7f803019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.414+0000 7f8035a00640 1 -- 172.21.15.83:0/1233525519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8030101770 msgr2=0x7f803019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.414+0000 7f8035a00640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8030101770 0x7f803019f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.414+0000 7f8035a00640 1 -- 172.21.15.83:0/1233525519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8030106250 msgr2=0x7f80301a3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.414+0000 7f8035a00640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8030106250 0x7f80301a3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.414+0000 7f8035a00640 1 -- 172.21.15.83:0/1233525519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80301a3f50 con 0x7f8030100e50 2024-04-23T14:49:43.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.414+0000 7f8035000640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8030101770 0x7f803019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:49:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.414+0000 7f8035a00640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8030100e50 0x7f803019ef60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f802400b450 tx=0x7f802400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.415+0000 7f802e000640 1 -- 172.21.15.83:0/1233525519 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024015020 con 0x7f8030100e50 2024-04-23T14:49:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.415+0000 7f802e000640 1 -- 172.21.15.83:0/1233525519 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8024011d70 con 0x7f8030100e50 2024-04-23T14:49:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.415+0000 7f802e000640 1 -- 172.21.15.83:0/1233525519 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802400f340 con 0x7f8030100e50 2024-04-23T14:49:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.415+0000 7f8036e00640 1 -- 172.21.15.83:0/1233525519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80301a4240 con 0x7f8030100e50 2024-04-23T14:49:43.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.415+0000 7f8036e00640 1 -- 172.21.15.83:0/1233525519 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80301abad0 con 0x7f8030100e50 2024-04-23T14:49:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.419+0000 7f803b3d0740 1 Processor -- start 2024-04-23T14:49:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.419+0000 7f803b3d0740 1 -- start start 2024-04-23T14:49:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.420+0000 7f803b3d0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d30f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.420+0000 7f803b3d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30f4350 0x5642d30fe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.420+0000 7f803b3d0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30fea20 0x5642d3100e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.420+0000 7f803b3d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5642d30591f0 con 0x5642d30f4350 2024-04-23T14:49:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.420+0000 7f803b3d0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5642d2fff350 con 0x5642d30f3a30 2024-04-23T14:49:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.420+0000 7f803b3d0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5642d30551b0 con 0x5642d30fea20 2024-04-23T14:49:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.420+0000 7f8036400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d30f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.420+0000 7f8035000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30fea20 0x5642d3100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.420+0000 7f8035000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30fea20 0x5642d3100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40454/0 (socket says 172.21.15.83:40454) 2024-04-23T14:49:43.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.420+0000 7f8035000640 1 -- 172.21.15.83:0/1745563652 learned_addr learned my addr 172.21.15.83:0/1745563652 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:43.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.421+0000 7f8036400640 1 -- 172.21.15.83:0/1745563652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30fea20 msgr2=0x5642d3100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.421+0000 7f8036400640 1 --2- 172.21.15.83:0/1745563652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30fea20 0x5642d3100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.421+0000 7f8036400640 1 -- 172.21.15.83:0/1745563652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30f4350 msgr2=0x5642d30fe4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:49:43.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.421+0000 7f8036400640 1 --2- 172.21.15.83:0/1745563652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30f4350 0x5642d30fe4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.421+0000 7f8036400640 1 -- 172.21.15.83:0/1745563652 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642d3053f50 con 0x5642d30f3a30 2024-04-23T14:49:43.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.421+0000 7f8036400640 1 --2- 172.21.15.83:0/1745563652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d30f3e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8028009ea0 tx=0x7f8028009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dfea7fdbb944c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:43.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.422+0000 7f802e000640 1 -- 172.21.15.83:0/1233525519 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f8024018020 con 0x7f8030100e50 2024-04-23T14:49:43.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.422+0000 7f8022a00640 1 -- 172.21.15.83:0/1745563652 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802801a070 con 0x5642d30f3a30 2024-04-23T14:49:43.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.422+0000 7f8022a00640 1 -- 172.21.15.83:0/1745563652 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80280158f0 con 0x5642d30f3a30 2024-04-23T14:49:43.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.422+0000 7f8022a00640 1 -- 172.21.15.83:0/1745563652 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802800a9c0 con 0x5642d30f3a30 2024-04-23T14:49:43.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.423+0000 7f803b3d0740 1 -- 172.21.15.83:0/1745563652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 msgr2=0x5642d30f3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.423+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1745563652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d30f3e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8028009ea0 tx=0x7f8028009f90 comp rx=0 tx=0).stop 2024-04-23T14:49:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.423+0000 7f803b3d0740 1 -- 172.21.15.83:0/1745563652 shutdown_connections 2024-04-23T14:49:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.423+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1745563652 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30fea20 0x5642d3100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.423+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1745563652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30f4350 0x5642d30fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.423+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1745563652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d30f3e10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.423+0000 7f803b3d0740 1 -- 172.21.15.83:0/1745563652 >> 172.21.15.83:0/1745563652 conn(0x5642d30efd50 msgr2=0x5642d30f0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.423+0000 7f803b3d0740 1 -- 172.21.15.83:0/1745563652 shutdown_connections 2024-04-23T14:49:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.423+0000 7f803b3d0740 1 -- 172.21.15.83:0/1745563652 wait complete. 2024-04-23T14:49:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.424+0000 7f803b3d0740 1 -- start start 2024-04-23T14:49:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.424+0000 7f803b3d0740 1 Processor -- start 2024-04-23T14:49:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.424+0000 7f802e000640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f80180a3e60 0x7f80180a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.424+0000 7f802e000640 1 -- 172.21.15.83:0/1233525519 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f80240d0510 con 0x7f8030100e50 2024-04-23T14:49:43.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.425+0000 7f8036400640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f80180a3e60 0x7f80180a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.425+0000 7f803b3d0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d30fba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.425+0000 7f803b3d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30f4350 0x5642d30fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.425+0000 7f803b3d0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30fea20 0x5642d30fc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.425+0000 7f803b3d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5642d3101810 con 0x5642d30f4350 2024-04-23T14:49:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.425+0000 7f803b3d0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5642d3012690 con 0x5642d30f3a30 2024-04-23T14:49:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.425+0000 7f803b3d0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5642d3101990 con 0x5642d30fea20 2024-04-23T14:49:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.425+0000 7f803b3d0740 10 client.0 _ll_register_callbacks cb 0x5642d304e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.425+0000 7f8035000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30f4350 0x5642d30fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.425+0000 7f8035000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30f4350 0x5642d30fbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50872/0 (socket says 172.21.15.83:50872) 2024-04-23T14:49:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.425+0000 7f8035000640 1 -- 172.21.15.83:0/2893345574 learned_addr learned my addr 172.21.15.83:0/2893345574 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.426+0000 7f8036400640 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30fea20 0x5642d30fc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.426+0000 7f8036400640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f80180a3e60 0x7f80180a6320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8028009cb0 tx=0x7f8028000f40 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.426+0000 7f8035a00640 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d30fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.426+0000 7f8035a00640 1 -- 172.21.15.83:0/2893345574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30fea20 msgr2=0x5642d30fc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.426+0000 7f8035a00640 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30fea20 0x5642d30fc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.426+0000 7f8035a00640 1 -- 172.21.15.83:0/2893345574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30f4350 msgr2=0x5642d30fbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.426+0000 7f8035a00640 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30f4350 0x5642d30fbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.426+0000 7f8035a00640 1 -- 172.21.15.83:0/2893345574 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642d3011920 con 0x5642d30f3a30 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.426+0000 7f8035000640 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30f4350 0x5642d30fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.427+0000 7f8035a00640 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d30fba70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f80240cdb40 tx=0x7f802409bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b234e0189210713 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.427+0000 7f8017400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.427+0000 7f8017400640 1 -- 172.21.15.83:0/2893345574 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80240d0250 con 0x5642d30f3a30 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.427+0000 7f803b3d0740 1 -- 172.21.15.83:0/2893345574 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80280158f0 con 0x5642d30f3a30 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.427+0000 7f8017400640 1 -- 172.21.15.83:0/2893345574 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f802409b7e0 con 0x5642d30f3a30 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.427+0000 7f8017400640 1 -- 172.21.15.83:0/2893345574 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f80240bb300 con 0x5642d30f3a30 2024-04-23T14:49:43.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.427+0000 7f803b3d0740 10 client.25679 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.427+0000 7f803b3d0740 1 -- 172.21.15.83:0/2893345574 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5642d2e357d0 con 0x5642d30f3a30 2024-04-23T14:49:43.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.428+0000 7f8017400640 1 -- 172.21.15.83:0/2893345574 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80280158f0 con 0x5642d30f3a30 2024-04-23T14:49:43.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.428+0000 7f8017400640 1 -- 172.21.15.83:0/2893345574 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f8024011890 con 0x5642d30f3a30 2024-04-23T14:49:43.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.428+0000 7f803b3d0740 10 client.25679 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:43.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.428+0000 7f803b3d0740 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.428+0000 7f803b3d0740 20 client.25679 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.428+0000 7f803b3d0740 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5642d310ee80 0x5642d31112a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.428+0000 7f803b3d0740 4 client.25679 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.428+0000 7f803b3d0740 1 -- 172.21.15.83:0/2893345574 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642d3066c10 con 0x5642d310ee80 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.429+0000 7f8035000640 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5642d310ee80 0x5642d31112a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.430+0000 7f8017400640 10 client.25679 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.430+0000 7f8035000640 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5642d310ee80 0x5642d31112a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db60dfeffc9bfce4 server_cookie=ce49220d0d93e2d8 in_seq=0 out_seq=0 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.430+0000 7f8017400640 1 -- 172.21.15.83:0/2893345574 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642d3066c10 con 0x5642d310ee80 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.430+0000 7f8017400640 10 client.25679 handle_command_reply: tid=0 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f803b3d0740 1 client.25679 shutdown 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f803b3d0740 1 -- 172.21.15.83:0/2893345574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 msgr2=0x5642d30fba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f803b3d0740 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d30fba70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f80240cdb40 tx=0x7f802409bf90 comp rx=0 tx=0).stop 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f803b3d0740 1 -- 172.21.15.83:0/2893345574 shutdown_connections 2024-04-23T14:49:43.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f803b3d0740 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5642d310ee80 0x5642d31112a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f803b3d0740 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30fea20 0x5642d30fc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f803b3d0740 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30f4350 0x5642d30fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f803b3d0740 1 --2- 172.21.15.83:0/2893345574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d30fba70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f803b3d0740 1 -- 172.21.15.83:0/2893345574 >> 172.21.15.83:0/2893345574 conn(0x5642d30efd50 msgr2=0x5642d30eefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f8017400640 0 client.25679 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f803b3d0740 1 -- 172.21.15.83:0/2893345574 shutdown_connections 2024-04-23T14:49:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f803b3d0740 1 -- 172.21.15.83:0/2893345574 wait complete. 2024-04-23T14:49:43.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.431+0000 7f803b3d0740 20 client.25679 trim_cache size 0 max 16384 2024-04-23T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.449+0000 7f803b3d0740 1 Processor -- start 2024-04-23T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.449+0000 7f803b3d0740 1 -- start start 2024-04-23T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f803b3d0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30f3a30 0x5642d310a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f803b3d0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f4350 0x5642d30faa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f803b3d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30fea20 0x5642d3116b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f803b3d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80301b6b20 con 0x5642d30fea20 2024-04-23T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f803b3d0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80301b43f0 con 0x5642d30f4350 2024-04-23T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f803b3d0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80301b2240 con 0x5642d30f3a30 2024-04-23T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f8035a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30f3a30 0x5642d310a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f8035a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30f3a30 0x5642d310a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40470/0 (socket says 172.21.15.83:40470) 2024-04-23T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f8035a00640 1 -- 172.21.15.83:0/1377443408 learned_addr learned my addr 172.21.15.83:0/1377443408 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:43.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f8035000640 1 --2- 172.21.15.83:0/1377443408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f4350 0x5642d30faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f8036400640 1 --2- 172.21.15.83:0/1377443408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30fea20 0x5642d3116b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f8035a00640 1 -- 172.21.15.83:0/1377443408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f4350 msgr2=0x5642d30faa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f8035a00640 1 --2- 172.21.15.83:0/1377443408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f4350 0x5642d30faa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f8035a00640 1 -- 172.21.15.83:0/1377443408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30fea20 msgr2=0x5642d3116b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.450+0000 7f8035a00640 1 --2- 172.21.15.83:0/1377443408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30fea20 0x5642d3116b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.451+0000 7f8035a00640 1 -- 172.21.15.83:0/1377443408 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642d2d6b3f0 con 0x5642d30f3a30 2024-04-23T14:49:43.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.451+0000 7f8035a00640 1 --2- 172.21.15.83:0/1377443408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30f3a30 0x5642d310a6d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f802400be10 tx=0x7f80240103e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f91d0f33e15701c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:43.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.451+0000 7f8022000640 1 -- 172.21.15.83:0/1377443408 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024011a90 con 0x5642d30f3a30 2024-04-23T14:49:43.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.451+0000 7f8022000640 1 -- 172.21.15.83:0/1377443408 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8024083070 con 0x5642d30f3a30 2024-04-23T14:49:43.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.451+0000 7f8022000640 1 -- 172.21.15.83:0/1377443408 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802409ba40 con 0x5642d30f3a30 2024-04-23T14:49:43.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.452+0000 7f803b3d0740 1 -- 172.21.15.83:0/1377443408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30f3a30 msgr2=0x5642d310a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.452+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1377443408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30f3a30 0x5642d310a6d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f802400be10 tx=0x7f80240103e0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.452+0000 7f803b3d0740 1 -- 172.21.15.83:0/1377443408 shutdown_connections 2024-04-23T14:49:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.452+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1377443408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30fea20 0x5642d3116b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.452+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1377443408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f4350 0x5642d30faa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.452+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1377443408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30f3a30 0x5642d310a6d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.452+0000 7f803b3d0740 1 -- 172.21.15.83:0/1377443408 >> 172.21.15.83:0/1377443408 conn(0x5642d30efd50 msgr2=0x5642d30fa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.452+0000 7f803b3d0740 1 -- 172.21.15.83:0/1377443408 shutdown_connections 2024-04-23T14:49:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.452+0000 7f803b3d0740 1 -- 172.21.15.83:0/1377443408 wait complete. 2024-04-23T14:49:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.453+0000 7f803b3d0740 1 -- start start 2024-04-23T14:49:43.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.453+0000 7f803b3d0740 1 Processor -- start 2024-04-23T14:49:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.453+0000 7f803b3d0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d3106dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.453+0000 7f803b3d0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30f4350 0x5642d30f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.453+0000 7f803b3d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30fea20 0x5642d30f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.453+0000 7f803b3d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f80301b28b0 con 0x5642d30fea20 2024-04-23T14:49:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.453+0000 7f803b3d0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f80301b3d20 con 0x5642d30f3a30 2024-04-23T14:49:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.453+0000 7f803b3d0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f80301b2f80 con 0x5642d30f4350 2024-04-23T14:49:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.454+0000 7f803b3d0740 10 client.0 _ll_register_callbacks cb 0x7f8024010f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:49:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.454+0000 7f8035a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d3106dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.454+0000 7f8035a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d3106dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37512/0 (socket says 172.21.15.83:37512) 2024-04-23T14:49:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.454+0000 7f8035a00640 1 -- 172.21.15.83:0/1660401665 learned_addr learned my addr 172.21.15.83:0/1660401665 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:49:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.454+0000 7f8036400640 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30fea20 0x5642d30f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.454+0000 7f8035000640 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30f4350 0x5642d30f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.454+0000 7f8035a00640 1 -- 172.21.15.83:0/1660401665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30f4350 msgr2=0x5642d30f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.454+0000 7f8035a00640 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30f4350 0x5642d30f82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.454+0000 7f8035a00640 1 -- 172.21.15.83:0/1660401665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30fea20 msgr2=0x5642d30f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.454+0000 7f8035a00640 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30fea20 0x5642d30f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.454+0000 7f8035a00640 1 -- 172.21.15.83:0/1660401665 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642d2d2b0f0 con 0x5642d30f3a30 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.455+0000 7f8035a00640 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d3106dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f80240cdc30 tx=0x7f802409ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c64c83c6f990486b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.455+0000 7f8017400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.455+0000 7f8017400640 1 -- 172.21.15.83:0/1660401665 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80240ad060 con 0x5642d30f3a30 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.455+0000 7f8017400640 1 -- 172.21.15.83:0/1660401665 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f80240980e0 con 0x5642d30f3a30 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.455+0000 7f803b3d0740 1 -- 172.21.15.83:0/1660401665 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8024083070 con 0x5642d30f3a30 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.455+0000 7f8017400640 1 -- 172.21.15.83:0/1660401665 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f80240a0740 con 0x5642d30f3a30 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.455+0000 7f803b3d0740 10 client.25685 fetch_fsmap learned FSMap version 24 2024-04-23T14:49:43.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.455+0000 7f803b3d0740 1 -- 172.21.15.83:0/1660401665 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5642d301e810 con 0x5642d30f3a30 2024-04-23T14:49:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.456+0000 7f8017400640 1 -- 172.21.15.83:0/1660401665 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8024083070 con 0x5642d30f3a30 2024-04-23T14:49:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.456+0000 7f8017400640 1 -- 172.21.15.83:0/1660401665 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f80240a7070 con 0x5642d30f3a30 2024-04-23T14:49:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.456+0000 7f803b3d0740 10 client.25685 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:49:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.456+0000 7f803b3d0740 10 client.25685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:49:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.456+0000 7f803b3d0740 20 client.25685 populate_metadata read hostname 'smithi083' 2024-04-23T14:49:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.456+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5642d3174fb0 0x5642d31773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:49:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.456+0000 7f803b3d0740 4 client.25685 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:49:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.456+0000 7f803b3d0740 1 -- 172.21.15.83:0/1660401665 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5642d3060a80 con 0x5642d3174fb0 2024-04-23T14:49:43.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.456+0000 7f8035000640 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5642d3174fb0 0x5642d31773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:49:43.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.457+0000 7f8035000640 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5642d3174fb0 0x5642d31773a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a97ce75ee8833981 server_cookie=43d724f9341d6729 in_seq=0 out_seq=0 2024-04-23T14:49:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.457+0000 7f8017400640 10 client.25685 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.457+0000 7f8017400640 1 -- 172.21.15.83:0/1660401665 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5642d3060a80 con 0x5642d3174fb0 2024-04-23T14:49:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.457+0000 7f8017400640 10 client.25685 handle_command_reply: tid=0 2024-04-23T14:49:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f803b3d0740 1 client.25685 shutdown 2024-04-23T14:49:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f803b3d0740 1 -- 172.21.15.83:0/1660401665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 msgr2=0x5642d3106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d3106dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f80240cdc30 tx=0x7f802409ba10 comp rx=0 tx=0).stop 2024-04-23T14:49:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f803b3d0740 1 -- 172.21.15.83:0/1660401665 shutdown_connections 2024-04-23T14:49:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5642d3174fb0 0x5642d31773a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5642d30fea20 0x5642d30f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5642d30f4350 0x5642d30f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f803b3d0740 1 --2- 172.21.15.83:0/1660401665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5642d30f3a30 0x5642d3106dd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f803b3d0740 1 -- 172.21.15.83:0/1660401665 >> 172.21.15.83:0/1660401665 conn(0x5642d30efd50 msgr2=0x5642d30fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f8017400640 0 client.25685 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:49:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f803b3d0740 1 -- 172.21.15.83:0/1660401665 shutdown_connections 2024-04-23T14:49:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f803b3d0740 1 -- 172.21.15.83:0/1660401665 wait complete. 2024-04-23T14:49:43.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.458+0000 7f803b3d0740 20 client.25685 trim_cache size 0 max 16384 2024-04-23T14:49:43.463 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:49:43.463 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:49:43.463 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:49:43.463 INFO:teuthology.orchestra.run.smithi083.stdout: "bf153c12-d4be-4afb-b1f1-f4b2e29a89e7": { 2024-04-23T14:49:43.463 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:49:43.463 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "bf153c12-d4be-4afb-b1f1-f4b2e29a89e7", 2024-04-23T14:49:43.464 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:49:43.464 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:49:43.464 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:49:43.464 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:49:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.459+0000 7f8036e00640 1 -- 172.21.15.83:0/1233525519 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f80180a3e60 msgr2=0x7f80180a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.459+0000 7f8036e00640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f80180a3e60 0x7f80180a6320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8028009cb0 tx=0x7f8028000f40 comp rx=0 tx=0).stop 2024-04-23T14:49:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.459+0000 7f8036e00640 1 -- 172.21.15.83:0/1233525519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8030100e50 msgr2=0x7f803019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:49:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.459+0000 7f8036e00640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8030100e50 0x7f803019ef60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f802400b450 tx=0x7f802400b910 comp rx=0 tx=0).stop 2024-04-23T14:49:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.459+0000 7f8036e00640 1 -- 172.21.15.83:0/1233525519 shutdown_connections 2024-04-23T14:49:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.459+0000 7f8036e00640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8030106250 0x7f80301a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.459+0000 7f8036e00640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8030101770 0x7f803019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.459+0000 7f8036e00640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f80180a3e60 0x7f80180a6320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.459+0000 7f8036e00640 1 --2- 172.21.15.83:0/1233525519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8030100e50 0x7f803019ef60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:49:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.459+0000 7f8036e00640 1 -- 172.21.15.83:0/1233525519 >> 172.21.15.83:0/1233525519 conn(0x7f80300fc880 msgr2=0x7f80300fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:49:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.460+0000 7f8036e00640 1 -- 172.21.15.83:0/1233525519 shutdown_connections 2024-04-23T14:49:43.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:49:43.460+0000 7f8036e00640 1 -- 172.21.15.83:0/1233525519 wait complete. 2024-04-23T14:49:43.484 INFO:tasks.cephfs.filesystem:scrub status for tag:bf153c12-d4be-4afb-b1f1-f4b2e29a89e7 - {'path': '/', 'tag': 'bf153c12-d4be-4afb-b1f1-f4b2e29a89e7', 'options': 'recursive,force'} 2024-04-23T14:49:43.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:43 smithi083 ceph-mon[27867]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-23T14:49:43.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:43 smithi203 ceph-mon[32800]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-23T14:49:43.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:43 smithi173 ceph-mon[32443]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-23T14:49:44.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:44 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:49:44.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:44 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:49:44.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:44 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:49:44.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:44 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:49:44.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:44 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:49:44.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:44 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:49:44.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:44 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:49:44.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:44 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:49:44.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:44 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:49:44.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:44 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:49:44.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:44 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:49:44.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:44 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:49:44.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:44 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:49:44.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:44 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:49:44.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:44 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:49:45.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:45 smithi203 ceph-mon[32800]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-23T14:49:45.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:45 smithi083 ceph-mon[27867]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-23T14:49:45.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:45 smithi173 ceph-mon[32443]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-23T14:49:47.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:47 smithi203 ceph-mon[32800]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:49:47.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:47 smithi083 ceph-mon[27867]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:49:47.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:47 smithi173 ceph-mon[32443]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:49:49.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:49 smithi203 ceph-mon[32800]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:49:49.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:49 smithi083 ceph-mon[27867]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:49:49.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:49 smithi173 ceph-mon[32443]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:49:51.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:51 smithi203 ceph-mon[32800]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-23T14:49:51.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:51 smithi083 ceph-mon[27867]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-23T14:49:51.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:51 smithi173 ceph-mon[32443]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-23T14:49:53.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:53 smithi203 ceph-mon[32800]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:49:53.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:53 smithi083 ceph-mon[27867]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:49:53.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:53 smithi173 ceph-mon[32443]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:49:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:55 smithi203 ceph-mon[32800]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-04-23T14:49:55.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:55 smithi083 ceph-mon[27867]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-04-23T14:49:55.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:55 smithi173 ceph-mon[32443]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-04-23T14:49:57.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:57 smithi203 ceph-mon[32800]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:49:57.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:57 smithi083 ceph-mon[27867]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:49:57.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:57 smithi173 ceph-mon[32443]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:49:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:49:59 smithi203 ceph-mon[32800]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:49:59.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:49:59 smithi083 ceph-mon[27867]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:49:59.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:49:59 smithi173 ceph-mon[32443]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-23T14:50:00.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:00 smithi203 ceph-mon[32800]: overall HEALTH_OK 2024-04-23T14:50:00.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:00 smithi083 ceph-mon[27867]: overall HEALTH_OK 2024-04-23T14:50:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:00 smithi173 ceph-mon[32443]: overall HEALTH_OK 2024-04-23T14:50:01.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:01 smithi203 ceph-mon[32800]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-23T14:50:01.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:01 smithi203 ceph-mon[32800]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-23T14:50:01.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:01 smithi083 ceph-mon[27867]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-23T14:50:01.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:01 smithi083 ceph-mon[27867]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-23T14:50:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:01 smithi173 ceph-mon[32443]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-23T14:50:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:01 smithi173 ceph-mon[32443]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-23T14:50:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:03 smithi203 ceph-mon[32800]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T14:50:03.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:03 smithi083 ceph-mon[27867]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T14:50:03.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:03 smithi173 ceph-mon[32443]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-23T14:50:05.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:05 smithi203 ceph-mon[32800]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-04-23T14:50:05.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:05 smithi083 ceph-mon[27867]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-04-23T14:50:05.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:05 smithi173 ceph-mon[32443]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-04-23T14:50:07.563 INFO:tasks.workunit.client.0.smithi083.stderr:Updating files: 63% (7768/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-23T14:50:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:07 smithi203 ceph-mon[32800]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-23T14:50:07.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:07 smithi083 ceph-mon[27867]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-23T14:50:07.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:07 smithi173 ceph-mon[32443]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-23T14:50:08.415 INFO:tasks.workunit.client.0.smithi083.stderr:Note: switching to 'd44fddd910b5e94da0f2357af4be37d7bd495ae0'. 2024-04-23T14:50:08.415 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-23T14:50:08.415 INFO:tasks.workunit.client.0.smithi083.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-23T14:50:08.415 INFO:tasks.workunit.client.0.smithi083.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-23T14:50:08.415 INFO:tasks.workunit.client.0.smithi083.stderr:state without impacting any branches by switching back to a branch. 2024-04-23T14:50:08.415 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-23T14:50:08.415 INFO:tasks.workunit.client.0.smithi083.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-23T14:50:08.415 INFO:tasks.workunit.client.0.smithi083.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-23T14:50:08.415 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-23T14:50:08.416 INFO:tasks.workunit.client.0.smithi083.stderr: git switch -c 2024-04-23T14:50:08.416 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-23T14:50:08.416 INFO:tasks.workunit.client.0.smithi083.stderr:Or undo this operation with: 2024-04-23T14:50:08.416 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-23T14:50:08.416 INFO:tasks.workunit.client.0.smithi083.stderr: git switch - 2024-04-23T14:50:08.416 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-23T14:50:08.416 INFO:tasks.workunit.client.0.smithi083.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-23T14:50:08.416 INFO:tasks.workunit.client.0.smithi083.stderr: 2024-04-23T14:50:08.416 INFO:tasks.workunit.client.0.smithi083.stderr:HEAD is now at d44fddd910b DNM: trigger builds again 2024-04-23T14:50:08.428 DEBUG:teuthology.orchestra.run.smithi083:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-23T14:50:08.534 INFO:tasks.workunit.client.0.smithi083.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-23T14:50:08.542 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T14:50:08.542 INFO:tasks.workunit.client.0.smithi083.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-23T14:50:09.338 INFO:tasks.workunit.client.0.smithi083.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-23T14:50:09.445 INFO:tasks.workunit.client.0.smithi083.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-23T14:50:09.534 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T14:50:09.539 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T14:50:09.540 INFO:tasks.workunit.client.0.smithi083.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-23T14:50:09.621 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T14:50:09.631 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:50:09.631 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-23T14:50:09.682 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-04-23T14:50:09.683 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-04-23T14:50:09.683 DEBUG:teuthology.orchestra.run.smithi083:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d44fddd910b5e94da0f2357af4be37d7bd495ae0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-04-23T14:50:09.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:09 smithi203 ceph-mon[32800]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-23T14:50:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:09 smithi083 ceph-mon[27867]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-23T14:50:09.758 INFO:tasks.workunit.client.0.smithi083.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-04-23T14:50:09.760 INFO:tasks.workunit.client.0.smithi083.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-04-23T14:50:09.761 INFO:tasks.workunit.client.0.smithi083.stderr:+ trap -- retry EXIT 2024-04-23T14:50:09.761 INFO:tasks.workunit.client.0.smithi083.stderr:+ rm -rf ffsb 2024-04-23T14:50:09.763 INFO:tasks.workunit.client.0.smithi083.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-04-23T14:50:09.772 INFO:tasks.workunit.client.0.smithi083.stderr:Cloning into 'ffsb'... 2024-04-23T14:50:09.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:09 smithi173 ceph-mon[32443]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-23T14:50:11.143 INFO:tasks.workunit.client.0.smithi083.stderr:+ trap - EXIT 2024-04-23T14:50:11.144 INFO:tasks.workunit.client.0.smithi083.stderr:+ cd ffsb 2024-04-23T14:50:11.144 INFO:tasks.workunit.client.0.smithi083.stderr:+ ./configure 2024-04-23T14:50:11.490 INFO:tasks.workunit.client.0.smithi083.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-23T14:50:11.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:11 smithi203 ceph-mon[32800]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-23T14:50:11.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:11 smithi083 ceph-mon[27867]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-23T14:50:11.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:11 smithi173 ceph-mon[32443]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-23T14:50:12.512 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether build environment is sane... yes 2024-04-23T14:50:12.544 INFO:tasks.workunit.client.0.smithi083.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-23T14:50:12.545 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gawk... gawk 2024-04-23T14:50:12.583 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether make sets $(MAKE)... yes 2024-04-23T14:50:12.685 INFO:tasks.workunit.client.0.smithi083.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-23T14:50:12.685 INFO:tasks.workunit.client.0.smithi083.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-23T14:50:12.710 INFO:tasks.workunit.client.0.smithi083.stdout:checking for style of include used by make... GNU 2024-04-23T14:50:12.719 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gcc... gcc 2024-04-23T14:50:12.846 INFO:tasks.workunit.client.0.smithi083.stdout:checking for C compiler default output file name... a.out 2024-04-23T14:50:12.852 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether the C compiler works... yes 2024-04-23T14:50:12.856 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether we are cross compiling... no 2024-04-23T14:50:12.934 INFO:tasks.workunit.client.0.smithi083.stdout:checking for suffix of executables... 2024-04-23T14:50:13.013 INFO:tasks.workunit.client.0.smithi083.stdout:checking for suffix of object files... o 2024-04-23T14:50:13.094 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether we are using the GNU C compiler... yes 2024-04-23T14:50:13.178 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether gcc accepts -g... yes 2024-04-23T14:50:13.323 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-23T14:50:13.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:50:13.485 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:50:13.533 INFO:tasks.workunit.client.0.smithi083.stdout:checking dependency style of gcc... gcc3 2024-04-23T14:50:13.660 INFO:tasks.workunit.client.0.smithi083.stdout:checking how to run the C preprocessor... gcc -E 2024-04-23T14:50:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.686+0000 7f490da00640 1 -- 172.21.15.83:0/1953809183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f490810e880 msgr2=0x7f4908110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.686+0000 7f490da00640 1 --2- 172.21.15.83:0/1953809183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f490810e880 0x7f4908110c90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4900009860 tx=0x7f490002a3d0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.686+0000 7f490da00640 1 -- 172.21.15.83:0/1953809183 shutdown_connections 2024-04-23T14:50:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.686+0000 7f490da00640 1 --2- 172.21.15.83:0/1953809183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f490810e880 0x7f4908110c90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.686+0000 7f490da00640 1 --2- 172.21.15.83:0/1953809183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4908074cf0 0x7f490810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.686+0000 7f490da00640 1 --2- 172.21.15.83:0/1953809183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49080743d0 0x7f49080747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.686+0000 7f490da00640 1 -- 172.21.15.83:0/1953809183 >> 172.21.15.83:0/1953809183 conn(0x7f4908079400 msgr2=0x7f4908079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:13.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.686+0000 7f490da00640 1 -- 172.21.15.83:0/1953809183 shutdown_connections 2024-04-23T14:50:13.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.687+0000 7f490da00640 1 -- 172.21.15.83:0/1953809183 wait complete. 2024-04-23T14:50:13.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.687+0000 7f490da00640 1 Processor -- start 2024-04-23T14:50:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.688+0000 7f490da00640 1 -- start start 2024-04-23T14:50:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.688+0000 7f490da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49080743d0 0x7f49081a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.688+0000 7f490da00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4908074cf0 0x7f49081a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.688+0000 7f490da00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f490810e880 0x7f490819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.688+0000 7f490da00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4908114610 con 0x7f49080743d0 2024-04-23T14:50:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.688+0000 7f490da00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4908114490 con 0x7f4908074cf0 2024-04-23T14:50:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.688+0000 7f490da00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4908114790 con 0x7f490810e880 2024-04-23T14:50:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.688+0000 7f4906a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49080743d0 0x7f49081a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.688+0000 7f4906a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49080743d0 0x7f49081a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36882/0 (socket says 172.21.15.83:36882) 2024-04-23T14:50:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.688+0000 7f4906a00640 1 -- 172.21.15.83:0/1395553265 learned_addr learned my addr 172.21.15.83:0/1395553265 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.688+0000 7f48fe000640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4908074cf0 0x7f49081a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.688+0000 7f4907400640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f490810e880 0x7f490819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.689+0000 7f4906a00640 1 -- 172.21.15.83:0/1395553265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f490810e880 msgr2=0x7f490819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.689+0000 7f4906a00640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f490810e880 0x7f490819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.689+0000 7f4906a00640 1 -- 172.21.15.83:0/1395553265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4908074cf0 msgr2=0x7f49081a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.689+0000 7f4906a00640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4908074cf0 0x7f49081a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.689+0000 7f4906a00640 1 -- 172.21.15.83:0/1395553265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490819b0f0 con 0x7f49080743d0 2024-04-23T14:50:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.689+0000 7f48fe000640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4908074cf0 0x7f49081a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.689+0000 7f4906a00640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49080743d0 0x7f49081a0770 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f48f800eb90 tx=0x7f48f800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.689+0000 7f48ffe00640 1 -- 172.21.15.83:0/1395553265 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8004b20 con 0x7f49080743d0 2024-04-23T14:50:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.689+0000 7f48ffe00640 1 -- 172.21.15.83:0/1395553265 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f8004cc0 con 0x7f49080743d0 2024-04-23T14:50:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.689+0000 7f490da00640 1 -- 172.21.15.83:0/1395553265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490819b3e0 con 0x7f49080743d0 2024-04-23T14:50:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.689+0000 7f48ffe00640 1 -- 172.21.15.83:0/1395553265 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8010720 con 0x7f49080743d0 2024-04-23T14:50:13.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.689+0000 7f490da00640 1 -- 172.21.15.83:0/1395553265 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49081a7630 con 0x7f49080743d0 2024-04-23T14:50:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.693+0000 7f48ffe00640 1 -- 172.21.15.83:0/1395553265 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f48f8005020 con 0x7f49080743d0 2024-04-23T14:50:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.693+0000 7f4910823740 1 Processor -- start 2024-04-23T14:50:13.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.693+0000 7f4910823740 1 -- start start 2024-04-23T14:50:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.693+0000 7f4910823740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ee9860 0x55ae21ee9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.693+0000 7f4910823740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21eea180 0x55ae21ef4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.693+0000 7f4910823740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21ef4850 0x55ae21ef6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.693+0000 7f4910823740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ae21e4f1f0 con 0x55ae21ef4850 2024-04-23T14:50:13.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.693+0000 7f4910823740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ae21df5350 con 0x55ae21eea180 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.693+0000 7f4910823740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ae21e4b1b0 con 0x55ae21ee9860 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f4906a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21eea180 0x55ae21ef4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f4907400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ee9860 0x55ae21ee9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f4907400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ee9860 0x55ae21ee9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:53818/0 (socket says 172.21.15.83:53818) 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f4906a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21eea180 0x55ae21ef4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:48840/0 (socket says 172.21.15.83:48840) 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f4907400640 1 -- 172.21.15.83:0/1195310175 learned_addr learned my addr 172.21.15.83:0/1195310175 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f48fe000640 1 --2- 172.21.15.83:0/1195310175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21ef4850 0x55ae21ef6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f48ffe00640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f48ec0a3d20 0x7f48ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f4906a00640 1 -- 172.21.15.83:0/1195310175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ee9860 msgr2=0x55ae21ee9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f4906a00640 1 --2- 172.21.15.83:0/1195310175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ee9860 0x55ae21ee9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f4906a00640 1 -- 172.21.15.83:0/1195310175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21ef4850 msgr2=0x55ae21ef6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f4906a00640 1 --2- 172.21.15.83:0/1195310175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21ef4850 0x55ae21ef6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f4906a00640 1 -- 172.21.15.83:0/1195310175 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae21e49f50 con 0x55ae21eea180 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.694+0000 7f48fe000640 1 --2- 172.21.15.83:0/1195310175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21ef4850 0x55ae21ef6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:50:13.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.695+0000 7f4907400640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f48ec0a3d20 0x7f48ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.695+0000 7f48ffe00640 1 -- 172.21.15.83:0/1395553265 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f48f80d0e40 con 0x7f49080743d0 2024-04-23T14:50:13.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.695+0000 7f4906a00640 1 --2- 172.21.15.83:0/1195310175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21eea180 0x55ae21ef4310 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f48f80ce540 tx=0x7f48f80058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a72740c7679b6f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:13.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.695+0000 7f48f7e00640 1 -- 172.21.15.83:0/1195310175 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f809cb60 con 0x55ae21eea180 2024-04-23T14:50:13.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.695+0000 7f48f7e00640 1 -- 172.21.15.83:0/1195310175 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f809d490 con 0x55ae21eea180 2024-04-23T14:50:13.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.695+0000 7f48f7e00640 1 -- 172.21.15.83:0/1195310175 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f80203a0 con 0x55ae21eea180 2024-04-23T14:50:13.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.696+0000 7f4910823740 1 -- 172.21.15.83:0/1195310175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21eea180 msgr2=0x55ae21ef4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:13.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.696+0000 7f4910823740 1 --2- 172.21.15.83:0/1195310175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21eea180 0x55ae21ef4310 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f48f80ce540 tx=0x7f48f80058d0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.696+0000 7f4907400640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f48ec0a3d20 0x7f48ec0a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4908075320 tx=0x7f4900005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.696+0000 7f4910823740 1 -- 172.21.15.83:0/1195310175 shutdown_connections 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.696+0000 7f4910823740 1 --2- 172.21.15.83:0/1195310175 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21ef4850 0x55ae21ef6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.696+0000 7f4910823740 1 --2- 172.21.15.83:0/1195310175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21eea180 0x55ae21ef4310 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.696+0000 7f4910823740 1 --2- 172.21.15.83:0/1195310175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ee9860 0x55ae21ee9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.696+0000 7f4910823740 1 -- 172.21.15.83:0/1195310175 >> 172.21.15.83:0/1195310175 conn(0x55ae21ee5b80 msgr2=0x55ae21ee5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.696+0000 7f4910823740 1 -- 172.21.15.83:0/1195310175 shutdown_connections 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.696+0000 7f4910823740 1 -- 172.21.15.83:0/1195310175 wait complete. 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.697+0000 7f4910823740 1 -- start start 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.697+0000 7f4910823740 1 Processor -- start 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.697+0000 7f4910823740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 0x55ae21ef1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4910823740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21eea180 0x55ae21ef1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4910823740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ef4850 0x55ae21ef2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4910823740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ae21ef7580 con 0x55ae21eea180 2024-04-23T14:50:13.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4910823740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ae21e08690 con 0x55ae21ee9860 2024-04-23T14:50:13.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4910823740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ae21ef7700 con 0x55ae21ef4850 2024-04-23T14:50:13.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4910823740 10 client.0 _ll_register_callbacks cb 0x55ae21e44650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:13.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4906a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 0x55ae21ef1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4906a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 0x55ae21ef1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:48842/0 (socket says 172.21.15.83:48842) 2024-04-23T14:50:13.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4906a00640 1 -- 172.21.15.83:0/318410041 learned_addr learned my addr 172.21.15.83:0/318410041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:13.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f48fe000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21eea180 0x55ae21ef1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4907400640 1 --2- 172.21.15.83:0/318410041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ef4850 0x55ae21ef2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4906a00640 1 -- 172.21.15.83:0/318410041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ef4850 msgr2=0x55ae21ef2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4906a00640 1 --2- 172.21.15.83:0/318410041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ef4850 0x55ae21ef2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4906a00640 1 -- 172.21.15.83:0/318410041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21eea180 msgr2=0x55ae21ef1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4906a00640 1 --2- 172.21.15.83:0/318410041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21eea180 0x55ae21ef1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.698+0000 7f4906a00640 1 -- 172.21.15.83:0/318410041 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae21e07920 con 0x55ae21ee9860 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.699+0000 7f4906a00640 1 --2- 172.21.15.83:0/318410041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 0x55ae21ef1710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48f809cd20 tx=0x7f48f80d46c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f22eae27e4a09a1e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.699+0000 7f48f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.699+0000 7f48f4c00640 1 -- 172.21.15.83:0/318410041 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8020590 con 0x55ae21ee9860 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.699+0000 7f4910823740 1 -- 172.21.15.83:0/318410041 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48f809d490 con 0x55ae21ee9860 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.699+0000 7f48f4c00640 1 -- 172.21.15.83:0/318410041 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f48f80b6200 con 0x55ae21ee9860 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.699+0000 7f48f4c00640 1 -- 172.21.15.83:0/318410041 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f48f809c6f0 con 0x55ae21ee9860 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.699+0000 7f4910823740 10 client.25697 fetch_fsmap learned FSMap version 24 2024-04-23T14:50:13.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.699+0000 7f4910823740 1 -- 172.21.15.83:0/318410041 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ae21c2b7d0 con 0x55ae21ee9860 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.699+0000 7f48f4c00640 1 -- 172.21.15.83:0/318410041 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f809d490 con 0x55ae21ee9860 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.700+0000 7f48f4c00640 1 -- 172.21.15.83:0/318410041 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f48f80ce7c0 con 0x55ae21ee9860 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.700+0000 7f4910823740 10 client.25697 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.700+0000 7f4910823740 10 client.25697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.700+0000 7f4910823740 20 client.25697 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.700+0000 7f4910823740 1 --2- 172.21.15.83:0/318410041 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae21f04930 0x55ae21f06d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.700+0000 7f4910823740 4 client.25697 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.700+0000 7f4910823740 1 -- 172.21.15.83:0/318410041 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae21e5cc10 con 0x55ae21f04930 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.700+0000 7f48fe000640 1 --2- 172.21.15.83:0/318410041 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae21f04930 0x55ae21f06d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.701+0000 7f48fe000640 1 --2- 172.21.15.83:0/318410041 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae21f04930 0x55ae21f06d20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d19bcd27523d72 server_cookie=a2dfd9b5211f4194 in_seq=0 out_seq=0 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.701+0000 7f48f4c00640 10 client.25697 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.701+0000 7f48f4c00640 1 -- 172.21.15.83:0/318410041 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae21e5cc10 con 0x55ae21f04930 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.701+0000 7f48f4c00640 10 client.25697 handle_command_reply: tid=0 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.701+0000 7f4910823740 1 client.25697 shutdown 2024-04-23T14:50:13.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.701+0000 7f4910823740 1 -- 172.21.15.83:0/318410041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 msgr2=0x55ae21ef1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:13.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.701+0000 7f4910823740 1 --2- 172.21.15.83:0/318410041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 0x55ae21ef1710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48f809cd20 tx=0x7f48f80d46c0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.702+0000 7f4910823740 1 -- 172.21.15.83:0/318410041 shutdown_connections 2024-04-23T14:50:13.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.702+0000 7f4910823740 1 --2- 172.21.15.83:0/318410041 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae21f04930 0x55ae21f06d20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.702+0000 7f4910823740 1 --2- 172.21.15.83:0/318410041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ef4850 0x55ae21ef2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.702+0000 7f4910823740 1 --2- 172.21.15.83:0/318410041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21eea180 0x55ae21ef1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.702+0000 7f4910823740 1 --2- 172.21.15.83:0/318410041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 0x55ae21ef1710 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.702+0000 7f48f4c00640 0 client.25697 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:13.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.702+0000 7f4910823740 1 -- 172.21.15.83:0/318410041 >> 172.21.15.83:0/318410041 conn(0x55ae21ee5b80 msgr2=0x55ae21eeb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:13.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.702+0000 7f4910823740 1 -- 172.21.15.83:0/318410041 shutdown_connections 2024-04-23T14:50:13.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.702+0000 7f4910823740 1 -- 172.21.15.83:0/318410041 wait complete. 2024-04-23T14:50:13.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.702+0000 7f4910823740 20 client.25697 trim_cache size 0 max 16384 2024-04-23T14:50:13.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:13 smithi203 ceph-mon[32800]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-23T14:50:13.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.723+0000 7f4910823740 1 Processor -- start 2024-04-23T14:50:13.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.723+0000 7f4910823740 1 -- start start 2024-04-23T14:50:13.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.723+0000 7f4910823740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21ee9860 0x55ae21f004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.723+0000 7f4910823740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21eea180 0x55ae21ef0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.723+0000 7f4910823740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ef4850 0x55ae21f0c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.723+0000 7f4910823740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49081b2870 con 0x55ae21ee9860 2024-04-23T14:50:13.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.723+0000 7f4910823740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f49081b0140 con 0x55ae21ef4850 2024-04-23T14:50:13.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.723+0000 7f4910823740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f49081adf90 con 0x55ae21eea180 2024-04-23T14:50:13.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.723+0000 7f4906a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21ee9860 0x55ae21f004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.723+0000 7f4907400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ef4850 0x55ae21f0c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.724+0000 7f4906a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21ee9860 0x55ae21f004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36904/0 (socket says 172.21.15.83:36904) 2024-04-23T14:50:13.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.724+0000 7f48fe000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21eea180 0x55ae21ef0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.724+0000 7f4907400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ef4850 0x55ae21f0c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:48856/0 (socket says 172.21.15.83:48856) 2024-04-23T14:50:13.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.724+0000 7f4906a00640 1 -- 172.21.15.83:0/2380975461 learned_addr learned my addr 172.21.15.83:0/2380975461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:13.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.724+0000 7f48fe000640 1 -- 172.21.15.83:0/2380975461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ef4850 msgr2=0x55ae21f0c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:13.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.724+0000 7f48fe000640 1 --2- 172.21.15.83:0/2380975461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ef4850 0x55ae21f0c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.724+0000 7f48fe000640 1 -- 172.21.15.83:0/2380975461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21ee9860 msgr2=0x55ae21f004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:13.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.724+0000 7f48fe000640 1 --2- 172.21.15.83:0/2380975461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21ee9860 0x55ae21f004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.724+0000 7f48fe000640 1 -- 172.21.15.83:0/2380975461 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae21b613f0 con 0x55ae21eea180 2024-04-23T14:50:13.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.724+0000 7f48fe000640 1 --2- 172.21.15.83:0/2380975461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21eea180 0x55ae21ef0650 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55ae21ef0f70 tx=0x7f48f0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=836a26f94ea30151 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:13.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.724+0000 7f48f7400640 1 -- 172.21.15.83:0/2380975461 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0005220 con 0x55ae21eea180 2024-04-23T14:50:13.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.725+0000 7f48f7400640 1 -- 172.21.15.83:0/2380975461 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f0004c80 con 0x55ae21eea180 2024-04-23T14:50:13.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.725+0000 7f48f7400640 1 -- 172.21.15.83:0/2380975461 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0020790 con 0x55ae21eea180 2024-04-23T14:50:13.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.726+0000 7f4910823740 1 -- 172.21.15.83:0/2380975461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21eea180 msgr2=0x55ae21ef0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:13.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.726+0000 7f4910823740 1 --2- 172.21.15.83:0/2380975461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21eea180 0x55ae21ef0650 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55ae21ef0f70 tx=0x7f48f0009260 comp rx=0 tx=0).stop 2024-04-23T14:50:13.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.726+0000 7f4910823740 1 -- 172.21.15.83:0/2380975461 shutdown_connections 2024-04-23T14:50:13.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.726+0000 7f4910823740 1 --2- 172.21.15.83:0/2380975461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ef4850 0x55ae21f0c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.726+0000 7f4910823740 1 --2- 172.21.15.83:0/2380975461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21eea180 0x55ae21ef0650 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.726+0000 7f4910823740 1 --2- 172.21.15.83:0/2380975461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21ee9860 0x55ae21f004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.726+0000 7f4910823740 1 -- 172.21.15.83:0/2380975461 >> 172.21.15.83:0/2380975461 conn(0x55ae21ee5b80 msgr2=0x55ae21ee4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.726+0000 7f4910823740 1 -- 172.21.15.83:0/2380975461 shutdown_connections 2024-04-23T14:50:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.726+0000 7f4910823740 1 -- 172.21.15.83:0/2380975461 wait complete. 2024-04-23T14:50:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.726+0000 7f4910823740 1 -- start start 2024-04-23T14:50:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.727+0000 7f4910823740 1 Processor -- start 2024-04-23T14:50:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.727+0000 7f4910823740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 0x55ae21ef83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.727+0000 7f4910823740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21eea180 0x55ae21ef88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.727+0000 7f4910823740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ef4850 0x55ae21ef8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.727+0000 7f4910823740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49081ae600 con 0x55ae21eea180 2024-04-23T14:50:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.727+0000 7f4910823740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f49081afa70 con 0x55ae21ee9860 2024-04-23T14:50:13.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.727+0000 7f4910823740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f49081aecd0 con 0x55ae21ef4850 2024-04-23T14:50:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f4910823740 10 client.0 _ll_register_callbacks cb 0x7f48f80bbf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f4906a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 0x55ae21ef83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f48fe000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21eea180 0x55ae21ef88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f4906a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 0x55ae21ef83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:48868/0 (socket says 172.21.15.83:48868) 2024-04-23T14:50:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f4906a00640 1 -- 172.21.15.83:0/2272328724 learned_addr learned my addr 172.21.15.83:0/2272328724 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f4907400640 1 --2- 172.21.15.83:0/2272328724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ef4850 0x55ae21ef8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f4906a00640 1 -- 172.21.15.83:0/2272328724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ef4850 msgr2=0x55ae21ef8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f4906a00640 1 --2- 172.21.15.83:0/2272328724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ef4850 0x55ae21ef8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f4906a00640 1 -- 172.21.15.83:0/2272328724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21eea180 msgr2=0x55ae21ef88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f4906a00640 1 --2- 172.21.15.83:0/2272328724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21eea180 0x55ae21ef88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f4906a00640 1 -- 172.21.15.83:0/2272328724 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae21b210f0 con 0x55ae21ee9860 2024-04-23T14:50:13.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f4906a00640 1 --2- 172.21.15.83:0/2272328724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 0x55ae21ef83b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f48f809cf50 tx=0x7f48f80b6410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6e5eb7385f5f947 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.728+0000 7f48f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f48f4c00640 1 -- 172.21.15.83:0/2272328724 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f809c8d0 con 0x55ae21ee9860 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f48f4c00640 1 -- 172.21.15.83:0/2272328724 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f48f8005310 con 0x55ae21ee9860 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f4910823740 1 -- 172.21.15.83:0/2272328724 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48f0004c80 con 0x55ae21ee9860 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f48f4c00640 1 -- 172.21.15.83:0/2272328724 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f48f8063e60 con 0x55ae21ee9860 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f4910823740 10 client.25703 fetch_fsmap learned FSMap version 24 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f4910823740 1 -- 172.21.15.83:0/2272328724 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ae21e14810 con 0x55ae21ee9860 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f48f4c00640 1 -- 172.21.15.83:0/2272328724 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f0004c80 con 0x55ae21ee9860 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f48f4c00640 1 -- 172.21.15.83:0/2272328724 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f48f807bbb0 con 0x55ae21ee9860 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f4910823740 10 client.25703 fetch_fsmap finished waiting for FSMap version 24 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f4910823740 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f4910823740 20 client.25703 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f4910823740 1 --2- 172.21.15.83:0/2272328724 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae21f73730 0x55ae21f75b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f4910823740 4 client.25703 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.729+0000 7f4910823740 1 -- 172.21.15.83:0/2272328724 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae21e56a80 con 0x55ae21f73730 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.730+0000 7f48fe000640 1 --2- 172.21.15.83:0/2272328724 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae21f73730 0x55ae21f75b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.730+0000 7f48fe000640 1 --2- 172.21.15.83:0/2272328724 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae21f73730 0x55ae21f75b20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a0c09c4f08cd0f server_cookie=f458750f657c91c1 in_seq=0 out_seq=0 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.730+0000 7f48f4c00640 10 client.25703 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.730+0000 7f48f4c00640 1 -- 172.21.15.83:0/2272328724 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ae21e56a80 con 0x55ae21f73730 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.730+0000 7f48f4c00640 10 client.25703 handle_command_reply: tid=0 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.730+0000 7f4910823740 1 client.25703 shutdown 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.731+0000 7f4910823740 1 -- 172.21.15.83:0/2272328724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 msgr2=0x55ae21ef83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.731+0000 7f4910823740 1 --2- 172.21.15.83:0/2272328724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 0x55ae21ef83b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f48f809cf50 tx=0x7f48f80b6410 comp rx=0 tx=0).stop 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.731+0000 7f4910823740 1 -- 172.21.15.83:0/2272328724 shutdown_connections 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.731+0000 7f4910823740 1 --2- 172.21.15.83:0/2272328724 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae21f73730 0x55ae21f75b20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.731+0000 7f4910823740 1 --2- 172.21.15.83:0/2272328724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae21ef4850 0x55ae21ef8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.731+0000 7f4910823740 1 --2- 172.21.15.83:0/2272328724 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae21eea180 0x55ae21ef88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.731+0000 7f4910823740 1 --2- 172.21.15.83:0/2272328724 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae21ee9860 0x55ae21ef83b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.731+0000 7f4910823740 1 -- 172.21.15.83:0/2272328724 >> 172.21.15.83:0/2272328724 conn(0x55ae21ee5b80 msgr2=0x55ae21ef1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:13.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.731+0000 7f48f4c00640 0 client.25703 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:13.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.731+0000 7f4910823740 1 -- 172.21.15.83:0/2272328724 shutdown_connections 2024-04-23T14:50:13.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.731+0000 7f4910823740 1 -- 172.21.15.83:0/2272328724 wait complete. 2024-04-23T14:50:13.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.731+0000 7f4910823740 20 client.25703 trim_cache size 0 max 16384 2024-04-23T14:50:13.737 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:50:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:50:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:50:13.737 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:50:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.732+0000 7f490da00640 1 -- 172.21.15.83:0/1395553265 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f48ec0a3d20 msgr2=0x7f48ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.732+0000 7f490da00640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f48ec0a3d20 0x7f48ec0a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4908075320 tx=0x7f4900005f90 comp rx=0 tx=0).stop 2024-04-23T14:50:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.732+0000 7f490da00640 1 -- 172.21.15.83:0/1395553265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49080743d0 msgr2=0x7f49081a0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.732+0000 7f490da00640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49080743d0 0x7f49081a0770 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f48f800eb90 tx=0x7f48f800ef90 comp rx=0 tx=0).stop 2024-04-23T14:50:13.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.732+0000 7f490da00640 1 -- 172.21.15.83:0/1395553265 shutdown_connections 2024-04-23T14:50:13.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.732+0000 7f490da00640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f490810e880 0x7f490819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.732+0000 7f490da00640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4908074cf0 0x7f49081a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.732+0000 7f490da00640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f48ec0a3d20 0x7f48ec0a61e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.732+0000 7f490da00640 1 --2- 172.21.15.83:0/1395553265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49080743d0 0x7f49081a0770 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.732+0000 7f490da00640 1 -- 172.21.15.83:0/1395553265 >> 172.21.15.83:0/1395553265 conn(0x7f4908079400 msgr2=0x7f49080fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:13.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.732+0000 7f490da00640 1 -- 172.21.15.83:0/1395553265 shutdown_connections 2024-04-23T14:50:13.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.732+0000 7f490da00640 1 -- 172.21.15.83:0/1395553265 wait complete. 2024-04-23T14:50:13.746 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:13 smithi083 ceph-mon[27867]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-23T14:50:13.749 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:50:13.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:50:13.758 INFO:tasks.workunit.client.0.smithi083.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-23T14:50:13.761 INFO:tasks.workunit.client.0.smithi083.stdout:checking for egrep... /usr/bin/grep -E 2024-04-23T14:50:13.787 INFO:tasks.workunit.client.0.smithi083.stdout:checking for AIX... no 2024-04-23T14:50:13.801 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gcc... (cached) gcc 2024-04-23T14:50:13.825 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-23T14:50:13.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:13 smithi173 ceph-mon[32443]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-23T14:50:13.828 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-23T14:50:13.828 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-23T14:50:13.828 INFO:tasks.workunit.client.0.smithi083.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-23T14:50:13.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.930+0000 7f390e600640 1 -- 172.21.15.83:0/1561891505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3908101b50 msgr2=0x7f3908105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:13.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.930+0000 7f390e600640 1 --2- 172.21.15.83:0/1561891505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3908101b50 0x7f3908105b10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009900 tx=0x7f38f402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.931+0000 7f390e600640 1 -- 172.21.15.83:0/1561891505 shutdown_connections 2024-04-23T14:50:13.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.931+0000 7f390e600640 1 --2- 172.21.15.83:0/1561891505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39081061f0 0x7f390810ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.931+0000 7f390e600640 1 --2- 172.21.15.83:0/1561891505 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3908101b50 0x7f3908105b10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.931+0000 7f390e600640 1 --2- 172.21.15.83:0/1561891505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39081011a0 0x7f3908101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.931+0000 7f390e600640 1 -- 172.21.15.83:0/1561891505 >> 172.21.15.83:0/1561891505 conn(0x7f3908077010 msgr2=0x7f3908077420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.931+0000 7f390e600640 1 -- 172.21.15.83:0/1561891505 shutdown_connections 2024-04-23T14:50:13.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.931+0000 7f390e600640 1 -- 172.21.15.83:0/1561891505 wait complete. 2024-04-23T14:50:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.932+0000 7f390e600640 1 Processor -- start 2024-04-23T14:50:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.932+0000 7f390e600640 1 -- start start 2024-04-23T14:50:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.932+0000 7f390e600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39081011a0 0x7f39081a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.932+0000 7f390e600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39081061f0 0x7f39081a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.932+0000 7f390e600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f390819a940 0x7f390819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.932+0000 7f390e600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3908111260 con 0x7f39081011a0 2024-04-23T14:50:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.932+0000 7f390e600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f39081110e0 con 0x7f390819a940 2024-04-23T14:50:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.932+0000 7f390e600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f39081113e0 con 0x7f39081061f0 2024-04-23T14:50:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.932+0000 7f3907400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39081011a0 0x7f39081a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.933+0000 7f3907400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39081011a0 0x7f39081a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36924/0 (socket says 172.21.15.83:36924) 2024-04-23T14:50:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.933+0000 7f3907400640 1 -- 172.21.15.83:0/2771834116 learned_addr learned my addr 172.21.15.83:0/2771834116 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:13.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.933+0000 7f3907e00640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f390819a940 0x7f390819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.933+0000 7f3906a00640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39081061f0 0x7f39081a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.933+0000 7f3907400640 1 -- 172.21.15.83:0/2771834116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39081061f0 msgr2=0x7f39081a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.933+0000 7f3907400640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39081061f0 0x7f39081a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.933+0000 7f3907400640 1 -- 172.21.15.83:0/2771834116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f390819a940 msgr2=0x7f390819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.933+0000 7f3907400640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f390819a940 0x7f390819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.933+0000 7f3907400640 1 -- 172.21.15.83:0/2771834116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390819b460 con 0x7f39081011a0 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.933+0000 7f3907e00640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f390819a940 0x7f390819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.933+0000 7f3907400640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39081011a0 0x7f39081a0790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f38f800e9f0 tx=0x7f38f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.934+0000 7f38ffe00640 1 -- 172.21.15.83:0/2771834116 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f800cd80 con 0x7f39081011a0 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.934+0000 7f390e600640 1 -- 172.21.15.83:0/2771834116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390819b750 con 0x7f39081011a0 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.934+0000 7f38ffe00640 1 -- 172.21.15.83:0/2771834116 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f8010910 con 0x7f39081011a0 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.934+0000 7f38ffe00640 1 -- 172.21.15.83:0/2771834116 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f80188f0 con 0x7f39081011a0 2024-04-23T14:50:13.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.934+0000 7f390e600640 1 -- 172.21.15.83:0/2771834116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39081a79f0 con 0x7f39081011a0 2024-04-23T14:50:13.937 INFO:tasks.workunit.client.0.smithi083.stdout:checking for main in -lm... yes 2024-04-23T14:50:13.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.935+0000 7f390e600640 1 -- 172.21.15.83:0/2771834116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39081025f0 con 0x7f39081011a0 2024-04-23T14:50:13.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.937+0000 7f38ffe00640 1 -- 172.21.15.83:0/2771834116 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f38f8010ac0 con 0x7f39081011a0 2024-04-23T14:50:13.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.938+0000 7f38ffe00640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38ec0a3d20 0x7f38ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:13.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.939+0000 7f38ffe00640 1 -- 172.21.15.83:0/2771834116 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f38f80d0c00 con 0x7f39081011a0 2024-04-23T14:50:13.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.939+0000 7f3906a00640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38ec0a3d20 0x7f38ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:13.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.940+0000 7f3906a00640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38ec0a3d20 0x7f38ec0a61e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f38f4004770 tx=0x7f38f40058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:13.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:13.943+0000 7f38ffe00640 1 -- 172.21.15.83:0/2771834116 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38f809cc00 con 0x7f39081011a0 2024-04-23T14:50:14.037 INFO:tasks.workunit.client.0.smithi083.stdout:checking for main in -lpthread... yes 2024-04-23T14:50:14.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.196+0000 7f390e600640 1 -- 172.21.15.83:0/2771834116 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3908104220 con 0x7f39081011a0 2024-04-23T14:50:14.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.197+0000 7f38ffe00640 1 -- 172.21.15.83:0/2771834116 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11863 (secure 0 0 0) 0x7f38f80a0f00 con 0x7f39081011a0 2024-04-23T14:50:14.198 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:50:14.198 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:48:27.525285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24599,24532,24542]},"id":1}]} 2024-04-23T14:50:14.198 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 24 2024-04-23T14:50:14.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.200+0000 7f390e600640 1 -- 172.21.15.83:0/2771834116 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38ec0a3d20 msgr2=0x7f38ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.200+0000 7f390e600640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38ec0a3d20 0x7f38ec0a61e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f38f4004770 tx=0x7f38f40058d0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.200+0000 7f390e600640 1 -- 172.21.15.83:0/2771834116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39081011a0 msgr2=0x7f39081a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.200+0000 7f390e600640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39081011a0 0x7f39081a0790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f38f800e9f0 tx=0x7f38f800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.200+0000 7f390e600640 1 -- 172.21.15.83:0/2771834116 shutdown_connections 2024-04-23T14:50:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.200+0000 7f390e600640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f390819a940 0x7f390819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.200+0000 7f390e600640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39081061f0 0x7f39081a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.200+0000 7f390e600640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38ec0a3d20 0x7f38ec0a61e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.200+0000 7f390e600640 1 --2- 172.21.15.83:0/2771834116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39081011a0 0x7f39081a0790 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.200+0000 7f390e600640 1 -- 172.21.15.83:0/2771834116 >> 172.21.15.83:0/2771834116 conn(0x7f3908077010 msgr2=0x7f39080ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.200+0000 7f390e600640 1 -- 172.21.15.83:0/2771834116 shutdown_connections 2024-04-23T14:50:14.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.200+0000 7f390e600640 1 -- 172.21.15.83:0/2771834116 wait complete. 2024-04-23T14:50:14.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:50:14.355 INFO:tasks.workunit.client.0.smithi083.stdout:checking for ANSI C header files... yes 2024-04-23T14:50:14.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.456+0000 7f3210e00640 1 -- 172.21.15.83:0/2044576864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320c112fd0 msgr2=0x7f320c1153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.456+0000 7f3210e00640 1 --2- 172.21.15.83:0/2044576864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320c112fd0 0x7f320c1153c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3204009ca0 tx=0x7f320402a950 comp rx=0 tx=0).stop 2024-04-23T14:50:14.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.456+0000 7f3210e00640 1 -- 172.21.15.83:0/2044576864 shutdown_connections 2024-04-23T14:50:14.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.456+0000 7f3210e00640 1 --2- 172.21.15.83:0/2044576864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320c112fd0 0x7f320c1153c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.456+0000 7f3210e00640 1 --2- 172.21.15.83:0/2044576864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f320c074cc0 0x7f320c112a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.456+0000 7f3210e00640 1 --2- 172.21.15.83:0/2044576864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0743a0 0x7f320c074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.456+0000 7f3210e00640 1 -- 172.21.15.83:0/2044576864 >> 172.21.15.83:0/2044576864 conn(0x7f320c0fda70 msgr2=0x7f320c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.456+0000 7f3210e00640 1 -- 172.21.15.83:0/2044576864 shutdown_connections 2024-04-23T14:50:14.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.456+0000 7f3210e00640 1 -- 172.21.15.83:0/2044576864 wait complete. 2024-04-23T14:50:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.457+0000 7f3210e00640 1 Processor -- start 2024-04-23T14:50:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f3210e00640 1 -- start start 2024-04-23T14:50:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f3210e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0743a0 0x7f320c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f3210e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f320c074cc0 0x7f320c19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f3210e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320c112fd0 0x7f320c1a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f3210e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1189d0 con 0x7f320c0743a0 2024-04-23T14:50:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f3210e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f320c118850 con 0x7f320c112fd0 2024-04-23T14:50:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f3210e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f320c118b50 con 0x7f320c074cc0 2024-04-23T14:50:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f3209e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0743a0 0x7f320c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f3203e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f320c074cc0 0x7f320c19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f3209e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0743a0 0x7f320c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:36936/0 (socket says 172.21.15.83:36936) 2024-04-23T14:50:14.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f3203e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f320c074cc0 0x7f320c19f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:53896/0 (socket says 172.21.15.83:53896) 2024-04-23T14:50:14.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f3209e00640 1 -- 172.21.15.83:0/3310557999 learned_addr learned my addr 172.21.15.83:0/3310557999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.458+0000 7f320a800640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320c112fd0 0x7f320c1a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.459+0000 7f320a800640 1 -- 172.21.15.83:0/3310557999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f320c074cc0 msgr2=0x7f320c19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.459+0000 7f320a800640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f320c074cc0 0x7f320c19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.459+0000 7f320a800640 1 -- 172.21.15.83:0/3310557999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0743a0 msgr2=0x7f320c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.459+0000 7f320a800640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0743a0 0x7f320c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.459+0000 7f320a800640 1 -- 172.21.15.83:0/3310557999 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320c1a3f80 con 0x7f320c112fd0 2024-04-23T14:50:14.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.459+0000 7f320a800640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320c112fd0 0x7f320c1a3850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f320402ae70 tx=0x7f320402f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.459+0000 7f3202a00640 1 -- 172.21.15.83:0/3310557999 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204034040 con 0x7f320c112fd0 2024-04-23T14:50:14.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.460+0000 7f3202a00640 1 -- 172.21.15.83:0/3310557999 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3204030920 con 0x7f320c112fd0 2024-04-23T14:50:14.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.460+0000 7f3210e00640 1 -- 172.21.15.83:0/3310557999 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320c1a4210 con 0x7f320c112fd0 2024-04-23T14:50:14.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.460+0000 7f3202a00640 1 -- 172.21.15.83:0/3310557999 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204004610 con 0x7f320c112fd0 2024-04-23T14:50:14.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.460+0000 7f3210e00640 1 -- 172.21.15.83:0/3310557999 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320c1abb00 con 0x7f320c112fd0 2024-04-23T14:50:14.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.464+0000 7f3213d97740 1 Processor -- start 2024-04-23T14:50:14.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.464+0000 7f3202a00640 1 -- 172.21.15.83:0/3310557999 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3204037020 con 0x7f320c112fd0 2024-04-23T14:50:14.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.464+0000 7f3213d97740 1 -- start start 2024-04-23T14:50:14.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.464+0000 7f3213d97740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abdd950 0x55fd7abddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.464+0000 7f3213d97740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 0x55fd7abe8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.464+0000 7f3213d97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abe8940 0x55fd7abead30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.464+0000 7f3213d97740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fd7ab431d0 con 0x55fd7abdd950 2024-04-23T14:50:14.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.464+0000 7f3213d97740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fd7aae9450 con 0x55fd7abde270 2024-04-23T14:50:14.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.464+0000 7f3213d97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fd7ab3f190 con 0x55fd7abe8940 2024-04-23T14:50:14.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.465+0000 7f3209e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 0x55fd7abe8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.465+0000 7f3209e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 0x55fd7abe8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:48896/0 (socket says 172.21.15.83:48896) 2024-04-23T14:50:14.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.465+0000 7f3209e00640 1 -- 172.21.15.83:0/1257220128 learned_addr learned my addr 172.21.15.83:0/1257220128 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.465+0000 7f3203e00640 1 --2- 172.21.15.83:0/1257220128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abdd950 0x55fd7abddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.465+0000 7f3203e00640 1 --2- 172.21.15.83:0/1257220128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abe8940 0x55fd7abead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.465+0000 7f3209e00640 1 -- 172.21.15.83:0/1257220128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abe8940 msgr2=0x55fd7abead30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.465+0000 7f3209e00640 1 --2- 172.21.15.83:0/1257220128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abe8940 0x55fd7abead30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.465+0000 7f3209e00640 1 -- 172.21.15.83:0/1257220128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abdd950 msgr2=0x55fd7abddd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.465+0000 7f3209e00640 1 --2- 172.21.15.83:0/1257220128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abdd950 0x55fd7abddd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.465+0000 7f3209e00640 1 -- 172.21.15.83:0/1257220128 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd7aba6310 con 0x55fd7abde270 2024-04-23T14:50:14.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.465+0000 7f3203e00640 1 --2- 172.21.15.83:0/1257220128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abdd950 0x55fd7abddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:50:14.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.465+0000 7f3209e00640 1 --2- 172.21.15.83:0/1257220128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 0x55fd7abe8400 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009dc0 tx=0x7f31fc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3b1c00fc6ad2eac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.466+0000 7f31fb400640 1 -- 172.21.15.83:0/1257220128 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc005220 con 0x55fd7abde270 2024-04-23T14:50:14.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.466+0000 7f31fb400640 1 -- 172.21.15.83:0/1257220128 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31fc004c80 con 0x55fd7abde270 2024-04-23T14:50:14.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.466+0000 7f31fb400640 1 -- 172.21.15.83:0/1257220128 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc019970 con 0x55fd7abde270 2024-04-23T14:50:14.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.466+0000 7f3202a00640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f31f00a3d20 0x7f31f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.466+0000 7f3202a00640 1 -- 172.21.15.83:0/3310557999 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f32040f14c0 con 0x7f320c112fd0 2024-04-23T14:50:14.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.466+0000 7f3203e00640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f31f00a3d20 0x7f31f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.467+0000 7f3213d97740 1 -- 172.21.15.83:0/1257220128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 msgr2=0x55fd7abe8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.467+0000 7f3213d97740 1 --2- 172.21.15.83:0/1257220128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 0x55fd7abe8400 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009dc0 tx=0x7f31fc011810 comp rx=0 tx=0).stop 2024-04-23T14:50:14.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.467+0000 7f3213d97740 1 -- 172.21.15.83:0/1257220128 shutdown_connections 2024-04-23T14:50:14.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.467+0000 7f3213d97740 1 --2- 172.21.15.83:0/1257220128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abe8940 0x55fd7abead30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.467+0000 7f3213d97740 1 --2- 172.21.15.83:0/1257220128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 0x55fd7abe8400 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.467+0000 7f3213d97740 1 --2- 172.21.15.83:0/1257220128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abdd950 0x55fd7abddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.467+0000 7f3213d97740 1 -- 172.21.15.83:0/1257220128 >> 172.21.15.83:0/1257220128 conn(0x55fd7abd9c70 msgr2=0x55fd7abda050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.467+0000 7f3213d97740 1 -- 172.21.15.83:0/1257220128 shutdown_connections 2024-04-23T14:50:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.467+0000 7f3203e00640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f31f00a3d20 0x7f31f00a61e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31f4004650 tx=0x7f31f4014040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.467+0000 7f3213d97740 1 -- 172.21.15.83:0/1257220128 wait complete. 2024-04-23T14:50:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.468+0000 7f3213d97740 1 -- start start 2024-04-23T14:50:14.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.468+0000 7f3213d97740 1 Processor -- start 2024-04-23T14:50:14.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.468+0000 7f3213d97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 0x55fd7abe83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.468+0000 7f3213d97740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 0x55fd7abe56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.468+0000 7f3213d97740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abe8940 0x55fd7abe5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.468+0000 7f3213d97740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fd7abeb7d0 con 0x55fd7abe8940 2024-04-23T14:50:14.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.468+0000 7f3213d97740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fd7aafc810 con 0x55fd7abde270 2024-04-23T14:50:14.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.468+0000 7f3213d97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fd7abeb950 con 0x55fd7abdd950 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f3213d97740 10 client.0 _ll_register_callbacks cb 0x55fd7aaca140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f3203e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 0x55fd7abe56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f3209e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 0x55fd7abe83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f3203e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 0x55fd7abe56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:48912/0 (socket says 172.21.15.83:48912) 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f3209e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 0x55fd7abe83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:53924/0 (socket says 172.21.15.83:53924) 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f3203e00640 1 -- 172.21.15.83:0/2286322174 learned_addr learned my addr 172.21.15.83:0/2286322174 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f320a800640 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abe8940 0x55fd7abe5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f3203e00640 1 -- 172.21.15.83:0/2286322174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 msgr2=0x55fd7abe83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f3203e00640 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 0x55fd7abe83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f3203e00640 1 -- 172.21.15.83:0/2286322174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abe8940 msgr2=0x55fd7abe5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f3203e00640 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abe8940 0x55fd7abe5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f3203e00640 1 -- 172.21.15.83:0/2286322174 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd7ab0d3f0 con 0x55fd7abde270 2024-04-23T14:50:14.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.469+0000 7f320a800640 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abe8940 0x55fd7abe5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:50:14.472 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:14 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2771834116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:50:14.472 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-23T14:50:14.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.470+0000 7f3203e00640 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 0x55fd7abe56c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31f4003e90 tx=0x7f31f40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1862d47b9f4d3a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.470+0000 7f31efe00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:50:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.472+0000 7f31efe00640 1 -- 172.21.15.83:0/2286322174 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4018030 con 0x55fd7abde270 2024-04-23T14:50:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.472+0000 7f31efe00640 1 -- 172.21.15.83:0/2286322174 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f31f4003ec0 con 0x55fd7abde270 2024-04-23T14:50:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.472+0000 7f3213d97740 1 -- 172.21.15.83:0/2286322174 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31fc004c80 con 0x55fd7abde270 2024-04-23T14:50:14.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.473+0000 7f31efe00640 1 -- 172.21.15.83:0/2286322174 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f31f4020650 con 0x55fd7abde270 2024-04-23T14:50:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.473+0000 7f3213d97740 10 client.25721 fetch_fsmap learned FSMap version 25 2024-04-23T14:50:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.473+0000 7f3213d97740 1 -- 172.21.15.83:0/2286322174 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fd7a8553f0 con 0x55fd7abde270 2024-04-23T14:50:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.473+0000 7f31efe00640 1 -- 172.21.15.83:0/2286322174 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31fc004c80 con 0x55fd7abde270 2024-04-23T14:50:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.473+0000 7f31efe00640 1 -- 172.21.15.83:0/2286322174 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f31f4032af0 con 0x55fd7abde270 2024-04-23T14:50:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.474+0000 7f3213d97740 10 client.25721 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:50:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.474+0000 7f3213d97740 10 client.25721 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:50:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.474+0000 7f3213d97740 20 client.25721 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.474+0000 7f3213d97740 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55fd7abf8b50 0x55fd7abfafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.474+0000 7f3213d97740 4 client.25721 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:50:14.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.474+0000 7f3213d97740 1 -- 172.21.15.83:0/2286322174 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd7ab50bf0 con 0x55fd7abf8b50 2024-04-23T14:50:14.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.474+0000 7f3209e00640 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55fd7abf8b50 0x55fd7abfafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.475+0000 7f3209e00640 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55fd7abf8b50 0x55fd7abfafb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ec905ef0425c7c9 server_cookie=d12d3b911c91d1f7 in_seq=0 out_seq=0 2024-04-23T14:50:14.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.475+0000 7f31efe00640 10 client.25721 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:50:14.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f31efe00640 1 -- 172.21.15.83:0/2286322174 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd7ab50bf0 con 0x55fd7abf8b50 2024-04-23T14:50:14.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f31efe00640 10 client.25721 handle_command_reply: tid=0 2024-04-23T14:50:14.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f3213d97740 1 client.25721 shutdown 2024-04-23T14:50:14.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f3213d97740 1 -- 172.21.15.83:0/2286322174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 msgr2=0x55fd7abe56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f3213d97740 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 0x55fd7abe56c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31f4003e90 tx=0x7f31f40040c0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f3213d97740 1 -- 172.21.15.83:0/2286322174 shutdown_connections 2024-04-23T14:50:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f3213d97740 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55fd7abf8b50 0x55fd7abfafb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f3213d97740 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abe8940 0x55fd7abe5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f3213d97740 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abde270 0x55fd7abe56c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f31efe00640 0 client.25721 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:50:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f3213d97740 1 --2- 172.21.15.83:0/2286322174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 0x55fd7abe83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f3213d97740 1 -- 172.21.15.83:0/2286322174 >> 172.21.15.83:0/2286322174 conn(0x55fd7abd9c70 msgr2=0x55fd7abdf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f3213d97740 1 -- 172.21.15.83:0/2286322174 shutdown_connections 2024-04-23T14:50:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f3213d97740 1 -- 172.21.15.83:0/2286322174 wait complete. 2024-04-23T14:50:14.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.476+0000 7f3213d97740 20 client.25721 trim_cache size 0 max 16384 2024-04-23T14:50:14.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.491+0000 7f3213d97740 1 Processor -- start 2024-04-23T14:50:14.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.491+0000 7f3213d97740 1 -- start start 2024-04-23T14:50:14.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.491+0000 7f3213d97740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abdd950 0x55fd7abd9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.491+0000 7f3213d97740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abde270 0x55fd7abe7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.491+0000 7f3213d97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abe8940 0x55fd7ac00880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.491+0000 7f3213d97740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1b6be0 con 0x55fd7abde270 2024-04-23T14:50:14.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.491+0000 7f3213d97740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1b44b0 con 0x55fd7abdd950 2024-04-23T14:50:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.491+0000 7f3213d97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1b2300 con 0x55fd7abe8940 2024-04-23T14:50:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.491+0000 7f320a800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abe8940 0x55fd7ac00880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f320a800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abe8940 0x55fd7ac00880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:53940/0 (socket says 172.21.15.83:53940) 2024-04-23T14:50:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f320a800640 1 -- 172.21.15.83:0/892726519 learned_addr learned my addr 172.21.15.83:0/892726519 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f3203e00640 1 --2- 172.21.15.83:0/892726519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abde270 0x55fd7abe7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f3209e00640 1 --2- 172.21.15.83:0/892726519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abdd950 0x55fd7abd9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f320a800640 1 -- 172.21.15.83:0/892726519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abdd950 msgr2=0x55fd7abd9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f320a800640 1 --2- 172.21.15.83:0/892726519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abdd950 0x55fd7abd9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f320a800640 1 -- 172.21.15.83:0/892726519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abde270 msgr2=0x55fd7abe7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f320a800640 1 --2- 172.21.15.83:0/892726519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abde270 0x55fd7abe7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f320a800640 1 -- 172.21.15.83:0/892726519 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd7a91f7d0 con 0x55fd7abe8940 2024-04-23T14:50:14.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f320a800640 1 --2- 172.21.15.83:0/892726519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abe8940 0x55fd7ac00880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32040bd510 tx=0x7f320402f820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dd763c91854be13 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f31faa00640 1 -- 172.21.15.83:0/892726519 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32040c1930 con 0x55fd7abe8940 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f31faa00640 1 -- 172.21.15.83:0/892726519 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32040c61f0 con 0x55fd7abe8940 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.492+0000 7f31faa00640 1 -- 172.21.15.83:0/892726519 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32040c5fc0 con 0x55fd7abe8940 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.493+0000 7f3213d97740 1 -- 172.21.15.83:0/892726519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abe8940 msgr2=0x55fd7ac00880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.493+0000 7f3213d97740 1 --2- 172.21.15.83:0/892726519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abe8940 0x55fd7ac00880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f32040bd510 tx=0x7f320402f820 comp rx=0 tx=0).stop 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.493+0000 7f3213d97740 1 -- 172.21.15.83:0/892726519 shutdown_connections 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.493+0000 7f3213d97740 1 --2- 172.21.15.83:0/892726519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abe8940 0x55fd7ac00880 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.493+0000 7f3213d97740 1 --2- 172.21.15.83:0/892726519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abde270 0x55fd7abe7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.493+0000 7f3213d97740 1 --2- 172.21.15.83:0/892726519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abdd950 0x55fd7abd9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.493+0000 7f3213d97740 1 -- 172.21.15.83:0/892726519 >> 172.21.15.83:0/892726519 conn(0x55fd7abd9c70 msgr2=0x55fd7abe0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.493+0000 7f3213d97740 1 -- 172.21.15.83:0/892726519 shutdown_connections 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.493+0000 7f3213d97740 1 -- 172.21.15.83:0/892726519 wait complete. 2024-04-23T14:50:14.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.494+0000 7f3213d97740 1 -- start start 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.494+0000 7f3213d97740 1 Processor -- start 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.494+0000 7f3213d97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 0x55fd7abe9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.494+0000 7f3213d97740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abde270 0x55fd7abea260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.494+0000 7f3213d97740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abe8940 0x55fd7abef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.494+0000 7f3213d97740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1b2970 con 0x55fd7abde270 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.494+0000 7f3213d97740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1b3de0 con 0x55fd7abe8940 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.494+0000 7f3213d97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f320c1b3040 con 0x55fd7abdd950 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f3213d97740 10 client.0 _ll_register_callbacks cb 0x7f31f400bf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f320a800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abe8940 0x55fd7abef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f3209e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 0x55fd7abe9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f3203e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abde270 0x55fd7abea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f3209e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 0x55fd7abe9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:53944/0 (socket says 172.21.15.83:53944) 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f320a800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abe8940 0x55fd7abef1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:48924/0 (socket says 172.21.15.83:48924) 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f3209e00640 1 -- 172.21.15.83:0/4218817575 learned_addr learned my addr 172.21.15.83:0/4218817575 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f3209e00640 1 -- 172.21.15.83:0/4218817575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abe8940 msgr2=0x55fd7abef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f3209e00640 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abe8940 0x55fd7abef1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f3209e00640 1 -- 172.21.15.83:0/4218817575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abde270 msgr2=0x55fd7abea260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f3209e00640 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abde270 0x55fd7abea260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f3209e00640 1 -- 172.21.15.83:0/4218817575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd7abef720 con 0x55fd7abdd950 2024-04-23T14:50:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f3203e00640 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abde270 0x55fd7abea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f320a800640 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abe8940 0x55fd7abef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f3209e00640 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 0x55fd7abe9d20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005e00 tx=0x7f31fc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad961026f7a75f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.495+0000 7f31efe00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:50:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.496+0000 7f31efe00640 1 -- 172.21.15.83:0/4218817575 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc02f070 con 0x55fd7abdd950 2024-04-23T14:50:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.496+0000 7f31efe00640 1 -- 172.21.15.83:0/4218817575 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f31fc0050b0 con 0x55fd7abdd950 2024-04-23T14:50:14.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.496+0000 7f3213d97740 1 -- 172.21.15.83:0/4218817575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32040c61f0 con 0x55fd7abdd950 2024-04-23T14:50:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.496+0000 7f31efe00640 1 -- 172.21.15.83:0/4218817575 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f31fc03b050 con 0x55fd7abdd950 2024-04-23T14:50:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.496+0000 7f3213d97740 10 client.25696 fetch_fsmap learned FSMap version 25 2024-04-23T14:50:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.496+0000 7f3213d97740 1 -- 172.21.15.83:0/4218817575 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fd7ac5f850 con 0x55fd7abdd950 2024-04-23T14:50:14.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.496+0000 7f31efe00640 1 -- 172.21.15.83:0/4218817575 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32040c61f0 con 0x55fd7abdd950 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.497+0000 7f31efe00640 1 -- 172.21.15.83:0/4218817575 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f31fc003820 con 0x55fd7abdd950 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.497+0000 7f3213d97740 10 client.25696 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.497+0000 7f3213d97740 10 client.25696 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.497+0000 7f3213d97740 20 client.25696 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.497+0000 7f3213d97740 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55fd7ac5fde0 0x55fd7ac621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.497+0000 7f3213d97740 4 client.25696 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.497+0000 7f3213d97740 1 -- 172.21.15.83:0/4218817575 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd7ab4aa60 con 0x55fd7ac5fde0 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.497+0000 7f3203e00640 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55fd7ac5fde0 0x55fd7ac621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.498+0000 7f3203e00640 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55fd7ac5fde0 0x55fd7ac621d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c9eb374fbb72b04 server_cookie=fdb7c300dfebb7ad in_seq=0 out_seq=0 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.498+0000 7f31efe00640 10 client.25696 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.498+0000 7f31efe00640 1 -- 172.21.15.83:0/4218817575 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd7ab4aa60 con 0x55fd7ac5fde0 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.498+0000 7f31efe00640 10 client.25696 handle_command_reply: tid=0 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.498+0000 7f3213d97740 1 client.25696 shutdown 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.498+0000 7f3213d97740 1 -- 172.21.15.83:0/4218817575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 msgr2=0x55fd7abe9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.498+0000 7f3213d97740 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 0x55fd7abe9d20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f31fc005e00 tx=0x7f31fc011810 comp rx=0 tx=0).stop 2024-04-23T14:50:14.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3213d97740 1 -- 172.21.15.83:0/4218817575 shutdown_connections 2024-04-23T14:50:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3213d97740 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55fd7ac5fde0 0x55fd7ac621d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3213d97740 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fd7abe8940 0x55fd7abef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3213d97740 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fd7abde270 0x55fd7abea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f31efe00640 0 client.25696 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:50:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3213d97740 1 --2- 172.21.15.83:0/4218817575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fd7abdd950 0x55fd7abe9d20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3213d97740 1 -- 172.21.15.83:0/4218817575 >> 172.21.15.83:0/4218817575 conn(0x55fd7abd9c70 msgr2=0x55fd7abdf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3213d97740 1 -- 172.21.15.83:0/4218817575 shutdown_connections 2024-04-23T14:50:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3213d97740 1 -- 172.21.15.83:0/4218817575 wait complete. 2024-04-23T14:50:14.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3213d97740 20 client.25696 trim_cache size 0 max 16384 2024-04-23T14:50:14.502 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:50:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3210e00640 1 -- 172.21.15.83:0/3310557999 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f31f00a3d20 msgr2=0x7f31f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3210e00640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f31f00a3d20 0x7f31f00a61e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31f4004650 tx=0x7f31f4014040 comp rx=0 tx=0).stop 2024-04-23T14:50:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3210e00640 1 -- 172.21.15.83:0/3310557999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320c112fd0 msgr2=0x7f320c1a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.499+0000 7f3210e00640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320c112fd0 0x7f320c1a3850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f320402ae70 tx=0x7f320402f420 comp rx=0 tx=0).stop 2024-04-23T14:50:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.500+0000 7f3210e00640 1 -- 172.21.15.83:0/3310557999 shutdown_connections 2024-04-23T14:50:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.500+0000 7f3210e00640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f320c112fd0 0x7f320c1a3850 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.500+0000 7f3210e00640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f320c074cc0 0x7f320c19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.500+0000 7f3210e00640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f31f00a3d20 0x7f31f00a61e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.500+0000 7f3210e00640 1 --2- 172.21.15.83:0/3310557999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f320c0743a0 0x7f320c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.500+0000 7f3210e00640 1 -- 172.21.15.83:0/3310557999 >> 172.21.15.83:0/3310557999 conn(0x7f320c0fda70 msgr2=0x7f320c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.500+0000 7f3210e00640 1 -- 172.21.15.83:0/3310557999 shutdown_connections 2024-04-23T14:50:14.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.500+0000 7f3210e00640 1 -- 172.21.15.83:0/3310557999 wait complete. 2024-04-23T14:50:14.518 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:50:14.657 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/types.h... yes 2024-04-23T14:50:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.701+0000 7f3a91600640 1 -- 172.21.15.83:0/3931683966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c104310 msgr2=0x7f3a8c1046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.701+0000 7f3a91600640 1 --2- 172.21.15.83:0/3931683966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c104310 0x7f3a8c1046f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009900 tx=0x7f3a7c02a410 comp rx=0 tx=0).stop 2024-04-23T14:50:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.701+0000 7f3a91600640 1 -- 172.21.15.83:0/3931683966 shutdown_connections 2024-04-23T14:50:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.701+0000 7f3a91600640 1 --2- 172.21.15.83:0/3931683966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8c1093e0 0x7f3a8c110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.701+0000 7f3a91600640 1 --2- 172.21.15.83:0/3931683966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8c104cc0 0x7f3a8c108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.701+0000 7f3a91600640 1 --2- 172.21.15.83:0/3931683966 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c104310 0x7f3a8c1046f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.701+0000 7f3a91600640 1 -- 172.21.15.83:0/3931683966 >> 172.21.15.83:0/3931683966 conn(0x7f3a8c0ffb70 msgr2=0x7f3a8c101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.701+0000 7f3a91600640 1 -- 172.21.15.83:0/3931683966 shutdown_connections 2024-04-23T14:50:14.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.702+0000 7f3a91600640 1 -- 172.21.15.83:0/3931683966 wait complete. 2024-04-23T14:50:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.702+0000 7f3a91600640 1 Processor -- start 2024-04-23T14:50:14.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.703+0000 7f3a91600640 1 -- start start 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.703+0000 7f3a91600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8c104310 0x7f3a8c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.703+0000 7f3a91600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c1093e0 0x7f3a8c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.703+0000 7f3a91600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8c19a7b0 0x7f3a8c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.703+0000 7f3a91600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c114470 con 0x7f3a8c19a7b0 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.703+0000 7f3a91600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1142f0 con 0x7f3a8c104310 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.703+0000 7f3a91600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1145f0 con 0x7f3a8c1093e0 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.703+0000 7f3a8be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8c104310 0x7f3a8c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.703+0000 7f3a8b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c1093e0 0x7f3a8c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.703+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8c19a7b0 0x7f3a8c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.703+0000 7f3a8be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8c104310 0x7f3a8c1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:48934/0 (socket says 172.21.15.83:48934) 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.703+0000 7f3a8be00640 1 -- 172.21.15.83:0/1785014603 learned_addr learned my addr 172.21.15.83:0/1785014603 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.704+0000 7f3a8be00640 1 -- 172.21.15.83:0/1785014603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c1093e0 msgr2=0x7f3a8c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.704+0000 7f3a8be00640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c1093e0 0x7f3a8c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.704+0000 7f3a8be00640 1 -- 172.21.15.83:0/1785014603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8c19a7b0 msgr2=0x7f3a8c19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.704+0000 7f3a8be00640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8c19a7b0 0x7f3a8c19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.704+0000 7f3a8be00640 1 -- 172.21.15.83:0/1785014603 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c19b340 con 0x7f3a8c104310 2024-04-23T14:50:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.704+0000 7f3a8b400640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c1093e0 0x7f3a8c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.704+0000 7f3a90c00640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8c19a7b0 0x7f3a8c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.704+0000 7f3a8be00640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8c104310 0x7f3a8c1a0640 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c006010 tx=0x7f3a7c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.704+0000 7f3a88c00640 1 -- 172.21.15.83:0/1785014603 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c03f030 con 0x7f3a8c104310 2024-04-23T14:50:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.704+0000 7f3a91600640 1 -- 172.21.15.83:0/1785014603 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8c19b5d0 con 0x7f3a8c104310 2024-04-23T14:50:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.704+0000 7f3a88c00640 1 -- 172.21.15.83:0/1785014603 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a7c004440 con 0x7f3a8c104310 2024-04-23T14:50:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.704+0000 7f3a88c00640 1 -- 172.21.15.83:0/1785014603 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c035070 con 0x7f3a8c104310 2024-04-23T14:50:14.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.705+0000 7f3a91600640 1 -- 172.21.15.83:0/1785014603 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8c1a77b0 con 0x7f3a8c104310 2024-04-23T14:50:14.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.707+0000 7f3a88c00640 1 -- 172.21.15.83:0/1785014603 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a7c042030 con 0x7f3a8c104310 2024-04-23T14:50:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.708+0000 7f3a959ed740 1 Processor -- start 2024-04-23T14:50:14.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.708+0000 7f3a959ed740 1 -- start start 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.708+0000 7f3a959ed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fa7b40 0x55f195fa7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.708+0000 7f3a959ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195fb25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.708+0000 7f3a959ed740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fb2b30 0x55f195fb4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.708+0000 7f3a959ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f195f0d1d0 con 0x55f195fa8460 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.708+0000 7f3a959ed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f195eb3450 con 0x55f195fa7b40 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.708+0000 7f3a959ed740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f195f09190 con 0x55f195fb2b30 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fa7b40 0x55f195fa7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fa7b40 0x55f195fa7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:48936/0 (socket says 172.21.15.83:48936) 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a90c00640 1 -- 172.21.15.83:0/2912060737 learned_addr learned my addr 172.21.15.83:0/2912060737 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a8b400640 1 --2- 172.21.15.83:0/2912060737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fb2b30 0x55f195fb4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a8be00640 1 --2- 172.21.15.83:0/2912060737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195fb25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a90c00640 1 -- 172.21.15.83:0/2912060737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fb2b30 msgr2=0x55f195fb4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a90c00640 1 --2- 172.21.15.83:0/2912060737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fb2b30 0x55f195fb4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a90c00640 1 -- 172.21.15.83:0/2912060737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 msgr2=0x55f195fb25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a90c00640 1 --2- 172.21.15.83:0/2912060737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195fb25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a90c00640 1 -- 172.21.15.83:0/2912060737 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f195f70310 con 0x55f195fa7b40 2024-04-23T14:50:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a8be00640 1 --2- 172.21.15.83:0/2912060737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195fb25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:50:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a90c00640 1 --2- 172.21.15.83:0/2912060737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fa7b40 0x55f195fa7f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3a84011790 tx=0x7f3a84011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c2bfbb9468b398 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a88c00640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f3a700a3da0 0x7f3a700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.709+0000 7f3a81200640 1 -- 172.21.15.83:0/2912060737 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84003c30 con 0x55f195fa7b40 2024-04-23T14:50:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.710+0000 7f3a81200640 1 -- 172.21.15.83:0/2912060737 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8400fe80 con 0x55f195fa7b40 2024-04-23T14:50:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.710+0000 7f3a81200640 1 -- 172.21.15.83:0/2912060737 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840193e0 con 0x55f195fa7b40 2024-04-23T14:50:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.710+0000 7f3a88c00640 1 -- 172.21.15.83:0/1785014603 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3a7c0f0b20 con 0x7f3a8c104310 2024-04-23T14:50:14.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.710+0000 7f3a8be00640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f3a700a3da0 0x7f3a700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.710+0000 7f3a959ed740 1 -- 172.21.15.83:0/2912060737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fa7b40 msgr2=0x55f195fa7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.710+0000 7f3a959ed740 1 --2- 172.21.15.83:0/2912060737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fa7b40 0x55f195fa7f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3a84011790 tx=0x7f3a84011c50 comp rx=0 tx=0).stop 2024-04-23T14:50:14.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.710+0000 7f3a959ed740 1 -- 172.21.15.83:0/2912060737 shutdown_connections 2024-04-23T14:50:14.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.710+0000 7f3a959ed740 1 --2- 172.21.15.83:0/2912060737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fb2b30 0x55f195fb4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.710+0000 7f3a959ed740 1 --2- 172.21.15.83:0/2912060737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195fb25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.710+0000 7f3a959ed740 1 --2- 172.21.15.83:0/2912060737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fa7b40 0x55f195fa7f20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.710+0000 7f3a959ed740 1 -- 172.21.15.83:0/2912060737 >> 172.21.15.83:0/2912060737 conn(0x55f195fa3e60 msgr2=0x55f195fa4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.710+0000 7f3a959ed740 1 -- 172.21.15.83:0/2912060737 shutdown_connections 2024-04-23T14:50:14.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.711+0000 7f3a8be00640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f3a700a3da0 0x7f3a700a6260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55f195fa9100 tx=0x7f3a7c02a410 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.711+0000 7f3a959ed740 1 -- 172.21.15.83:0/2912060737 wait complete. 2024-04-23T14:50:14.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.711+0000 7f3a959ed740 1 -- start start 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.711+0000 7f3a959ed740 1 Processor -- start 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a959ed740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 0x55f195fb2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a959ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195faf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a959ed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 0x55f195fafe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a959ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f195fb59c0 con 0x55f195fa8460 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a959ed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f195ec6810 con 0x55f195fb2b30 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a959ed740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f195fb5b40 con 0x55f195fa7b40 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a959ed740 10 client.0 _ll_register_callbacks cb 0x55f195e94140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 0x55f195fb2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 0x55f195fb2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:53986/0 (socket says 172.21.15.83:53986) 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a90c00640 1 -- 172.21.15.83:0/4029404525 learned_addr learned my addr 172.21.15.83:0/4029404525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a90c00640 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 0x55f195fafe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a8b400640 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195faf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a90c00640 1 -- 172.21.15.83:0/4029404525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 msgr2=0x55f195fafe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.712+0000 7f3a90c00640 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 0x55f195fafe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.713+0000 7f3a90c00640 1 -- 172.21.15.83:0/4029404525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 msgr2=0x55f195faf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.713+0000 7f3a90c00640 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195faf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.713+0000 7f3a90c00640 1 -- 172.21.15.83:0/4029404525 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f195ed73f0 con 0x55f195fa7b40 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.713+0000 7f3a8b400640 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195faf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.713+0000 7f3a90c00640 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 0x55f195fb2540 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a84005f40 tx=0x7f3a840045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de5e2fa4f99ea947 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.713+0000 7f3a75600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.713+0000 7f3a75600640 1 -- 172.21.15.83:0/4029404525 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84004320 con 0x55f195fa7b40 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.713+0000 7f3a75600640 1 -- 172.21.15.83:0/4029404525 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3a84011020 con 0x55f195fa7b40 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.713+0000 7f3a959ed740 1 -- 172.21.15.83:0/4029404525 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f195f1abf0 con 0x55f195fa7b40 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.713+0000 7f3a75600640 1 -- 172.21.15.83:0/4029404525 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3a84011220 con 0x55f195fa7b40 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.713+0000 7f3a959ed740 10 client.25708 fetch_fsmap learned FSMap version 25 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.713+0000 7f3a959ed740 1 -- 172.21.15.83:0/4029404525 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f195c1f3f0 con 0x55f195fa7b40 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.714+0000 7f3a75600640 1 -- 172.21.15.83:0/4029404525 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f195f1abf0 con 0x55f195fa7b40 2024-04-23T14:50:14.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.714+0000 7f3a75600640 1 -- 172.21.15.83:0/4029404525 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3a84018070 con 0x55f195fa7b40 2024-04-23T14:50:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.714+0000 7f3a959ed740 10 client.25708 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:50:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.714+0000 7f3a959ed740 10 client.25708 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:50:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.714+0000 7f3a959ed740 20 client.25708 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.714+0000 7f3a959ed740 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f195fc2b70 0x55f195fc4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.714+0000 7f3a959ed740 4 client.25708 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:50:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.714+0000 7f3a959ed740 1 -- 172.21.15.83:0/4029404525 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f195f14a60 con 0x55f195fc2b70 2024-04-23T14:50:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.714+0000 7f3a8b400640 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f195fc2b70 0x55f195fc4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.715+0000 7f3a75600640 10 client.25708 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:50:14.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.715+0000 7f3a8b400640 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f195fc2b70 0x55f195fc4f60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dee779bde238e00e server_cookie=763e3f538867ca2b in_seq=0 out_seq=0 2024-04-23T14:50:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.715+0000 7f3a75600640 1 -- 172.21.15.83:0/4029404525 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f195f14a60 con 0x55f195fc2b70 2024-04-23T14:50:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.715+0000 7f3a75600640 10 client.25708 handle_command_reply: tid=0 2024-04-23T14:50:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.715+0000 7f3a959ed740 1 client.25708 shutdown 2024-04-23T14:50:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.716+0000 7f3a959ed740 1 -- 172.21.15.83:0/4029404525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 msgr2=0x55f195fb2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.716+0000 7f3a959ed740 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 0x55f195fb2540 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a84005f40 tx=0x7f3a840045c0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.716+0000 7f3a959ed740 1 -- 172.21.15.83:0/4029404525 shutdown_connections 2024-04-23T14:50:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.716+0000 7f3a959ed740 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 0x55f195fafe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.716+0000 7f3a959ed740 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195faf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.716+0000 7f3a959ed740 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f195fc2b70 0x55f195fc4f60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.716+0000 7f3a959ed740 1 --2- 172.21.15.83:0/4029404525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 0x55f195fb2540 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.716+0000 7f3a75600640 0 client.25708 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:50:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.716+0000 7f3a959ed740 1 -- 172.21.15.83:0/4029404525 >> 172.21.15.83:0/4029404525 conn(0x55f195fa3e60 msgr2=0x55f195fa9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.716+0000 7f3a959ed740 1 -- 172.21.15.83:0/4029404525 shutdown_connections 2024-04-23T14:50:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.716+0000 7f3a959ed740 1 -- 172.21.15.83:0/4029404525 wait complete. 2024-04-23T14:50:14.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.716+0000 7f3a959ed740 20 client.25708 trim_cache size 0 max 16384 2024-04-23T14:50:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:14 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2771834116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:50:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.732+0000 7f3a959ed740 1 Processor -- start 2024-04-23T14:50:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.732+0000 7f3a959ed740 1 -- start start 2024-04-23T14:50:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.732+0000 7f3a959ed740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 0x55f195fbe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a959ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195fb2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a959ed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 0x55f195fca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a959ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b2660 con 0x55f195fa8460 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a959ed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1aff30 con 0x55f195fb2b30 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a959ed740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1add80 con 0x55f195fa7b40 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 0x55f195fbe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 0x55f195fbe8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:53996/0 (socket says 172.21.15.83:53996) 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a90c00640 1 -- 172.21.15.83:0/2838774186 learned_addr learned my addr 172.21.15.83:0/2838774186 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a90c00640 1 --2- 172.21.15.83:0/2838774186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 0x55f195fca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a8b400640 1 --2- 172.21.15.83:0/2838774186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195fb2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a90c00640 1 -- 172.21.15.83:0/2838774186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 msgr2=0x55f195fca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a90c00640 1 --2- 172.21.15.83:0/2838774186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 0x55f195fca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a90c00640 1 -- 172.21.15.83:0/2838774186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 msgr2=0x55f195fb2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a90c00640 1 --2- 172.21.15.83:0/2838774186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195fb2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.733+0000 7f3a90c00640 1 -- 172.21.15.83:0/2838774186 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f195fb01b0 con 0x55f195fa7b40 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.734+0000 7f3a90c00640 1 --2- 172.21.15.83:0/2838774186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 0x55f195fbe8b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3a84011790 tx=0x7f3a840113c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4727728e410716a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.734+0000 7f3a77e00640 1 -- 172.21.15.83:0/2838774186 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84003ec0 con 0x55f195fa7b40 2024-04-23T14:50:14.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.734+0000 7f3a77e00640 1 -- 172.21.15.83:0/2838774186 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a84005960 con 0x55f195fa7b40 2024-04-23T14:50:14.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.734+0000 7f3a77e00640 1 -- 172.21.15.83:0/2838774186 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84030d50 con 0x55f195fa7b40 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.735+0000 7f3a959ed740 1 -- 172.21.15.83:0/2838774186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 msgr2=0x55f195fbe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.735+0000 7f3a959ed740 1 --2- 172.21.15.83:0/2838774186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 0x55f195fbe8b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3a84011790 tx=0x7f3a840113c0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.735+0000 7f3a959ed740 1 -- 172.21.15.83:0/2838774186 shutdown_connections 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.735+0000 7f3a959ed740 1 --2- 172.21.15.83:0/2838774186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 0x55f195fca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.735+0000 7f3a959ed740 1 --2- 172.21.15.83:0/2838774186 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa8460 0x55f195fb2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.735+0000 7f3a959ed740 1 --2- 172.21.15.83:0/2838774186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa7b40 0x55f195fbe8b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.735+0000 7f3a959ed740 1 -- 172.21.15.83:0/2838774186 >> 172.21.15.83:0/2838774186 conn(0x55f195fa3e60 msgr2=0x55f195fa2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.735+0000 7f3a959ed740 1 -- 172.21.15.83:0/2838774186 shutdown_connections 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.735+0000 7f3a959ed740 1 -- 172.21.15.83:0/2838774186 wait complete. 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.735+0000 7f3a959ed740 1 -- start start 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.735+0000 7f3a959ed740 1 Processor -- start 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a959ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa7b40 0x55f195fbadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a959ed740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa8460 0x55f195fbb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a959ed740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 0x55f195facad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a959ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1ae3f0 con 0x55f195fa7b40 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a959ed740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1af860 con 0x55f195fb2b30 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a959ed740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1aeac0 con 0x55f195fa8460 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a959ed740 10 client.0 _ll_register_callbacks cb 0x7f3a78009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa7b40 0x55f195fbadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa7b40 0x55f195fbadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37010/0 (socket says 172.21.15.83:37010) 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a90c00640 1 -- 172.21.15.83:0/3068865580 learned_addr learned my addr 172.21.15.83:0/3068865580 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a8b400640 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa8460 0x55f195fbb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a90c00640 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 0x55f195facad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a90c00640 1 -- 172.21.15.83:0/3068865580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa8460 msgr2=0x55f195fbb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a90c00640 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa8460 0x55f195fbb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.736+0000 7f3a90c00640 1 -- 172.21.15.83:0/3068865580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 msgr2=0x55f195facad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.737+0000 7f3a90c00640 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 0x55f195facad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.737+0000 7f3a90c00640 1 -- 172.21.15.83:0/3068865580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f195ce97d0 con 0x55f195fa7b40 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.737+0000 7f3a90c00640 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa7b40 0x55f195fbadc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400f850 tx=0x7f3a84003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7488df9bb9fffbe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.737+0000 7f3a75600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.737+0000 7f3a75600640 1 -- 172.21.15.83:0/3068865580 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8401f040 con 0x55f195fa7b40 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.737+0000 7f3a75600640 1 -- 172.21.15.83:0/3068865580 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3a84030cf0 con 0x55f195fa7b40 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.737+0000 7f3a959ed740 1 -- 172.21.15.83:0/3068865580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a84005960 con 0x55f195fa7b40 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.737+0000 7f3a75600640 1 -- 172.21.15.83:0/3068865580 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3a84003ec0 con 0x55f195fa7b40 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.737+0000 7f3a959ed740 10 client.15624 fetch_fsmap learned FSMap version 25 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.737+0000 7f3a959ed740 1 -- 172.21.15.83:0/3068865580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f195fb3980 con 0x55f195fa7b40 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.738+0000 7f3a75600640 1 -- 172.21.15.83:0/3068865580 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a84005960 con 0x55f195fa7b40 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.738+0000 7f3a75600640 1 -- 172.21.15.83:0/3068865580 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3a84018070 con 0x55f195fa7b40 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.738+0000 7f3a959ed740 10 client.15624 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.738+0000 7f3a959ed740 10 client.15624 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.738+0000 7f3a959ed740 20 client.15624 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.738+0000 7f3a959ed740 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f196028fe0 0x55f19602b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.738+0000 7f3a959ed740 4 client.15624 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:50:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.738+0000 7f3a959ed740 1 -- 172.21.15.83:0/3068865580 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f195e904d0 con 0x55f196028fe0 2024-04-23T14:50:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.738+0000 7f3a8b400640 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f196028fe0 0x55f19602b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a8b400640 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f196028fe0 0x55f19602b3d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1fcb7eec52c61821 server_cookie=2d23d41061e06fdf in_seq=0 out_seq=0 2024-04-23T14:50:14.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a75600640 10 client.15624 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:50:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a75600640 1 -- 172.21.15.83:0/3068865580 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f195e904d0 con 0x55f196028fe0 2024-04-23T14:50:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a75600640 10 client.15624 handle_command_reply: tid=0 2024-04-23T14:50:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a959ed740 1 client.15624 shutdown 2024-04-23T14:50:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a959ed740 1 -- 172.21.15.83:0/3068865580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa7b40 msgr2=0x55f195fbadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a959ed740 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa7b40 0x55f195fbadc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400f850 tx=0x7f3a84003c40 comp rx=0 tx=0).stop 2024-04-23T14:50:14.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a959ed740 1 -- 172.21.15.83:0/3068865580 shutdown_connections 2024-04-23T14:50:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a959ed740 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f195fb2b30 0x55f195facad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a959ed740 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f196028fe0 0x55f19602b3d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a959ed740 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f195fa8460 0x55f195fbb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a75600640 0 client.15624 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:50:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a959ed740 1 --2- 172.21.15.83:0/3068865580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f195fa7b40 0x55f195fbadc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.739+0000 7f3a959ed740 1 -- 172.21.15.83:0/3068865580 >> 172.21.15.83:0/3068865580 conn(0x55f195fa3e60 msgr2=0x55f195fadfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.740+0000 7f3a959ed740 1 -- 172.21.15.83:0/3068865580 shutdown_connections 2024-04-23T14:50:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.740+0000 7f3a959ed740 1 -- 172.21.15.83:0/3068865580 wait complete. 2024-04-23T14:50:14.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.740+0000 7f3a959ed740 20 client.15624 trim_cache size 0 max 16384 2024-04-23T14:50:14.745 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:50:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.740+0000 7f3a91600640 1 -- 172.21.15.83:0/1785014603 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f3a700a3da0 msgr2=0x7f3a700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.740+0000 7f3a91600640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f3a700a3da0 0x7f3a700a6260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55f195fa9100 tx=0x7f3a7c02a410 comp rx=0 tx=0).stop 2024-04-23T14:50:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.740+0000 7f3a91600640 1 -- 172.21.15.83:0/1785014603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8c104310 msgr2=0x7f3a8c1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.740+0000 7f3a91600640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8c104310 0x7f3a8c1a0640 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c006010 tx=0x7f3a7c004040 comp rx=0 tx=0).stop 2024-04-23T14:50:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.741+0000 7f3a91600640 1 -- 172.21.15.83:0/1785014603 shutdown_connections 2024-04-23T14:50:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.741+0000 7f3a91600640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f3a700a3da0 0x7f3a700a6260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.741+0000 7f3a91600640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3a8c19a7b0 0x7f3a8c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.741+0000 7f3a91600640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c1093e0 0x7f3a8c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.741+0000 7f3a91600640 1 --2- 172.21.15.83:0/1785014603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8c104310 0x7f3a8c1a0640 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.741+0000 7f3a91600640 1 -- 172.21.15.83:0/1785014603 >> 172.21.15.83:0/1785014603 conn(0x7f3a8c0ffb70 msgr2=0x7f3a8c100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.741+0000 7f3a91600640 1 -- 172.21.15.83:0/1785014603 shutdown_connections 2024-04-23T14:50:14.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.741+0000 7f3a91600640 1 -- 172.21.15.83:0/1785014603 wait complete. 2024-04-23T14:50:14.759 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:50:14.811 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/stat.h... yes 2024-04-23T14:50:14.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:14 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2771834116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:50:14.906 INFO:tasks.workunit.client.0.smithi083.stdout:checking for stdlib.h... yes 2024-04-23T14:50:14.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.936+0000 7fa7df800640 1 -- 172.21.15.83:0/4268819860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d8102bd0 msgr2=0x7fa7d8106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.936+0000 7fa7df800640 1 --2- 172.21.15.83:0/4268819860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d8102bd0 0x7fa7d8106c10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009900 tx=0x7fa7c402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.936+0000 7fa7df800640 1 -- 172.21.15.83:0/4268819860 shutdown_connections 2024-04-23T14:50:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.936+0000 7fa7df800640 1 --2- 172.21.15.83:0/4268819860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7d81072f0 0x7fa7d810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.936+0000 7fa7df800640 1 --2- 172.21.15.83:0/4268819860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d8102bd0 0x7fa7d8106c10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.936+0000 7fa7df800640 1 --2- 172.21.15.83:0/4268819860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7d8102220 0x7fa7d8102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.936+0000 7fa7df800640 1 -- 172.21.15.83:0/4268819860 >> 172.21.15.83:0/4268819860 conn(0x7fa7d80fda80 msgr2=0x7fa7d80ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.936+0000 7fa7df800640 1 -- 172.21.15.83:0/4268819860 shutdown_connections 2024-04-23T14:50:14.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.936+0000 7fa7df800640 1 -- 172.21.15.83:0/4268819860 wait complete. 2024-04-23T14:50:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.937+0000 7fa7df800640 1 Processor -- start 2024-04-23T14:50:14.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.938+0000 7fa7df800640 1 -- start start 2024-04-23T14:50:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.938+0000 7fa7df800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d8102220 0x7fa7d81a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.938+0000 7fa7df800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7d81072f0 0x7fa7d81a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.938+0000 7fa7df800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7d819a950 0x7fa7d819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.938+0000 7fa7df800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81123b0 con 0x7fa7d819a950 2024-04-23T14:50:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.938+0000 7fa7df800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d8112230 con 0x7fa7d81072f0 2024-04-23T14:50:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.938+0000 7fa7df800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d8112530 con 0x7fa7d8102220 2024-04-23T14:50:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.938+0000 7fa7dd800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7d819a950 0x7fa7d819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.938+0000 7fa7dce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d8102220 0x7fa7d81a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.938+0000 7fa7dce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d8102220 0x7fa7d81a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:54150/0 (socket says 172.21.15.83:54150) 2024-04-23T14:50:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.938+0000 7fa7dce00640 1 -- 172.21.15.83:0/947127621 learned_addr learned my addr 172.21.15.83:0/947127621 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.938+0000 7fa7d7e00640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7d81072f0 0x7fa7d81a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.939+0000 7fa7dce00640 1 -- 172.21.15.83:0/947127621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7d81072f0 msgr2=0x7fa7d81a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.939+0000 7fa7dce00640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7d81072f0 0x7fa7d81a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.939+0000 7fa7dce00640 1 -- 172.21.15.83:0/947127621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7d819a950 msgr2=0x7fa7d819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.939+0000 7fa7dce00640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7d819a950 0x7fa7d819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.939+0000 7fa7dce00640 1 -- 172.21.15.83:0/947127621 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d819b470 con 0x7fa7d8102220 2024-04-23T14:50:14.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.939+0000 7fa7dce00640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d8102220 0x7fa7d81a07a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00e9f0 tx=0x7fa7cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.939+0000 7fa7d5600640 1 -- 172.21.15.83:0/947127621 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00cd80 con 0x7fa7d8102220 2024-04-23T14:50:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.939+0000 7fa7d5600640 1 -- 172.21.15.83:0/947127621 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7cc010910 con 0x7fa7d8102220 2024-04-23T14:50:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.939+0000 7fa7df800640 1 -- 172.21.15.83:0/947127621 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d819b760 con 0x7fa7d8102220 2024-04-23T14:50:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.939+0000 7fa7d5600640 1 -- 172.21.15.83:0/947127621 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc018990 con 0x7fa7d8102220 2024-04-23T14:50:14.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.940+0000 7fa7df800640 1 -- 172.21.15.83:0/947127621 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d81a7a00 con 0x7fa7d8102220 2024-04-23T14:50:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.944+0000 7fa7e25f7740 1 Processor -- start 2024-04-23T14:50:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.944+0000 7fa7e25f7740 1 -- start start 2024-04-23T14:50:14.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.944+0000 7fa7d5600640 1 -- 172.21.15.83:0/947127621 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa7cc010ac0 con 0x7fa7d8102220 2024-04-23T14:50:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.944+0000 7fa7e25f7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35df0950 0x55ac35df0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.944+0000 7fa7e25f7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df1270 0x55ac35dfb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.944+0000 7fa7e25f7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35dfb940 0x55ac35dfdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.944+0000 7fa7e25f7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ac35d561d0 con 0x55ac35df0950 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.944+0000 7fa7e25f7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ac35cfc450 con 0x55ac35dfb940 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.944+0000 7fa7e25f7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ac35d52190 con 0x55ac35df1270 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.945+0000 7fa7dce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df1270 0x55ac35dfb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.945+0000 7fa7dd800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35df0950 0x55ac35df0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.945+0000 7fa7dd800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35df0950 0x55ac35df0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60966/0 (socket says 172.21.15.83:60966) 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.945+0000 7fa7dd800640 1 -- 172.21.15.83:0/120339905 learned_addr learned my addr 172.21.15.83:0/120339905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.945+0000 7fa7dce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df1270 0x55ac35dfb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:54160/0 (socket says 172.21.15.83:54160) 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.945+0000 7fa7d7e00640 1 --2- 172.21.15.83:0/120339905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35dfb940 0x55ac35dfdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.945+0000 7fa7dce00640 1 -- 172.21.15.83:0/120339905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35dfb940 msgr2=0x55ac35dfdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.945+0000 7fa7dce00640 1 --2- 172.21.15.83:0/120339905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35dfb940 0x55ac35dfdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.945+0000 7fa7dce00640 1 -- 172.21.15.83:0/120339905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35df0950 msgr2=0x55ac35df0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.945+0000 7fa7dce00640 1 --2- 172.21.15.83:0/120339905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35df0950 0x55ac35df0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.945+0000 7fa7dce00640 1 -- 172.21.15.83:0/120339905 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac35db9310 con 0x55ac35df1270 2024-04-23T14:50:14.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.945+0000 7fa7dd800640 1 --2- 172.21.15.83:0/120339905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35df0950 0x55ac35df0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.946+0000 7fa7dce00640 1 --2- 172.21.15.83:0/120339905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df1270 0x55ac35dfb400 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0ce1d0 tx=0x7fa7cc0d9590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcd251a774998f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.946+0000 7fa7d5600640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa7c00a3da0 0x7fa7c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.946+0000 7fa7ca000640 1 -- 172.21.15.83:0/120339905 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0de640 con 0x55ac35df1270 2024-04-23T14:50:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.946+0000 7fa7ca000640 1 -- 172.21.15.83:0/120339905 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7cc0d4e20 con 0x55ac35df1270 2024-04-23T14:50:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.946+0000 7fa7ca000640 1 -- 172.21.15.83:0/120339905 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0c2610 con 0x55ac35df1270 2024-04-23T14:50:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.946+0000 7fa7d5600640 1 -- 172.21.15.83:0/947127621 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa7cc0d0ad0 con 0x7fa7d8102220 2024-04-23T14:50:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.946+0000 7fa7dd800640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa7c00a3da0 0x7fa7c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.947+0000 7fa7e25f7740 1 -- 172.21.15.83:0/120339905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df1270 msgr2=0x55ac35dfb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.947+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/120339905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df1270 0x55ac35dfb400 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0ce1d0 tx=0x7fa7cc0d9590 comp rx=0 tx=0).stop 2024-04-23T14:50:14.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.947+0000 7fa7e25f7740 1 -- 172.21.15.83:0/120339905 shutdown_connections 2024-04-23T14:50:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.947+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/120339905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35dfb940 0x55ac35dfdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.947+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/120339905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df1270 0x55ac35dfb400 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.947+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/120339905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35df0950 0x55ac35df0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.947+0000 7fa7dd800640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa7c00a3da0 0x7fa7c00a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0005f40 tx=0x7fa7d0009600 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.947+0000 7fa7e25f7740 1 -- 172.21.15.83:0/120339905 >> 172.21.15.83:0/120339905 conn(0x55ac35decc70 msgr2=0x55ac35ded050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.947+0000 7fa7e25f7740 1 -- 172.21.15.83:0/120339905 shutdown_connections 2024-04-23T14:50:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.947+0000 7fa7e25f7740 1 -- 172.21.15.83:0/120339905 wait complete. 2024-04-23T14:50:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.948+0000 7fa7e25f7740 1 -- start start 2024-04-23T14:50:14.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.948+0000 7fa7e25f7740 1 Processor -- start 2024-04-23T14:50:14.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.948+0000 7fa7e25f7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35dfb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.948+0000 7fa7e25f7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35df1270 0x55ac35df8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.948+0000 7fa7e25f7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35dfb940 0x55ac35df8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.948+0000 7fa7e25f7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ac35dfe7d0 con 0x55ac35df1270 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.948+0000 7fa7e25f7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ac35d0f810 con 0x55ac35dfb940 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.948+0000 7fa7e25f7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ac35dfe950 con 0x55ac35df0950 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.948+0000 7fa7e25f7740 10 client.0 _ll_register_callbacks cb 0x55ac35cdd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.949+0000 7fa7dce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35dfb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.949+0000 7fa7d7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35df1270 0x55ac35df8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.949+0000 7fa7dce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35dfb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:54168/0 (socket says 172.21.15.83:54168) 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.949+0000 7fa7dce00640 1 -- 172.21.15.83:0/4010587852 learned_addr learned my addr 172.21.15.83:0/4010587852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.949+0000 7fa7dd800640 1 --2- 172.21.15.83:0/4010587852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35dfb940 0x55ac35df8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.949+0000 7fa7dce00640 1 -- 172.21.15.83:0/4010587852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35dfb940 msgr2=0x55ac35df8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.949+0000 7fa7dce00640 1 --2- 172.21.15.83:0/4010587852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35dfb940 0x55ac35df8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.949+0000 7fa7dce00640 1 -- 172.21.15.83:0/4010587852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35df1270 msgr2=0x55ac35df8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.949+0000 7fa7dce00640 1 --2- 172.21.15.83:0/4010587852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35df1270 0x55ac35df8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.949+0000 7fa7dce00640 1 -- 172.21.15.83:0/4010587852 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac35d203f0 con 0x55ac35df0950 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.949+0000 7fa7dce00640 1 --2- 172.21.15.83:0/4010587852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35dfb350 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0a5160 tx=0x7fa7cc0d0d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=345e4acee38a45da server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.949+0000 7fa7bea00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7bea00640 1 -- 172.21.15.83:0/4010587852 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0c7d90 con 0x55ac35df0950 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7bea00640 1 -- 172.21.15.83:0/4010587852 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa7cc099140 con 0x55ac35df0950 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7e25f7740 1 -- 172.21.15.83:0/4010587852 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7cc0d4e20 con 0x55ac35df0950 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7bea00640 1 -- 172.21.15.83:0/4010587852 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa7cc0c87c0 con 0x55ac35df0950 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7e25f7740 10 client.25726 fetch_fsmap learned FSMap version 25 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7e25f7740 1 -- 172.21.15.83:0/4010587852 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac35a683f0 con 0x55ac35df0950 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7bea00640 1 -- 172.21.15.83:0/4010587852 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7cc0d4e20 con 0x55ac35df0950 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7bea00640 1 -- 172.21.15.83:0/4010587852 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa7cc0ce450 con 0x55ac35df0950 2024-04-23T14:50:14.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7e25f7740 10 client.25726 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7e25f7740 10 client.25726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7e25f7740 20 client.25726 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/4010587852 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ac35e0b980 0x55ac35e0dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7e25f7740 4 client.25726 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.950+0000 7fa7e25f7740 1 -- 172.21.15.83:0/4010587852 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac35d63bf0 con 0x55ac35e0b980 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.951+0000 7fa7d7e00640 1 --2- 172.21.15.83:0/4010587852 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ac35e0b980 0x55ac35e0dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.951+0000 7fa7d7e00640 1 --2- 172.21.15.83:0/4010587852 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ac35e0b980 0x55ac35e0dd70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e194a9c4c7dc59d3 server_cookie=984953060ab0797c in_seq=0 out_seq=0 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.951+0000 7fa7bea00640 10 client.25726 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7bea00640 1 -- 172.21.15.83:0/4010587852 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac35d63bf0 con 0x55ac35e0b980 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7bea00640 10 client.25726 handle_command_reply: tid=0 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7e25f7740 1 client.25726 shutdown 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7e25f7740 1 -- 172.21.15.83:0/4010587852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 msgr2=0x55ac35dfb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/4010587852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35dfb350 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0a5160 tx=0x7fa7cc0d0d50 comp rx=0 tx=0).stop 2024-04-23T14:50:14.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7e25f7740 1 -- 172.21.15.83:0/4010587852 shutdown_connections 2024-04-23T14:50:14.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/4010587852 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ac35e0b980 0x55ac35e0dd70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/4010587852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35dfb940 0x55ac35df8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/4010587852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35df1270 0x55ac35df8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/4010587852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35dfb350 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7bea00640 0 client.25726 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:14.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7e25f7740 1 -- 172.21.15.83:0/4010587852 >> 172.21.15.83:0/4010587852 conn(0x55ac35decc70 msgr2=0x55ac35df2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7e25f7740 1 -- 172.21.15.83:0/4010587852 shutdown_connections 2024-04-23T14:50:14.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.952+0000 7fa7e25f7740 1 -- 172.21.15.83:0/4010587852 wait complete. 2024-04-23T14:50:14.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.953+0000 7fa7e25f7740 20 client.25726 trim_cache size 0 max 16384 2024-04-23T14:50:14.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.967+0000 7fa7e25f7740 1 Processor -- start 2024-04-23T14:50:14.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.967+0000 7fa7e25f7740 1 -- start start 2024-04-23T14:50:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7e25f7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35df2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7e25f7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35df1270 0x55ac35dfae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7e25f7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35dfb940 0x55ac35e13340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7e25f7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81b2910 con 0x55ac35dfb940 2024-04-23T14:50:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7e25f7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81b01e0 con 0x55ac35df1270 2024-04-23T14:50:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7e25f7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81ae030 con 0x55ac35df0950 2024-04-23T14:50:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7dce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35df2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7dce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35df2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:54184/0 (socket says 172.21.15.83:54184) 2024-04-23T14:50:14.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7dce00640 1 -- 172.21.15.83:0/3940912255 learned_addr learned my addr 172.21.15.83:0/3940912255 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7dd800640 1 --2- 172.21.15.83:0/3940912255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35dfb940 0x55ac35e13340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7d7e00640 1 --2- 172.21.15.83:0/3940912255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35df1270 0x55ac35dfae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7dce00640 1 -- 172.21.15.83:0/3940912255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35df1270 msgr2=0x55ac35dfae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7dce00640 1 --2- 172.21.15.83:0/3940912255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35df1270 0x55ac35dfae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7dce00640 1 -- 172.21.15.83:0/3940912255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35dfb940 msgr2=0x55ac35e13340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7dce00640 1 --2- 172.21.15.83:0/3940912255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35dfb940 0x55ac35e13340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.968+0000 7fa7dce00640 1 -- 172.21.15.83:0/3940912255 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac35b327d0 con 0x55ac35df0950 2024-04-23T14:50:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.969+0000 7fa7dce00640 1 --2- 172.21.15.83:0/3940912255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35df2e90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc09c7d0 tx=0x7fa7cc0d9590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75cc9968bfa6d46e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.969+0000 7fa7c9600640 1 -- 172.21.15.83:0/3940912255 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0c8d40 con 0x55ac35df0950 2024-04-23T14:50:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.969+0000 7fa7c9600640 1 -- 172.21.15.83:0/3940912255 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7cc0d4e20 con 0x55ac35df0950 2024-04-23T14:50:14.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.969+0000 7fa7c9600640 1 -- 172.21.15.83:0/3940912255 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc0c6e80 con 0x55ac35df0950 2024-04-23T14:50:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.970+0000 7fa7e25f7740 1 -- 172.21.15.83:0/3940912255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 msgr2=0x55ac35df2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.970+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/3940912255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35df2e90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc09c7d0 tx=0x7fa7cc0d9590 comp rx=0 tx=0).stop 2024-04-23T14:50:14.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.970+0000 7fa7e25f7740 1 -- 172.21.15.83:0/3940912255 shutdown_connections 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.970+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/3940912255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35dfb940 0x55ac35e13340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.970+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/3940912255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35df1270 0x55ac35dfae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.970+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/3940912255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35df2e90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.970+0000 7fa7e25f7740 1 -- 172.21.15.83:0/3940912255 >> 172.21.15.83:0/3940912255 conn(0x55ac35decc70 msgr2=0x55ac35debdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.970+0000 7fa7e25f7740 1 -- 172.21.15.83:0/3940912255 shutdown_connections 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.970+0000 7fa7e25f7740 1 -- 172.21.15.83:0/3940912255 wait complete. 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.970+0000 7fa7e25f7740 1 -- start start 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7e25f7740 1 Processor -- start 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7e25f7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35df5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7e25f7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35df1270 0x55ac35df61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7e25f7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35dfb940 0x55ac35dfc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7e25f7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81ae6a0 con 0x55ac35dfb940 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7e25f7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81afb10 con 0x55ac35df1270 2024-04-23T14:50:14.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7e25f7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d81aed70 con 0x55ac35df0950 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7e25f7740 10 client.0 _ll_register_callbacks cb 0x7fa7d0009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7dd800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35dfb940 0x55ac35dfc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7dd800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35dfb940 0x55ac35dfc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:60986/0 (socket says 172.21.15.83:60986) 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7d7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35df1270 0x55ac35df61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7dd800640 1 -- 172.21.15.83:0/1260369576 learned_addr learned my addr 172.21.15.83:0/1260369576 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.971+0000 7fa7dce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35df5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.972+0000 7fa7d7e00640 1 -- 172.21.15.83:0/1260369576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 msgr2=0x55ac35df5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.972+0000 7fa7d7e00640 1 --2- 172.21.15.83:0/1260369576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35df5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.972+0000 7fa7d7e00640 1 -- 172.21.15.83:0/1260369576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35dfb940 msgr2=0x55ac35dfc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.972+0000 7fa7d7e00640 1 --2- 172.21.15.83:0/1260369576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35dfb940 0x55ac35dfc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.972+0000 7fa7d7e00640 1 -- 172.21.15.83:0/1260369576 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac35e04470 con 0x55ac35df1270 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.972+0000 7fa7dce00640 1 --2- 172.21.15.83:0/1260369576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35df5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.972+0000 7fa7d7e00640 1 --2- 172.21.15.83:0/1260369576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35df1270 0x55ac35df61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa7c402a910 tx=0x7fa7c4004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a24e180fe0e596a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.972+0000 7fa7bea00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:50:14.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.972+0000 7fa7bea00640 1 -- 172.21.15.83:0/1260369576 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c404a070 con 0x55ac35df1270 2024-04-23T14:50:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.972+0000 7fa7bea00640 1 -- 172.21.15.83:0/1260369576 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa7c4005040 con 0x55ac35df1270 2024-04-23T14:50:14.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.972+0000 7fa7e25f7740 1 -- 172.21.15.83:0/1260369576 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7cc0d4e20 con 0x55ac35df1270 2024-04-23T14:50:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.973+0000 7fa7bea00640 1 -- 172.21.15.83:0/1260369576 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa7c4056050 con 0x55ac35df1270 2024-04-23T14:50:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.973+0000 7fa7e25f7740 10 client.25763 fetch_fsmap learned FSMap version 25 2024-04-23T14:50:14.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.973+0000 7fa7e25f7740 1 -- 172.21.15.83:0/1260369576 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac35e72180 con 0x55ac35df1270 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.973+0000 7fa7bea00640 1 -- 172.21.15.83:0/1260369576 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7cc0d4e20 con 0x55ac35df1270 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.973+0000 7fa7bea00640 1 -- 172.21.15.83:0/1260369576 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa7c40046c0 con 0x55ac35df1270 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.973+0000 7fa7e25f7740 10 client.25763 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.973+0000 7fa7e25f7740 10 client.25763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.973+0000 7fa7e25f7740 20 client.25763 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.973+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/1260369576 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ac35e72930 0x55ac35e74d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.973+0000 7fa7e25f7740 4 client.25763 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.973+0000 7fa7e25f7740 1 -- 172.21.15.83:0/1260369576 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac35d5da60 con 0x55ac35e72930 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.974+0000 7fa7dce00640 1 --2- 172.21.15.83:0/1260369576 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ac35e72930 0x55ac35e74d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.974+0000 7fa7dce00640 1 --2- 172.21.15.83:0/1260369576 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ac35e72930 0x55ac35e74d50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc86c072f3724f68 server_cookie=78cdc9cb39c13b7c in_seq=0 out_seq=0 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.974+0000 7fa7bea00640 10 client.25763 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.974+0000 7fa7bea00640 1 -- 172.21.15.83:0/1260369576 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac35d5da60 con 0x55ac35e72930 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.974+0000 7fa7bea00640 10 client.25763 handle_command_reply: tid=0 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.974+0000 7fa7e25f7740 1 client.25763 shutdown 2024-04-23T14:50:14.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.975+0000 7fa7e25f7740 1 -- 172.21.15.83:0/1260369576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35df1270 msgr2=0x55ac35df61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.975+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/1260369576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35df1270 0x55ac35df61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa7c402a910 tx=0x7fa7c4004180 comp rx=0 tx=0).stop 2024-04-23T14:50:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.975+0000 7fa7e25f7740 1 -- 172.21.15.83:0/1260369576 shutdown_connections 2024-04-23T14:50:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.975+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/1260369576 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ac35e72930 0x55ac35e74d50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.975+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/1260369576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ac35dfb940 0x55ac35dfc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.975+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/1260369576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ac35df1270 0x55ac35df61e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.975+0000 7fa7bea00640 0 client.25763 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.975+0000 7fa7e25f7740 1 --2- 172.21.15.83:0/1260369576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ac35df0950 0x55ac35df5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.975+0000 7fa7e25f7740 1 -- 172.21.15.83:0/1260369576 >> 172.21.15.83:0/1260369576 conn(0x55ac35decc70 msgr2=0x55ac35df6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.975+0000 7fa7e25f7740 1 -- 172.21.15.83:0/1260369576 shutdown_connections 2024-04-23T14:50:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.975+0000 7fa7e25f7740 1 -- 172.21.15.83:0/1260369576 wait complete. 2024-04-23T14:50:14.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.975+0000 7fa7e25f7740 20 client.25763 trim_cache size 0 max 16384 2024-04-23T14:50:14.978 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:50:14.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.976+0000 7fa7df800640 1 -- 172.21.15.83:0/947127621 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa7c00a3da0 msgr2=0x7fa7c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.976+0000 7fa7df800640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa7c00a3da0 0x7fa7c00a6260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0005f40 tx=0x7fa7d0009600 comp rx=0 tx=0).stop 2024-04-23T14:50:14.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.976+0000 7fa7df800640 1 -- 172.21.15.83:0/947127621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d8102220 msgr2=0x7fa7d81a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:14.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.976+0000 7fa7df800640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d8102220 0x7fa7d81a07a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00e9f0 tx=0x7fa7cc00eeb0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.976+0000 7fa7df800640 1 -- 172.21.15.83:0/947127621 shutdown_connections 2024-04-23T14:50:14.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.976+0000 7fa7df800640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa7c00a3da0 0x7fa7c00a6260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.976+0000 7fa7df800640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa7d819a950 0x7fa7d819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.976+0000 7fa7df800640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa7d81072f0 0x7fa7d81a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.976+0000 7fa7df800640 1 --2- 172.21.15.83:0/947127621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d8102220 0x7fa7d81a07a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.976+0000 7fa7df800640 1 -- 172.21.15.83:0/947127621 >> 172.21.15.83:0/947127621 conn(0x7fa7d80fda80 msgr2=0x7fa7d80ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.976+0000 7fa7df800640 1 -- 172.21.15.83:0/947127621 shutdown_connections 2024-04-23T14:50:14.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:14.976+0000 7fa7df800640 1 -- 172.21.15.83:0/947127621 wait complete. 2024-04-23T14:50:15.009 INFO:tasks.workunit.client.0.smithi083.stdout:checking for string.h... yes 2024-04-23T14:50:15.119 INFO:tasks.workunit.client.0.smithi083.stdout:checking for memory.h... yes 2024-04-23T14:50:15.228 INFO:tasks.workunit.client.0.smithi083.stdout:checking for strings.h... yes 2024-04-23T14:50:15.380 INFO:tasks.workunit.client.0.smithi083.stdout:checking for inttypes.h... yes 2024-04-23T14:50:15.496 INFO:tasks.workunit.client.0.smithi083.stdout:checking for stdint.h... yes 2024-04-23T14:50:15.608 INFO:tasks.workunit.client.0.smithi083.stdout:checking for unistd.h... yes 2024-04-23T14:50:15.726 INFO:tasks.workunit.client.0.smithi083.stdout:checking pthread.h usability... yes 2024-04-23T14:50:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:15 smithi083 ceph-mon[27867]: pgmap v56: 97 pgs: 97 active+clean; 3.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 181 KiB/s wr, 29 op/s 2024-04-23T14:50:15.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:15 smithi083 ceph-mon[27867]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:50:15.785 INFO:tasks.workunit.client.0.smithi083.stdout:checking pthread.h presence... yes 2024-04-23T14:50:15.786 INFO:tasks.workunit.client.0.smithi083.stdout:checking for pthread.h... yes 2024-04-23T14:50:15.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:15 smithi173 ceph-mon[32443]: pgmap v56: 97 pgs: 97 active+clean; 3.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 181 KiB/s wr, 29 op/s 2024-04-23T14:50:15.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:15 smithi173 ceph-mon[32443]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:50:15.909 INFO:tasks.workunit.client.0.smithi083.stdout:checking fcntl.h usability... yes 2024-04-23T14:50:15.968 INFO:tasks.workunit.client.0.smithi083.stdout:checking fcntl.h presence... yes 2024-04-23T14:50:15.971 INFO:tasks.workunit.client.0.smithi083.stdout:checking for fcntl.h... yes 2024-04-23T14:50:15.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:15 smithi203 ceph-mon[32800]: pgmap v56: 97 pgs: 97 active+clean; 3.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 181 KiB/s wr, 29 op/s 2024-04-23T14:50:15.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:15 smithi203 ceph-mon[32800]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:50:15.994 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:50:15.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:50:15.995 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:50:16.084 INFO:tasks.workunit.client.0.smithi083.stdout:checking limits.h usability... yes 2024-04-23T14:50:16.135 INFO:tasks.workunit.client.0.smithi083.stdout:checking limits.h presence... yes 2024-04-23T14:50:16.137 INFO:tasks.workunit.client.0.smithi083.stdout:checking for limits.h... yes 2024-04-23T14:50:16.167 INFO:tasks.workunit.client.0.smithi083.stdout:checking for stdint.h... (cached) yes 2024-04-23T14:50:16.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.186+0000 7fc3f7c00640 1 -- 172.21.15.83:0/57682579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f0109580 msgr2=0x7fc3f0111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.186+0000 7fc3f7c00640 1 --2- 172.21.15.83:0/57682579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f0109580 0x7fc3f0111150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009ca0 tx=0x7fc3e802a8c0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.186+0000 7fc3f7c00640 1 -- 172.21.15.83:0/57682579 shutdown_connections 2024-04-23T14:50:16.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.186+0000 7fc3f7c00640 1 --2- 172.21.15.83:0/57682579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f0109580 0x7fc3f0111150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.186+0000 7fc3f7c00640 1 --2- 172.21.15.83:0/57682579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3f0104e60 0x7fc3f0108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.186+0000 7fc3f7c00640 1 --2- 172.21.15.83:0/57682579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3f01044b0 0x7fc3f0104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.186+0000 7fc3f7c00640 1 -- 172.21.15.83:0/57682579 >> 172.21.15.83:0/57682579 conn(0x7fc3f00ffd90 msgr2=0x7fc3f01021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:16.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.186+0000 7fc3f7c00640 1 -- 172.21.15.83:0/57682579 shutdown_connections 2024-04-23T14:50:16.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.187+0000 7fc3f7c00640 1 -- 172.21.15.83:0/57682579 wait complete. 2024-04-23T14:50:16.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.187+0000 7fc3f7c00640 1 Processor -- start 2024-04-23T14:50:16.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.188+0000 7fc3f7c00640 1 -- start start 2024-04-23T14:50:16.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.188+0000 7fc3f7c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3f01044b0 0x7fc3f01a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.188+0000 7fc3f7c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3f0109580 0x7fc3f01a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.188+0000 7fc3f7c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f019a9d0 0x7fc3f019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.188+0000 7fc3f7c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f0114690 con 0x7fc3f0109580 2024-04-23T14:50:16.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.188+0000 7fc3f7c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f0114510 con 0x7fc3f01044b0 2024-04-23T14:50:16.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.188+0000 7fc3f7c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f0114810 con 0x7fc3f019a9d0 2024-04-23T14:50:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.188+0000 7fc3efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3f0109580 0x7fc3f01a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.188+0000 7fc3efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3f0109580 0x7fc3f01a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32774/0 (socket says 172.21.15.83:32774) 2024-04-23T14:50:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.188+0000 7fc3efe00640 1 -- 172.21.15.83:0/1911018981 learned_addr learned my addr 172.21.15.83:0/1911018981 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.188+0000 7fc3f5200640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3f01044b0 0x7fc3f01a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.189+0000 7fc3f5c00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f019a9d0 0x7fc3f019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.189+0000 7fc3efe00640 1 -- 172.21.15.83:0/1911018981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f019a9d0 msgr2=0x7fc3f019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.189+0000 7fc3efe00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f019a9d0 0x7fc3f019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.189+0000 7fc3efe00640 1 -- 172.21.15.83:0/1911018981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3f01044b0 msgr2=0x7fc3f01a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.189+0000 7fc3efe00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3f01044b0 0x7fc3f01a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.189+0000 7fc3efe00640 1 -- 172.21.15.83:0/1911018981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f019b4f0 con 0x7fc3f0109580 2024-04-23T14:50:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.189+0000 7fc3f5c00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f019a9d0 0x7fc3f019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:50:16.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.189+0000 7fc3efe00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3f0109580 0x7fc3f01a0d10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc005fd0 tx=0x7fc3dc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:16.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.189+0000 7fc3ed600640 1 -- 172.21.15.83:0/1911018981 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc017070 con 0x7fc3f0109580 2024-04-23T14:50:16.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.189+0000 7fc3ed600640 1 -- 172.21.15.83:0/1911018981 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3dc009c30 con 0x7fc3f0109580 2024-04-23T14:50:16.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.189+0000 7fc3f7c00640 1 -- 172.21.15.83:0/1911018981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f019b7e0 con 0x7fc3f0109580 2024-04-23T14:50:16.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.189+0000 7fc3ed600640 1 -- 172.21.15.83:0/1911018981 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc010460 con 0x7fc3f0109580 2024-04-23T14:50:16.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.190+0000 7fc3f7c00640 1 -- 172.21.15.83:0/1911018981 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f01a7a30 con 0x7fc3f0109580 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.193+0000 7fc3fa9c7740 1 Processor -- start 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.193+0000 7fc3fa9c7740 1 -- start start 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.193+0000 7fc3fa9c7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634662f6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.193+0000 7fc3fa9c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x563466300b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.193+0000 7fc3fa9c7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x563466303470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.193+0000 7fc3fa9c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56346625be70 con 0x5634662f69b0 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.193+0000 7fc3fa9c7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563466202380 con 0x5634662f6090 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.193+0000 7fc3fa9c7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563466257e30 con 0x563466301080 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.193+0000 7fc3ed600640 1 -- 172.21.15.83:0/1911018981 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3dc003e70 con 0x7fc3f0109580 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.194+0000 7fc3f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x563466300b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.194+0000 7fc3f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x563466300b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32776/0 (socket says 172.21.15.83:32776) 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.194+0000 7fc3f5200640 1 -- 172.21.15.83:0/1753281209 learned_addr learned my addr 172.21.15.83:0/1753281209 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:16.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.194+0000 7fc3f5c00640 1 --2- 172.21.15.83:0/1753281209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634662f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.194+0000 7fc3efe00640 1 --2- 172.21.15.83:0/1753281209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x563466303470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.194+0000 7fc3f5200640 1 -- 172.21.15.83:0/1753281209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 msgr2=0x563466303470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.194+0000 7fc3f5200640 1 --2- 172.21.15.83:0/1753281209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x563466303470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.194+0000 7fc3f5200640 1 -- 172.21.15.83:0/1753281209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 msgr2=0x5634662f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.194+0000 7fc3f5200640 1 --2- 172.21.15.83:0/1753281209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634662f6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.194+0000 7fc3f5200640 1 -- 172.21.15.83:0/1753281209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634660367e0 con 0x5634662f69b0 2024-04-23T14:50:16.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.194+0000 7fc3f5c00640 1 --2- 172.21.15.83:0/1753281209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634662f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:16.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.194+0000 7fc3f5200640 1 --2- 172.21.15.83:0/1753281209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x563466300b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009ca0 tx=0x7fc3e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba66d3a4804049e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:16.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.195+0000 7fc3e2000640 1 -- 172.21.15.83:0/1753281209 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4005220 con 0x5634662f69b0 2024-04-23T14:50:16.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.195+0000 7fc3e2000640 1 -- 172.21.15.83:0/1753281209 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e4004c80 con 0x5634662f69b0 2024-04-23T14:50:16.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.195+0000 7fc3e2000640 1 -- 172.21.15.83:0/1753281209 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40197a0 con 0x5634662f69b0 2024-04-23T14:50:16.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.195+0000 7fc3ed600640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc3d80a3da0 0x7fc3d80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.196+0000 7fc3ed600640 1 -- 172.21.15.83:0/1911018981 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc3dc0d0380 con 0x7fc3f0109580 2024-04-23T14:50:16.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.196+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/1753281209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 msgr2=0x563466300b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.196+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/1753281209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x563466300b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009ca0 tx=0x7fc3e4011810 comp rx=0 tx=0).stop 2024-04-23T14:50:16.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.196+0000 7fc3f5c00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc3d80a3da0 0x7fc3d80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.196+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/1753281209 shutdown_connections 2024-04-23T14:50:16.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.196+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/1753281209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x563466303470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.196+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/1753281209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x563466300b40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.196+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/1753281209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634662f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.196+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/1753281209 >> 172.21.15.83:0/1753281209 conn(0x5634662f23b0 msgr2=0x5634662f2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:16.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.196+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/1753281209 shutdown_connections 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.196+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/1753281209 wait complete. 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.197+0000 7fc3f5c00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc3d80a3da0 0x7fc3d80a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009430 tx=0x7fc3e8004a40 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.197+0000 7fc3fa9c7740 1 -- start start 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.197+0000 7fc3fa9c7740 1 Processor -- start 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.197+0000 7fc3fa9c7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634662fe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.197+0000 7fc3fa9c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x5634662fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.197+0000 7fc3fa9c7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x5634662fea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.197+0000 7fc3fa9c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563466303e70 con 0x5634662f69b0 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.197+0000 7fc3fa9c7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563466215660 con 0x5634662f6090 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.197+0000 7fc3fa9c7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563466303ff0 con 0x563466301080 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.197+0000 7fc3fa9c7740 10 client.0 _ll_register_callbacks cb 0x563466203aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3f5c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x5634662fea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x5634662fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3f5c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x5634662fea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:54228/0 (socket says 172.21.15.83:54228) 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3f5c00640 1 -- 172.21.15.83:0/2724918741 learned_addr learned my addr 172.21.15.83:0/2724918741 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3f5200640 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634662fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3efe00640 1 -- 172.21.15.83:0/2724918741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 msgr2=0x5634662fea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3efe00640 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x5634662fea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3efe00640 1 -- 172.21.15.83:0/2724918741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 msgr2=0x5634662fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3efe00640 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634662fe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3efe00640 1 -- 172.21.15.83:0/2724918741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563466096f70 con 0x5634662f69b0 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3f5c00640 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x5634662fea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3efe00640 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x5634662fe540 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0cdae0 tx=0x7fc3dc09be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6524f7a5279711e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.198+0000 7fc3d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.199+0000 7fc3d6a00640 1 -- 172.21.15.83:0/2724918741 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0cdf60 con 0x5634662f69b0 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.199+0000 7fc3d6a00640 1 -- 172.21.15.83:0/2724918741 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc3dc0a4b10 con 0x5634662f69b0 2024-04-23T14:50:16.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.199+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/2724918741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3e4004c80 con 0x5634662f69b0 2024-04-23T14:50:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.199+0000 7fc3d6a00640 1 -- 172.21.15.83:0/2724918741 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc3dc009de0 con 0x5634662f69b0 2024-04-23T14:50:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.199+0000 7fc3fa9c7740 10 client.15654 fetch_fsmap learned FSMap version 25 2024-04-23T14:50:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.199+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/2724918741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563465f2e170 con 0x5634662f69b0 2024-04-23T14:50:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.199+0000 7fc3d6a00640 1 -- 172.21.15.83:0/2724918741 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e4004c80 con 0x5634662f69b0 2024-04-23T14:50:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.199+0000 7fc3d6a00640 1 -- 172.21.15.83:0/2724918741 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc3dc09b780 con 0x5634662f69b0 2024-04-23T14:50:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.200+0000 7fc3fa9c7740 10 client.15654 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:50:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.200+0000 7fc3fa9c7740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.200+0000 7fc3fa9c7740 20 client.15654 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.200+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5634663113e0 0x5634663137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.200+0000 7fc3fa9c7740 4 client.15654 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:50:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.200+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/2724918741 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563466269890 con 0x5634663113e0 2024-04-23T14:50:16.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.200+0000 7fc3f5200640 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5634663113e0 0x5634663137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.201+0000 7fc3f5200640 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5634663113e0 0x5634663137d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57be9c0ae6f41cc8 server_cookie=c8cfbc9c5993345 in_seq=0 out_seq=0 2024-04-23T14:50:16.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.201+0000 7fc3d6a00640 10 client.15654 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:16.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.201+0000 7fc3d6a00640 1 -- 172.21.15.83:0/2724918741 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563466269890 con 0x5634663113e0 2024-04-23T14:50:16.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.201+0000 7fc3d6a00640 10 client.15654 handle_command_reply: tid=0 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.201+0000 7fc3fa9c7740 1 client.15654 shutdown 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.202+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/2724918741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 msgr2=0x5634662fe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.202+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x5634662fe540 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0cdae0 tx=0x7fc3dc09be70 comp rx=0 tx=0).stop 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.202+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/2724918741 shutdown_connections 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.202+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5634663113e0 0x5634663137d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.202+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x5634662fea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.202+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x5634662fe540 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.202+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/2724918741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634662fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.202+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/2724918741 >> 172.21.15.83:0/2724918741 conn(0x5634662f23b0 msgr2=0x5634662f1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.202+0000 7fc3d6a00640 0 client.15654 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.202+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/2724918741 shutdown_connections 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.202+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/2724918741 wait complete. 2024-04-23T14:50:16.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.202+0000 7fc3fa9c7740 20 client.15654 trim_cache size 0 max 16384 2024-04-23T14:50:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.221+0000 7fc3fa9c7740 1 Processor -- start 2024-04-23T14:50:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.221+0000 7fc3fa9c7740 1 -- start start 2024-04-23T14:50:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3fa9c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f6090 0x56346630cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3fa9c7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f69b0 0x5634662f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3fa9c7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x5634663190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3fa9c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01b2850 con 0x5634662f6090 2024-04-23T14:50:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3fa9c7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01b0120 con 0x5634662f69b0 2024-04-23T14:50:16.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3fa9c7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01adf90 con 0x563466301080 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3f5c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x5634663190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3efe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f69b0 0x5634662f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3f5c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x5634663190c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:54234/0 (socket says 172.21.15.83:54234) 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3f5200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f6090 0x56346630cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3f5c00640 1 -- 172.21.15.83:0/1609929364 learned_addr learned my addr 172.21.15.83:0/1609929364 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3efe00640 1 -- 172.21.15.83:0/1609929364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 msgr2=0x5634663190c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3efe00640 1 --2- 172.21.15.83:0/1609929364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x5634663190c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3efe00640 1 -- 172.21.15.83:0/1609929364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f6090 msgr2=0x56346630cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3efe00640 1 --2- 172.21.15.83:0/1609929364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f6090 0x56346630cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3efe00640 1 -- 172.21.15.83:0/1609929364 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634662f8d30 con 0x5634662f69b0 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.222+0000 7fc3f5c00640 1 --2- 172.21.15.83:0/1609929364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x5634663190c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.223+0000 7fc3efe00640 1 --2- 172.21.15.83:0/1609929364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f69b0 0x5634662f87f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc098790 tx=0x7fc3dc09bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f4088b4ee9dfe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:16.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.223+0000 7fc3e1600640 1 -- 172.21.15.83:0/1609929364 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc09c330 con 0x5634662f69b0 2024-04-23T14:50:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.223+0000 7fc3e1600640 1 -- 172.21.15.83:0/1609929364 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3dc0a06e0 con 0x5634662f69b0 2024-04-23T14:50:16.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.223+0000 7fc3e1600640 1 -- 172.21.15.83:0/1609929364 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0cdd60 con 0x5634662f69b0 2024-04-23T14:50:16.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.224+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/1609929364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f69b0 msgr2=0x5634662f87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.224+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/1609929364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f69b0 0x5634662f87f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc098790 tx=0x7fc3dc09bfa0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.224+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/1609929364 shutdown_connections 2024-04-23T14:50:16.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.224+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/1609929364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x5634663190c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.224+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/1609929364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f69b0 0x5634662f87f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.224+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/1609929364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f6090 0x56346630cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.224+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/1609929364 >> 172.21.15.83:0/1609929364 conn(0x5634662f23b0 msgr2=0x5634662fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.224+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/1609929364 shutdown_connections 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.224+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/1609929364 wait complete. 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.225+0000 7fc3fa9c7740 1 -- start start 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.225+0000 7fc3fa9c7740 1 Processor -- start 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.225+0000 7fc3fa9c7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634663094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.225+0000 7fc3fa9c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x563466304860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.225+0000 7fc3fa9c7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x563466304da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.225+0000 7fc3fa9c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01ae5e0 con 0x5634662f69b0 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.225+0000 7fc3fa9c7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01afa50 con 0x5634662f6090 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.225+0000 7fc3fa9c7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f01aecb0 con 0x563466301080 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.225+0000 7fc3fa9c7740 10 client.0 _ll_register_callbacks cb 0x7fc3dc00df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3efe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x563466304860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3f5200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634663094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3f5200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634663094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:49638/0 (socket says 172.21.15.83:49638) 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3f5200640 1 -- 172.21.15.83:0/516280212 learned_addr learned my addr 172.21.15.83:0/516280212 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3efe00640 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x563466304860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32790/0 (socket says 172.21.15.83:32790) 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3f5c00640 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x563466304da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3efe00640 1 -- 172.21.15.83:0/516280212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 msgr2=0x563466304da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3efe00640 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x563466304da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3efe00640 1 -- 172.21.15.83:0/516280212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 msgr2=0x5634663094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3efe00640 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634663094a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3efe00640 1 -- 172.21.15.83:0/516280212 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563466305440 con 0x5634662f69b0 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3f5200640 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634663094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3efe00640 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x563466304860 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009610 tx=0x7fc3dc0a4dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4481dbb64130559 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.226+0000 7fc3d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3d6a00640 1 -- 172.21.15.83:0/516280212 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0cdf70 con 0x5634662f69b0 2024-04-23T14:50:16.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3d6a00640 1 -- 172.21.15.83:0/516280212 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc3dc004160 con 0x5634662f69b0 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/516280212 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3dc0a06e0 con 0x5634662f69b0 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3d6a00640 1 -- 172.21.15.83:0/516280212 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc3dc0035f0 con 0x5634662f69b0 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3fa9c7740 10 client.15660 fetch_fsmap learned FSMap version 25 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/516280212 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563466377800 con 0x5634662f69b0 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3d6a00640 1 -- 172.21.15.83:0/516280212 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3dc0a06e0 con 0x5634662f69b0 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3d6a00640 1 -- 172.21.15.83:0/516280212 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc3dc09b1f0 con 0x5634662f69b0 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3fa9c7740 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3fa9c7740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3fa9c7740 20 client.15660 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563466377d60 0x56346637a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3fa9c7740 4 client.15660 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.227+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/516280212 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563466263700 con 0x563466377d60 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.228+0000 7fc3f5200640 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563466377d60 0x56346637a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.228+0000 7fc3f5200640 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563466377d60 0x56346637a180 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed8e783a92f4fb4 server_cookie=a38ee58f66c93d4b in_seq=0 out_seq=0 2024-04-23T14:50:16.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.228+0000 7fc3d6a00640 10 client.15660 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.229+0000 7fc3d6a00640 1 -- 172.21.15.83:0/516280212 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563466263700 con 0x563466377d60 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.229+0000 7fc3d6a00640 10 client.15660 handle_command_reply: tid=0 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.229+0000 7fc3fa9c7740 1 client.15660 shutdown 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.229+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/516280212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 msgr2=0x563466304860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.229+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x563466304860 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009610 tx=0x7fc3dc0a4dc0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.230+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/516280212 shutdown_connections 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.230+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563466377d60 0x56346637a180 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.230+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563466301080 0x563466304da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.230+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5634662f69b0 0x563466304860 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.230+0000 7fc3fa9c7740 1 --2- 172.21.15.83:0/516280212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5634662f6090 0x5634663094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.230+0000 7fc3d6a00640 0 client.15660 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.230+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/516280212 >> 172.21.15.83:0/516280212 conn(0x5634662f23b0 msgr2=0x5634662f0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.230+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/516280212 shutdown_connections 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.230+0000 7fc3fa9c7740 1 -- 172.21.15.83:0/516280212 wait complete. 2024-04-23T14:50:16.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.230+0000 7fc3fa9c7740 20 client.15660 trim_cache size 0 max 16384 2024-04-23T14:50:16.234 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:50:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:50:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "6e3cefac-51ff-44c5-9f5b-05c9b613678f", 2024-04-23T14:50:16.234 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:50:16.234 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:50:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.231+0000 7fc3f7c00640 1 -- 172.21.15.83:0/1911018981 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc3d80a3da0 msgr2=0x7fc3d80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.231+0000 7fc3f7c00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc3d80a3da0 0x7fc3d80a6260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009430 tx=0x7fc3e8004a40 comp rx=0 tx=0).stop 2024-04-23T14:50:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.231+0000 7fc3f7c00640 1 -- 172.21.15.83:0/1911018981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3f0109580 msgr2=0x7fc3f01a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.231+0000 7fc3f7c00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3f0109580 0x7fc3f01a0d10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc005fd0 tx=0x7fc3dc00dde0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.231+0000 7fc3f7c00640 1 -- 172.21.15.83:0/1911018981 shutdown_connections 2024-04-23T14:50:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.231+0000 7fc3f7c00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3f019a9d0 0x7fc3f019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.231+0000 7fc3f7c00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc3d80a3da0 0x7fc3d80a6260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.231+0000 7fc3f7c00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc3f0109580 0x7fc3f01a0d10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.231+0000 7fc3f7c00640 1 --2- 172.21.15.83:0/1911018981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3f01044b0 0x7fc3f01a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.231+0000 7fc3f7c00640 1 -- 172.21.15.83:0/1911018981 >> 172.21.15.83:0/1911018981 conn(0x7fc3f00ffd90 msgr2=0x7fc3f0102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:16.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.231+0000 7fc3f7c00640 1 -- 172.21.15.83:0/1911018981 shutdown_connections 2024-04-23T14:50:16.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.231+0000 7fc3f7c00640 1 -- 172.21.15.83:0/1911018981 wait complete. 2024-04-23T14:50:16.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:50:16.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:50:16.261 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/time.h usability... yes 2024-04-23T14:50:16.307 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/time.h presence... yes 2024-04-23T14:50:16.309 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/time.h... yes 2024-04-23T14:50:16.330 INFO:tasks.workunit.client.0.smithi083.stdout:checking for unistd.h... (cached) yes 2024-04-23T14:50:16.426 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/vfs.h usability... yes 2024-04-23T14:50:16.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.451+0000 7fa374c00640 1 -- 172.21.15.83:0/1877317689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370104e60 msgr2=0x7fa370108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.451+0000 7fa374c00640 1 --2- 172.21.15.83:0/1877317689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370104e60 0x7fa370108ea0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa358009900 tx=0x7fa35802a410 comp rx=0 tx=0).stop 2024-04-23T14:50:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.451+0000 7fa374c00640 1 -- 172.21.15.83:0/1877317689 shutdown_connections 2024-04-23T14:50:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.451+0000 7fa374c00640 1 --2- 172.21.15.83:0/1877317689 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa370109580 0x7fa370111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.451+0000 7fa374c00640 1 --2- 172.21.15.83:0/1877317689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa370104e60 0x7fa370108ea0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.451+0000 7fa374c00640 1 --2- 172.21.15.83:0/1877317689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa3701044b0 0x7fa370104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.451+0000 7fa374c00640 1 -- 172.21.15.83:0/1877317689 >> 172.21.15.83:0/1877317689 conn(0x7fa3700ffd90 msgr2=0x7fa3701021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.451+0000 7fa374c00640 1 -- 172.21.15.83:0/1877317689 shutdown_connections 2024-04-23T14:50:16.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.451+0000 7fa374c00640 1 -- 172.21.15.83:0/1877317689 wait complete. 2024-04-23T14:50:16.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.452+0000 7fa374c00640 1 Processor -- start 2024-04-23T14:50:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.452+0000 7fa374c00640 1 -- start start 2024-04-23T14:50:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa374c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3701044b0 0x7fa3701a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa374c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa370109580 0x7fa3701a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa374c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa37019a990 0x7fa37019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa374c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701145f0 con 0x7fa3701044b0 2024-04-23T14:50:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa374c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa370114470 con 0x7fa370109580 2024-04-23T14:50:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa374c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa370114770 con 0x7fa37019a990 2024-04-23T14:50:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa36de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3701044b0 0x7fa3701a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa367e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa370109580 0x7fa3701a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa36de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3701044b0 0x7fa3701a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32810/0 (socket says 172.21.15.83:32810) 2024-04-23T14:50:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa367e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa370109580 0x7fa3701a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:49658/0 (socket says 172.21.15.83:49658) 2024-04-23T14:50:16.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa36de00640 1 -- 172.21.15.83:0/3180967840 learned_addr learned my addr 172.21.15.83:0/3180967840 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa36e800640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa37019a990 0x7fa37019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa36de00640 1 -- 172.21.15.83:0/3180967840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa37019a990 msgr2=0x7fa37019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa36de00640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa37019a990 0x7fa37019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa36de00640 1 -- 172.21.15.83:0/3180967840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa370109580 msgr2=0x7fa3701a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa36de00640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa370109580 0x7fa3701a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.453+0000 7fa36de00640 1 -- 172.21.15.83:0/3180967840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37019b4b0 con 0x7fa3701044b0 2024-04-23T14:50:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.454+0000 7fa36de00640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3701044b0 0x7fa3701a07e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa36000e9f0 tx=0x7fa36000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.454+0000 7fa366a00640 1 -- 172.21.15.83:0/3180967840 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36000cd80 con 0x7fa3701044b0 2024-04-23T14:50:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.454+0000 7fa366a00640 1 -- 172.21.15.83:0/3180967840 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa360010910 con 0x7fa3701044b0 2024-04-23T14:50:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.454+0000 7fa374c00640 1 -- 172.21.15.83:0/3180967840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37019b7a0 con 0x7fa3701044b0 2024-04-23T14:50:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.454+0000 7fa366a00640 1 -- 172.21.15.83:0/3180967840 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360018990 con 0x7fa3701044b0 2024-04-23T14:50:16.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.454+0000 7fa374c00640 1 -- 172.21.15.83:0/3180967840 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3701a7a40 con 0x7fa3701044b0 2024-04-23T14:50:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.457+0000 7fa377bab740 1 Processor -- start 2024-04-23T14:50:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa377bab740 1 -- start start 2024-04-23T14:50:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa366a00640 1 -- 172.21.15.83:0/3180967840 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa360010ac0 con 0x7fa3701044b0 2024-04-23T14:50:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa377bab740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ad860 0x55cd5c4adc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa377bab740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa377bab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4b8850 0x55cd5c4bac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa377bab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd5c4131f0 con 0x55cd5c4b8850 2024-04-23T14:50:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa377bab740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cd5c3b9350 con 0x55cd5c4ad860 2024-04-23T14:50:16.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa377bab740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cd5c40f1b0 con 0x55cd5c4ae180 2024-04-23T14:50:16.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa367e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4b8850 0x55cd5c4bac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa36de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa367e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4b8850 0x55cd5c4bac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32826/0 (socket says 172.21.15.83:32826) 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa36de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:54280/0 (socket says 172.21.15.83:54280) 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa367e00640 1 -- 172.21.15.83:0/2617845877 learned_addr learned my addr 172.21.15.83:0/2617845877 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.458+0000 7fa36e800640 1 --2- 172.21.15.83:0/2617845877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ad860 0x55cd5c4adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.459+0000 7fa36e800640 1 -- 172.21.15.83:0/2617845877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4ae180 msgr2=0x55cd5c4b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.459+0000 7fa36e800640 1 --2- 172.21.15.83:0/2617845877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.459+0000 7fa36e800640 1 -- 172.21.15.83:0/2617845877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4b8850 msgr2=0x55cd5c4bac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.459+0000 7fa36e800640 1 --2- 172.21.15.83:0/2617845877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4b8850 0x55cd5c4bac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.459+0000 7fa36e800640 1 -- 172.21.15.83:0/2617845877 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd5c40df50 con 0x55cd5c4ad860 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.459+0000 7fa36e800640 1 --2- 172.21.15.83:0/2617845877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ad860 0x55cd5c4adc40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa368009f00 tx=0x7fa368011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92c56fb0f53c3e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.459+0000 7fa35f000640 1 -- 172.21.15.83:0/2617845877 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368004d20 con 0x55cd5c4ad860 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.459+0000 7fa35f000640 1 -- 172.21.15.83:0/2617845877 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa368004050 con 0x55cd5c4ad860 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.459+0000 7fa35f000640 1 -- 172.21.15.83:0/2617845877 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368005260 con 0x55cd5c4ad860 2024-04-23T14:50:16.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.460+0000 7fa366a00640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa3540a3da0 0x7fa3540a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.460+0000 7fa366a00640 1 -- 172.21.15.83:0/3180967840 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa3600d0a70 con 0x7fa3701044b0 2024-04-23T14:50:16.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.460+0000 7fa36de00640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa3540a3da0 0x7fa3540a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.460+0000 7fa377bab740 1 -- 172.21.15.83:0/2617845877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ad860 msgr2=0x55cd5c4adc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.460+0000 7fa377bab740 1 --2- 172.21.15.83:0/2617845877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ad860 0x55cd5c4adc40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa368009f00 tx=0x7fa368011810 comp rx=0 tx=0).stop 2024-04-23T14:50:16.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.460+0000 7fa377bab740 1 -- 172.21.15.83:0/2617845877 shutdown_connections 2024-04-23T14:50:16.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.460+0000 7fa377bab740 1 --2- 172.21.15.83:0/2617845877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4b8850 0x55cd5c4bac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.460+0000 7fa377bab740 1 --2- 172.21.15.83:0/2617845877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.460+0000 7fa377bab740 1 --2- 172.21.15.83:0/2617845877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ad860 0x55cd5c4adc40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.460+0000 7fa377bab740 1 -- 172.21.15.83:0/2617845877 >> 172.21.15.83:0/2617845877 conn(0x55cd5c4a9b80 msgr2=0x55cd5c4a9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.461+0000 7fa377bab740 1 -- 172.21.15.83:0/2617845877 shutdown_connections 2024-04-23T14:50:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.461+0000 7fa377bab740 1 -- 172.21.15.83:0/2617845877 wait complete. 2024-04-23T14:50:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.461+0000 7fa36de00640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa3540a3da0 0x7fa3540a6260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa3600ce280 tx=0x7fa3600d45c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.461+0000 7fa377bab740 1 -- start start 2024-04-23T14:50:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.461+0000 7fa377bab740 1 Processor -- start 2024-04-23T14:50:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa377bab740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4ad860 0x55cd5c4b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa377bab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa377bab740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4b8850 0x55cd5c4b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa377bab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd5c4bb640 con 0x55cd5c4ae180 2024-04-23T14:50:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa377bab740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cd5c3cc690 con 0x55cd5c4b8850 2024-04-23T14:50:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa377bab740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cd5c4bb7c0 con 0x55cd5c4ad860 2024-04-23T14:50:16.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa377bab740 10 client.0 _ll_register_callbacks cb 0x55cd5c408650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa367e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa36e800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4ad860 0x55cd5c4b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa367e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32828/0 (socket says 172.21.15.83:32828) 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa367e00640 1 -- 172.21.15.83:0/173319890 learned_addr learned my addr 172.21.15.83:0/173319890 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa36e800640 1 --2- 172.21.15.83:0/173319890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4b8850 0x55cd5c4b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa36e800640 1 -- 172.21.15.83:0/173319890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4b8850 msgr2=0x55cd5c4b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa36e800640 1 --2- 172.21.15.83:0/173319890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4b8850 0x55cd5c4b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa36e800640 1 -- 172.21.15.83:0/173319890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ae180 msgr2=0x55cd5c4b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa36e800640 1 --2- 172.21.15.83:0/173319890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.462+0000 7fa36e800640 1 -- 172.21.15.83:0/173319890 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd5c3cb920 con 0x55cd5c4ad860 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.463+0000 7fa36e800640 1 --2- 172.21.15.83:0/173319890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4ad860 0x55cd5c4b57c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa368009730 tx=0x7fa3680048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47bd068ff700e5cf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.463+0000 7fa353400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.463+0000 7fa353400640 1 -- 172.21.15.83:0/173319890 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36802c020 con 0x55cd5c4ad860 2024-04-23T14:50:16.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.463+0000 7fa353400640 1 -- 172.21.15.83:0/173319890 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa368005890 con 0x55cd5c4ad860 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.463+0000 7fa377bab740 1 -- 172.21.15.83:0/173319890 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa368004050 con 0x55cd5c4ad860 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.463+0000 7fa353400640 1 -- 172.21.15.83:0/173319890 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa368037050 con 0x55cd5c4ad860 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.463+0000 7fa377bab740 10 client.25762 fetch_fsmap learned FSMap version 25 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.464+0000 7fa377bab740 1 -- 172.21.15.83:0/173319890 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cd5c1ef7d0 con 0x55cd5c4ad860 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.464+0000 7fa353400640 1 -- 172.21.15.83:0/173319890 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa368004050 con 0x55cd5c4ad860 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.464+0000 7fa353400640 1 -- 172.21.15.83:0/173319890 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa368028720 con 0x55cd5c4ad860 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.464+0000 7fa377bab740 10 client.25762 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.464+0000 7fa377bab740 10 client.25762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.464+0000 7fa377bab740 20 client.25762 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.464+0000 7fa377bab740 1 --2- 172.21.15.83:0/173319890 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd5c4c8bb0 0x55cd5c4cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.464+0000 7fa377bab740 4 client.25762 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.464+0000 7fa377bab740 1 -- 172.21.15.83:0/173319890 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd5c420c10 con 0x55cd5c4c8bb0 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.465+0000 7fa367e00640 1 --2- 172.21.15.83:0/173319890 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd5c4c8bb0 0x55cd5c4cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.465+0000 7fa367e00640 1 --2- 172.21.15.83:0/173319890 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd5c4c8bb0 0x55cd5c4cafa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cc5ddaf71a53f2c server_cookie=1e0a3cf2ec2657b5 in_seq=0 out_seq=0 2024-04-23T14:50:16.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.465+0000 7fa353400640 10 client.25762 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa353400640 1 -- 172.21.15.83:0/173319890 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd5c420c10 con 0x55cd5c4c8bb0 2024-04-23T14:50:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa353400640 10 client.25762 handle_command_reply: tid=0 2024-04-23T14:50:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa377bab740 1 client.25762 shutdown 2024-04-23T14:50:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa377bab740 1 -- 172.21.15.83:0/173319890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4ad860 msgr2=0x55cd5c4b57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa377bab740 1 --2- 172.21.15.83:0/173319890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4ad860 0x55cd5c4b57c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa368009730 tx=0x7fa3680048f0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa377bab740 1 -- 172.21.15.83:0/173319890 shutdown_connections 2024-04-23T14:50:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa377bab740 1 --2- 172.21.15.83:0/173319890 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd5c4c8bb0 0x55cd5c4cafa0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa377bab740 1 --2- 172.21.15.83:0/173319890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4b8850 0x55cd5c4b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa377bab740 1 --2- 172.21.15.83:0/173319890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa377bab740 1 --2- 172.21.15.83:0/173319890 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4ad860 0x55cd5c4b57c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa353400640 0 client.25762 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:16.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa377bab740 1 -- 172.21.15.83:0/173319890 >> 172.21.15.83:0/173319890 conn(0x55cd5c4a9b80 msgr2=0x55cd5c4af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.466+0000 7fa377bab740 1 -- 172.21.15.83:0/173319890 shutdown_connections 2024-04-23T14:50:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.467+0000 7fa377bab740 1 -- 172.21.15.83:0/173319890 wait complete. 2024-04-23T14:50:16.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.467+0000 7fa377bab740 20 client.25762 trim_cache size 0 max 16384 2024-04-23T14:50:16.471 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/vfs.h presence... yes 2024-04-23T14:50:16.471 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/vfs.h... yes 2024-04-23T14:50:16.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.487+0000 7fa377bab740 1 Processor -- start 2024-04-23T14:50:16.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.487+0000 7fa377bab740 1 -- start start 2024-04-23T14:50:16.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.487+0000 7fa377bab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ad860 0x55cd5c4b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.487+0000 7fa377bab740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 0x55cd5c4c4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.487+0000 7fa377bab740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4b8850 0x55cd5c4affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.487+0000 7fa377bab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701b28a0 con 0x55cd5c4ad860 2024-04-23T14:50:16.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.487+0000 7fa377bab740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701b0170 con 0x55cd5c4ae180 2024-04-23T14:50:16.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.487+0000 7fa377bab740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701adfc0 con 0x55cd5c4b8850 2024-04-23T14:50:16.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.487+0000 7fa36e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ad860 0x55cd5c4b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.488+0000 7fa367e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 0x55cd5c4c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.488+0000 7fa367e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 0x55cd5c4c4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:49676/0 (socket says 172.21.15.83:49676) 2024-04-23T14:50:16.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.488+0000 7fa367e00640 1 -- 172.21.15.83:0/3228502296 learned_addr learned my addr 172.21.15.83:0/3228502296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:16.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.488+0000 7fa36e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ad860 0x55cd5c4b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:32834/0 (socket says 172.21.15.83:32834) 2024-04-23T14:50:16.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.488+0000 7fa36e800640 1 --2- 172.21.15.83:0/3228502296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4b8850 0x55cd5c4affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.488+0000 7fa367e00640 1 -- 172.21.15.83:0/3228502296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4b8850 msgr2=0x55cd5c4affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.488+0000 7fa367e00640 1 --2- 172.21.15.83:0/3228502296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4b8850 0x55cd5c4affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.488+0000 7fa367e00640 1 -- 172.21.15.83:0/3228502296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ad860 msgr2=0x55cd5c4b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.488+0000 7fa367e00640 1 --2- 172.21.15.83:0/3228502296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ad860 0x55cd5c4b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.488+0000 7fa367e00640 1 -- 172.21.15.83:0/3228502296 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd5c1253f0 con 0x55cd5c4ae180 2024-04-23T14:50:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.488+0000 7fa36e800640 1 --2- 172.21.15.83:0/3228502296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4b8850 0x55cd5c4affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:50:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.488+0000 7fa367e00640 1 --2- 172.21.15.83:0/3228502296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 0x55cd5c4c4500 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa358000c00 tx=0x7fa358031920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73cd18876cd82cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.489+0000 7fa35e600640 1 -- 172.21.15.83:0/3228502296 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35804c020 con 0x55cd5c4ae180 2024-04-23T14:50:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.489+0000 7fa35e600640 1 -- 172.21.15.83:0/3228502296 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa358048420 con 0x55cd5c4ae180 2024-04-23T14:50:16.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.489+0000 7fa35e600640 1 -- 172.21.15.83:0/3228502296 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358035040 con 0x55cd5c4ae180 2024-04-23T14:50:16.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.489+0000 7fa377bab740 1 -- 172.21.15.83:0/3228502296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 msgr2=0x55cd5c4c4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.489+0000 7fa377bab740 1 --2- 172.21.15.83:0/3228502296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 0x55cd5c4c4500 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa358000c00 tx=0x7fa358031920 comp rx=0 tx=0).stop 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.489+0000 7fa377bab740 1 -- 172.21.15.83:0/3228502296 shutdown_connections 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.489+0000 7fa377bab740 1 --2- 172.21.15.83:0/3228502296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4b8850 0x55cd5c4affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.490+0000 7fa377bab740 1 --2- 172.21.15.83:0/3228502296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 0x55cd5c4c4500 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.490+0000 7fa377bab740 1 --2- 172.21.15.83:0/3228502296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ad860 0x55cd5c4b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.490+0000 7fa377bab740 1 -- 172.21.15.83:0/3228502296 >> 172.21.15.83:0/3228502296 conn(0x55cd5c4a9b80 msgr2=0x55cd5c4a8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.490+0000 7fa377bab740 1 -- 172.21.15.83:0/3228502296 shutdown_connections 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.490+0000 7fa377bab740 1 -- 172.21.15.83:0/3228502296 wait complete. 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.490+0000 7fa377bab740 1 -- start start 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.490+0000 7fa377bab740 1 Processor -- start 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa377bab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ad860 0x55cd5c4c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa377bab740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa377bab740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4b8850 0x55cd5c4b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa377bab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701ae630 con 0x55cd5c4ad860 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa377bab740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701afaa0 con 0x55cd5c4ae180 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa377bab740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa3701aed00 con 0x55cd5c4b8850 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa377bab740 10 client.0 _ll_register_callbacks cb 0x7fa3680093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa36e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ad860 0x55cd5c4c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa367e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa367e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:49692/0 (socket says 172.21.15.83:49692) 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa367e00640 1 -- 172.21.15.83:0/520560137 learned_addr learned my addr 172.21.15.83:0/520560137 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa36e800640 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4b8850 0x55cd5c4b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa367e00640 1 -- 172.21.15.83:0/520560137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4b8850 msgr2=0x55cd5c4b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa367e00640 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4b8850 0x55cd5c4b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa367e00640 1 -- 172.21.15.83:0/520560137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ad860 msgr2=0x55cd5c4c0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa367e00640 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ad860 0x55cd5c4c0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa367e00640 1 -- 172.21.15.83:0/520560137 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd5c0e50f0 con 0x55cd5c4ae180 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.491+0000 7fa36e800640 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ad860 0x55cd5c4c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.492+0000 7fa367e00640 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b21d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa35802a930 tx=0x7fa358031650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23b7c0b8dabcc6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.492+0000 7fa353400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.492+0000 7fa353400640 1 -- 172.21.15.83:0/520560137 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358035040 con 0x55cd5c4ae180 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.492+0000 7fa377bab740 1 -- 172.21.15.83:0/520560137 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa358048420 con 0x55cd5c4ae180 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.492+0000 7fa353400640 1 -- 172.21.15.83:0/520560137 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa358005030 con 0x55cd5c4ae180 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.492+0000 7fa353400640 1 -- 172.21.15.83:0/520560137 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa358005230 con 0x55cd5c4ae180 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.492+0000 7fa377bab740 10 client.25799 fetch_fsmap learned FSMap version 25 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.492+0000 7fa377bab740 1 -- 172.21.15.83:0/520560137 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cd5c3d8810 con 0x55cd5c4ae180 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.493+0000 7fa353400640 1 -- 172.21.15.83:0/520560137 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa358048420 con 0x55cd5c4ae180 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.493+0000 7fa353400640 1 -- 172.21.15.83:0/520560137 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa35803d4e0 con 0x55cd5c4ae180 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.493+0000 7fa377bab740 10 client.25799 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.493+0000 7fa377bab740 10 client.25799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:16.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.493+0000 7fa377bab740 20 client.25799 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.493+0000 7fa377bab740 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd5c52f380 0x55cd5c5317a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.493+0000 7fa377bab740 4 client.25799 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.493+0000 7fa377bab740 1 -- 172.21.15.83:0/520560137 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd5c41aa80 con 0x55cd5c52f380 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.493+0000 7fa36e800640 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd5c52f380 0x55cd5c5317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa36e800640 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd5c52f380 0x55cd5c5317a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0150506a7a98fc7 server_cookie=10648d51b161e086 in_seq=0 out_seq=0 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa353400640 10 client.25799 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa353400640 1 -- 172.21.15.83:0/520560137 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cd5c41aa80 con 0x55cd5c52f380 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa353400640 10 client.25799 handle_command_reply: tid=0 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa377bab740 1 client.25799 shutdown 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa377bab740 1 -- 172.21.15.83:0/520560137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 msgr2=0x55cd5c4b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa377bab740 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b21d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa35802a930 tx=0x7fa358031650 comp rx=0 tx=0).stop 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa377bab740 1 -- 172.21.15.83:0/520560137 shutdown_connections 2024-04-23T14:50:16.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa377bab740 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd5c52f380 0x55cd5c5317a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa377bab740 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd5c4b8850 0x55cd5c4b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa377bab740 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd5c4ae180 0x55cd5c4b21d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa353400640 0 client.25799 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa377bab740 1 --2- 172.21.15.83:0/520560137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd5c4ad860 0x55cd5c4c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa377bab740 1 -- 172.21.15.83:0/520560137 >> 172.21.15.83:0/520560137 conn(0x55cd5c4a9b80 msgr2=0x55cd5c4b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa377bab740 1 -- 172.21.15.83:0/520560137 shutdown_connections 2024-04-23T14:50:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa377bab740 1 -- 172.21.15.83:0/520560137 wait complete. 2024-04-23T14:50:16.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.494+0000 7fa377bab740 20 client.25799 trim_cache size 0 max 16384 2024-04-23T14:50:16.499 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:50:16.499 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:50:16.500 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:50:16.500 INFO:teuthology.orchestra.run.smithi083.stdout: "6e3cefac-51ff-44c5-9f5b-05c9b613678f": { 2024-04-23T14:50:16.500 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:50:16.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "6e3cefac-51ff-44c5-9f5b-05c9b613678f", 2024-04-23T14:50:16.500 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:50:16.500 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:50:16.500 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:50:16.500 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:50:16.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.495+0000 7fa374c00640 1 -- 172.21.15.83:0/3180967840 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa3540a3da0 msgr2=0x7fa3540a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.495+0000 7fa374c00640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa3540a3da0 0x7fa3540a6260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa3600ce280 tx=0x7fa3600d45c0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.495+0000 7fa374c00640 1 -- 172.21.15.83:0/3180967840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3701044b0 msgr2=0x7fa3701a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:16.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.495+0000 7fa374c00640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3701044b0 0x7fa3701a07e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa36000e9f0 tx=0x7fa36000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.495+0000 7fa374c00640 1 -- 172.21.15.83:0/3180967840 shutdown_connections 2024-04-23T14:50:16.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.495+0000 7fa374c00640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa37019a990 0x7fa37019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.495+0000 7fa374c00640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa370109580 0x7fa3701a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.495+0000 7fa374c00640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa3540a3da0 0x7fa3540a6260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.495+0000 7fa374c00640 1 --2- 172.21.15.83:0/3180967840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3701044b0 0x7fa3701a07e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:16.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.495+0000 7fa374c00640 1 -- 172.21.15.83:0/3180967840 >> 172.21.15.83:0/3180967840 conn(0x7fa3700ffd90 msgr2=0x7fa3701010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:16.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.495+0000 7fa374c00640 1 -- 172.21.15.83:0/3180967840 shutdown_connections 2024-04-23T14:50:16.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:16.495+0000 7fa374c00640 1 -- 172.21.15.83:0/3180967840 wait complete. 2024-04-23T14:50:16.512 INFO:tasks.cephfs.filesystem:scrub status for tag:6e3cefac-51ff-44c5-9f5b-05c9b613678f - {'path': '/', 'tag': '6e3cefac-51ff-44c5-9f5b-05c9b613678f', 'options': 'recursive,force'} 2024-04-23T14:50:16.566 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/limits.h usability... no 2024-04-23T14:50:16.625 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/limits.h presence... no 2024-04-23T14:50:16.628 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/limits.h... no 2024-04-23T14:50:16.716 INFO:tasks.workunit.client.0.smithi083.stdout:checking for an ANSI C-conforming const... yes 2024-04-23T14:50:16.813 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-23T14:50:16.816 INFO:tasks.workunit.client.0.smithi083.stdout:checking for function prototypes... yes 2024-04-23T14:50:16.902 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether setvbuf arguments are reversed... no 2024-04-23T14:50:17.073 INFO:tasks.workunit.client.0.smithi083.stdout:checking for system... yes 2024-04-23T14:50:17.217 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gettimeofday... yes 2024-04-23T14:50:17.358 INFO:tasks.workunit.client.0.smithi083.stdout:checking for mkdir... yes 2024-04-23T14:50:17.503 INFO:tasks.workunit.client.0.smithi083.stdout:checking for strerror... yes 2024-04-23T14:50:17.656 INFO:tasks.workunit.client.0.smithi083.stdout:checking for open64... yes 2024-04-23T14:50:17.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: pgmap v57: 97 pgs: 97 active+clean; 3.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 172 KiB/s wr, 27 op/s 2024-04-23T14:50:17.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:50:17.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:50:17.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:50:17.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:50:17.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:50:17.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:50:17.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:17 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.798 INFO:tasks.workunit.client.0.smithi083.stdout:checking for stat64... yes 2024-04-23T14:50:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: pgmap v57: 97 pgs: 97 active+clean; 3.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 172 KiB/s wr, 27 op/s 2024-04-23T14:50:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:50:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:50:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:50:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:50:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:50:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:50:17.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:17 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.948 INFO:tasks.workunit.client.0.smithi083.stdout:checking for fseeko64... yes 2024-04-23T14:50:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: pgmap v57: 97 pgs: 97 active+clean; 3.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 172 KiB/s wr, 27 op/s 2024-04-23T14:50:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:50:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:50:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:50:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:50:17.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:50:17.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:50:17.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:17.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:17 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:18.093 INFO:tasks.workunit.client.0.smithi083.stdout:checking for lrand48_r... yes 2024-04-23T14:50:18.243 INFO:tasks.workunit.client.0.smithi083.stdout:checking for srand48_r... yes 2024-04-23T14:50:18.302 INFO:tasks.workunit.client.0.smithi083.stdout:configure: creating ./config.status 2024-04-23T14:50:18.572 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: creating Makefile 2024-04-23T14:50:18.612 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: creating config.h 2024-04-23T14:50:18.659 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: executing depfiles commands 2024-04-23T14:50:18.791 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: executing default commands 2024-04-23T14:50:18.882 INFO:tasks.workunit.client.0.smithi083.stderr:+ make 2024-04-23T14:50:18.895 INFO:tasks.workunit.client.0.smithi083.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-04-23T14:50:18.904 INFO:tasks.workunit.client.0.smithi083.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-04-23T14:50:18.907 INFO:tasks.workunit.client.0.smithi083.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-04-23T14:50:18.907 INFO:tasks.workunit.client.0.smithi083.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-04-23T14:50:18.907 INFO:tasks.workunit.client.0.smithi083.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-04-23T14:50:18.907 INFO:tasks.workunit.client.0.smithi083.stderr: any GNU archive site. 2024-04-23T14:50:18.913 INFO:tasks.workunit.client.0.smithi083.stdout: cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-04-23T14:50:18.918 INFO:tasks.workunit.client.0.smithi083.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-04-23T14:50:18.921 INFO:tasks.workunit.client.0.smithi083.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-04-23T14:50:18.921 INFO:tasks.workunit.client.0.smithi083.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-04-23T14:50:18.921 INFO:tasks.workunit.client.0.smithi083.stderr: You might want to install the `Automake' and `Perl' packages. 2024-04-23T14:50:18.921 INFO:tasks.workunit.client.0.smithi083.stderr: Grab them from any GNU archive site. 2024-04-23T14:50:19.008 INFO:tasks.workunit.client.0.smithi083.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-04-23T14:50:19.509 INFO:tasks.workunit.client.0.smithi083.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-04-23T14:50:19.509 INFO:tasks.workunit.client.0.smithi083.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-04-23T14:50:19.509 INFO:tasks.workunit.client.0.smithi083.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-04-23T14:50:19.510 INFO:tasks.workunit.client.0.smithi083.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-04-23T14:50:19.707 INFO:tasks.workunit.client.0.smithi083.stdout:/bin/sh ./config.status --recheck 2024-04-23T14:50:19.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:19 smithi083 ceph-mon[27867]: pgmap v58: 97 pgs: 97 active+clean; 3.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 161 KiB/s wr, 13 op/s 2024-04-23T14:50:19.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:19 smithi083 ceph-mon[27867]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:50:19.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:19 smithi173 ceph-mon[32443]: pgmap v58: 97 pgs: 97 active+clean; 3.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 161 KiB/s wr, 13 op/s 2024-04-23T14:50:19.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:19 smithi173 ceph-mon[32443]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:50:19.865 INFO:tasks.workunit.client.0.smithi083.stdout:running CONFIG_SHELL=/bin/sh /bin/sh ./configure --no-create --no-recursion 2024-04-23T14:50:19.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:19 smithi203 ceph-mon[32800]: pgmap v58: 97 pgs: 97 active+clean; 3.2 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 161 KiB/s wr, 13 op/s 2024-04-23T14:50:19.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:19 smithi203 ceph-mon[32800]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:50:20.122 INFO:tasks.workunit.client.0.smithi083.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-04-23T14:50:21.142 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether build environment is sane... yes 2024-04-23T14:50:21.160 INFO:tasks.workunit.client.0.smithi083.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-04-23T14:50:21.160 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gawk... gawk 2024-04-23T14:50:21.192 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether make sets $(MAKE)... yes 2024-04-23T14:50:21.288 INFO:tasks.workunit.client.0.smithi083.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-04-23T14:50:21.288 INFO:tasks.workunit.client.0.smithi083.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-04-23T14:50:21.309 INFO:tasks.workunit.client.0.smithi083.stdout:checking for style of include used by make... GNU 2024-04-23T14:50:21.314 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gcc... gcc 2024-04-23T14:50:21.501 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether the C compiler works... yes 2024-04-23T14:50:21.501 INFO:tasks.workunit.client.0.smithi083.stdout:checking for C compiler default output file name... a.out 2024-04-23T14:50:21.586 INFO:tasks.workunit.client.0.smithi083.stdout:checking for suffix of executables... 2024-04-23T14:50:21.674 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether we are cross compiling... no 2024-04-23T14:50:21.750 INFO:tasks.workunit.client.0.smithi083.stdout:checking for suffix of object files... o 2024-04-23T14:50:21.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:21 smithi083 ceph-mon[27867]: pgmap v59: 97 pgs: 97 active+clean; 7.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 578 KiB/s wr, 60 op/s 2024-04-23T14:50:21.809 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether we are using the GNU C compiler... yes 2024-04-23T14:50:21.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:21 smithi173 ceph-mon[32443]: pgmap v59: 97 pgs: 97 active+clean; 7.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 578 KiB/s wr, 60 op/s 2024-04-23T14:50:21.869 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether gcc accepts -g... yes 2024-04-23T14:50:21.955 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-23T14:50:22.005 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:21 smithi203 ceph-mon[32800]: pgmap v59: 97 pgs: 97 active+clean; 7.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 578 KiB/s wr, 60 op/s 2024-04-23T14:50:22.156 INFO:tasks.workunit.client.0.smithi083.stdout:checking dependency style of gcc... gcc3 2024-04-23T14:50:22.258 INFO:tasks.workunit.client.0.smithi083.stdout:checking how to run the C preprocessor... gcc -E 2024-04-23T14:50:22.368 INFO:tasks.workunit.client.0.smithi083.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-23T14:50:22.373 INFO:tasks.workunit.client.0.smithi083.stdout:checking for egrep... /usr/bin/grep -E 2024-04-23T14:50:22.637 INFO:tasks.workunit.client.0.smithi083.stdout:checking for ANSI C header files... yes 2024-04-23T14:50:22.723 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/types.h... yes 2024-04-23T14:50:22.804 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/stat.h... yes 2024-04-23T14:50:22.891 INFO:tasks.workunit.client.0.smithi083.stdout:checking for stdlib.h... yes 2024-04-23T14:50:22.976 INFO:tasks.workunit.client.0.smithi083.stdout:checking for string.h... yes 2024-04-23T14:50:23.067 INFO:tasks.workunit.client.0.smithi083.stdout:checking for memory.h... yes 2024-04-23T14:50:23.155 INFO:tasks.workunit.client.0.smithi083.stdout:checking for strings.h... yes 2024-04-23T14:50:23.242 INFO:tasks.workunit.client.0.smithi083.stdout:checking for inttypes.h... yes 2024-04-23T14:50:23.333 INFO:tasks.workunit.client.0.smithi083.stdout:checking for stdint.h... yes 2024-04-23T14:50:23.425 INFO:tasks.workunit.client.0.smithi083.stdout:checking for unistd.h... yes 2024-04-23T14:50:23.514 INFO:tasks.workunit.client.0.smithi083.stdout:checking minix/config.h usability... no 2024-04-23T14:50:23.569 INFO:tasks.workunit.client.0.smithi083.stdout:checking minix/config.h presence... no 2024-04-23T14:50:23.569 INFO:tasks.workunit.client.0.smithi083.stdout:checking for minix/config.h... no 2024-04-23T14:50:23.646 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-04-23T14:50:23.647 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gcc... (cached) gcc 2024-04-23T14:50:23.729 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-23T14:50:23.729 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-23T14:50:23.730 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-23T14:50:23.730 INFO:tasks.workunit.client.0.smithi083.stdout:checking dependency style of gcc... (cached) gcc3 2024-04-23T14:50:23.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:23 smithi083 ceph-mon[27867]: pgmap v60: 97 pgs: 97 active+clean; 7.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 578 KiB/s wr, 57 op/s 2024-04-23T14:50:23.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:23 smithi173 ceph-mon[32443]: pgmap v60: 97 pgs: 97 active+clean; 7.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 578 KiB/s wr, 57 op/s 2024-04-23T14:50:23.829 INFO:tasks.workunit.client.0.smithi083.stdout:checking for main in -lm... yes 2024-04-23T14:50:23.925 INFO:tasks.workunit.client.0.smithi083.stdout:checking for main in -lpthread... yes 2024-04-23T14:50:23.929 INFO:tasks.workunit.client.0.smithi083.stdout:checking for ANSI C header files... (cached) yes 2024-04-23T14:50:23.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:23 smithi203 ceph-mon[32800]: pgmap v60: 97 pgs: 97 active+clean; 7.6 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 578 KiB/s wr, 57 op/s 2024-04-23T14:50:24.001 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-04-23T14:50:24.121 INFO:tasks.workunit.client.0.smithi083.stdout:checking pthread.h usability... yes 2024-04-23T14:50:24.166 INFO:tasks.workunit.client.0.smithi083.stdout:checking pthread.h presence... yes 2024-04-23T14:50:24.167 INFO:tasks.workunit.client.0.smithi083.stdout:checking for pthread.h... yes 2024-04-23T14:50:24.275 INFO:tasks.workunit.client.0.smithi083.stdout:checking fcntl.h usability... yes 2024-04-23T14:50:24.317 INFO:tasks.workunit.client.0.smithi083.stdout:checking fcntl.h presence... yes 2024-04-23T14:50:24.317 INFO:tasks.workunit.client.0.smithi083.stdout:checking for fcntl.h... yes 2024-04-23T14:50:24.421 INFO:tasks.workunit.client.0.smithi083.stdout:checking limits.h usability... yes 2024-04-23T14:50:24.463 INFO:tasks.workunit.client.0.smithi083.stdout:checking limits.h presence... yes 2024-04-23T14:50:24.463 INFO:tasks.workunit.client.0.smithi083.stdout:checking for limits.h... yes 2024-04-23T14:50:24.493 INFO:tasks.workunit.client.0.smithi083.stdout:checking for stdint.h... (cached) yes 2024-04-23T14:50:24.580 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/time.h usability... yes 2024-04-23T14:50:24.620 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/time.h presence... yes 2024-04-23T14:50:24.620 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/time.h... yes 2024-04-23T14:50:24.649 INFO:tasks.workunit.client.0.smithi083.stdout:checking for unistd.h... (cached) yes 2024-04-23T14:50:24.735 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/vfs.h usability... yes 2024-04-23T14:50:24.773 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/vfs.h presence... yes 2024-04-23T14:50:24.773 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/vfs.h... yes 2024-04-23T14:50:24.883 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/limits.h usability... no 2024-04-23T14:50:24.943 INFO:tasks.workunit.client.0.smithi083.stdout:checking sys/limits.h presence... no 2024-04-23T14:50:24.943 INFO:tasks.workunit.client.0.smithi083.stdout:checking for sys/limits.h... no 2024-04-23T14:50:25.009 INFO:tasks.workunit.client.0.smithi083.stdout:checking for an ANSI C-conforming const... yes 2024-04-23T14:50:25.078 INFO:tasks.workunit.client.0.smithi083.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-04-23T14:50:25.202 INFO:tasks.workunit.client.0.smithi083.stdout:checking for system... yes 2024-04-23T14:50:25.324 INFO:tasks.workunit.client.0.smithi083.stdout:checking for gettimeofday... yes 2024-04-23T14:50:25.443 INFO:tasks.workunit.client.0.smithi083.stdout:checking for mkdir... yes 2024-04-23T14:50:25.565 INFO:tasks.workunit.client.0.smithi083.stdout:checking for strerror... yes 2024-04-23T14:50:25.684 INFO:tasks.workunit.client.0.smithi083.stdout:checking for open64... yes 2024-04-23T14:50:25.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:25 smithi083 ceph-mon[27867]: pgmap v61: 97 pgs: 97 active+clean; 8.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 671 KiB/s wr, 67 op/s 2024-04-23T14:50:25.809 INFO:tasks.workunit.client.0.smithi083.stdout:checking for stat64... yes 2024-04-23T14:50:25.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:25 smithi173 ceph-mon[32443]: pgmap v61: 97 pgs: 97 active+clean; 8.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 671 KiB/s wr, 67 op/s 2024-04-23T14:50:25.934 INFO:tasks.workunit.client.0.smithi083.stdout:checking for fseeko64... yes 2024-04-23T14:50:25.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:25 smithi203 ceph-mon[32800]: pgmap v61: 97 pgs: 97 active+clean; 8.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 671 KiB/s wr, 67 op/s 2024-04-23T14:50:26.054 INFO:tasks.workunit.client.0.smithi083.stdout:checking for lrand48_r... yes 2024-04-23T14:50:26.173 INFO:tasks.workunit.client.0.smithi083.stdout:checking for srand48_r... yes 2024-04-23T14:50:26.224 INFO:tasks.workunit.client.0.smithi083.stdout:configure: creating ./config.status 2024-04-23T14:50:26.390 INFO:tasks.workunit.client.0.smithi083.stdout: /bin/sh ./config.status 2024-04-23T14:50:26.514 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: creating Makefile 2024-04-23T14:50:26.549 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: creating config.h 2024-04-23T14:50:26.597 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: executing depfiles commands 2024-04-23T14:50:26.642 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: executing default commands 2024-04-23T14:50:26.675 INFO:tasks.workunit.client.0.smithi083.stdout:cd . && /bin/sh /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-04-23T14:50:26.923 INFO:tasks.workunit.client.0.smithi083.stdout:rm -f stamp-h1 2024-04-23T14:50:26.928 INFO:tasks.workunit.client.0.smithi083.stdout:touch config.h.in 2024-04-23T14:50:26.931 INFO:tasks.workunit.client.0.smithi083.stdout:cd . && /bin/sh ./config.status config.h 2024-04-23T14:50:27.036 INFO:tasks.workunit.client.0.smithi083.stdout:config.status: creating config.h 2024-04-23T14:50:27.096 INFO:tasks.workunit.client.0.smithi083.stdout:make all-am 2024-04-23T14:50:27.109 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-04-23T14:50:27.109 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-04-23T14:50:27.389 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-04-23T14:50:27.394 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-04-23T14:50:27.544 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-04-23T14:50:27.551 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-04-23T14:50:27.680 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-04-23T14:50:27.685 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-04-23T14:50:27.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:27 smithi083 ceph-mon[27867]: pgmap v62: 97 pgs: 97 active+clean; 8.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 513 KiB/s wr, 58 op/s 2024-04-23T14:50:27.822 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-04-23T14:50:27.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:27 smithi173 ceph-mon[32443]: pgmap v62: 97 pgs: 97 active+clean; 8.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 513 KiB/s wr, 58 op/s 2024-04-23T14:50:27.828 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-04-23T14:50:27.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:27 smithi203 ceph-mon[32800]: pgmap v62: 97 pgs: 97 active+clean; 8.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 513 KiB/s wr, 58 op/s 2024-04-23T14:50:28.006 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-04-23T14:50:28.011 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-04-23T14:50:28.096 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-04-23T14:50:28.102 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-04-23T14:50:28.178 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-04-23T14:50:28.183 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-04-23T14:50:28.261 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-04-23T14:50:28.266 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-04-23T14:50:28.534 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-04-23T14:50:28.541 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-04-23T14:50:28.730 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-04-23T14:50:28.736 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-04-23T14:50:28.949 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-04-23T14:50:28.956 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-04-23T14:50:29.069 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-04-23T14:50:29.075 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-04-23T14:50:29.189 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-04-23T14:50:29.194 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-04-23T14:50:29.351 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-04-23T14:50:29.356 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-04-23T14:50:29.409 INFO:tasks.workunit.client.0.smithi083.stderr:In file included from parser.h:22, 2024-04-23T14:50:29.409 INFO:tasks.workunit.client.0.smithi083.stderr: from parser.c:25: 2024-04-23T14:50:29.409 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c: In function ‘init_filesys’: 2024-04-23T14:50:29.409 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-23T14:50:29.409 INFO:tasks.workunit.client.0.smithi083.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-23T14:50:29.410 INFO:tasks.workunit.client.0.smithi083.stderr: | ^ 2024-04-23T14:50:29.410 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-23T14:50:29.410 INFO:tasks.workunit.client.0.smithi083.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-23T14:50:29.410 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~ 2024-04-23T14:50:29.410 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-23T14:50:29.410 INFO:tasks.workunit.client.0.smithi083.stderr: 36 | container_of(ptr, type, member) 2024-04-23T14:50:29.410 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~ 2024-04-23T14:50:29.410 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-23T14:50:29.410 INFO:tasks.workunit.client.0.smithi083.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-23T14:50:29.410 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~ 2024-04-23T14:50:29.410 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-23T14:50:29.411 INFO:tasks.workunit.client.0.smithi083.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-23T14:50:29.411 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-23T14:50:29.411 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-23T14:50:29.411 INFO:tasks.workunit.client.0.smithi083.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-23T14:50:29.411 INFO:tasks.workunit.client.0.smithi083.stderr: | ^ 2024-04-23T14:50:29.411 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-23T14:50:29.411 INFO:tasks.workunit.client.0.smithi083.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-23T14:50:29.411 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~ 2024-04-23T14:50:29.411 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-23T14:50:29.411 INFO:tasks.workunit.client.0.smithi083.stderr: 36 | container_of(ptr, type, member) 2024-04-23T14:50:29.411 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~ 2024-04-23T14:50:29.411 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-23T14:50:29.412 INFO:tasks.workunit.client.0.smithi083.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-23T14:50:29.412 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~ 2024-04-23T14:50:29.412 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c:934:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-23T14:50:29.412 INFO:tasks.workunit.client.0.smithi083.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-04-23T14:50:29.412 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-23T14:50:29.412 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-23T14:50:29.412 INFO:tasks.workunit.client.0.smithi083.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-23T14:50:29.412 INFO:tasks.workunit.client.0.smithi083.stderr: | ^ 2024-04-23T14:50:29.412 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-23T14:50:29.412 INFO:tasks.workunit.client.0.smithi083.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-23T14:50:29.412 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~ 2024-04-23T14:50:29.412 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-23T14:50:29.413 INFO:tasks.workunit.client.0.smithi083.stderr: 36 | container_of(ptr, type, member) 2024-04-23T14:50:29.413 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~ 2024-04-23T14:50:29.413 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-23T14:50:29.413 INFO:tasks.workunit.client.0.smithi083.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-23T14:50:29.413 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~ 2024-04-23T14:50:29.413 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-23T14:50:29.413 INFO:tasks.workunit.client.0.smithi083.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-23T14:50:29.413 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-23T14:50:29.413 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-23T14:50:29.413 INFO:tasks.workunit.client.0.smithi083.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-23T14:50:29.413 INFO:tasks.workunit.client.0.smithi083.stderr: | ^ 2024-04-23T14:50:29.413 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-23T14:50:29.414 INFO:tasks.workunit.client.0.smithi083.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-23T14:50:29.414 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~ 2024-04-23T14:50:29.414 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-23T14:50:29.414 INFO:tasks.workunit.client.0.smithi083.stderr: 36 | container_of(ptr, type, member) 2024-04-23T14:50:29.414 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~ 2024-04-23T14:50:29.414 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-23T14:50:29.414 INFO:tasks.workunit.client.0.smithi083.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-23T14:50:29.414 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~ 2024-04-23T14:50:29.414 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c:941:17: note: in expansion of macro ‘list_for_each_entry’ 2024-04-23T14:50:29.414 INFO:tasks.workunit.client.0.smithi083.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-04-23T14:50:29.414 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c: In function ‘init_tg_stats’: 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr: | ^ 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~ 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr: 36 | container_of(ptr, type, member) 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~ 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~ 2024-04-23T14:50:29.415 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr: | ^ 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~ 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr: 36 | container_of(ptr, type, member) 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~ 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-23T14:50:29.416 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~ 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c:971:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr: 971 | list_for_each_entry(tmp_list, 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr: | ^ 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~ 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr: 36 | container_of(ptr, type, member) 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~ 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:39:20: note: in expansion of macro ‘list_entry’ 2024-04-23T14:50:29.417 INFO:tasks.workunit.client.0.smithi083.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~ 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr: | ^ 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:33:35: note: in expansion of macro ‘offsetof’ 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~ 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:36:9: note: in expansion of macro ‘container_of’ 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr: 36 | container_of(ptr, type, member) 2024-04-23T14:50:29.418 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~ 2024-04-23T14:50:29.419 INFO:tasks.workunit.client.0.smithi083.stderr:list.h:41:20: note: in expansion of macro ‘list_entry’ 2024-04-23T14:50:29.419 INFO:tasks.workunit.client.0.smithi083.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-04-23T14:50:29.419 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~ 2024-04-23T14:50:29.419 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c:980:41: note: in expansion of macro ‘list_for_each_entry’ 2024-04-23T14:50:29.419 INFO:tasks.workunit.client.0.smithi083.stderr: 980 | list_for_each_entry(tmp_list, 2024-04-23T14:50:29.419 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-23T14:50:29.469 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c: In function ‘get_optstr’: 2024-04-23T14:50:29.469 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c:206:23: warning: ‘strnlen’ specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-04-23T14:50:29.469 INFO:tasks.workunit.client.0.smithi083.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-04-23T14:50:29.469 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-23T14:50:29.469 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c:200:14: note: source object declared here 2024-04-23T14:50:29.469 INFO:tasks.workunit.client.0.smithi083.stderr: 200 | char temp[BUFSIZE]; 2024-04-23T14:50:29.470 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~ 2024-04-23T14:50:29.477 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c: In function ‘size64_convert’: 2024-04-23T14:50:29.477 INFO:tasks.workunit.client.0.smithi083.stderr:parser.c:119:9: warning: ‘memcpy’ reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-04-23T14:50:29.477 INFO:tasks.workunit.client.0.smithi083.stderr: 119 | memcpy(unit, "\0", 3); 2024-04-23T14:50:29.477 INFO:tasks.workunit.client.0.smithi083.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-23T14:50:29.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:29 smithi083 ceph-mon[27867]: pgmap v63: 97 pgs: 97 active+clean; 8.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 510 KiB/s wr, 56 op/s 2024-04-23T14:50:29.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:29 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:50:29.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:29 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:50:29.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:29 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:50:29.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:29 smithi173 ceph-mon[32443]: pgmap v63: 97 pgs: 97 active+clean; 8.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 510 KiB/s wr, 56 op/s 2024-04-23T14:50:29.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:29 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:50:29.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:29 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:50:29.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:29 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:50:29.834 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-04-23T14:50:29.840 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-04-23T14:50:29.937 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-04-23T14:50:29.944 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-04-23T14:50:29.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:29 smithi203 ceph-mon[32800]: pgmap v63: 97 pgs: 97 active+clean; 8.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 510 KiB/s wr, 56 op/s 2024-04-23T14:50:29.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:29 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:50:29.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:29 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:50:29.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:29 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:50:30.075 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-04-23T14:50:30.082 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-04-23T14:50:30.143 INFO:tasks.workunit.client.0.smithi083.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-04-23T14:50:30.153 INFO:tasks.workunit.client.0.smithi083.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-04-23T14:50:30.221 INFO:tasks.workunit.client.0.smithi083.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-04-23T14:50:30.222 INFO:tasks.workunit.client.0.smithi083.stderr:+ cd .. 2024-04-23T14:50:30.222 INFO:tasks.workunit.client.0.smithi083.stderr:+ mkdir tmp 2024-04-23T14:50:30.229 INFO:tasks.workunit.client.0.smithi083.stderr:+ cd tmp 2024-04-23T14:50:30.258 INFO:tasks.workunit.client.0.smithi083.stderr:+ for f in $mydir/*.ffsb 2024-04-23T14:50:30.258 INFO:tasks.workunit.client.0.smithi083.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-04-23T14:50:30.261 INFO:tasks.workunit.client.0.smithi083.stdout:FFSB version 6.0-RC2 started 2024-04-23T14:50:30.261 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-23T14:50:30.269 INFO:tasks.workunit.client.0.smithi083.stdout:benchmark time = 300 2024-04-23T14:50:30.269 INFO:tasks.workunit.client.0.smithi083.stdout:ThreadGroup 0 2024-04-23T14:50:30.269 INFO:tasks.workunit.client.0.smithi083.stdout:================ 2024-04-23T14:50:30.269 INFO:tasks.workunit.client.0.smithi083.stdout: num_threads = 32 2024-04-23T14:50:30.269 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-23T14:50:30.269 INFO:tasks.workunit.client.0.smithi083.stdout: read_random = off 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: read_size = 0 (0B) 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: read_blocksize = 0 (0B) 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: read_skip = off 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: read_skipsize = 0 (0B) 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: write_random = on 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: write_size = 5242880 (5MB) 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: fsync_file = 0 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: write_blocksize = 4096 (4KB) 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: wait time = 0 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: op weights 2024-04-23T14:50:30.270 INFO:tasks.workunit.client.0.smithi083.stdout: read = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: readall = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: write = 1 (100.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: create = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: append = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: delete = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: metaop = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: createdir = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: stat = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: writeall = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: writeall_fsync = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: open_close = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: write_fsync = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: create_fsync = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: append_fsync = 0 (0.00%) 2024-04-23T14:50:30.271 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-23T14:50:30.272 INFO:tasks.workunit.client.0.smithi083.stdout:FileSystem . 2024-04-23T14:50:30.272 INFO:tasks.workunit.client.0.smithi083.stdout:========== 2024-04-23T14:50:30.272 INFO:tasks.workunit.client.0.smithi083.stdout: num_dirs = 0 2024-04-23T14:50:30.272 INFO:tasks.workunit.client.0.smithi083.stdout: starting files = 128 2024-04-23T14:50:30.272 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-23T14:50:30.272 INFO:tasks.workunit.client.0.smithi083.stdout: min file size = 104857600 (100MB) 2024-04-23T14:50:30.272 INFO:tasks.workunit.client.0.smithi083.stdout: max file size = 104857600 (100MB) 2024-04-23T14:50:30.272 INFO:tasks.workunit.client.0.smithi083.stdout: directio = off 2024-04-23T14:50:30.272 INFO:tasks.workunit.client.0.smithi083.stdout: alignedio = on 2024-04-23T14:50:30.272 INFO:tasks.workunit.client.0.smithi083.stdout: bufferedio = off 2024-04-23T14:50:30.272 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-23T14:50:30.272 INFO:tasks.workunit.client.0.smithi083.stdout: aging is off 2024-04-23T14:50:30.273 INFO:tasks.workunit.client.0.smithi083.stdout: current utilization = 0.00% 2024-04-23T14:50:30.273 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-23T14:50:30.273 INFO:tasks.workunit.client.0.smithi083.stdout:checking existing fs: . 2024-04-23T14:50:30.273 INFO:tasks.workunit.client.0.smithi083.stdout:recreating new fileset 2024-04-23T14:50:30.273 INFO:tasks.workunit.client.0.smithi083.stderr:opendir: No such file or directory 2024-04-23T14:50:31.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:31 smithi173 ceph-mon[32443]: pgmap v64: 97 pgs: 97 active+clean; 9.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 600 KiB/s wr, 65 op/s 2024-04-23T14:50:31.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:31 smithi203 ceph-mon[32800]: pgmap v64: 97 pgs: 97 active+clean; 9.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 600 KiB/s wr, 65 op/s 2024-04-23T14:50:32.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:31 smithi083 ceph-mon[27867]: pgmap v64: 97 pgs: 97 active+clean; 9.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 600 KiB/s wr, 65 op/s 2024-04-23T14:50:33.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:33 smithi173 ceph-mon[32443]: pgmap v65: 97 pgs: 97 active+clean; 9.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 183 KiB/s wr, 18 op/s 2024-04-23T14:50:33.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:33 smithi203 ceph-mon[32800]: pgmap v65: 97 pgs: 97 active+clean; 9.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 183 KiB/s wr, 18 op/s 2024-04-23T14:50:34.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:33 smithi083 ceph-mon[27867]: pgmap v65: 97 pgs: 97 active+clean; 9.5 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 183 KiB/s wr, 18 op/s 2024-04-23T14:50:35.773 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:35 smithi203 ceph-mon[32800]: pgmap v66: 97 pgs: 97 active+clean; 64 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 4.8 MiB/s wr, 77 op/s 2024-04-23T14:50:35.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:35 smithi173 ceph-mon[32443]: pgmap v66: 97 pgs: 97 active+clean; 64 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 4.8 MiB/s wr, 77 op/s 2024-04-23T14:50:36.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:35 smithi083 ceph-mon[27867]: pgmap v66: 97 pgs: 97 active+clean; 64 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 4.8 MiB/s wr, 77 op/s 2024-04-23T14:50:37.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:37 smithi173 ceph-mon[32443]: pgmap v67: 97 pgs: 97 active+clean; 64 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 4.7 MiB/s wr, 67 op/s 2024-04-23T14:50:37.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:37 smithi203 ceph-mon[32800]: pgmap v67: 97 pgs: 97 active+clean; 64 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 4.7 MiB/s wr, 67 op/s 2024-04-23T14:50:38.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:37 smithi083 ceph-mon[27867]: pgmap v67: 97 pgs: 97 active+clean; 64 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 4.7 MiB/s wr, 67 op/s 2024-04-23T14:50:39.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:39 smithi173 ceph-mon[32443]: pgmap v68: 97 pgs: 97 active+clean; 64 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 4.7 MiB/s wr, 67 op/s 2024-04-23T14:50:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:39 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:50:39.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:39 smithi203 ceph-mon[32800]: pgmap v68: 97 pgs: 97 active+clean; 64 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 4.7 MiB/s wr, 67 op/s 2024-04-23T14:50:39.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:39 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:50:40.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:39 smithi083 ceph-mon[27867]: pgmap v68: 97 pgs: 97 active+clean; 64 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 4.7 MiB/s wr, 67 op/s 2024-04-23T14:50:40.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:39 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:50:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:41 smithi173 ceph-mon[32443]: pgmap v69: 97 pgs: 97 active+clean; 453 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 37 MiB/s wr, 90 op/s 2024-04-23T14:50:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:41 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:50:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:41 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:50:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:41 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:50:41.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:41 smithi203 ceph-mon[32800]: pgmap v69: 97 pgs: 97 active+clean; 453 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 37 MiB/s wr, 90 op/s 2024-04-23T14:50:41.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:41 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:50:41.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:41 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:50:41.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:41 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:50:42.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:41 smithi083 ceph-mon[27867]: pgmap v69: 97 pgs: 97 active+clean; 453 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 37 MiB/s wr, 90 op/s 2024-04-23T14:50:42.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:41 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:50:42.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:41 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:50:42.011 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:41 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:50:43.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:43 smithi173 ceph-mon[32443]: pgmap v70: 97 pgs: 97 active+clean; 453 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 37 MiB/s wr, 82 op/s 2024-04-23T14:50:43.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:43 smithi203 ceph-mon[32800]: pgmap v70: 97 pgs: 97 active+clean; 453 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 37 MiB/s wr, 82 op/s 2024-04-23T14:50:44.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:43 smithi083 ceph-mon[27867]: pgmap v70: 97 pgs: 97 active+clean; 453 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 37 MiB/s wr, 82 op/s 2024-04-23T14:50:45.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:45 smithi173 ceph-mon[32443]: pgmap v71: 97 pgs: 97 active+clean; 1.8 GiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 153 MiB/s wr, 123 op/s 2024-04-23T14:50:45.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:45 smithi203 ceph-mon[32800]: pgmap v71: 97 pgs: 97 active+clean; 1.8 GiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 153 MiB/s wr, 123 op/s 2024-04-23T14:50:46.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:45 smithi083 ceph-mon[27867]: pgmap v71: 97 pgs: 97 active+clean; 1.8 GiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 153 MiB/s wr, 123 op/s 2024-04-23T14:50:46.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:50:46.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:50:46.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.896+0000 7f9d67400640 1 -- 172.21.15.83:0/1998824449 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6002f300 con 0x7f9d68109580 2024-04-23T14:50:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.896+0000 7f9d6ea00640 1 -- 172.21.15.83:0/1998824449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d68109580 msgr2=0x7f9d68111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.896+0000 7f9d6ea00640 1 --2- 172.21.15.83:0/1998824449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d68109580 0x7f9d68111150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009be0 tx=0x7f9d6002a6f0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.896+0000 7f9d6ea00640 1 -- 172.21.15.83:0/1998824449 shutdown_connections 2024-04-23T14:50:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.896+0000 7f9d6ea00640 1 --2- 172.21.15.83:0/1998824449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d68109580 0x7f9d68111150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.896+0000 7f9d6ea00640 1 --2- 172.21.15.83:0/1998824449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d68104e60 0x7f9d68108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.896+0000 7f9d6ea00640 1 --2- 172.21.15.83:0/1998824449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d681044b0 0x7f9d68104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.896+0000 7f9d6ea00640 1 -- 172.21.15.83:0/1998824449 >> 172.21.15.83:0/1998824449 conn(0x7f9d680ffd90 msgr2=0x7f9d681021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:46.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.897+0000 7f9d6ea00640 1 -- 172.21.15.83:0/1998824449 shutdown_connections 2024-04-23T14:50:46.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.897+0000 7f9d6ea00640 1 -- 172.21.15.83:0/1998824449 wait complete. 2024-04-23T14:50:46.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.898+0000 7f9d6ea00640 1 Processor -- start 2024-04-23T14:50:46.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.899+0000 7f9d6ea00640 1 -- start start 2024-04-23T14:50:46.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.900+0000 7f9d6ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d681044b0 0x7f9d68110fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.900+0000 7f9d6ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d68109580 0x7f9d6810e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.900+0000 7f9d6ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d6810e5f0 0x7f9d6810ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.900+0000 7f9d6ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68114670 con 0x7f9d6810e5f0 2024-04-23T14:50:46.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.900+0000 7f9d6ea00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681144f0 con 0x7f9d681044b0 2024-04-23T14:50:46.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.900+0000 7f9d6ea00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681147f0 con 0x7f9d68109580 2024-04-23T14:50:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.901+0000 7f9d67e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d681044b0 0x7f9d68110fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.901+0000 7f9d67e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d681044b0 0x7f9d68110fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54674/0 (socket says 172.21.15.83:54674) 2024-04-23T14:50:46.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.901+0000 7f9d67e00640 1 -- 172.21.15.83:0/3191796823 learned_addr learned my addr 172.21.15.83:0/3191796823 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:46.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.902+0000 7f9d67e00640 1 -- 172.21.15.83:0/3191796823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d68109580 msgr2=0x7f9d6810e0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:50:46.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.902+0000 7f9d67e00640 1 --2- 172.21.15.83:0/3191796823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d68109580 0x7f9d6810e0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.902+0000 7f9d67e00640 1 -- 172.21.15.83:0/3191796823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d6810e5f0 msgr2=0x7f9d6810ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:46.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.902+0000 7f9d67e00640 1 --2- 172.21.15.83:0/3191796823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d6810e5f0 0x7f9d6810ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.902+0000 7f9d67e00640 1 -- 172.21.15.83:0/3191796823 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d681ab5a0 con 0x7f9d681044b0 2024-04-23T14:50:46.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.905+0000 7f9d67e00640 1 --2- 172.21.15.83:0/3191796823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d681044b0 0x7f9d68110fc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000d950 tx=0x7f9d5000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:46.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.906+0000 7f9d65600640 1 -- 172.21.15.83:0/3191796823 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50017070 con 0x7f9d681044b0 2024-04-23T14:50:46.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.906+0000 7f9d65600640 1 -- 172.21.15.83:0/3191796823 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d50009c30 con 0x7f9d681044b0 2024-04-23T14:50:46.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.906+0000 7f9d6ea00640 1 -- 172.21.15.83:0/3191796823 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d681ab890 con 0x7f9d681044b0 2024-04-23T14:50:46.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.906+0000 7f9d65600640 1 -- 172.21.15.83:0/3191796823 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5000f6b0 con 0x7f9d681044b0 2024-04-23T14:50:46.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.906+0000 7f9d6ea00640 1 -- 172.21.15.83:0/3191796823 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d681abe30 con 0x7f9d681044b0 2024-04-23T14:50:46.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.910+0000 7f9d717c3740 1 Processor -- start 2024-04-23T14:50:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.910+0000 7f9d717c3740 1 -- start start 2024-04-23T14:50:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.911+0000 7f9d717c3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb957860 0x5584eb957c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.911+0000 7f9d717c3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb958180 0x5584eb962310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.911+0000 7f9d717c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb962850 0x5584eb964c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.911+0000 7f9d717c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5584eb8bd1f0 con 0x5584eb962850 2024-04-23T14:50:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.911+0000 7f9d717c3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5584eb863350 con 0x5584eb958180 2024-04-23T14:50:46.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.911+0000 7f9d717c3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5584eb8b91b0 con 0x5584eb957860 2024-04-23T14:50:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.911+0000 7f9d6ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb957860 0x5584eb957c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.912+0000 7f9d6ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb957860 0x5584eb957c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55288/0 (socket says 172.21.15.83:55288) 2024-04-23T14:50:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.912+0000 7f9d6ca00640 1 -- 172.21.15.83:0/2097277951 learned_addr learned my addr 172.21.15.83:0/2097277951 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.912+0000 7f9d5f400640 1 --2- 172.21.15.83:0/2097277951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb962850 0x5584eb964c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.912+0000 7f9d6ca00640 1 -- 172.21.15.83:0/2097277951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb958180 msgr2=0x5584eb962310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:50:46.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.912+0000 7f9d6ca00640 1 --2- 172.21.15.83:0/2097277951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb958180 0x5584eb962310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.912+0000 7f9d6ca00640 1 -- 172.21.15.83:0/2097277951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb962850 msgr2=0x5584eb964c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:46.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.912+0000 7f9d6ca00640 1 --2- 172.21.15.83:0/2097277951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb962850 0x5584eb964c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.912+0000 7f9d6ca00640 1 -- 172.21.15.83:0/2097277951 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584eb8b7f50 con 0x5584eb957860 2024-04-23T14:50:46.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.913+0000 7f9d65600640 1 -- 172.21.15.83:0/3191796823 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9d5000f850 con 0x7f9d681044b0 2024-04-23T14:50:46.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.916+0000 7f9d65600640 1 --2- 172.21.15.83:0/3191796823 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9d4c0a3e20 0x7f9d4c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.916+0000 7f9d65600640 1 -- 172.21.15.83:0/3191796823 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9d500d04b0 con 0x7f9d681044b0 2024-04-23T14:50:46.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.918+0000 7f9d67e00640 1 --2- 172.21.15.83:0/3191796823 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9d4c0a3e20 0x7f9d4c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.918+0000 7f9d6ca00640 1 --2- 172.21.15.83:0/2097277951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb957860 0x5584eb957c40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9d60004af0 tx=0x7f9d6002f390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7926cb575f46524f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:46.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.919+0000 7f9d5d200640 1 -- 172.21.15.83:0/2097277951 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60040020 con 0x5584eb957860 2024-04-23T14:50:46.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.919+0000 7f9d5d200640 1 -- 172.21.15.83:0/2097277951 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d60003e50 con 0x5584eb957860 2024-04-23T14:50:46.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.919+0000 7f9d5d200640 1 -- 172.21.15.83:0/2097277951 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60036040 con 0x5584eb957860 2024-04-23T14:50:46.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.920+0000 7f9d67e00640 1 --2- 172.21.15.83:0/3191796823 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9d4c0a3e20 0x7f9d4c0a62e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5584eb958e20 tx=0x7f9d5000bf60 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:46.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.920+0000 7f9d717c3740 1 -- 172.21.15.83:0/2097277951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb957860 msgr2=0x5584eb957c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:46.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.920+0000 7f9d717c3740 1 --2- 172.21.15.83:0/2097277951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb957860 0x5584eb957c40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9d60004af0 tx=0x7f9d6002f390 comp rx=0 tx=0).stop 2024-04-23T14:50:46.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.921+0000 7f9d717c3740 1 -- 172.21.15.83:0/2097277951 shutdown_connections 2024-04-23T14:50:46.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.921+0000 7f9d717c3740 1 --2- 172.21.15.83:0/2097277951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb962850 0x5584eb964c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.921+0000 7f9d717c3740 1 --2- 172.21.15.83:0/2097277951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb958180 0x5584eb962310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.921+0000 7f9d717c3740 1 --2- 172.21.15.83:0/2097277951 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb957860 0x5584eb957c40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.921+0000 7f9d717c3740 1 -- 172.21.15.83:0/2097277951 >> 172.21.15.83:0/2097277951 conn(0x5584eb953b80 msgr2=0x5584eb953f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:46.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.922+0000 7f9d717c3740 1 -- 172.21.15.83:0/2097277951 shutdown_connections 2024-04-23T14:50:46.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.922+0000 7f9d717c3740 1 -- 172.21.15.83:0/2097277951 wait complete. 2024-04-23T14:50:46.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.923+0000 7f9d717c3740 1 -- start start 2024-04-23T14:50:46.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.924+0000 7f9d717c3740 1 Processor -- start 2024-04-23T14:50:46.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.925+0000 7f9d717c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 0x5584eb95f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.925+0000 7f9d717c3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb958180 0x5584eb95fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.925+0000 7f9d6ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 0x5584eb95f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.926+0000 7f9d6ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 0x5584eb95f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42082/0 (socket says 172.21.15.83:42082) 2024-04-23T14:50:46.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.926+0000 7f9d6ca00640 1 -- 172.21.15.83:0/3358265596 learned_addr learned my addr 172.21.15.83:0/3358265596 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:46.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.926+0000 7f9d717c3740 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb962850 0x5584eb9601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.927+0000 7f9d5f400640 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb958180 0x5584eb95fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.927+0000 7f9d717c3740 1 -- 172.21.15.83:0/3358265596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5584eb9765f0 con 0x5584eb957860 2024-04-23T14:50:46.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.927+0000 7f9d717c3740 1 -- 172.21.15.83:0/3358265596 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5584eb876690 con 0x5584eb962850 2024-04-23T14:50:46.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.927+0000 7f9d717c3740 1 -- 172.21.15.83:0/3358265596 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5584eb976770 con 0x5584eb958180 2024-04-23T14:50:46.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.927+0000 7f9d717c3740 10 client.0 _ll_register_callbacks cb 0x5584eb8b2650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:46.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.928+0000 7f9d6ca00640 1 -- 172.21.15.83:0/3358265596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb958180 msgr2=0x5584eb95fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:46.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.929+0000 7f9d6ca00640 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb958180 0x5584eb95fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.929+0000 7f9d6ca00640 1 -- 172.21.15.83:0/3358265596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb962850 msgr2=0x5584eb9601d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T14:50:46.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.929+0000 7f9d6ca00640 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb962850 0x5584eb9601d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.929+0000 7f9d6ca00640 1 -- 172.21.15.83:0/3358265596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584eb875920 con 0x5584eb957860 2024-04-23T14:50:46.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.930+0000 7f9d5f400640 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb958180 0x5584eb95fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:46.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.930+0000 7f9d6ca00640 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 0x5584eb95f750 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9d60003bf0 tx=0x7f9d60009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661c7bad978739a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:46.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.932+0000 7f9d55600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:50:46.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.933+0000 7f9d55600640 1 -- 172.21.15.83:0/3358265596 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60040030 con 0x5584eb957860 2024-04-23T14:50:46.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.933+0000 7f9d55600640 1 -- 172.21.15.83:0/3358265596 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9d60004000 con 0x5584eb957860 2024-04-23T14:50:46.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.933+0000 7f9d717c3740 1 -- 172.21.15.83:0/3358265596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d60003e50 con 0x5584eb957860 2024-04-23T14:50:46.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.935+0000 7f9d55600640 1 -- 172.21.15.83:0/3358265596 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d600097e0 con 0x5584eb957860 2024-04-23T14:50:46.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.935+0000 7f9d717c3740 10 client.15687 fetch_fsmap learned FSMap version 26 2024-04-23T14:50:46.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.935+0000 7f9d717c3740 1 -- 172.21.15.83:0/3358265596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584eb6997d0 con 0x5584eb957860 2024-04-23T14:50:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.936+0000 7f9d55600640 1 -- 172.21.15.83:0/3358265596 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d60003e50 con 0x5584eb957860 2024-04-23T14:50:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.937+0000 7f9d55600640 1 -- 172.21.15.83:0/3358265596 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f9d6005a020 con 0x5584eb957860 2024-04-23T14:50:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.937+0000 7f9d717c3740 10 client.15687 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:50:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.937+0000 7f9d717c3740 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:46.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.937+0000 7f9d717c3740 20 client.15687 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.937+0000 7f9d717c3740 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5584eb983760 0x5584eb985b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.937+0000 7f9d717c3740 4 client.15687 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.937+0000 7f9d717c3740 1 -- 172.21.15.83:0/3358265596 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584eb8cac10 con 0x5584eb983760 2024-04-23T14:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.937+0000 7f9d5f400640 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5584eb983760 0x5584eb985b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.938+0000 7f9d5f400640 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5584eb983760 0x5584eb985b50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e46479316f139c9 server_cookie=d1832aafaec58ca0 in_seq=0 out_seq=0 2024-04-23T14:50:46.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.938+0000 7f9d55600640 10 client.15687 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.939+0000 7f9d55600640 1 -- 172.21.15.83:0/3358265596 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584eb8cac10 con 0x5584eb983760 2024-04-23T14:50:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.939+0000 7f9d55600640 10 client.15687 handle_command_reply: tid=0 2024-04-23T14:50:46.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.939+0000 7f9d717c3740 1 client.15687 shutdown 2024-04-23T14:50:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.940+0000 7f9d717c3740 1 -- 172.21.15.83:0/3358265596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 msgr2=0x5584eb95f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.940+0000 7f9d717c3740 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 0x5584eb95f750 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9d60003bf0 tx=0x7f9d60009500 comp rx=0 tx=0).stop 2024-04-23T14:50:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.942+0000 7f9d717c3740 1 -- 172.21.15.83:0/3358265596 shutdown_connections 2024-04-23T14:50:46.942 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.942+0000 7f9d717c3740 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5584eb983760 0x5584eb985b50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.942+0000 7f9d717c3740 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb962850 0x5584eb9601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.942+0000 7f9d717c3740 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb958180 0x5584eb95fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.942+0000 7f9d717c3740 1 --2- 172.21.15.83:0/3358265596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 0x5584eb95f750 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.942+0000 7f9d717c3740 1 -- 172.21.15.83:0/3358265596 >> 172.21.15.83:0/3358265596 conn(0x5584eb953b80 msgr2=0x5584eb959980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.942+0000 7f9d55600640 0 client.15687 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.942+0000 7f9d717c3740 1 -- 172.21.15.83:0/3358265596 shutdown_connections 2024-04-23T14:50:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.942+0000 7f9d717c3740 1 -- 172.21.15.83:0/3358265596 wait complete. 2024-04-23T14:50:46.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.942+0000 7f9d717c3740 20 client.15687 trim_cache size 0 max 16384 2024-04-23T14:50:46.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.967+0000 7f9d717c3740 1 Processor -- start 2024-04-23T14:50:46.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.968+0000 7f9d717c3740 1 -- start start 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.968+0000 7f9d717c3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb957860 0x5584eb97f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.968+0000 7f9d717c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb958180 0x5584eb95fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.968+0000 7f9d717c3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb962850 0x5584eb952aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.968+0000 7f9d717c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681ae490 con 0x5584eb958180 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.968+0000 7f9d717c3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68073e10 con 0x5584eb957860 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.968+0000 7f9d717c3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68071c80 con 0x5584eb962850 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.968+0000 7f9d6ca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb957860 0x5584eb97f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.968+0000 7f9d6ca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb957860 0x5584eb97f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54682/0 (socket says 172.21.15.83:54682) 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.968+0000 7f9d6ca00640 1 -- 172.21.15.83:0/2942947564 learned_addr learned my addr 172.21.15.83:0/2942947564 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.968+0000 7f9d5f400640 1 --2- 172.21.15.83:0/2942947564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb958180 0x5584eb95fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.969+0000 7f9d6ca00640 1 --2- 172.21.15.83:0/2942947564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb962850 0x5584eb952aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.969+0000 7f9d5f400640 1 -- 172.21.15.83:0/2942947564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb962850 msgr2=0x5584eb952aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:46.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.969+0000 7f9d5f400640 1 --2- 172.21.15.83:0/2942947564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb962850 0x5584eb952aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.969+0000 7f9d5f400640 1 -- 172.21.15.83:0/2942947564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb957860 msgr2=0x5584eb97f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:46.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.969+0000 7f9d5f400640 1 --2- 172.21.15.83:0/2942947564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb957860 0x5584eb97f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.969+0000 7f9d5f400640 1 -- 172.21.15.83:0/2942947564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584eb5cf3f0 con 0x5584eb958180 2024-04-23T14:50:46.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.969+0000 7f9d5f400640 1 --2- 172.21.15.83:0/2942947564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb958180 0x5584eb95fff0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x5584eb954d10 tx=0x7f9d58005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9270a990fb5898ab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:46.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.969+0000 7f9d57e00640 1 -- 172.21.15.83:0/2942947564 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58003b00 con 0x5584eb958180 2024-04-23T14:50:46.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.969+0000 7f9d57e00640 1 -- 172.21.15.83:0/2942947564 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d58005530 con 0x5584eb958180 2024-04-23T14:50:46.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.969+0000 7f9d57e00640 1 -- 172.21.15.83:0/2942947564 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58004e10 con 0x5584eb958180 2024-04-23T14:50:46.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.970+0000 7f9d717c3740 1 -- 172.21.15.83:0/2942947564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb958180 msgr2=0x5584eb95fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:46.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.970+0000 7f9d717c3740 1 --2- 172.21.15.83:0/2942947564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb958180 0x5584eb95fff0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x5584eb954d10 tx=0x7f9d58005a70 comp rx=0 tx=0).stop 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.970+0000 7f9d717c3740 1 -- 172.21.15.83:0/2942947564 shutdown_connections 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.970+0000 7f9d717c3740 1 --2- 172.21.15.83:0/2942947564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb962850 0x5584eb952aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.970+0000 7f9d717c3740 1 --2- 172.21.15.83:0/2942947564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb958180 0x5584eb95fff0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.970+0000 7f9d717c3740 1 --2- 172.21.15.83:0/2942947564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb957860 0x5584eb97f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.970+0000 7f9d717c3740 1 -- 172.21.15.83:0/2942947564 >> 172.21.15.83:0/2942947564 conn(0x5584eb953b80 msgr2=0x5584eb952390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.970+0000 7f9d717c3740 1 -- 172.21.15.83:0/2942947564 shutdown_connections 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.970+0000 7f9d717c3740 1 -- 172.21.15.83:0/2942947564 wait complete. 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.971+0000 7f9d717c3740 1 -- start start 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.971+0000 7f9d717c3740 1 Processor -- start 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.971+0000 7f9d717c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 0x5584eb97b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.971+0000 7f9d717c3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb958180 0x5584eb97bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.971+0000 7f9d717c3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb962850 0x5584eb95bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.971+0000 7f9d717c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d680722d0 con 0x5584eb957860 2024-04-23T14:50:46.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.971+0000 7f9d717c3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68073740 con 0x5584eb958180 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.971+0000 7f9d717c3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9d680729a0 con 0x5584eb962850 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.972+0000 7f9d717c3740 10 client.0 _ll_register_callbacks cb 0x7f9d58005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.972+0000 7f9d6ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 0x5584eb97b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.972+0000 7f9d5f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb958180 0x5584eb97bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.972+0000 7f9d6ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 0x5584eb97b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42102/0 (socket says 172.21.15.83:42102) 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.972+0000 7f9d6ca00640 1 -- 172.21.15.83:0/1137402938 learned_addr learned my addr 172.21.15.83:0/1137402938 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.972+0000 7f9d6ca00640 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb962850 0x5584eb95bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.972+0000 7f9d5f400640 1 -- 172.21.15.83:0/1137402938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb962850 msgr2=0x5584eb95bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.972+0000 7f9d5f400640 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb962850 0x5584eb95bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.972+0000 7f9d5f400640 1 -- 172.21.15.83:0/1137402938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 msgr2=0x5584eb97b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.972+0000 7f9d5f400640 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 0x5584eb97b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.972+0000 7f9d5f400640 1 -- 172.21.15.83:0/1137402938 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584eb58f0f0 con 0x5584eb958180 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.972+0000 7f9d6ca00640 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb962850 0x5584eb95bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:50:46.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.973+0000 7f9d5f400640 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb958180 0x5584eb97bf00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5584eb982510 tx=0x7f9d580049d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9feb4bf9a214b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.973+0000 7f9d55600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.973+0000 7f9d55600640 1 -- 172.21.15.83:0/1137402938 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58004d50 con 0x5584eb958180 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.973+0000 7f9d717c3740 1 -- 172.21.15.83:0/1137402938 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d58005530 con 0x5584eb958180 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.973+0000 7f9d55600640 1 -- 172.21.15.83:0/1137402938 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f9d58005050 con 0x5584eb958180 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.973+0000 7f9d55600640 1 -- 172.21.15.83:0/1137402938 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d58036050 con 0x5584eb958180 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.973+0000 7f9d717c3740 10 client.25811 fetch_fsmap learned FSMap version 26 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.973+0000 7f9d717c3740 1 -- 172.21.15.83:0/1137402938 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584eb882810 con 0x5584eb958180 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.973+0000 7f9d55600640 1 -- 172.21.15.83:0/1137402938 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d58005530 con 0x5584eb958180 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.974+0000 7f9d55600640 1 -- 172.21.15.83:0/1137402938 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f9d60030720 con 0x5584eb958180 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.974+0000 7f9d717c3740 10 client.25811 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.974+0000 7f9d717c3740 10 client.25811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.974+0000 7f9d717c3740 20 client.25811 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:46.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.974+0000 7f9d717c3740 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5584eb9e9760 0x5584eb9ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.974+0000 7f9d717c3740 4 client.25811 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.974+0000 7f9d717c3740 1 -- 172.21.15.83:0/1137402938 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5584eb8c4a80 con 0x5584eb9e9760 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.974+0000 7f9d6ca00640 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5584eb9e9760 0x5584eb9ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.974+0000 7f9d6ca00640 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5584eb9e9760 0x5584eb9ebb80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8196d1ed230745fa server_cookie=e36fd7b3f9b86c9 in_seq=0 out_seq=0 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.974+0000 7f9d55600640 10 client.25811 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d55600640 1 -- 172.21.15.83:0/1137402938 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5584eb8c4a80 con 0x5584eb9e9760 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d55600640 10 client.25811 handle_command_reply: tid=0 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d717c3740 1 client.25811 shutdown 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d717c3740 1 -- 172.21.15.83:0/1137402938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb958180 msgr2=0x5584eb97bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d717c3740 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb958180 0x5584eb97bf00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5584eb982510 tx=0x7f9d580049d0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d717c3740 1 -- 172.21.15.83:0/1137402938 shutdown_connections 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d717c3740 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5584eb9e9760 0x5584eb9ebb80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d717c3740 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584eb962850 0x5584eb95bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d717c3740 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5584eb958180 0x5584eb97bf00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d55600640 0 client.25811 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:46.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d717c3740 1 --2- 172.21.15.83:0/1137402938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5584eb957860 0x5584eb97b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d717c3740 1 -- 172.21.15.83:0/1137402938 >> 172.21.15.83:0/1137402938 conn(0x5584eb953b80 msgr2=0x5584eb952c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:46.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d717c3740 1 -- 172.21.15.83:0/1137402938 shutdown_connections 2024-04-23T14:50:46.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d717c3740 1 -- 172.21.15.83:0/1137402938 wait complete. 2024-04-23T14:50:46.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.975+0000 7f9d717c3740 20 client.25811 trim_cache size 0 max 16384 2024-04-23T14:50:46.980 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:50:46.980 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:50:46.980 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:50:46.980 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:50:46.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.976+0000 7f9d6ea00640 1 -- 172.21.15.83:0/3191796823 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9d4c0a3e20 msgr2=0x7f9d4c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:46.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.976+0000 7f9d6ea00640 1 --2- 172.21.15.83:0/3191796823 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9d4c0a3e20 0x7f9d4c0a62e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5584eb958e20 tx=0x7f9d5000bf60 comp rx=0 tx=0).stop 2024-04-23T14:50:46.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.976+0000 7f9d6ea00640 1 -- 172.21.15.83:0/3191796823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d681044b0 msgr2=0x7f9d68110fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:46.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.976+0000 7f9d6ea00640 1 --2- 172.21.15.83:0/3191796823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d681044b0 0x7f9d68110fc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000d950 tx=0x7f9d5000de10 comp rx=0 tx=0).stop 2024-04-23T14:50:46.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.976+0000 7f9d6ea00640 1 -- 172.21.15.83:0/3191796823 shutdown_connections 2024-04-23T14:50:46.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.976+0000 7f9d6ea00640 1 --2- 172.21.15.83:0/3191796823 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9d4c0a3e20 0x7f9d4c0a62e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.976+0000 7f9d6ea00640 1 --2- 172.21.15.83:0/3191796823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d6810e5f0 0x7f9d6810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.976+0000 7f9d6ea00640 1 --2- 172.21.15.83:0/3191796823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9d68109580 0x7f9d6810e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.976+0000 7f9d6ea00640 1 --2- 172.21.15.83:0/3191796823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d681044b0 0x7f9d68110fc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:46.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.976+0000 7f9d6ea00640 1 -- 172.21.15.83:0/3191796823 >> 172.21.15.83:0/3191796823 conn(0x7f9d680ffd90 msgr2=0x7f9d681008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:46.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.976+0000 7f9d6ea00640 1 -- 172.21.15.83:0/3191796823 shutdown_connections 2024-04-23T14:50:46.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:46.976+0000 7f9d6ea00640 1 -- 172.21.15.83:0/3191796823 wait complete. 2024-04-23T14:50:46.992 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:50:46.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:50:47.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.161+0000 7f5802c00640 1 -- 172.21.15.83:0/2243740227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc109360 msgr2=0x7f57fc110f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.161+0000 7f5802c00640 1 --2- 172.21.15.83:0/2243740227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc109360 0x7f57fc110f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f57f4009be0 tx=0x7f57f402a6c0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.161+0000 7f5802c00640 1 -- 172.21.15.83:0/2243740227 shutdown_connections 2024-04-23T14:50:47.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.161+0000 7f5802c00640 1 --2- 172.21.15.83:0/2243740227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc109360 0x7f57fc110f30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.161+0000 7f5802c00640 1 --2- 172.21.15.83:0/2243740227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc104c40 0x7f57fc108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.161+0000 7f5802c00640 1 --2- 172.21.15.83:0/2243740227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57fc104290 0x7f57fc104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.161+0000 7f5802c00640 1 -- 172.21.15.83:0/2243740227 >> 172.21.15.83:0/2243740227 conn(0x7f57fc0ffad0 msgr2=0x7f57fc101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.162+0000 7f5802c00640 1 -- 172.21.15.83:0/2243740227 shutdown_connections 2024-04-23T14:50:47.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.162+0000 7f5802c00640 1 -- 172.21.15.83:0/2243740227 wait complete. 2024-04-23T14:50:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.162+0000 7f5802c00640 1 Processor -- start 2024-04-23T14:50:47.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.163+0000 7f5802c00640 1 -- start start 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.163+0000 7f5802c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc104290 0x7f57fc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.163+0000 7f5802c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57fc109360 0x7f57fc1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.163+0000 7f5802c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc19a700 0x7f57fc19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.163+0000 7f5802c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc114400 con 0x7f57fc104290 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.163+0000 7f5802c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc114280 con 0x7f57fc19a700 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.163+0000 7f5802c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc114580 con 0x7f57fc109360 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.163+0000 7f5801800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc104290 0x7f57fc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.163+0000 7f5801800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc104290 0x7f57fc1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42132/0 (socket says 172.21.15.83:42132) 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.163+0000 7f5800e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57fc109360 0x7f57fc1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.163+0000 7f5801800640 1 -- 172.21.15.83:0/1263233140 learned_addr learned my addr 172.21.15.83:0/1263233140 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.163+0000 7f5802200640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc19a700 0x7f57fc19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.164+0000 7f5800e00640 1 -- 172.21.15.83:0/1263233140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc19a700 msgr2=0x7f57fc19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.164+0000 7f5800e00640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc19a700 0x7f57fc19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.164+0000 7f5800e00640 1 -- 172.21.15.83:0/1263233140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc104290 msgr2=0x7f57fc1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.164+0000 7f5800e00640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc104290 0x7f57fc1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.164+0000 7f5800e00640 1 -- 172.21.15.83:0/1263233140 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc19b300 con 0x7f57fc109360 2024-04-23T14:50:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.164+0000 7f5801800640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc104290 0x7f57fc1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.164+0000 7f5800e00640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57fc109360 0x7f57fc1a0b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57e800e9c0 tx=0x7f57e800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.164+0000 7f57fa000640 1 -- 172.21.15.83:0/1263233140 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e800ccc0 con 0x7f57fc109360 2024-04-23T14:50:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.164+0000 7f5802c00640 1 -- 172.21.15.83:0/1263233140 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57fc1a73e0 con 0x7f57fc109360 2024-04-23T14:50:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.164+0000 7f57fa000640 1 -- 172.21.15.83:0/1263233140 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57e8010910 con 0x7f57fc109360 2024-04-23T14:50:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.164+0000 7f57fa000640 1 -- 172.21.15.83:0/1263233140 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8018880 con 0x7f57fc109360 2024-04-23T14:50:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.164+0000 7f5802c00640 1 -- 172.21.15.83:0/1263233140 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57fc1a7840 con 0x7f57fc109360 2024-04-23T14:50:47.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.165+0000 7f5802c00640 1 -- 172.21.15.83:0/1263233140 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57fc105760 con 0x7f57fc109360 2024-04-23T14:50:47.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.169+0000 7f57fa000640 1 -- 172.21.15.83:0/1263233140 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f57e8010ac0 con 0x7f57fc109360 2024-04-23T14:50:47.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.170+0000 7f57fa000640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f57e40a3d20 0x7f57e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.170+0000 7f57fa000640 1 -- 172.21.15.83:0/1263233140 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f57e80d0ad0 con 0x7f57fc109360 2024-04-23T14:50:47.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.170+0000 7f5801800640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f57e40a3d20 0x7f57e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.171+0000 7f5801800640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f57e40a3d20 0x7f57e40a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f57f00096c0 tx=0x7f57f0009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.175+0000 7f57fa000640 1 -- 172.21.15.83:0/1263233140 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57e809cb30 con 0x7f57fc109360 2024-04-23T14:50:47.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.415+0000 7f5802c00640 1 -- 172.21.15.83:0/1263233140 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57fc104c40 con 0x7f57fc109360 2024-04-23T14:50:47.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.417+0000 7f57fa000640 1 -- 172.21.15.83:0/1263233140 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11867 (secure 0 0 0) 0x7f57e80bd0c0 con 0x7f57fc109360 2024-04-23T14:50:47.417 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:50:47.417 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:50:17.708313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:50:47.418 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-04-23T14:50:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.419+0000 7f5802c00640 1 -- 172.21.15.83:0/1263233140 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f57e40a3d20 msgr2=0x7f57e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.419+0000 7f5802c00640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f57e40a3d20 0x7f57e40a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f57f00096c0 tx=0x7f57f0009260 comp rx=0 tx=0).stop 2024-04-23T14:50:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.419+0000 7f5802c00640 1 -- 172.21.15.83:0/1263233140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57fc109360 msgr2=0x7f57fc1a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.419+0000 7f5802c00640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57fc109360 0x7f57fc1a0b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57e800e9c0 tx=0x7f57e800ee80 comp rx=0 tx=0).stop 2024-04-23T14:50:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.419+0000 7f5802c00640 1 -- 172.21.15.83:0/1263233140 shutdown_connections 2024-04-23T14:50:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.419+0000 7f5802c00640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57fc19a700 0x7f57fc19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.419+0000 7f5802c00640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f57fc109360 0x7f57fc1a0b80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.419+0000 7f5802c00640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f57e40a3d20 0x7f57e40a61e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.419+0000 7f5802c00640 1 --2- 172.21.15.83:0/1263233140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57fc104290 0x7f57fc1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.419+0000 7f5802c00640 1 -- 172.21.15.83:0/1263233140 >> 172.21.15.83:0/1263233140 conn(0x7f57fc0ffad0 msgr2=0x7f57fc101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.419+0000 7f5802c00640 1 -- 172.21.15.83:0/1263233140 shutdown_connections 2024-04-23T14:50:47.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.419+0000 7f5802c00640 1 -- 172.21.15.83:0/1263233140 wait complete. 2024-04-23T14:50:47.437 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:50:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.624+0000 7f293ca00640 1 -- 172.21.15.83:0/4102386121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2938102120 msgr2=0x7f2938102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.624+0000 7f293ca00640 1 --2- 172.21.15.83:0/4102386121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2938102120 0x7f2938102500 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2928009ca0 tx=0x7f292802a8c0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.624+0000 7f293ca00640 1 -- 172.21.15.83:0/4102386121 shutdown_connections 2024-04-23T14:50:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.624+0000 7f293ca00640 1 --2- 172.21.15.83:0/4102386121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29381071f0 0x7f293810edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.624+0000 7f293ca00640 1 --2- 172.21.15.83:0/4102386121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2938102ad0 0x7f2938106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.624+0000 7f293ca00640 1 --2- 172.21.15.83:0/4102386121 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2938102120 0x7f2938102500 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.624+0000 7f293ca00640 1 -- 172.21.15.83:0/4102386121 >> 172.21.15.83:0/4102386121 conn(0x7f29380fd9a0 msgr2=0x7f29380ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.625+0000 7f293ca00640 1 -- 172.21.15.83:0/4102386121 shutdown_connections 2024-04-23T14:50:47.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.625+0000 7f293ca00640 1 -- 172.21.15.83:0/4102386121 wait complete. 2024-04-23T14:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.625+0000 7f293ca00640 1 Processor -- start 2024-04-23T14:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.626+0000 7f293ca00640 1 -- start start 2024-04-23T14:50:47.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.626+0000 7f293ca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2938102120 0x7f293819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.626+0000 7f293ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29381071f0 0x7f293819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.626+0000 7f293ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29381985d0 0x7f2938198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.626+0000 7f293ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29381122d0 con 0x7f29381071f0 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.626+0000 7f293ca00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2938112150 con 0x7f2938102120 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.626+0000 7f293ca00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2938112450 con 0x7f29381985d0 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.626+0000 7f2937e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29381985d0 0x7f2938198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.626+0000 7f292ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29381071f0 0x7f293819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.626+0000 7f2937400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2938102120 0x7f293819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.626+0000 7f2937e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29381985d0 0x7f2938198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55354/0 (socket says 172.21.15.83:55354) 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.626+0000 7f2937e00640 1 -- 172.21.15.83:0/1464143591 learned_addr learned my addr 172.21.15.83:0/1464143591 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.627+0000 7f2937e00640 1 -- 172.21.15.83:0/1464143591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2938102120 msgr2=0x7f293819e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.627+0000 7f2937e00640 1 --2- 172.21.15.83:0/1464143591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2938102120 0x7f293819e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.627+0000 7f2937e00640 1 -- 172.21.15.83:0/1464143591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29381071f0 msgr2=0x7f293819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.627+0000 7f2937e00640 1 --2- 172.21.15.83:0/1464143591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29381071f0 0x7f293819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.627+0000 7f2937e00640 1 -- 172.21.15.83:0/1464143591 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29381990f0 con 0x7f29381985d0 2024-04-23T14:50:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.627+0000 7f2937e00640 1 --2- 172.21.15.83:0/1464143591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29381985d0 0x7f2938198a80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f293000d950 tx=0x7f293000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.627+0000 7f2934c00640 1 -- 172.21.15.83:0/1464143591 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930017030 con 0x7f29381985d0 2024-04-23T14:50:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.627+0000 7f293ca00640 1 -- 172.21.15.83:0/1464143591 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29381993e0 con 0x7f29381985d0 2024-04-23T14:50:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.627+0000 7f2934c00640 1 -- 172.21.15.83:0/1464143591 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2930004020 con 0x7f29381985d0 2024-04-23T14:50:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.627+0000 7f2934c00640 1 -- 172.21.15.83:0/1464143591 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930003c00 con 0x7f29381985d0 2024-04-23T14:50:47.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.627+0000 7f293ca00640 1 -- 172.21.15.83:0/1464143591 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29381a5630 con 0x7f29381985d0 2024-04-23T14:50:47.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.630+0000 7f2934c00640 1 -- 172.21.15.83:0/1464143591 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f293001a070 con 0x7f29381985d0 2024-04-23T14:50:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.630+0000 7f2940d7c740 1 Processor -- start 2024-04-23T14:50:47.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.631+0000 7f2940d7c740 1 -- start start 2024-04-23T14:50:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.631+0000 7f2940d7c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115cf50 0x558e0115d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.631+0000 7f2940d7c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e0115d870 0x558e01167a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.631+0000 7f2940d7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e01167f40 0x558e0116a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.631+0000 7f2940d7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558e010be1d0 con 0x558e01167f40 2024-04-23T14:50:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.631+0000 7f2940d7c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558e01064450 con 0x558e0115d870 2024-04-23T14:50:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.631+0000 7f2940d7c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558e010ba190 con 0x558e0115cf50 2024-04-23T14:50:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.631+0000 7f292ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e01167f40 0x558e0116a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.631+0000 7f292ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115cf50 0x558e0115d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.631+0000 7f292ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115cf50 0x558e0115d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55356/0 (socket says 172.21.15.83:55356) 2024-04-23T14:50:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.631+0000 7f292ea00640 1 -- 172.21.15.83:0/1135764443 learned_addr learned my addr 172.21.15.83:0/1135764443 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:47.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.631+0000 7f2937400640 1 --2- 172.21.15.83:0/1135764443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e0115d870 0x558e01167a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.632+0000 7f292ea00640 1 -- 172.21.15.83:0/1135764443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e0115d870 msgr2=0x558e01167a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.632+0000 7f292ea00640 1 --2- 172.21.15.83:0/1135764443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e0115d870 0x558e01167a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.632+0000 7f292ea00640 1 -- 172.21.15.83:0/1135764443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e01167f40 msgr2=0x558e0116a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.632+0000 7f292ea00640 1 --2- 172.21.15.83:0/1135764443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e01167f40 0x558e0116a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.632+0000 7f292ea00640 1 -- 172.21.15.83:0/1135764443 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e01121310 con 0x558e0115cf50 2024-04-23T14:50:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.632+0000 7f292ea00640 1 --2- 172.21.15.83:0/1135764443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115cf50 0x558e0115d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2920010900 tx=0x7f2920005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a33196340c34bb9e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.632+0000 7f2934c00640 1 --2- 172.21.15.83:0/1464143591 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f291c0a3d20 0x7f291c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.632+0000 7f2934c00640 1 -- 172.21.15.83:0/1464143591 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f29300cffb0 con 0x7f29381985d0 2024-04-23T14:50:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.632+0000 7f2927e00640 1 -- 172.21.15.83:0/1135764443 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920016070 con 0x558e0115cf50 2024-04-23T14:50:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.632+0000 7f2927e00640 1 -- 172.21.15.83:0/1135764443 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f292000f7c0 con 0x558e0115cf50 2024-04-23T14:50:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.632+0000 7f2927e00640 1 -- 172.21.15.83:0/1135764443 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920019040 con 0x558e0115cf50 2024-04-23T14:50:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.633+0000 7f2937400640 1 --2- 172.21.15.83:0/1464143591 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f291c0a3d20 0x7f291c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.633+0000 7f2940d7c740 1 -- 172.21.15.83:0/1135764443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115cf50 msgr2=0x558e0115d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.633+0000 7f2940d7c740 1 --2- 172.21.15.83:0/1135764443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115cf50 0x558e0115d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2920010900 tx=0x7f2920005f90 comp rx=0 tx=0).stop 2024-04-23T14:50:47.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.633+0000 7f2940d7c740 1 -- 172.21.15.83:0/1135764443 shutdown_connections 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.633+0000 7f2940d7c740 1 --2- 172.21.15.83:0/1135764443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e01167f40 0x558e0116a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.633+0000 7f2940d7c740 1 --2- 172.21.15.83:0/1135764443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e0115d870 0x558e01167a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.633+0000 7f2940d7c740 1 --2- 172.21.15.83:0/1135764443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115cf50 0x558e0115d330 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.633+0000 7f2940d7c740 1 -- 172.21.15.83:0/1135764443 >> 172.21.15.83:0/1135764443 conn(0x558e01159270 msgr2=0x558e01159650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.633+0000 7f2940d7c740 1 -- 172.21.15.83:0/1135764443 shutdown_connections 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.633+0000 7f2937400640 1 --2- 172.21.15.83:0/1464143591 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f291c0a3d20 0x7f291c0a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2928006010 tx=0x7f2928009820 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.633+0000 7f2940d7c740 1 -- 172.21.15.83:0/1135764443 wait complete. 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.634+0000 7f2940d7c740 1 -- start start 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.634+0000 7f2940d7c740 1 Processor -- start 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.634+0000 7f2940d7c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e0115cf50 0x558e01164d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.634+0000 7f2940d7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115d870 0x558e01165290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.634+0000 7f2940d7c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e01167f40 0x558e011657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.634+0000 7f2940d7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558e0116add0 con 0x558e0115d870 2024-04-23T14:50:47.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.634+0000 7f2940d7c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558e01077810 con 0x558e0115cf50 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.634+0000 7f2940d7c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558e0116af50 con 0x558e01167f40 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.634+0000 7f2940d7c740 10 client.0 _ll_register_callbacks cb 0x558e01045140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.635+0000 7f292ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115d870 0x558e01165290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.635+0000 7f292ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115d870 0x558e01165290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42186/0 (socket says 172.21.15.83:42186) 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.635+0000 7f2937e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e01167f40 0x558e011657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.635+0000 7f2937400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e0115cf50 0x558e01164d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.635+0000 7f292ea00640 1 -- 172.21.15.83:0/290065192 learned_addr learned my addr 172.21.15.83:0/290065192 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.635+0000 7f2937e00640 1 -- 172.21.15.83:0/290065192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e0115cf50 msgr2=0x558e01164d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.635+0000 7f2937e00640 1 --2- 172.21.15.83:0/290065192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e0115cf50 0x558e01164d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.635+0000 7f2937e00640 1 -- 172.21.15.83:0/290065192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115d870 msgr2=0x558e01165290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.635+0000 7f2937e00640 1 --2- 172.21.15.83:0/290065192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115d870 0x558e01165290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.635+0000 7f2937e00640 1 -- 172.21.15.83:0/290065192 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e010883f0 con 0x558e01167f40 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.635+0000 7f2937e00640 1 --2- 172.21.15.83:0/290065192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e01167f40 0x558e011657d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f29300cd710 tx=0x7f293009b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98243b8184eedf89 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.635+0000 7f2924c00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:50:47.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.636+0000 7f2924c00640 1 -- 172.21.15.83:0/290065192 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300cde60 con 0x558e01167f40 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.636+0000 7f2924c00640 1 -- 172.21.15.83:0/290065192 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f29300cd990 con 0x558e01167f40 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.636+0000 7f2940d7c740 1 -- 172.21.15.83:0/290065192 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f292000f7c0 con 0x558e01167f40 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.636+0000 7f2924c00640 1 -- 172.21.15.83:0/290065192 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f29300a0310 con 0x558e01167f40 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.636+0000 7f2940d7c740 10 client.25801 fetch_fsmap learned FSMap version 26 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.636+0000 7f2940d7c740 1 -- 172.21.15.83:0/290065192 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558e00dd03f0 con 0x558e01167f40 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.636+0000 7f2924c00640 1 -- 172.21.15.83:0/290065192 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f292000f7c0 con 0x558e01167f40 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.636+0000 7f2924c00640 1 -- 172.21.15.83:0/290065192 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f29300b5220 con 0x558e01167f40 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.636+0000 7f2940d7c740 10 client.25801 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.637+0000 7f2940d7c740 10 client.25801 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.637+0000 7f2940d7c740 20 client.25801 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.637+0000 7f2940d7c740 1 --2- 172.21.15.83:0/290065192 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558e01178240 0x558e0117a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.637+0000 7f2940d7c740 4 client.25801 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.637+0000 7f2940d7c740 1 -- 172.21.15.83:0/290065192 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e010cbbf0 con 0x558e01178240 2024-04-23T14:50:47.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.637+0000 7f2937400640 1 --2- 172.21.15.83:0/290065192 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558e01178240 0x558e0117a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.637+0000 7f2937400640 1 --2- 172.21.15.83:0/290065192 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558e01178240 0x558e0117a650 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=406b9cd5b05ddee4 server_cookie=96514bde8d552f8c in_seq=0 out_seq=0 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.637+0000 7f2924c00640 10 client.25801 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2924c00640 1 -- 172.21.15.83:0/290065192 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e010cbbf0 con 0x558e01178240 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2924c00640 10 client.25801 handle_command_reply: tid=0 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2940d7c740 1 client.25801 shutdown 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2940d7c740 1 -- 172.21.15.83:0/290065192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e01167f40 msgr2=0x558e011657d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2940d7c740 1 --2- 172.21.15.83:0/290065192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e01167f40 0x558e011657d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f29300cd710 tx=0x7f293009b980 comp rx=0 tx=0).stop 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2940d7c740 1 -- 172.21.15.83:0/290065192 shutdown_connections 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2940d7c740 1 --2- 172.21.15.83:0/290065192 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558e01178240 0x558e0117a650 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2940d7c740 1 --2- 172.21.15.83:0/290065192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e01167f40 0x558e011657d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2924c00640 0 client.25801 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2940d7c740 1 --2- 172.21.15.83:0/290065192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115d870 0x558e01165290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2940d7c740 1 --2- 172.21.15.83:0/290065192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e0115cf50 0x558e01164d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2940d7c740 1 -- 172.21.15.83:0/290065192 >> 172.21.15.83:0/290065192 conn(0x558e01159270 msgr2=0x558e0115f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2940d7c740 1 -- 172.21.15.83:0/290065192 shutdown_connections 2024-04-23T14:50:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2940d7c740 1 -- 172.21.15.83:0/290065192 wait complete. 2024-04-23T14:50:47.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.638+0000 7f2940d7c740 20 client.25801 trim_cache size 0 max 16384 2024-04-23T14:50:47.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.652+0000 7f2940d7c740 1 Processor -- start 2024-04-23T14:50:47.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f2940d7c740 1 -- start start 2024-04-23T14:50:47.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f2940d7c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115cf50 0x558e01173c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f2940d7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115d870 0x558e0115f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f2940d7c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e01167f40 0x558e01165420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f2940d7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29381ae150 con 0x558e0115d870 2024-04-23T14:50:47.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f2940d7c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29381ab9f0 con 0x558e01167f40 2024-04-23T14:50:47.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f2940d7c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f29381a9840 con 0x558e0115cf50 2024-04-23T14:50:47.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f292ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115d870 0x558e0115f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f2937e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e01167f40 0x558e01165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f2937e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e01167f40 0x558e01165420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54758/0 (socket says 172.21.15.83:54758) 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f292ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115d870 0x558e0115f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42202/0 (socket says 172.21.15.83:42202) 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f2937e00640 1 -- 172.21.15.83:0/173296704 learned_addr learned my addr 172.21.15.83:0/173296704 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.653+0000 7f2937400640 1 --2- 172.21.15.83:0/173296704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115cf50 0x558e01173c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.654+0000 7f2937e00640 1 -- 172.21.15.83:0/173296704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115cf50 msgr2=0x558e01173c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.654+0000 7f2937e00640 1 --2- 172.21.15.83:0/173296704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115cf50 0x558e01173c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.654+0000 7f2937e00640 1 -- 172.21.15.83:0/173296704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115d870 msgr2=0x558e0115f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.654+0000 7f2937e00640 1 --2- 172.21.15.83:0/173296704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115d870 0x558e0115f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.654+0000 7f2937e00640 1 -- 172.21.15.83:0/173296704 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e00e9a7d0 con 0x558e01167f40 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.654+0000 7f2937e00640 1 --2- 172.21.15.83:0/173296704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e01167f40 0x558e01165420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f293009bef0 tx=0x7f293009c0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c96bd4a4e6ff45cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.654+0000 7f2927400640 1 -- 172.21.15.83:0/173296704 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300cde60 con 0x558e01167f40 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.654+0000 7f2927400640 1 -- 172.21.15.83:0/173296704 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293009b3b0 con 0x558e01167f40 2024-04-23T14:50:47.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.654+0000 7f2927400640 1 -- 172.21.15.83:0/173296704 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300985d0 con 0x558e01167f40 2024-04-23T14:50:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.655+0000 7f2940d7c740 1 -- 172.21.15.83:0/173296704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e01167f40 msgr2=0x558e01165420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.655+0000 7f2940d7c740 1 --2- 172.21.15.83:0/173296704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e01167f40 0x558e01165420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f293009bef0 tx=0x7f293009c0e0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.655+0000 7f2940d7c740 1 -- 172.21.15.83:0/173296704 shutdown_connections 2024-04-23T14:50:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.655+0000 7f2940d7c740 1 --2- 172.21.15.83:0/173296704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e01167f40 0x558e01165420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.655+0000 7f2940d7c740 1 --2- 172.21.15.83:0/173296704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115d870 0x558e0115f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.655+0000 7f2940d7c740 1 --2- 172.21.15.83:0/173296704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115cf50 0x558e01173c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.655+0000 7f2940d7c740 1 -- 172.21.15.83:0/173296704 >> 172.21.15.83:0/173296704 conn(0x558e01159270 msgr2=0x558e01164d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.655+0000 7f2940d7c740 1 -- 172.21.15.83:0/173296704 shutdown_connections 2024-04-23T14:50:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.655+0000 7f2940d7c740 1 -- 172.21.15.83:0/173296704 wait complete. 2024-04-23T14:50:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.655+0000 7f2940d7c740 1 -- start start 2024-04-23T14:50:47.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f2940d7c740 1 Processor -- start 2024-04-23T14:50:47.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f2940d7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115cf50 0x558e01170390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f2940d7c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115d870 0x558e011613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f2940d7c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e01167f40 0x558e01161930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f2940d7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29381a9eb0 con 0x558e0115cf50 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f2940d7c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29381ab320 con 0x558e01167f40 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f2940d7c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f29381aa580 con 0x558e0115d870 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f2940d7c740 10 client.0 _ll_register_callbacks cb 0x7f293000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f2937400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115cf50 0x558e01170390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f292ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115d870 0x558e011613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f2937400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115cf50 0x558e01170390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42212/0 (socket says 172.21.15.83:42212) 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f2937400640 1 -- 172.21.15.83:0/703570131 learned_addr learned my addr 172.21.15.83:0/703570131 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.656+0000 7f2937e00640 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e01167f40 0x558e01161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.657+0000 7f292ea00640 1 -- 172.21.15.83:0/703570131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e01167f40 msgr2=0x558e01161930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.657+0000 7f292ea00640 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e01167f40 0x558e01161930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.657+0000 7f292ea00640 1 -- 172.21.15.83:0/703570131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115cf50 msgr2=0x558e01170390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.657+0000 7f292ea00640 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115cf50 0x558e01170390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.657+0000 7f292ea00640 1 -- 172.21.15.83:0/703570131 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e01161e70 con 0x558e0115d870 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.657+0000 7f2937400640 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115cf50 0x558e01170390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.657+0000 7f292ea00640 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115d870 0x558e011613f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x558e011645b0 tx=0x7f292000fcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28cdc4f5174601a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.657+0000 7f2924c00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.657+0000 7f2924c00640 1 -- 172.21.15.83:0/703570131 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920016040 con 0x558e0115d870 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.657+0000 7f2940d7c740 1 -- 172.21.15.83:0/703570131 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f293009b3b0 con 0x558e0115d870 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.657+0000 7f2924c00640 1 -- 172.21.15.83:0/703570131 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f292001f650 con 0x558e0115d870 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.658+0000 7f2924c00640 1 -- 172.21.15.83:0/703570131 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2920037050 con 0x558e0115d870 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.658+0000 7f2940d7c740 10 client.25807 fetch_fsmap learned FSMap version 26 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.658+0000 7f2940d7c740 1 -- 172.21.15.83:0/703570131 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558e011de3f0 con 0x558e0115d870 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.658+0000 7f2924c00640 1 -- 172.21.15.83:0/703570131 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293009b3b0 con 0x558e0115d870 2024-04-23T14:50:47.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.658+0000 7f2924c00640 1 -- 172.21.15.83:0/703570131 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f2920029890 con 0x558e0115d870 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.658+0000 7f2940d7c740 10 client.25807 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.658+0000 7f2940d7c740 10 client.25807 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.658+0000 7f2940d7c740 20 client.25807 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.658+0000 7f2940d7c740 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558e011de920 0x558e011e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.658+0000 7f2940d7c740 4 client.25807 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.658+0000 7f2940d7c740 1 -- 172.21.15.83:0/703570131 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e010c5a60 con 0x558e011de920 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.658+0000 7f2937400640 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558e011de920 0x558e011e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.659+0000 7f2937400640 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558e011de920 0x558e011e0d40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f46ce5a26110bee server_cookie=7a07d6af4921a2a6 in_seq=0 out_seq=0 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.659+0000 7f2924c00640 10 client.25807 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2924c00640 1 -- 172.21.15.83:0/703570131 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e010c5a60 con 0x558e011de920 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2924c00640 10 client.25807 handle_command_reply: tid=0 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2940d7c740 1 client.25807 shutdown 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2940d7c740 1 -- 172.21.15.83:0/703570131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115d870 msgr2=0x558e011613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2940d7c740 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115d870 0x558e011613f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x558e011645b0 tx=0x7f292000fcb0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2940d7c740 1 -- 172.21.15.83:0/703570131 shutdown_connections 2024-04-23T14:50:47.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2940d7c740 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558e011de920 0x558e011e0d40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2940d7c740 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558e01167f40 0x558e01161930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2924c00640 0 client.25807 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:50:47.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2940d7c740 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e0115d870 0x558e011613f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2940d7c740 1 --2- 172.21.15.83:0/703570131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e0115cf50 0x558e01170390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2940d7c740 1 -- 172.21.15.83:0/703570131 >> 172.21.15.83:0/703570131 conn(0x558e01159270 msgr2=0x558e01153f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2940d7c740 1 -- 172.21.15.83:0/703570131 shutdown_connections 2024-04-23T14:50:47.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2940d7c740 1 -- 172.21.15.83:0/703570131 wait complete. 2024-04-23T14:50:47.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.660+0000 7f2940d7c740 20 client.25807 trim_cache size 0 max 16384 2024-04-23T14:50:47.664 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:50:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.661+0000 7f293ca00640 1 -- 172.21.15.83:0/1464143591 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f291c0a3d20 msgr2=0x7f291c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.661+0000 7f293ca00640 1 --2- 172.21.15.83:0/1464143591 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f291c0a3d20 0x7f291c0a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2928006010 tx=0x7f2928009820 comp rx=0 tx=0).stop 2024-04-23T14:50:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.661+0000 7f293ca00640 1 -- 172.21.15.83:0/1464143591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29381985d0 msgr2=0x7f2938198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.661+0000 7f293ca00640 1 --2- 172.21.15.83:0/1464143591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29381985d0 0x7f2938198a80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f293000d950 tx=0x7f293000de10 comp rx=0 tx=0).stop 2024-04-23T14:50:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.661+0000 7f293ca00640 1 -- 172.21.15.83:0/1464143591 shutdown_connections 2024-04-23T14:50:47.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.661+0000 7f293ca00640 1 --2- 172.21.15.83:0/1464143591 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29381985d0 0x7f2938198a80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.661+0000 7f293ca00640 1 --2- 172.21.15.83:0/1464143591 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f291c0a3d20 0x7f291c0a61e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.661+0000 7f293ca00640 1 --2- 172.21.15.83:0/1464143591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29381071f0 0x7f293819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.661+0000 7f293ca00640 1 --2- 172.21.15.83:0/1464143591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2938102120 0x7f293819e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.661+0000 7f293ca00640 1 -- 172.21.15.83:0/1464143591 >> 172.21.15.83:0/1464143591 conn(0x7f29380fd9a0 msgr2=0x7f29380ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.661+0000 7f293ca00640 1 -- 172.21.15.83:0/1464143591 shutdown_connections 2024-04-23T14:50:47.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.661+0000 7f293ca00640 1 -- 172.21.15.83:0/1464143591 wait complete. 2024-04-23T14:50:47.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:50:47.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:47 smithi083 ceph-mon[27867]: pgmap v72: 97 pgs: 97 active+clean; 1.8 GiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 148 MiB/s wr, 64 op/s 2024-04-23T14:50:47.738 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:47 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1263233140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:50:47.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:47 smithi173 ceph-mon[32443]: pgmap v72: 97 pgs: 97 active+clean; 1.8 GiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 148 MiB/s wr, 64 op/s 2024-04-23T14:50:47.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:47 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1263233140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:50:47.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.864+0000 7fbc50c00640 1 -- 172.21.15.83:0/3855947883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4c109580 msgr2=0x7fbc4c111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.864+0000 7fbc50c00640 1 --2- 172.21.15.83:0/3855947883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4c109580 0x7fbc4c111150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009ca0 tx=0x7fbc4402a8c0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.865+0000 7fbc50c00640 1 -- 172.21.15.83:0/3855947883 shutdown_connections 2024-04-23T14:50:47.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.865+0000 7fbc50c00640 1 --2- 172.21.15.83:0/3855947883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4c109580 0x7fbc4c111150 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.865+0000 7fbc50c00640 1 --2- 172.21.15.83:0/3855947883 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc4c104e60 0x7fbc4c108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.865+0000 7fbc50c00640 1 --2- 172.21.15.83:0/3855947883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc4c1044b0 0x7fbc4c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.865+0000 7fbc50c00640 1 -- 172.21.15.83:0/3855947883 >> 172.21.15.83:0/3855947883 conn(0x7fbc4c0ffd90 msgr2=0x7fbc4c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.865+0000 7fbc50c00640 1 -- 172.21.15.83:0/3855947883 shutdown_connections 2024-04-23T14:50:47.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.865+0000 7fbc50c00640 1 -- 172.21.15.83:0/3855947883 wait complete. 2024-04-23T14:50:47.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc50c00640 1 Processor -- start 2024-04-23T14:50:47.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc50c00640 1 -- start start 2024-04-23T14:50:47.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc50c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4c1044b0 0x7fbc4c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc50c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc4c109580 0x7fbc4c1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc50c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc4c19a980 0x7fbc4c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc50c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c114620 con 0x7fbc4c1044b0 2024-04-23T14:50:47.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc50c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1144a0 con 0x7fbc4c19a980 2024-04-23T14:50:47.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc50c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1147a0 con 0x7fbc4c109580 2024-04-23T14:50:47.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc49e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4c1044b0 0x7fbc4c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc49400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc4c109580 0x7fbc4c1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc49e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4c1044b0 0x7fbc4c1a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42244/0 (socket says 172.21.15.83:42244) 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc49e00640 1 -- 172.21.15.83:0/2506954070 learned_addr learned my addr 172.21.15.83:0/2506954070 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.866+0000 7fbc49400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc4c109580 0x7fbc4c1a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55418/0 (socket says 172.21.15.83:55418) 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.867+0000 7fbc4a800640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc4c19a980 0x7fbc4c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.867+0000 7fbc49e00640 1 -- 172.21.15.83:0/2506954070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc4c109580 msgr2=0x7fbc4c1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.867+0000 7fbc49e00640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc4c109580 0x7fbc4c1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.867+0000 7fbc49e00640 1 -- 172.21.15.83:0/2506954070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc4c19a980 msgr2=0x7fbc4c19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.867+0000 7fbc49e00640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc4c19a980 0x7fbc4c19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.867+0000 7fbc49e00640 1 -- 172.21.15.83:0/2506954070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4c19b4a0 con 0x7fbc4c1044b0 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.867+0000 7fbc49400640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc4c109580 0x7fbc4c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.867+0000 7fbc49e00640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4c1044b0 0x7fbc4c1a0780 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00ea60 tx=0x7fbc3c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.867+0000 7fbc42a00640 1 -- 172.21.15.83:0/2506954070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c00cde0 con 0x7fbc4c1044b0 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.867+0000 7fbc42a00640 1 -- 172.21.15.83:0/2506954070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc3c0035f0 con 0x7fbc4c1044b0 2024-04-23T14:50:47.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.867+0000 7fbc50c00640 1 -- 172.21.15.83:0/2506954070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4c19b790 con 0x7fbc4c1044b0 2024-04-23T14:50:47.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.868+0000 7fbc42a00640 1 -- 172.21.15.83:0/2506954070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c010670 con 0x7fbc4c1044b0 2024-04-23T14:50:47.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.868+0000 7fbc50c00640 1 -- 172.21.15.83:0/2506954070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc4c1a7990 con 0x7fbc4c1044b0 2024-04-23T14:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.871+0000 7fbc539c7740 1 Processor -- start 2024-04-23T14:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.871+0000 7fbc539c7740 1 -- start start 2024-04-23T14:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc539c7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 0x5571c6f58d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc539c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f63400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc539c7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f65d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc539c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5571c6ebe1d0 con 0x5571c6f59270 2024-04-23T14:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc539c7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5571c6e64450 con 0x5571c6f63940 2024-04-23T14:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc539c7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5571c6eba190 con 0x5571c6f58950 2024-04-23T14:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc4a800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 0x5571c6f58d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc4a800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 0x5571c6f58d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55420/0 (socket says 172.21.15.83:55420) 2024-04-23T14:50:47.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc4a800640 1 -- 172.21.15.83:0/898645320 learned_addr learned my addr 172.21.15.83:0/898645320 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:47.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc49400640 1 --2- 172.21.15.83:0/898645320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f65d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc4a800640 1 -- 172.21.15.83:0/898645320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 msgr2=0x5571c6f65d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc4a800640 1 --2- 172.21.15.83:0/898645320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f65d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc4a800640 1 -- 172.21.15.83:0/898645320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 msgr2=0x5571c6f63400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:50:47.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc4a800640 1 --2- 172.21.15.83:0/898645320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f63400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.872+0000 7fbc4a800640 1 -- 172.21.15.83:0/898645320 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c6f21310 con 0x5571c6f58950 2024-04-23T14:50:47.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.873+0000 7fbc4a800640 1 --2- 172.21.15.83:0/898645320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 0x5571c6f58d30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009ca0 tx=0x7fbc44005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96d97c48e579ae61 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.873+0000 7fbc42a00640 1 -- 172.21.15.83:0/2506954070 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbc3c0037a0 con 0x7fbc4c1044b0 2024-04-23T14:50:47.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.873+0000 7fbc37000640 1 -- 172.21.15.83:0/898645320 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44031db0 con 0x5571c6f58950 2024-04-23T14:50:47.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.873+0000 7fbc37000640 1 -- 172.21.15.83:0/898645320 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc44036040 con 0x5571c6f58950 2024-04-23T14:50:47.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.873+0000 7fbc37000640 1 -- 172.21.15.83:0/898645320 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc44004280 con 0x5571c6f58950 2024-04-23T14:50:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.874+0000 7fbc539c7740 1 -- 172.21.15.83:0/898645320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 msgr2=0x5571c6f58d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.874+0000 7fbc539c7740 1 --2- 172.21.15.83:0/898645320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 0x5571c6f58d30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009ca0 tx=0x7fbc44005f10 comp rx=0 tx=0).stop 2024-04-23T14:50:47.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.874+0000 7fbc539c7740 1 -- 172.21.15.83:0/898645320 shutdown_connections 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.874+0000 7fbc539c7740 1 --2- 172.21.15.83:0/898645320 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f65d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.874+0000 7fbc539c7740 1 --2- 172.21.15.83:0/898645320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f63400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.874+0000 7fbc539c7740 1 --2- 172.21.15.83:0/898645320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 0x5571c6f58d30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.874+0000 7fbc539c7740 1 -- 172.21.15.83:0/898645320 >> 172.21.15.83:0/898645320 conn(0x5571c6f54c70 msgr2=0x5571c6f55050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.874+0000 7fbc539c7740 1 -- 172.21.15.83:0/898645320 shutdown_connections 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.874+0000 7fbc539c7740 1 -- 172.21.15.83:0/898645320 wait complete. 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.875+0000 7fbc539c7740 1 -- start start 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.875+0000 7fbc539c7740 1 Processor -- start 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.875+0000 7fbc42a00640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fbc300a3c30 0x7fbc300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.875+0000 7fbc42a00640 1 -- 172.21.15.83:0/2506954070 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fbc3c0d0b10 con 0x7fbc4c1044b0 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.875+0000 7fbc4a800640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fbc300a3c30 0x7fbc300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.875+0000 7fbc539c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f58950 0x5571c6f633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.875+0000 7fbc539c7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f59270 0x5571c6f606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.875+0000 7fbc539c7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f60de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.875+0000 7fbc539c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5571c6f665f0 con 0x5571c6f58950 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.875+0000 7fbc539c7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5571c6e77810 con 0x5571c6f63940 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.875+0000 7fbc539c7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5571c6f66770 con 0x5571c6f59270 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.875+0000 7fbc539c7740 10 client.0 _ll_register_callbacks cb 0x5571c6e45140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc49e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f58950 0x5571c6f633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc49e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f58950 0x5571c6f633b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42254/0 (socket says 172.21.15.83:42254) 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc49e00640 1 -- 172.21.15.83:0/480132041 learned_addr learned my addr 172.21.15.83:0/480132041 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc4a800640 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f60de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc49400640 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f59270 0x5571c6f606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc4a800640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fbc300a3c30 0x7fbc300a60f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc4402f470 tx=0x7fbc44031a80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc4a800640 1 -- 172.21.15.83:0/480132041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f59270 msgr2=0x5571c6f606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc4a800640 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f59270 0x5571c6f606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc4a800640 1 -- 172.21.15.83:0/480132041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f58950 msgr2=0x5571c6f633b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc4a800640 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f58950 0x5571c6f633b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc4a800640 1 -- 172.21.15.83:0/480132041 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c6e883f0 con 0x5571c6f63940 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc4a800640 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f60de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009620 tx=0x7fbc44004a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dafe90c391ad5422 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.876+0000 7fbc2b400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.877+0000 7fbc2b400640 1 -- 172.21.15.83:0/480132041 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4405e060 con 0x5571c6f63940 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.877+0000 7fbc539c7740 1 -- 172.21.15.83:0/480132041 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc44036040 con 0x5571c6f63940 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.877+0000 7fbc2b400640 1 -- 172.21.15.83:0/480132041 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fbc44004790 con 0x5571c6f63940 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.877+0000 7fbc2b400640 1 -- 172.21.15.83:0/480132041 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc4402f610 con 0x5571c6f63940 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.877+0000 7fbc539c7740 10 client.25847 fetch_fsmap learned FSMap version 26 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.877+0000 7fbc539c7740 1 -- 172.21.15.83:0/480132041 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5571c6bd03f0 con 0x5571c6f63940 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.877+0000 7fbc2b400640 1 -- 172.21.15.83:0/480132041 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc44036040 con 0x5571c6f63940 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.878+0000 7fbc2b400640 1 -- 172.21.15.83:0/480132041 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fbc44043070 con 0x5571c6f63940 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.878+0000 7fbc539c7740 10 client.25847 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.878+0000 7fbc539c7740 10 client.25847 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.878+0000 7fbc539c7740 20 client.25847 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.878+0000 7fbc539c7740 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5571c6f737f0 0x5571c6f75be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.878+0000 7fbc539c7740 4 client.25847 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:50:47.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.878+0000 7fbc539c7740 1 -- 172.21.15.83:0/480132041 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571c6ecbbf0 con 0x5571c6f737f0 2024-04-23T14:50:47.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.878+0000 7fbc49e00640 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5571c6f737f0 0x5571c6f75be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.878+0000 7fbc49e00640 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5571c6f737f0 0x5571c6f75be0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e99b26adebbab27 server_cookie=ac38fb4df1144c65 in_seq=0 out_seq=0 2024-04-23T14:50:47.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.878+0000 7fbc2b400640 10 client.25847 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:50:47.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.879+0000 7fbc2b400640 1 -- 172.21.15.83:0/480132041 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571c6ecbbf0 con 0x5571c6f737f0 2024-04-23T14:50:47.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.879+0000 7fbc2b400640 10 client.25847 handle_command_reply: tid=0 2024-04-23T14:50:47.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.879+0000 7fbc539c7740 1 client.25847 shutdown 2024-04-23T14:50:47.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.879+0000 7fbc539c7740 1 -- 172.21.15.83:0/480132041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 msgr2=0x5571c6f60de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.880+0000 7fbc539c7740 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f60de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbc44009620 tx=0x7fbc44004a80 comp rx=0 tx=0).stop 2024-04-23T14:50:47.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.880+0000 7fbc539c7740 1 -- 172.21.15.83:0/480132041 shutdown_connections 2024-04-23T14:50:47.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.880+0000 7fbc539c7740 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f60de0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.880+0000 7fbc539c7740 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5571c6f737f0 0x5571c6f75be0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.880+0000 7fbc539c7740 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f59270 0x5571c6f606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.880+0000 7fbc539c7740 1 --2- 172.21.15.83:0/480132041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f58950 0x5571c6f633b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.880+0000 7fbc2b400640 0 client.25847 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:50:47.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.880+0000 7fbc539c7740 1 -- 172.21.15.83:0/480132041 >> 172.21.15.83:0/480132041 conn(0x5571c6f54c70 msgr2=0x5571c6f5a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.880+0000 7fbc539c7740 1 -- 172.21.15.83:0/480132041 shutdown_connections 2024-04-23T14:50:47.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.880+0000 7fbc539c7740 1 -- 172.21.15.83:0/480132041 wait complete. 2024-04-23T14:50:47.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.880+0000 7fbc539c7740 20 client.25847 trim_cache size 0 max 16384 2024-04-23T14:50:47.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.894+0000 7fbc539c7740 1 Processor -- start 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.894+0000 7fbc539c7740 1 -- start start 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.894+0000 7fbc539c7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f58950 0x5571c6f53c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.894+0000 7fbc539c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f62ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.894+0000 7fbc539c7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f63940 0x5571c6f7b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.894+0000 7fbc539c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1b27e0 con 0x5571c6f59270 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.894+0000 7fbc539c7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1b0080 con 0x5571c6f58950 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.894+0000 7fbc539c7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1adef0 con 0x5571c6f63940 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.894+0000 7fbc4a800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f63940 0x5571c6f7b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.894+0000 7fbc4a800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f63940 0x5571c6f7b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55436/0 (socket says 172.21.15.83:55436) 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.894+0000 7fbc4a800640 1 -- 172.21.15.83:0/1675807303 learned_addr learned my addr 172.21.15.83:0/1675807303 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.894+0000 7fbc49400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f62ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.895+0000 7fbc49e00640 1 --2- 172.21.15.83:0/1675807303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f58950 0x5571c6f53c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.895+0000 7fbc4a800640 1 -- 172.21.15.83:0/1675807303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f58950 msgr2=0x5571c6f53c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.895+0000 7fbc4a800640 1 --2- 172.21.15.83:0/1675807303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f58950 0x5571c6f53c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.895+0000 7fbc4a800640 1 -- 172.21.15.83:0/1675807303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 msgr2=0x5571c6f62ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.895+0000 7fbc4a800640 1 --2- 172.21.15.83:0/1675807303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f62ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.895+0000 7fbc4a800640 1 -- 172.21.15.83:0/1675807303 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c6c9a7d0 con 0x5571c6f63940 2024-04-23T14:50:47.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.895+0000 7fbc49e00640 1 --2- 172.21.15.83:0/1675807303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f58950 0x5571c6f53c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:50:47.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.895+0000 7fbc49400640 1 --2- 172.21.15.83:0/1675807303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f62ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:50:47.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.895+0000 7fbc4a800640 1 --2- 172.21.15.83:0/1675807303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f63940 0x5571c6f7b2a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbc44030c10 tx=0x7fbc44032f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d2438995599466 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.895+0000 7fbc36600640 1 -- 172.21.15.83:0/1675807303 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4405e030 con 0x5571c6f63940 2024-04-23T14:50:47.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.895+0000 7fbc36600640 1 -- 172.21.15.83:0/1675807303 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc44036040 con 0x5571c6f63940 2024-04-23T14:50:47.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.895+0000 7fbc36600640 1 -- 172.21.15.83:0/1675807303 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4402fa40 con 0x5571c6f63940 2024-04-23T14:50:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.896+0000 7fbc539c7740 1 -- 172.21.15.83:0/1675807303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f63940 msgr2=0x5571c6f7b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.896+0000 7fbc539c7740 1 --2- 172.21.15.83:0/1675807303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f63940 0x5571c6f7b2a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbc44030c10 tx=0x7fbc44032f90 comp rx=0 tx=0).stop 2024-04-23T14:50:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.896+0000 7fbc539c7740 1 -- 172.21.15.83:0/1675807303 shutdown_connections 2024-04-23T14:50:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.896+0000 7fbc539c7740 1 --2- 172.21.15.83:0/1675807303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f63940 0x5571c6f7b2a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.896+0000 7fbc539c7740 1 --2- 172.21.15.83:0/1675807303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f62ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.896+0000 7fbc539c7740 1 --2- 172.21.15.83:0/1675807303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f58950 0x5571c6f53c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.896+0000 7fbc539c7740 1 -- 172.21.15.83:0/1675807303 >> 172.21.15.83:0/1675807303 conn(0x5571c6f54c70 msgr2=0x5571c6f53560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.896+0000 7fbc539c7740 1 -- 172.21.15.83:0/1675807303 shutdown_connections 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.896+0000 7fbc539c7740 1 -- 172.21.15.83:0/1675807303 wait complete. 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc539c7740 1 -- start start 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc539c7740 1 Processor -- start 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc539c7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 0x5571c6f6bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc539c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f5dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc539c7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f5e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc539c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1ae540 con 0x5571c6f59270 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc539c7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1af9b0 con 0x5571c6f63940 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc539c7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4c1aec10 con 0x5571c6f58950 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc539c7740 10 client.0 _ll_register_callbacks cb 0x7fbc38009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc49400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f5dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc49400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f5dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42278/0 (socket says 172.21.15.83:42278) 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc49e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 0x5571c6f6bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc49400640 1 -- 172.21.15.83:0/3883608491 learned_addr learned my addr 172.21.15.83:0/3883608491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.897+0000 7fbc4a800640 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f5e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc49e00640 1 -- 172.21.15.83:0/3883608491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 msgr2=0x5571c6f5e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc49e00640 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f5e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc49e00640 1 -- 172.21.15.83:0/3883608491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 msgr2=0x5571c6f5dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc49e00640 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f5dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc49e00640 1 -- 172.21.15.83:0/3883608491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c6f64910 con 0x5571c6f58950 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc4a800640 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f5e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc49e00640 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 0x5571c6f6bec0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c09c7c0 tx=0x7fbc3c09c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c80bb969d5112e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc2b400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc2b400640 1 -- 172.21.15.83:0/3883608491 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0a0e70 con 0x5571c6f58950 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc2b400640 1 -- 172.21.15.83:0/3883608491 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fbc3c09adc0 con 0x5571c6f58950 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc539c7740 1 -- 172.21.15.83:0/3883608491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc44036040 con 0x5571c6f58950 2024-04-23T14:50:47.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc2b400640 1 -- 172.21.15.83:0/3883608491 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc3c0a5730 con 0x5571c6f58950 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc539c7740 10 client.25825 fetch_fsmap learned FSMap version 26 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.898+0000 7fbc539c7740 1 -- 172.21.15.83:0/3883608491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5571c6fda010 con 0x5571c6f58950 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.899+0000 7fbc2b400640 1 -- 172.21.15.83:0/3883608491 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc44036040 con 0x5571c6f58950 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.899+0000 7fbc2b400640 1 -- 172.21.15.83:0/3883608491 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fbc3c09bf10 con 0x5571c6f58950 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.899+0000 7fbc539c7740 10 client.25825 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.899+0000 7fbc539c7740 10 client.25825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.899+0000 7fbc539c7740 20 client.25825 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.899+0000 7fbc539c7740 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5571c6fda6f0 0x5571c6fdcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.899+0000 7fbc539c7740 4 client.25825 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.899+0000 7fbc539c7740 1 -- 172.21.15.83:0/3883608491 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571c6ec5a60 con 0x5571c6fda6f0 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.899+0000 7fbc49400640 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5571c6fda6f0 0x5571c6fdcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc49400640 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5571c6fda6f0 0x5571c6fdcb10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21c0294c96aebb88 server_cookie=6ee7ff04c6910074 in_seq=0 out_seq=0 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc2b400640 10 client.25825 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:50:47.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc2b400640 1 -- 172.21.15.83:0/3883608491 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571c6ec5a60 con 0x5571c6fda6f0 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc2b400640 10 client.25825 handle_command_reply: tid=0 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc539c7740 1 client.25825 shutdown 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc539c7740 1 -- 172.21.15.83:0/3883608491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 msgr2=0x5571c6f6bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc539c7740 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 0x5571c6f6bec0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c09c7c0 tx=0x7fbc3c09c960 comp rx=0 tx=0).stop 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc539c7740 1 -- 172.21.15.83:0/3883608491 shutdown_connections 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc539c7740 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5571c6f63940 0x5571c6f5e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc539c7740 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571c6f59270 0x5571c6f5dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc539c7740 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5571c6fda6f0 0x5571c6fdcb10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc539c7740 1 --2- 172.21.15.83:0/3883608491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5571c6f58950 0x5571c6f6bec0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc539c7740 1 -- 172.21.15.83:0/3883608491 >> 172.21.15.83:0/3883608491 conn(0x5571c6f54c70 msgr2=0x5571c6f53df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc2b400640 0 client.25825 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc539c7740 1 -- 172.21.15.83:0/3883608491 shutdown_connections 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.900+0000 7fbc539c7740 1 -- 172.21.15.83:0/3883608491 wait complete. 2024-04-23T14:50:47.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.901+0000 7fbc539c7740 20 client.25825 trim_cache size 0 max 16384 2024-04-23T14:50:47.906 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:50:47.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.901+0000 7fbc50c00640 1 -- 172.21.15.83:0/2506954070 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fbc300a3c30 msgr2=0x7fbc300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.901+0000 7fbc50c00640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fbc300a3c30 0x7fbc300a60f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbc4402f470 tx=0x7fbc44031a80 comp rx=0 tx=0).stop 2024-04-23T14:50:47.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.901+0000 7fbc50c00640 1 -- 172.21.15.83:0/2506954070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4c1044b0 msgr2=0x7fbc4c1a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:47.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.901+0000 7fbc50c00640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4c1044b0 0x7fbc4c1a0780 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00ea60 tx=0x7fbc3c00ef20 comp rx=0 tx=0).stop 2024-04-23T14:50:47.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.901+0000 7fbc50c00640 1 -- 172.21.15.83:0/2506954070 shutdown_connections 2024-04-23T14:50:47.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.901+0000 7fbc50c00640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc4c19a980 0x7fbc4c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.901+0000 7fbc50c00640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc4c109580 0x7fbc4c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.901+0000 7fbc50c00640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fbc300a3c30 0x7fbc300a60f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.901+0000 7fbc50c00640 1 --2- 172.21.15.83:0/2506954070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc4c1044b0 0x7fbc4c1a0780 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:47.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.901+0000 7fbc50c00640 1 -- 172.21.15.83:0/2506954070 >> 172.21.15.83:0/2506954070 conn(0x7fbc4c0ffd90 msgr2=0x7fbc4c1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:47.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.902+0000 7fbc50c00640 1 -- 172.21.15.83:0/2506954070 shutdown_connections 2024-04-23T14:50:47.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:47.902+0000 7fbc50c00640 1 -- 172.21.15.83:0/2506954070 wait complete. 2024-04-23T14:50:47.923 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:50:47.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:47 smithi203 ceph-mon[32800]: pgmap v72: 97 pgs: 97 active+clean; 1.8 GiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 148 MiB/s wr, 64 op/s 2024-04-23T14:50:47.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:47 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1263233140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:50:48.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.110+0000 7f3894600640 1 -- 172.21.15.83:0/3370550415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c10c5c0 msgr2=0x7f388c10ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:48.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.110+0000 7f3894600640 1 --2- 172.21.15.83:0/3370550415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c10c5c0 0x7f388c10ea60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3884009ca0 tx=0x7f388402a9e0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.110+0000 7f3894600640 1 -- 172.21.15.83:0/3370550415 shutdown_connections 2024-04-23T14:50:48.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.110+0000 7f3894600640 1 --2- 172.21.15.83:0/3370550415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c10c5c0 0x7f388c10ea60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.110+0000 7f3894600640 1 --2- 172.21.15.83:0/3370550415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388c103ba0 0x7f388c10c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.110+0000 7f3894600640 1 --2- 172.21.15.83:0/3370550415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f388c103280 0x7f388c103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.110+0000 7f3894600640 1 -- 172.21.15.83:0/3370550415 >> 172.21.15.83:0/3370550415 conn(0x7f388c078000 msgr2=0x7f388c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:48.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.110+0000 7f3894600640 1 -- 172.21.15.83:0/3370550415 shutdown_connections 2024-04-23T14:50:48.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.110+0000 7f3894600640 1 -- 172.21.15.83:0/3370550415 wait complete. 2024-04-23T14:50:48.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.111+0000 7f3894600640 1 Processor -- start 2024-04-23T14:50:48.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.111+0000 7f3894600640 1 -- start start 2024-04-23T14:50:48.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3894600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f388c103280 0x7f388c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3894600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388c103ba0 0x7f388c1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3894600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c10c5c0 0x7f388c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3894600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f388c1123a0 con 0x7f388c103ba0 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3894600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f388c112220 con 0x7f388c10c5c0 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3894600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f388c112520 con 0x7f388c103280 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3891200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388c103ba0 0x7f388c1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3891200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388c103ba0 0x7f388c1a0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42306/0 (socket says 172.21.15.83:42306) 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3892600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c10c5c0 0x7f388c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3891200640 1 -- 172.21.15.83:0/2745035294 learned_addr learned my addr 172.21.15.83:0/2745035294 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3891c00640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f388c103280 0x7f388c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3891c00640 1 -- 172.21.15.83:0/2745035294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c10c5c0 msgr2=0x7f388c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3891c00640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c10c5c0 0x7f388c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3891c00640 1 -- 172.21.15.83:0/2745035294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388c103ba0 msgr2=0x7f388c1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:48.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3891c00640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388c103ba0 0x7f388c1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3891c00640 1 -- 172.21.15.83:0/2745035294 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c19b120 con 0x7f388c103280 2024-04-23T14:50:48.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.112+0000 7f3891200640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388c103ba0 0x7f388c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:48.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.113+0000 7f3891c00640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f388c103280 0x7f388c1a07f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f388000e9f0 tx=0x7f388000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:48.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.113+0000 7f388a000640 1 -- 172.21.15.83:0/2745035294 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388000ccc0 con 0x7f388c103280 2024-04-23T14:50:48.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.113+0000 7f3894600640 1 -- 172.21.15.83:0/2745035294 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c19b410 con 0x7f388c103280 2024-04-23T14:50:48.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.113+0000 7f388a000640 1 -- 172.21.15.83:0/2745035294 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3880005020 con 0x7f388c103280 2024-04-23T14:50:48.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.113+0000 7f388a000640 1 -- 172.21.15.83:0/2745035294 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38800049e0 con 0x7f388c103280 2024-04-23T14:50:48.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.113+0000 7f3894600640 1 -- 172.21.15.83:0/2745035294 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c1a76b0 con 0x7f388c103280 2024-04-23T14:50:48.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.116+0000 7f388a000640 1 -- 172.21.15.83:0/2745035294 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3880005560 con 0x7f388c103280 2024-04-23T14:50:48.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.116+0000 7f3897378740 1 Processor -- start 2024-04-23T14:50:48.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.116+0000 7f3897378740 1 -- start start 2024-04-23T14:50:48.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3897378740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b76199950 0x559b76199d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3897378740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b7619a270 0x559b761a4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3897378740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b761a4940 0x559b761a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3897378740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b760ff1d0 con 0x559b761a4940 2024-04-23T14:50:48.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3897378740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559b760a5450 con 0x559b76199950 2024-04-23T14:50:48.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3897378740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559b760fb190 con 0x559b7619a270 2024-04-23T14:50:48.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3891c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b7619a270 0x559b761a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3891200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b761a4940 0x559b761a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3891200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b761a4940 0x559b761a6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42322/0 (socket says 172.21.15.83:42322) 2024-04-23T14:50:48.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3891200640 1 -- 172.21.15.83:0/731215673 learned_addr learned my addr 172.21.15.83:0/731215673 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3891c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b7619a270 0x559b761a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55472/0 (socket says 172.21.15.83:55472) 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3892600640 1 --2- 172.21.15.83:0/731215673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b76199950 0x559b76199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3891200640 1 -- 172.21.15.83:0/731215673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b7619a270 msgr2=0x559b761a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3891200640 1 --2- 172.21.15.83:0/731215673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b7619a270 0x559b761a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3891200640 1 -- 172.21.15.83:0/731215673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b76199950 msgr2=0x559b76199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3891200640 1 --2- 172.21.15.83:0/731215673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b76199950 0x559b76199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3891200640 1 -- 172.21.15.83:0/731215673 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b76162310 con 0x559b761a4940 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.117+0000 7f3891c00640 1 --2- 172.21.15.83:0/731215673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b7619a270 0x559b761a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.118+0000 7f3891200640 1 --2- 172.21.15.83:0/731215673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b761a4940 0x559b761a6d30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3878011510 tx=0x7f38780119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829b2716a5dc2184 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.118+0000 7f388a000640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38740a3d20 0x7f38740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.118+0000 7f387ea00640 1 -- 172.21.15.83:0/731215673 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878005020 con 0x559b761a4940 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.118+0000 7f387ea00640 1 -- 172.21.15.83:0/731215673 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38780047a0 con 0x559b761a4940 2024-04-23T14:50:48.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.118+0000 7f387ea00640 1 -- 172.21.15.83:0/731215673 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387800f990 con 0x559b761a4940 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.118+0000 7f388a000640 1 -- 172.21.15.83:0/2745035294 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f38800d07d0 con 0x7f388c103280 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.118+0000 7f3892600640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38740a3d20 0x7f38740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.119+0000 7f3897378740 1 -- 172.21.15.83:0/731215673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b761a4940 msgr2=0x559b761a6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.119+0000 7f3897378740 1 --2- 172.21.15.83:0/731215673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b761a4940 0x559b761a6d30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3878011510 tx=0x7f38780119d0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.119+0000 7f3897378740 1 -- 172.21.15.83:0/731215673 shutdown_connections 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.119+0000 7f3897378740 1 --2- 172.21.15.83:0/731215673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b761a4940 0x559b761a6d30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.119+0000 7f3897378740 1 --2- 172.21.15.83:0/731215673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b7619a270 0x559b761a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.119+0000 7f3897378740 1 --2- 172.21.15.83:0/731215673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b76199950 0x559b76199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.119+0000 7f3897378740 1 -- 172.21.15.83:0/731215673 >> 172.21.15.83:0/731215673 conn(0x559b76195c70 msgr2=0x559b76196050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.119+0000 7f3892600640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38740a3d20 0x7f38740a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3884005e30 tx=0x7f388403a040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.119+0000 7f3897378740 1 -- 172.21.15.83:0/731215673 shutdown_connections 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.119+0000 7f3897378740 1 -- 172.21.15.83:0/731215673 wait complete. 2024-04-23T14:50:48.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.119+0000 7f3897378740 1 -- start start 2024-04-23T14:50:48.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.120+0000 7f3897378740 1 Processor -- start 2024-04-23T14:50:48.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.120+0000 7f3897378740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b76199950 0x559b761a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.120+0000 7f3897378740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b7619a270 0x559b761a1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.120+0000 7f3897378740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b761a4940 0x559b761a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.120+0000 7f3897378740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b761a77d0 con 0x559b7619a270 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.120+0000 7f3897378740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559b760b8810 con 0x559b761a4940 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.120+0000 7f3897378740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559b761a7950 con 0x559b76199950 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.120+0000 7f3897378740 10 client.0 _ll_register_callbacks cb 0x559b76086140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.120+0000 7f3891200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b7619a270 0x559b761a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.120+0000 7f3891200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b7619a270 0x559b761a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42338/0 (socket says 172.21.15.83:42338) 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.120+0000 7f3891c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b76199950 0x559b761a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.120+0000 7f3891200640 1 -- 172.21.15.83:0/1412166203 learned_addr learned my addr 172.21.15.83:0/1412166203 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.121+0000 7f3892600640 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b761a4940 0x559b761a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.121+0000 7f3891200640 1 -- 172.21.15.83:0/1412166203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b76199950 msgr2=0x559b761a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.121+0000 7f3891200640 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b76199950 0x559b761a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.121+0000 7f3891200640 1 -- 172.21.15.83:0/1412166203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b761a4940 msgr2=0x559b761a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.121+0000 7f3891200640 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b761a4940 0x559b761a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.121+0000 7f3891200640 1 -- 172.21.15.83:0/1412166203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b760c93f0 con 0x559b7619a270 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.121+0000 7f3891c00640 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b76199950 0x559b761a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.121+0000 7f3891200640 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b7619a270 0x559b761a1610 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f38780052b0 tx=0x7f38780052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9202c4561f47242c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.121+0000 7f3873400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.121+0000 7f3873400640 1 -- 172.21.15.83:0/1412166203 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878027070 con 0x559b7619a270 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.121+0000 7f3897378740 1 -- 172.21.15.83:0/1412166203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38780047a0 con 0x559b7619a270 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.121+0000 7f3873400640 1 -- 172.21.15.83:0/1412166203 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3878004040 con 0x559b7619a270 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.122+0000 7f3873400640 1 -- 172.21.15.83:0/1412166203 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3878033050 con 0x559b7619a270 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.122+0000 7f3897378740 10 client.15744 fetch_fsmap learned FSMap version 26 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.122+0000 7f3897378740 1 -- 172.21.15.83:0/1412166203 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b75e113f0 con 0x559b7619a270 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.122+0000 7f3873400640 1 -- 172.21.15.83:0/1412166203 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38780047a0 con 0x559b7619a270 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.122+0000 7f3873400640 1 -- 172.21.15.83:0/1412166203 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f387800fce0 con 0x559b7619a270 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.122+0000 7f3897378740 10 client.15744 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:50:48.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.122+0000 7f3897378740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.122+0000 7f3897378740 20 client.15744 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.122+0000 7f3897378740 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b761b4b20 0x559b761b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.122+0000 7f3897378740 4 client.15744 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.122+0000 7f3897378740 1 -- 172.21.15.83:0/1412166203 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b7610cbf0 con 0x559b761b4b20 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.123+0000 7f3891c00640 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b761b4b20 0x559b761b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.123+0000 7f3891c00640 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b761b4b20 0x559b761b6f10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fc824cfbab186ca server_cookie=a1563d49c44d066d in_seq=0 out_seq=0 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.123+0000 7f3873400640 10 client.15744 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3873400640 1 -- 172.21.15.83:0/1412166203 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b7610cbf0 con 0x559b761b4b20 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3873400640 10 client.15744 handle_command_reply: tid=0 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3897378740 1 client.15744 shutdown 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3897378740 1 -- 172.21.15.83:0/1412166203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b7619a270 msgr2=0x559b761a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3897378740 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b7619a270 0x559b761a1610 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f38780052b0 tx=0x7f38780052e0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3897378740 1 -- 172.21.15.83:0/1412166203 shutdown_connections 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3897378740 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b761b4b20 0x559b761b6f10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3897378740 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b761a4940 0x559b761a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3897378740 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b7619a270 0x559b761a1610 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3897378740 1 --2- 172.21.15.83:0/1412166203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b76199950 0x559b761a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3897378740 1 -- 172.21.15.83:0/1412166203 >> 172.21.15.83:0/1412166203 conn(0x559b76195c70 msgr2=0x559b7619ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3873400640 0 client.15744 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3897378740 1 -- 172.21.15.83:0/1412166203 shutdown_connections 2024-04-23T14:50:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3897378740 1 -- 172.21.15.83:0/1412166203 wait complete. 2024-04-23T14:50:48.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.124+0000 7f3897378740 20 client.15744 trim_cache size 0 max 16384 2024-04-23T14:50:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.138+0000 7f3897378740 1 Processor -- start 2024-04-23T14:50:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.139+0000 7f3897378740 1 -- start start 2024-04-23T14:50:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.139+0000 7f3897378740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b76199950 0x559b761b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.139+0000 7f3897378740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b7619a270 0x559b761a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.139+0000 7f3897378740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 0x559b761a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.139+0000 7f3897378740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f388c1b27c0 con 0x559b7619a270 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.139+0000 7f3897378740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f388c1b0060 con 0x559b76199950 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.139+0000 7f3897378740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f388c1adeb0 con 0x559b761a4940 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.139+0000 7f3892600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 0x559b761a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.139+0000 7f3892600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 0x559b761a3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55492/0 (socket says 172.21.15.83:55492) 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.139+0000 7f3892600640 1 -- 172.21.15.83:0/3171128696 learned_addr learned my addr 172.21.15.83:0/3171128696 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.139+0000 7f3891200640 1 --2- 172.21.15.83:0/3171128696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b7619a270 0x559b761a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.139+0000 7f3891c00640 1 --2- 172.21.15.83:0/3171128696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b76199950 0x559b761b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.140+0000 7f3892600640 1 -- 172.21.15.83:0/3171128696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b76199950 msgr2=0x559b761b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.140+0000 7f3892600640 1 --2- 172.21.15.83:0/3171128696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b76199950 0x559b761b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.140+0000 7f3892600640 1 -- 172.21.15.83:0/3171128696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b7619a270 msgr2=0x559b761a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.140+0000 7f3892600640 1 --2- 172.21.15.83:0/3171128696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b7619a270 0x559b761a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.140+0000 7f3892600640 1 -- 172.21.15.83:0/3171128696 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b75edb7d0 con 0x559b761a4940 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.140+0000 7f3892600640 1 --2- 172.21.15.83:0/3171128696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 0x559b761a3e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f388402ee50 tx=0x7f3884042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6ba57f00ce0fa2e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.140+0000 7f387e000640 1 -- 172.21.15.83:0/3171128696 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884056070 con 0x559b761a4940 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.140+0000 7f387e000640 1 -- 172.21.15.83:0/3171128696 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f388404c040 con 0x559b761a4940 2024-04-23T14:50:48.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.140+0000 7f387e000640 1 -- 172.21.15.83:0/3171128696 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388404b030 con 0x559b761a4940 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.141+0000 7f3897378740 1 -- 172.21.15.83:0/3171128696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 msgr2=0x559b761a3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.141+0000 7f3897378740 1 --2- 172.21.15.83:0/3171128696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 0x559b761a3e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f388402ee50 tx=0x7f3884042000 comp rx=0 tx=0).stop 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.141+0000 7f3897378740 1 -- 172.21.15.83:0/3171128696 shutdown_connections 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.141+0000 7f3897378740 1 --2- 172.21.15.83:0/3171128696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 0x559b761a3e50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.141+0000 7f3897378740 1 --2- 172.21.15.83:0/3171128696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b7619a270 0x559b761a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.141+0000 7f3897378740 1 --2- 172.21.15.83:0/3171128696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b76199950 0x559b761b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.141+0000 7f3897378740 1 -- 172.21.15.83:0/3171128696 >> 172.21.15.83:0/3171128696 conn(0x559b76195c70 msgr2=0x559b7619fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.141+0000 7f3897378740 1 -- 172.21.15.83:0/3171128696 shutdown_connections 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.141+0000 7f3897378740 1 -- 172.21.15.83:0/3171128696 wait complete. 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.142+0000 7f3897378740 1 -- start start 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.142+0000 7f3897378740 1 Processor -- start 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.142+0000 7f3897378740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b76199950 0x559b761acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.142+0000 7f3897378740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b7619a270 0x559b7619e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.142+0000 7f3897378740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 0x559b7619edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.142+0000 7f3897378740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f388c1ae520 con 0x559b76199950 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.142+0000 7f3897378740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f388c1af990 con 0x559b7619a270 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.142+0000 7f3897378740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f388c1aebf0 con 0x559b761a4940 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.142+0000 7f3897378740 10 client.0 _ll_register_callbacks cb 0x7f3880005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.142+0000 7f3891c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b76199950 0x559b761acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.143+0000 7f3892600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 0x559b7619edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.143+0000 7f3891c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b76199950 0x559b761acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42350/0 (socket says 172.21.15.83:42350) 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.143+0000 7f3891c00640 1 -- 172.21.15.83:0/1040501064 learned_addr learned my addr 172.21.15.83:0/1040501064 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.143+0000 7f3891200640 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b7619a270 0x559b7619e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.143+0000 7f3892600640 1 -- 172.21.15.83:0/1040501064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b7619a270 msgr2=0x559b7619e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.143+0000 7f3892600640 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b7619a270 0x559b7619e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.143+0000 7f3892600640 1 -- 172.21.15.83:0/1040501064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b76199950 msgr2=0x559b761acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.143+0000 7f3892600640 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b76199950 0x559b761acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.143+0000 7f3892600640 1 -- 172.21.15.83:0/1040501064 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b7619f310 con 0x559b761a4940 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.143+0000 7f3891c00640 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b76199950 0x559b761acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:48.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.143+0000 7f3892600640 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 0x559b7619edd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3884003d40 tx=0x7f38840042d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccc1e51d684db58c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.144+0000 7f3873400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.144+0000 7f3873400640 1 -- 172.21.15.83:0/1040501064 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388405e020 con 0x559b761a4940 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.144+0000 7f3897378740 1 -- 172.21.15.83:0/1040501064 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f388404c040 con 0x559b761a4940 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.144+0000 7f3873400640 1 -- 172.21.15.83:0/1040501064 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f38840047f0 con 0x559b761a4940 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.144+0000 7f3873400640 1 -- 172.21.15.83:0/1040501064 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3884069050 con 0x559b761a4940 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.144+0000 7f3897378740 10 client.25843 fetch_fsmap learned FSMap version 26 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.144+0000 7f3897378740 1 -- 172.21.15.83:0/1040501064 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b761a5ec0 con 0x559b761a4940 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.144+0000 7f3873400640 1 -- 172.21.15.83:0/1040501064 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f388404c040 con 0x559b761a4940 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.144+0000 7f3873400640 1 -- 172.21.15.83:0/1040501064 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f3884059030 con 0x559b761a4940 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.145+0000 7f3897378740 10 client.25843 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.145+0000 7f3897378740 10 client.25843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.145+0000 7f3897378740 20 client.25843 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.145+0000 7f3897378740 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b7621b300 0x559b7621d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.145+0000 7f3897378740 4 client.25843 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:50:48.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.145+0000 7f3897378740 1 -- 172.21.15.83:0/1040501064 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b76106a60 con 0x559b7621b300 2024-04-23T14:50:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.145+0000 7f3891c00640 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b7621b300 0x559b7621d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.145+0000 7f3891c00640 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b7621b300 0x559b7621d6f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d82aa6764f89857 server_cookie=4f98b9d3293e083c in_seq=0 out_seq=0 2024-04-23T14:50:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.145+0000 7f3873400640 10 client.25843 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3873400640 1 -- 172.21.15.83:0/1040501064 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b76106a60 con 0x559b7621b300 2024-04-23T14:50:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3873400640 10 client.25843 handle_command_reply: tid=0 2024-04-23T14:50:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3897378740 1 client.25843 shutdown 2024-04-23T14:50:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3897378740 1 -- 172.21.15.83:0/1040501064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 msgr2=0x559b7619edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3897378740 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 0x559b7619edd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3884003d40 tx=0x7f38840042d0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3897378740 1 -- 172.21.15.83:0/1040501064 shutdown_connections 2024-04-23T14:50:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3897378740 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b7621b300 0x559b7621d6f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3897378740 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b761a4940 0x559b7619edd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3897378740 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b7619a270 0x559b7619e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3873400640 0 client.25843 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:48.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3897378740 1 --2- 172.21.15.83:0/1040501064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b76199950 0x559b761acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3897378740 1 -- 172.21.15.83:0/1040501064 >> 172.21.15.83:0/1040501064 conn(0x559b76195c70 msgr2=0x559b76194560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:48.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3897378740 1 -- 172.21.15.83:0/1040501064 shutdown_connections 2024-04-23T14:50:48.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3897378740 1 -- 172.21.15.83:0/1040501064 wait complete. 2024-04-23T14:50:48.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.146+0000 7f3897378740 20 client.25843 trim_cache size 0 max 16384 2024-04-23T14:50:48.150 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:50:48.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.147+0000 7f3894600640 1 -- 172.21.15.83:0/2745035294 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38740a3d20 msgr2=0x7f38740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:48.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.147+0000 7f3894600640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38740a3d20 0x7f38740a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3884005e30 tx=0x7f388403a040 comp rx=0 tx=0).stop 2024-04-23T14:50:48.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.147+0000 7f3894600640 1 -- 172.21.15.83:0/2745035294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f388c103280 msgr2=0x7f388c1a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:48.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.147+0000 7f3894600640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f388c103280 0x7f388c1a07f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f388000e9f0 tx=0x7f388000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.147+0000 7f3894600640 1 -- 172.21.15.83:0/2745035294 shutdown_connections 2024-04-23T14:50:48.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.147+0000 7f3894600640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c10c5c0 0x7f388c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.147+0000 7f3894600640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38740a3d20 0x7f38740a61e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.147+0000 7f3894600640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f388c103ba0 0x7f388c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.147+0000 7f3894600640 1 --2- 172.21.15.83:0/2745035294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f388c103280 0x7f388c1a07f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:48.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.147+0000 7f3894600640 1 -- 172.21.15.83:0/2745035294 >> 172.21.15.83:0/2745035294 conn(0x7f388c078000 msgr2=0x7f388c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:48.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.147+0000 7f3894600640 1 -- 172.21.15.83:0/2745035294 shutdown_connections 2024-04-23T14:50:48.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:48.147+0000 7f3894600640 1 -- 172.21.15.83:0/2745035294 wait complete. 2024-04-23T14:50:49.164 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:50:49.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:50:49.164 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:50:49.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.382+0000 7fdb5a200640 1 -- 172.21.15.83:0/4266231168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb541028e0 msgr2=0x7fdb5410adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:49.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.382+0000 7fdb5a200640 1 --2- 172.21.15.83:0/4266231168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb541028e0 0x7fdb5410adc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009860 tx=0x7fdb4002a340 comp rx=0 tx=0).stop 2024-04-23T14:50:49.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.383+0000 7fdb5a200640 1 -- 172.21.15.83:0/4266231168 shutdown_connections 2024-04-23T14:50:49.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.383+0000 7fdb5a200640 1 --2- 172.21.15.83:0/4266231168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb5410b390 0x7fdb5410d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.383+0000 7fdb5a200640 1 --2- 172.21.15.83:0/4266231168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb541028e0 0x7fdb5410adc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.383+0000 7fdb5a200640 1 --2- 172.21.15.83:0/4266231168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb54101fc0 0x7fdb541023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.383+0000 7fdb5a200640 1 -- 172.21.15.83:0/4266231168 >> 172.21.15.83:0/4266231168 conn(0x7fdb54075d00 msgr2=0x7fdb54076110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:49.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.383+0000 7fdb5a200640 1 -- 172.21.15.83:0/4266231168 shutdown_connections 2024-04-23T14:50:49.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.383+0000 7fdb5a200640 1 -- 172.21.15.83:0/4266231168 wait complete. 2024-04-23T14:50:49.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.385+0000 7fdb5a200640 1 Processor -- start 2024-04-23T14:50:49.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.385+0000 7fdb5a200640 1 -- start start 2024-04-23T14:50:49.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.386+0000 7fdb5a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb54101fc0 0x7fdb5419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.386+0000 7fdb5a200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb541028e0 0x7fdb5419bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.386+0000 7fdb5a200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb5410b390 0x7fdb541a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.386+0000 7fdb5a200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54101a00 con 0x7fdb54101fc0 2024-04-23T14:50:49.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.386+0000 7fdb5a200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54101880 con 0x7fdb541028e0 2024-04-23T14:50:49.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.386+0000 7fdb5a200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54101b80 con 0x7fdb5410b390 2024-04-23T14:50:49.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.386+0000 7fdb58e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb54101fc0 0x7fdb5419b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.387+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb541028e0 0x7fdb5419bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.387+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb541028e0 0x7fdb5419bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54914/0 (socket says 172.21.15.83:54914) 2024-04-23T14:50:49.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.387+0000 7fdb53e00640 1 -- 172.21.15.83:0/1171545602 learned_addr learned my addr 172.21.15.83:0/1171545602 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:49.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.387+0000 7fdb59800640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb5410b390 0x7fdb541a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.387+0000 7fdb53e00640 1 -- 172.21.15.83:0/1171545602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb5410b390 msgr2=0x7fdb541a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:49.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.387+0000 7fdb53e00640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb5410b390 0x7fdb541a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.387+0000 7fdb53e00640 1 -- 172.21.15.83:0/1171545602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb54101fc0 msgr2=0x7fdb5419b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:49.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.387+0000 7fdb53e00640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb54101fc0 0x7fdb5419b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.387+0000 7fdb53e00640 1 -- 172.21.15.83:0/1171545602 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb541a6c70 con 0x7fdb541028e0 2024-04-23T14:50:49.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.387+0000 7fdb53e00640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb541028e0 0x7fdb5419bbf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009980 tx=0x7fdb40033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:49.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.388+0000 7fdb51600640 1 -- 172.21.15.83:0/1171545602 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40037040 con 0x7fdb541028e0 2024-04-23T14:50:49.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.388+0000 7fdb51600640 1 -- 172.21.15.83:0/1171545602 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb40040070 con 0x7fdb541028e0 2024-04-23T14:50:49.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.388+0000 7fdb51600640 1 -- 172.21.15.83:0/1171545602 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40003a10 con 0x7fdb541028e0 2024-04-23T14:50:49.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.388+0000 7fdb5a200640 1 -- 172.21.15.83:0/1171545602 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb541a6f00 con 0x7fdb541028e0 2024-04-23T14:50:49.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.388+0000 7fdb5a200640 1 -- 172.21.15.83:0/1171545602 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb541a7360 con 0x7fdb541028e0 2024-04-23T14:50:49.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.392+0000 7fdb51600640 1 -- 172.21.15.83:0/1171545602 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdb40004250 con 0x7fdb541028e0 2024-04-23T14:50:49.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.394+0000 7fdb5e5f3740 1 Processor -- start 2024-04-23T14:50:49.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.394+0000 7fdb51600640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdb3c0a3d20 0x7fdb3c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.394+0000 7fdb5e5f3740 1 -- start start 2024-04-23T14:50:49.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.394+0000 7fdb51600640 1 -- 172.21.15.83:0/1171545602 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fdb400f0700 con 0x7fdb541028e0 2024-04-23T14:50:49.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.394+0000 7fdb59800640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdb3c0a3d20 0x7fdb3c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.394+0000 7fdb5e5f3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f7e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.395+0000 7fdb5e5f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7f9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.395+0000 7fdb5e5f3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f7fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.395+0000 7fdb5e5f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558b7f74be70 con 0x558b7f7e6970 2024-04-23T14:50:49.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.395+0000 7fdb5e5f3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558b7f6f2380 con 0x558b7f7f9a70 2024-04-23T14:50:49.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.395+0000 7fdb5e5f3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558b7f747e30 con 0x558b7f7e5fc0 2024-04-23T14:50:49.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.395+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7f9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.395+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7f9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42358/0 (socket says 172.21.15.83:42358) 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.395+0000 7fdb53e00640 1 -- 172.21.15.83:0/108935151 learned_addr learned my addr 172.21.15.83:0/108935151 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.395+0000 7fdb59800640 1 --2- 172.21.15.83:0/108935151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f7fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.395+0000 7fdb58e00640 1 --2- 172.21.15.83:0/108935151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f7e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.395+0000 7fdb59800640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdb3c0a3d20 0x7fdb3c0a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c005fd0 tx=0x7fdb4c0107a0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.396+0000 7fdb59800640 1 -- 172.21.15.83:0/108935151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 msgr2=0x558b7f7e63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.396+0000 7fdb59800640 1 --2- 172.21.15.83:0/108935151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f7e63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.396+0000 7fdb59800640 1 -- 172.21.15.83:0/108935151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 msgr2=0x558b7f7f9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.396+0000 7fdb59800640 1 --2- 172.21.15.83:0/108935151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7f9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.396+0000 7fdb59800640 1 -- 172.21.15.83:0/108935151 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b7f5267e0 con 0x558b7f7f9a70 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.396+0000 7fdb53e00640 1 --2- 172.21.15.83:0/108935151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.396+0000 7fdb59800640 1 --2- 172.21.15.83:0/108935151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f7fbe60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c013730 tx=0x7fdb4c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d9bdd6a515ded7f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.396+0000 7fdb46000640 1 -- 172.21.15.83:0/108935151 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c003970 con 0x558b7f7f9a70 2024-04-23T14:50:49.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.396+0000 7fdb46000640 1 -- 172.21.15.83:0/108935151 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4c004030 con 0x558b7f7f9a70 2024-04-23T14:50:49.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.396+0000 7fdb46000640 1 -- 172.21.15.83:0/108935151 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00e7e0 con 0x558b7f7f9a70 2024-04-23T14:50:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.397+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/108935151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 msgr2=0x558b7f7fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.397+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/108935151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f7fbe60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c013730 tx=0x7fdb4c01b000 comp rx=0 tx=0).stop 2024-04-23T14:50:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.397+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/108935151 shutdown_connections 2024-04-23T14:50:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.397+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/108935151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f7fbe60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.397+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/108935151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7f9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.397+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/108935151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f7e63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.397+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/108935151 >> 172.21.15.83:0/108935151 conn(0x558b7f7e2590 msgr2=0x558b7f7e2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.397+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/108935151 shutdown_connections 2024-04-23T14:50:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.397+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/108935151 wait complete. 2024-04-23T14:50:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.398+0000 7fdb5e5f3740 1 -- start start 2024-04-23T14:50:49.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.398+0000 7fdb5e5f3740 1 Processor -- start 2024-04-23T14:50:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb5e5f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e5fc0 0x558b7f7f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb5e5f3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e6970 0x558b7f7f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb5e5f3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f7f7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb5e5f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558b7f7fc830 con 0x558b7f7e5fc0 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb5e5f3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558b7f705660 con 0x558b7f7f9a70 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb5e5f3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558b7f7fc9b0 con 0x558b7f7e6970 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e6970 0x558b7f7f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb59800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f7f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb58e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e5fc0 0x558b7f7f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb5e5f3740 10 client.0 _ll_register_callbacks cb 0x558b7f6f3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e6970 0x558b7f7f6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55530/0 (socket says 172.21.15.83:55530) 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb53e00640 1 -- 172.21.15.83:0/1246576347 learned_addr learned my addr 172.21.15.83:0/1246576347 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.399+0000 7fdb59800640 1 --2- 172.21.15.83:0/1246576347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f7f7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54922/0 (socket says 172.21.15.83:54922) 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.400+0000 7fdb59800640 1 -- 172.21.15.83:0/1246576347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e6970 msgr2=0x558b7f7f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.400+0000 7fdb59800640 1 --2- 172.21.15.83:0/1246576347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e6970 0x558b7f7f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.400+0000 7fdb59800640 1 -- 172.21.15.83:0/1246576347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e5fc0 msgr2=0x558b7f7f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:49.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.400+0000 7fdb59800640 1 --2- 172.21.15.83:0/1246576347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e5fc0 0x558b7f7f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.400+0000 7fdb59800640 1 -- 172.21.15.83:0/1246576347 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b7f586f70 con 0x558b7f7f9a70 2024-04-23T14:50:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.400+0000 7fdb59800640 1 --2- 172.21.15.83:0/1246576347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f7f7480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c010be0 tx=0x7fdb4c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd34f0e208b6111 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.400+0000 7fdb3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:50:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.401+0000 7fdb3aa00640 1 -- 172.21.15.83:0/1246576347 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c013400 con 0x558b7f7f9a70 2024-04-23T14:50:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.401+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1246576347 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb4c004030 con 0x558b7f7f9a70 2024-04-23T14:50:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.401+0000 7fdb3aa00640 1 -- 172.21.15.83:0/1246576347 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fdb4c004a00 con 0x558b7f7f9a70 2024-04-23T14:50:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.401+0000 7fdb3aa00640 1 -- 172.21.15.83:0/1246576347 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdb4c004c00 con 0x558b7f7f9a70 2024-04-23T14:50:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.401+0000 7fdb5e5f3740 10 client.25883 fetch_fsmap learned FSMap version 26 2024-04-23T14:50:49.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.401+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1246576347 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558b7f41e170 con 0x558b7f7f9a70 2024-04-23T14:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.402+0000 7fdb3aa00640 1 -- 172.21.15.83:0/1246576347 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4c004030 con 0x558b7f7f9a70 2024-04-23T14:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.402+0000 7fdb3aa00640 1 -- 172.21.15.83:0/1246576347 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fdb4c005050 con 0x558b7f7f9a70 2024-04-23T14:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.402+0000 7fdb5e5f3740 10 client.25883 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.402+0000 7fdb5e5f3740 10 client.25883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.402+0000 7fdb5e5f3740 20 client.25883 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.402+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/1246576347 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558b7f81ad60 0x558b7f81d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.402+0000 7fdb5e5f3740 4 client.25883 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.402+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1246576347 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b7f759890 con 0x558b7f81ad60 2024-04-23T14:50:49.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.403+0000 7fdb58e00640 1 --2- 172.21.15.83:0/1246576347 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558b7f81ad60 0x558b7f81d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.403+0000 7fdb58e00640 1 --2- 172.21.15.83:0/1246576347 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558b7f81ad60 0x558b7f81d150 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e853f45d9e543bf9 server_cookie=2892ae76c9adbeab in_seq=0 out_seq=0 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.403+0000 7fdb3aa00640 10 client.25883 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.404+0000 7fdb3aa00640 1 -- 172.21.15.83:0/1246576347 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b7f759890 con 0x558b7f81ad60 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.404+0000 7fdb3aa00640 10 client.25883 handle_command_reply: tid=0 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.404+0000 7fdb5e5f3740 1 client.25883 shutdown 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.404+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1246576347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 msgr2=0x558b7f7f7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.404+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/1246576347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f7f7480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c010be0 tx=0x7fdb4c003ba0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.404+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1246576347 shutdown_connections 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.404+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/1246576347 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558b7f81ad60 0x558b7f81d150 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.404+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/1246576347 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f7f7480 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.404+0000 7fdb3aa00640 0 client.25883 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.404+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/1246576347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e6970 0x558b7f7f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.404+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/1246576347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e5fc0 0x558b7f7f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.404+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1246576347 >> 172.21.15.83:0/1246576347 conn(0x558b7f7e2590 msgr2=0x558b7f7e8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.405+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1246576347 shutdown_connections 2024-04-23T14:50:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.405+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1246576347 wait complete. 2024-04-23T14:50:49.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.405+0000 7fdb5e5f3740 20 client.25883 trim_cache size 0 max 16384 2024-04-23T14:50:49.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.456+0000 7fdb5e5f3740 1 Processor -- start 2024-04-23T14:50:49.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.457+0000 7fdb5e5f3740 1 -- start start 2024-04-23T14:50:49.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.457+0000 7fdb5e5f3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f805790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.457+0000 7fdb5e5f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7e8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.457+0000 7fdb5e5f3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f822750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.458+0000 7fdb5e5f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54072630 con 0x558b7f7e6970 2024-04-23T14:50:49.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.458+0000 7fdb5e5f3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5406fed0 con 0x558b7f7f9a70 2024-04-23T14:50:49.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.458+0000 7fdb5e5f3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5406dd20 con 0x558b7f7e5fc0 2024-04-23T14:50:49.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.458+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.458+0000 7fdb58e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f805790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.458+0000 7fdb53e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7e8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42380/0 (socket says 172.21.15.83:42380) 2024-04-23T14:50:49.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.458+0000 7fdb53e00640 1 -- 172.21.15.83:0/1478691866 learned_addr learned my addr 172.21.15.83:0/1478691866 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:49.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.458+0000 7fdb59800640 1 --2- 172.21.15.83:0/1478691866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f822750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.459+0000 7fdb53e00640 1 -- 172.21.15.83:0/1478691866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 msgr2=0x558b7f805790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.459+0000 7fdb53e00640 1 --2- 172.21.15.83:0/1478691866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f805790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.459+0000 7fdb53e00640 1 -- 172.21.15.83:0/1478691866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 msgr2=0x558b7f822750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.459+0000 7fdb53e00640 1 --2- 172.21.15.83:0/1478691866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f822750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.459+0000 7fdb53e00640 1 -- 172.21.15.83:0/1478691866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b7f7f1840 con 0x558b7f7e6970 2024-04-23T14:50:49.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.463+0000 7fdb53e00640 1 --2- 172.21.15.83:0/1478691866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7e8390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdb40035040 tx=0x7fdb400a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfc9a276de93384 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:49.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.464+0000 7fdb45600640 1 -- 172.21.15.83:0/1478691866 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb400bcec0 con 0x558b7f7e6970 2024-04-23T14:50:49.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.464+0000 7fdb45600640 1 -- 172.21.15.83:0/1478691866 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb400bc7c0 con 0x558b7f7e6970 2024-04-23T14:50:49.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.464+0000 7fdb45600640 1 -- 172.21.15.83:0/1478691866 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb400b8e30 con 0x558b7f7e6970 2024-04-23T14:50:49.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.466+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1478691866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 msgr2=0x558b7f7e8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:49.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.466+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/1478691866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7e8390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdb40035040 tx=0x7fdb400a7040 comp rx=0 tx=0).stop 2024-04-23T14:50:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.469+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1478691866 shutdown_connections 2024-04-23T14:50:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.469+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/1478691866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f822750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.469+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/1478691866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7e8390 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.469+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/1478691866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f805790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.469+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1478691866 >> 172.21.15.83:0/1478691866 conn(0x558b7f7e2590 msgr2=0x558b7f7e1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:49.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.469+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1478691866 shutdown_connections 2024-04-23T14:50:49.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.469+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/1478691866 wait complete. 2024-04-23T14:50:49.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.470+0000 7fdb5e5f3740 1 -- start start 2024-04-23T14:50:49.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.470+0000 7fdb5e5f3740 1 Processor -- start 2024-04-23T14:50:49.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.471+0000 7fdb5e5f3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f7f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.471+0000 7fdb5e5f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7f4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.471+0000 7fdb5e5f3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f87ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.471+0000 7fdb5e5f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5406e390 con 0x558b7f7e6970 2024-04-23T14:50:49.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.471+0000 7fdb5e5f3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5406f800 con 0x558b7f7f9a70 2024-04-23T14:50:49.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.471+0000 7fdb5e5f3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5406ea60 con 0x558b7f7e5fc0 2024-04-23T14:50:49.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.471+0000 7fdb5e5f3740 10 client.0 _ll_register_callbacks cb 0x7fdb48009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:49.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.472+0000 7fdb59800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f87ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.472+0000 7fdb59800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f87ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54932/0 (socket says 172.21.15.83:54932) 2024-04-23T14:50:49.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.472+0000 7fdb59800640 1 -- 172.21.15.83:0/2224056483 learned_addr learned my addr 172.21.15.83:0/2224056483 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:49.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.472+0000 7fdb59800640 1 -- 172.21.15.83:0/2224056483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 msgr2=0x558b7f7f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:49.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.473+0000 7fdb58e00640 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f7f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.473+0000 7fdb59800640 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f7f3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.473+0000 7fdb59800640 1 -- 172.21.15.83:0/2224056483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 msgr2=0x558b7f7f4040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:50:49.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.474+0000 7fdb53e00640 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.475+0000 7fdb59800640 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.477+0000 7fdb59800640 1 -- 172.21.15.83:0/2224056483 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b7f7f4940 con 0x558b7f7f9a70 2024-04-23T14:50:49.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.477+0000 7fdb58e00640 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f7f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:50:49.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.477+0000 7fdb53e00640 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:50:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.478+0000 7fdb59800640 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f87ffc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c010c80 tx=0x7fdb4c003c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8c5d4b5250b27b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.478+0000 7fdb3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:50:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.479+0000 7fdb3aa00640 1 -- 172.21.15.83:0/2224056483 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0049e0 con 0x558b7f7f9a70 2024-04-23T14:50:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.479+0000 7fdb3aa00640 1 -- 172.21.15.83:0/2224056483 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fdb4c013190 con 0x558b7f7f9a70 2024-04-23T14:50:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.479+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/2224056483 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb400bc7c0 con 0x558b7f7f9a70 2024-04-23T14:50:49.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.480+0000 7fdb3aa00640 1 -- 172.21.15.83:0/2224056483 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdb4c013390 con 0x558b7f7f9a70 2024-04-23T14:50:49.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.481+0000 7fdb5e5f3740 10 client.25889 fetch_fsmap learned FSMap version 26 2024-04-23T14:50:49.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.481+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/2224056483 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558b7f7e9df0 con 0x558b7f7f9a70 2024-04-23T14:50:49.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.482+0000 7fdb3aa00640 1 -- 172.21.15.83:0/2224056483 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb400bc7c0 con 0x558b7f7f9a70 2024-04-23T14:50:49.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.483+0000 7fdb3aa00640 1 -- 172.21.15.83:0/2224056483 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fdb4c03e020 con 0x558b7f7f9a70 2024-04-23T14:50:49.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.483+0000 7fdb5e5f3740 10 client.25889 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:50:49.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.483+0000 7fdb5e5f3740 10 client.25889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:49.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.483+0000 7fdb5e5f3740 20 client.25889 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.484+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558b7f81ad60 0x558b7f7ec600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.484+0000 7fdb5e5f3740 4 client.25889 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:50:49.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.484+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/2224056483 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558b7f753700 con 0x558b7f81ad60 2024-04-23T14:50:49.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.485+0000 7fdb58e00640 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558b7f81ad60 0x558b7f7ec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.490+0000 7fdb58e00640 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558b7f81ad60 0x558b7f7ec600 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ae04f5f8d8b3b2 server_cookie=17a2e8a3b223854e in_seq=0 out_seq=0 2024-04-23T14:50:49.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.490+0000 7fdb3aa00640 10 client.25889 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.491+0000 7fdb3aa00640 1 -- 172.21.15.83:0/2224056483 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558b7f753700 con 0x558b7f81ad60 2024-04-23T14:50:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.491+0000 7fdb3aa00640 10 client.25889 handle_command_reply: tid=0 2024-04-23T14:50:49.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.491+0000 7fdb5e5f3740 1 client.25889 shutdown 2024-04-23T14:50:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.491+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/2224056483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 msgr2=0x558b7f87ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.491+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f87ffc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c010c80 tx=0x7fdb4c003c90 comp rx=0 tx=0).stop 2024-04-23T14:50:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.492+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/2224056483 shutdown_connections 2024-04-23T14:50:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.492+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558b7f81ad60 0x558b7f7ec600 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.492+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558b7f7f9a70 0x558b7f87ffc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.492+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b7f7e6970 0x558b7f7f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.492+0000 7fdb5e5f3740 1 --2- 172.21.15.83:0/2224056483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558b7f7e5fc0 0x558b7f7f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.492+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/2224056483 >> 172.21.15.83:0/2224056483 conn(0x558b7f7e2590 msgr2=0x558b7f7f5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:49.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.492+0000 7fdb3aa00640 0 client.25889 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:49.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.495+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/2224056483 shutdown_connections 2024-04-23T14:50:49.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.495+0000 7fdb5e5f3740 1 -- 172.21.15.83:0/2224056483 wait complete. 2024-04-23T14:50:49.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.495+0000 7fdb5e5f3740 20 client.25889 trim_cache size 0 max 16384 2024-04-23T14:50:49.497 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:50:49.498 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:50:49.498 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "51c2fd65-83fe-4041-b9cf-3cc0d7adc56c", 2024-04-23T14:50:49.498 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:50:49.498 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:50:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.510+0000 7fdb5a200640 1 -- 172.21.15.83:0/1171545602 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdb3c0a3d20 msgr2=0x7fdb3c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.510+0000 7fdb5a200640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdb3c0a3d20 0x7fdb3c0a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c005fd0 tx=0x7fdb4c0107a0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.510+0000 7fdb5a200640 1 -- 172.21.15.83:0/1171545602 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb541028e0 msgr2=0x7fdb5419bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:49.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.510+0000 7fdb5a200640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb541028e0 0x7fdb5419bbf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009980 tx=0x7fdb40033d90 comp rx=0 tx=0).stop 2024-04-23T14:50:49.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.518+0000 7fdb5a200640 1 -- 172.21.15.83:0/1171545602 shutdown_connections 2024-04-23T14:50:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.518+0000 7fdb5a200640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb5410b390 0x7fdb541a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.518+0000 7fdb5a200640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb541028e0 0x7fdb5419bbf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.518+0000 7fdb5a200640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fdb3c0a3d20 0x7fdb3c0a61e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.518+0000 7fdb5a200640 1 --2- 172.21.15.83:0/1171545602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb54101fc0 0x7fdb5419b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.518+0000 7fdb5a200640 1 -- 172.21.15.83:0/1171545602 >> 172.21.15.83:0/1171545602 conn(0x7fdb54075d00 msgr2=0x7fdb5410be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.519+0000 7fdb5a200640 1 -- 172.21.15.83:0/1171545602 shutdown_connections 2024-04-23T14:50:49.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.519+0000 7fdb5a200640 1 -- 172.21.15.83:0/1171545602 wait complete. 2024-04-23T14:50:49.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:50:49.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:50:49.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:49 smithi173 ceph-mon[32443]: pgmap v73: 97 pgs: 97 active+clean; 1.8 GiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 148 MiB/s wr, 64 op/s 2024-04-23T14:50:49.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.955+0000 7f1a60200640 1 -- 172.21.15.83:0/788188432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a58108610 msgr2=0x7f1a58108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:49.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.956+0000 7f1a60200640 1 --2- 172.21.15.83:0/788188432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a58108610 0x7f1a58108a90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009900 tx=0x7f1a4402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.958+0000 7f1a60200640 1 -- 172.21.15.83:0/788188432 shutdown_connections 2024-04-23T14:50:49.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.958+0000 7f1a60200640 1 --2- 172.21.15.83:0/788188432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a58108fd0 0x7f1a58115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.959+0000 7f1a60200640 1 --2- 172.21.15.83:0/788188432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a58108610 0x7f1a58108a90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.960+0000 7f1a60200640 1 --2- 172.21.15.83:0/788188432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a5810e660 0x7f1a5810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.960+0000 7f1a60200640 1 -- 172.21.15.83:0/788188432 >> 172.21.15.83:0/788188432 conn(0x7f1a58076540 msgr2=0x7f1a58078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:49.966 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.962+0000 7f1a60200640 1 -- 172.21.15.83:0/788188432 shutdown_connections 2024-04-23T14:50:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.963+0000 7f1a60200640 1 -- 172.21.15.83:0/788188432 wait complete. 2024-04-23T14:50:49.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.966+0000 7f1a60200640 1 Processor -- start 2024-04-23T14:50:49.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.967+0000 7f1a60200640 1 -- start start 2024-04-23T14:50:49.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.968+0000 7f1a60200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a58108610 0x7f1a580825c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.969+0000 7f1a60200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a58108fd0 0x7f1a58082b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.969+0000 7f1a5d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a58108610 0x7f1a580825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.970+0000 7f1a60200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a5810e660 0x7f1a5807b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.970+0000 7f1a5d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a58108610 0x7f1a580825c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42414/0 (socket says 172.21.15.83:42414) 2024-04-23T14:50:49.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.970+0000 7f1a5ce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a58108fd0 0x7f1a58082b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:49 smithi203 ceph-mon[32800]: pgmap v73: 97 pgs: 97 active+clean; 1.8 GiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 148 MiB/s wr, 64 op/s 2024-04-23T14:50:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.970+0000 7f1a5ce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a58108fd0 0x7f1a58082b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55572/0 (socket says 172.21.15.83:55572) 2024-04-23T14:50:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.970+0000 7f1a5ce00640 1 -- 172.21.15.83:0/3292701310 learned_addr learned my addr 172.21.15.83:0/3292701310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.970+0000 7f1a5e200640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a5810e660 0x7f1a5807b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.970+0000 7f1a60200640 1 -- 172.21.15.83:0/3292701310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581189f0 con 0x7f1a58108610 2024-04-23T14:50:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.970+0000 7f1a60200640 1 -- 172.21.15.83:0/3292701310 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a58118870 con 0x7f1a5810e660 2024-04-23T14:50:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.970+0000 7f1a60200640 1 -- 172.21.15.83:0/3292701310 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1a58118b70 con 0x7f1a58108fd0 2024-04-23T14:50:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.971+0000 7f1a5e200640 1 -- 172.21.15.83:0/3292701310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a58108fd0 msgr2=0x7f1a58082b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.971+0000 7f1a5e200640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a58108fd0 0x7f1a58082b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.971+0000 7f1a5e200640 1 -- 172.21.15.83:0/3292701310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a58108610 msgr2=0x7f1a580825c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.972+0000 7f1a5e200640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a58108610 0x7f1a580825c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.972+0000 7f1a5e200640 1 -- 172.21.15.83:0/3292701310 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5807c0c0 con 0x7f1a5810e660 2024-04-23T14:50:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.972+0000 7f1a5d800640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a58108610 0x7f1a580825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.973+0000 7f1a5e200640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a5810e660 0x7f1a5807b800 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000e940 tx=0x7f1a5000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:49.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.973+0000 7f1a56000640 1 -- 172.21.15.83:0/3292701310 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50004d20 con 0x7f1a5810e660 2024-04-23T14:50:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.974+0000 7f1a56000640 1 -- 172.21.15.83:0/3292701310 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a50005020 con 0x7f1a5810e660 2024-04-23T14:50:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.974+0000 7f1a56000640 1 -- 172.21.15.83:0/3292701310 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500116b0 con 0x7f1a5810e660 2024-04-23T14:50:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.974+0000 7f1a60200640 1 -- 172.21.15.83:0/3292701310 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5807c3b0 con 0x7f1a5810e660 2024-04-23T14:50:49.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.974+0000 7f1a60200640 1 -- 172.21.15.83:0/3292701310 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a58080380 con 0x7f1a5810e660 2024-04-23T14:50:49.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.983+0000 7f1a63174740 1 Processor -- start 2024-04-23T14:50:49.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.988+0000 7f1a56000640 1 -- 172.21.15.83:0/3292701310 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1a5000cb20 con 0x7f1a5810e660 2024-04-23T14:50:49.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.988+0000 7f1a63174740 1 -- start start 2024-04-23T14:50:49.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.989+0000 7f1a63174740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a437c860 0x55d6a437cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.989+0000 7f1a63174740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437d180 0x55d6a4387310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.989+0000 7f1a63174740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 0x55d6a4389c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.989+0000 7f1a63174740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6a42e21f0 con 0x55d6a4387850 2024-04-23T14:50:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.989+0000 7f1a63174740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d6a4288350 con 0x55d6a437d180 2024-04-23T14:50:49.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.989+0000 7f1a63174740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d6a42de1b0 con 0x55d6a437c860 2024-04-23T14:50:49.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.989+0000 7f1a5ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 0x55d6a4389c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.990+0000 7f1a5ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 0x55d6a4389c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42420/0 (socket says 172.21.15.83:42420) 2024-04-23T14:50:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.990+0000 7f1a5ce00640 1 -- 172.21.15.83:0/482673849 learned_addr learned my addr 172.21.15.83:0/482673849 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.990+0000 7f1a5d800640 1 --2- 172.21.15.83:0/482673849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437d180 0x55d6a4387310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.990+0000 7f1a5ce00640 1 -- 172.21.15.83:0/482673849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a437c860 msgr2=0x55d6a437cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.990+0000 7f1a5ce00640 1 --2- 172.21.15.83:0/482673849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a437c860 0x55d6a437cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.990+0000 7f1a5ce00640 1 -- 172.21.15.83:0/482673849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437d180 msgr2=0x55d6a4387310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.990+0000 7f1a5ce00640 1 --2- 172.21.15.83:0/482673849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437d180 0x55d6a4387310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.990+0000 7f1a5ce00640 1 -- 172.21.15.83:0/482673849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6a42dcf50 con 0x55d6a4387850 2024-04-23T14:50:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.990+0000 7f1a5d800640 1 --2- 172.21.15.83:0/482673849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437d180 0x55d6a4387310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:50:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.991+0000 7f1a56000640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1a400a3e70 0x7f1a400a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.991+0000 7f1a56000640 1 -- 172.21.15.83:0/3292701310 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a500d0a60 con 0x7f1a5810e660 2024-04-23T14:50:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.991+0000 7f1a5d800640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1a400a3e70 0x7f1a400a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:49.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.992+0000 7f1a5ce00640 1 --2- 172.21.15.83:0/482673849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 0x55d6a4389c40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005ab0 tx=0x7f1a44009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebab78c8de7adf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:49.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.992+0000 7f1a5d800640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1a400a3e70 0x7f1a400a6330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009fd0 tx=0x7f1a4c005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:49.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.993+0000 7f1a4aa00640 1 -- 172.21.15.83:0/482673849 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44041070 con 0x55d6a4387850 2024-04-23T14:50:49.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.993+0000 7f1a4aa00640 1 -- 172.21.15.83:0/482673849 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a440043c0 con 0x55d6a4387850 2024-04-23T14:50:49.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.993+0000 7f1a4aa00640 1 -- 172.21.15.83:0/482673849 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44035040 con 0x55d6a4387850 2024-04-23T14:50:50.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.994+0000 7f1a63174740 1 -- 172.21.15.83:0/482673849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 msgr2=0x55d6a4389c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.994+0000 7f1a63174740 1 --2- 172.21.15.83:0/482673849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 0x55d6a4389c40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005ab0 tx=0x7f1a44009630 comp rx=0 tx=0).stop 2024-04-23T14:50:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.996+0000 7f1a63174740 1 -- 172.21.15.83:0/482673849 shutdown_connections 2024-04-23T14:50:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.996+0000 7f1a63174740 1 --2- 172.21.15.83:0/482673849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 0x55d6a4389c40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.996+0000 7f1a63174740 1 --2- 172.21.15.83:0/482673849 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437d180 0x55d6a4387310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.996+0000 7f1a63174740 1 --2- 172.21.15.83:0/482673849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a437c860 0x55d6a437cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.996+0000 7f1a63174740 1 -- 172.21.15.83:0/482673849 >> 172.21.15.83:0/482673849 conn(0x55d6a4378b80 msgr2=0x55d6a4378f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.996+0000 7f1a63174740 1 -- 172.21.15.83:0/482673849 shutdown_connections 2024-04-23T14:50:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.996+0000 7f1a63174740 1 -- 172.21.15.83:0/482673849 wait complete. 2024-04-23T14:50:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.997+0000 7f1a63174740 1 -- start start 2024-04-23T14:50:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.997+0000 7f1a63174740 1 Processor -- start 2024-04-23T14:50:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.998+0000 7f1a63174740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a437c860 0x55d6a43847d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:50.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.998+0000 7f1a63174740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437d180 0x55d6a4384d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.998+0000 7f1a63174740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 0x55d6a4385250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.998+0000 7f1a63174740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d6a438a5e0 con 0x55d6a4387850 2024-04-23T14:50:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.998+0000 7f1a63174740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d6a429b690 con 0x55d6a437d180 2024-04-23T14:50:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.998+0000 7f1a63174740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d6a438a760 con 0x55d6a437c860 2024-04-23T14:50:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.998+0000 7f1a63174740 10 client.0 _ll_register_callbacks cb 0x55d6a42d7650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:50.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.998+0000 7f1a5d800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a437c860 0x55d6a43847d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.998+0000 7f1a5d800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a437c860 0x55d6a43847d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55592/0 (socket says 172.21.15.83:55592) 2024-04-23T14:50:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.998+0000 7f1a5d800640 1 -- 172.21.15.83:0/2933232096 learned_addr learned my addr 172.21.15.83:0/2933232096 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:50.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.998+0000 7f1a5e200640 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 0x55d6a4385250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:50.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.998+0000 7f1a5ce00640 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437d180 0x55d6a4384d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:50.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.999+0000 7f1a5d800640 1 -- 172.21.15.83:0/2933232096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437d180 msgr2=0x55d6a4384d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:50.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.999+0000 7f1a5d800640 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437d180 0x55d6a4384d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.999+0000 7f1a5d800640 1 -- 172.21.15.83:0/2933232096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 msgr2=0x55d6a4385250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:50.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.999+0000 7f1a5d800640 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 0x55d6a4385250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.999+0000 7f1a5d800640 1 -- 172.21.15.83:0/2933232096 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6a429a920 con 0x55d6a437c860 2024-04-23T14:50:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.999+0000 7f1a5e200640 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 0x55d6a4385250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:50:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.999+0000 7f1a5d800640 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a437c860 0x55d6a43847d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00f740 tx=0x7f1a4c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae83788038ad1532 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.999+0000 7f1a3f400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:50:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.999+0000 7f1a3f400640 1 -- 172.21.15.83:0/2933232096 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c0118d0 con 0x55d6a437c860 2024-04-23T14:50:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.999+0000 7f1a3f400640 1 -- 172.21.15.83:0/2933232096 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a4c011a70 con 0x55d6a437c860 2024-04-23T14:50:50.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:49.999+0000 7f1a63174740 1 -- 172.21.15.83:0/2933232096 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a440043c0 con 0x55d6a437c860 2024-04-23T14:50:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.000+0000 7f1a3f400640 1 -- 172.21.15.83:0/2933232096 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a4c011c70 con 0x55d6a437c860 2024-04-23T14:50:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.000+0000 7f1a63174740 10 client.25867 fetch_fsmap learned FSMap version 26 2024-04-23T14:50:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.000+0000 7f1a63174740 1 -- 172.21.15.83:0/2933232096 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d6a40be7d0 con 0x55d6a437c860 2024-04-23T14:50:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.000+0000 7f1a3f400640 1 -- 172.21.15.83:0/2933232096 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a440043c0 con 0x55d6a437c860 2024-04-23T14:50:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.000+0000 7f1a3f400640 1 -- 172.21.15.83:0/2933232096 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f1a4c027070 con 0x55d6a437c860 2024-04-23T14:50:50.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.000+0000 7f1a63174740 10 client.25867 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:50:50.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.000+0000 7f1a63174740 10 client.25867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:50.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.000+0000 7f1a63174740 20 client.25867 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:50.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.000+0000 7f1a63174740 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d6a4397d60 0x55d6a439a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:50.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.000+0000 7f1a63174740 4 client.25867 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:50:50.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.000+0000 7f1a63174740 1 -- 172.21.15.83:0/2933232096 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6a42efc10 con 0x55d6a4397d60 2024-04-23T14:50:50.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.000+0000 7f1a5ce00640 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d6a4397d60 0x55d6a439a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.001+0000 7f1a5ce00640 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d6a4397d60 0x55d6a439a200 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d65a35b61f95a58 server_cookie=ccd952b8ec29f717 in_seq=0 out_seq=0 2024-04-23T14:50:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.001+0000 7f1a3f400640 10 client.25867 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.002+0000 7f1a3f400640 1 -- 172.21.15.83:0/2933232096 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6a42efc10 con 0x55d6a4397d60 2024-04-23T14:50:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.002+0000 7f1a3f400640 10 client.25867 handle_command_reply: tid=0 2024-04-23T14:50:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.002+0000 7f1a63174740 1 client.25867 shutdown 2024-04-23T14:50:50.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.002+0000 7f1a63174740 1 -- 172.21.15.83:0/2933232096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a437c860 msgr2=0x55d6a43847d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.002+0000 7f1a63174740 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a437c860 0x55d6a43847d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00f740 tx=0x7f1a4c004730 comp rx=0 tx=0).stop 2024-04-23T14:50:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.002+0000 7f1a63174740 1 -- 172.21.15.83:0/2933232096 shutdown_connections 2024-04-23T14:50:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.002+0000 7f1a63174740 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d6a4397d60 0x55d6a439a200 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.002+0000 7f1a63174740 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a4387850 0x55d6a4385250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.002+0000 7f1a63174740 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437d180 0x55d6a4384d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.002+0000 7f1a3f400640 0 client.25867 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:50.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.002+0000 7f1a63174740 1 --2- 172.21.15.83:0/2933232096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a437c860 0x55d6a43847d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.002+0000 7f1a63174740 1 -- 172.21.15.83:0/2933232096 >> 172.21.15.83:0/2933232096 conn(0x55d6a4378b80 msgr2=0x55d6a437e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:50.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.003+0000 7f1a63174740 1 -- 172.21.15.83:0/2933232096 shutdown_connections 2024-04-23T14:50:50.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.003+0000 7f1a63174740 1 -- 172.21.15.83:0/2933232096 wait complete. 2024-04-23T14:50:50.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.003+0000 7f1a63174740 20 client.25867 trim_cache size 0 max 16384 2024-04-23T14:50:50.015 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:49 smithi083 ceph-mon[27867]: pgmap v73: 97 pgs: 97 active+clean; 1.8 GiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 148 MiB/s wr, 64 op/s 2024-04-23T14:50:50.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.059+0000 7f1a63174740 1 Processor -- start 2024-04-23T14:50:50.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.059+0000 7f1a63174740 1 -- start start 2024-04-23T14:50:50.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.061+0000 7f1a63174740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 0x55d6a4393530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:50.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.061+0000 7f1a63174740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a437d180 0x55d6a437efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:50.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.061+0000 7f1a63174740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a4387850 0x55d6a439fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:50.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.062+0000 7f1a63174740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1a38011040 con 0x55d6a437d180 2024-04-23T14:50:50.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.062+0000 7f1a63174740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3800e8e0 con 0x55d6a437c860 2024-04-23T14:50:50.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.062+0000 7f1a63174740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3800c730 con 0x55d6a4387850 2024-04-23T14:50:50.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.062+0000 7f1a5e200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a4387850 0x55d6a439fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:50.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.062+0000 7f1a5e200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a4387850 0x55d6a439fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55604/0 (socket says 172.21.15.83:55604) 2024-04-23T14:50:50.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.062+0000 7f1a5e200640 1 -- 172.21.15.83:0/2181597326 learned_addr learned my addr 172.21.15.83:0/2181597326 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:50.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.062+0000 7f1a5d800640 1 --2- 172.21.15.83:0/2181597326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 0x55d6a4393530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:50.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.063+0000 7f1a5ce00640 1 --2- 172.21.15.83:0/2181597326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a437d180 0x55d6a437efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:50.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.063+0000 7f1a5e200640 1 -- 172.21.15.83:0/2181597326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 msgr2=0x55d6a4393530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:50.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.063+0000 7f1a5e200640 1 --2- 172.21.15.83:0/2181597326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 0x55d6a4393530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.063+0000 7f1a5e200640 1 -- 172.21.15.83:0/2181597326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a437d180 msgr2=0x55d6a437efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:50.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.063+0000 7f1a5e200640 1 --2- 172.21.15.83:0/2181597326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a437d180 0x55d6a437efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.063+0000 7f1a5e200640 1 -- 172.21.15.83:0/2181597326 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6a3ff43f0 con 0x55d6a4387850 2024-04-23T14:50:50.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.063+0000 7f1a5d800640 1 --2- 172.21.15.83:0/2181597326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 0x55d6a4393530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:50:50.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.063+0000 7f1a5ce00640 1 --2- 172.21.15.83:0/2181597326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a437d180 0x55d6a437efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:50:50.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.064+0000 7f1a5e200640 1 --2- 172.21.15.83:0/2181597326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a4387850 0x55d6a439fad0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a500059e0 tx=0x7f1a5009d1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b98841695adeb07f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.064+0000 7f1a4a000640 1 -- 172.21.15.83:0/2181597326 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5001f040 con 0x55d6a4387850 2024-04-23T14:50:50.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.064+0000 7f1a4a000640 1 -- 172.21.15.83:0/2181597326 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a500a0dc0 con 0x55d6a4387850 2024-04-23T14:50:50.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.064+0000 7f1a4a000640 1 -- 172.21.15.83:0/2181597326 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500bbe30 con 0x55d6a4387850 2024-04-23T14:50:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.070+0000 7f1a63174740 1 -- 172.21.15.83:0/2181597326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a4387850 msgr2=0x55d6a439fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.070+0000 7f1a63174740 1 --2- 172.21.15.83:0/2181597326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a4387850 0x55d6a439fad0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1a500059e0 tx=0x7f1a5009d1d0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.070+0000 7f1a63174740 1 -- 172.21.15.83:0/2181597326 shutdown_connections 2024-04-23T14:50:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.070+0000 7f1a63174740 1 --2- 172.21.15.83:0/2181597326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a4387850 0x55d6a439fad0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.070+0000 7f1a63174740 1 --2- 172.21.15.83:0/2181597326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a437d180 0x55d6a437efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.070+0000 7f1a63174740 1 --2- 172.21.15.83:0/2181597326 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 0x55d6a4393530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.070+0000 7f1a63174740 1 -- 172.21.15.83:0/2181597326 >> 172.21.15.83:0/2181597326 conn(0x55d6a4378b80 msgr2=0x55d6a4377c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:50.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.071+0000 7f1a63174740 1 -- 172.21.15.83:0/2181597326 shutdown_connections 2024-04-23T14:50:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.071+0000 7f1a63174740 1 -- 172.21.15.83:0/2181597326 wait complete. 2024-04-23T14:50:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.071+0000 7f1a63174740 1 -- start start 2024-04-23T14:50:50.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.072+0000 7f1a63174740 1 Processor -- start 2024-04-23T14:50:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.074+0000 7f1a63174740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 0x55d6a4381c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:50.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.074+0000 7f1a63174740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a437d180 0x55d6a4382140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:50.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.075+0000 7f1a63174740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a4387850 0x55d6a43fd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.075+0000 7f1a5d800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 0x55d6a4381c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.076+0000 7f1a5d800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 0x55d6a4381c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54994/0 (socket says 172.21.15.83:54994) 2024-04-23T14:50:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.076+0000 7f1a63174740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3800cda0 con 0x55d6a437d180 2024-04-23T14:50:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.076+0000 7f1a63174740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3800e210 con 0x55d6a437c860 2024-04-23T14:50:50.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.077+0000 7f1a63174740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3800d470 con 0x55d6a4387850 2024-04-23T14:50:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.077+0000 7f1a63174740 10 client.0 _ll_register_callbacks cb 0x7f1a4c009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:50:50.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.077+0000 7f1a5d800640 1 -- 172.21.15.83:0/1838245407 learned_addr learned my addr 172.21.15.83:0/1838245407 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:50:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.077+0000 7f1a5e200640 1 --2- 172.21.15.83:0/1838245407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a4387850 0x55d6a43fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.078+0000 7f1a5d800640 1 -- 172.21.15.83:0/1838245407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a4387850 msgr2=0x55d6a43fd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.078+0000 7f1a5d800640 1 --2- 172.21.15.83:0/1838245407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a4387850 0x55d6a43fd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.078+0000 7f1a5d800640 1 -- 172.21.15.83:0/1838245407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a437d180 msgr2=0x55d6a4382140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:50:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.078+0000 7f1a5d800640 1 --2- 172.21.15.83:0/1838245407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a437d180 0x55d6a4382140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.078+0000 7f1a5d800640 1 -- 172.21.15.83:0/1838245407 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6a3fb40f0 con 0x55d6a437c860 2024-04-23T14:50:50.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.081+0000 7f1a5d800640 1 --2- 172.21.15.83:0/1838245407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 0x55d6a4381c00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00f3e0 tx=0x7f1a4c0114d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a4a9ebfa8ea21c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:50:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.081+0000 7f1a3f400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:50:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.081+0000 7f1a3f400640 1 -- 172.21.15.83:0/1838245407 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c01fc30 con 0x55d6a437c860 2024-04-23T14:50:50.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.082+0000 7f1a3f400640 1 -- 172.21.15.83:0/1838245407 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a4c01fdd0 con 0x55d6a437c860 2024-04-23T14:50:50.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.083+0000 7f1a63174740 1 -- 172.21.15.83:0/1838245407 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a500a0dc0 con 0x55d6a437c860 2024-04-23T14:50:50.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.084+0000 7f1a3f400640 1 -- 172.21.15.83:0/1838245407 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a4c03f050 con 0x55d6a437c860 2024-04-23T14:50:50.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.085+0000 7f1a63174740 10 client.25904 fetch_fsmap learned FSMap version 26 2024-04-23T14:50:50.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.085+0000 7f1a63174740 1 -- 172.21.15.83:0/1838245407 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d6a42a7810 con 0x55d6a437c860 2024-04-23T14:50:50.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.086+0000 7f1a3f400640 1 -- 172.21.15.83:0/1838245407 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a4c0215b0 con 0x55d6a437c860 2024-04-23T14:50:50.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.087+0000 7f1a3f400640 1 -- 172.21.15.83:0/1838245407 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f1a4c027040 con 0x55d6a437c860 2024-04-23T14:50:50.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.087+0000 7f1a63174740 10 client.25904 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:50:50.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.087+0000 7f1a63174740 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:50:50.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.087+0000 7f1a63174740 20 client.25904 populate_metadata read hostname 'smithi083' 2024-04-23T14:50:50.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.087+0000 7f1a63174740 1 --2- 172.21.15.83:0/1838245407 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d6a43fefb0 0x55d6a44013d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:50:50.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.087+0000 7f1a63174740 4 client.25904 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:50:50.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.087+0000 7f1a63174740 1 -- 172.21.15.83:0/1838245407 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1a500a0dc0 con 0x55d6a43fefb0 2024-04-23T14:50:50.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.088+0000 7f1a5ce00640 1 --2- 172.21.15.83:0/1838245407 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d6a43fefb0 0x55d6a44013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:50:50.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.089+0000 7f1a3f400640 10 client.25904 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:50.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.094+0000 7f1a5ce00640 1 --2- 172.21.15.83:0/1838245407 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d6a43fefb0 0x55d6a44013d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7122eb9b12aa2819 server_cookie=2c92bd877967d0b0 in_seq=0 out_seq=0 2024-04-23T14:50:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.096+0000 7f1a3f400640 1 -- 172.21.15.83:0/1838245407 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f1a500a0dc0 con 0x55d6a43fefb0 2024-04-23T14:50:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.096+0000 7f1a3f400640 10 client.25904 handle_command_reply: tid=0 2024-04-23T14:50:50.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.096+0000 7f1a63174740 1 client.25904 shutdown 2024-04-23T14:50:50.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.097+0000 7f1a63174740 1 -- 172.21.15.83:0/1838245407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 msgr2=0x55d6a4381c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.097+0000 7f1a63174740 1 --2- 172.21.15.83:0/1838245407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 0x55d6a4381c00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00f3e0 tx=0x7f1a4c0114d0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.097+0000 7f1a63174740 1 -- 172.21.15.83:0/1838245407 shutdown_connections 2024-04-23T14:50:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.097+0000 7f1a63174740 1 --2- 172.21.15.83:0/1838245407 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d6a43fefb0 0x55d6a44013d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.097+0000 7f1a63174740 1 --2- 172.21.15.83:0/1838245407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d6a4387850 0x55d6a43fd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.097+0000 7f1a63174740 1 --2- 172.21.15.83:0/1838245407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d6a437d180 0x55d6a4382140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.097+0000 7f1a63174740 1 --2- 172.21.15.83:0/1838245407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d6a437c860 0x55d6a4381c00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.097+0000 7f1a63174740 1 -- 172.21.15.83:0/1838245407 >> 172.21.15.83:0/1838245407 conn(0x55d6a4378b80 msgr2=0x55d6a4384790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:50.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.097+0000 7f1a3f400640 0 client.25904 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:50:50.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.101+0000 7f1a63174740 1 -- 172.21.15.83:0/1838245407 shutdown_connections 2024-04-23T14:50:50.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.101+0000 7f1a63174740 1 -- 172.21.15.83:0/1838245407 wait complete. 2024-04-23T14:50:50.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.102+0000 7f1a63174740 20 client.25904 trim_cache size 0 max 16384 2024-04-23T14:50:50.103 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:50:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:50:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:50:50.103 INFO:teuthology.orchestra.run.smithi083.stdout: "51c2fd65-83fe-4041-b9cf-3cc0d7adc56c": { 2024-04-23T14:50:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:50:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "51c2fd65-83fe-4041-b9cf-3cc0d7adc56c", 2024-04-23T14:50:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:50:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:50:50.104 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:50:50.104 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:50:50.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.103+0000 7f1a60200640 1 -- 172.21.15.83:0/3292701310 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1a400a3e70 msgr2=0x7f1a400a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:50.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.103+0000 7f1a60200640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1a400a3e70 0x7f1a400a6330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009fd0 tx=0x7f1a4c005f90 comp rx=0 tx=0).stop 2024-04-23T14:50:50.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.103+0000 7f1a60200640 1 -- 172.21.15.83:0/3292701310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a5810e660 msgr2=0x7f1a5807b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:50:50.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.103+0000 7f1a60200640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a5810e660 0x7f1a5807b800 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000e940 tx=0x7f1a5000ee00 comp rx=0 tx=0).stop 2024-04-23T14:50:50.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.104+0000 7f1a60200640 1 -- 172.21.15.83:0/3292701310 shutdown_connections 2024-04-23T14:50:50.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.104+0000 7f1a60200640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a5810e660 0x7f1a5807b800 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.104+0000 7f1a60200640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a58108fd0 0x7f1a58082b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.104+0000 7f1a60200640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1a400a3e70 0x7f1a400a6330 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.104+0000 7f1a60200640 1 --2- 172.21.15.83:0/3292701310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a58108610 0x7f1a580825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:50:50.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.104+0000 7f1a60200640 1 -- 172.21.15.83:0/3292701310 >> 172.21.15.83:0/3292701310 conn(0x7f1a58076540 msgr2=0x7f1a58077ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:50:50.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.104+0000 7f1a60200640 1 -- 172.21.15.83:0/3292701310 shutdown_connections 2024-04-23T14:50:50.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:50:50.105+0000 7f1a60200640 1 -- 172.21.15.83:0/3292701310 wait complete. 2024-04-23T14:50:50.133 INFO:tasks.cephfs.filesystem:scrub status for tag:51c2fd65-83fe-4041-b9cf-3cc0d7adc56c - {'path': '/', 'tag': '51c2fd65-83fe-4041-b9cf-3cc0d7adc56c', 'options': 'recursive,force'} 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:50:50.948 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.949 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:50:50.949 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.949 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:50 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:50:50.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:50:50.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:50:50.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:50:50.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:50:50.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:50:50.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:50:50.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:50:50.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:50.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:50 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:51.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:50:51.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:51.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:50:51.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:50:51.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:51.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:50:51.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:51.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:50:51.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:51.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:50:51.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:51.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:50:51.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:51.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:50:51.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:51.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:50 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:50:51.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:51 smithi203 ceph-mon[32800]: pgmap v74: 97 pgs: 97 active+clean; 2.9 GiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 246 MiB/s wr, 102 op/s 2024-04-23T14:50:52.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:51 smithi083 ceph-mon[27867]: pgmap v74: 97 pgs: 97 active+clean; 2.9 GiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 246 MiB/s wr, 102 op/s 2024-04-23T14:50:52.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:51 smithi173 ceph-mon[32443]: pgmap v74: 97 pgs: 97 active+clean; 2.9 GiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 246 MiB/s wr, 102 op/s 2024-04-23T14:50:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:53 smithi203 ceph-mon[32800]: pgmap v75: 97 pgs: 97 active+clean; 2.9 GiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 213 MiB/s wr, 78 op/s 2024-04-23T14:50:54.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:53 smithi083 ceph-mon[27867]: pgmap v75: 97 pgs: 97 active+clean; 2.9 GiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 213 MiB/s wr, 78 op/s 2024-04-23T14:50:54.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:53 smithi173 ceph-mon[32443]: pgmap v75: 97 pgs: 97 active+clean; 2.9 GiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 213 MiB/s wr, 78 op/s 2024-04-23T14:50:54.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:54 smithi203 ceph-mon[32800]: pgmap v76: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 318 MiB/s wr, 118 op/s 2024-04-23T14:50:55.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:54 smithi083 ceph-mon[27867]: pgmap v76: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 318 MiB/s wr, 118 op/s 2024-04-23T14:50:55.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:54 smithi173 ceph-mon[32443]: pgmap v76: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 318 MiB/s wr, 118 op/s 2024-04-23T14:50:57.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:56 smithi203 ceph-mon[32800]: pgmap v77: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 202 MiB/s wr, 76 op/s 2024-04-23T14:50:57.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:56 smithi083 ceph-mon[27867]: pgmap v77: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 202 MiB/s wr, 76 op/s 2024-04-23T14:50:57.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:56 smithi173 ceph-mon[32443]: pgmap v77: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 202 MiB/s wr, 76 op/s 2024-04-23T14:50:59.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:50:58 smithi203 ceph-mon[32800]: pgmap v78: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 202 MiB/s wr, 76 op/s 2024-04-23T14:50:59.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:50:58 smithi083 ceph-mon[27867]: pgmap v78: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 202 MiB/s wr, 76 op/s 2024-04-23T14:50:59.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:50:58 smithi173 ceph-mon[32443]: pgmap v78: 97 pgs: 97 active+clean; 4.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 202 MiB/s wr, 76 op/s 2024-04-23T14:51:01.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:00 smithi203 ceph-mon[32800]: pgmap v79: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 347 MiB/s wr, 126 op/s 2024-04-23T14:51:01.263 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:00 smithi083 ceph-mon[27867]: pgmap v79: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 347 MiB/s wr, 126 op/s 2024-04-23T14:51:01.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:00 smithi173 ceph-mon[32443]: pgmap v79: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 347 MiB/s wr, 126 op/s 2024-04-23T14:51:03.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:02 smithi203 ceph-mon[32800]: pgmap v80: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 250 MiB/s wr, 88 op/s 2024-04-23T14:51:03.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:02 smithi083 ceph-mon[27867]: pgmap v80: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 250 MiB/s wr, 88 op/s 2024-04-23T14:51:03.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:02 smithi173 ceph-mon[32443]: pgmap v80: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 250 MiB/s wr, 88 op/s 2024-04-23T14:51:04.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:03 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:51:04.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:03 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:51:04.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:03 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:51:04.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:03 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:51:04.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:03 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:51:04.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:03 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:51:04.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:03 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:51:04.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:03 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:51:04.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:03 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:51:05.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:04 smithi203 ceph-mon[32800]: pgmap v81: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 330 MiB/s wr, 137 op/s 2024-04-23T14:51:05.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:04 smithi083 ceph-mon[27867]: pgmap v81: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 330 MiB/s wr, 137 op/s 2024-04-23T14:51:05.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:04 smithi173 ceph-mon[32443]: pgmap v81: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 330 MiB/s wr, 137 op/s 2024-04-23T14:51:07.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:06 smithi203 ceph-mon[32800]: pgmap v82: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 226 MiB/s wr, 98 op/s 2024-04-23T14:51:07.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:06 smithi083 ceph-mon[27867]: pgmap v82: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 226 MiB/s wr, 98 op/s 2024-04-23T14:51:07.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:06 smithi173 ceph-mon[32443]: pgmap v82: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 226 MiB/s wr, 98 op/s 2024-04-23T14:51:09.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:08 smithi203 ceph-mon[32800]: pgmap v83: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 226 MiB/s wr, 98 op/s 2024-04-23T14:51:09.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:08 smithi083 ceph-mon[27867]: pgmap v83: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 226 MiB/s wr, 98 op/s 2024-04-23T14:51:09.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:08 smithi173 ceph-mon[32443]: pgmap v83: 97 pgs: 97 active+clean; 6.8 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 226 MiB/s wr, 98 op/s 2024-04-23T14:51:11.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:10 smithi203 ceph-mon[32800]: pgmap v84: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 356 MiB/s wr, 142 op/s 2024-04-23T14:51:11.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:10 smithi083 ceph-mon[27867]: pgmap v84: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 356 MiB/s wr, 142 op/s 2024-04-23T14:51:11.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:10 smithi173 ceph-mon[32443]: pgmap v84: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 356 MiB/s wr, 142 op/s 2024-04-23T14:51:13.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:12 smithi203 ceph-mon[32800]: pgmap v85: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 210 MiB/s wr, 92 op/s 2024-04-23T14:51:13.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:12 smithi083 ceph-mon[27867]: pgmap v85: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 210 MiB/s wr, 92 op/s 2024-04-23T14:51:13.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:12 smithi173 ceph-mon[32443]: pgmap v85: 97 pgs: 97 active+clean; 8.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 210 MiB/s wr, 92 op/s 2024-04-23T14:51:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:14 smithi203 ceph-mon[32800]: pgmap v86: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 242 MiB/s wr, 108 op/s 2024-04-23T14:51:15.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:14 smithi083 ceph-mon[27867]: pgmap v86: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 242 MiB/s wr, 108 op/s 2024-04-23T14:51:15.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:14 smithi173 ceph-mon[32443]: pgmap v86: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 242 MiB/s wr, 108 op/s 2024-04-23T14:51:17.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:16 smithi203 ceph-mon[32800]: pgmap v87: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 162 MiB/s wr, 58 op/s 2024-04-23T14:51:17.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:16 smithi083 ceph-mon[27867]: pgmap v87: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 162 MiB/s wr, 58 op/s 2024-04-23T14:51:17.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:16 smithi173 ceph-mon[32443]: pgmap v87: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 162 MiB/s wr, 58 op/s 2024-04-23T14:51:19.217 INFO:tasks.workunit.client.0.smithi083.stdout:fs setup took 48 secs 2024-04-23T14:51:19.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:18 smithi203 ceph-mon[32800]: pgmap v88: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 162 MiB/s wr, 58 op/s 2024-04-23T14:51:19.261 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:18 smithi083 ceph-mon[27867]: pgmap v88: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 162 MiB/s wr, 58 op/s 2024-04-23T14:51:19.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:18 smithi173 ceph-mon[32443]: pgmap v88: 97 pgs: 97 active+clean; 8.7 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 162 MiB/s wr, 58 op/s 2024-04-23T14:51:20.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:51:20.134 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:51:20.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.422+0000 7fe3e7200640 1 -- 172.21.15.83:0/1160279350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3e8102110 msgr2=0x7fe3e81024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.422+0000 7fe3e7200640 1 --2- 172.21.15.83:0/1160279350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3e8102110 0x7fe3e81024f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009be0 tx=0x7fe3e002a6d0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.423+0000 7fe3e7200640 1 -- 172.21.15.83:0/1160279350 shutdown_connections 2024-04-23T14:51:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.423+0000 7fe3e7200640 1 --2- 172.21.15.83:0/1160279350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3e81071e0 0x7fe3e810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.423+0000 7fe3e7200640 1 --2- 172.21.15.83:0/1160279350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8102ac0 0x7fe3e8106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.423+0000 7fe3e7200640 1 --2- 172.21.15.83:0/1160279350 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3e8102110 0x7fe3e81024f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.423+0000 7fe3e7200640 1 -- 172.21.15.83:0/1160279350 >> 172.21.15.83:0/1160279350 conn(0x7fe3e80fd9d0 msgr2=0x7fe3e80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:20.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.425+0000 7fe3e7200640 1 -- 172.21.15.83:0/1160279350 shutdown_connections 2024-04-23T14:51:20.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.426+0000 7fe3e7200640 1 -- 172.21.15.83:0/1160279350 wait complete. 2024-04-23T14:51:20.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.428+0000 7fe3e7200640 1 Processor -- start 2024-04-23T14:51:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.429+0000 7fe3e7200640 1 -- start start 2024-04-23T14:51:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.429+0000 7fe3e7200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8102110 0x7fe3e8116610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.429+0000 7fe3e7200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3e81071e0 0x7fe3e8116b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.429+0000 7fe3e7200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3e8111720 0x7fe3e8111bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.429+0000 7fe3e7200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e8075d30 con 0x7fe3e8102110 2024-04-23T14:51:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.429+0000 7fe3e7200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e8075bb0 con 0x7fe3e81071e0 2024-04-23T14:51:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.429+0000 7fe3e7200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e8075eb0 con 0x7fe3e8111720 2024-04-23T14:51:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.429+0000 7fe3e5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8102110 0x7fe3e8116610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.429+0000 7fe3e5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8102110 0x7fe3e8116610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56558/0 (socket says 172.21.15.83:56558) 2024-04-23T14:51:20.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.430+0000 7fe3e5e00640 1 -- 172.21.15.83:0/3300554601 learned_addr learned my addr 172.21.15.83:0/3300554601 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.430+0000 7fe3e5e00640 1 -- 172.21.15.83:0/3300554601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3e8111720 msgr2=0x7fe3e8111bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.430+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/3300554601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3e8111720 0x7fe3e8111bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.430+0000 7fe3e5e00640 1 -- 172.21.15.83:0/3300554601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3e81071e0 msgr2=0x7fe3e8116b50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T14:51:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.430+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/3300554601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3e81071e0 0x7fe3e8116b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.430+0000 7fe3e5e00640 1 -- 172.21.15.83:0/3300554601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e8112490 con 0x7fe3e8102110 2024-04-23T14:51:20.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.432+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/3300554601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8102110 0x7fe3e8116610 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0031040 tx=0x7fe3e0030640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.434+0000 7fe3dea00640 1 -- 172.21.15.83:0/3300554601 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0041020 con 0x7fe3e8102110 2024-04-23T14:51:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.434+0000 7fe3dea00640 1 -- 172.21.15.83:0/3300554601 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3e0031920 con 0x7fe3e8102110 2024-04-23T14:51:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.434+0000 7fe3dea00640 1 -- 172.21.15.83:0/3300554601 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0036040 con 0x7fe3e8102110 2024-04-23T14:51:20.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.435+0000 7fe3e7200640 1 -- 172.21.15.83:0/3300554601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e811c320 con 0x7fe3e8102110 2024-04-23T14:51:20.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.435+0000 7fe3e7200640 1 -- 172.21.15.83:0/3300554601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e811c820 con 0x7fe3e8102110 2024-04-23T14:51:20.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.450+0000 7fe3dea00640 1 -- 172.21.15.83:0/3300554601 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe3e0044020 con 0x7fe3e8102110 2024-04-23T14:51:20.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.450+0000 7fe3efbaa740 1 Processor -- start 2024-04-23T14:51:20.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.451+0000 7fe3dea00640 1 --2- 172.21.15.83:0/3300554601 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe3cc0a3ef0 0x7fe3cc0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.451+0000 7fe3efbaa740 1 -- start start 2024-04-23T14:51:20.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.451+0000 7fe3dea00640 1 -- 172.21.15.83:0/3300554601 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe3e00f0dc0 con 0x7fe3e8102110 2024-04-23T14:51:20.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.452+0000 7fe3e6800640 1 --2- 172.21.15.83:0/3300554601 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe3cc0a3ef0 0x7fe3cc0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.452+0000 7fe3efbaa740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7aa30 0x5651b7f7ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.452+0000 7fe3efbaa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7b350 0x5651b7f854e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.452+0000 7fe3efbaa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f85a20 0x5651b7f87e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.452+0000 7fe3efbaa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5651b7ee01f0 con 0x5651b7f85a20 2024-04-23T14:51:20.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.452+0000 7fe3efbaa740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5651b7e86350 con 0x5651b7f7aa30 2024-04-23T14:51:20.454 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.452+0000 7fe3efbaa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5651b7edc1b0 con 0x5651b7f7b350 2024-04-23T14:51:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.453+0000 7fe3e5e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7aa30 0x5651b7f7ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.453+0000 7fe3e6800640 1 --2- 172.21.15.83:0/3300554601 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe3cc0a3ef0 0x7fe3cc0a63b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005ab0 tx=0x7fe3d4005a00 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.453+0000 7fe3e5e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7aa30 0x5651b7f7ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37784/0 (socket says 172.21.15.83:37784) 2024-04-23T14:51:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.453+0000 7fe3e5e00640 1 -- 172.21.15.83:0/2213168984 learned_addr learned my addr 172.21.15.83:0/2213168984 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.453+0000 7fe3e6800640 1 --2- 172.21.15.83:0/2213168984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f85a20 0x5651b7f87e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.453+0000 7fe3e5400640 1 --2- 172.21.15.83:0/2213168984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7b350 0x5651b7f854e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.453+0000 7fe3e5e00640 1 -- 172.21.15.83:0/2213168984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7b350 msgr2=0x5651b7f854e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.453+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/2213168984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7b350 0x5651b7f854e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.453+0000 7fe3e5e00640 1 -- 172.21.15.83:0/2213168984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f85a20 msgr2=0x5651b7f87e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.453+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/2213168984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f85a20 0x5651b7f87e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.453+0000 7fe3e5e00640 1 -- 172.21.15.83:0/2213168984 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651b7edaf50 con 0x5651b7f7aa30 2024-04-23T14:51:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.453+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/2213168984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7aa30 0x5651b7f7ae10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00bcdb0 tx=0x7fe3e002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cef289d5eeb076b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.454+0000 7fe3d3400640 1 -- 172.21.15.83:0/2213168984 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00c5a90 con 0x5651b7f7aa30 2024-04-23T14:51:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.454+0000 7fe3d3400640 1 -- 172.21.15.83:0/2213168984 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3e00d6210 con 0x5651b7f7aa30 2024-04-23T14:51:20.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.454+0000 7fe3d3400640 1 -- 172.21.15.83:0/2213168984 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0004600 con 0x5651b7f7aa30 2024-04-23T14:51:20.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.455+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2213168984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7aa30 msgr2=0x5651b7f7ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:20.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.455+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/2213168984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7aa30 0x5651b7f7ae10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00bcdb0 tx=0x7fe3e002a6d0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.455+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2213168984 shutdown_connections 2024-04-23T14:51:20.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.455+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/2213168984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f85a20 0x5651b7f87e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.455+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/2213168984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7b350 0x5651b7f854e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.455+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/2213168984 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7aa30 0x5651b7f7ae10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.455+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2213168984 >> 172.21.15.83:0/2213168984 conn(0x5651b7f76d50 msgr2=0x5651b7f77130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.455+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2213168984 shutdown_connections 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.455+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2213168984 wait complete. 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.455+0000 7fe3efbaa740 1 -- start start 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.456+0000 7fe3efbaa740 1 Processor -- start 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.456+0000 7fe3efbaa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7aa30 0x5651b7f82a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.456+0000 7fe3efbaa740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7b350 0x5651b7f82f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.456+0000 7fe3efbaa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f85a20 0x5651b7f834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.456+0000 7fe3efbaa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5651b7f885d0 con 0x5651b7f85a20 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.456+0000 7fe3efbaa740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5651b7e99690 con 0x5651b7f7b350 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.456+0000 7fe3efbaa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5651b7f88750 con 0x5651b7f7aa30 2024-04-23T14:51:20.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.456+0000 7fe3efbaa740 10 client.0 _ll_register_callbacks cb 0x5651b7ed5650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.457+0000 7fe3e5400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7b350 0x5651b7f82f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.457+0000 7fe3e5400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7b350 0x5651b7f82f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37792/0 (socket says 172.21.15.83:37792) 2024-04-23T14:51:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.457+0000 7fe3e5400640 1 -- 172.21.15.83:0/4210267056 learned_addr learned my addr 172.21.15.83:0/4210267056 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.457+0000 7fe3e6800640 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f85a20 0x5651b7f834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.457+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7aa30 0x5651b7f82a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.457+0000 7fe3e5400640 1 -- 172.21.15.83:0/4210267056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7aa30 msgr2=0x5651b7f82a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.457+0000 7fe3e5400640 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7aa30 0x5651b7f82a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.457+0000 7fe3e5400640 1 -- 172.21.15.83:0/4210267056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f85a20 msgr2=0x5651b7f834a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.457+0000 7fe3e5400640 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f85a20 0x5651b7f834a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.457+0000 7fe3e5400640 1 -- 172.21.15.83:0/4210267056 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651b7e98920 con 0x5651b7f7b350 2024-04-23T14:51:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.457+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7aa30 0x5651b7f82a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:51:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.457+0000 7fe3e6800640 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f85a20 0x5651b7f834a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:51:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.458+0000 7fe3e5400640 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7b350 0x5651b7f82f60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e650 tx=0x7fe3d800eb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8337c47f77ae5d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.458+0000 7fe3c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:51:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.458+0000 7fe3c7e00640 1 -- 172.21.15.83:0/4210267056 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800aa90 con 0x5651b7f7b350 2024-04-23T14:51:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.458+0000 7fe3efbaa740 1 -- 172.21.15.83:0/4210267056 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3e00d6210 con 0x5651b7f7b350 2024-04-23T14:51:20.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.458+0000 7fe3c7e00640 1 -- 172.21.15.83:0/4210267056 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe3d800ac30 con 0x5651b7f7b350 2024-04-23T14:51:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.462+0000 7fe3c7e00640 1 -- 172.21.15.83:0/4210267056 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3d800ae30 con 0x5651b7f7b350 2024-04-23T14:51:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.462+0000 7fe3efbaa740 10 client.25913 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.462+0000 7fe3efbaa740 1 -- 172.21.15.83:0/4210267056 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651b7cbc7d0 con 0x5651b7f7b350 2024-04-23T14:51:20.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.462+0000 7fe3c7e00640 1 -- 172.21.15.83:0/4210267056 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3e00d6210 con 0x5651b7f7b350 2024-04-23T14:51:20.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.463+0000 7fe3c7e00640 1 -- 172.21.15.83:0/4210267056 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fe3d80104c0 con 0x5651b7f7b350 2024-04-23T14:51:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.463+0000 7fe3efbaa740 10 client.25913 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.463+0000 7fe3efbaa740 10 client.25913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.463+0000 7fe3efbaa740 20 client.25913 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.463+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5651b7f95b30 0x5651b7f97f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.463+0000 7fe3efbaa740 4 client.25913 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:51:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.463+0000 7fe3efbaa740 1 -- 172.21.15.83:0/4210267056 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651b7eedc10 con 0x5651b7f95b30 2024-04-23T14:51:20.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.464+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5651b7f95b30 0x5651b7f97f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.465+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5651b7f95b30 0x5651b7f97f20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7101766aea06b817 server_cookie=edfbe2cfafe384ba in_seq=0 out_seq=0 2024-04-23T14:51:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.465+0000 7fe3c7e00640 10 client.25913 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:20.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.466+0000 7fe3c7e00640 1 -- 172.21.15.83:0/4210267056 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651b7eedc10 con 0x5651b7f95b30 2024-04-23T14:51:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.466+0000 7fe3c7e00640 10 client.25913 handle_command_reply: tid=0 2024-04-23T14:51:20.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.467+0000 7fe3efbaa740 1 client.25913 shutdown 2024-04-23T14:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.467+0000 7fe3efbaa740 1 -- 172.21.15.83:0/4210267056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7b350 msgr2=0x5651b7f82f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.467+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7b350 0x5651b7f82f60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e650 tx=0x7fe3d800eb10 comp rx=0 tx=0).stop 2024-04-23T14:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.467+0000 7fe3efbaa740 1 -- 172.21.15.83:0/4210267056 shutdown_connections 2024-04-23T14:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.467+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5651b7f95b30 0x5651b7f97f20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.467+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f85a20 0x5651b7f834a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.467+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7b350 0x5651b7f82f60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.467+0000 7fe3c7e00640 0 client.25913 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.467+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/4210267056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7aa30 0x5651b7f82a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.467+0000 7fe3efbaa740 1 -- 172.21.15.83:0/4210267056 >> 172.21.15.83:0/4210267056 conn(0x5651b7f76d50 msgr2=0x5651b7f7cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.467+0000 7fe3efbaa740 1 -- 172.21.15.83:0/4210267056 shutdown_connections 2024-04-23T14:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.467+0000 7fe3efbaa740 1 -- 172.21.15.83:0/4210267056 wait complete. 2024-04-23T14:51:20.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.468+0000 7fe3efbaa740 20 client.25913 trim_cache size 0 max 16384 2024-04-23T14:51:20.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.509+0000 7fe3efbaa740 1 Processor -- start 2024-04-23T14:51:20.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.509+0000 7fe3efbaa740 1 -- start start 2024-04-23T14:51:20.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.509+0000 7fe3efbaa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7aa30 0x5651b7f913e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.509+0000 7fe3efbaa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7b350 0x5651b7f832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.509+0000 7fe3efbaa740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f85a20 0x5651b7f75e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.509+0000 7fe3efbaa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e81bf290 con 0x5651b7f7b350 2024-04-23T14:51:20.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.509+0000 7fe3efbaa740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e81bcb30 con 0x5651b7f85a20 2024-04-23T14:51:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.509+0000 7fe3efbaa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e81ba980 con 0x5651b7f7aa30 2024-04-23T14:51:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.510+0000 7fe3e5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7b350 0x5651b7f832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.510+0000 7fe3e6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f85a20 0x5651b7f75e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.510+0000 7fe3e6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f85a20 0x5651b7f75e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37802/0 (socket says 172.21.15.83:37802) 2024-04-23T14:51:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.510+0000 7fe3e6800640 1 -- 172.21.15.83:0/614886525 learned_addr learned my addr 172.21.15.83:0/614886525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:20.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.510+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/614886525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7aa30 0x5651b7f913e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.511+0000 7fe3e6800640 1 -- 172.21.15.83:0/614886525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7aa30 msgr2=0x5651b7f913e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.511+0000 7fe3e6800640 1 --2- 172.21.15.83:0/614886525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7aa30 0x5651b7f913e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.511+0000 7fe3e6800640 1 -- 172.21.15.83:0/614886525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7b350 msgr2=0x5651b7f832c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.511+0000 7fe3e6800640 1 --2- 172.21.15.83:0/614886525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7b350 0x5651b7f832c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.511+0000 7fe3e6800640 1 -- 172.21.15.83:0/614886525 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651b7f7b180 con 0x5651b7f85a20 2024-04-23T14:51:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.511+0000 7fe3e5400640 1 --2- 172.21.15.83:0/614886525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7b350 0x5651b7f832c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:20.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.512+0000 7fe3e6800640 1 --2- 172.21.15.83:0/614886525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f85a20 0x5651b7f75e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400dea0 tx=0x7fe3d4010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dd1a45705b3bf0a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.512+0000 7fe3d2a00640 1 -- 172.21.15.83:0/614886525 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4025070 con 0x5651b7f85a20 2024-04-23T14:51:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.512+0000 7fe3d2a00640 1 -- 172.21.15.83:0/614886525 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d401a040 con 0x5651b7f85a20 2024-04-23T14:51:20.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.512+0000 7fe3d2a00640 1 -- 172.21.15.83:0/614886525 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4019070 con 0x5651b7f85a20 2024-04-23T14:51:20.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.513+0000 7fe3efbaa740 1 -- 172.21.15.83:0/614886525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f85a20 msgr2=0x5651b7f75e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:20.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.513+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/614886525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f85a20 0x5651b7f75e50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400dea0 tx=0x7fe3d4010000 comp rx=0 tx=0).stop 2024-04-23T14:51:20.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.513+0000 7fe3efbaa740 1 -- 172.21.15.83:0/614886525 shutdown_connections 2024-04-23T14:51:20.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.513+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/614886525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f85a20 0x5651b7f75e50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.513+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/614886525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7b350 0x5651b7f832c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.513+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/614886525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f7aa30 0x5651b7f913e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.513+0000 7fe3efbaa740 1 -- 172.21.15.83:0/614886525 >> 172.21.15.83:0/614886525 conn(0x5651b7f76d50 msgr2=0x5651b7f75740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:20.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.513+0000 7fe3efbaa740 1 -- 172.21.15.83:0/614886525 shutdown_connections 2024-04-23T14:51:20.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.514+0000 7fe3efbaa740 1 -- 172.21.15.83:0/614886525 wait complete. 2024-04-23T14:51:20.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.514+0000 7fe3efbaa740 1 -- start start 2024-04-23T14:51:20.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.514+0000 7fe3efbaa740 1 Processor -- start 2024-04-23T14:51:20.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.515+0000 7fe3efbaa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7aa30 0x5651b7f8db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.515+0000 7fe3efbaa740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7b350 0x5651b7f7f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.515+0000 7fe3efbaa740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f85a20 0x5651b7f7f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.515+0000 7fe3efbaa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e81baff0 con 0x5651b7f7aa30 2024-04-23T14:51:20.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.515+0000 7fe3efbaa740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e81bc460 con 0x5651b7f7b350 2024-04-23T14:51:20.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.515+0000 7fe3efbaa740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e81bb6c0 con 0x5651b7f85a20 2024-04-23T14:51:20.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.515+0000 7fe3efbaa740 10 client.0 _ll_register_callbacks cb 0x7fe3d8005ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:20.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.515+0000 7fe3e5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7aa30 0x5651b7f8db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.516+0000 7fe3e6800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f85a20 0x5651b7f7f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.516+0000 7fe3e5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7aa30 0x5651b7f8db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56608/0 (socket says 172.21.15.83:56608) 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.516+0000 7fe3e5e00640 1 -- 172.21.15.83:0/2254031704 learned_addr learned my addr 172.21.15.83:0/2254031704 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.516+0000 7fe3e5400640 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7b350 0x5651b7f7f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.516+0000 7fe3e5e00640 1 -- 172.21.15.83:0/2254031704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f85a20 msgr2=0x5651b7f7f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.516+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f85a20 0x5651b7f7f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.516+0000 7fe3e5e00640 1 -- 172.21.15.83:0/2254031704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7b350 msgr2=0x5651b7f7f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.516+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7b350 0x5651b7f7f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.516+0000 7fe3e5e00640 1 -- 172.21.15.83:0/2254031704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651b7bf23f0 con 0x5651b7f7aa30 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.516+0000 7fe3e6800640 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f85a20 0x5651b7f7f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.516+0000 7fe3e5e00640 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7aa30 0x5651b7f8db50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00b97f0 tx=0x7fe3e0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d4473483cfe08d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.516+0000 7fe3c7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.517+0000 7fe3c7e00640 1 -- 172.21.15.83:0/2254031704 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e00c5a90 con 0x5651b7f7aa30 2024-04-23T14:51:20.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.517+0000 7fe3c7e00640 1 -- 172.21.15.83:0/2254031704 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe3e00b9490 con 0x5651b7f7aa30 2024-04-23T14:51:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.517+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2254031704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d401a040 con 0x5651b7f7aa30 2024-04-23T14:51:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.517+0000 7fe3c7e00640 1 -- 172.21.15.83:0/2254031704 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3e0004600 con 0x5651b7f7aa30 2024-04-23T14:51:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.517+0000 7fe3efbaa740 10 client.15786 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.517+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2254031704 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651b7bb20f0 con 0x5651b7f7aa30 2024-04-23T14:51:20.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.517+0000 7fe3c7e00640 1 -- 172.21.15.83:0/2254031704 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d401a040 con 0x5651b7f7aa30 2024-04-23T14:51:20.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.518+0000 7fe3c7e00640 1 -- 172.21.15.83:0/2254031704 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fe3e00c5ce0 con 0x5651b7f7aa30 2024-04-23T14:51:20.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.518+0000 7fe3efbaa740 10 client.15786 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.518+0000 7fe3efbaa740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.518+0000 7fe3efbaa740 20 client.15786 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.518+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5651b7ffc180 0x5651b7ffe5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.518+0000 7fe3efbaa740 4 client.15786 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.518+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2254031704 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651b7ee7a80 con 0x5651b7ffc180 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.518+0000 7fe3e5400640 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5651b7ffc180 0x5651b7ffe5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3e5400640 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5651b7ffc180 0x5651b7ffe5a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb64e3376ba57dc server_cookie=f0ffcc230f6ab475 in_seq=0 out_seq=0 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3c7e00640 10 client.15786 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3c7e00640 1 -- 172.21.15.83:0/2254031704 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5651b7ee7a80 con 0x5651b7ffc180 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3c7e00640 10 client.15786 handle_command_reply: tid=0 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3efbaa740 1 client.15786 shutdown 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2254031704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7aa30 msgr2=0x5651b7f8db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7aa30 0x5651b7f8db50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00b97f0 tx=0x7fe3e0004040 comp rx=0 tx=0).stop 2024-04-23T14:51:20.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2254031704 shutdown_connections 2024-04-23T14:51:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5651b7ffc180 0x5651b7ffe5a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651b7f85a20 0x5651b7f7f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651b7f7b350 0x5651b7f7f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3efbaa740 1 --2- 172.21.15.83:0/2254031704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651b7f7aa30 0x5651b7f8db50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.519+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2254031704 >> 172.21.15.83:0/2254031704 conn(0x5651b7f76d50 msgr2=0x5651b7f75fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.520+0000 7fe3c7e00640 0 client.15786 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.520+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2254031704 shutdown_connections 2024-04-23T14:51:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.520+0000 7fe3efbaa740 1 -- 172.21.15.83:0/2254031704 wait complete. 2024-04-23T14:51:20.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.520+0000 7fe3efbaa740 20 client.15786 trim_cache size 0 max 16384 2024-04-23T14:51:20.523 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:51:20.523 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:51:20.523 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:51:20.524 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:51:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.521+0000 7fe3e7200640 1 -- 172.21.15.83:0/3300554601 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe3cc0a3ef0 msgr2=0x7fe3cc0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:20.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.521+0000 7fe3e7200640 1 --2- 172.21.15.83:0/3300554601 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe3cc0a3ef0 0x7fe3cc0a63b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005ab0 tx=0x7fe3d4005a00 comp rx=0 tx=0).stop 2024-04-23T14:51:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.521+0000 7fe3e7200640 1 -- 172.21.15.83:0/3300554601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8102110 msgr2=0x7fe3e8116610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.521+0000 7fe3e7200640 1 --2- 172.21.15.83:0/3300554601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8102110 0x7fe3e8116610 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0031040 tx=0x7fe3e0030640 comp rx=0 tx=0).stop 2024-04-23T14:51:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.521+0000 7fe3e7200640 1 -- 172.21.15.83:0/3300554601 shutdown_connections 2024-04-23T14:51:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.521+0000 7fe3e7200640 1 --2- 172.21.15.83:0/3300554601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3e8111720 0x7fe3e8111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.521+0000 7fe3e7200640 1 --2- 172.21.15.83:0/3300554601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3e81071e0 0x7fe3e8116b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.521+0000 7fe3e7200640 1 --2- 172.21.15.83:0/3300554601 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe3cc0a3ef0 0x7fe3cc0a63b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.521+0000 7fe3e7200640 1 --2- 172.21.15.83:0/3300554601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e8102110 0x7fe3e8116610 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.521+0000 7fe3e7200640 1 -- 172.21.15.83:0/3300554601 >> 172.21.15.83:0/3300554601 conn(0x7fe3e80fd9d0 msgr2=0x7fe3e80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.523+0000 7fe3e7200640 1 -- 172.21.15.83:0/3300554601 shutdown_connections 2024-04-23T14:51:20.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.523+0000 7fe3e7200640 1 -- 172.21.15.83:0/3300554601 wait complete. 2024-04-23T14:51:20.566 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:51:20.566 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:51:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.853+0000 7f62e2200640 1 -- 172.21.15.83:0/2526044539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62dc076e50 msgr2=0x7f62dc112940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.854+0000 7f62e2200640 1 --2- 172.21.15.83:0/2526044539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62dc076e50 0x7f62dc112940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009900 tx=0x7f62c402a410 comp rx=0 tx=0).stop 2024-04-23T14:51:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.854+0000 7f62e2200640 1 -- 172.21.15.83:0/2526044539 shutdown_connections 2024-04-23T14:51:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.854+0000 7f62e2200640 1 --2- 172.21.15.83:0/2526044539 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62dc112e80 0x7f62dc115270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.854+0000 7f62e2200640 1 --2- 172.21.15.83:0/2526044539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62dc076e50 0x7f62dc112940 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.854+0000 7f62e2200640 1 --2- 172.21.15.83:0/2526044539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62dc076530 0x7f62dc076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.854+0000 7f62e2200640 1 -- 172.21.15.83:0/2526044539 >> 172.21.15.83:0/2526044539 conn(0x7f62dc0ffcb0 msgr2=0x7f62dc1020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.854+0000 7f62e2200640 1 -- 172.21.15.83:0/2526044539 shutdown_connections 2024-04-23T14:51:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.854+0000 7f62e2200640 1 -- 172.21.15.83:0/2526044539 wait complete. 2024-04-23T14:51:20.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.855+0000 7f62e2200640 1 Processor -- start 2024-04-23T14:51:20.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.857+0000 7f62e2200640 1 -- start start 2024-04-23T14:51:20.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.859+0000 7f62e2200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62dc076530 0x7f62dc19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.860+0000 7f62e2200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62dc076e50 0x7f62dc19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.860+0000 7f62e2200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62dc112e80 0x7f62dc1a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.860+0000 7f62e2200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc1189e0 con 0x7f62dc076e50 2024-04-23T14:51:20.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.860+0000 7f62e2200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc118860 con 0x7f62dc112e80 2024-04-23T14:51:20.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.860+0000 7f62e2200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc118b60 con 0x7f62dc076530 2024-04-23T14:51:20.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.860+0000 7f62db400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62dc076530 0x7f62dc19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.860+0000 7f62db400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62dc076530 0x7f62dc19eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:47432/0 (socket says 172.21.15.83:47432) 2024-04-23T14:51:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.860+0000 7f62db400640 1 -- 172.21.15.83:0/459660201 learned_addr learned my addr 172.21.15.83:0/459660201 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.861+0000 7f62db400640 1 -- 172.21.15.83:0/459660201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62dc112e80 msgr2=0x7f62dc1a3780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T14:51:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.861+0000 7f62db400640 1 --2- 172.21.15.83:0/459660201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62dc112e80 0x7f62dc1a3780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.861+0000 7f62db400640 1 -- 172.21.15.83:0/459660201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62dc076e50 msgr2=0x7f62dc19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.861+0000 7f62db400640 1 --2- 172.21.15.83:0/459660201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62dc076e50 0x7f62dc19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:20.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.861+0000 7f62db400640 1 -- 172.21.15.83:0/459660201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62dc1a3eb0 con 0x7f62dc076530 2024-04-23T14:51:20.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.861+0000 7f62db400640 1 --2- 172.21.15.83:0/459660201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62dc076530 0x7f62dc19eec0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00d950 tx=0x7f62cc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:20.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.861+0000 7f62d8c00640 1 -- 172.21.15.83:0/459660201 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc017070 con 0x7f62dc076530 2024-04-23T14:51:20.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.862+0000 7f62d8c00640 1 -- 172.21.15.83:0/459660201 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62cc009c30 con 0x7f62dc076530 2024-04-23T14:51:20.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.861+0000 7f62e2200640 1 -- 172.21.15.83:0/459660201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62dc1a41a0 con 0x7f62dc076530 2024-04-23T14:51:20.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.862+0000 7f62e2200640 1 -- 172.21.15.83:0/459660201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62dc1aba30 con 0x7f62dc076530 2024-04-23T14:51:20.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.863+0000 7f62d8c00640 1 -- 172.21.15.83:0/459660201 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00f690 con 0x7f62dc076530 2024-04-23T14:51:20.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.865+0000 7f62e2200640 1 -- 172.21.15.83:0/459660201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62dc069270 con 0x7f62dc076530 2024-04-23T14:51:20.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.866+0000 7f62d8c00640 1 -- 172.21.15.83:0/459660201 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62cc00f830 con 0x7f62dc076530 2024-04-23T14:51:20.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.878+0000 7f62d8c00640 1 --2- 172.21.15.83:0/459660201 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f62c00a3da0 0x7f62c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:20.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.879+0000 7f62d8c00640 1 -- 172.21.15.83:0/459660201 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f62cc0d1540 con 0x7f62dc076530 2024-04-23T14:51:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.879+0000 7f62d8c00640 1 -- 172.21.15.83:0/459660201 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62cc009de0 con 0x7f62dc076530 2024-04-23T14:51:20.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.881+0000 7f62d2a00640 1 --2- 172.21.15.83:0/459660201 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f62c00a3da0 0x7f62c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:20.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:20.884+0000 7f62d2a00640 1 --2- 172.21.15.83:0/459660201 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f62c00a3da0 0x7f62c00a6260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f62c40096c0 tx=0x7f62c4000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:20.953 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:20 smithi083 ceph-mon[27867]: pgmap v89: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 239 MiB/s wr, 93 op/s 2024-04-23T14:51:21.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:20 smithi203 ceph-mon[32800]: pgmap v89: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 239 MiB/s wr, 93 op/s 2024-04-23T14:51:21.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.282+0000 7f62e2200640 1 -- 172.21.15.83:0/459660201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62dc10da80 con 0x7f62dc076530 2024-04-23T14:51:21.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.283+0000 7f62d8c00640 1 -- 172.21.15.83:0/459660201 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11867 (secure 0 0 0) 0x7f62cc09db90 con 0x7f62dc076530 2024-04-23T14:51:21.284 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:51:21.284 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:50:17.708313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:51:21.285 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-04-23T14:51:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.286+0000 7f62e2200640 1 -- 172.21.15.83:0/459660201 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f62c00a3da0 msgr2=0x7f62c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.286+0000 7f62e2200640 1 --2- 172.21.15.83:0/459660201 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f62c00a3da0 0x7f62c00a6260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f62c40096c0 tx=0x7f62c4000bc0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.286+0000 7f62e2200640 1 -- 172.21.15.83:0/459660201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62dc076530 msgr2=0x7f62dc19eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.286+0000 7f62e2200640 1 --2- 172.21.15.83:0/459660201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62dc076530 0x7f62dc19eec0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00d950 tx=0x7f62cc00de10 comp rx=0 tx=0).stop 2024-04-23T14:51:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.286+0000 7f62e2200640 1 -- 172.21.15.83:0/459660201 shutdown_connections 2024-04-23T14:51:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.286+0000 7f62e2200640 1 --2- 172.21.15.83:0/459660201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f62dc112e80 0x7f62dc1a3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.286+0000 7f62e2200640 1 --2- 172.21.15.83:0/459660201 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f62c00a3da0 0x7f62c00a6260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.286+0000 7f62e2200640 1 --2- 172.21.15.83:0/459660201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62dc076e50 0x7f62dc19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.286+0000 7f62e2200640 1 --2- 172.21.15.83:0/459660201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62dc076530 0x7f62dc19eec0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.286+0000 7f62e2200640 1 -- 172.21.15.83:0/459660201 >> 172.21.15.83:0/459660201 conn(0x7f62dc0ffcb0 msgr2=0x7f62dc1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.287+0000 7f62e2200640 1 -- 172.21.15.83:0/459660201 shutdown_connections 2024-04-23T14:51:21.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.287+0000 7f62e2200640 1 -- 172.21.15.83:0/459660201 wait complete. 2024-04-23T14:51:21.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:51:21.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:20 smithi173 ceph-mon[32443]: pgmap v89: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 239 MiB/s wr, 93 op/s 2024-04-23T14:51:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.608+0000 7ff01b400640 1 -- 172.21.15.83:0/1370475763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff01410a9a0 msgr2=0x7ff01410ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.608+0000 7ff01b400640 1 --2- 172.21.15.83:0/1370475763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff01410a9a0 0x7ff01410ad80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff008009900 tx=0x7ff00802a3f0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.609+0000 7ff01b400640 1 -- 172.21.15.83:0/1370475763 shutdown_connections 2024-04-23T14:51:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.609+0000 7ff01b400640 1 --2- 172.21.15.83:0/1370475763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff014113f50 0x7ff014116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.609+0000 7ff01b400640 1 --2- 172.21.15.83:0/1370475763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff01410b2c0 0x7ff014113a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.609+0000 7ff01b400640 1 --2- 172.21.15.83:0/1370475763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff01410a9a0 0x7ff01410ad80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.609+0000 7ff01b400640 1 -- 172.21.15.83:0/1370475763 >> 172.21.15.83:0/1370475763 conn(0x7ff01406d110 msgr2=0x7ff01406d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:21.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.609+0000 7ff01b400640 1 -- 172.21.15.83:0/1370475763 shutdown_connections 2024-04-23T14:51:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.611+0000 7ff01b400640 1 -- 172.21.15.83:0/1370475763 wait complete. 2024-04-23T14:51:21.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.613+0000 7ff01b400640 1 Processor -- start 2024-04-23T14:51:21.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.614+0000 7ff01b400640 1 -- start start 2024-04-23T14:51:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.616+0000 7ff01b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff01410a9a0 0x7ff014113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.616+0000 7ff01b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff01410b2c0 0x7ff0141109e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.617+0000 7ff01b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff014113f50 0x7ff014110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.617+0000 7ff01b400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff01411cdf0 con 0x7ff01410b2c0 2024-04-23T14:51:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.617+0000 7ff01b400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff01411cc70 con 0x7ff01410a9a0 2024-04-23T14:51:21.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.617+0000 7ff01b400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff01411cf70 con 0x7ff014113f50 2024-04-23T14:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.617+0000 7ff018a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff01410a9a0 0x7ff014113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.617+0000 7ff018a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff01410a9a0 0x7ff014113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37850/0 (socket says 172.21.15.83:37850) 2024-04-23T14:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.617+0000 7ff018a00640 1 -- 172.21.15.83:0/1860091044 learned_addr learned my addr 172.21.15.83:0/1860091044 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.617+0000 7ff019400640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff014113f50 0x7ff014110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.617+0000 7ff018a00640 1 -- 172.21.15.83:0/1860091044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff014113f50 msgr2=0x7ff014110fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.617+0000 7ff018a00640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff014113f50 0x7ff014110fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.617+0000 7ff018a00640 1 -- 172.21.15.83:0/1860091044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff01410b2c0 msgr2=0x7ff0141109e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.618+0000 7ff013e00640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff01410b2c0 0x7ff0141109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.618+0000 7ff018a00640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff01410b2c0 0x7ff0141109e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.618+0000 7ff018a00640 1 -- 172.21.15.83:0/1860091044 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0141a92b0 con 0x7ff01410a9a0 2024-04-23T14:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.618+0000 7ff013e00640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff01410b2c0 0x7ff0141109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.618+0000 7ff018a00640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff01410a9a0 0x7ff014113940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff00802a9e0 tx=0x7ff008003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.618+0000 7ff011600640 1 -- 172.21.15.83:0/1860091044 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00803f070 con 0x7ff01410a9a0 2024-04-23T14:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.618+0000 7ff011600640 1 -- 172.21.15.83:0/1860091044 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff008031c30 con 0x7ff01410a9a0 2024-04-23T14:51:21.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.618+0000 7ff011600640 1 -- 172.21.15.83:0/1860091044 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008035030 con 0x7ff01410a9a0 2024-04-23T14:51:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.619+0000 7ff01b400640 1 -- 172.21.15.83:0/1860091044 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0141a9540 con 0x7ff01410a9a0 2024-04-23T14:51:21.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.619+0000 7ff01b400640 1 -- 172.21.15.83:0/1860091044 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0141a99e0 con 0x7ff01410a9a0 2024-04-23T14:51:21.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.628+0000 7ff01e1f4740 1 Processor -- start 2024-04-23T14:51:21.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.631+0000 7ff01e1f4740 1 -- start start 2024-04-23T14:51:21.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.631+0000 7ff01e1f4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 0x5608e6390d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.631+0000 7ff01e1f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6391270 0x5608e639b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.632+0000 7ff01e1f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e639b940 0x5608e639dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.632+0000 7ff01e1f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5608e62f61d0 con 0x5608e639b940 2024-04-23T14:51:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.632+0000 7ff01e1f4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5608e629c450 con 0x5608e6390950 2024-04-23T14:51:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.632+0000 7ff01e1f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5608e62f2190 con 0x5608e6391270 2024-04-23T14:51:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.633+0000 7ff019400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 0x5608e6390d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.633+0000 7ff019400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 0x5608e6390d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37864/0 (socket says 172.21.15.83:37864) 2024-04-23T14:51:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.633+0000 7ff019400640 1 -- 172.21.15.83:0/1346236602 learned_addr learned my addr 172.21.15.83:0/1346236602 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:21.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.633+0000 7ff018a00640 1 --2- 172.21.15.83:0/1346236602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6391270 0x5608e639b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.633+0000 7ff011600640 1 -- 172.21.15.83:0/1860091044 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff008042070 con 0x7ff01410a9a0 2024-04-23T14:51:21.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.633+0000 7ff018a00640 1 -- 172.21.15.83:0/1346236602 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 msgr2=0x5608e6390d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:21.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.633+0000 7ff018a00640 1 --2- 172.21.15.83:0/1346236602 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 0x5608e6390d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.633+0000 7ff018a00640 1 -- 172.21.15.83:0/1346236602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e639b940 msgr2=0x5608e639dd30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:21.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.637+0000 7ff013e00640 1 --2- 172.21.15.83:0/1346236602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e639b940 0x5608e639dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.637+0000 7ff018a00640 1 --2- 172.21.15.83:0/1346236602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e639b940 0x5608e639dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.637+0000 7ff018a00640 1 -- 172.21.15.83:0/1346236602 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e6359310 con 0x5608e6391270 2024-04-23T14:51:21.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.637+0000 7ff013e00640 1 --2- 172.21.15.83:0/1346236602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e639b940 0x5608e639dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:21.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.638+0000 7ff018a00640 1 --2- 172.21.15.83:0/1346236602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6391270 0x5608e639b400 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ee590 tx=0x7ff0080f90e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d19d70f4a8532bad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.638+0000 7ff005c00640 1 -- 172.21.15.83:0/1346236602 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0080f9cf0 con 0x5608e6391270 2024-04-23T14:51:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.638+0000 7ff005c00640 1 -- 172.21.15.83:0/1346236602 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0080fe070 con 0x5608e6391270 2024-04-23T14:51:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.638+0000 7ff005c00640 1 -- 172.21.15.83:0/1346236602 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0081015d0 con 0x5608e6391270 2024-04-23T14:51:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.638+0000 7ff011600640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feffc0a3e20 0x7feffc0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.639+0000 7ff011600640 1 -- 172.21.15.83:0/1860091044 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ff0080f0e90 con 0x7ff01410a9a0 2024-04-23T14:51:21.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.639+0000 7ff019400640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feffc0a3e20 0x7feffc0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.641+0000 7ff019400640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feffc0a3e20 0x7feffc0a62e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff00c005f40 tx=0x7ff00c0096a0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:21.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.641+0000 7ff01e1f4740 1 -- 172.21.15.83:0/1346236602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6391270 msgr2=0x5608e639b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.641+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/1346236602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6391270 0x5608e639b400 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ee590 tx=0x7ff0080f90e0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.641+0000 7ff01e1f4740 1 -- 172.21.15.83:0/1346236602 shutdown_connections 2024-04-23T14:51:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.641+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/1346236602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e639b940 0x5608e639dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.641+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/1346236602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6391270 0x5608e639b400 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.641+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/1346236602 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 0x5608e6390d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.641+0000 7ff01e1f4740 1 -- 172.21.15.83:0/1346236602 >> 172.21.15.83:0/1346236602 conn(0x5608e638cc70 msgr2=0x5608e638d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.641+0000 7ff01e1f4740 1 -- 172.21.15.83:0/1346236602 shutdown_connections 2024-04-23T14:51:21.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.641+0000 7ff01e1f4740 1 -- 172.21.15.83:0/1346236602 wait complete. 2024-04-23T14:51:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.642+0000 7ff01e1f4740 1 -- start start 2024-04-23T14:51:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.642+0000 7ff01e1f4740 1 Processor -- start 2024-04-23T14:51:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.643+0000 7ff01e1f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6390950 0x5608e639b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.643+0000 7ff01e1f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 0x5608e63986d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.643+0000 7ff01e1f4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e639b940 0x5608e6398dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.643+0000 7ff01e1f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5608e639e4f0 con 0x5608e6391270 2024-04-23T14:51:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.643+0000 7ff01e1f4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5608e62af810 con 0x5608e639b940 2024-04-23T14:51:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.643+0000 7ff01e1f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5608e639e670 con 0x5608e6390950 2024-04-23T14:51:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.643+0000 7ff01e1f4740 10 client.0 _ll_register_callbacks cb 0x5608e627d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.643+0000 7ff018a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6390950 0x5608e639b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.643+0000 7ff019400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e639b940 0x5608e6398dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.643+0000 7ff018a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6390950 0x5608e639b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:47458/0 (socket says 172.21.15.83:47458) 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.643+0000 7ff018a00640 1 -- 172.21.15.83:0/4018639371 learned_addr learned my addr 172.21.15.83:0/4018639371 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.644+0000 7ff013e00640 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 0x5608e63986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.644+0000 7ff019400640 1 -- 172.21.15.83:0/4018639371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6390950 msgr2=0x5608e639b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.644+0000 7ff019400640 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6390950 0x5608e639b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.644+0000 7ff019400640 1 -- 172.21.15.83:0/4018639371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 msgr2=0x5608e63986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.644+0000 7ff019400640 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 0x5608e63986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.644+0000 7ff019400640 1 -- 172.21.15.83:0/4018639371 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e62c03f0 con 0x5608e639b940 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.644+0000 7ff018a00640 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6390950 0x5608e639b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.644+0000 7ff013e00640 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 0x5608e63986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.644+0000 7ff019400640 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e639b940 0x5608e6398dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff00c011e90 tx=0x7ff00c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16e08dc305741c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.644+0000 7feffa000640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.645+0000 7feffa000640 1 -- 172.21.15.83:0/4018639371 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c003c40 con 0x5608e639b940 2024-04-23T14:51:21.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.645+0000 7ff01e1f4740 1 -- 172.21.15.83:0/4018639371 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0080fe070 con 0x5608e639b940 2024-04-23T14:51:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.645+0000 7feffa000640 1 -- 172.21.15.83:0/4018639371 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ff00c003de0 con 0x5608e639b940 2024-04-23T14:51:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.645+0000 7feffa000640 1 -- 172.21.15.83:0/4018639371 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff00c004b20 con 0x5608e639b940 2024-04-23T14:51:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.645+0000 7ff01e1f4740 10 client.25934 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:21.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.645+0000 7ff01e1f4740 1 -- 172.21.15.83:0/4018639371 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608e60083f0 con 0x5608e639b940 2024-04-23T14:51:21.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.645+0000 7feffa000640 1 -- 172.21.15.83:0/4018639371 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0080fe070 con 0x5608e639b940 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.645+0000 7feffa000640 1 -- 172.21.15.83:0/4018639371 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7ff00c02f050 con 0x5608e639b940 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.646+0000 7ff01e1f4740 10 client.25934 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.646+0000 7ff01e1f4740 10 client.25934 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.646+0000 7ff01e1f4740 20 client.25934 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.646+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x5608e63ab8f0 0x5608e63add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.646+0000 7ff01e1f4740 4 client.25934 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.646+0000 7ff01e1f4740 1 -- 172.21.15.83:0/4018639371 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608e6303bf0 con 0x5608e63ab8f0 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.646+0000 7ff018a00640 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x5608e63ab8f0 0x5608e63add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.647+0000 7ff018a00640 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x5608e63ab8f0 0x5608e63add30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72f3644846add49d server_cookie=4a5791e44cf92ac2 in_seq=0 out_seq=0 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.647+0000 7feffa000640 10 client.25934 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.648+0000 7feffa000640 1 -- 172.21.15.83:0/4018639371 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608e6303bf0 con 0x5608e63ab8f0 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.648+0000 7feffa000640 10 client.25934 handle_command_reply: tid=0 2024-04-23T14:51:21.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.648+0000 7ff01e1f4740 1 client.25934 shutdown 2024-04-23T14:51:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.649+0000 7ff01e1f4740 1 -- 172.21.15.83:0/4018639371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e639b940 msgr2=0x5608e6398dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.649+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e639b940 0x5608e6398dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff00c011e90 tx=0x7ff00c004ab0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.649+0000 7ff01e1f4740 1 -- 172.21.15.83:0/4018639371 shutdown_connections 2024-04-23T14:51:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.649+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x5608e63ab8f0 0x5608e63add30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.649+0000 7feffa000640 0 client.25934 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:51:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.649+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e639b940 0x5608e6398dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.649+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 0x5608e63986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.649+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/4018639371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6390950 0x5608e639b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.649+0000 7ff01e1f4740 1 -- 172.21.15.83:0/4018639371 >> 172.21.15.83:0/4018639371 conn(0x5608e638cc70 msgr2=0x5608e6392a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.649+0000 7ff01e1f4740 1 -- 172.21.15.83:0/4018639371 shutdown_connections 2024-04-23T14:51:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.649+0000 7ff01e1f4740 1 -- 172.21.15.83:0/4018639371 wait complete. 2024-04-23T14:51:21.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.650+0000 7ff01e1f4740 20 client.25934 trim_cache size 0 max 16384 2024-04-23T14:51:21.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.677+0000 7ff01e1f4740 1 Processor -- start 2024-04-23T14:51:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.678+0000 7ff01e1f4740 1 -- start start 2024-04-23T14:51:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.678+0000 7ff01e1f4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 0x5608e63a7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.678+0000 7ff01e1f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6391270 0x5608e639af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.678+0000 7ff01e1f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e639b940 0x5608e63b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.678+0000 7ff01e1f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141c7d50 con 0x5608e639b940 2024-04-23T14:51:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.678+0000 7ff01e1f4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141c55f0 con 0x5608e6390950 2024-04-23T14:51:21.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.678+0000 7ff01e1f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141c3440 con 0x5608e6391270 2024-04-23T14:51:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.679+0000 7ff013e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6391270 0x5608e639af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.679+0000 7ff018a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 0x5608e63a7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.680+0000 7ff018a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 0x5608e63a7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37888/0 (socket says 172.21.15.83:37888) 2024-04-23T14:51:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.680+0000 7ff018a00640 1 -- 172.21.15.83:0/1428630878 learned_addr learned my addr 172.21.15.83:0/1428630878 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:21.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.680+0000 7ff013e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6391270 0x5608e639af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:47466/0 (socket says 172.21.15.83:47466) 2024-04-23T14:51:21.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.680+0000 7ff019400640 1 --2- 172.21.15.83:0/1428630878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e639b940 0x5608e63b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.681+0000 7ff018a00640 1 -- 172.21.15.83:0/1428630878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6391270 msgr2=0x5608e639af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:21.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.681+0000 7ff018a00640 1 --2- 172.21.15.83:0/1428630878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6391270 0x5608e639af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.681+0000 7ff018a00640 1 -- 172.21.15.83:0/1428630878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e639b940 msgr2=0x5608e63b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:21.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.681+0000 7ff018a00640 1 --2- 172.21.15.83:0/1428630878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e639b940 0x5608e63b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.681+0000 7ff018a00640 1 -- 172.21.15.83:0/1428630878 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608e60d27d0 con 0x5608e6390950 2024-04-23T14:51:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.682+0000 7ff018a00640 1 --2- 172.21.15.83:0/1428630878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 0x5608e63a7410 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff0080b97b0 tx=0x7ff0080f4900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc5af28ebc6cbcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.683+0000 7ff005200640 1 -- 172.21.15.83:0/1428630878 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0080c0070 con 0x5608e6390950 2024-04-23T14:51:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.683+0000 7ff005200640 1 -- 172.21.15.83:0/1428630878 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0080fe040 con 0x5608e6390950 2024-04-23T14:51:21.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.683+0000 7ff005200640 1 -- 172.21.15.83:0/1428630878 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0080bd4e0 con 0x5608e6390950 2024-04-23T14:51:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.684+0000 7ff01e1f4740 1 -- 172.21.15.83:0/1428630878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 msgr2=0x5608e63a7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:21.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.684+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/1428630878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 0x5608e63a7410 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff0080b97b0 tx=0x7ff0080f4900 comp rx=0 tx=0).stop 2024-04-23T14:51:21.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.685+0000 7ff01e1f4740 1 -- 172.21.15.83:0/1428630878 shutdown_connections 2024-04-23T14:51:21.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.686+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/1428630878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e639b940 0x5608e63b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.686+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/1428630878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6391270 0x5608e639af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.686+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/1428630878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e6390950 0x5608e63a7410 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.686+0000 7ff01e1f4740 1 -- 172.21.15.83:0/1428630878 >> 172.21.15.83:0/1428630878 conn(0x5608e638cc70 msgr2=0x5608e6396fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:21.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.690+0000 7ff01e1f4740 1 -- 172.21.15.83:0/1428630878 shutdown_connections 2024-04-23T14:51:21.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.690+0000 7ff01e1f4740 1 -- 172.21.15.83:0/1428630878 wait complete. 2024-04-23T14:51:21.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.692+0000 7ff01e1f4740 1 -- start start 2024-04-23T14:51:21.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.692+0000 7ff01e1f4740 1 Processor -- start 2024-04-23T14:51:21.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.694+0000 7ff01e1f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6390950 0x5608e639d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.694+0000 7ff01e1f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 0x5608e639bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.694+0000 7ff01e1f4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e639b940 0x5608e639c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.695+0000 7ff01e1f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141c3ab0 con 0x5608e6391270 2024-04-23T14:51:21.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.695+0000 7ff01e1f4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141c4f20 con 0x5608e639b940 2024-04-23T14:51:21.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.695+0000 7ff01e1f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141c4180 con 0x5608e6390950 2024-04-23T14:51:21.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.697+0000 7ff01e1f4740 10 client.0 _ll_register_callbacks cb 0x7ff00c010680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:21.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.697+0000 7ff018a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6390950 0x5608e639d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.698+0000 7ff013e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 0x5608e639bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.698+0000 7ff013e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 0x5608e639bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56680/0 (socket says 172.21.15.83:56680) 2024-04-23T14:51:21.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.698+0000 7ff013e00640 1 -- 172.21.15.83:0/147135514 learned_addr learned my addr 172.21.15.83:0/147135514 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:21.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.698+0000 7ff019400640 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e639b940 0x5608e639c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.699+0000 7ff013e00640 1 -- 172.21.15.83:0/147135514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6390950 msgr2=0x5608e639d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.700+0000 7ff013e00640 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6390950 0x5608e639d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.700+0000 7ff013e00640 1 -- 172.21.15.83:0/147135514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e639b940 msgr2=0x5608e639c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.700+0000 7ff013e00640 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e639b940 0x5608e639c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.700+0000 7ff013e00640 1 -- 172.21.15.83:0/147135514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608e639ca40 con 0x5608e6391270 2024-04-23T14:51:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.700+0000 7ff019400640 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e639b940 0x5608e639c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:51:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.700+0000 7ff013e00640 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 0x5608e639bd20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff014112230 tx=0x7ff000009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2c9e6617f70e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:21.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.700+0000 7feffa000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:51:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.701+0000 7feffa000640 1 -- 172.21.15.83:0/147135514 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00001d030 con 0x5608e6391270 2024-04-23T14:51:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.701+0000 7feffa000640 1 -- 172.21.15.83:0/147135514 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ff000004020 con 0x5608e6391270 2024-04-23T14:51:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.701+0000 7ff01e1f4740 1 -- 172.21.15.83:0/147135514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0080fe040 con 0x5608e6391270 2024-04-23T14:51:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.701+0000 7feffa000640 1 -- 172.21.15.83:0/147135514 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff000004220 con 0x5608e6391270 2024-04-23T14:51:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.701+0000 7ff01e1f4740 10 client.15798 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:21.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.702+0000 7ff01e1f4740 1 -- 172.21.15.83:0/147135514 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608e63a3990 con 0x5608e6391270 2024-04-23T14:51:21.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.702+0000 7feffa000640 1 -- 172.21.15.83:0/147135514 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0080fe040 con 0x5608e6391270 2024-04-23T14:51:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.703+0000 7feffa000640 1 -- 172.21.15.83:0/147135514 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7ff0000047a0 con 0x5608e6391270 2024-04-23T14:51:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.704+0000 7ff01e1f4740 10 client.15798 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.704+0000 7ff01e1f4740 10 client.15798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:51:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.704+0000 7ff01e1f4740 20 client.15798 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.704+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x5608e63ab8f0 0x5608e6413e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.704+0000 7ff01e1f4740 4 client.15798 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:51:21.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.704+0000 7ff01e1f4740 1 -- 172.21.15.83:0/147135514 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608e62fda60 con 0x5608e63ab8f0 2024-04-23T14:51:21.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.705+0000 7ff018a00640 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x5608e63ab8f0 0x5608e6413e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.706+0000 7ff018a00640 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x5608e63ab8f0 0x5608e6413e80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f25f78f36a815ba3 server_cookie=c664e6627b125bf7 in_seq=0 out_seq=0 2024-04-23T14:51:21.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.706+0000 7feffa000640 10 client.15798 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:51:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.707+0000 7feffa000640 1 -- 172.21.15.83:0/147135514 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608e62fda60 con 0x5608e63ab8f0 2024-04-23T14:51:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.707+0000 7feffa000640 10 client.15798 handle_command_reply: tid=0 2024-04-23T14:51:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.707+0000 7ff01e1f4740 1 client.15798 shutdown 2024-04-23T14:51:21.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.707+0000 7ff01e1f4740 1 -- 172.21.15.83:0/147135514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 msgr2=0x5608e639bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.707+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 0x5608e639bd20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff014112230 tx=0x7ff000009d20 comp rx=0 tx=0).stop 2024-04-23T14:51:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.708+0000 7ff01e1f4740 1 -- 172.21.15.83:0/147135514 shutdown_connections 2024-04-23T14:51:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.708+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x5608e63ab8f0 0x5608e6413e80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.708+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5608e639b940 0x5608e639c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.708+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5608e6391270 0x5608e639bd20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.708+0000 7ff01e1f4740 1 --2- 172.21.15.83:0/147135514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5608e6390950 0x5608e639d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.708+0000 7ff01e1f4740 1 -- 172.21.15.83:0/147135514 >> 172.21.15.83:0/147135514 conn(0x5608e638cc70 msgr2=0x5608e6396ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.708+0000 7feffa000640 0 client.15798 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:51:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.709+0000 7ff01e1f4740 1 -- 172.21.15.83:0/147135514 shutdown_connections 2024-04-23T14:51:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.709+0000 7ff01e1f4740 1 -- 172.21.15.83:0/147135514 wait complete. 2024-04-23T14:51:21.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.709+0000 7ff01e1f4740 20 client.15798 trim_cache size 0 max 16384 2024-04-23T14:51:21.711 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:51:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.710+0000 7ff01b400640 1 -- 172.21.15.83:0/1860091044 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feffc0a3e20 msgr2=0x7feffc0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.710+0000 7ff01b400640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feffc0a3e20 0x7feffc0a62e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff00c005f40 tx=0x7ff00c0096a0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.710+0000 7ff01b400640 1 -- 172.21.15.83:0/1860091044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff01410a9a0 msgr2=0x7ff014113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:21.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.710+0000 7ff01b400640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff01410a9a0 0x7ff014113940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff00802a9e0 tx=0x7ff008003c10 comp rx=0 tx=0).stop 2024-04-23T14:51:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.710+0000 7ff01b400640 1 -- 172.21.15.83:0/1860091044 shutdown_connections 2024-04-23T14:51:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.710+0000 7ff01b400640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff014113f50 0x7ff014110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.710+0000 7ff01b400640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feffc0a3e20 0x7feffc0a62e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.710+0000 7ff01b400640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff01410b2c0 0x7ff0141109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.710+0000 7ff01b400640 1 --2- 172.21.15.83:0/1860091044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff01410a9a0 0x7ff014113940 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.710+0000 7ff01b400640 1 -- 172.21.15.83:0/1860091044 >> 172.21.15.83:0/1860091044 conn(0x7ff01406d110 msgr2=0x7ff014070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.710+0000 7ff01b400640 1 -- 172.21.15.83:0/1860091044 shutdown_connections 2024-04-23T14:51:21.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:21.711+0000 7ff01b400640 1 -- 172.21.15.83:0/1860091044 wait complete. 2024-04-23T14:51:21.768 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:51:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.101+0000 7fd01b200640 1 -- 172.21.15.83:0/1931370102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd01c104320 msgr2=0x7fd01c104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.101+0000 7fd01b200640 1 --2- 172.21.15.83:0/1931370102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd01c104320 0x7fd01c104700 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009900 tx=0x7fd00c02a400 comp rx=0 tx=0).stop 2024-04-23T14:51:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.101+0000 7fd01b200640 1 -- 172.21.15.83:0/1931370102 shutdown_connections 2024-04-23T14:51:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.101+0000 7fd01b200640 1 --2- 172.21.15.83:0/1931370102 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd01c1093f0 0x7fd01c110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.101+0000 7fd01b200640 1 --2- 172.21.15.83:0/1931370102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd01c104cd0 0x7fd01c108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.101+0000 7fd01b200640 1 --2- 172.21.15.83:0/1931370102 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd01c104320 0x7fd01c104700 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.101+0000 7fd01b200640 1 -- 172.21.15.83:0/1931370102 >> 172.21.15.83:0/1931370102 conn(0x7fd01c0ffb40 msgr2=0x7fd01c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.101+0000 7fd01b200640 1 -- 172.21.15.83:0/1931370102 shutdown_connections 2024-04-23T14:51:22.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.101+0000 7fd01b200640 1 -- 172.21.15.83:0/1931370102 wait complete. 2024-04-23T14:51:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.102+0000 7fd01b200640 1 Processor -- start 2024-04-23T14:51:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.103+0000 7fd01b200640 1 -- start start 2024-04-23T14:51:22.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.103+0000 7fd01b200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd01c104320 0x7fd01c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.103+0000 7fd01b200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd01c1093f0 0x7fd01c1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.103+0000 7fd01b200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd01c19a6d0 0x7fd01c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.103+0000 7fd01b200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1144d0 con 0x7fd01c19a6d0 2024-04-23T14:51:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.103+0000 7fd01b200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c114350 con 0x7fd01c104320 2024-04-23T14:51:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.103+0000 7fd01b200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c114650 con 0x7fd01c1093f0 2024-04-23T14:51:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.103+0000 7fd019e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd01c104320 0x7fd01c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.103+0000 7fd019e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd01c104320 0x7fd01c1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37908/0 (socket says 172.21.15.83:37908) 2024-04-23T14:51:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.103+0000 7fd019e00640 1 -- 172.21.15.83:0/3577562281 learned_addr learned my addr 172.21.15.83:0/3577562281 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.103+0000 7fd013e00640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd01c1093f0 0x7fd01c1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.103+0000 7fd019e00640 1 -- 172.21.15.83:0/3577562281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd01c1093f0 msgr2=0x7fd01c1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.104+0000 7fd019e00640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd01c1093f0 0x7fd01c1a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.104+0000 7fd019e00640 1 -- 172.21.15.83:0/3577562281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd01c19a6d0 msgr2=0x7fd01c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.104+0000 7fd01a800640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd01c19a6d0 0x7fd01c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.104+0000 7fd019e00640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd01c19a6d0 0x7fd01c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.104+0000 7fd019e00640 1 -- 172.21.15.83:0/3577562281 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01c19b2d0 con 0x7fd01c104320 2024-04-23T14:51:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.104+0000 7fd013e00640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd01c1093f0 0x7fd01c1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:51:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.104+0000 7fd01a800640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd01c19a6d0 0x7fd01c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.104+0000 7fd019e00640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd01c104320 0x7fd01c1a0610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd00c006010 tx=0x7fd00c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.105+0000 7fd012a00640 1 -- 172.21.15.83:0/3577562281 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c03f070 con 0x7fd01c104320 2024-04-23T14:51:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.105+0000 7fd012a00640 1 -- 172.21.15.83:0/3577562281 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd00c031c90 con 0x7fd01c104320 2024-04-23T14:51:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.105+0000 7fd012a00640 1 -- 172.21.15.83:0/3577562281 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c035030 con 0x7fd01c104320 2024-04-23T14:51:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.105+0000 7fd01b200640 1 -- 172.21.15.83:0/3577562281 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01c19b560 con 0x7fd01c104320 2024-04-23T14:51:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.105+0000 7fd01b200640 1 -- 172.21.15.83:0/3577562281 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01c1a76a0 con 0x7fd01c104320 2024-04-23T14:51:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.108+0000 7fd023f9a740 1 Processor -- start 2024-04-23T14:51:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.108+0000 7fd023f9a740 1 -- start start 2024-04-23T14:51:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd023f9a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d82b40 0x55af35d82f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd023f9a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d83460 0x55af35d8d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd023f9a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 0x55af35d8ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd023f9a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55af35ce81d0 con 0x55af35d82b40 2024-04-23T14:51:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd023f9a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55af35c8e450 con 0x55af35d83460 2024-04-23T14:51:22.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd023f9a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55af35ce4190 con 0x55af35d8db30 2024-04-23T14:51:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd013e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 0x55af35d8ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd013e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 0x55af35d8ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:47506/0 (socket says 172.21.15.83:47506) 2024-04-23T14:51:22.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd013e00640 1 -- 172.21.15.83:0/1904024782 learned_addr learned my addr 172.21.15.83:0/1904024782 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd013e00640 1 -- 172.21.15.83:0/1904024782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d83460 msgr2=0x55af35d8d5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd013e00640 1 --2- 172.21.15.83:0/1904024782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d83460 0x55af35d8d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd013e00640 1 -- 172.21.15.83:0/1904024782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d82b40 msgr2=0x55af35d82f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.109+0000 7fd012a00640 1 -- 172.21.15.83:0/3577562281 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd00c042070 con 0x7fd01c104320 2024-04-23T14:51:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.110+0000 7fd01a800640 1 --2- 172.21.15.83:0/1904024782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d82b40 0x55af35d82f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.110+0000 7fd013e00640 1 --2- 172.21.15.83:0/1904024782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d82b40 0x55af35d82f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.110+0000 7fd013e00640 1 -- 172.21.15.83:0/1904024782 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af35d4b310 con 0x55af35d8db30 2024-04-23T14:51:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.110+0000 7fd01a800640 1 --2- 172.21.15.83:0/1904024782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d82b40 0x55af35d82f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:22.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.110+0000 7fd013e00640 1 --2- 172.21.15.83:0/1904024782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 0x55af35d8ff20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd004008c00 tx=0x7fd00400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b37d260cece15b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.111+0000 7fd012a00640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0000a3e70 0x7fd0000a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.111+0000 7fd00b400640 1 -- 172.21.15.83:0/1904024782 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004014070 con 0x55af35d8db30 2024-04-23T14:51:22.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.112+0000 7fd00b400640 1 -- 172.21.15.83:0/1904024782 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0040047a0 con 0x55af35d8db30 2024-04-23T14:51:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.112+0000 7fd00b400640 1 -- 172.21.15.83:0/1904024782 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004010670 con 0x55af35d8db30 2024-04-23T14:51:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.112+0000 7fd012a00640 1 -- 172.21.15.83:0/3577562281 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd00c0bb000 con 0x7fd01c104320 2024-04-23T14:51:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.112+0000 7fd01a800640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0000a3e70 0x7fd0000a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.112+0000 7fd023f9a740 1 -- 172.21.15.83:0/1904024782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 msgr2=0x55af35d8ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.112+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1904024782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 0x55af35d8ff20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd004008c00 tx=0x7fd00400e720 comp rx=0 tx=0).stop 2024-04-23T14:51:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.112+0000 7fd023f9a740 1 -- 172.21.15.83:0/1904024782 shutdown_connections 2024-04-23T14:51:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.112+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1904024782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 0x55af35d8ff20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.112+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1904024782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d83460 0x55af35d8d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.112+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1904024782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d82b40 0x55af35d82f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.112+0000 7fd023f9a740 1 -- 172.21.15.83:0/1904024782 >> 172.21.15.83:0/1904024782 conn(0x55af35d7ee60 msgr2=0x55af35d7f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:22.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.113+0000 7fd01a800640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0000a3e70 0x7fd0000a6330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd014009dc0 tx=0x7fd014009390 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:22.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.113+0000 7fd023f9a740 1 -- 172.21.15.83:0/1904024782 shutdown_connections 2024-04-23T14:51:22.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.114+0000 7fd023f9a740 1 -- 172.21.15.83:0/1904024782 wait complete. 2024-04-23T14:51:22.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.114+0000 7fd023f9a740 1 -- start start 2024-04-23T14:51:22.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.114+0000 7fd023f9a740 1 Processor -- start 2024-04-23T14:51:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.115+0000 7fd023f9a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 0x55af35d8d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.115+0000 7fd023f9a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d83460 0x55af35d8a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.115+0000 7fd023f9a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 0x55af35d8b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.115+0000 7fd023f9a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55af35d909c0 con 0x55af35d83460 2024-04-23T14:51:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.115+0000 7fd019e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 0x55af35d8d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.115+0000 7fd01a800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 0x55af35d8b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.115+0000 7fd019e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 0x55af35d8d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37910/0 (socket says 172.21.15.83:37910) 2024-04-23T14:51:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.115+0000 7fd019e00640 1 -- 172.21.15.83:0/2918918310 learned_addr learned my addr 172.21.15.83:0/2918918310 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.115+0000 7fd023f9a740 1 -- 172.21.15.83:0/2918918310 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55af35ca1810 con 0x55af35d82b40 2024-04-23T14:51:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.115+0000 7fd023f9a740 1 -- 172.21.15.83:0/2918918310 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55af35d90b40 con 0x55af35d8db30 2024-04-23T14:51:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.115+0000 7fd023f9a740 10 client.0 _ll_register_callbacks cb 0x55af35c6f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:22.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.115+0000 7fd013e00640 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d83460 0x55af35d8a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.116+0000 7fd019e00640 1 -- 172.21.15.83:0/2918918310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 msgr2=0x55af35d8b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.116+0000 7fd019e00640 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 0x55af35d8b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.116+0000 7fd019e00640 1 -- 172.21.15.83:0/2918918310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d83460 msgr2=0x55af35d8a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.116+0000 7fd019e00640 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d83460 0x55af35d8a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.116+0000 7fd019e00640 1 -- 172.21.15.83:0/2918918310 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af35cb23f0 con 0x55af35d82b40 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.116+0000 7fd013e00640 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d83460 0x55af35d8a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.116+0000 7fd01a800640 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 0x55af35d8b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.116+0000 7fd019e00640 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 0x55af35d8d5e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0ba3b0 tx=0x7fd00c0c56d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3fddb7a4eb49b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.116+0000 7fcfffe00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.116+0000 7fcfffe00640 1 -- 172.21.15.83:0/2918918310 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c0b9990 con 0x55af35d82b40 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.116+0000 7fcfffe00640 1 -- 172.21.15.83:0/2918918310 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd00c03b580 con 0x55af35d82b40 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.116+0000 7fd023f9a740 1 -- 172.21.15.83:0/2918918310 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0040047a0 con 0x55af35d82b40 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.117+0000 7fcfffe00640 1 -- 172.21.15.83:0/2918918310 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd00c0b9b30 con 0x55af35d82b40 2024-04-23T14:51:22.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.117+0000 7fd023f9a740 10 client.25949 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.117+0000 7fd023f9a740 1 -- 172.21.15.83:0/2918918310 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af359fa3f0 con 0x55af35d82b40 2024-04-23T14:51:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.117+0000 7fcfffe00640 1 -- 172.21.15.83:0/2918918310 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0040047a0 con 0x55af35d82b40 2024-04-23T14:51:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.118+0000 7fcfffe00640 1 -- 172.21.15.83:0/2918918310 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fd00c0433b0 con 0x55af35d82b40 2024-04-23T14:51:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.118+0000 7fd023f9a740 10 client.25949 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.118+0000 7fd023f9a740 10 client.25949 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:51:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.118+0000 7fd023f9a740 20 client.25949 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.118+0000 7fd023f9a740 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55af35da6550 0x55af35da89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.118+0000 7fd023f9a740 4 client.25949 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:51:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.118+0000 7fd023f9a740 1 -- 172.21.15.83:0/2918918310 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af35cf5bf0 con 0x55af35da6550 2024-04-23T14:51:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.118+0000 7fd013e00640 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55af35da6550 0x55af35da89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.119+0000 7fd013e00640 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55af35da6550 0x55af35da89c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49dadc0ed7e05a0b server_cookie=d429dc0d3fb7ee85 in_seq=0 out_seq=0 2024-04-23T14:51:22.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.119+0000 7fcfffe00640 10 client.25949 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:51:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.119+0000 7fcfffe00640 1 -- 172.21.15.83:0/2918918310 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af35cf5bf0 con 0x55af35da6550 2024-04-23T14:51:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.119+0000 7fcfffe00640 10 client.25949 handle_command_reply: tid=0 2024-04-23T14:51:22.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.121+0000 7fd023f9a740 1 client.25949 shutdown 2024-04-23T14:51:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.121+0000 7fd023f9a740 1 -- 172.21.15.83:0/2918918310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 msgr2=0x55af35d8d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.121+0000 7fd023f9a740 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 0x55af35d8d5e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0ba3b0 tx=0x7fd00c0c56d0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.121+0000 7fd023f9a740 1 -- 172.21.15.83:0/2918918310 shutdown_connections 2024-04-23T14:51:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.121+0000 7fd023f9a740 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55af35da6550 0x55af35da89c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.121+0000 7fd023f9a740 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d8db30 0x55af35d8b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.121+0000 7fcfffe00640 0 client.25949 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:51:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.121+0000 7fd023f9a740 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d83460 0x55af35d8a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.121+0000 7fd023f9a740 1 --2- 172.21.15.83:0/2918918310 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 0x55af35d8d5e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.121+0000 7fd023f9a740 1 -- 172.21.15.83:0/2918918310 >> 172.21.15.83:0/2918918310 conn(0x55af35d7ee60 msgr2=0x55af35d84b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.122+0000 7fd023f9a740 1 -- 172.21.15.83:0/2918918310 shutdown_connections 2024-04-23T14:51:22.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.122+0000 7fd023f9a740 1 -- 172.21.15.83:0/2918918310 wait complete. 2024-04-23T14:51:22.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.125+0000 7fd023f9a740 20 client.25949 trim_cache size 0 max 16384 2024-04-23T14:51:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.153+0000 7fd023f9a740 1 Processor -- start 2024-04-23T14:51:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.153+0000 7fd023f9a740 1 -- start start 2024-04-23T14:51:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.153+0000 7fd023f9a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 0x55af35d852a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.153+0000 7fd023f9a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d83460 0x55af35d8d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.153+0000 7fd023f9a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d8db30 0x55af35dae260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.153+0000 7fd023f9a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1ae030 con 0x55af35d8db30 2024-04-23T14:51:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.153+0000 7fd023f9a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1ab8d0 con 0x55af35d82b40 2024-04-23T14:51:22.157 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.153+0000 7fd023f9a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1a9740 con 0x55af35d83460 2024-04-23T14:51:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.154+0000 7fd01a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d8db30 0x55af35dae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.154+0000 7fd01a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d8db30 0x55af35dae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56728/0 (socket says 172.21.15.83:56728) 2024-04-23T14:51:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.154+0000 7fd01a800640 1 -- 172.21.15.83:0/1239787914 learned_addr learned my addr 172.21.15.83:0/1239787914 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.155+0000 7fd01a800640 1 -- 172.21.15.83:0/1239787914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d83460 msgr2=0x55af35d8d110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T14:51:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.155+0000 7fd01a800640 1 --2- 172.21.15.83:0/1239787914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d83460 0x55af35d8d110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.155+0000 7fd01a800640 1 -- 172.21.15.83:0/1239787914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 msgr2=0x55af35d852a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T14:51:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.155+0000 7fd01a800640 1 --2- 172.21.15.83:0/1239787914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 0x55af35d852a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.155+0000 7fd01a800640 1 -- 172.21.15.83:0/1239787914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af35d83290 con 0x55af35d8db30 2024-04-23T14:51:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.155+0000 7fd01a800640 1 --2- 172.21.15.83:0/1239787914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d8db30 0x55af35dae260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd01400fa90 tx=0x7fd014011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd69cc2c642e4be server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.156+0000 7fd00aa00640 1 -- 172.21.15.83:0/1239787914 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014004b20 con 0x55af35d8db30 2024-04-23T14:51:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.156+0000 7fd00aa00640 1 -- 172.21.15.83:0/1239787914 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd014004cc0 con 0x55af35d8db30 2024-04-23T14:51:22.158 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.156+0000 7fd00aa00640 1 -- 172.21.15.83:0/1239787914 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014018600 con 0x55af35d8db30 2024-04-23T14:51:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.157+0000 7fd023f9a740 1 -- 172.21.15.83:0/1239787914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d8db30 msgr2=0x55af35dae260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.157+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1239787914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d8db30 0x55af35dae260 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd01400fa90 tx=0x7fd014011000 comp rx=0 tx=0).stop 2024-04-23T14:51:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.157+0000 7fd023f9a740 1 -- 172.21.15.83:0/1239787914 shutdown_connections 2024-04-23T14:51:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.157+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1239787914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d8db30 0x55af35dae260 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.157+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1239787914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d83460 0x55af35d8d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.157+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1239787914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 0x55af35d852a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.157+0000 7fd023f9a740 1 -- 172.21.15.83:0/1239787914 >> 172.21.15.83:0/1239787914 conn(0x55af35d7ee60 msgr2=0x55af35d7f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.157+0000 7fd023f9a740 1 -- 172.21.15.83:0/1239787914 shutdown_connections 2024-04-23T14:51:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.158+0000 7fd023f9a740 1 -- 172.21.15.83:0/1239787914 wait complete. 2024-04-23T14:51:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.158+0000 7fd023f9a740 1 -- start start 2024-04-23T14:51:22.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.159+0000 7fd023f9a740 1 Processor -- start 2024-04-23T14:51:22.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.159+0000 7fd023f9a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 0x55af35d9e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.160+0000 7fd023f9a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d83460 0x55af35d87b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.160+0000 7fd023f9a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d8db30 0x55af35d88090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.160+0000 7fd023f9a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1a9d90 con 0x55af35d8db30 2024-04-23T14:51:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.160+0000 7fd023f9a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1ab200 con 0x55af35d82b40 2024-04-23T14:51:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.160+0000 7fd023f9a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd01c1aa460 con 0x55af35d83460 2024-04-23T14:51:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.160+0000 7fd023f9a740 10 client.0 _ll_register_callbacks cb 0x7fd00c043760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.160+0000 7fd013e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d83460 0x55af35d87b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.160+0000 7fd013e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d83460 0x55af35d87b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:47540/0 (socket says 172.21.15.83:47540) 2024-04-23T14:51:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.160+0000 7fd013e00640 1 -- 172.21.15.83:0/1455663664 learned_addr learned my addr 172.21.15.83:0/1455663664 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.160+0000 7fd013e00640 1 -- 172.21.15.83:0/1455663664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 msgr2=0x55af35d9e9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.160+0000 7fd013e00640 1 --2- 172.21.15.83:0/1455663664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 0x55af35d9e9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.160+0000 7fd013e00640 1 -- 172.21.15.83:0/1455663664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d8db30 msgr2=0x55af35d88090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.161+0000 7fd013e00640 1 --2- 172.21.15.83:0/1455663664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d8db30 0x55af35d88090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.161+0000 7fd013e00640 1 -- 172.21.15.83:0/1455663664 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af35ac47d0 con 0x55af35d83460 2024-04-23T14:51:22.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.161+0000 7fd013e00640 1 --2- 172.21.15.83:0/1455663664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d83460 0x55af35d87b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55af35d7fff0 tx=0x7fd0040047a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98f0ce816f4a5e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.161+0000 7fcfffe00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:51:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.162+0000 7fcfffe00640 1 -- 172.21.15.83:0/1455663664 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004014040 con 0x55af35d83460 2024-04-23T14:51:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.162+0000 7fcfffe00640 1 -- 172.21.15.83:0/1455663664 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd0040282c0 con 0x55af35d83460 2024-04-23T14:51:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.162+0000 7fd023f9a740 1 -- 172.21.15.83:0/1455663664 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd014004cc0 con 0x55af35d83460 2024-04-23T14:51:22.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.163+0000 7fcfffe00640 1 -- 172.21.15.83:0/1455663664 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd0040284c0 con 0x55af35d83460 2024-04-23T14:51:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.163+0000 7fd023f9a740 10 client.25924 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.163+0000 7fd023f9a740 1 -- 172.21.15.83:0/1455663664 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af35d8f3b0 con 0x55af35d83460 2024-04-23T14:51:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.164+0000 7fcfffe00640 1 -- 172.21.15.83:0/1455663664 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd014004cc0 con 0x55af35d83460 2024-04-23T14:51:22.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.164+0000 7fcfffe00640 1 -- 172.21.15.83:0/1455663664 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fd0040107a0 con 0x55af35d83460 2024-04-23T14:51:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.164+0000 7fd023f9a740 10 client.25924 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.164+0000 7fd023f9a740 10 client.25924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:51:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.164+0000 7fd023f9a740 20 client.25924 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.165+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1455663664 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55af35e0cda0 0x55af35e0f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.165+0000 7fd023f9a740 4 client.25924 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:51:22.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.165+0000 7fd023f9a740 1 -- 172.21.15.83:0/1455663664 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af35cefa60 con 0x55af35e0cda0 2024-04-23T14:51:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.165+0000 7fd019e00640 1 --2- 172.21.15.83:0/1455663664 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55af35e0cda0 0x55af35e0f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.166+0000 7fd019e00640 1 --2- 172.21.15.83:0/1455663664 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55af35e0cda0 0x55af35e0f190 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19b085e5418b80d8 server_cookie=d8e1510f7d9fcbef in_seq=0 out_seq=0 2024-04-23T14:51:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.166+0000 7fcfffe00640 10 client.25924 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:51:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.166+0000 7fcfffe00640 1 -- 172.21.15.83:0/1455663664 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af35cefa60 con 0x55af35e0cda0 2024-04-23T14:51:22.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.166+0000 7fcfffe00640 10 client.25924 handle_command_reply: tid=0 2024-04-23T14:51:22.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.166+0000 7fd023f9a740 1 client.25924 shutdown 2024-04-23T14:51:22.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.167+0000 7fd023f9a740 1 -- 172.21.15.83:0/1455663664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d83460 msgr2=0x55af35d87b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.167+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1455663664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d83460 0x55af35d87b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55af35d7fff0 tx=0x7fd0040047a0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.168+0000 7fd023f9a740 1 -- 172.21.15.83:0/1455663664 shutdown_connections 2024-04-23T14:51:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.168+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1455663664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af35d8db30 0x55af35d88090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.168+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1455663664 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55af35e0cda0 0x55af35e0f190 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.168+0000 7fcfffe00640 0 client.25924 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:51:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.168+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1455663664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55af35d83460 0x55af35d87b50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.168+0000 7fd023f9a740 1 --2- 172.21.15.83:0/1455663664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55af35d82b40 0x55af35d9e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.168+0000 7fd023f9a740 1 -- 172.21.15.83:0/1455663664 >> 172.21.15.83:0/1455663664 conn(0x55af35d7ee60 msgr2=0x55af35d7d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.168+0000 7fd023f9a740 1 -- 172.21.15.83:0/1455663664 shutdown_connections 2024-04-23T14:51:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.168+0000 7fd023f9a740 1 -- 172.21.15.83:0/1455663664 wait complete. 2024-04-23T14:51:22.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.169+0000 7fd023f9a740 20 client.25924 trim_cache size 0 max 16384 2024-04-23T14:51:22.171 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:51:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.171+0000 7fd01b200640 1 -- 172.21.15.83:0/3577562281 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0000a3e70 msgr2=0x7fd0000a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.171+0000 7fd01b200640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0000a3e70 0x7fd0000a6330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd014009dc0 tx=0x7fd014009390 comp rx=0 tx=0).stop 2024-04-23T14:51:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.171+0000 7fd01b200640 1 -- 172.21.15.83:0/3577562281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd01c104320 msgr2=0x7fd01c1a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.171+0000 7fd01b200640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd01c104320 0x7fd01c1a0610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd00c006010 tx=0x7fd00c003ba0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.171+0000 7fd01b200640 1 -- 172.21.15.83:0/3577562281 shutdown_connections 2024-04-23T14:51:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.171+0000 7fd01b200640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0000a3e70 0x7fd0000a6330 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.171+0000 7fd01b200640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd01c19a6d0 0x7fd01c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.171+0000 7fd01b200640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd01c1093f0 0x7fd01c1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.171+0000 7fd01b200640 1 --2- 172.21.15.83:0/3577562281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd01c104320 0x7fd01c1a0610 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.171+0000 7fd01b200640 1 -- 172.21.15.83:0/3577562281 >> 172.21.15.83:0/3577562281 conn(0x7fd01c0ffb40 msgr2=0x7fd01c100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.172+0000 7fd01b200640 1 -- 172.21.15.83:0/3577562281 shutdown_connections 2024-04-23T14:51:22.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.172+0000 7fd01b200640 1 -- 172.21.15.83:0/3577562281 wait complete. 2024-04-23T14:51:22.199 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:21 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/459660201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:51:22.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:51:22.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:21 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/459660201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:51:22.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:21 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/459660201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:51:22.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.530+0000 7f4fddc00640 1 -- 172.21.15.83:0/2363012147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd8107490 msgr2=0x7f4fd810ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.530+0000 7f4fddc00640 1 --2- 172.21.15.83:0/2363012147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd8107490 0x7f4fd810ef90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009be0 tx=0x7f4fd002a6f0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.531+0000 7f4fddc00640 1 -- 172.21.15.83:0/2363012147 shutdown_connections 2024-04-23T14:51:22.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.531+0000 7f4fddc00640 1 --2- 172.21.15.83:0/2363012147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd8107490 0x7f4fd810ef90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.531+0000 7f4fddc00640 1 --2- 172.21.15.83:0/2363012147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd8100da0 0x7f4fd8106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.531+0000 7f4fddc00640 1 --2- 172.21.15.83:0/2363012147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd81003f0 0x7f4fd81007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.531+0000 7f4fddc00640 1 -- 172.21.15.83:0/2363012147 >> 172.21.15.83:0/2363012147 conn(0x7f4fd8077060 msgr2=0x7f4fd80fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:22.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.531+0000 7f4fddc00640 1 -- 172.21.15.83:0/2363012147 shutdown_connections 2024-04-23T14:51:22.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.531+0000 7f4fddc00640 1 -- 172.21.15.83:0/2363012147 wait complete. 2024-04-23T14:51:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.533+0000 7f4fddc00640 1 Processor -- start 2024-04-23T14:51:22.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.534+0000 7f4fddc00640 1 -- start start 2024-04-23T14:51:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.534+0000 7f4fddc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd81003f0 0x7f4fd810e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.534+0000 7f4fddc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd8107490 0x7f4fd810e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.535+0000 7f4fddc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd81a37d0 0x7f4fd81a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.535+0000 7f4fddc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8112470 con 0x7f4fd81003f0 2024-04-23T14:51:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.535+0000 7f4fddc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81122f0 con 0x7f4fd81a37d0 2024-04-23T14:51:22.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.535+0000 7f4fddc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd81125f0 con 0x7f4fd8107490 2024-04-23T14:51:22.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.535+0000 7f4fd6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd81003f0 0x7f4fd810e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.535+0000 7f4fd6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd81003f0 0x7f4fd810e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56740/0 (socket says 172.21.15.83:56740) 2024-04-23T14:51:22.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.535+0000 7f4fd6a00640 1 -- 172.21.15.83:0/372591800 learned_addr learned my addr 172.21.15.83:0/372591800 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:22.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.535+0000 7f4fd7400640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd81a37d0 0x7f4fd81a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.536+0000 7f4fd6000640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd8107490 0x7f4fd810e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.537+0000 7f4fd6a00640 1 -- 172.21.15.83:0/372591800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd8107490 msgr2=0x7f4fd810e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.537+0000 7f4fd6a00640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd8107490 0x7f4fd810e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.537+0000 7f4fd6a00640 1 -- 172.21.15.83:0/372591800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd81a37d0 msgr2=0x7f4fd81a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.537+0000 7f4fd6a00640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd81a37d0 0x7f4fd81a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.537+0000 7f4fd6a00640 1 -- 172.21.15.83:0/372591800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd81a42c0 con 0x7f4fd81003f0 2024-04-23T14:51:22.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.538+0000 7f4fd6a00640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd81003f0 0x7f4fd810e1c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800e9c0 tx=0x7f4fc800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:22.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.538+0000 7f4fcf400640 1 -- 172.21.15.83:0/372591800 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800cde0 con 0x7f4fd81003f0 2024-04-23T14:51:22.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.538+0000 7f4fddc00640 1 -- 172.21.15.83:0/372591800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd81a45b0 con 0x7f4fd81003f0 2024-04-23T14:51:22.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.539+0000 7f4fddc00640 1 -- 172.21.15.83:0/372591800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81abdf0 con 0x7f4fd81003f0 2024-04-23T14:51:22.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.540+0000 7f4fcf400640 1 -- 172.21.15.83:0/372591800 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc8009460 con 0x7f4fd81003f0 2024-04-23T14:51:22.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.540+0000 7f4fcf400640 1 -- 172.21.15.83:0/372591800 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc80107f0 con 0x7f4fd81003f0 2024-04-23T14:51:22.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.543+0000 7f4fe09f4740 1 Processor -- start 2024-04-23T14:51:22.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.543+0000 7f4fe09f4740 1 -- start start 2024-04-23T14:51:22.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.543+0000 7f4fe09f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2e950 0x5563c6e2ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.543+0000 7f4fe09f4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2f270 0x5563c6e39400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.543+0000 7f4fe09f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e3bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.543+0000 7f4fe09f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563c6d941d0 con 0x5563c6e39940 2024-04-23T14:51:22.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.543+0000 7f4fe09f4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5563c6d3a450 con 0x5563c6e2f270 2024-04-23T14:51:22.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.543+0000 7f4fe09f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5563c6d90190 con 0x5563c6e2e950 2024-04-23T14:51:22.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.544+0000 7f4fcf400640 1 -- 172.21.15.83:0/372591800 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4fc8010990 con 0x7f4fd81003f0 2024-04-23T14:51:22.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.544+0000 7f4fd7400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2e950 0x5563c6e2ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.544+0000 7f4fd7400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2e950 0x5563c6e2ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:47576/0 (socket says 172.21.15.83:47576) 2024-04-23T14:51:22.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.544+0000 7f4fd7400640 1 -- 172.21.15.83:0/637376778 learned_addr learned my addr 172.21.15.83:0/637376778 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:22.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.544+0000 7f4fd6000640 1 --2- 172.21.15.83:0/637376778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e3bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.544+0000 7f4fd6a00640 1 --2- 172.21.15.83:0/637376778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2f270 0x5563c6e39400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.545+0000 7f4fd7400640 1 -- 172.21.15.83:0/637376778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2f270 msgr2=0x5563c6e39400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.545+0000 7f4fd7400640 1 --2- 172.21.15.83:0/637376778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2f270 0x5563c6e39400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.545+0000 7f4fd7400640 1 -- 172.21.15.83:0/637376778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 msgr2=0x5563c6e3bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.545+0000 7f4fd7400640 1 --2- 172.21.15.83:0/637376778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e3bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.545+0000 7f4fd7400640 1 -- 172.21.15.83:0/637376778 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c6df7310 con 0x5563c6e2e950 2024-04-23T14:51:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.545+0000 7f4fd7400640 1 --2- 172.21.15.83:0/637376778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2e950 0x5563c6e2ed30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0004af0 tx=0x7f4fd002f470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3afcc2d1e4c28682 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.545+0000 7f4fcf400640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4fbc0a3da0 0x7f4fbc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.546+0000 7f4fcf400640 1 -- 172.21.15.83:0/372591800 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4fc80d0be0 con 0x7f4fd81003f0 2024-04-23T14:51:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.546+0000 7f4fd6a00640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4fbc0a3da0 0x7f4fbc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.546+0000 7f4fc3e00640 1 -- 172.21.15.83:0/637376778 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0041030 con 0x5563c6e2e950 2024-04-23T14:51:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.546+0000 7f4fc3e00640 1 -- 172.21.15.83:0/637376778 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd0003970 con 0x5563c6e2e950 2024-04-23T14:51:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.546+0000 7f4fc3e00640 1 -- 172.21.15.83:0/637376778 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0036040 con 0x5563c6e2e950 2024-04-23T14:51:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.547+0000 7f4fe09f4740 1 -- 172.21.15.83:0/637376778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2e950 msgr2=0x5563c6e2ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.547+0000 7f4fd6a00640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4fbc0a3da0 0x7f4fbc0a6260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8000c00 tx=0x7f4fc80d4690 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.547+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/637376778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2e950 0x5563c6e2ed30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0004af0 tx=0x7f4fd002f470 comp rx=0 tx=0).stop 2024-04-23T14:51:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.547+0000 7f4fe09f4740 1 -- 172.21.15.83:0/637376778 shutdown_connections 2024-04-23T14:51:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.547+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/637376778 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e3bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.547+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/637376778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2f270 0x5563c6e39400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.547+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/637376778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2e950 0x5563c6e2ed30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.547+0000 7f4fe09f4740 1 -- 172.21.15.83:0/637376778 >> 172.21.15.83:0/637376778 conn(0x5563c6e2ac70 msgr2=0x5563c6e2b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:22.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.547+0000 7f4fe09f4740 1 -- 172.21.15.83:0/637376778 shutdown_connections 2024-04-23T14:51:22.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.547+0000 7f4fe09f4740 1 -- 172.21.15.83:0/637376778 wait complete. 2024-04-23T14:51:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.547+0000 7f4fe09f4740 1 -- start start 2024-04-23T14:51:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.547+0000 7f4fe09f4740 1 Processor -- start 2024-04-23T14:51:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.548+0000 7f4fe09f4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2e950 0x5563c6e39360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.548+0000 7f4fe09f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2f270 0x5563c6e36650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.548+0000 7f4fe09f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e36c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.548+0000 7f4fe09f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563c6e3c770 con 0x5563c6e39940 2024-04-23T14:51:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.548+0000 7f4fe09f4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5563c6d4d810 con 0x5563c6e2e950 2024-04-23T14:51:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.548+0000 7f4fe09f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5563c6e3c8f0 con 0x5563c6e2f270 2024-04-23T14:51:22.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.548+0000 7f4fe09f4740 10 client.0 _ll_register_callbacks cb 0x5563c6d1b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.548+0000 7f4fd7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e36c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.548+0000 7f4fd6000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2f270 0x5563c6e36650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.548+0000 7f4fd6000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2f270 0x5563c6e36650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:47580/0 (socket says 172.21.15.83:47580) 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.548+0000 7f4fd6000640 1 -- 172.21.15.83:0/1932465278 learned_addr learned my addr 172.21.15.83:0/1932465278 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.549+0000 7f4fd7400640 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2e950 0x5563c6e39360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.549+0000 7f4fd6000640 1 -- 172.21.15.83:0/1932465278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2e950 msgr2=0x5563c6e39360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.549+0000 7f4fd6000640 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2e950 0x5563c6e39360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.549+0000 7f4fd6000640 1 -- 172.21.15.83:0/1932465278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 msgr2=0x5563c6e36c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.549+0000 7f4fd6000640 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e36c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.549+0000 7f4fd6000640 1 -- 172.21.15.83:0/1932465278 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c6d5e3f0 con 0x5563c6e2f270 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.549+0000 7f4fd7400640 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e36c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.549+0000 7f4fd6000640 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2f270 0x5563c6e36650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4fd81011b0 tx=0x7f4fc4013b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=973ba348bbcdf4c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.549+0000 7f4fc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.549+0000 7f4fc0c00640 1 -- 172.21.15.83:0/1932465278 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4011dd0 con 0x5563c6e2f270 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.550+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1932465278 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd0003970 con 0x5563c6e2f270 2024-04-23T14:51:22.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.550+0000 7f4fc0c00640 1 -- 172.21.15.83:0/1932465278 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4fc4004020 con 0x5563c6e2f270 2024-04-23T14:51:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.550+0000 7f4fc0c00640 1 -- 172.21.15.83:0/1932465278 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4fc4004220 con 0x5563c6e2f270 2024-04-23T14:51:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.550+0000 7f4fe09f4740 10 client.25933 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.550+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1932465278 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5563c6aa63f0 con 0x5563c6e2f270 2024-04-23T14:51:22.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.550+0000 7f4fc0c00640 1 -- 172.21.15.83:0/1932465278 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd0003970 con 0x5563c6e2f270 2024-04-23T14:51:22.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.550+0000 7f4fc0c00640 1 -- 172.21.15.83:0/1932465278 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f4fc401a070 con 0x5563c6e2f270 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.551+0000 7f4fe09f4740 10 client.25933 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.551+0000 7f4fe09f4740 10 client.25933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.551+0000 7f4fe09f4740 20 client.25933 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.551+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5563c6e49b70 0x5563c6e4bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.551+0000 7f4fe09f4740 4 client.25933 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.551+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1932465278 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563c6da1bf0 con 0x5563c6e49b70 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.551+0000 7f4fd7400640 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5563c6e49b70 0x5563c6e4bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.551+0000 7f4fd7400640 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5563c6e49b70 0x5563c6e4bfd0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48e1ffacb7523e4 server_cookie=fd0c709303379931 in_seq=0 out_seq=0 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.551+0000 7f4fc0c00640 10 client.25933 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fc0c00640 1 -- 172.21.15.83:0/1932465278 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5563c6da1bf0 con 0x5563c6e49b70 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fc0c00640 10 client.25933 handle_command_reply: tid=0 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fe09f4740 1 client.25933 shutdown 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1932465278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2f270 msgr2=0x5563c6e36650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2f270 0x5563c6e36650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4fd81011b0 tx=0x7f4fc4013b70 comp rx=0 tx=0).stop 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1932465278 shutdown_connections 2024-04-23T14:51:22.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5563c6e49b70 0x5563c6e4bfd0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fc0c00640 0 client.25933 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e36c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2f270 0x5563c6e36650 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/1932465278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2e950 0x5563c6e39360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1932465278 >> 172.21.15.83:0/1932465278 conn(0x5563c6e2ac70 msgr2=0x5563c6e30950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1932465278 shutdown_connections 2024-04-23T14:51:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.552+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1932465278 wait complete. 2024-04-23T14:51:22.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.553+0000 7f4fe09f4740 20 client.25933 trim_cache size 0 max 16384 2024-04-23T14:51:22.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.567+0000 7f4fe09f4740 1 Processor -- start 2024-04-23T14:51:22.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.567+0000 7f4fe09f4740 1 -- start start 2024-04-23T14:51:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.567+0000 7f4fe09f4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2e950 0x5563c6e310b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.567+0000 7f4fe09f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2f270 0x5563c6e357d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.567+0000 7f4fe09f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e29c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.567+0000 7f4fe09f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8011030 con 0x5563c6e39940 2024-04-23T14:51:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.567+0000 7f4fe09f4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb800e8d0 con 0x5563c6e2e950 2024-04-23T14:51:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.567+0000 7f4fe09f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb800c720 con 0x5563c6e2f270 2024-04-23T14:51:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.567+0000 7f4fd7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2e950 0x5563c6e310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.567+0000 7f4fd7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2e950 0x5563c6e310b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37960/0 (socket says 172.21.15.83:37960) 2024-04-23T14:51:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.567+0000 7f4fd7400640 1 -- 172.21.15.83:0/1773024632 learned_addr learned my addr 172.21.15.83:0/1773024632 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:22.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.567+0000 7f4fd7400640 1 --2- 172.21.15.83:0/1773024632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e29c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.568+0000 7f4fd6000640 1 --2- 172.21.15.83:0/1773024632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2f270 0x5563c6e357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.568+0000 7f4fd7400640 1 -- 172.21.15.83:0/1773024632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2f270 msgr2=0x5563c6e357d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.568+0000 7f4fd7400640 1 --2- 172.21.15.83:0/1773024632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2f270 0x5563c6e357d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.568+0000 7f4fd7400640 1 -- 172.21.15.83:0/1773024632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 msgr2=0x5563c6e29c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.568+0000 7f4fd7400640 1 --2- 172.21.15.83:0/1773024632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e29c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.568+0000 7f4fd7400640 1 -- 172.21.15.83:0/1773024632 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563c6b707d0 con 0x5563c6e2e950 2024-04-23T14:51:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.568+0000 7f4fd7400640 1 --2- 172.21.15.83:0/1773024632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2e950 0x5563c6e310b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00092f0 tx=0x7f4fd0030620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=118221a026d2ebdc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.568+0000 7f4fc3400640 1 -- 172.21.15.83:0/1773024632 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0041040 con 0x5563c6e2e950 2024-04-23T14:51:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.568+0000 7f4fc3400640 1 -- 172.21.15.83:0/1773024632 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd00097e0 con 0x5563c6e2e950 2024-04-23T14:51:22.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.568+0000 7f4fc3400640 1 -- 172.21.15.83:0/1773024632 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00506f0 con 0x5563c6e2e950 2024-04-23T14:51:22.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.569+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1773024632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2e950 msgr2=0x5563c6e310b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.569+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/1773024632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2e950 0x5563c6e310b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4fd00092f0 tx=0x7f4fd0030620 comp rx=0 tx=0).stop 2024-04-23T14:51:22.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.569+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1773024632 shutdown_connections 2024-04-23T14:51:22.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.569+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/1773024632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e39940 0x5563c6e29c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.569+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/1773024632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e2f270 0x5563c6e357d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.569+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/1773024632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2e950 0x5563c6e310b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.569+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1773024632 >> 172.21.15.83:0/1773024632 conn(0x5563c6e2ac70 msgr2=0x5563c6e29560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:22.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.569+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1773024632 shutdown_connections 2024-04-23T14:51:22.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.569+0000 7f4fe09f4740 1 -- 172.21.15.83:0/1773024632 wait complete. 2024-04-23T14:51:22.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.570+0000 7f4fe09f4740 1 -- start start 2024-04-23T14:51:22.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.570+0000 7f4fe09f4740 1 Processor -- start 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.570+0000 7f4fe09f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e2e950 0x5563c6e41ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.570+0000 7f4fe09f4740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2f270 0x5563c6e420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.570+0000 7f4fe09f4740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e39940 0x5563c6e328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.570+0000 7f4fe09f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb800cd90 con 0x5563c6e2e950 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.570+0000 7f4fe09f4740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb800e200 con 0x5563c6e2f270 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.570+0000 7f4fe09f4740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb800d460 con 0x5563c6e39940 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.570+0000 7f4fe09f4740 10 client.0 _ll_register_callbacks cb 0x7f4fc40096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fd6000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2f270 0x5563c6e420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fd7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e2e950 0x5563c6e41ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fd6000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2f270 0x5563c6e420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37976/0 (socket says 172.21.15.83:37976) 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fd6000640 1 -- 172.21.15.83:0/118700566 learned_addr learned my addr 172.21.15.83:0/118700566 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fd7400640 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e39940 0x5563c6e328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fd6000640 1 -- 172.21.15.83:0/118700566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e39940 msgr2=0x5563c6e328e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:22.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fd6000640 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e39940 0x5563c6e328e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fd6000640 1 -- 172.21.15.83:0/118700566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e2e950 msgr2=0x5563c6e41ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:22.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fd6000640 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e2e950 0x5563c6e41ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fd6000640 1 -- 172.21.15.83:0/118700566 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563c6e32f20 con 0x5563c6e2f270 2024-04-23T14:51:22.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fd7400640 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e39940 0x5563c6e328e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:51:22.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fd6000640 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2f270 0x5563c6e420e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x5563c6e35eb0 tx=0x7f4fc4011840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b70e8753599bd74 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:22.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:51:22.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fc0c00640 1 -- 172.21.15.83:0/118700566 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4027070 con 0x5563c6e2f270 2024-04-23T14:51:22.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fe09f4740 1 -- 172.21.15.83:0/118700566 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fd00097e0 con 0x5563c6e2f270 2024-04-23T14:51:22.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fc0c00640 1 -- 172.21.15.83:0/118700566 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4fc4004f10 con 0x5563c6e2f270 2024-04-23T14:51:22.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fc0c00640 1 -- 172.21.15.83:0/118700566 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4fc4039050 con 0x5563c6e2f270 2024-04-23T14:51:22.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fe09f4740 10 client.25967 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:22.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.572+0000 7f4fe09f4740 1 -- 172.21.15.83:0/118700566 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5563c6e34210 con 0x5563c6e2f270 2024-04-23T14:51:22.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.573+0000 7f4fc0c00640 1 -- 172.21.15.83:0/118700566 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd00097e0 con 0x5563c6e2f270 2024-04-23T14:51:22.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.573+0000 7f4fc0c00640 1 -- 172.21.15.83:0/118700566 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f4fc4034030 con 0x5563c6e2f270 2024-04-23T14:51:22.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.573+0000 7f4fe09f4740 10 client.25967 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:22.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.573+0000 7f4fe09f4740 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:22.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.573+0000 7f4fe09f4740 20 client.25967 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:22.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.574+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5563c6eb12f0 0x5563c6eb3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:22.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.574+0000 7f4fe09f4740 4 client.25967 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:51:22.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.574+0000 7f4fe09f4740 1 -- 172.21.15.83:0/118700566 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563c6d9ba60 con 0x5563c6eb12f0 2024-04-23T14:51:22.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.574+0000 7f4fd7400640 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5563c6eb12f0 0x5563c6eb3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:22.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.575+0000 7f4fd7400640 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5563c6eb12f0 0x5563c6eb3710 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19e559c4d93c1e29 server_cookie=4769d934cebff686 in_seq=0 out_seq=0 2024-04-23T14:51:22.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.575+0000 7f4fc0c00640 10 client.25967 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:22.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.575+0000 7f4fc0c00640 1 -- 172.21.15.83:0/118700566 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563c6d9ba60 con 0x5563c6eb12f0 2024-04-23T14:51:22.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.575+0000 7f4fc0c00640 10 client.25967 handle_command_reply: tid=0 2024-04-23T14:51:22.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.576+0000 7f4fe09f4740 1 client.25967 shutdown 2024-04-23T14:51:22.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.576+0000 7f4fe09f4740 1 -- 172.21.15.83:0/118700566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2f270 msgr2=0x5563c6e420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.576+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2f270 0x5563c6e420e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x5563c6e35eb0 tx=0x7f4fc4011840 comp rx=0 tx=0).stop 2024-04-23T14:51:22.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.576+0000 7f4fe09f4740 1 -- 172.21.15.83:0/118700566 shutdown_connections 2024-04-23T14:51:22.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.576+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5563c6eb12f0 0x5563c6eb3710 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.576+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5563c6e39940 0x5563c6e328e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.576+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5563c6e2f270 0x5563c6e420e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.576+0000 7f4fe09f4740 1 --2- 172.21.15.83:0/118700566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563c6e2e950 0x5563c6e41ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.576+0000 7f4fe09f4740 1 -- 172.21.15.83:0/118700566 >> 172.21.15.83:0/118700566 conn(0x5563c6e2ac70 msgr2=0x5563c6e29df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:22.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.576+0000 7f4fc0c00640 0 client.25967 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:22.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.576+0000 7f4fe09f4740 1 -- 172.21.15.83:0/118700566 shutdown_connections 2024-04-23T14:51:22.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.577+0000 7f4fe09f4740 1 -- 172.21.15.83:0/118700566 wait complete. 2024-04-23T14:51:22.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.578+0000 7f4fe09f4740 20 client.25967 trim_cache size 0 max 16384 2024-04-23T14:51:22.581 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:51:22.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.579+0000 7f4fddc00640 1 -- 172.21.15.83:0/372591800 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4fbc0a3da0 msgr2=0x7f4fbc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.579+0000 7f4fddc00640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4fbc0a3da0 0x7f4fbc0a6260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8000c00 tx=0x7f4fc80d4690 comp rx=0 tx=0).stop 2024-04-23T14:51:22.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.579+0000 7f4fddc00640 1 -- 172.21.15.83:0/372591800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd81003f0 msgr2=0x7f4fd810e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:22.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.579+0000 7f4fddc00640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd81003f0 0x7f4fd810e1c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800e9c0 tx=0x7f4fc800ee80 comp rx=0 tx=0).stop 2024-04-23T14:51:22.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.579+0000 7f4fddc00640 1 -- 172.21.15.83:0/372591800 shutdown_connections 2024-04-23T14:51:22.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.579+0000 7f4fddc00640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4fd81a37d0 0x7f4fd81a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.579+0000 7f4fddc00640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4fd8107490 0x7f4fd810e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.579+0000 7f4fddc00640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4fbc0a3da0 0x7f4fbc0a6260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.579+0000 7f4fddc00640 1 --2- 172.21.15.83:0/372591800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4fd81003f0 0x7f4fd810e1c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:22.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.579+0000 7f4fddc00640 1 -- 172.21.15.83:0/372591800 >> 172.21.15.83:0/372591800 conn(0x7f4fd8077060 msgr2=0x7f4fd81052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:22.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.580+0000 7f4fddc00640 1 -- 172.21.15.83:0/372591800 shutdown_connections 2024-04-23T14:51:22.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:22.580+0000 7f4fddc00640 1 -- 172.21.15.83:0/372591800 wait complete. 2024-04-23T14:51:23.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:22 smithi203 ceph-mon[32800]: pgmap v90: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 109 MiB/s wr, 49 op/s 2024-04-23T14:51:23.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:22 smithi083 ceph-mon[27867]: pgmap v90: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 109 MiB/s wr, 49 op/s 2024-04-23T14:51:23.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:22 smithi173 ceph-mon[32443]: pgmap v90: 97 pgs: 97 active+clean; 9.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 109 MiB/s wr, 49 op/s 2024-04-23T14:51:23.623 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:51:23.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:51:23.624 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:51:23.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.845+0000 7f17bbc00640 1 -- 172.21.15.83:0/2278022180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17b41063f0 msgr2=0x7f17b41068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:23.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.845+0000 7f17bbc00640 1 --2- 172.21.15.83:0/2278022180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17b41063f0 0x7f17b41068a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009be0 tx=0x7f17ac02a750 comp rx=0 tx=0).stop 2024-04-23T14:51:23.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.846+0000 7f17bbc00640 1 -- 172.21.15.83:0/2278022180 shutdown_connections 2024-04-23T14:51:23.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.846+0000 7f17bbc00640 1 --2- 172.21.15.83:0/2278022180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17b41063f0 0x7f17b41068a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.846+0000 7f17bbc00640 1 --2- 172.21.15.83:0/2278022180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b4101810 0x7f17b4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.846+0000 7f17bbc00640 1 --2- 172.21.15.83:0/2278022180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17b4100ef0 0x7f17b41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.846+0000 7f17bbc00640 1 -- 172.21.15.83:0/2278022180 >> 172.21.15.83:0/2278022180 conn(0x7f17b4078060 msgr2=0x7f17b4078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:23.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.846+0000 7f17bbc00640 1 -- 172.21.15.83:0/2278022180 shutdown_connections 2024-04-23T14:51:23.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.847+0000 7f17bbc00640 1 -- 172.21.15.83:0/2278022180 wait complete. 2024-04-23T14:51:23.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.849+0000 7f17bbc00640 1 Processor -- start 2024-04-23T14:51:23.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.849+0000 7f17bbc00640 1 -- start start 2024-04-23T14:51:23.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.849+0000 7f17bbc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17b4100ef0 0x7f17b419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.849+0000 7f17bbc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17b4101810 0x7f17b419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.849+0000 7f17bbc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b41063f0 0x7f17b41a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.849+0000 7f17bbc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41189c0 con 0x7f17b41063f0 2024-04-23T14:51:23.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.849+0000 7f17bbc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f17b4118840 con 0x7f17b4100ef0 2024-04-23T14:51:23.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.849+0000 7f17bbc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f17b4118b40 con 0x7f17b4101810 2024-04-23T14:51:23.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.850+0000 7f17b9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b41063f0 0x7f17b41a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.850+0000 7f17b9c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b41063f0 0x7f17b41a36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56814/0 (socket says 172.21.15.83:56814) 2024-04-23T14:51:23.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.850+0000 7f17b9c00640 1 -- 172.21.15.83:0/8473069 learned_addr learned my addr 172.21.15.83:0/8473069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:23.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.850+0000 7f17b9200640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17b4100ef0 0x7f17b419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.850+0000 7f17b9c00640 1 -- 172.21.15.83:0/8473069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17b4101810 msgr2=0x7f17b419f320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:51:23.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.850+0000 7f17b9c00640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17b4101810 0x7f17b419f320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.850+0000 7f17b9c00640 1 -- 172.21.15.83:0/8473069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17b4100ef0 msgr2=0x7f17b419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:23.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.850+0000 7f17b9c00640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17b4100ef0 0x7f17b419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.850+0000 7f17b9c00640 1 -- 172.21.15.83:0/8473069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b41a3dd0 con 0x7f17b41063f0 2024-04-23T14:51:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.851+0000 7f17b9c00640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b41063f0 0x7f17b41a36a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009b30 tx=0x7f17ac0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.851+0000 7f17b1600640 1 -- 172.21.15.83:0/8473069 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac041020 con 0x7f17b41063f0 2024-04-23T14:51:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.851+0000 7f17bbc00640 1 -- 172.21.15.83:0/8473069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b41a4060 con 0x7f17b41063f0 2024-04-23T14:51:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.851+0000 7f17bbc00640 1 -- 172.21.15.83:0/8473069 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b41ab950 con 0x7f17b41063f0 2024-04-23T14:51:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.851+0000 7f17b1600640 1 -- 172.21.15.83:0/8473069 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac003e50 con 0x7f17b41063f0 2024-04-23T14:51:23.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.851+0000 7f17b1600640 1 -- 172.21.15.83:0/8473069 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac036040 con 0x7f17b41063f0 2024-04-23T14:51:23.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.857+0000 7f17be9ce740 1 Processor -- start 2024-04-23T14:51:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.857+0000 7f17b1600640 1 -- 172.21.15.83:0/8473069 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f17ac044020 con 0x7f17b41063f0 2024-04-23T14:51:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.858+0000 7f17be9ce740 1 -- start start 2024-04-23T14:51:23.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.858+0000 7f17be9ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x558280834470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.858+0000 7f17be9ce740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x55828083eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.858+0000 7f17be9ce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x558280841470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.858+0000 7f17be9ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558280799e70 con 0x558280834090 2024-04-23T14:51:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.858+0000 7f17be9ce740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558280740380 con 0x5582808349b0 2024-04-23T14:51:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.858+0000 7f17be9ce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558280795e30 con 0x55828083f080 2024-04-23T14:51:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.859+0000 7f17b9200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x55828083eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.859+0000 7f17b9200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x55828083eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38018/0 (socket says 172.21.15.83:38018) 2024-04-23T14:51:23.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.859+0000 7f17b9200640 1 -- 172.21.15.83:0/869684986 learned_addr learned my addr 172.21.15.83:0/869684986 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:23.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.860+0000 7f17b9200640 1 -- 172.21.15.83:0/869684986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 msgr2=0x558280841470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:23.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.860+0000 7f17b9200640 1 --2- 172.21.15.83:0/869684986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x558280841470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.860+0000 7f17b9200640 1 -- 172.21.15.83:0/869684986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 msgr2=0x558280834470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:23.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.860+0000 7f17b9200640 1 --2- 172.21.15.83:0/869684986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x558280834470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.860+0000 7f17b9200640 1 -- 172.21.15.83:0/869684986 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582805747e0 con 0x5582808349b0 2024-04-23T14:51:23.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.860+0000 7f17b1600640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f179c0a3e20 0x7f179c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.861+0000 7f17b9200640 1 --2- 172.21.15.83:0/869684986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x55828083eb40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009c50 tx=0x7f17a80117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e4e2820015bb7a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:23.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.861+0000 7f17a6000640 1 -- 172.21.15.83:0/869684986 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8012450 con 0x5582808349b0 2024-04-23T14:51:23.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.860+0000 7f17b1600640 1 -- 172.21.15.83:0/8473069 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f17ac0f1270 con 0x7f17b41063f0 2024-04-23T14:51:23.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.861+0000 7f17a6000640 1 -- 172.21.15.83:0/869684986 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a8012ad0 con 0x5582808349b0 2024-04-23T14:51:23.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.861+0000 7f17a6000640 1 -- 172.21.15.83:0/869684986 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a800fc20 con 0x5582808349b0 2024-04-23T14:51:23.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.862+0000 7f17b3e00640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f179c0a3e20 0x7f179c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.862+0000 7f17be9ce740 1 -- 172.21.15.83:0/869684986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 msgr2=0x55828083eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:23.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.862+0000 7f17be9ce740 1 --2- 172.21.15.83:0/869684986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x55828083eb40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f17a8009c50 tx=0x7f17a80117b0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.863+0000 7f17be9ce740 1 -- 172.21.15.83:0/869684986 shutdown_connections 2024-04-23T14:51:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.863+0000 7f17be9ce740 1 --2- 172.21.15.83:0/869684986 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x558280841470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.863+0000 7f17be9ce740 1 --2- 172.21.15.83:0/869684986 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x55828083eb40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.863+0000 7f17be9ce740 1 --2- 172.21.15.83:0/869684986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x558280834470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.863+0000 7f17be9ce740 1 -- 172.21.15.83:0/869684986 >> 172.21.15.83:0/869684986 conn(0x5582808303b0 msgr2=0x558280830790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.863+0000 7f17b3e00640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f179c0a3e20 0x7f179c0a62e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x558280831540 tx=0x7f17a0005d40 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:23.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.863+0000 7f17be9ce740 1 -- 172.21.15.83:0/869684986 shutdown_connections 2024-04-23T14:51:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.863+0000 7f17be9ce740 1 -- 172.21.15.83:0/869684986 wait complete. 2024-04-23T14:51:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.863+0000 7f17be9ce740 1 -- start start 2024-04-23T14:51:23.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.864+0000 7f17be9ce740 1 Processor -- start 2024-04-23T14:51:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.864+0000 7f17be9ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x55828083bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.864+0000 7f17be9ce740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x55828083c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.864+0000 7f17be9ce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x55828083ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.864+0000 7f17be9ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558280841e70 con 0x558280834090 2024-04-23T14:51:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.864+0000 7f17be9ce740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558280753660 con 0x5582808349b0 2024-04-23T14:51:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.864+0000 7f17be9ce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558280841ff0 con 0x55828083f080 2024-04-23T14:51:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.864+0000 7f17be9ce740 10 client.0 _ll_register_callbacks cb 0x558280741aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.865+0000 7f17b9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x55828083bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.865+0000 7f17b9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x55828083bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56830/0 (socket says 172.21.15.83:56830) 2024-04-23T14:51:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.865+0000 7f17b9200640 1 -- 172.21.15.83:0/1167162352 learned_addr learned my addr 172.21.15.83:0/1167162352 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.865+0000 7f17b3e00640 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x55828083c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.865+0000 7f17b9c00640 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x55828083ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.865+0000 7f17b9200640 1 -- 172.21.15.83:0/1167162352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 msgr2=0x55828083ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.865+0000 7f17b9200640 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x55828083ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.865+0000 7f17b9200640 1 -- 172.21.15.83:0/1167162352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 msgr2=0x55828083c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.865+0000 7f17b9200640 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x55828083c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.865+0000 7f17b9200640 1 -- 172.21.15.83:0/1167162352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582805d4f70 con 0x558280834090 2024-04-23T14:51:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.865+0000 7f17b9c00640 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x55828083ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:51:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.866+0000 7f17b9200640 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x55828083bfc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f17a8005900 tx=0x7f17a8012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cba2a42b1a4bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.866+0000 7f179aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:51:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.866+0000 7f179aa00640 1 -- 172.21.15.83:0/1167162352 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8028070 con 0x558280834090 2024-04-23T14:51:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.866+0000 7f179aa00640 1 -- 172.21.15.83:0/1167162352 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f17a8004450 con 0x558280834090 2024-04-23T14:51:23.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.867+0000 7f17be9ce740 1 -- 172.21.15.83:0/1167162352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a8012ad0 con 0x558280834090 2024-04-23T14:51:23.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.867+0000 7f179aa00640 1 -- 172.21.15.83:0/1167162352 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17a8034050 con 0x558280834090 2024-04-23T14:51:23.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.867+0000 7f17be9ce740 10 client.15828 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:23.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.867+0000 7f17be9ce740 1 -- 172.21.15.83:0/1167162352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55828046c170 con 0x558280834090 2024-04-23T14:51:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.867+0000 7f179aa00640 1 -- 172.21.15.83:0/1167162352 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a8012ad0 con 0x558280834090 2024-04-23T14:51:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.867+0000 7f179aa00640 1 -- 172.21.15.83:0/1167162352 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f17a80035f0 con 0x558280834090 2024-04-23T14:51:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.868+0000 7f17be9ce740 10 client.15828 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.868+0000 7f17be9ce740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.868+0000 7f17be9ce740 20 client.15828 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.868+0000 7f17be9ce740 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55828084f300 0x5582808516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.868+0000 7f17be9ce740 4 client.15828 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:51:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.868+0000 7f17be9ce740 1 -- 172.21.15.83:0/1167162352 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582807a7890 con 0x55828084f300 2024-04-23T14:51:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.868+0000 7f17b3e00640 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55828084f300 0x5582808516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.869+0000 7f17b3e00640 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55828084f300 0x5582808516f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1bc1ffd04c26939 server_cookie=3810fec0e766f1 in_seq=0 out_seq=0 2024-04-23T14:51:23.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.869+0000 7f179aa00640 10 client.15828 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:23.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.869+0000 7f179aa00640 1 -- 172.21.15.83:0/1167162352 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582807a7890 con 0x55828084f300 2024-04-23T14:51:23.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.869+0000 7f179aa00640 10 client.15828 handle_command_reply: tid=0 2024-04-23T14:51:23.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.870+0000 7f17be9ce740 1 client.15828 shutdown 2024-04-23T14:51:23.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.871+0000 7f17be9ce740 1 -- 172.21.15.83:0/1167162352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 msgr2=0x55828083bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:23.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.871+0000 7f17be9ce740 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x55828083bfc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f17a8005900 tx=0x7f17a8012f90 comp rx=0 tx=0).stop 2024-04-23T14:51:23.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.871+0000 7f17be9ce740 1 -- 172.21.15.83:0/1167162352 shutdown_connections 2024-04-23T14:51:23.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.871+0000 7f17be9ce740 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55828084f300 0x5582808516f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.871+0000 7f17be9ce740 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x55828083ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.871+0000 7f17be9ce740 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x55828083c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.871+0000 7f179aa00640 0 client.15828 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:23.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.871+0000 7f17be9ce740 1 --2- 172.21.15.83:0/1167162352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x55828083bfc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.871+0000 7f17be9ce740 1 -- 172.21.15.83:0/1167162352 >> 172.21.15.83:0/1167162352 conn(0x5582808303b0 msgr2=0x55828082f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:23.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.871+0000 7f17be9ce740 1 -- 172.21.15.83:0/1167162352 shutdown_connections 2024-04-23T14:51:23.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.871+0000 7f17be9ce740 1 -- 172.21.15.83:0/1167162352 wait complete. 2024-04-23T14:51:23.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.871+0000 7f17be9ce740 20 client.15828 trim_cache size 0 max 16384 2024-04-23T14:51:23.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.911+0000 7f17be9ce740 1 Processor -- start 2024-04-23T14:51:23.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.912+0000 7f17be9ce740 1 -- start start 2024-04-23T14:51:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.913+0000 7f17be9ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x55828084ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.913+0000 7f17be9ce740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x558280836330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.913+0000 7f17be9ce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x558280856db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.913+0000 7f17be9ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41b6ba0 con 0x558280834090 2024-04-23T14:51:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.913+0000 7f17be9ce740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41b4440 con 0x5582808349b0 2024-04-23T14:51:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.913+0000 7f17be9ce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41b2290 con 0x55828083f080 2024-04-23T14:51:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.913+0000 7f17b9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x55828084ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.913+0000 7f17b9200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x55828084ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56840/0 (socket says 172.21.15.83:56840) 2024-04-23T14:51:23.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.913+0000 7f17b9200640 1 -- 172.21.15.83:0/3131886451 learned_addr learned my addr 172.21.15.83:0/3131886451 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.913+0000 7f17b9c00640 1 --2- 172.21.15.83:0/3131886451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x558280856db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.913+0000 7f17b3e00640 1 --2- 172.21.15.83:0/3131886451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x558280836330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.914+0000 7f17b9200640 1 -- 172.21.15.83:0/3131886451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 msgr2=0x558280856db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.914+0000 7f17b9200640 1 --2- 172.21.15.83:0/3131886451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x558280856db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.914+0000 7f17b9200640 1 -- 172.21.15.83:0/3131886451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 msgr2=0x558280836330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.914+0000 7f17b9200640 1 --2- 172.21.15.83:0/3131886451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x558280836330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.914+0000 7f17b9200640 1 -- 172.21.15.83:0/3131886451 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582808347e0 con 0x558280834090 2024-04-23T14:51:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.914+0000 7f17b9c00640 1 --2- 172.21.15.83:0/3131886451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x558280856db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:51:23.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.914+0000 7f17b9200640 1 --2- 172.21.15.83:0/3131886451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x55828084ad30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f17a8004010 tx=0x7f17a80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e4ddc8e7a8bdd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:23.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.915+0000 7f17a5600640 1 -- 172.21.15.83:0/3131886451 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a803d070 con 0x558280834090 2024-04-23T14:51:23.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.915+0000 7f17a5600640 1 -- 172.21.15.83:0/3131886451 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a8009600 con 0x558280834090 2024-04-23T14:51:23.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.915+0000 7f17a5600640 1 -- 172.21.15.83:0/3131886451 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8028040 con 0x558280834090 2024-04-23T14:51:23.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.916+0000 7f17be9ce740 1 -- 172.21.15.83:0/3131886451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 msgr2=0x55828084ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:23.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.916+0000 7f17be9ce740 1 --2- 172.21.15.83:0/3131886451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x55828084ad30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f17a8004010 tx=0x7f17a80117b0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.916+0000 7f17be9ce740 1 -- 172.21.15.83:0/3131886451 shutdown_connections 2024-04-23T14:51:23.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.916+0000 7f17be9ce740 1 --2- 172.21.15.83:0/3131886451 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x558280856db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.916+0000 7f17be9ce740 1 --2- 172.21.15.83:0/3131886451 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582808349b0 0x558280836330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.916+0000 7f17be9ce740 1 --2- 172.21.15.83:0/3131886451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558280834090 0x55828084ad30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.916+0000 7f17be9ce740 1 -- 172.21.15.83:0/3131886451 >> 172.21.15.83:0/3131886451 conn(0x5582808303b0 msgr2=0x55828082f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:23.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.916+0000 7f17be9ce740 1 -- 172.21.15.83:0/3131886451 shutdown_connections 2024-04-23T14:51:23.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.916+0000 7f17be9ce740 1 -- 172.21.15.83:0/3131886451 wait complete. 2024-04-23T14:51:23.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.917+0000 7f17be9ce740 1 -- start start 2024-04-23T14:51:23.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.917+0000 7f17be9ce740 1 Processor -- start 2024-04-23T14:51:23.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.918+0000 7f17be9ce740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558280834090 0x558280846240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.918+0000 7f17be9ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582808349b0 0x558280846780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.918+0000 7f17be9ce740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x558280846cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.918+0000 7f17be9ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41b2900 con 0x5582808349b0 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.918+0000 7f17be9ce740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41b3d70 con 0x558280834090 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.918+0000 7f17be9ce740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41b2fd0 con 0x55828083f080 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.918+0000 7f17be9ce740 10 client.0 _ll_register_callbacks cb 0x7f17a8009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.918+0000 7f17b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582808349b0 0x558280846780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.918+0000 7f17b3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582808349b0 0x558280846780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56854/0 (socket says 172.21.15.83:56854) 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.918+0000 7f17b3e00640 1 -- 172.21.15.83:0/1806616946 learned_addr learned my addr 172.21.15.83:0/1806616946 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.918+0000 7f17b9200640 1 --2- 172.21.15.83:0/1806616946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558280834090 0x558280846240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.919+0000 7f17b3e00640 1 -- 172.21.15.83:0/1806616946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 msgr2=0x558280846cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.919+0000 7f17b3e00640 1 --2- 172.21.15.83:0/1806616946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x558280846cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.919+0000 7f17b3e00640 1 -- 172.21.15.83:0/1806616946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558280834090 msgr2=0x558280846240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:23.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.919+0000 7f17b3e00640 1 --2- 172.21.15.83:0/1806616946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558280834090 0x558280846240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.919+0000 7f17b3e00640 1 -- 172.21.15.83:0/1806616946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558280847490 con 0x5582808349b0 2024-04-23T14:51:23.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.919+0000 7f17b3e00640 1 --2- 172.21.15.83:0/1806616946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582808349b0 0x558280846780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f17a0005fd0 tx=0x7f17a0005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da9ab08c5922037 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:23.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.919+0000 7f179aa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:51:23.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.920+0000 7f179aa00640 1 -- 172.21.15.83:0/1806616946 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a000fe30 con 0x5582808349b0 2024-04-23T14:51:23.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.920+0000 7f179aa00640 1 -- 172.21.15.83:0/1806616946 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f17a0024250 con 0x5582808349b0 2024-04-23T14:51:23.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.920+0000 7f17be9ce740 1 -- 172.21.15.83:0/1806616946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a8009600 con 0x5582808349b0 2024-04-23T14:51:23.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.920+0000 7f179aa00640 1 -- 172.21.15.83:0/1806616946 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17a0024450 con 0x5582808349b0 2024-04-23T14:51:23.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.920+0000 7f17be9ce740 10 client.15834 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:23.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.920+0000 7f17be9ce740 1 -- 172.21.15.83:0/1806616946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5582808c7070 con 0x5582808349b0 2024-04-23T14:51:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.920+0000 7f179aa00640 1 -- 172.21.15.83:0/1806616946 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a8009600 con 0x5582808349b0 2024-04-23T14:51:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.921+0000 7f179aa00640 1 -- 172.21.15.83:0/1806616946 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f17a002c3c0 con 0x5582808349b0 2024-04-23T14:51:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.921+0000 7f17be9ce740 10 client.15834 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.921+0000 7f17be9ce740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.921+0000 7f17be9ce740 20 client.15834 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.921+0000 7f17be9ce740 1 --2- 172.21.15.83:0/1806616946 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5582808c78d0 0x5582808c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.921+0000 7f17be9ce740 4 client.15834 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:51:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.921+0000 7f17be9ce740 1 -- 172.21.15.83:0/1806616946 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5582807a1700 con 0x5582808c78d0 2024-04-23T14:51:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.921+0000 7f17b9200640 1 --2- 172.21.15.83:0/1806616946 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5582808c78d0 0x5582808c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.922+0000 7f17b9200640 1 --2- 172.21.15.83:0/1806616946 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5582808c78d0 0x5582808c9cf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f1721f39310708c server_cookie=616519576332844b in_seq=0 out_seq=0 2024-04-23T14:51:23.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.922+0000 7f179aa00640 10 client.15834 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.923+0000 7f179aa00640 1 -- 172.21.15.83:0/1806616946 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5582807a1700 con 0x5582808c78d0 2024-04-23T14:51:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.923+0000 7f179aa00640 10 client.15834 handle_command_reply: tid=0 2024-04-23T14:51:23.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.923+0000 7f17be9ce740 1 client.15834 shutdown 2024-04-23T14:51:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.923+0000 7f17be9ce740 1 -- 172.21.15.83:0/1806616946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582808349b0 msgr2=0x558280846780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.923+0000 7f17be9ce740 1 --2- 172.21.15.83:0/1806616946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582808349b0 0x558280846780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f17a0005fd0 tx=0x7f17a0005b80 comp rx=0 tx=0).stop 2024-04-23T14:51:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.923+0000 7f17be9ce740 1 -- 172.21.15.83:0/1806616946 shutdown_connections 2024-04-23T14:51:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.923+0000 7f17be9ce740 1 --2- 172.21.15.83:0/1806616946 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5582808c78d0 0x5582808c9cf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.923+0000 7f17be9ce740 1 --2- 172.21.15.83:0/1806616946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55828083f080 0x558280846cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.923+0000 7f17be9ce740 1 --2- 172.21.15.83:0/1806616946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582808349b0 0x558280846780 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.923+0000 7f17be9ce740 1 --2- 172.21.15.83:0/1806616946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558280834090 0x558280846240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.923+0000 7f179aa00640 0 client.15834 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.924+0000 7f17be9ce740 1 -- 172.21.15.83:0/1806616946 >> 172.21.15.83:0/1806616946 conn(0x5582808303b0 msgr2=0x55828083bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:23.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.924+0000 7f17be9ce740 1 -- 172.21.15.83:0/1806616946 shutdown_connections 2024-04-23T14:51:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.924+0000 7f17be9ce740 1 -- 172.21.15.83:0/1806616946 wait complete. 2024-04-23T14:51:23.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.924+0000 7f17be9ce740 20 client.15834 trim_cache size 0 max 16384 2024-04-23T14:51:23.927 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:51:23.927 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:51:23.927 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "78716592-708e-4c0a-b6a7-532f486118b6", 2024-04-23T14:51:23.927 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:51:23.927 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:51:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.926+0000 7f17bbc00640 1 -- 172.21.15.83:0/8473069 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f179c0a3e20 msgr2=0x7f179c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.926+0000 7f17bbc00640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f179c0a3e20 0x7f179c0a62e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x558280831540 tx=0x7f17a0005d40 comp rx=0 tx=0).stop 2024-04-23T14:51:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.926+0000 7f17bbc00640 1 -- 172.21.15.83:0/8473069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b41063f0 msgr2=0x7f17b41a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.926+0000 7f17bbc00640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b41063f0 0x7f17b41a36a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009b30 tx=0x7f17ac0305d0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.926+0000 7f17bbc00640 1 -- 172.21.15.83:0/8473069 shutdown_connections 2024-04-23T14:51:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.926+0000 7f17bbc00640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f179c0a3e20 0x7f179c0a62e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.926+0000 7f17bbc00640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17b41063f0 0x7f17b41a36a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.926+0000 7f17bbc00640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17b4101810 0x7f17b419f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.926+0000 7f17bbc00640 1 --2- 172.21.15.83:0/8473069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17b4100ef0 0x7f17b419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:23.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.926+0000 7f17bbc00640 1 -- 172.21.15.83:0/8473069 >> 172.21.15.83:0/8473069 conn(0x7f17b4078060 msgr2=0x7f17b4103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:23.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.927+0000 7f17bbc00640 1 -- 172.21.15.83:0/8473069 shutdown_connections 2024-04-23T14:51:23.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:23.927+0000 7f17bbc00640 1 -- 172.21.15.83:0/8473069 wait complete. 2024-04-23T14:51:23.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:51:23.951 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:51:24.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.206+0000 7fa276800640 1 -- 172.21.15.83:0/1239858910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa270109540 msgr2=0x7fa2701099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:24.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.206+0000 7fa276800640 1 --2- 172.21.15.83:0/1239858910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa270109540 0x7fa2701099a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009900 tx=0x7fa25c02a420 comp rx=0 tx=0).stop 2024-04-23T14:51:24.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.207+0000 7fa276800640 1 -- 172.21.15.83:0/1239858910 shutdown_connections 2024-04-23T14:51:24.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.207+0000 7fa276800640 1 --2- 172.21.15.83:0/1239858910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa270117000 0x7fa2701193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.207+0000 7fa276800640 1 --2- 172.21.15.83:0/1239858910 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa270109540 0x7fa2701099a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.207+0000 7fa276800640 1 --2- 172.21.15.83:0/1239858910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27010c5b0 0x7fa270109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.207+0000 7fa276800640 1 -- 172.21.15.83:0/1239858910 >> 172.21.15.83:0/1239858910 conn(0x7fa27006cf80 msgr2=0x7fa27006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:24.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.208+0000 7fa276800640 1 -- 172.21.15.83:0/1239858910 shutdown_connections 2024-04-23T14:51:24.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.208+0000 7fa276800640 1 -- 172.21.15.83:0/1239858910 wait complete. 2024-04-23T14:51:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.209+0000 7fa276800640 1 Processor -- start 2024-04-23T14:51:24.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.209+0000 7fa276800640 1 -- start start 2024-04-23T14:51:24.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.210+0000 7fa276800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa270109540 0x7fa2701a3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.210+0000 7fa276800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27010c5b0 0x7fa2701a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.210+0000 7fa276800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa270117000 0x7fa2701a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.210+0000 7fa276800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa27011cd40 con 0x7fa270117000 2024-04-23T14:51:24.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.210+0000 7fa276800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa27011cbc0 con 0x7fa27010c5b0 2024-04-23T14:51:24.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.210+0000 7fa276800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa27011cec0 con 0x7fa270109540 2024-04-23T14:51:24.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.210+0000 7fa274a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27010c5b0 0x7fa2701a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.210+0000 7fa274a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27010c5b0 0x7fa2701a3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38064/0 (socket says 172.21.15.83:38064) 2024-04-23T14:51:24.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.210+0000 7fa274a00640 1 -- 172.21.15.83:0/3459337362 learned_addr learned my addr 172.21.15.83:0/3459337362 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:24.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.210+0000 7fa275400640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa270109540 0x7fa2701a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.211+0000 7fa274a00640 1 -- 172.21.15.83:0/3459337362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa270109540 msgr2=0x7fa2701a3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:24.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.211+0000 7fa274a00640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa270109540 0x7fa2701a3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.211+0000 7fa274a00640 1 -- 172.21.15.83:0/3459337362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa270117000 msgr2=0x7fa2701a7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:24.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.211+0000 7fa274a00640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa270117000 0x7fa2701a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.211+0000 7fa274a00640 1 -- 172.21.15.83:0/3459337362 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2701a8230 con 0x7fa27010c5b0 2024-04-23T14:51:24.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.211+0000 7fa274a00640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27010c5b0 0x7fa2701a3780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa25c02aa10 tx=0x7fa25c005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:24.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.212+0000 7fa26e000640 1 -- 172.21.15.83:0/3459337362 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c040030 con 0x7fa27010c5b0 2024-04-23T14:51:24.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.212+0000 7fa26e000640 1 -- 172.21.15.83:0/3459337362 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa25c003e30 con 0x7fa27010c5b0 2024-04-23T14:51:24.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.212+0000 7fa26e000640 1 -- 172.21.15.83:0/3459337362 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c03ca70 con 0x7fa27010c5b0 2024-04-23T14:51:24.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.212+0000 7fa276800640 1 -- 172.21.15.83:0/3459337362 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2701a84c0 con 0x7fa27010c5b0 2024-04-23T14:51:24.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.212+0000 7fa276800640 1 -- 172.21.15.83:0/3459337362 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2701afdb0 con 0x7fa27010c5b0 2024-04-23T14:51:24.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.216+0000 7fa27ad69740 1 Processor -- start 2024-04-23T14:51:24.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.217+0000 7fa27ad69740 1 -- start start 2024-04-23T14:51:24.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.217+0000 7fa27ad69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69240 0x562d41d69620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.217+0000 7fa27ad69740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69b60 0x562d41d73cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.217+0000 7fa27ad69740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d76620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.217+0000 7fa27ad69740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562d41cc61f0 con 0x562d41d69b60 2024-04-23T14:51:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.217+0000 7fa27ad69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562d41c6c350 con 0x562d41d69240 2024-04-23T14:51:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.217+0000 7fa27ad69740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562d41cc21b0 con 0x562d41d74230 2024-04-23T14:51:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.218+0000 7fa275e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69240 0x562d41d69620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.218+0000 7fa275e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69240 0x562d41d69620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38080/0 (socket says 172.21.15.83:38080) 2024-04-23T14:51:24.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.218+0000 7fa275e00640 1 -- 172.21.15.83:0/266634034 learned_addr learned my addr 172.21.15.83:0/266634034 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:24.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.218+0000 7fa26e000640 1 -- 172.21.15.83:0/3459337362 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa25c043030 con 0x7fa27010c5b0 2024-04-23T14:51:24.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.220+0000 7fa274a00640 1 --2- 172.21.15.83:0/266634034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d76620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.220+0000 7fa275e00640 1 -- 172.21.15.83:0/266634034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 msgr2=0x562d41d76620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:24.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.220+0000 7fa275e00640 1 --2- 172.21.15.83:0/266634034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d76620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.220+0000 7fa275e00640 1 -- 172.21.15.83:0/266634034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69b60 msgr2=0x562d41d73cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:24.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.220+0000 7fa275e00640 1 --2- 172.21.15.83:0/266634034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69b60 0x562d41d73cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.220+0000 7fa275e00640 1 -- 172.21.15.83:0/266634034 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d41cc0f50 con 0x562d41d69240 2024-04-23T14:51:24.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.220+0000 7fa275e00640 1 --2- 172.21.15.83:0/266634034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69240 0x562d41d69620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa268001fd0 tx=0x7fa26800c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12d23d268610b596 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:24.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.220+0000 7fa26e000640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa2580a3e60 0x7fa2580a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.220+0000 7fa262600640 1 -- 172.21.15.83:0/266634034 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268018070 con 0x562d41d69240 2024-04-23T14:51:24.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.221+0000 7fa26e000640 1 -- 172.21.15.83:0/3459337362 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa25c0f11a0 con 0x7fa27010c5b0 2024-04-23T14:51:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.221+0000 7fa262600640 1 -- 172.21.15.83:0/266634034 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa268003e50 con 0x562d41d69240 2024-04-23T14:51:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.221+0000 7fa262600640 1 -- 172.21.15.83:0/266634034 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268018070 con 0x562d41d69240 2024-04-23T14:51:24.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.221+0000 7fa275400640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa2580a3e60 0x7fa2580a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.223+0000 7fa275400640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa2580a3e60 0x7fa2580a6320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x562d41d6a800 tx=0x7fa264005f30 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.223+0000 7fa27ad69740 1 -- 172.21.15.83:0/266634034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69240 msgr2=0x562d41d69620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:24.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.223+0000 7fa27ad69740 1 --2- 172.21.15.83:0/266634034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69240 0x562d41d69620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa268001fd0 tx=0x7fa26800c4b0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.224+0000 7fa27ad69740 1 -- 172.21.15.83:0/266634034 shutdown_connections 2024-04-23T14:51:24.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.224+0000 7fa27ad69740 1 --2- 172.21.15.83:0/266634034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d76620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.224+0000 7fa27ad69740 1 --2- 172.21.15.83:0/266634034 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69b60 0x562d41d73cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.224+0000 7fa27ad69740 1 --2- 172.21.15.83:0/266634034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69240 0x562d41d69620 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.224+0000 7fa27ad69740 1 -- 172.21.15.83:0/266634034 >> 172.21.15.83:0/266634034 conn(0x562d41d65560 msgr2=0x562d41d65940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:24.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.225+0000 7fa27ad69740 1 -- 172.21.15.83:0/266634034 shutdown_connections 2024-04-23T14:51:24.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.225+0000 7fa27ad69740 1 -- 172.21.15.83:0/266634034 wait complete. 2024-04-23T14:51:24.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.226+0000 7fa27ad69740 1 -- start start 2024-04-23T14:51:24.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.226+0000 7fa27ad69740 1 Processor -- start 2024-04-23T14:51:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.227+0000 7fa27ad69740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69240 0x562d41d71250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.227+0000 7fa27ad69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69b60 0x562d41d71790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.227+0000 7fa27ad69740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d71cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.227+0000 7fa27ad69740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562d41d77020 con 0x562d41d69240 2024-04-23T14:51:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.227+0000 7fa27ad69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562d41c7f690 con 0x562d41d69b60 2024-04-23T14:51:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.227+0000 7fa27ad69740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562d41d771a0 con 0x562d41d74230 2024-04-23T14:51:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.227+0000 7fa27ad69740 10 client.0 _ll_register_callbacks cb 0x562d41cbb650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.228+0000 7fa274a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69b60 0x562d41d71790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.228+0000 7fa275e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d71cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.228+0000 7fa275e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d71cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:47690/0 (socket says 172.21.15.83:47690) 2024-04-23T14:51:24.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.228+0000 7fa275e00640 1 -- 172.21.15.83:0/3511349469 learned_addr learned my addr 172.21.15.83:0/3511349469 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:24.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.229+0000 7fa275e00640 1 -- 172.21.15.83:0/3511349469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69b60 msgr2=0x562d41d71790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:24.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.229+0000 7fa275e00640 1 --2- 172.21.15.83:0/3511349469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69b60 0x562d41d71790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.229+0000 7fa275e00640 1 -- 172.21.15.83:0/3511349469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69240 msgr2=0x562d41d71250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:24.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.229+0000 7fa275e00640 1 --2- 172.21.15.83:0/3511349469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69240 0x562d41d71250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.229+0000 7fa275e00640 1 -- 172.21.15.83:0/3511349469 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d41c7e920 con 0x562d41d74230 2024-04-23T14:51:24.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.229+0000 7fa275e00640 1 --2- 172.21.15.83:0/3511349469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d71cd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa268000f80 tx=0x7fa268004230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f35bfbd76a79de8e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:24.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.229+0000 7fa256a00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:51:24.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.230+0000 7fa256a00640 1 -- 172.21.15.83:0/3511349469 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268021070 con 0x562d41d74230 2024-04-23T14:51:24.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.230+0000 7fa256a00640 1 -- 172.21.15.83:0/3511349469 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa268003970 con 0x562d41d74230 2024-04-23T14:51:24.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.230+0000 7fa27ad69740 1 -- 172.21.15.83:0/3511349469 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa268003e50 con 0x562d41d74230 2024-04-23T14:51:24.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.231+0000 7fa256a00640 1 -- 172.21.15.83:0/3511349469 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa26802d050 con 0x562d41d74230 2024-04-23T14:51:24.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.231+0000 7fa27ad69740 10 client.25960 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:24.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.231+0000 7fa27ad69740 1 -- 172.21.15.83:0/3511349469 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d41aa27d0 con 0x562d41d74230 2024-04-23T14:51:24.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.232+0000 7fa256a00640 1 -- 172.21.15.83:0/3511349469 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa268003e50 con 0x562d41d74230 2024-04-23T14:51:24.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.232+0000 7fa256a00640 1 -- 172.21.15.83:0/3511349469 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fa268013c20 con 0x562d41d74230 2024-04-23T14:51:24.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.232+0000 7fa27ad69740 10 client.25960 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:24.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.232+0000 7fa27ad69740 10 client.25960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:24.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.233+0000 7fa27ad69740 20 client.25960 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:24.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.233+0000 7fa27ad69740 1 --2- 172.21.15.83:0/3511349469 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562d41d84660 0x562d41d86a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.233+0000 7fa27ad69740 4 client.25960 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:51:24.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.233+0000 7fa27ad69740 1 -- 172.21.15.83:0/3511349469 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d41cd3c10 con 0x562d41d84660 2024-04-23T14:51:24.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.234+0000 7fa275400640 1 --2- 172.21.15.83:0/3511349469 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562d41d84660 0x562d41d86a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.236+0000 7fa275400640 1 --2- 172.21.15.83:0/3511349469 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562d41d84660 0x562d41d86a50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abafd27b198a85da server_cookie=f44cb3e8de400914 in_seq=0 out_seq=0 2024-04-23T14:51:24.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.236+0000 7fa256a00640 10 client.25960 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:24.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.247+0000 7fa256a00640 1 -- 172.21.15.83:0/3511349469 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d41cd3c10 con 0x562d41d84660 2024-04-23T14:51:24.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.247+0000 7fa256a00640 10 client.25960 handle_command_reply: tid=0 2024-04-23T14:51:24.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.247+0000 7fa27ad69740 1 client.25960 shutdown 2024-04-23T14:51:24.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.249+0000 7fa27ad69740 1 -- 172.21.15.83:0/3511349469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 msgr2=0x562d41d71cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:24.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.249+0000 7fa27ad69740 1 --2- 172.21.15.83:0/3511349469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d71cd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa268000f80 tx=0x7fa268004230 comp rx=0 tx=0).stop 2024-04-23T14:51:24.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.249+0000 7fa27ad69740 1 -- 172.21.15.83:0/3511349469 shutdown_connections 2024-04-23T14:51:24.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.249+0000 7fa27ad69740 1 --2- 172.21.15.83:0/3511349469 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562d41d84660 0x562d41d86a50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.249+0000 7fa27ad69740 1 --2- 172.21.15.83:0/3511349469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d71cd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.249+0000 7fa27ad69740 1 --2- 172.21.15.83:0/3511349469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69b60 0x562d41d71790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.249+0000 7fa27ad69740 1 --2- 172.21.15.83:0/3511349469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69240 0x562d41d71250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.249+0000 7fa256a00640 0 client.25960 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:24.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.250+0000 7fa27ad69740 1 -- 172.21.15.83:0/3511349469 >> 172.21.15.83:0/3511349469 conn(0x562d41d65560 msgr2=0x562d41d5bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:24.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.250+0000 7fa27ad69740 1 -- 172.21.15.83:0/3511349469 shutdown_connections 2024-04-23T14:51:24.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.250+0000 7fa27ad69740 1 -- 172.21.15.83:0/3511349469 wait complete. 2024-04-23T14:51:24.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.250+0000 7fa27ad69740 20 client.25960 trim_cache size 0 max 16384 2024-04-23T14:51:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.296+0000 7fa27ad69740 1 Processor -- start 2024-04-23T14:51:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.296+0000 7fa27ad69740 1 -- start start 2024-04-23T14:51:24.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.296+0000 7fa27ad69740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d69240 0x562d41d7ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.296+0000 7fa27ad69740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69b60 0x562d41d71af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.296+0000 7fa27ad69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d74230 0x562d41d8c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.296+0000 7fa27ad69740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701b6c10 con 0x562d41d69b60 2024-04-23T14:51:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.296+0000 7fa27ad69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701b44b0 con 0x562d41d74230 2024-04-23T14:51:24.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.296+0000 7fa27ad69740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701b2320 con 0x562d41d69240 2024-04-23T14:51:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.297+0000 7fa275400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d69240 0x562d41d7ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.297+0000 7fa275400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d69240 0x562d41d7ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:47706/0 (socket says 172.21.15.83:47706) 2024-04-23T14:51:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.297+0000 7fa275400640 1 -- 172.21.15.83:0/1987881436 learned_addr learned my addr 172.21.15.83:0/1987881436 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:24.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.298+0000 7fa275e00640 1 --2- 172.21.15.83:0/1987881436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d74230 0x562d41d8c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.298+0000 7fa274a00640 1 --2- 172.21.15.83:0/1987881436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69b60 0x562d41d71af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.299+0000 7fa275e00640 1 -- 172.21.15.83:0/1987881436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d69240 msgr2=0x562d41d7ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.299+0000 7fa275e00640 1 --2- 172.21.15.83:0/1987881436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d69240 0x562d41d7ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.299+0000 7fa275e00640 1 -- 172.21.15.83:0/1987881436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69b60 msgr2=0x562d41d71af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:24.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.299+0000 7fa275e00640 1 --2- 172.21.15.83:0/1987881436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69b60 0x562d41d71af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.300+0000 7fa275e00640 1 -- 172.21.15.83:0/1987881436 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d419d83f0 con 0x562d41d74230 2024-04-23T14:51:24.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.301+0000 7fa275e00640 1 --2- 172.21.15.83:0/1987881436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d74230 0x562d41d8c190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa26801d3e0 tx=0x7fa268004450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cce1e038cf17ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:24.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.303+0000 7fa261c00640 1 -- 172.21.15.83:0/1987881436 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268021070 con 0x562d41d74230 2024-04-23T14:51:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.303+0000 7fa261c00640 1 -- 172.21.15.83:0/1987881436 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26800b070 con 0x562d41d74230 2024-04-23T14:51:24.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.303+0000 7fa261c00640 1 -- 172.21.15.83:0/1987881436 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268018040 con 0x562d41d74230 2024-04-23T14:51:24.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.304+0000 7fa27ad69740 1 -- 172.21.15.83:0/1987881436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d74230 msgr2=0x562d41d8c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:24.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.304+0000 7fa27ad69740 1 --2- 172.21.15.83:0/1987881436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d74230 0x562d41d8c190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa26801d3e0 tx=0x7fa268004450 comp rx=0 tx=0).stop 2024-04-23T14:51:24.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.305+0000 7fa27ad69740 1 -- 172.21.15.83:0/1987881436 shutdown_connections 2024-04-23T14:51:24.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.305+0000 7fa27ad69740 1 --2- 172.21.15.83:0/1987881436 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d74230 0x562d41d8c190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.305+0000 7fa27ad69740 1 --2- 172.21.15.83:0/1987881436 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69b60 0x562d41d71af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.305+0000 7fa27ad69740 1 --2- 172.21.15.83:0/1987881436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d69240 0x562d41d7ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.305+0000 7fa27ad69740 1 -- 172.21.15.83:0/1987881436 >> 172.21.15.83:0/1987881436 conn(0x562d41d65560 msgr2=0x562d41d5bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.305+0000 7fa27ad69740 1 -- 172.21.15.83:0/1987881436 shutdown_connections 2024-04-23T14:51:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.305+0000 7fa27ad69740 1 -- 172.21.15.83:0/1987881436 wait complete. 2024-04-23T14:51:24.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.306+0000 7fa27ad69740 1 -- start start 2024-04-23T14:51:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.306+0000 7fa27ad69740 1 Processor -- start 2024-04-23T14:51:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.307+0000 7fa27ad69740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69240 0x562d41d7c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.307+0000 7fa27ad69740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69b60 0x562d41d7c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.307+0000 7fa27ad69740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d6d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.307+0000 7fa27ad69740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701b2970 con 0x562d41d69240 2024-04-23T14:51:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.307+0000 7fa27ad69740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701b3de0 con 0x562d41d69b60 2024-04-23T14:51:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.307+0000 7fa27ad69740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2701b3040 con 0x562d41d74230 2024-04-23T14:51:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.307+0000 7fa27ad69740 10 client.0 _ll_register_callbacks cb 0x7fa268016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:24.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.307+0000 7fa275400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69240 0x562d41d7c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.307+0000 7fa275400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69240 0x562d41d7c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56894/0 (socket says 172.21.15.83:56894) 2024-04-23T14:51:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.307+0000 7fa275400640 1 -- 172.21.15.83:0/1433422482 learned_addr learned my addr 172.21.15.83:0/1433422482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:24.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.308+0000 7fa274a00640 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69b60 0x562d41d7c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.308+0000 7fa275400640 1 -- 172.21.15.83:0/1433422482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 msgr2=0x562d41d6d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.308+0000 7fa275400640 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d6d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.308+0000 7fa275400640 1 -- 172.21.15.83:0/1433422482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69b60 msgr2=0x562d41d7c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.308+0000 7fa275400640 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69b60 0x562d41d7c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.308+0000 7fa275400640 1 -- 172.21.15.83:0/1433422482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d419980f0 con 0x562d41d69240 2024-04-23T14:51:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.308+0000 7fa274a00640 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69b60 0x562d41d7c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:51:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.310+0000 7fa275400640 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69240 0x562d41d7c420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa26400fdf0 tx=0x7fa264005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d6b8d23ca89e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:24.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.310+0000 7fa256a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:51:24.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.311+0000 7fa256a00640 1 -- 172.21.15.83:0/1433422482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26402e480 con 0x562d41d69240 2024-04-23T14:51:24.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.311+0000 7fa256a00640 1 -- 172.21.15.83:0/1433422482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa26402e620 con 0x562d41d69240 2024-04-23T14:51:24.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.311+0000 7fa27ad69740 1 -- 172.21.15.83:0/1433422482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa26800b070 con 0x562d41d69240 2024-04-23T14:51:24.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.311+0000 7fa256a00640 1 -- 172.21.15.83:0/1433422482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa26402e820 con 0x562d41d69240 2024-04-23T14:51:24.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.313+0000 7fa27ad69740 10 client.15840 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:24.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.313+0000 7fa27ad69740 1 -- 172.21.15.83:0/1433422482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d41c8b810 con 0x562d41d69240 2024-04-23T14:51:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.313+0000 7fa256a00640 1 -- 172.21.15.83:0/1433422482 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26800b070 con 0x562d41d69240 2024-04-23T14:51:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.313+0000 7fa256a00640 1 -- 172.21.15.83:0/1433422482 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fa264038600 con 0x562d41d69240 2024-04-23T14:51:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.313+0000 7fa27ad69740 10 client.15840 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.313+0000 7fa27ad69740 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.313+0000 7fa27ad69740 20 client.15840 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.313+0000 7fa27ad69740 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562d41dea810 0x562d41decc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.313+0000 7fa27ad69740 4 client.15840 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:51:24.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.313+0000 7fa27ad69740 1 -- 172.21.15.83:0/1433422482 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562d41ccda80 con 0x562d41dea810 2024-04-23T14:51:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.313+0000 7fa274a00640 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562d41dea810 0x562d41decc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:24.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.314+0000 7fa256a00640 10 client.15840 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.315+0000 7fa274a00640 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562d41dea810 0x562d41decc70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7de4460a33d567ea server_cookie=65fe59aa23699a01 in_seq=0 out_seq=0 2024-04-23T14:51:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.315+0000 7fa256a00640 1 -- 172.21.15.83:0/1433422482 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562d41ccda80 con 0x562d41dea810 2024-04-23T14:51:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.315+0000 7fa256a00640 10 client.15840 handle_command_reply: tid=0 2024-04-23T14:51:24.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.315+0000 7fa27ad69740 1 client.15840 shutdown 2024-04-23T14:51:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.316+0000 7fa27ad69740 1 -- 172.21.15.83:0/1433422482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69240 msgr2=0x562d41d7c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.316+0000 7fa27ad69740 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69240 0x562d41d7c420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa26400fdf0 tx=0x7fa264005850 comp rx=0 tx=0).stop 2024-04-23T14:51:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.316+0000 7fa27ad69740 1 -- 172.21.15.83:0/1433422482 shutdown_connections 2024-04-23T14:51:24.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.316+0000 7fa27ad69740 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562d41dea810 0x562d41decc70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.316+0000 7fa27ad69740 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562d41d74230 0x562d41d6d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.316+0000 7fa27ad69740 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562d41d69b60 0x562d41d7c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.316+0000 7fa27ad69740 1 --2- 172.21.15.83:0/1433422482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d41d69240 0x562d41d7c420 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.316+0000 7fa27ad69740 1 -- 172.21.15.83:0/1433422482 >> 172.21.15.83:0/1433422482 conn(0x562d41d65560 msgr2=0x562d41d6a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.316+0000 7fa256a00640 0 client.15840 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.316+0000 7fa27ad69740 1 -- 172.21.15.83:0/1433422482 shutdown_connections 2024-04-23T14:51:24.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.316+0000 7fa27ad69740 1 -- 172.21.15.83:0/1433422482 wait complete. 2024-04-23T14:51:24.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa27ad69740 20 client.15840 trim_cache size 0 max 16384 2024-04-23T14:51:24.319 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:51:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:51:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:51:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "78716592-708e-4c0a-b6a7-532f486118b6": { 2024-04-23T14:51:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:51:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "78716592-708e-4c0a-b6a7-532f486118b6", 2024-04-23T14:51:24.319 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:51:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:51:24.320 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:51:24.320 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:51:24.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa276800640 1 -- 172.21.15.83:0/3459337362 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa2580a3e60 msgr2=0x7fa2580a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:24.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa276800640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa2580a3e60 0x7fa2580a6320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x562d41d6a800 tx=0x7fa264005f30 comp rx=0 tx=0).stop 2024-04-23T14:51:24.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa276800640 1 -- 172.21.15.83:0/3459337362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27010c5b0 msgr2=0x7fa2701a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa276800640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27010c5b0 0x7fa2701a3780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa25c02aa10 tx=0x7fa25c005230 comp rx=0 tx=0).stop 2024-04-23T14:51:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa276800640 1 -- 172.21.15.83:0/3459337362 shutdown_connections 2024-04-23T14:51:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa276800640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa2580a3e60 0x7fa2580a6320 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa276800640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa270117000 0x7fa2701a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa276800640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa27010c5b0 0x7fa2701a3780 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa276800640 1 --2- 172.21.15.83:0/3459337362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa270109540 0x7fa2701a3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa276800640 1 -- 172.21.15.83:0/3459337362 >> 172.21.15.83:0/3459337362 conn(0x7fa27006cf80 msgr2=0x7fa2701173e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa276800640 1 -- 172.21.15.83:0/3459337362 shutdown_connections 2024-04-23T14:51:24.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:24.318+0000 7fa276800640 1 -- 172.21.15.83:0/3459337362 wait complete. 2024-04-23T14:51:24.340 INFO:tasks.cephfs.filesystem:scrub status for tag:78716592-708e-4c0a-b6a7-532f486118b6 - {'path': '/', 'tag': '78716592-708e-4c0a-b6a7-532f486118b6', 'options': 'recursive,force'} 2024-04-23T14:51:25.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: pgmap v91: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 267 MiB/s wr, 94 op/s 2024-04-23T14:51:25.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:51:25.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:51:25.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:51:25.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:51:25.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:51:25.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:51:25.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:24 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: pgmap v91: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 267 MiB/s wr, 94 op/s 2024-04-23T14:51:25.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:51:25.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:51:25.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:51:25.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:51:25.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:51:25.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:51:25.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:24 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: pgmap v91: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 267 MiB/s wr, 94 op/s 2024-04-23T14:51:25.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:51:25.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:51:25.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:51:25.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:51:25.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:51:25.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:51:25.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:25.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:24 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:26.333 INFO:tasks.workunit.client.0.smithi083.stdout:Syncing()...7 sec 2024-04-23T14:51:26.336 INFO:tasks.workunit.client.0.smithi083.stdout:Starting Actual Benchmark At: Tue Apr 23 14:51:26 2024 2024-04-23T14:51:26.336 INFO:tasks.workunit.client.0.smithi083.stdout: 2024-04-23T14:51:27.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:26 smithi083 ceph-mon[27867]: pgmap v92: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 235 MiB/s wr, 78 op/s 2024-04-23T14:51:27.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:26 smithi173 ceph-mon[32443]: pgmap v92: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 235 MiB/s wr, 78 op/s 2024-04-23T14:51:27.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:26 smithi203 ceph-mon[32800]: pgmap v92: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 235 MiB/s wr, 78 op/s 2024-04-23T14:51:29.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:28 smithi083 ceph-mon[27867]: pgmap v93: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 235 MiB/s wr, 78 op/s 2024-04-23T14:51:29.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:28 smithi173 ceph-mon[32443]: pgmap v93: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 235 MiB/s wr, 78 op/s 2024-04-23T14:51:29.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:28 smithi203 ceph-mon[32800]: pgmap v93: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 235 MiB/s wr, 78 op/s 2024-04-23T14:51:31.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:30 smithi083 ceph-mon[27867]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 326 MiB/s wr, 2.00k op/s 2024-04-23T14:51:31.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:30 smithi173 ceph-mon[32443]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 326 MiB/s wr, 2.00k op/s 2024-04-23T14:51:31.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:30 smithi203 ceph-mon[32800]: pgmap v94: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 326 MiB/s wr, 2.00k op/s 2024-04-23T14:51:33.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:33 smithi173 ceph-mon[32443]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 258 MiB/s wr, 2.13k op/s 2024-04-23T14:51:33.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:33 smithi203 ceph-mon[32800]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 258 MiB/s wr, 2.13k op/s 2024-04-23T14:51:33.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:33 smithi083 ceph-mon[27867]: pgmap v95: 97 pgs: 97 active+clean; 13 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 258 MiB/s wr, 2.13k op/s 2024-04-23T14:51:35.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:35 smithi173 ceph-mon[32443]: pgmap v96: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 272 MiB/s wr, 4.21k op/s 2024-04-23T14:51:35.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:35 smithi203 ceph-mon[32800]: pgmap v96: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 272 MiB/s wr, 4.21k op/s 2024-04-23T14:51:35.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:35 smithi083 ceph-mon[27867]: pgmap v96: 97 pgs: 97 active+clean; 13 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 272 MiB/s wr, 4.21k op/s 2024-04-23T14:51:37.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:37 smithi173 ceph-mon[32443]: pgmap v97: 97 pgs: 97 active+clean; 13 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 157 MiB/s wr, 10.82k op/s 2024-04-23T14:51:37.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:37 smithi203 ceph-mon[32800]: pgmap v97: 97 pgs: 97 active+clean; 13 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 157 MiB/s wr, 10.82k op/s 2024-04-23T14:51:37.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:37 smithi083 ceph-mon[27867]: pgmap v97: 97 pgs: 97 active+clean; 13 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 157 MiB/s wr, 10.82k op/s 2024-04-23T14:51:39.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:39 smithi203 ceph-mon[32800]: pgmap v98: 97 pgs: 97 active+clean; 13 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 157 MiB/s wr, 10.82k op/s 2024-04-23T14:51:39.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:39 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:51:39.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:39 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:51:39.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:39 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:51:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:39 smithi083 ceph-mon[27867]: pgmap v98: 97 pgs: 97 active+clean; 13 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 157 MiB/s wr, 10.82k op/s 2024-04-23T14:51:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:39 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:51:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:39 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:51:39.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:39 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:51:39.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:39 smithi173 ceph-mon[32443]: pgmap v98: 97 pgs: 97 active+clean; 13 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 157 MiB/s wr, 10.82k op/s 2024-04-23T14:51:39.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:39 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:51:39.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:39 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:51:39.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:39 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:51:41.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:41 smithi203 ceph-mon[32800]: pgmap v99: 97 pgs: 97 active+clean; 14 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 199 MiB/s wr, 16.66k op/s 2024-04-23T14:51:41.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:41 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:51:41.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:41 smithi083 ceph-mon[27867]: pgmap v99: 97 pgs: 97 active+clean; 14 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 199 MiB/s wr, 16.66k op/s 2024-04-23T14:51:41.509 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:41 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:51:41.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:41 smithi173 ceph-mon[32443]: pgmap v99: 97 pgs: 97 active+clean; 14 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 199 MiB/s wr, 16.66k op/s 2024-04-23T14:51:41.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:41 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:51:43.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:43 smithi203 ceph-mon[32800]: pgmap v100: 97 pgs: 97 active+clean; 14 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 130 MiB/s wr, 17.86k op/s 2024-04-23T14:51:43.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:43 smithi083 ceph-mon[27867]: pgmap v100: 97 pgs: 97 active+clean; 14 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 130 MiB/s wr, 17.86k op/s 2024-04-23T14:51:43.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:43 smithi173 ceph-mon[32443]: pgmap v100: 97 pgs: 97 active+clean; 14 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 130 MiB/s wr, 17.86k op/s 2024-04-23T14:51:44.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:44 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:51:44.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:44 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:51:44.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:44 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:51:44.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:44 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:51:44.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:44 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:51:44.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:44 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:51:44.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:44 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:51:44.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:44 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:51:44.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:44 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:51:45.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:45 smithi203 ceph-mon[32800]: pgmap v101: 97 pgs: 97 active+clean; 14 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 123 MiB/s wr, 17.96k op/s 2024-04-23T14:51:45.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:45 smithi083 ceph-mon[27867]: pgmap v101: 97 pgs: 97 active+clean; 14 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 123 MiB/s wr, 17.96k op/s 2024-04-23T14:51:45.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:45 smithi173 ceph-mon[32443]: pgmap v101: 97 pgs: 97 active+clean; 14 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 123 MiB/s wr, 17.96k op/s 2024-04-23T14:51:47.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:47 smithi203 ceph-mon[32800]: pgmap v102: 97 pgs: 97 active+clean; 14 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 170 MiB/s wr, 23.73k op/s 2024-04-23T14:51:47.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:47 smithi083 ceph-mon[27867]: pgmap v102: 97 pgs: 97 active+clean; 14 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 170 MiB/s wr, 23.73k op/s 2024-04-23T14:51:47.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:47 smithi173 ceph-mon[32443]: pgmap v102: 97 pgs: 97 active+clean; 14 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 170 MiB/s wr, 23.73k op/s 2024-04-23T14:51:49.627 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:49 smithi173 ceph-mon[32443]: pgmap v103: 97 pgs: 97 active+clean; 14 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 135 MiB/s wr, 18.14k op/s 2024-04-23T14:51:49.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:49 smithi203 ceph-mon[32800]: pgmap v103: 97 pgs: 97 active+clean; 14 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 135 MiB/s wr, 18.14k op/s 2024-04-23T14:51:49.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:49 smithi083 ceph-mon[27867]: pgmap v103: 97 pgs: 97 active+clean; 14 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 135 MiB/s wr, 18.14k op/s 2024-04-23T14:51:51.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:51 smithi203 ceph-mon[32800]: pgmap v104: 97 pgs: 97 active+clean; 15 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 205 MiB/s wr, 28.49k op/s 2024-04-23T14:51:51.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:51 smithi083 ceph-mon[27867]: pgmap v104: 97 pgs: 97 active+clean; 15 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 205 MiB/s wr, 28.49k op/s 2024-04-23T14:51:51.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:51 smithi173 ceph-mon[32443]: pgmap v104: 97 pgs: 97 active+clean; 15 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 205 MiB/s wr, 28.49k op/s 2024-04-23T14:51:53.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:53 smithi203 ceph-mon[32800]: pgmap v105: 97 pgs: 97 active+clean; 15 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 185 MiB/s wr, 25.63k op/s 2024-04-23T14:51:53.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:53 smithi083 ceph-mon[27867]: pgmap v105: 97 pgs: 97 active+clean; 15 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 185 MiB/s wr, 25.63k op/s 2024-04-23T14:51:53.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:53 smithi173 ceph-mon[32443]: pgmap v105: 97 pgs: 97 active+clean; 15 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 185 MiB/s wr, 25.63k op/s 2024-04-23T14:51:54.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:51:54.342 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:51:54.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.669+0000 7f58d6800640 1 -- 172.21.15.83:0/2265848023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58d0076e50 msgr2=0x7f58d01129a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:54.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.669+0000 7f58d6800640 1 --2- 172.21.15.83:0/2265848023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58d0076e50 0x7f58d01129a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f58c0009930 tx=0x7f58c002a440 comp rx=0 tx=0).stop 2024-04-23T14:51:54.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.669+0000 7f58d6800640 1 -- 172.21.15.83:0/2265848023 shutdown_connections 2024-04-23T14:51:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.669+0000 7f58d6800640 1 --2- 172.21.15.83:0/2265848023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58d0112ee0 0x7f58d01152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.669+0000 7f58d6800640 1 --2- 172.21.15.83:0/2265848023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58d0076e50 0x7f58d01129a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.669+0000 7f58d6800640 1 --2- 172.21.15.83:0/2265848023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58d0076530 0x7f58d0076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.669+0000 7f58d6800640 1 -- 172.21.15.83:0/2265848023 >> 172.21.15.83:0/2265848023 conn(0x7f58d00ffd90 msgr2=0x7f58d01021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.669+0000 7f58d6800640 1 -- 172.21.15.83:0/2265848023 shutdown_connections 2024-04-23T14:51:54.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.670+0000 7f58d6800640 1 -- 172.21.15.83:0/2265848023 wait complete. 2024-04-23T14:51:54.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.671+0000 7f58d6800640 1 Processor -- start 2024-04-23T14:51:54.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.672+0000 7f58d6800640 1 -- start start 2024-04-23T14:51:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.672+0000 7f58d6800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58d0076530 0x7f58d0073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.672+0000 7f58d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58d0076e50 0x7f58d006db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.672+0000 7f58d6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58d0112ee0 0x7f58d006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.672+0000 7f58d6800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58d006c470 con 0x7f58d0076e50 2024-04-23T14:51:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.672+0000 7f58d6800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f58d006c2f0 con 0x7f58d0112ee0 2024-04-23T14:51:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.672+0000 7f58d6800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58d006c5f0 con 0x7f58d0076530 2024-04-23T14:51:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.673+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58d0112ee0 0x7f58d006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.673+0000 7f58cea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58d0076e50 0x7f58d006db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.673+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58d0112ee0 0x7f58d006e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:52402/0 (socket says 172.21.15.83:52402) 2024-04-23T14:51:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.673+0000 7f58cfe00640 1 -- 172.21.15.83:0/1949573364 learned_addr learned my addr 172.21.15.83:0/1949573364 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.673+0000 7f58cfe00640 1 -- 172.21.15.83:0/1949573364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58d0076530 msgr2=0x7f58d0073a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-23T14:51:54.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.673+0000 7f58cfe00640 1 --2- 172.21.15.83:0/1949573364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58d0076530 0x7f58d0073a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.673+0000 7f58cfe00640 1 -- 172.21.15.83:0/1949573364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58d0076e50 msgr2=0x7f58d006db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.673+0000 7f58cfe00640 1 --2- 172.21.15.83:0/1949573364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58d0076e50 0x7f58d006db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.673+0000 7f58cfe00640 1 -- 172.21.15.83:0/1949573364 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d006e930 con 0x7f58d0112ee0 2024-04-23T14:51:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.673+0000 7f58cfe00640 1 --2- 172.21.15.83:0/1949573364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58d0112ee0 0x7f58d006e0c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58c800eb40 tx=0x7f58c800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.674+0000 7f58c7e00640 1 -- 172.21.15.83:0/1949573364 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8004b20 con 0x7f58d0112ee0 2024-04-23T14:51:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.674+0000 7f58c7e00640 1 -- 172.21.15.83:0/1949573364 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c8004cc0 con 0x7f58d0112ee0 2024-04-23T14:51:54.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.674+0000 7f58c7e00640 1 -- 172.21.15.83:0/1949573364 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8010850 con 0x7f58d0112ee0 2024-04-23T14:51:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.675+0000 7f58d6800640 1 -- 172.21.15.83:0/1949573364 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d01b3e70 con 0x7f58d0112ee0 2024-04-23T14:51:54.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.676+0000 7f58d6800640 1 -- 172.21.15.83:0/1949573364 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d01b42e0 con 0x7f58d0112ee0 2024-04-23T14:51:54.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.683+0000 7f58d9769740 1 Processor -- start 2024-04-23T14:51:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.688+0000 7f58c7e00640 1 -- 172.21.15.83:0/1949573364 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f58c8005000 con 0x7f58d0112ee0 2024-04-23T14:51:54.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.690+0000 7f58d9769740 1 -- start start 2024-04-23T14:51:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.691+0000 7f58c7e00640 1 --2- 172.21.15.83:0/1949573364 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f58b40a3e70 0x7f58b40a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.691+0000 7f58c7e00640 1 -- 172.21.15.83:0/1949573364 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f58c80d1300 con 0x7f58d0112ee0 2024-04-23T14:51:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.691+0000 7f58cea00640 1 --2- 172.21.15.83:0/1949573364 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f58b40a3e70 0x7f58b40a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.693+0000 7f58cea00640 1 --2- 172.21.15.83:0/1949573364 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f58b40a3e70 0x7f58b40a6330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f58c0006010 tx=0x7f58c0000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:54.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.694+0000 7f58d9769740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b3860 0x55963a0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.694+0000 7f58d9769740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0b4180 0x55963a0be310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.694+0000 7f58d9769740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 0x55963a0c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.694+0000 7f58d9769740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55963a0191f0 con 0x55963a0b3860 2024-04-23T14:51:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.694+0000 7f58d9769740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559639fbf350 con 0x55963a0be850 2024-04-23T14:51:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.694+0000 7f58d9769740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55963a0151b0 con 0x55963a0b4180 2024-04-23T14:51:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.695+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 0x55963a0c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.695+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 0x55963a0c0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:52412/0 (socket says 172.21.15.83:52412) 2024-04-23T14:51:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.695+0000 7f58cfe00640 1 -- 172.21.15.83:0/3235372266 learned_addr learned my addr 172.21.15.83:0/3235372266 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.695+0000 7f58cfe00640 1 -- 172.21.15.83:0/3235372266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0b4180 msgr2=0x55963a0be310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.695+0000 7f58cf400640 1 --2- 172.21.15.83:0/3235372266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b3860 0x55963a0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.695+0000 7f58cfe00640 1 --2- 172.21.15.83:0/3235372266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0b4180 0x55963a0be310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.695+0000 7f58cfe00640 1 -- 172.21.15.83:0/3235372266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b3860 msgr2=0x55963a0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.695+0000 7f58cfe00640 1 --2- 172.21.15.83:0/3235372266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b3860 0x55963a0b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.695+0000 7f58cfe00640 1 -- 172.21.15.83:0/3235372266 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55963a013f50 con 0x55963a0be850 2024-04-23T14:51:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.695+0000 7f58cf400640 1 --2- 172.21.15.83:0/3235372266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b3860 0x55963a0b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.695+0000 7f58cfe00640 1 --2- 172.21.15.83:0/3235372266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 0x55963a0c0c40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f58c8099c80 tx=0x7f58c809ceb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfe0f7c9d5d77486 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.696+0000 7f58c4c00640 1 -- 172.21.15.83:0/3235372266 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c80b5070 con 0x55963a0be850 2024-04-23T14:51:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.696+0000 7f58c4c00640 1 -- 172.21.15.83:0/3235372266 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c809d2b0 con 0x55963a0be850 2024-04-23T14:51:54.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.696+0000 7f58c4c00640 1 -- 172.21.15.83:0/3235372266 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8099920 con 0x55963a0be850 2024-04-23T14:51:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.698+0000 7f58d9769740 1 -- 172.21.15.83:0/3235372266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 msgr2=0x55963a0c0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:54.699 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.698+0000 7f58d9769740 1 --2- 172.21.15.83:0/3235372266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 0x55963a0c0c40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f58c8099c80 tx=0x7f58c809ceb0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.698+0000 7f58d9769740 1 -- 172.21.15.83:0/3235372266 shutdown_connections 2024-04-23T14:51:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.698+0000 7f58d9769740 1 --2- 172.21.15.83:0/3235372266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 0x55963a0c0c40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.698+0000 7f58d9769740 1 --2- 172.21.15.83:0/3235372266 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0b4180 0x55963a0be310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.698+0000 7f58d9769740 1 --2- 172.21.15.83:0/3235372266 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b3860 0x55963a0b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.698+0000 7f58d9769740 1 -- 172.21.15.83:0/3235372266 >> 172.21.15.83:0/3235372266 conn(0x55963a0afb80 msgr2=0x55963a0aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:54.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.699+0000 7f58d9769740 1 -- 172.21.15.83:0/3235372266 shutdown_connections 2024-04-23T14:51:54.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.700+0000 7f58d9769740 1 -- 172.21.15.83:0/3235372266 wait complete. 2024-04-23T14:51:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.700+0000 7f58d9769740 1 -- start start 2024-04-23T14:51:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.700+0000 7f58d9769740 1 Processor -- start 2024-04-23T14:51:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.701+0000 7f58d9769740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0b3860 0x55963a0bb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.701+0000 7f58d9769740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.701+0000 7f58d9769740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 0x55963a0bc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.701+0000 7f58d9769740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55963a0c1640 con 0x55963a0b4180 2024-04-23T14:51:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.701+0000 7f58d9769740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559639fd2690 con 0x55963a0be850 2024-04-23T14:51:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.701+0000 7f58d9769740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55963a0c17c0 con 0x55963a0b3860 2024-04-23T14:51:54.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.701+0000 7f58d9769740 10 client.0 _ll_register_callbacks cb 0x55963a00e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:54.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.702+0000 7f58cea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.702+0000 7f58cfe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 0x55963a0bc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.702+0000 7f58cea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0bbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52436/0 (socket says 172.21.15.83:52436) 2024-04-23T14:51:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.702+0000 7f58cea00640 1 -- 172.21.15.83:0/2968079226 learned_addr learned my addr 172.21.15.83:0/2968079226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.702+0000 7f58cf400640 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0b3860 0x55963a0bb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.702+0000 7f58cfe00640 1 -- 172.21.15.83:0/2968079226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0b3860 msgr2=0x55963a0bb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.703+0000 7f58cfe00640 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0b3860 0x55963a0bb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.703+0000 7f58cfe00640 1 -- 172.21.15.83:0/2968079226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 msgr2=0x55963a0bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.703+0000 7f58cfe00640 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0bbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.703+0000 7f58cfe00640 1 -- 172.21.15.83:0/2968079226 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559639fd1920 con 0x55963a0be850 2024-04-23T14:51:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.703+0000 7f58cea00640 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:51:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.703+0000 7f58cfe00640 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 0x55963a0bc370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f58c8099c80 tx=0x7f58c8099b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ea7131d5b160d4f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:54.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.703+0000 7f58b9600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:51:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.703+0000 7f58b9600640 1 -- 172.21.15.83:0/2968079226 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c80a7070 con 0x55963a0be850 2024-04-23T14:51:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.703+0000 7f58b9600640 1 -- 172.21.15.83:0/2968079226 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f58c80a1380 con 0x55963a0be850 2024-04-23T14:51:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.703+0000 7f58d9769740 1 -- 172.21.15.83:0/2968079226 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c809d2b0 con 0x55963a0be850 2024-04-23T14:51:54.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.704+0000 7f58b9600640 1 -- 172.21.15.83:0/2968079226 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f58c80b5050 con 0x55963a0be850 2024-04-23T14:51:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.704+0000 7f58d9769740 10 client.26015 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.704+0000 7f58d9769740 1 -- 172.21.15.83:0/2968079226 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559639df57d0 con 0x55963a0be850 2024-04-23T14:51:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.704+0000 7f58b9600640 1 -- 172.21.15.83:0/2968079226 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c809d2b0 con 0x55963a0be850 2024-04-23T14:51:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.704+0000 7f58b9600640 1 -- 172.21.15.83:0/2968079226 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f58c8020070 con 0x55963a0be850 2024-04-23T14:51:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.704+0000 7f58d9769740 10 client.26015 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.704+0000 7f58d9769740 10 client.26015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.704+0000 7f58d9769740 20 client.26015 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.704+0000 7f58d9769740 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55963a0dfb10 0x55963a0e1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.704+0000 7f58d9769740 4 client.26015 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:51:54.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.704+0000 7f58d9769740 1 -- 172.21.15.83:0/2968079226 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55963a026c10 con 0x55963a0dfb10 2024-04-23T14:51:54.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.706+0000 7f58cf400640 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55963a0dfb10 0x55963a0e1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.709+0000 7f58cf400640 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55963a0dfb10 0x55963a0e1f00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d09cf646a29867 server_cookie=f03a87d1e46f8141 in_seq=0 out_seq=0 2024-04-23T14:51:54.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.709+0000 7f58b9600640 10 client.26015 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:54.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.711+0000 7f58b9600640 1 -- 172.21.15.83:0/2968079226 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55963a026c10 con 0x55963a0dfb10 2024-04-23T14:51:54.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.711+0000 7f58b9600640 10 client.26015 handle_command_reply: tid=0 2024-04-23T14:51:54.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.711+0000 7f58d9769740 1 client.26015 shutdown 2024-04-23T14:51:54.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.712+0000 7f58d9769740 1 -- 172.21.15.83:0/2968079226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 msgr2=0x55963a0bc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:54.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.712+0000 7f58d9769740 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 0x55963a0bc370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f58c8099c80 tx=0x7f58c8099b50 comp rx=0 tx=0).stop 2024-04-23T14:51:54.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.712+0000 7f58d9769740 1 -- 172.21.15.83:0/2968079226 shutdown_connections 2024-04-23T14:51:54.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.712+0000 7f58d9769740 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55963a0dfb10 0x55963a0e1f00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.712+0000 7f58d9769740 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0be850 0x55963a0bc370 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.712+0000 7f58d9769740 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.712+0000 7f58d9769740 1 --2- 172.21.15.83:0/2968079226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0b3860 0x55963a0bb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.712+0000 7f58b9600640 0 client.26015 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:54.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.712+0000 7f58d9769740 1 -- 172.21.15.83:0/2968079226 >> 172.21.15.83:0/2968079226 conn(0x55963a0afb80 msgr2=0x55963a0b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:54.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.712+0000 7f58d9769740 1 -- 172.21.15.83:0/2968079226 shutdown_connections 2024-04-23T14:51:54.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.713+0000 7f58d9769740 1 -- 172.21.15.83:0/2968079226 wait complete. 2024-04-23T14:51:54.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.713+0000 7f58d9769740 20 client.26015 trim_cache size 0 max 16384 2024-04-23T14:51:54.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.750+0000 7f58d9769740 1 Processor -- start 2024-04-23T14:51:54.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.750+0000 7f58d9769740 1 -- start start 2024-04-23T14:51:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.750+0000 7f58d9769740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0b3860 0x55963a0ba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.750+0000 7f58d9769740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0bc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.751+0000 7f58d9769740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0be850 0x55963a0e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.751+0000 7f58d9769740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58d01b6c40 con 0x55963a0b4180 2024-04-23T14:51:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.751+0000 7f58d9769740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f58d01b45d0 con 0x55963a0b3860 2024-04-23T14:51:54.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.751+0000 7f58d9769740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010c000 con 0x55963a0be850 2024-04-23T14:51:54.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.751+0000 7f58cf400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0b3860 0x55963a0ba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.751+0000 7f58cf400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0b3860 0x55963a0ba850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:52430/0 (socket says 172.21.15.83:52430) 2024-04-23T14:51:54.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.751+0000 7f58cf400640 1 -- 172.21.15.83:0/2852895324 learned_addr learned my addr 172.21.15.83:0/2852895324 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:54.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.751+0000 7f58cfe00640 1 --2- 172.21.15.83:0/2852895324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0be850 0x55963a0e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.751+0000 7f58cf400640 1 -- 172.21.15.83:0/2852895324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0be850 msgr2=0x55963a0e4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:54.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.752+0000 7f58cf400640 1 --2- 172.21.15.83:0/2852895324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0be850 0x55963a0e4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.752+0000 7f58cf400640 1 -- 172.21.15.83:0/2852895324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 msgr2=0x55963a0bc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:54.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.751+0000 7f58cea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0bc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.753+0000 7f58cf400640 1 --2- 172.21.15.83:0/2852895324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0bc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.753+0000 7f58cf400640 1 -- 172.21.15.83:0/2852895324 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559639d2b3f0 con 0x55963a0b3860 2024-04-23T14:51:54.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.754+0000 7f58cf400640 1 --2- 172.21.15.83:0/2852895324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0b3860 0x55963a0ba850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00ade0 tx=0x7f58bc00ae10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81276f398352e325 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:54.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.754+0000 7f58bbe00640 1 -- 172.21.15.83:0/2852895324 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc01a490 con 0x55963a0b3860 2024-04-23T14:51:54.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.755+0000 7f58bbe00640 1 -- 172.21.15.83:0/2852895324 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58bc01a630 con 0x55963a0b3860 2024-04-23T14:51:54.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.755+0000 7f58bbe00640 1 -- 172.21.15.83:0/2852895324 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc013660 con 0x55963a0b3860 2024-04-23T14:51:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.756+0000 7f58d9769740 1 -- 172.21.15.83:0/2852895324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0b3860 msgr2=0x55963a0ba850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.756+0000 7f58d9769740 1 --2- 172.21.15.83:0/2852895324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0b3860 0x55963a0ba850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00ade0 tx=0x7f58bc00ae10 comp rx=0 tx=0).stop 2024-04-23T14:51:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.756+0000 7f58d9769740 1 -- 172.21.15.83:0/2852895324 shutdown_connections 2024-04-23T14:51:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.756+0000 7f58d9769740 1 --2- 172.21.15.83:0/2852895324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0be850 0x55963a0e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.756+0000 7f58d9769740 1 --2- 172.21.15.83:0/2852895324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0bc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.756+0000 7f58d9769740 1 --2- 172.21.15.83:0/2852895324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a0b3860 0x55963a0ba850 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00ade0 tx=0x7f58bc00ae10 comp rx=0 tx=0).stop 2024-04-23T14:51:54.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.756+0000 7f58d9769740 1 -- 172.21.15.83:0/2852895324 >> 172.21.15.83:0/2852895324 conn(0x55963a0afb80 msgr2=0x55963a0bd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:54.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.756+0000 7f58d9769740 1 -- 172.21.15.83:0/2852895324 shutdown_connections 2024-04-23T14:51:54.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.756+0000 7f58d9769740 1 -- 172.21.15.83:0/2852895324 wait complete. 2024-04-23T14:51:54.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.757+0000 7f58d9769740 1 -- start start 2024-04-23T14:51:54.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.757+0000 7f58d9769740 1 Processor -- start 2024-04-23T14:51:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.758+0000 7f58d9769740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.758+0000 7f58d9769740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0be850 0x55963a0e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.758+0000 7f58d9769740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a142ce0 0x55963a1450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.758+0000 7f58d9769740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010c670 con 0x55963a0b4180 2024-04-23T14:51:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.758+0000 7f58d9769740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010dae0 con 0x55963a142ce0 2024-04-23T14:51:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.758+0000 7f58d9769740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010cd40 con 0x55963a0be850 2024-04-23T14:51:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.758+0000 7f58d9769740 10 client.0 _ll_register_callbacks cb 0x7f58c809d080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.759+0000 7f58cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.760+0000 7f58cf400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0e2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52460/0 (socket says 172.21.15.83:52460) 2024-04-23T14:51:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.760+0000 7f58cf400640 1 -- 172.21.15.83:0/2079865981 learned_addr learned my addr 172.21.15.83:0/2079865981 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:54.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.760+0000 7f58cea00640 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0be850 0x55963a0e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.761+0000 7f58cfe00640 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a142ce0 0x55963a1450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.762+0000 7f58cea00640 1 -- 172.21.15.83:0/2079865981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a142ce0 msgr2=0x55963a1450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:54.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.762+0000 7f58cea00640 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a142ce0 0x55963a1450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.762+0000 7f58cea00640 1 -- 172.21.15.83:0/2079865981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 msgr2=0x55963a0e2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:54.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.762+0000 7f58cea00640 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0e2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.762+0000 7f58cea00640 1 -- 172.21.15.83:0/2079865981 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559639ceb0f0 con 0x55963a0be850 2024-04-23T14:51:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.764+0000 7f58cea00640 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0be850 0x55963a0e34d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58c00096c0 tx=0x7f58c0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=649a52fc8d14e1ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.764+0000 7f58b9600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:51:54.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.764+0000 7f58b9600640 1 -- 172.21.15.83:0/2079865981 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0035040 con 0x55963a0be850 2024-04-23T14:51:54.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.764+0000 7f58b9600640 1 -- 172.21.15.83:0/2079865981 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f58c0004ad0 con 0x55963a0be850 2024-04-23T14:51:54.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.764+0000 7f58d9769740 1 -- 172.21.15.83:0/2079865981 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58bc01a630 con 0x55963a0be850 2024-04-23T14:51:54.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.768+0000 7f58b9600640 1 -- 172.21.15.83:0/2079865981 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f58c0004d10 con 0x55963a0be850 2024-04-23T14:51:54.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.768+0000 7f58d9769740 10 client.25972 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:54.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.768+0000 7f58d9769740 1 -- 172.21.15.83:0/2079865981 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559639fde810 con 0x55963a0be850 2024-04-23T14:51:54.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.768+0000 7f58b9600640 1 -- 172.21.15.83:0/2079865981 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58bc01a630 con 0x55963a0be850 2024-04-23T14:51:54.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.769+0000 7f58b9600640 1 -- 172.21.15.83:0/2079865981 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f58c0003710 con 0x55963a0be850 2024-04-23T14:51:54.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.770+0000 7f58d9769740 10 client.25972 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:54.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.770+0000 7f58d9769740 10 client.25972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:54.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.770+0000 7f58d9769740 20 client.25972 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.770+0000 7f58d9769740 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55963a146bd0 0x55963a148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.770+0000 7f58d9769740 4 client.25972 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:51:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.770+0000 7f58d9769740 1 -- 172.21.15.83:0/2079865981 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55963a020a80 con 0x55963a146bd0 2024-04-23T14:51:54.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.772+0000 7f58cf400640 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55963a146bd0 0x55963a148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.773+0000 7f58cf400640 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55963a146bd0 0x55963a148ff0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f3e6496f582ba59 server_cookie=55730da0133c2e7f in_seq=0 out_seq=0 2024-04-23T14:51:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.773+0000 7f58b9600640 10 client.25972 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.773+0000 7f58b9600640 1 -- 172.21.15.83:0/2079865981 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55963a020a80 con 0x55963a146bd0 2024-04-23T14:51:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.773+0000 7f58b9600640 10 client.25972 handle_command_reply: tid=0 2024-04-23T14:51:54.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.773+0000 7f58d9769740 1 client.25972 shutdown 2024-04-23T14:51:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.774+0000 7f58d9769740 1 -- 172.21.15.83:0/2079865981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0be850 msgr2=0x55963a0e34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.774+0000 7f58d9769740 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0be850 0x55963a0e34d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58c00096c0 tx=0x7f58c0009260 comp rx=0 tx=0).stop 2024-04-23T14:51:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.774+0000 7f58d9769740 1 -- 172.21.15.83:0/2079865981 shutdown_connections 2024-04-23T14:51:54.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.774+0000 7f58d9769740 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55963a146bd0 0x55963a148ff0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.774+0000 7f58d9769740 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55963a142ce0 0x55963a1450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.774+0000 7f58d9769740 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55963a0be850 0x55963a0e34d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.774+0000 7f58d9769740 1 --2- 172.21.15.83:0/2079865981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55963a0b4180 0x55963a0e2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.774+0000 7f58d9769740 1 -- 172.21.15.83:0/2079865981 >> 172.21.15.83:0/2079865981 conn(0x55963a0afb80 msgr2=0x55963a0ae390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:54.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.774+0000 7f58b9600640 0 client.25972 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:54.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.774+0000 7f58d9769740 1 -- 172.21.15.83:0/2079865981 shutdown_connections 2024-04-23T14:51:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.776+0000 7f58d9769740 1 -- 172.21.15.83:0/2079865981 wait complete. 2024-04-23T14:51:54.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.776+0000 7f58d9769740 20 client.25972 trim_cache size 0 max 16384 2024-04-23T14:51:54.778 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:51:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:51:54.778 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:51:54.778 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:51:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.777+0000 7f58d6800640 1 -- 172.21.15.83:0/1949573364 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f58b40a3e70 msgr2=0x7f58b40a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.777+0000 7f58d6800640 1 --2- 172.21.15.83:0/1949573364 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f58b40a3e70 0x7f58b40a6330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f58c0006010 tx=0x7f58c0000bc0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.777+0000 7f58d6800640 1 -- 172.21.15.83:0/1949573364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58d0112ee0 msgr2=0x7f58d006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.777+0000 7f58d6800640 1 --2- 172.21.15.83:0/1949573364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58d0112ee0 0x7f58d006e0c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58c800eb40 tx=0x7f58c800c2d0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.777+0000 7f58d6800640 1 -- 172.21.15.83:0/1949573364 shutdown_connections 2024-04-23T14:51:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.777+0000 7f58d6800640 1 --2- 172.21.15.83:0/1949573364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58d0112ee0 0x7f58d006e0c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.777+0000 7f58d6800640 1 --2- 172.21.15.83:0/1949573364 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f58b40a3e70 0x7f58b40a6330 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.777+0000 7f58d6800640 1 --2- 172.21.15.83:0/1949573364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f58d0076e50 0x7f58d006db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.777+0000 7f58d6800640 1 --2- 172.21.15.83:0/1949573364 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58d0076530 0x7f58d0073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.777+0000 7f58d6800640 1 -- 172.21.15.83:0/1949573364 >> 172.21.15.83:0/1949573364 conn(0x7f58d00ffd90 msgr2=0x7f58d0114670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.778+0000 7f58d6800640 1 -- 172.21.15.83:0/1949573364 shutdown_connections 2024-04-23T14:51:54.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:54.778+0000 7f58d6800640 1 -- 172.21.15.83:0/1949573364 wait complete. 2024-04-23T14:51:54.807 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:51:54.808 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:51:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.103+0000 7f7e02c00640 1 -- 172.21.15.83:0/4021754862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc104890 msgr2=0x7f7dfc104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.103+0000 7f7e02c00640 1 --2- 172.21.15.83:0/4021754862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc104890 0x7f7dfc104d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009ca0 tx=0x7f7df402a8c0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.103+0000 7f7e02c00640 1 -- 172.21.15.83:0/4021754862 shutdown_connections 2024-04-23T14:51:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.103+0000 7f7e02c00640 1 --2- 172.21.15.83:0/4021754862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc104890 0x7f7dfc104d40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.103+0000 7f7e02c00640 1 --2- 172.21.15.83:0/4021754862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7dfc1071b0 0x7f7dfc0696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.103+0000 7f7e02c00640 1 --2- 172.21.15.83:0/4021754862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dfc106d40 0x7f7dfc069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.103+0000 7f7e02c00640 1 -- 172.21.15.83:0/4021754862 >> 172.21.15.83:0/4021754862 conn(0x7f7dfc076840 msgr2=0x7f7dfc076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.104+0000 7f7e02c00640 1 -- 172.21.15.83:0/4021754862 shutdown_connections 2024-04-23T14:51:55.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.104+0000 7f7e02c00640 1 -- 172.21.15.83:0/4021754862 wait complete. 2024-04-23T14:51:55.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.105+0000 7f7e02c00640 1 Processor -- start 2024-04-23T14:51:55.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.105+0000 7f7e02c00640 1 -- start start 2024-04-23T14:51:55.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.105+0000 7f7e02c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc104890 0x7f7dfc1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.105+0000 7f7e02c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dfc106d40 0x7f7dfc1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.105+0000 7f7e02c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7dfc1071b0 0x7f7dfc19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.106+0000 7f7e01800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc104890 0x7f7dfc1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.106+0000 7f7e01800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc104890 0x7f7dfc1a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40830/0 (socket says 172.21.15.83:40830) 2024-04-23T14:51:55.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.106+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc118780 con 0x7f7dfc106d40 2024-04-23T14:51:55.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.106+0000 7f7e01800640 1 -- 172.21.15.83:0/2627848957 learned_addr learned my addr 172.21.15.83:0/2627848957 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:55.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.106+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc118600 con 0x7f7dfc1071b0 2024-04-23T14:51:55.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.106+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc118900 con 0x7f7dfc104890 2024-04-23T14:51:55.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.106+0000 7f7e02200640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7dfc1071b0 0x7f7dfc19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:55.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.106+0000 7f7e00e00640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dfc106d40 0x7f7dfc1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:55.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.107+0000 7f7e02200640 1 -- 172.21.15.83:0/2627848957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc104890 msgr2=0x7f7dfc1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:55.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.107+0000 7f7e02200640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc104890 0x7f7dfc1a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.107+0000 7f7e02200640 1 -- 172.21.15.83:0/2627848957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dfc106d40 msgr2=0x7f7dfc1a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.107+0000 7f7e02200640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dfc106d40 0x7f7dfc1a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.107+0000 7f7e02200640 1 -- 172.21.15.83:0/2627848957 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc19eff0 con 0x7f7dfc1071b0 2024-04-23T14:51:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.107+0000 7f7e01800640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc104890 0x7f7dfc1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:51:55.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.107+0000 7f7e02200640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7dfc1071b0 0x7f7dfc19e820 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009bf0 tx=0x7f7df4031980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.109+0000 7f7dfa000640 1 -- 172.21.15.83:0/2627848957 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4004450 con 0x7f7dfc1071b0 2024-04-23T14:51:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.109+0000 7f7dfa000640 1 -- 172.21.15.83:0/2627848957 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df4040070 con 0x7f7dfc1071b0 2024-04-23T14:51:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.109+0000 7f7dfa000640 1 -- 172.21.15.83:0/2627848957 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df40456d0 con 0x7f7dfc1071b0 2024-04-23T14:51:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.109+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dfc19f280 con 0x7f7dfc1071b0 2024-04-23T14:51:55.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.109+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dfc1ab500 con 0x7f7dfc1071b0 2024-04-23T14:51:55.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.112+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dfc101370 con 0x7f7dfc1071b0 2024-04-23T14:51:55.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.115+0000 7f7dfa000640 1 -- 172.21.15.83:0/2627848957 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7df40305c0 con 0x7f7dfc1071b0 2024-04-23T14:51:55.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.117+0000 7f7dfa000640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7de40a3ef0 0x7f7de40a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.118+0000 7f7dfa000640 1 -- 172.21.15.83:0/2627848957 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7df40f1f20 con 0x7f7dfc1071b0 2024-04-23T14:51:55.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.119+0000 7f7e01800640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7de40a3ef0 0x7f7de40a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:55.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.121+0000 7f7e01800640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7de40a3ef0 0x7f7de40a63b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7de8005fd0 tx=0x7f7de800b480 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:55.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.125+0000 7f7dfa000640 1 -- 172.21.15.83:0/2627848957 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7df40be700 con 0x7f7dfc1071b0 2024-04-23T14:51:55.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.639+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dfc1ab6d0 con 0x7f7dfc1071b0 2024-04-23T14:51:55.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.640+0000 7f7dfa000640 1 -- 172.21.15.83:0/2627848957 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11867 (secure 0 0 0) 0x7f7df40be000 con 0x7f7dfc1071b0 2024-04-23T14:51:55.642 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:51:55.642 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:50:17.708313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:51:55.643 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-04-23T14:51:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.647+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7de40a3ef0 msgr2=0x7f7de40a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.647+0000 7f7e02c00640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7de40a3ef0 0x7f7de40a63b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7de8005fd0 tx=0x7f7de800b480 comp rx=0 tx=0).stop 2024-04-23T14:51:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.647+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7dfc1071b0 msgr2=0x7f7dfc19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:55.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.647+0000 7f7e02c00640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7dfc1071b0 0x7f7dfc19e820 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009bf0 tx=0x7f7df4031980 comp rx=0 tx=0).stop 2024-04-23T14:51:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.650+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 shutdown_connections 2024-04-23T14:51:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.650+0000 7f7e02c00640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7dfc1071b0 0x7f7dfc19e820 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.650+0000 7f7e02c00640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7de40a3ef0 0x7f7de40a63b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.650+0000 7f7e02c00640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dfc106d40 0x7f7dfc1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.650+0000 7f7e02c00640 1 --2- 172.21.15.83:0/2627848957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7dfc104890 0x7f7dfc1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.650+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 >> 172.21.15.83:0/2627848957 conn(0x7f7dfc076840 msgr2=0x7f7dfc0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:55.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.651+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 shutdown_connections 2024-04-23T14:51:55.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.653+0000 7f7e02c00640 1 -- 172.21.15.83:0/2627848957 wait complete. 2024-04-23T14:51:55.691 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:51:55.691 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:55 smithi083 ceph-mon[27867]: pgmap v106: 97 pgs: 97 active+clean; 15 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 162 MiB/s wr, 22.50k op/s 2024-04-23T14:51:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:55 smithi203 ceph-mon[32800]: pgmap v106: 97 pgs: 97 active+clean; 15 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 162 MiB/s wr, 22.50k op/s 2024-04-23T14:51:55.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:55 smithi173 ceph-mon[32443]: pgmap v106: 97 pgs: 97 active+clean; 15 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 162 MiB/s wr, 22.50k op/s 2024-04-23T14:51:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.987+0000 7f7936800640 1 -- 172.21.15.83:0/2535560571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7930109270 msgr2=0x7f7930109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.987+0000 7f7936800640 1 --2- 172.21.15.83:0/2535560571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7930109270 0x7f7930109650 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f791c00b1e0 tx=0x7f791c02a820 comp rx=0 tx=0).stop 2024-04-23T14:51:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.988+0000 7f7936800640 1 -- 172.21.15.83:0/2535560571 shutdown_connections 2024-04-23T14:51:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.988+0000 7f7936800640 1 --2- 172.21.15.83:0/2535560571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f793010dd30 0x7f7930115900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.988+0000 7f7936800640 1 --2- 172.21.15.83:0/2535560571 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7930109c20 0x7f793010d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.988+0000 7f7936800640 1 --2- 172.21.15.83:0/2535560571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7930109270 0x7f7930109650 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.988+0000 7f7936800640 1 -- 172.21.15.83:0/2535560571 >> 172.21.15.83:0/2535560571 conn(0x7f793006d110 msgr2=0x7f793006d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.988+0000 7f7936800640 1 -- 172.21.15.83:0/2535560571 shutdown_connections 2024-04-23T14:51:55.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.988+0000 7f7936800640 1 -- 172.21.15.83:0/2535560571 wait complete. 2024-04-23T14:51:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.990+0000 7f7936800640 1 Processor -- start 2024-04-23T14:51:55.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.990+0000 7f7936800640 1 -- start start 2024-04-23T14:51:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.991+0000 7f7936800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7930109270 0x7f79301a7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:55.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.991+0000 7f7936800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793010dd30 0x7f79301a7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.992+0000 7f792f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7930109270 0x7f79301a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.992+0000 7f7936800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79301abfd0 0x7f79301ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:55.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.992+0000 7f7936800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7930118a00 con 0x7f793010dd30 2024-04-23T14:51:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.992+0000 7f7936800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7930118880 con 0x7f7930109270 2024-04-23T14:51:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.992+0000 7f7936800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7930118b80 con 0x7f79301abfd0 2024-04-23T14:51:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.992+0000 7f792fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79301abfd0 0x7f79301ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.992+0000 7f792fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79301abfd0 0x7f79301ac480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40852/0 (socket says 172.21.15.83:40852) 2024-04-23T14:51:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.992+0000 7f792fe00640 1 -- 172.21.15.83:0/3616935379 learned_addr learned my addr 172.21.15.83:0/3616935379 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.992+0000 7f792ea00640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793010dd30 0x7f79301a7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.993+0000 7f792ea00640 1 -- 172.21.15.83:0/3616935379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79301abfd0 msgr2=0x7f79301ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.993+0000 7f792ea00640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79301abfd0 0x7f79301ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.993+0000 7f792ea00640 1 -- 172.21.15.83:0/3616935379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7930109270 msgr2=0x7f79301a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.993+0000 7f792ea00640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7930109270 0x7f79301a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:55.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.993+0000 7f792ea00640 1 -- 172.21.15.83:0/3616935379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79301acac0 con 0x7f793010dd30 2024-04-23T14:51:55.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.995+0000 7f792ea00640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793010dd30 0x7f79301a7c50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f792000e9c0 tx=0x7f792000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:55.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.996+0000 7f7927e00640 1 -- 172.21.15.83:0/3616935379 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792000cd80 con 0x7f793010dd30 2024-04-23T14:51:55.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.996+0000 7f7936800640 1 -- 172.21.15.83:0/3616935379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79301acdb0 con 0x7f793010dd30 2024-04-23T14:51:55.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.997+0000 7f7936800640 1 -- 172.21.15.83:0/3616935379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79301b45f0 con 0x7f793010dd30 2024-04-23T14:51:56.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.998+0000 7f7927e00640 1 -- 172.21.15.83:0/3616935379 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7920009390 con 0x7f793010dd30 2024-04-23T14:51:56.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:55.998+0000 7f7927e00640 1 -- 172.21.15.83:0/3616935379 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920010430 con 0x7f793010dd30 2024-04-23T14:51:56.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.003+0000 7f79395c0740 1 Processor -- start 2024-04-23T14:51:56.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.003+0000 7f79395c0740 1 -- start start 2024-04-23T14:51:56.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.003+0000 7f79395c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 0x56462c7cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.003+0000 7f79395c0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 0x56462c7d6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.003+0000 7f79395c0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7d6940 0x56462c7d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.003+0000 7f79395c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56462c7311d0 con 0x56462c7cb950 2024-04-23T14:51:56.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.003+0000 7f79395c0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56462c6d7450 con 0x56462c7cc270 2024-04-23T14:51:56.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.003+0000 7f79395c0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56462c72d190 con 0x56462c7d6940 2024-04-23T14:51:56.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.004+0000 7f792f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 0x56462c7d6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.004+0000 7f792f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 0x56462c7d6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33150/0 (socket says 172.21.15.83:33150) 2024-04-23T14:51:56.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.004+0000 7f792f400640 1 -- 172.21.15.83:0/3980009160 learned_addr learned my addr 172.21.15.83:0/3980009160 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.004+0000 7f792ea00640 1 --2- 172.21.15.83:0/3980009160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7d6940 0x56462c7d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.004+0000 7f792f400640 1 -- 172.21.15.83:0/3980009160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7d6940 msgr2=0x56462c7d8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.004+0000 7f792f400640 1 --2- 172.21.15.83:0/3980009160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7d6940 0x56462c7d8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.004+0000 7f792f400640 1 -- 172.21.15.83:0/3980009160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 msgr2=0x56462c7cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.004+0000 7f792f400640 1 --2- 172.21.15.83:0/3980009160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 0x56462c7cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.004+0000 7f792f400640 1 -- 172.21.15.83:0/3980009160 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56462c794310 con 0x56462c7cc270 2024-04-23T14:51:56.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.005+0000 7f792f400640 1 --2- 172.21.15.83:0/3980009160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 0x56462c7d6400 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f791c02ad90 tx=0x7f791c009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faf8c09a567a8fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.006+0000 7f7924c00640 1 -- 172.21.15.83:0/3980009160 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c042020 con 0x56462c7cc270 2024-04-23T14:51:56.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.006+0000 7f7924c00640 1 -- 172.21.15.83:0/3980009160 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f791c009820 con 0x56462c7cc270 2024-04-23T14:51:56.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.006+0000 7f7924c00640 1 -- 172.21.15.83:0/3980009160 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c0071c0 con 0x56462c7cc270 2024-04-23T14:51:56.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.007+0000 7f79395c0740 1 -- 172.21.15.83:0/3980009160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 msgr2=0x56462c7d6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.007+0000 7f79395c0740 1 --2- 172.21.15.83:0/3980009160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 0x56462c7d6400 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f791c02ad90 tx=0x7f791c009630 comp rx=0 tx=0).stop 2024-04-23T14:51:56.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.009+0000 7f79395c0740 1 -- 172.21.15.83:0/3980009160 shutdown_connections 2024-04-23T14:51:56.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.009+0000 7f79395c0740 1 --2- 172.21.15.83:0/3980009160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7d6940 0x56462c7d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.009+0000 7f79395c0740 1 --2- 172.21.15.83:0/3980009160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 0x56462c7d6400 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.009+0000 7f79395c0740 1 --2- 172.21.15.83:0/3980009160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 0x56462c7cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.009+0000 7f79395c0740 1 -- 172.21.15.83:0/3980009160 >> 172.21.15.83:0/3980009160 conn(0x56462c7c7c70 msgr2=0x56462c7c8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.011+0000 7f7927e00640 1 -- 172.21.15.83:0/3616935379 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7920003f30 con 0x7f793010dd30 2024-04-23T14:51:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.013+0000 7f79395c0740 1 -- 172.21.15.83:0/3980009160 shutdown_connections 2024-04-23T14:51:56.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.013+0000 7f79395c0740 1 -- 172.21.15.83:0/3980009160 wait complete. 2024-04-23T14:51:56.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.014+0000 7f7927e00640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79140a3cf0 0x7f79140a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.014+0000 7f7927e00640 1 -- 172.21.15.83:0/3616935379 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f79200d60d0 con 0x7f793010dd30 2024-04-23T14:51:56.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.014+0000 7f792f400640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79140a3cf0 0x7f79140a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.015+0000 7f79395c0740 1 -- start start 2024-04-23T14:51:56.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.015+0000 7f79395c0740 1 Processor -- start 2024-04-23T14:51:56.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f792f400640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79140a3cf0 0x7f79140a61b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f791c0047e0 tx=0x7f791c02a820 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f79395c0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cb950 0x56462c7d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f79395c0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 0x56462c7d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f79395c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7d41c0 0x56462c7e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f79395c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56462c7d9770 con 0x56462c7d41c0 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f79395c0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56462c6ea810 con 0x56462c7cc270 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f79395c0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56462c7d98f0 con 0x56462c7cb950 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f79395c0740 10 client.0 _ll_register_callbacks cb 0x56462c6b8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f792f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cb950 0x56462c7d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f792ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 0x56462c7d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f792f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cb950 0x56462c7d3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40878/0 (socket says 172.21.15.83:40878) 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f792f400640 1 -- 172.21.15.83:0/3465436644 learned_addr learned my addr 172.21.15.83:0/3465436644 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.016+0000 7f792fe00640 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7d41c0 0x56462c7e2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.017+0000 7f792ea00640 1 -- 172.21.15.83:0/3465436644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cb950 msgr2=0x56462c7d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.017+0000 7f792ea00640 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cb950 0x56462c7d3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.017+0000 7f792ea00640 1 -- 172.21.15.83:0/3465436644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7d41c0 msgr2=0x56462c7e2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.017+0000 7f792ea00640 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7d41c0 0x56462c7e2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.017+0000 7f792ea00640 1 -- 172.21.15.83:0/3465436644 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56462c6fb3f0 con 0x56462c7cc270 2024-04-23T14:51:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.017+0000 7f792fe00640 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7d41c0 0x56462c7e2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:51:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.017+0000 7f792ea00640 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 0x56462c7d3c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f79200d38d0 tx=0x7f79200039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2705b40c32c8dacb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.017+0000 7f7919600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:51:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.018+0000 7f7919600640 1 -- 172.21.15.83:0/3465436644 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79200ba020 con 0x56462c7cc270 2024-04-23T14:51:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.018+0000 7f7919600640 1 -- 172.21.15.83:0/3465436644 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f792009e700 con 0x56462c7cc270 2024-04-23T14:51:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.018+0000 7f79395c0740 1 -- 172.21.15.83:0/3465436644 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f791c009820 con 0x56462c7cc270 2024-04-23T14:51:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.018+0000 7f7919600640 1 -- 172.21.15.83:0/3465436644 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f79200a1a80 con 0x56462c7cc270 2024-04-23T14:51:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.019+0000 7f79395c0740 10 client.26036 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.019+0000 7f79395c0740 1 -- 172.21.15.83:0/3465436644 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56462c4433f0 con 0x56462c7cc270 2024-04-23T14:51:56.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.019+0000 7f7919600640 1 -- 172.21.15.83:0/3465436644 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f791c009820 con 0x56462c7cc270 2024-04-23T14:51:56.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.020+0000 7f7919600640 1 -- 172.21.15.83:0/3465436644 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f7920004220 con 0x56462c7cc270 2024-04-23T14:51:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.020+0000 7f79395c0740 10 client.26036 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.020+0000 7f79395c0740 10 client.26036 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:51:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.020+0000 7f79395c0740 20 client.26036 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:56.023 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.021+0000 7f79395c0740 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x56462c7f8040 0x56462c7fa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.021+0000 7f79395c0740 4 client.26036 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:51:56.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.021+0000 7f79395c0740 1 -- 172.21.15.83:0/3465436644 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56462c73ebf0 con 0x56462c7f8040 2024-04-23T14:51:56.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.021+0000 7f792f400640 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x56462c7f8040 0x56462c7fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.022+0000 7f792f400640 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x56462c7f8040 0x56462c7fa430 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e11297bd13094ca3 server_cookie=5039b4b13826270d in_seq=0 out_seq=0 2024-04-23T14:51:56.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.022+0000 7f7919600640 10 client.26036 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:51:56.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.023+0000 7f7919600640 1 -- 172.21.15.83:0/3465436644 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56462c73ebf0 con 0x56462c7f8040 2024-04-23T14:51:56.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.023+0000 7f7919600640 10 client.26036 handle_command_reply: tid=0 2024-04-23T14:51:56.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.023+0000 7f79395c0740 1 client.26036 shutdown 2024-04-23T14:51:56.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.025+0000 7f79395c0740 1 -- 172.21.15.83:0/3465436644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 msgr2=0x56462c7d3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.025+0000 7f79395c0740 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 0x56462c7d3c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f79200d38d0 tx=0x7f79200039e0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.025+0000 7f79395c0740 1 -- 172.21.15.83:0/3465436644 shutdown_connections 2024-04-23T14:51:56.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.025+0000 7f79395c0740 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x56462c7f8040 0x56462c7fa430 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.025+0000 7f79395c0740 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7d41c0 0x56462c7e2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.025+0000 7f79395c0740 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7cc270 0x56462c7d3c80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.025+0000 7f79395c0740 1 --2- 172.21.15.83:0/3465436644 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cb950 0x56462c7d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.025+0000 7f79395c0740 1 -- 172.21.15.83:0/3465436644 >> 172.21.15.83:0/3465436644 conn(0x56462c7c7c70 msgr2=0x56462c7cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.025+0000 7f7919600640 0 client.26036 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:51:56.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.026+0000 7f79395c0740 1 -- 172.21.15.83:0/3465436644 shutdown_connections 2024-04-23T14:51:56.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.026+0000 7f79395c0740 1 -- 172.21.15.83:0/3465436644 wait complete. 2024-04-23T14:51:56.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.026+0000 7f79395c0740 20 client.26036 trim_cache size 0 max 16384 2024-04-23T14:51:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.054+0000 7f79395c0740 1 Processor -- start 2024-04-23T14:51:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.055+0000 7f79395c0740 1 -- start start 2024-04-23T14:51:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.055+0000 7f79395c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 0x56462c7ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.055+0000 7f79395c0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 0x56462c800260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.055+0000 7f79395c0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7d41c0 0x56462c7e26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.055+0000 7f79395c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79100110f0 con 0x56462c7cb950 2024-04-23T14:51:56.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.055+0000 7f79395c0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f791000e990 con 0x56462c7d41c0 2024-04-23T14:51:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.055+0000 7f79395c0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f791000c7e0 con 0x56462c7cc270 2024-04-23T14:51:56.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.056+0000 7f792ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 0x56462c800260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.056+0000 7f792f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 0x56462c7ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.056+0000 7f792ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 0x56462c800260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40894/0 (socket says 172.21.15.83:40894) 2024-04-23T14:51:56.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.056+0000 7f792ea00640 1 -- 172.21.15.83:0/3488449999 learned_addr learned my addr 172.21.15.83:0/3488449999 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.058 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.056+0000 7f792fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7d41c0 0x56462c7e26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.057+0000 7f792ea00640 1 -- 172.21.15.83:0/3488449999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7d41c0 msgr2=0x56462c7e26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.057+0000 7f792ea00640 1 --2- 172.21.15.83:0/3488449999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7d41c0 0x56462c7e26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.057+0000 7f792ea00640 1 -- 172.21.15.83:0/3488449999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 msgr2=0x56462c7ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.057+0000 7f792ea00640 1 --2- 172.21.15.83:0/3488449999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 0x56462c7ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.057+0000 7f792ea00640 1 -- 172.21.15.83:0/3488449999 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56462c7fa7a0 con 0x56462c7cc270 2024-04-23T14:51:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.057+0000 7f792fe00640 1 --2- 172.21.15.83:0/3488449999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7d41c0 0x56462c7e26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:51:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.057+0000 7f792ea00640 1 --2- 172.21.15.83:0/3488449999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 0x56462c800260 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f792000cfd0 tx=0x7f7920005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a1127dc307e45d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.057+0000 7f791be00640 1 -- 172.21.15.83:0/3488449999 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920085070 con 0x56462c7cc270 2024-04-23T14:51:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.058+0000 7f791be00640 1 -- 172.21.15.83:0/3488449999 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79200a2820 con 0x56462c7cc270 2024-04-23T14:51:56.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.058+0000 7f791be00640 1 -- 172.21.15.83:0/3488449999 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79200ba070 con 0x56462c7cc270 2024-04-23T14:51:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.058+0000 7f79395c0740 1 -- 172.21.15.83:0/3488449999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 msgr2=0x56462c800260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.058+0000 7f79395c0740 1 --2- 172.21.15.83:0/3488449999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 0x56462c800260 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f792000cfd0 tx=0x7f7920005e30 comp rx=0 tx=0).stop 2024-04-23T14:51:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.058+0000 7f79395c0740 1 -- 172.21.15.83:0/3488449999 shutdown_connections 2024-04-23T14:51:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.058+0000 7f79395c0740 1 --2- 172.21.15.83:0/3488449999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7d41c0 0x56462c7e26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.058+0000 7f79395c0740 1 --2- 172.21.15.83:0/3488449999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 0x56462c800260 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.058+0000 7f79395c0740 1 --2- 172.21.15.83:0/3488449999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 0x56462c7ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.058+0000 7f79395c0740 1 -- 172.21.15.83:0/3488449999 >> 172.21.15.83:0/3488449999 conn(0x56462c7c7c70 msgr2=0x56462c7d2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.059+0000 7f79395c0740 1 -- 172.21.15.83:0/3488449999 shutdown_connections 2024-04-23T14:51:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.059+0000 7f79395c0740 1 -- 172.21.15.83:0/3488449999 wait complete. 2024-04-23T14:51:56.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.059+0000 7f79395c0740 1 -- start start 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.059+0000 7f79395c0740 1 Processor -- start 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f79395c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 0x56462c7debd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f79395c0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 0x56462c7df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f79395c0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7d41c0 0x56462c7d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f79395c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f791000ce50 con 0x56462c7cb950 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f79395c0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f791000e2c0 con 0x56462c7d41c0 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f79395c0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f791000d520 con 0x56462c7cc270 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f79395c0740 10 client.0 _ll_register_callbacks cb 0x7f792000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f792ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 0x56462c7df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f792ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 0x56462c7df110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40898/0 (socket says 172.21.15.83:40898) 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f792ea00640 1 -- 172.21.15.83:0/930369491 learned_addr learned my addr 172.21.15.83:0/930369491 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f792fe00640 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7d41c0 0x56462c7d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f792f400640 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 0x56462c7debd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f792ea00640 1 -- 172.21.15.83:0/930369491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7d41c0 msgr2=0x56462c7d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f792ea00640 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7d41c0 0x56462c7d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.060+0000 7f792ea00640 1 -- 172.21.15.83:0/930369491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 msgr2=0x56462c7debd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.061+0000 7f792ea00640 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 0x56462c7debd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.061+0000 7f792ea00640 1 -- 172.21.15.83:0/930369491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56462c50d7d0 con 0x56462c7cc270 2024-04-23T14:51:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.061+0000 7f792f400640 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 0x56462c7debd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.061+0000 7f792ea00640 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 0x56462c7df110 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f79200b6e90 tx=0x7f79200d3d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fa8a50f93385fc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.061+0000 7f7919600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:51:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.061+0000 7f7919600640 1 -- 172.21.15.83:0/930369491 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920085070 con 0x56462c7cc270 2024-04-23T14:51:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.061+0000 7f7919600640 1 -- 172.21.15.83:0/930369491 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7920004420 con 0x56462c7cc270 2024-04-23T14:51:56.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.061+0000 7f79395c0740 1 -- 172.21.15.83:0/930369491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79200a2820 con 0x56462c7cc270 2024-04-23T14:51:56.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.062+0000 7f7919600640 1 -- 172.21.15.83:0/930369491 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f79200ba070 con 0x56462c7cc270 2024-04-23T14:51:56.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.062+0000 7f79395c0740 10 client.25996 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:56.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.062+0000 7f79395c0740 1 -- 172.21.15.83:0/930369491 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56462c7d7c10 con 0x56462c7cc270 2024-04-23T14:51:56.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.063+0000 7f7919600640 1 -- 172.21.15.83:0/930369491 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79200a2820 con 0x56462c7cc270 2024-04-23T14:51:56.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.063+0000 7f7919600640 1 -- 172.21.15.83:0/930369491 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f7920088030 con 0x56462c7cc270 2024-04-23T14:51:56.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.063+0000 7f79395c0740 10 client.25996 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:56.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.063+0000 7f79395c0740 10 client.25996 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:51:56.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.063+0000 7f79395c0740 20 client.25996 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:56.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.063+0000 7f79395c0740 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x56462c85e590 0x56462c860980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.063+0000 7f79395c0740 4 client.25996 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:51:56.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.063+0000 7f79395c0740 1 -- 172.21.15.83:0/930369491 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56462c738a60 con 0x56462c85e590 2024-04-23T14:51:56.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.064+0000 7f792f400640 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x56462c85e590 0x56462c860980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.064+0000 7f792f400640 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x56462c85e590 0x56462c860980 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28902446fc3b7e6e server_cookie=8ee5d612eafdf89d in_seq=0 out_seq=0 2024-04-23T14:51:56.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.064+0000 7f7919600640 10 client.25996 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:51:56.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.064+0000 7f7919600640 1 -- 172.21.15.83:0/930369491 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56462c738a60 con 0x56462c85e590 2024-04-23T14:51:56.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.064+0000 7f7919600640 10 client.25996 handle_command_reply: tid=0 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.064+0000 7f79395c0740 1 client.25996 shutdown 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.065+0000 7f79395c0740 1 -- 172.21.15.83:0/930369491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 msgr2=0x56462c7df110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.065+0000 7f79395c0740 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 0x56462c7df110 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f79200b6e90 tx=0x7f79200d3d60 comp rx=0 tx=0).stop 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.065+0000 7f79395c0740 1 -- 172.21.15.83:0/930369491 shutdown_connections 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.065+0000 7f79395c0740 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x56462c85e590 0x56462c860980 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.065+0000 7f7919600640 0 client.25996 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.065+0000 7f79395c0740 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56462c7d41c0 0x56462c7d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.065+0000 7f79395c0740 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56462c7cc270 0x56462c7df110 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.065+0000 7f79395c0740 1 --2- 172.21.15.83:0/930369491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56462c7cb950 0x56462c7debd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.065+0000 7f79395c0740 1 -- 172.21.15.83:0/930369491 >> 172.21.15.83:0/930369491 conn(0x56462c7c7c70 msgr2=0x56462c7c6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.065+0000 7f79395c0740 1 -- 172.21.15.83:0/930369491 shutdown_connections 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.065+0000 7f79395c0740 1 -- 172.21.15.83:0/930369491 wait complete. 2024-04-23T14:51:56.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.066+0000 7f79395c0740 20 client.25996 trim_cache size 0 max 16384 2024-04-23T14:51:56.070 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:51:56.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.067+0000 7f7936800640 1 -- 172.21.15.83:0/3616935379 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79140a3cf0 msgr2=0x7f79140a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.067+0000 7f7936800640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79140a3cf0 0x7f79140a61b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f791c0047e0 tx=0x7f791c02a820 comp rx=0 tx=0).stop 2024-04-23T14:51:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.067+0000 7f7936800640 1 -- 172.21.15.83:0/3616935379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793010dd30 msgr2=0x7f79301a7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.067+0000 7f7936800640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793010dd30 0x7f79301a7c50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f792000e9c0 tx=0x7f792000ee80 comp rx=0 tx=0).stop 2024-04-23T14:51:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.067+0000 7f7936800640 1 -- 172.21.15.83:0/3616935379 shutdown_connections 2024-04-23T14:51:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.067+0000 7f7936800640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79301abfd0 0x7f79301ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.067+0000 7f7936800640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79140a3cf0 0x7f79140a61b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.067+0000 7f7936800640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f793010dd30 0x7f79301a7c50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.067+0000 7f7936800640 1 --2- 172.21.15.83:0/3616935379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7930109270 0x7f79301a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.067+0000 7f7936800640 1 -- 172.21.15.83:0/3616935379 >> 172.21.15.83:0/3616935379 conn(0x7f793006d110 msgr2=0x7f7930071b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.068+0000 7f7936800640 1 -- 172.21.15.83:0/3616935379 shutdown_connections 2024-04-23T14:51:56.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.068+0000 7f7936800640 1 -- 172.21.15.83:0/3616935379 wait complete. 2024-04-23T14:51:56.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:51:56.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.375+0000 7f42e5e00640 1 -- 172.21.15.83:0/1192559710 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8035020 con 0x7f42e806d9c0 2024-04-23T14:51:56.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.376+0000 7f42ed400640 1 -- 172.21.15.83:0/1192559710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42e806d9c0 msgr2=0x7f42e806dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.376+0000 7f42ed400640 1 --2- 172.21.15.83:0/1192559710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42e806d9c0 0x7f42e806dda0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f42d8009900 tx=0x7f42d802a480 comp rx=0 tx=0).stop 2024-04-23T14:51:56.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.376+0000 7f42ed400640 1 -- 172.21.15.83:0/1192559710 shutdown_connections 2024-04-23T14:51:56.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.376+0000 7f42ed400640 1 --2- 172.21.15.83:0/1192559710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42e811b6a0 0x7f42e811da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.376+0000 7f42ed400640 1 --2- 172.21.15.83:0/1192559710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e806e2e0 0x7f42e811b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.376+0000 7f42ed400640 1 --2- 172.21.15.83:0/1192559710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42e806d9c0 0x7f42e806dda0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.376+0000 7f42ed400640 1 -- 172.21.15.83:0/1192559710 >> 172.21.15.83:0/1192559710 conn(0x7f42e806d030 msgr2=0x7f42e810b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.377+0000 7f42ed400640 1 -- 172.21.15.83:0/1192559710 shutdown_connections 2024-04-23T14:51:56.384 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:56 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2627848957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:51:56.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.380+0000 7f42ed400640 1 -- 172.21.15.83:0/1192559710 wait complete. 2024-04-23T14:51:56.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.382+0000 7f42ed400640 1 Processor -- start 2024-04-23T14:51:56.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.382+0000 7f42ed400640 1 -- start start 2024-04-23T14:51:56.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.383+0000 7f42ed400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e806d9c0 0x7f42e81a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.383+0000 7f42ed400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42e806e2e0 0x7f42e81a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.383+0000 7f42ed400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42e811b6a0 0x7f42e81abd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.383+0000 7f42ed400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81211b0 con 0x7f42e806d9c0 2024-04-23T14:51:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.383+0000 7f42ed400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8121030 con 0x7f42e811b6a0 2024-04-23T14:51:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.383+0000 7f42ed400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8121330 con 0x7f42e806e2e0 2024-04-23T14:51:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.383+0000 7f42dde00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42e806e2e0 0x7f42e81a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.383+0000 7f42e7200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42e811b6a0 0x7f42e81abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.384+0000 7f42e7200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42e811b6a0 0x7f42e81abd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33198/0 (socket says 172.21.15.83:33198) 2024-04-23T14:51:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.384+0000 7f42dde00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42e806e2e0 0x7f42e81a79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40918/0 (socket says 172.21.15.83:40918) 2024-04-23T14:51:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.384+0000 7f42e7200640 1 -- 172.21.15.83:0/2958660069 learned_addr learned my addr 172.21.15.83:0/2958660069 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.384+0000 7f42e7200640 1 -- 172.21.15.83:0/2958660069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42e806e2e0 msgr2=0x7f42e81a79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.384+0000 7f42e7200640 1 --2- 172.21.15.83:0/2958660069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42e806e2e0 0x7f42e81a79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.384+0000 7f42e7200640 1 -- 172.21.15.83:0/2958660069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e806d9c0 msgr2=0x7f42e81a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.384+0000 7f42e7200640 1 --2- 172.21.15.83:0/2958660069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e806d9c0 0x7f42e81a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.384+0000 7f42e7200640 1 -- 172.21.15.83:0/2958660069 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e81ac460 con 0x7f42e811b6a0 2024-04-23T14:51:56.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.385+0000 7f42e7200640 1 --2- 172.21.15.83:0/2958660069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42e811b6a0 0x7f42e81abd30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f42e000cce0 tx=0x7f42e000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.389+0000 7f42dfe00640 1 -- 172.21.15.83:0/2958660069 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e000f450 con 0x7f42e811b6a0 2024-04-23T14:51:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.389+0000 7f42dfe00640 1 -- 172.21.15.83:0/2958660069 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42e000fad0 con 0x7f42e811b6a0 2024-04-23T14:51:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.389+0000 7f42dfe00640 1 -- 172.21.15.83:0/2958660069 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e0018c10 con 0x7f42e811b6a0 2024-04-23T14:51:56.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.389+0000 7f42ed400640 1 -- 172.21.15.83:0/2958660069 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e81ac750 con 0x7f42e811b6a0 2024-04-23T14:51:56.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.389+0000 7f42ed400640 1 -- 172.21.15.83:0/2958660069 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e807a260 con 0x7f42e811b6a0 2024-04-23T14:51:56.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.395+0000 7f42f023d740 1 Processor -- start 2024-04-23T14:51:56.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.399+0000 7f42dfe00640 1 -- 172.21.15.83:0/2958660069 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f42e000f5f0 con 0x7f42e811b6a0 2024-04-23T14:51:56.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.400+0000 7f42f023d740 1 -- start start 2024-04-23T14:51:56.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.400+0000 7f42f023d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdca9950 0x555ccdca9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.401+0000 7f42f023d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcaa270 0x555ccdcb4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.401+0000 7f42f023d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdcb4940 0x555ccdcb6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.401+0000 7f42f023d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555ccdc0f1d0 con 0x555ccdcaa270 2024-04-23T14:51:56.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.401+0000 7f42f023d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555ccdbb5450 con 0x555ccdcb4940 2024-04-23T14:51:56.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.401+0000 7f42f023d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555ccdc0b190 con 0x555ccdca9950 2024-04-23T14:51:56.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.401+0000 7f42dde00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdca9950 0x555ccdca9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.401+0000 7f42dde00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdca9950 0x555ccdca9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40924/0 (socket says 172.21.15.83:40924) 2024-04-23T14:51:56.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.401+0000 7f42dde00640 1 -- 172.21.15.83:0/575018954 learned_addr learned my addr 172.21.15.83:0/575018954 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.401+0000 7f42dde00640 1 --2- 172.21.15.83:0/575018954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdcb4940 0x555ccdcb6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.402+0000 7f42dde00640 1 -- 172.21.15.83:0/575018954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdca9950 msgr2=0x555ccdca9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.402+0000 7f42dde00640 1 --2- 172.21.15.83:0/575018954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdca9950 0x555ccdca9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.402+0000 7f42dde00640 1 -- 172.21.15.83:0/575018954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcaa270 msgr2=0x555ccdcb4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.402+0000 7f42dde00640 1 --2- 172.21.15.83:0/575018954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcaa270 0x555ccdcb4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.402+0000 7f42dde00640 1 -- 172.21.15.83:0/575018954 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ccdc72310 con 0x555ccdcb4940 2024-04-23T14:51:56.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.402+0000 7f42dde00640 1 --2- 172.21.15.83:0/575018954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdcb4940 0x555ccdcb6d30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f42d00149c0 tx=0x7f42d00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7615c327c529a2ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.403+0000 7f42dfe00640 1 --2- 172.21.15.83:0/2958660069 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f42cc0a3da0 0x7f42cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.404+0000 7f42dfe00640 1 -- 172.21.15.83:0/2958660069 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f42e00d10e0 con 0x7f42e811b6a0 2024-04-23T14:51:56.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.404+0000 7f42e6800640 1 --2- 172.21.15.83:0/2958660069 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f42cc0a3da0 0x7f42cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.405+0000 7f42d7400640 1 -- 172.21.15.83:0/575018954 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d00142d0 con 0x555ccdcb4940 2024-04-23T14:51:56.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.406+0000 7f42d7400640 1 -- 172.21.15.83:0/575018954 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d0011cc0 con 0x555ccdcb4940 2024-04-23T14:51:56.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.407+0000 7f42e6800640 1 --2- 172.21.15.83:0/2958660069 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f42cc0a3da0 0x7f42cc0a6260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f42d80096c0 tx=0x7f42d8005e50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.407+0000 7f42f023d740 1 -- 172.21.15.83:0/575018954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdcb4940 msgr2=0x555ccdcb6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.407+0000 7f42f023d740 1 --2- 172.21.15.83:0/575018954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdcb4940 0x555ccdcb6d30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f42d00149c0 tx=0x7f42d00046c0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.408+0000 7f42f023d740 1 -- 172.21.15.83:0/575018954 shutdown_connections 2024-04-23T14:51:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.408+0000 7f42f023d740 1 --2- 172.21.15.83:0/575018954 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdcb4940 0x555ccdcb6d30 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f42d00149c0 tx=0x7f42d00046c0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.408+0000 7f42f023d740 1 --2- 172.21.15.83:0/575018954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcaa270 0x555ccdcb4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.408+0000 7f42f023d740 1 --2- 172.21.15.83:0/575018954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdca9950 0x555ccdca9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.408+0000 7f42f023d740 1 -- 172.21.15.83:0/575018954 >> 172.21.15.83:0/575018954 conn(0x555ccdca5c70 msgr2=0x555ccdca6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.410+0000 7f42f023d740 1 -- 172.21.15.83:0/575018954 shutdown_connections 2024-04-23T14:51:56.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.410+0000 7f42f023d740 1 -- 172.21.15.83:0/575018954 wait complete. 2024-04-23T14:51:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.411+0000 7f42f023d740 1 -- start start 2024-04-23T14:51:56.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.411+0000 7f42f023d740 1 Processor -- start 2024-04-23T14:51:56.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.417+0000 7f42f023d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdca9950 0x555ccdcb18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.417+0000 7f42f023d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcaa270 0x555ccdcb1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.417+0000 7f42f023d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdcb2360 0x555ccdcc0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.417+0000 7f42f023d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555ccdcb7810 con 0x555ccdca9950 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.417+0000 7f42f023d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555ccdbc8810 con 0x555ccdcb2360 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.417+0000 7f42f023d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555ccdcb7990 con 0x555ccdcaa270 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.418+0000 7f42f023d740 10 client.0 _ll_register_callbacks cb 0x555ccdb96140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.418+0000 7f42dde00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcaa270 0x555ccdcb1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.418+0000 7f42dde00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcaa270 0x555ccdcb1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40928/0 (socket says 172.21.15.83:40928) 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.418+0000 7f42dde00640 1 -- 172.21.15.83:0/1537992869 learned_addr learned my addr 172.21.15.83:0/1537992869 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.419+0000 7f42dde00640 1 -- 172.21.15.83:0/1537992869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdcb2360 msgr2=0x555ccdcc0d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.419+0000 7f42dde00640 1 --2- 172.21.15.83:0/1537992869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdcb2360 0x555ccdcc0d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.419+0000 7f42dde00640 1 -- 172.21.15.83:0/1537992869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdca9950 msgr2=0x555ccdcb18e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:56.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.419+0000 7f42dde00640 1 --2- 172.21.15.83:0/1537992869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdca9950 0x555ccdcb18e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.419+0000 7f42dde00640 1 -- 172.21.15.83:0/1537992869 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ccdbd93f0 con 0x555ccdcaa270 2024-04-23T14:51:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.420+0000 7f42dde00640 1 --2- 172.21.15.83:0/1537992869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcaa270 0x555ccdcb1e20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f42d001c000 tx=0x7f42d000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89e3987852d1d3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.420+0000 7f42cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:51:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.421+0000 7f42cbe00640 1 -- 172.21.15.83:0/1537992869 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d003e060 con 0x555ccdcaa270 2024-04-23T14:51:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.421+0000 7f42f023d740 1 -- 172.21.15.83:0/1537992869 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555ccdc1cbf0 con 0x555ccdcaa270 2024-04-23T14:51:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.421+0000 7f42cbe00640 1 -- 172.21.15.83:0/1537992869 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f42d00140d0 con 0x555ccdcaa270 2024-04-23T14:51:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.421+0000 7f42cbe00640 1 -- 172.21.15.83:0/1537992869 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42d00142d0 con 0x555ccdcaa270 2024-04-23T14:51:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.421+0000 7f42f023d740 10 client.26005 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:56.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.421+0000 7f42f023d740 1 -- 172.21.15.83:0/1537992869 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555ccd9213f0 con 0x555ccdcaa270 2024-04-23T14:51:56.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.422+0000 7f42cbe00640 1 -- 172.21.15.83:0/1537992869 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555ccdc1cbf0 con 0x555ccdcaa270 2024-04-23T14:51:56.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.424+0000 7f42cbe00640 1 -- 172.21.15.83:0/1537992869 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f42d0004040 con 0x555ccdcaa270 2024-04-23T14:51:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.424+0000 7f42f023d740 10 client.26005 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.424+0000 7f42f023d740 10 client.26005 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:51:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.424+0000 7f42f023d740 20 client.26005 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.425+0000 7f42f023d740 1 --2- 172.21.15.83:0/1537992869 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555ccdcc5560 0x555ccdcc7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.425+0000 7f42f023d740 4 client.26005 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:51:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.425+0000 7f42f023d740 1 -- 172.21.15.83:0/1537992869 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ccdc16a60 con 0x555ccdcc5560 2024-04-23T14:51:56.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.425+0000 7f42e6800640 1 --2- 172.21.15.83:0/1537992869 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555ccdcc5560 0x555ccdcc7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.427+0000 7f42cbe00640 10 client.26005 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:51:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.427+0000 7f42e6800640 1 --2- 172.21.15.83:0/1537992869 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555ccdcc5560 0x555ccdcc7a20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7898811766261b3d server_cookie=82ff342cf57a6dec in_seq=0 out_seq=0 2024-04-23T14:51:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.428+0000 7f42cbe00640 1 -- 172.21.15.83:0/1537992869 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ccdc16a60 con 0x555ccdcc5560 2024-04-23T14:51:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.428+0000 7f42cbe00640 10 client.26005 handle_command_reply: tid=0 2024-04-23T14:51:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.428+0000 7f42f023d740 1 client.26005 shutdown 2024-04-23T14:51:56.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.429+0000 7f42f023d740 1 -- 172.21.15.83:0/1537992869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcaa270 msgr2=0x555ccdcb1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.429+0000 7f42f023d740 1 --2- 172.21.15.83:0/1537992869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcaa270 0x555ccdcb1e20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f42d001c000 tx=0x7f42d000ce30 comp rx=0 tx=0).stop 2024-04-23T14:51:56.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.429+0000 7f42f023d740 1 -- 172.21.15.83:0/1537992869 shutdown_connections 2024-04-23T14:51:56.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.429+0000 7f42f023d740 1 --2- 172.21.15.83:0/1537992869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdcb2360 0x555ccdcc0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.429+0000 7f42f023d740 1 --2- 172.21.15.83:0/1537992869 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555ccdcc5560 0x555ccdcc7a20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.429+0000 7f42f023d740 1 --2- 172.21.15.83:0/1537992869 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcaa270 0x555ccdcb1e20 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f42d001c000 tx=0x7f42d000ce30 comp rx=0 tx=0).stop 2024-04-23T14:51:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.429+0000 7f42f023d740 1 --2- 172.21.15.83:0/1537992869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdca9950 0x555ccdcb18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.429+0000 7f42f023d740 1 -- 172.21.15.83:0/1537992869 >> 172.21.15.83:0/1537992869 conn(0x555ccdca5c70 msgr2=0x555ccdcaba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.429+0000 7f42cbe00640 0 client.26005 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:51:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.430+0000 7f42f023d740 1 -- 172.21.15.83:0/1537992869 shutdown_connections 2024-04-23T14:51:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.430+0000 7f42f023d740 1 -- 172.21.15.83:0/1537992869 wait complete. 2024-04-23T14:51:56.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.430+0000 7f42f023d740 20 client.26005 trim_cache size 0 max 16384 2024-04-23T14:51:56.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.469+0000 7f42f023d740 1 Processor -- start 2024-04-23T14:51:56.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.469+0000 7f42f023d740 1 -- start start 2024-04-23T14:51:56.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.469+0000 7f42f023d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdca9950 0x555ccdcac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.469+0000 7f42f023d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb2360 0x555ccdccd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.469+0000 7f42f023d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcb4120 0x555ccdccd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.469+0000 7f42f023d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42e807c6a0 con 0x555ccdcb4120 2024-04-23T14:51:56.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.469+0000 7f42f023d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f42e807cd70 con 0x555ccdca9950 2024-04-23T14:51:56.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.469+0000 7f42f023d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42e807a650 con 0x555ccdcb2360 2024-04-23T14:51:56.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.471+0000 7f42dde00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb2360 0x555ccdccd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.471+0000 7f42dde00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb2360 0x555ccdccd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40930/0 (socket says 172.21.15.83:40930) 2024-04-23T14:51:56.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.471+0000 7f42dde00640 1 -- 172.21.15.83:0/3776039286 learned_addr learned my addr 172.21.15.83:0/3776039286 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.472+0000 7f42e7200640 1 --2- 172.21.15.83:0/3776039286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcb4120 0x555ccdccd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.473+0000 7f42dde00640 1 -- 172.21.15.83:0/3776039286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdca9950 msgr2=0x555ccdcac0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:56.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.473+0000 7f42dde00640 1 --2- 172.21.15.83:0/3776039286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdca9950 0x555ccdcac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.473+0000 7f42dde00640 1 -- 172.21.15.83:0/3776039286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcb4120 msgr2=0x555ccdccd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.473+0000 7f42dde00640 1 --2- 172.21.15.83:0/3776039286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcb4120 0x555ccdccd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.473+0000 7f42dde00640 1 -- 172.21.15.83:0/3776039286 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ccd9eb7d0 con 0x555ccdcb2360 2024-04-23T14:51:56.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.474+0000 7f42e7200640 1 --2- 172.21.15.83:0/3776039286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcb4120 0x555ccdccd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:51:56.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.475+0000 7f42dde00640 1 --2- 172.21.15.83:0/3776039286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb2360 0x555ccdccd2f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f42d0014800 tx=0x7f42d0012730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b53ca52cec108d1a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.476+0000 7f42d6a00640 1 -- 172.21.15.83:0/3776039286 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d00142d0 con 0x555ccdcb2360 2024-04-23T14:51:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.476+0000 7f42d6a00640 1 -- 172.21.15.83:0/3776039286 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555ccdc1cbf0 con 0x555ccdcb2360 2024-04-23T14:51:56.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.477+0000 7f42d6a00640 1 -- 172.21.15.83:0/3776039286 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d00142d0 con 0x555ccdcb2360 2024-04-23T14:51:56.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.480+0000 7f42f023d740 1 -- 172.21.15.83:0/3776039286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb2360 msgr2=0x555ccdccd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.480+0000 7f42f023d740 1 --2- 172.21.15.83:0/3776039286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb2360 0x555ccdccd2f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f42d0014800 tx=0x7f42d0012730 comp rx=0 tx=0).stop 2024-04-23T14:51:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.481+0000 7f42f023d740 1 -- 172.21.15.83:0/3776039286 shutdown_connections 2024-04-23T14:51:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.481+0000 7f42f023d740 1 --2- 172.21.15.83:0/3776039286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcb4120 0x555ccdccd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.481+0000 7f42f023d740 1 --2- 172.21.15.83:0/3776039286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb2360 0x555ccdccd2f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.481+0000 7f42f023d740 1 --2- 172.21.15.83:0/3776039286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdca9950 0x555ccdcac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.481+0000 7f42f023d740 1 -- 172.21.15.83:0/3776039286 >> 172.21.15.83:0/3776039286 conn(0x555ccdca5c70 msgr2=0x555ccdcacb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.481+0000 7f42f023d740 1 -- 172.21.15.83:0/3776039286 shutdown_connections 2024-04-23T14:51:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.481+0000 7f42f023d740 1 -- 172.21.15.83:0/3776039286 wait complete. 2024-04-23T14:51:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.482+0000 7f42f023d740 1 -- start start 2024-04-23T14:51:56.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.482+0000 7f42f023d740 1 Processor -- start 2024-04-23T14:51:56.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.485+0000 7f42f023d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdca9950 0x555ccdcaec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.485+0000 7f42f023d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcb2360 0x555ccdcaf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.486+0000 7f42f023d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb4120 0x555ccdcadb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.486+0000 7f42f023d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42e81147f0 con 0x555ccdcb2360 2024-04-23T14:51:56.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.486+0000 7f42f023d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8115c60 con 0x555ccdca9950 2024-04-23T14:51:56.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.486+0000 7f42f023d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f42e8114ec0 con 0x555ccdcb4120 2024-04-23T14:51:56.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.486+0000 7f42f023d740 10 client.0 _ll_register_callbacks cb 0x7f42d000cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:56.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.486+0000 7f42e6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdca9950 0x555ccdcaec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.486+0000 7f42e7200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb4120 0x555ccdcadb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.486+0000 7f42e7200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb4120 0x555ccdcadb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40934/0 (socket says 172.21.15.83:40934) 2024-04-23T14:51:56.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.486+0000 7f42e7200640 1 -- 172.21.15.83:0/2115386054 learned_addr learned my addr 172.21.15.83:0/2115386054 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.486+0000 7f42dde00640 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcb2360 0x555ccdcaf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.487+0000 7f42e6800640 1 -- 172.21.15.83:0/2115386054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb4120 msgr2=0x555ccdcadb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.487+0000 7f42e6800640 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb4120 0x555ccdcadb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.487+0000 7f42e6800640 1 -- 172.21.15.83:0/2115386054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcb2360 msgr2=0x555ccdcaf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.487+0000 7f42e6800640 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcb2360 0x555ccdcaf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.487+0000 7f42e6800640 1 -- 172.21.15.83:0/2115386054 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ccdcbd250 con 0x555ccdca9950 2024-04-23T14:51:56.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.487+0000 7f42dde00640 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcb2360 0x555ccdcaf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:51:56.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.487+0000 7f42e6800640 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdca9950 0x555ccdcaec80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f42d8000c00 tx=0x7f42d8031af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e18d1cfc68c31c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.489+0000 7f42cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:51:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.489+0000 7f42cbe00640 1 -- 172.21.15.83:0/2115386054 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d8035040 con 0x555ccdca9950 2024-04-23T14:51:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.489+0000 7f42cbe00640 1 -- 172.21.15.83:0/2115386054 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f42d804c6a0 con 0x555ccdca9950 2024-04-23T14:51:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.489+0000 7f42f023d740 1 -- 172.21.15.83:0/2115386054 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555ccdc1cbf0 con 0x555ccdca9950 2024-04-23T14:51:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.490+0000 7f42cbe00640 1 -- 172.21.15.83:0/2115386054 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42d805a050 con 0x555ccdca9950 2024-04-23T14:51:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.490+0000 7f42f023d740 10 client.26054 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:56.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.490+0000 7f42f023d740 1 -- 172.21.15.83:0/2115386054 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555ccdd2aef0 con 0x555ccdca9950 2024-04-23T14:51:56.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.491+0000 7f42cbe00640 1 -- 172.21.15.83:0/2115386054 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555ccdc1cbf0 con 0x555ccdca9950 2024-04-23T14:51:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.492+0000 7f42cbe00640 1 -- 172.21.15.83:0/2115386054 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f42d80555d0 con 0x555ccdca9950 2024-04-23T14:51:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.492+0000 7f42f023d740 10 client.26054 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.492+0000 7f42f023d740 10 client.26054 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:51:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.492+0000 7f42f023d740 20 client.26054 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.492+0000 7f42f023d740 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555ccdd2b640 0x555ccdd2da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.492+0000 7f42f023d740 4 client.26054 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:51:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.492+0000 7f42f023d740 1 -- 172.21.15.83:0/2115386054 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ccdb924d0 con 0x555ccdd2b640 2024-04-23T14:51:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.492+0000 7f42dde00640 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555ccdd2b640 0x555ccdd2da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.494+0000 7f42dde00640 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555ccdd2b640 0x555ccdd2da60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ee1c10473098751 server_cookie=a30ab3fe1b35b5da in_seq=0 out_seq=0 2024-04-23T14:51:56.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.494+0000 7f42cbe00640 10 client.26054 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:51:56.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.494+0000 7f42cbe00640 1 -- 172.21.15.83:0/2115386054 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ccdb924d0 con 0x555ccdd2b640 2024-04-23T14:51:56.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.494+0000 7f42cbe00640 10 client.26054 handle_command_reply: tid=0 2024-04-23T14:51:56.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.494+0000 7f42f023d740 1 client.26054 shutdown 2024-04-23T14:51:56.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.494+0000 7f42f023d740 1 -- 172.21.15.83:0/2115386054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdca9950 msgr2=0x555ccdcaec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.494+0000 7f42f023d740 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdca9950 0x555ccdcaec80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f42d8000c00 tx=0x7f42d8031af0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.495+0000 7f42f023d740 1 -- 172.21.15.83:0/2115386054 shutdown_connections 2024-04-23T14:51:56.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.495+0000 7f42f023d740 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x555ccdd2b640 0x555ccdd2da60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.495+0000 7f42f023d740 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555ccdcb4120 0x555ccdcadb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.495+0000 7f42f023d740 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ccdcb2360 0x555ccdcaf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.495+0000 7f42f023d740 1 --2- 172.21.15.83:0/2115386054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ccdca9950 0x555ccdcaec80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.495+0000 7f42f023d740 1 -- 172.21.15.83:0/2115386054 >> 172.21.15.83:0/2115386054 conn(0x555ccdca5c70 msgr2=0x555ccdcb3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.495+0000 7f42cbe00640 0 client.26054 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:51:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.495+0000 7f42f023d740 1 -- 172.21.15.83:0/2115386054 shutdown_connections 2024-04-23T14:51:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.495+0000 7f42f023d740 1 -- 172.21.15.83:0/2115386054 wait complete. 2024-04-23T14:51:56.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.496+0000 7f42f023d740 20 client.26054 trim_cache size 0 max 16384 2024-04-23T14:51:56.498 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:51:56.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.498+0000 7f42ed400640 1 -- 172.21.15.83:0/2958660069 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f42cc0a3da0 msgr2=0x7f42cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.498+0000 7f42ed400640 1 --2- 172.21.15.83:0/2958660069 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f42cc0a3da0 0x7f42cc0a6260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f42d80096c0 tx=0x7f42d8005e50 comp rx=0 tx=0).stop 2024-04-23T14:51:56.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.498+0000 7f42ed400640 1 -- 172.21.15.83:0/2958660069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42e811b6a0 msgr2=0x7f42e81abd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.498+0000 7f42ed400640 1 --2- 172.21.15.83:0/2958660069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42e811b6a0 0x7f42e81abd30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f42e000cce0 tx=0x7f42e000a620 comp rx=0 tx=0).stop 2024-04-23T14:51:56.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.498+0000 7f42ed400640 1 -- 172.21.15.83:0/2958660069 shutdown_connections 2024-04-23T14:51:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.498+0000 7f42ed400640 1 --2- 172.21.15.83:0/2958660069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42e811b6a0 0x7f42e81abd30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.498+0000 7f42ed400640 1 --2- 172.21.15.83:0/2958660069 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42e806e2e0 0x7f42e81a79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.498+0000 7f42ed400640 1 --2- 172.21.15.83:0/2958660069 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f42cc0a3da0 0x7f42cc0a6260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.498+0000 7f42ed400640 1 --2- 172.21.15.83:0/2958660069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42e806d9c0 0x7f42e81a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.498+0000 7f42ed400640 1 -- 172.21.15.83:0/2958660069 >> 172.21.15.83:0/2958660069 conn(0x7f42e806d030 msgr2=0x7f42e811bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.499+0000 7f42ed400640 1 -- 172.21.15.83:0/2958660069 shutdown_connections 2024-04-23T14:51:56.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.499+0000 7f42ed400640 1 -- 172.21.15.83:0/2958660069 wait complete. 2024-04-23T14:51:56.533 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:51:56.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:56 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2627848957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:51:56.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:56 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2627848957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:51:56.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.872+0000 7f0588a00640 1 -- 172.21.15.83:0/2213614946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0584117130 msgr2=0x7f0584119520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.872+0000 7f0588a00640 1 --2- 172.21.15.83:0/2213614946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0584117130 0x7f0584119520 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f057c009be0 tx=0x7f057c02a6e0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.872+0000 7f0588a00640 1 -- 172.21.15.83:0/2213614946 shutdown_connections 2024-04-23T14:51:56.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.872+0000 7f0588a00640 1 --2- 172.21.15.83:0/2213614946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0584117130 0x7f0584119520 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.872+0000 7f0588a00640 1 --2- 172.21.15.83:0/2213614946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05841095d0 0x7f0584109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.872+0000 7f0588a00640 1 --2- 172.21.15.83:0/2213614946 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f058410c770 0x7f0584109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.872+0000 7f0588a00640 1 -- 172.21.15.83:0/2213614946 >> 172.21.15.83:0/2213614946 conn(0x7f058406d3e0 msgr2=0x7f058406d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.873+0000 7f0588a00640 1 -- 172.21.15.83:0/2213614946 shutdown_connections 2024-04-23T14:51:56.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.873+0000 7f0588a00640 1 -- 172.21.15.83:0/2213614946 wait complete. 2024-04-23T14:51:56.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.875+0000 7f0588a00640 1 Processor -- start 2024-04-23T14:51:56.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.875+0000 7f0588a00640 1 -- start start 2024-04-23T14:51:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.876+0000 7f0588a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05841095d0 0x7f05841a32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.876+0000 7f0588a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f058410c770 0x7f05841a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.876+0000 7f0588a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0584117130 0x7f05841a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.876+0000 7f0588a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f058411cde0 con 0x7f0584117130 2024-04-23T14:51:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.876+0000 7f0588a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f058411cc60 con 0x7f05841095d0 2024-04-23T14:51:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.876+0000 7f0588a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f058411cf60 con 0x7f058410c770 2024-04-23T14:51:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.876+0000 7f0583400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05841095d0 0x7f05841a32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.876+0000 7f0582a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f058410c770 0x7f05841a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.876+0000 7f0583400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05841095d0 0x7f05841a32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33240/0 (socket says 172.21.15.83:33240) 2024-04-23T14:51:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.876+0000 7f0583400640 1 -- 172.21.15.83:0/2181772158 learned_addr learned my addr 172.21.15.83:0/2181772158 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.877+0000 7f0583e00640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0584117130 0x7f05841a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.877+0000 7f0583400640 1 -- 172.21.15.83:0/2181772158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f058410c770 msgr2=0x7f05841a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.877+0000 7f0583400640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f058410c770 0x7f05841a37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.877+0000 7f0583400640 1 -- 172.21.15.83:0/2181772158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0584117130 msgr2=0x7f05841a7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.877+0000 7f0583400640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0584117130 0x7f05841a7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.877+0000 7f0583400640 1 -- 172.21.15.83:0/2181772158 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05841a8290 con 0x7f05841095d0 2024-04-23T14:51:56.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.877+0000 7f0583e00640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0584117130 0x7f05841a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:51:56.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.878+0000 7f0583400640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05841095d0 0x7f05841a32a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f057400b450 tx=0x7f057400b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.878+0000 7f057be00640 1 -- 172.21.15.83:0/2181772158 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574015020 con 0x7f05841095d0 2024-04-23T14:51:56.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.878+0000 7f057be00640 1 -- 172.21.15.83:0/2181772158 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0574011d10 con 0x7f05841095d0 2024-04-23T14:51:56.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.878+0000 7f057be00640 1 -- 172.21.15.83:0/2181772158 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057400f2e0 con 0x7f05841095d0 2024-04-23T14:51:56.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.878+0000 7f0588a00640 1 -- 172.21.15.83:0/2181772158 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05841a8580 con 0x7f05841095d0 2024-04-23T14:51:56.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.879+0000 7f0588a00640 1 -- 172.21.15.83:0/2181772158 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05841afe10 con 0x7f05841095d0 2024-04-23T14:51:56.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.883+0000 7f058cdda740 1 Processor -- start 2024-04-23T14:51:56.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.883+0000 7f058cdda740 1 -- start start 2024-04-23T14:51:56.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.884+0000 7f058cdda740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f9a03c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.884+0000 7f057be00640 1 -- 172.21.15.83:0/2181772158 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0574018020 con 0x7f05841095d0 2024-04-23T14:51:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.884+0000 7f058cdda740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 0x5615f9a16e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.884+0000 7f058cdda740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a19730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.884+0000 7f058cdda740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5615f99691d0 con 0x5615f9a04240 2024-04-23T14:51:56.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.884+0000 7f058cdda740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5615f990f450 con 0x5615f9a17340 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.884+0000 7f058cdda740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5615f9965190 con 0x5615f9a03890 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.884+0000 7f0583e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f9a03c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.884+0000 7f0583e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f9a03c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40966/0 (socket says 172.21.15.83:40966) 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.884+0000 7f0582a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a19730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.884+0000 7f0583e00640 1 -- 172.21.15.83:0/932393953 learned_addr learned my addr 172.21.15.83:0/932393953 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.885+0000 7f0582a00640 1 -- 172.21.15.83:0/932393953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 msgr2=0x5615f9a03c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.885+0000 7f0582a00640 1 --2- 172.21.15.83:0/932393953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f9a03c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.885+0000 7f0582a00640 1 -- 172.21.15.83:0/932393953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 msgr2=0x5615f9a16e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.885+0000 7f0582a00640 1 --2- 172.21.15.83:0/932393953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 0x5615f9a16e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.885+0000 7f0582a00640 1 -- 172.21.15.83:0/932393953 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f99cc310 con 0x5615f9a17340 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.885+0000 7f0583e00640 1 --2- 172.21.15.83:0/932393953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f9a03c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.885+0000 7f0582a00640 1 --2- 172.21.15.83:0/932393953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a19730 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0584069d00 tx=0x7f0570011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b4aed3f7b11d991 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.885+0000 7f056fe00640 1 -- 172.21.15.83:0/932393953 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570009db0 con 0x5615f9a17340 2024-04-23T14:51:56.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.885+0000 7f056fe00640 1 -- 172.21.15.83:0/932393953 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05700035f0 con 0x5615f9a17340 2024-04-23T14:51:56.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.885+0000 7f056fe00640 1 -- 172.21.15.83:0/932393953 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05700196b0 con 0x5615f9a17340 2024-04-23T14:51:56.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.887+0000 7f058cdda740 1 -- 172.21.15.83:0/932393953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 msgr2=0x5615f9a19730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.887+0000 7f058cdda740 1 --2- 172.21.15.83:0/932393953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a19730 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0584069d00 tx=0x7f0570011810 comp rx=0 tx=0).stop 2024-04-23T14:51:56.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.887+0000 7f058cdda740 1 -- 172.21.15.83:0/932393953 shutdown_connections 2024-04-23T14:51:56.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.887+0000 7f058cdda740 1 --2- 172.21.15.83:0/932393953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a19730 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.887+0000 7f058cdda740 1 --2- 172.21.15.83:0/932393953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 0x5615f9a16e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.887+0000 7f058cdda740 1 --2- 172.21.15.83:0/932393953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f9a03c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.887+0000 7f058cdda740 1 -- 172.21.15.83:0/932393953 >> 172.21.15.83:0/932393953 conn(0x5615f99ffe60 msgr2=0x5615f9a00240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.888+0000 7f057be00640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f05680a3de0 0x7f05680a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.888+0000 7f057be00640 1 -- 172.21.15.83:0/2181772158 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f05740cff90 con 0x7f05841095d0 2024-04-23T14:51:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.889+0000 7f0582a00640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f05680a3de0 0x7f05680a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.890+0000 7f058cdda740 1 -- 172.21.15.83:0/932393953 shutdown_connections 2024-04-23T14:51:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.890+0000 7f0582a00640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f05680a3de0 0x7f05680a62a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0570003f10 tx=0x7f0570009600 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.890+0000 7f058cdda740 1 -- 172.21.15.83:0/932393953 wait complete. 2024-04-23T14:51:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.890+0000 7f058cdda740 1 -- start start 2024-04-23T14:51:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.891+0000 7f058cdda740 1 Processor -- start 2024-04-23T14:51:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.891+0000 7f058cdda740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f9a16de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.891+0000 7f058cdda740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 0x5615f9a140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.891+0000 7f058cdda740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.891+0000 7f058cdda740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5615f9a1a1a0 con 0x5615f9a04240 2024-04-23T14:51:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.891+0000 7f058cdda740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5615f9922810 con 0x5615f9a17340 2024-04-23T14:51:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.891+0000 7f058cdda740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5615f9a1a320 con 0x5615f9a03890 2024-04-23T14:51:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.892+0000 7f058cdda740 10 client.0 _ll_register_callbacks cb 0x5615f98f0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.892+0000 7f0583400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f9a16de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.892+0000 7f0583400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f9a16de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:40978/0 (socket says 172.21.15.83:40978) 2024-04-23T14:51:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.892+0000 7f0583400640 1 -- 172.21.15.83:0/2091484520 learned_addr learned my addr 172.21.15.83:0/2091484520 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.892+0000 7f0583e00640 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.893+0000 7f0582a00640 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 0x5615f9a140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.893+0000 7f0583400640 1 -- 172.21.15.83:0/2091484520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 msgr2=0x5615f9a147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.893+0000 7f0583400640 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a147c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.893+0000 7f0583400640 1 -- 172.21.15.83:0/2091484520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 msgr2=0x5615f9a140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.894+0000 7f0583400640 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 0x5615f9a140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.894+0000 7f0583400640 1 -- 172.21.15.83:0/2091484520 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f99333f0 con 0x5615f9a03890 2024-04-23T14:51:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.894+0000 7f0583e00640 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.894+0000 7f0582a00640 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 0x5615f9a140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.894+0000 7f0583400640 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f9a16de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f057400bd50 tx=0x7f057409bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0140c58281f5b3d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.894+0000 7f056cc00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:51:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.895+0000 7f056cc00640 1 -- 172.21.15.83:0/2091484520 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05740b5060 con 0x5615f9a03890 2024-04-23T14:51:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.895+0000 7f056cc00640 1 -- 172.21.15.83:0/2091484520 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f05740986c0 con 0x5615f9a03890 2024-04-23T14:51:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.895+0000 7f058cdda740 1 -- 172.21.15.83:0/2091484520 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05700035f0 con 0x5615f9a03890 2024-04-23T14:51:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.896+0000 7f056cc00640 1 -- 172.21.15.83:0/2091484520 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05740ba050 con 0x5615f9a03890 2024-04-23T14:51:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.896+0000 7f058cdda740 10 client.26023 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:56.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.896+0000 7f058cdda740 1 -- 172.21.15.83:0/2091484520 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5615f967b3f0 con 0x5615f9a03890 2024-04-23T14:51:56.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.897+0000 7f056cc00640 1 -- 172.21.15.83:0/2091484520 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05700035f0 con 0x5615f9a03890 2024-04-23T14:51:56.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.898+0000 7f056cc00640 1 -- 172.21.15.83:0/2091484520 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f05740a4830 con 0x5615f9a03890 2024-04-23T14:51:56.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.898+0000 7f058cdda740 10 client.26023 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:56.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.898+0000 7f058cdda740 10 client.26023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:56.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.898+0000 7f058cdda740 20 client.26023 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:56.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.898+0000 7f058cdda740 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5615f9a27500 0x5615f9a29980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.898+0000 7f058cdda740 4 client.26023 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:51:56.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.898+0000 7f058cdda740 1 -- 172.21.15.83:0/2091484520 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615f9976bf0 con 0x5615f9a27500 2024-04-23T14:51:56.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.899+0000 7f0582a00640 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5615f9a27500 0x5615f9a29980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.902+0000 7f0582a00640 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5615f9a27500 0x5615f9a29980 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5142927e8e366e04 server_cookie=7b0e4e6964446d77 in_seq=0 out_seq=0 2024-04-23T14:51:56.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.902+0000 7f056cc00640 10 client.26023 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:56.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.903+0000 7f056cc00640 1 -- 172.21.15.83:0/2091484520 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615f9976bf0 con 0x5615f9a27500 2024-04-23T14:51:56.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.903+0000 7f056cc00640 10 client.26023 handle_command_reply: tid=0 2024-04-23T14:51:56.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.903+0000 7f058cdda740 1 client.26023 shutdown 2024-04-23T14:51:56.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.903+0000 7f058cdda740 1 -- 172.21.15.83:0/2091484520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 msgr2=0x5615f9a16de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.903+0000 7f058cdda740 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f9a16de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f057400bd50 tx=0x7f057409bb90 comp rx=0 tx=0).stop 2024-04-23T14:51:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.904+0000 7f058cdda740 1 -- 172.21.15.83:0/2091484520 shutdown_connections 2024-04-23T14:51:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.904+0000 7f058cdda740 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5615f9a27500 0x5615f9a29980 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.904+0000 7f058cdda740 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a147c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.904+0000 7f058cdda740 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 0x5615f9a140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.904+0000 7f056cc00640 0 client.26023 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.904+0000 7f058cdda740 1 --2- 172.21.15.83:0/2091484520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f9a16de0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.904+0000 7f058cdda740 1 -- 172.21.15.83:0/2091484520 >> 172.21.15.83:0/2091484520 conn(0x5615f99ffe60 msgr2=0x5615f9a05c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.907+0000 7f058cdda740 1 -- 172.21.15.83:0/2091484520 shutdown_connections 2024-04-23T14:51:56.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.907+0000 7f058cdda740 1 -- 172.21.15.83:0/2091484520 wait complete. 2024-04-23T14:51:56.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.908+0000 7f058cdda740 20 client.26023 trim_cache size 0 max 16384 2024-04-23T14:51:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.943+0000 7f058cdda740 1 Processor -- start 2024-04-23T14:51:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.944+0000 7f058cdda740 1 -- start start 2024-04-23T14:51:56.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.944+0000 7f058cdda740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f99fedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.944+0000 7f058cdda740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 0x5615f9a16910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.944+0000 7f058cdda740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a2f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.944+0000 7f058cdda740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05841b6ab0 con 0x5615f9a04240 2024-04-23T14:51:56.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.944+0000 7f058cdda740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05841b4350 con 0x5615f9a17340 2024-04-23T14:51:56.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.944+0000 7f058cdda740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f05841b21c0 con 0x5615f9a03890 2024-04-23T14:51:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.945+0000 7f0583e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a2f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.945+0000 7f0583e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a2f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33268/0 (socket says 172.21.15.83:33268) 2024-04-23T14:51:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.945+0000 7f0583e00640 1 -- 172.21.15.83:0/2291134166 learned_addr learned my addr 172.21.15.83:0/2291134166 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.945+0000 7f0582a00640 1 --2- 172.21.15.83:0/2291134166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 0x5615f9a16910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.945+0000 7f0583400640 1 --2- 172.21.15.83:0/2291134166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f99fedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.946+0000 7f0583e00640 1 -- 172.21.15.83:0/2291134166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 msgr2=0x5615f99fedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.946+0000 7f0583e00640 1 --2- 172.21.15.83:0/2291134166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f99fedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.946+0000 7f0583e00640 1 -- 172.21.15.83:0/2291134166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 msgr2=0x5615f9a16910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.946+0000 7f0583e00640 1 --2- 172.21.15.83:0/2291134166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 0x5615f9a16910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.946+0000 7f0583e00640 1 -- 172.21.15.83:0/2291134166 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f97457d0 con 0x5615f9a17340 2024-04-23T14:51:56.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.948+0000 7f0583e00640 1 --2- 172.21.15.83:0/2291134166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a2f250 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f057c034040 tx=0x7f057c0305e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=930e086fb061e11d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.950+0000 7f056f400640 1 -- 172.21.15.83:0/2291134166 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c041070 con 0x5615f9a17340 2024-04-23T14:51:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.950+0000 7f056f400640 1 -- 172.21.15.83:0/2291134166 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f057c003970 con 0x5615f9a17340 2024-04-23T14:51:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.950+0000 7f056f400640 1 -- 172.21.15.83:0/2291134166 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c036040 con 0x5615f9a17340 2024-04-23T14:51:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.951+0000 7f058cdda740 1 -- 172.21.15.83:0/2291134166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 msgr2=0x5615f9a2f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.951+0000 7f058cdda740 1 --2- 172.21.15.83:0/2291134166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a2f250 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f057c034040 tx=0x7f057c0305e0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.951+0000 7f058cdda740 1 -- 172.21.15.83:0/2291134166 shutdown_connections 2024-04-23T14:51:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.951+0000 7f058cdda740 1 --2- 172.21.15.83:0/2291134166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a2f250 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.951+0000 7f058cdda740 1 --2- 172.21.15.83:0/2291134166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a04240 0x5615f9a16910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.951+0000 7f058cdda740 1 --2- 172.21.15.83:0/2291134166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a03890 0x5615f99fedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.951+0000 7f058cdda740 1 -- 172.21.15.83:0/2291134166 >> 172.21.15.83:0/2291134166 conn(0x5615f99ffe60 msgr2=0x5615f99fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.952+0000 7f058cdda740 1 -- 172.21.15.83:0/2291134166 shutdown_connections 2024-04-23T14:51:56.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.953+0000 7f058cdda740 1 -- 172.21.15.83:0/2291134166 wait complete. 2024-04-23T14:51:56.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.953+0000 7f058cdda740 1 -- start start 2024-04-23T14:51:56.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.953+0000 7f058cdda740 1 Processor -- start 2024-04-23T14:51:56.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.956+0000 7f058cdda740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a03890 0x5615f9a10940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.956+0000 7f058cdda740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a04240 0x5615f9a10e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.956+0000 7f058cdda740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a1e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.956+0000 7f058cdda740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05841b2810 con 0x5615f9a03890 2024-04-23T14:51:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.956+0000 7f058cdda740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f05841b3c80 con 0x5615f9a17340 2024-04-23T14:51:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.956+0000 7f058cdda740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f05841b2ee0 con 0x5615f9a04240 2024-04-23T14:51:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.956+0000 7f058cdda740 10 client.0 _ll_register_callbacks cb 0x7f0574004700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:56.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.957+0000 7f0583e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a1e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.957+0000 7f0583e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a1e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33274/0 (socket says 172.21.15.83:33274) 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.957+0000 7f0583e00640 1 -- 172.21.15.83:0/925745127 learned_addr learned my addr 172.21.15.83:0/925745127 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.957+0000 7f0583e00640 1 -- 172.21.15.83:0/925745127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a04240 msgr2=0x5615f9a10e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.957+0000 7f0583e00640 1 --2- 172.21.15.83:0/925745127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a04240 0x5615f9a10e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.957+0000 7f0583e00640 1 -- 172.21.15.83:0/925745127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a03890 msgr2=0x5615f9a10940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.957+0000 7f0583e00640 1 --2- 172.21.15.83:0/925745127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a03890 0x5615f9a10940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.957+0000 7f0583e00640 1 -- 172.21.15.83:0/925745127 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f9a1eaf0 con 0x5615f9a17340 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.957+0000 7f0583e00640 1 --2- 172.21.15.83:0/925745127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a1e5b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f057c009b30 tx=0x7f057c0304e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd0d79ac0b2c056c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.957+0000 7f056cc00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.958+0000 7f056cc00640 1 -- 172.21.15.83:0/925745127 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c036070 con 0x5615f9a17340 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.958+0000 7f056cc00640 1 -- 172.21.15.83:0/925745127 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f057c0047d0 con 0x5615f9a17340 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.958+0000 7f058cdda740 1 -- 172.21.15.83:0/925745127 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f057c003970 con 0x5615f9a17340 2024-04-23T14:51:56.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.958+0000 7f056cc00640 1 -- 172.21.15.83:0/925745127 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f057c054050 con 0x5615f9a17340 2024-04-23T14:51:56.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.959+0000 7f058cdda740 10 client.26066 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:56.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.959+0000 7f058cdda740 1 -- 172.21.15.83:0/925745127 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5615f9a1fd90 con 0x5615f9a17340 2024-04-23T14:51:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.959+0000 7f056cc00640 1 -- 172.21.15.83:0/925745127 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f057c003970 con 0x5615f9a17340 2024-04-23T14:51:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.960+0000 7f056cc00640 1 -- 172.21.15.83:0/925745127 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f057c03d560 con 0x5615f9a17340 2024-04-23T14:51:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.960+0000 7f058cdda740 10 client.26066 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.960+0000 7f058cdda740 10 client.26066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:56.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.960+0000 7f058cdda740 20 client.26066 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.960+0000 7f058cdda740 1 --2- 172.21.15.83:0/925745127 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5615f9a8df00 0x5615f9a90340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.962+0000 7f058cdda740 4 client.26066 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:51:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.962+0000 7f058cdda740 1 -- 172.21.15.83:0/925745127 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615f9970a60 con 0x5615f9a8df00 2024-04-23T14:51:56.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.962+0000 7f0583400640 1 --2- 172.21.15.83:0/925745127 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5615f9a8df00 0x5615f9a90340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.963+0000 7f0583400640 1 --2- 172.21.15.83:0/925745127 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5615f9a8df00 0x5615f9a90340 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99bd6ac05e63c663 server_cookie=597816acd0527761 in_seq=0 out_seq=0 2024-04-23T14:51:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.963+0000 7f056cc00640 10 client.26066 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:56.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.963+0000 7f056cc00640 1 -- 172.21.15.83:0/925745127 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615f9970a60 con 0x5615f9a8df00 2024-04-23T14:51:56.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.963+0000 7f056cc00640 10 client.26066 handle_command_reply: tid=0 2024-04-23T14:51:56.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.963+0000 7f058cdda740 1 client.26066 shutdown 2024-04-23T14:51:56.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.967+0000 7f058cdda740 1 -- 172.21.15.83:0/925745127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 msgr2=0x5615f9a1e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.967+0000 7f058cdda740 1 --2- 172.21.15.83:0/925745127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a1e5b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f057c009b30 tx=0x7f057c0304e0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.972+0000 7f058cdda740 1 -- 172.21.15.83:0/925745127 shutdown_connections 2024-04-23T14:51:56.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.972+0000 7f058cdda740 1 --2- 172.21.15.83:0/925745127 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5615f9a8df00 0x5615f9a90340 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.972+0000 7f058cdda740 1 --2- 172.21.15.83:0/925745127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615f9a17340 0x5615f9a1e5b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.972+0000 7f058cdda740 1 --2- 172.21.15.83:0/925745127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5615f9a04240 0x5615f9a10e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.972+0000 7f056cc00640 0 client.26066 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:56.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.972+0000 7f058cdda740 1 --2- 172.21.15.83:0/925745127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5615f9a03890 0x5615f9a10940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.972+0000 7f058cdda740 1 -- 172.21.15.83:0/925745127 >> 172.21.15.83:0/925745127 conn(0x5615f99ffe60 msgr2=0x5615f9a05c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.973+0000 7f058cdda740 1 -- 172.21.15.83:0/925745127 shutdown_connections 2024-04-23T14:51:56.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.973+0000 7f058cdda740 1 -- 172.21.15.83:0/925745127 wait complete. 2024-04-23T14:51:56.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.974+0000 7f058cdda740 20 client.26066 trim_cache size 0 max 16384 2024-04-23T14:51:56.976 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:51:56.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.975+0000 7f0588a00640 1 -- 172.21.15.83:0/2181772158 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f05680a3de0 msgr2=0x7f05680a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.975+0000 7f0588a00640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f05680a3de0 0x7f05680a62a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0570003f10 tx=0x7f0570009600 comp rx=0 tx=0).stop 2024-04-23T14:51:56.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.975+0000 7f0588a00640 1 -- 172.21.15.83:0/2181772158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05841095d0 msgr2=0x7f05841a32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:56.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.975+0000 7f0588a00640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05841095d0 0x7f05841a32a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f057400b450 tx=0x7f057400b850 comp rx=0 tx=0).stop 2024-04-23T14:51:56.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.975+0000 7f0588a00640 1 -- 172.21.15.83:0/2181772158 shutdown_connections 2024-04-23T14:51:56.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.975+0000 7f0588a00640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f05680a3de0 0x7f05680a62a0 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0570003f10 tx=0x7f0570009600 comp rx=0 tx=0).stop 2024-04-23T14:51:56.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.975+0000 7f0588a00640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0584117130 0x7f05841a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.975+0000 7f0588a00640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f058410c770 0x7f05841a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.975+0000 7f0588a00640 1 --2- 172.21.15.83:0/2181772158 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05841095d0 0x7f05841a32a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:56.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.975+0000 7f0588a00640 1 -- 172.21.15.83:0/2181772158 >> 172.21.15.83:0/2181772158 conn(0x7f058406d3e0 msgr2=0x7f0584072990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:56.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.976+0000 7f0588a00640 1 -- 172.21.15.83:0/2181772158 shutdown_connections 2024-04-23T14:51:56.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:56.979+0000 7f0588a00640 1 -- 172.21.15.83:0/2181772158 wait complete. 2024-04-23T14:51:57.405 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:57 smithi083 ceph-mon[27867]: pgmap v107: 97 pgs: 97 active+clean; 16 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 174 MiB/s wr, 24.17k op/s 2024-04-23T14:51:57.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:57 smithi203 ceph-mon[32800]: pgmap v107: 97 pgs: 97 active+clean; 16 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 174 MiB/s wr, 24.17k op/s 2024-04-23T14:51:57.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:57 smithi173 ceph-mon[32443]: pgmap v107: 97 pgs: 97 active+clean; 16 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 174 MiB/s wr, 24.17k op/s 2024-04-23T14:51:58.012 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:51:58.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:51:58.012 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:51:58.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.388+0000 7f83c1200640 1 -- 172.21.15.83:0/4114247671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83bc115150 msgr2=0x7f83bc117540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:58.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.388+0000 7f83c1200640 1 --2- 172.21.15.83:0/4114247671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83bc115150 0x7f83bc117540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009ca0 tx=0x7f83b402a8e0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.389+0000 7f83c1200640 1 -- 172.21.15.83:0/4114247671 shutdown_connections 2024-04-23T14:51:58.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.389+0000 7f83c1200640 1 --2- 172.21.15.83:0/4114247671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83bc115150 0x7f83bc117540 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.389+0000 7f83c1200640 1 --2- 172.21.15.83:0/4114247671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83bc109b30 0x7f83bc114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.389+0000 7f83c1200640 1 --2- 172.21.15.83:0/4114247671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83bc109210 0x7f83bc1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.389+0000 7f83c1200640 1 -- 172.21.15.83:0/4114247671 >> 172.21.15.83:0/4114247671 conn(0x7f83bc06d110 msgr2=0x7f83bc06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.390+0000 7f83c1200640 1 -- 172.21.15.83:0/4114247671 shutdown_connections 2024-04-23T14:51:58.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.390+0000 7f83c1200640 1 -- 172.21.15.83:0/4114247671 wait complete. 2024-04-23T14:51:58.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.393+0000 7f83c1200640 1 Processor -- start 2024-04-23T14:51:58.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.393+0000 7f83c1200640 1 -- start start 2024-04-23T14:51:58.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.397+0000 7f83c1200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83bc109210 0x7f83bc1a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.397+0000 7f83c1200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83bc109b30 0x7f83bc1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.398+0000 7f83c1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83bc115150 0x7f83bc1a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.398+0000 7f83b9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83bc109210 0x7f83bc1a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.398+0000 7f83b9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83bc109210 0x7f83bc1a1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33280/0 (socket says 172.21.15.83:33280) 2024-04-23T14:51:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.398+0000 7f83b9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83bc109b30 0x7f83bc1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.398+0000 7f83b9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83bc109b30 0x7f83bc1a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51424/0 (socket says 172.21.15.83:51424) 2024-04-23T14:51:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.399+0000 7f83c1200640 1 -- 172.21.15.83:0/1030935699 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc11ac20 con 0x7f83bc109b30 2024-04-23T14:51:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.399+0000 7f83c1200640 1 -- 172.21.15.83:0/1030935699 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc11aaa0 con 0x7f83bc109210 2024-04-23T14:51:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.399+0000 7f83c1200640 1 -- 172.21.15.83:0/1030935699 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc11ada0 con 0x7f83bc115150 2024-04-23T14:51:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.399+0000 7f83b9e00640 1 -- 172.21.15.83:0/1030935699 learned_addr learned my addr 172.21.15.83:0/1030935699 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.400+0000 7f83ba800640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83bc115150 0x7f83bc1a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.401+0000 7f83b9e00640 1 -- 172.21.15.83:0/1030935699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83bc115150 msgr2=0x7f83bc1a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.401+0000 7f83b9e00640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83bc115150 0x7f83bc1a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.401+0000 7f83b9e00640 1 -- 172.21.15.83:0/1030935699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83bc109b30 msgr2=0x7f83bc1a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.401+0000 7f83b9e00640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83bc109b30 0x7f83bc1a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.401+0000 7f83b9e00640 1 -- 172.21.15.83:0/1030935699 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc1a60e0 con 0x7f83bc109210 2024-04-23T14:51:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.402+0000 7f83ba800640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83bc115150 0x7f83bc1a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.403+0000 7f83b9e00640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83bc109210 0x7f83bc1a1140 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00e9f0 tx=0x7f83ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.404+0000 7f83b2a00640 1 -- 172.21.15.83:0/1030935699 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00cd80 con 0x7f83bc109210 2024-04-23T14:51:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.404+0000 7f83b2a00640 1 -- 172.21.15.83:0/1030935699 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83ac010910 con 0x7f83bc109210 2024-04-23T14:51:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.404+0000 7f83b2a00640 1 -- 172.21.15.83:0/1030935699 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac018990 con 0x7f83bc109210 2024-04-23T14:51:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.405+0000 7f83c1200640 1 -- 172.21.15.83:0/1030935699 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc1a63d0 con 0x7f83bc109210 2024-04-23T14:51:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.405+0000 7f83c1200640 1 -- 172.21.15.83:0/1030935699 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83bc1adc60 con 0x7f83bc109210 2024-04-23T14:51:58.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.420+0000 7f83c3ff1740 1 Processor -- start 2024-04-23T14:51:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.420+0000 7f83c3ff1740 1 -- start start 2024-04-23T14:51:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.420+0000 7f83b2a00640 1 -- 172.21.15.83:0/1030935699 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f83ac010ac0 con 0x7f83bc109210 2024-04-23T14:51:58.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.425+0000 7f83b2a00640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f83a00a3ea0 0x7f83a00a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.425+0000 7f83b2a00640 1 -- 172.21.15.83:0/1030935699 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f83ac0d0b50 con 0x7f83bc109210 2024-04-23T14:51:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.427+0000 7f83ba800640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f83a00a3ea0 0x7f83a00a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.428+0000 7f83c3ff1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfb02150 0x55c3bfb02530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.428+0000 7f83c3ff1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfafd9b0 0x55c3bfafde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.428+0000 7f83c3ff1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafe370 0x55c3bfb0a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.428+0000 7f83c3ff1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c3bfa59e70 con 0x55c3bfafd9b0 2024-04-23T14:51:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.428+0000 7f83c3ff1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c3bfa00380 con 0x55c3bfafe370 2024-04-23T14:51:58.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.428+0000 7f83c3ff1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c3bfa55e30 con 0x55c3bfb02150 2024-04-23T14:51:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.429+0000 7f83b9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfafd9b0 0x55c3bfafde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.429+0000 7f83b9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfafd9b0 0x55c3bfafde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51430/0 (socket says 172.21.15.83:51430) 2024-04-23T14:51:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.429+0000 7f83b9400640 1 -- 172.21.15.83:0/3788844050 learned_addr learned my addr 172.21.15.83:0/3788844050 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:58.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.430+0000 7f83b9e00640 1 --2- 172.21.15.83:0/3788844050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfb02150 0x55c3bfb02530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.430+0000 7f83b9e00640 1 -- 172.21.15.83:0/3788844050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafe370 msgr2=0x55c3bfb0a0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.430+0000 7f83b9e00640 1 --2- 172.21.15.83:0/3788844050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafe370 0x55c3bfb0a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.430+0000 7f83b9e00640 1 -- 172.21.15.83:0/3788844050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfafd9b0 msgr2=0x55c3bfafde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.431+0000 7f83ba800640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f83a00a3ea0 0x7f83a00a6360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009bf0 tx=0x7f83b4005f80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.431+0000 7f83b9e00640 1 --2- 172.21.15.83:0/3788844050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfafd9b0 0x55c3bfafde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.431+0000 7f83b9e00640 1 -- 172.21.15.83:0/3788844050 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3bf8347e0 con 0x55c3bfb02150 2024-04-23T14:51:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.431+0000 7f83b9400640 1 --2- 172.21.15.83:0/3788844050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfafd9b0 0x55c3bfafde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:51:58.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.431+0000 7f83b9e00640 1 --2- 172.21.15.83:0/3788844050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfb02150 0x55c3bfb02530 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0ce2b0 tx=0x7f83ac09c580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f26fa6d7b6aa8d41 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.435+0000 7f83a7400640 1 -- 172.21.15.83:0/3788844050 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac0a5770 con 0x55c3bfb02150 2024-04-23T14:51:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.435+0000 7f83a7400640 1 -- 172.21.15.83:0/3788844050 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83ac09ae00 con 0x55c3bfb02150 2024-04-23T14:51:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.435+0000 7f83a7400640 1 -- 172.21.15.83:0/3788844050 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac09bf50 con 0x55c3bfb02150 2024-04-23T14:51:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.436+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3788844050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfb02150 msgr2=0x55c3bfb02530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.436+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3788844050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfb02150 0x55c3bfb02530 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0ce2b0 tx=0x7f83ac09c580 comp rx=0 tx=0).stop 2024-04-23T14:51:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.436+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3788844050 shutdown_connections 2024-04-23T14:51:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.436+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3788844050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafe370 0x55c3bfb0a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.436+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3788844050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfafd9b0 0x55c3bfafde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.436+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3788844050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfb02150 0x55c3bfb02530 secure :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0ce2b0 tx=0x7f83ac09c580 comp rx=0 tx=0).stop 2024-04-23T14:51:58.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.437+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3788844050 >> 172.21.15.83:0/3788844050 conn(0x55c3bfaefe60 msgr2=0x55c3bfaf0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:58.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.439+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3788844050 shutdown_connections 2024-04-23T14:51:58.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.439+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3788844050 wait complete. 2024-04-23T14:51:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.441+0000 7f83c3ff1740 1 -- start start 2024-04-23T14:51:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.441+0000 7f83c3ff1740 1 Processor -- start 2024-04-23T14:51:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.443+0000 7f83c3ff1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb077c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.444+0000 7f83c3ff1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfafe370 0x55c3bfb07d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.444+0000 7f83c3ff1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfb08240 0x55c3bfb13a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.445+0000 7f83ba800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfb08240 0x55c3bfb13a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.445+0000 7f83ba800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfb08240 0x55c3bfb13a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41040/0 (socket says 172.21.15.83:41040) 2024-04-23T14:51:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.445+0000 7f83ba800640 1 -- 172.21.15.83:0/3989482013 learned_addr learned my addr 172.21.15.83:0/3989482013 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.445+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3989482013 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c3bfb0a5e0 con 0x55c3bfafe370 2024-04-23T14:51:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.444+0000 7f83b9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb077c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.445+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3989482013 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c3bfa13660 con 0x55c3bfafd9b0 2024-04-23T14:51:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.445+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3989482013 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c3bfb0a760 con 0x55c3bfb08240 2024-04-23T14:51:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.446+0000 7f83c3ff1740 10 client.0 _ll_register_callbacks cb 0x55c3bfa01aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.450+0000 7f83b9400640 1 --2- 172.21.15.83:0/3989482013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfafe370 0x55c3bfb07d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.450+0000 7f83b9e00640 1 -- 172.21.15.83:0/3989482013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfb08240 msgr2=0x55c3bfb13a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.450+0000 7f83b9e00640 1 --2- 172.21.15.83:0/3989482013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfb08240 0x55c3bfb13a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.450+0000 7f83b9e00640 1 -- 172.21.15.83:0/3989482013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfafe370 msgr2=0x55c3bfb07d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.450+0000 7f83b9e00640 1 --2- 172.21.15.83:0/3989482013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfafe370 0x55c3bfb07d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.450+0000 7f83b9e00640 1 -- 172.21.15.83:0/3989482013 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3bf894f70 con 0x55c3bfafd9b0 2024-04-23T14:51:58.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.451+0000 7f83b9e00640 1 --2- 172.21.15.83:0/3989482013 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb077c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f83ac09ca90 tx=0x7f83ac09cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9311af108fdd767 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:58.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.451+0000 7f839be00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:51:58.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.451+0000 7f839be00640 1 -- 172.21.15.83:0/3989482013 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac09c150 con 0x55c3bfafd9b0 2024-04-23T14:51:58.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.451+0000 7f839be00640 1 -- 172.21.15.83:0/3989482013 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f83ac010430 con 0x55c3bfafd9b0 2024-04-23T14:51:58.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.452+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3989482013 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83ac09ae00 con 0x55c3bfafd9b0 2024-04-23T14:51:58.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.452+0000 7f839be00640 1 -- 172.21.15.83:0/3989482013 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f83ac0ab050 con 0x55c3bfafd9b0 2024-04-23T14:51:58.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.458+0000 7f83c3ff1740 10 client.26075 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:58.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.458+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3989482013 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c3bf72c170 con 0x55c3bfafd9b0 2024-04-23T14:51:58.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.459+0000 7f839be00640 1 -- 172.21.15.83:0/3989482013 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83ac09ae00 con 0x55c3bfafd9b0 2024-04-23T14:51:58.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.459+0000 7f839be00640 1 -- 172.21.15.83:0/3989482013 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f83ac0a2040 con 0x55c3bfafd9b0 2024-04-23T14:51:58.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.460+0000 7f83c3ff1740 10 client.26075 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:58.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.460+0000 7f83c3ff1740 10 client.26075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:58.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.460+0000 7f83c3ff1740 20 client.26075 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:58.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.460+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3989482013 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c3bfb18060 0x55c3bfb1a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.460+0000 7f83c3ff1740 4 client.26075 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:51:58.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.460+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3989482013 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3bfa67890 con 0x55c3bfb18060 2024-04-23T14:51:58.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.461+0000 7f83b9400640 1 --2- 172.21.15.83:0/3989482013 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c3bfb18060 0x55c3bfb1a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.462+0000 7f83b9400640 1 --2- 172.21.15.83:0/3989482013 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c3bfb18060 0x55c3bfb1a450 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa9e8f4b752bca87 server_cookie=a0bc41d74c1ebf36 in_seq=0 out_seq=0 2024-04-23T14:51:58.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.462+0000 7f839be00640 10 client.26075 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:58.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.465+0000 7f839be00640 1 -- 172.21.15.83:0/3989482013 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c3bfa67890 con 0x55c3bfb18060 2024-04-23T14:51:58.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.465+0000 7f839be00640 10 client.26075 handle_command_reply: tid=0 2024-04-23T14:51:58.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.465+0000 7f83c3ff1740 1 client.26075 shutdown 2024-04-23T14:51:58.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.466+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3989482013 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 msgr2=0x55c3bfb077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:58.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.466+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3989482013 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb077c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f83ac09ca90 tx=0x7f83ac09cc80 comp rx=0 tx=0).stop 2024-04-23T14:51:58.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.466+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3989482013 shutdown_connections 2024-04-23T14:51:58.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.466+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3989482013 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c3bfb18060 0x55c3bfb1a450 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.466+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3989482013 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfb08240 0x55c3bfb13a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.466+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3989482013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfafe370 0x55c3bfb07d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.466+0000 7f839be00640 0 client.26075 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:58.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.466+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3989482013 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb077c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.467+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3989482013 >> 172.21.15.83:0/3989482013 conn(0x55c3bfaefe60 msgr2=0x55c3bfafd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:58.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.469+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3989482013 shutdown_connections 2024-04-23T14:51:58.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.469+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3989482013 wait complete. 2024-04-23T14:51:58.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.469+0000 7f83c3ff1740 20 client.26075 trim_cache size 0 max 16384 2024-04-23T14:51:58.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.515+0000 7f83c3ff1740 1 Processor -- start 2024-04-23T14:51:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.516+0000 7f83c3ff1740 1 -- start start 2024-04-23T14:51:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.517+0000 7f83c3ff1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb13a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.517+0000 7f83c3ff1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafe370 0x55c3bfb09700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.517+0000 7f83c3ff1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfb08240 0x55c3bfaffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.517+0000 7f83c3ff1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1b6ce0 con 0x55c3bfb08240 2024-04-23T14:51:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.517+0000 7f83c3ff1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1b4580 con 0x55c3bfafe370 2024-04-23T14:51:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.517+0000 7f83b9e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb13a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.517+0000 7f83b9400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafe370 0x55c3bfb09700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.518+0000 7f83b9e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb13a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41044/0 (socket says 172.21.15.83:41044) 2024-04-23T14:51:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.518+0000 7f83b9400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafe370 0x55c3bfb09700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33300/0 (socket says 172.21.15.83:33300) 2024-04-23T14:51:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.518+0000 7f83b9400640 1 -- 172.21.15.83:0/3821043782 learned_addr learned my addr 172.21.15.83:0/3821043782 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:58.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.518+0000 7f83ba800640 1 --2- 172.21.15.83:0/3821043782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfb08240 0x55c3bfaffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.519+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3821043782 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1b23d0 con 0x55c3bfafd9b0 2024-04-23T14:51:58.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.522+0000 7f83ba800640 1 -- 172.21.15.83:0/3821043782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfafd9b0 msgr2=0x55c3bfb13a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:58.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.522+0000 7f83ba800640 1 --2- 172.21.15.83:0/3821043782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb13a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.522+0000 7f83ba800640 1 -- 172.21.15.83:0/3821043782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafe370 msgr2=0x55c3bfb09700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:58.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.522+0000 7f83ba800640 1 --2- 172.21.15.83:0/3821043782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafe370 0x55c3bfb09700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.522+0000 7f83ba800640 1 -- 172.21.15.83:0/3821043782 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3bfb06770 con 0x55c3bfb08240 2024-04-23T14:51:58.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.524+0000 7f83ba800640 1 --2- 172.21.15.83:0/3821043782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfb08240 0x55c3bfaffcd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009a60 tx=0x7f83b403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b734d6ccff54172 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:58.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.525+0000 7f83a6a00640 1 -- 172.21.15.83:0/3821043782 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4030d30 con 0x55c3bfb08240 2024-04-23T14:51:58.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.525+0000 7f83a6a00640 1 -- 172.21.15.83:0/3821043782 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b4036040 con 0x55c3bfb08240 2024-04-23T14:51:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.525+0000 7f83a6a00640 1 -- 172.21.15.83:0/3821043782 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b403b660 con 0x55c3bfb08240 2024-04-23T14:51:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.526+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3821043782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfb08240 msgr2=0x55c3bfaffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.526+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3821043782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfb08240 0x55c3bfaffcd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009a60 tx=0x7f83b403c000 comp rx=0 tx=0).stop 2024-04-23T14:51:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.527+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3821043782 shutdown_connections 2024-04-23T14:51:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.527+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3821043782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfb08240 0x55c3bfaffcd0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.527+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3821043782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafe370 0x55c3bfb09700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.527+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3821043782 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb13a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.527+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3821043782 >> 172.21.15.83:0/3821043782 conn(0x55c3bfaefe60 msgr2=0x55c3bfb08ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:58.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.527+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3821043782 shutdown_connections 2024-04-23T14:51:58.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.527+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3821043782 wait complete. 2024-04-23T14:51:58.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.528+0000 7f83c3ff1740 1 -- start start 2024-04-23T14:51:58.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.528+0000 7f83c3ff1740 1 Processor -- start 2024-04-23T14:51:58.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.530+0000 7f83c3ff1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb79ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.530+0000 7f83c3ff1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfafe370 0x55c3bfb1cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.530+0000 7f83c3ff1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfb08240 0x55c3bfb1d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.531+0000 7f83b9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb79ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.531+0000 7f83b9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb79ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33312/0 (socket says 172.21.15.83:33312) 2024-04-23T14:51:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.532+0000 7f83b9400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfafe370 0x55c3bfb1cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.532+0000 7f83ba800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfb08240 0x55c3bfb1d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.532+0000 7f83ba800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfb08240 0x55c3bfb1d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51444/0 (socket says 172.21.15.83:51444) 2024-04-23T14:51:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.533+0000 7f83b9400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfafe370 0x55c3bfb1cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41052/0 (socket says 172.21.15.83:41052) 2024-04-23T14:51:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.533+0000 7f83b9e00640 1 -- 172.21.15.83:0/3619279760 learned_addr learned my addr 172.21.15.83:0/3619279760 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.533+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3619279760 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1b2a40 con 0x55c3bfb08240 2024-04-23T14:51:58.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.533+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3619279760 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1b3eb0 con 0x55c3bfafd9b0 2024-04-23T14:51:58.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.534+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3619279760 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc1b3110 con 0x55c3bfafe370 2024-04-23T14:51:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.534+0000 7f83b9e00640 1 -- 172.21.15.83:0/3619279760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfafe370 msgr2=0x55c3bfb1cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.534+0000 7f83b9e00640 1 --2- 172.21.15.83:0/3619279760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfafe370 0x55c3bfb1cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.534+0000 7f83b9e00640 1 -- 172.21.15.83:0/3619279760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfb08240 msgr2=0x55c3bfb1d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.534+0000 7f83b9e00640 1 --2- 172.21.15.83:0/3619279760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfb08240 0x55c3bfb1d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.535+0000 7f83b9e00640 1 -- 172.21.15.83:0/3619279760 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3bfb777b0 con 0x55c3bfafd9b0 2024-04-23T14:51:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.535+0000 7f83c3ff1740 10 client.0 _ll_register_callbacks cb 0x7f83a800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:58.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.535+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3619279760 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b4036040 con 0x55c3bfafd9b0 2024-04-23T14:51:58.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.537+0000 7f83b9e00640 1 --2- 172.21.15.83:0/3619279760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb79ca0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0a2460 tx=0x7f83ac00cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d41a23e72ff14179 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:58.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.537+0000 7f839be00640 10 client.26081 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:51:58.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.538+0000 7f839be00640 1 -- 172.21.15.83:0/3619279760 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac09c460 con 0x55c3bfafd9b0 2024-04-23T14:51:58.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.538+0000 7f839be00640 1 -- 172.21.15.83:0/3619279760 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f83ac09bbd0 con 0x55c3bfafd9b0 2024-04-23T14:51:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.539+0000 7f839be00640 1 -- 172.21.15.83:0/3619279760 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f83ac0a5770 con 0x55c3bfafd9b0 2024-04-23T14:51:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.539+0000 7f83c3ff1740 10 client.26081 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.539+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3619279760 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55c3bfb777b0 con 0x55c3bfafd9b0 2024-04-23T14:51:58.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.540+0000 7f839be00640 1 -- 172.21.15.83:0/3619279760 <== mon.2 v2:172.21.15.173:3300/0 4 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f83ac0b1040 con 0x55c3bfafd9b0 2024-04-23T14:51:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.541+0000 7f83c3ff1740 10 client.26081 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.541+0000 7f83c3ff1740 10 client.26081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.541+0000 7f83c3ff1740 20 client.26081 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.541+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3619279760 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c3bfb18060 0x55c3bfafc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.541+0000 7f83c3ff1740 4 client.26081 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:51:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.541+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3619279760 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c3bfa61700 con 0x55c3bfb18060 2024-04-23T14:51:58.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.542+0000 7f83b9400640 1 --2- 172.21.15.83:0/3619279760 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c3bfb18060 0x55c3bfafc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.551+0000 7f83b9400640 1 --2- 172.21.15.83:0/3619279760 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c3bfb18060 0x55c3bfafc4c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd001cb0b0bab3d1 server_cookie=28d3ce3aafc23996 in_seq=0 out_seq=0 2024-04-23T14:51:58.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.551+0000 7f839be00640 10 client.26081 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:58.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.554+0000 7f839be00640 1 -- 172.21.15.83:0/3619279760 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c3bfa61700 con 0x55c3bfb18060 2024-04-23T14:51:58.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.554+0000 7f839be00640 10 client.26081 handle_command_reply: tid=0 2024-04-23T14:51:58.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.554+0000 7f83c3ff1740 1 client.26081 shutdown 2024-04-23T14:51:58.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.557+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3619279760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 msgr2=0x55c3bfb79ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.557+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3619279760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb79ca0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0a2460 tx=0x7f83ac00cf20 comp rx=0 tx=0).stop 2024-04-23T14:51:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.557+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3619279760 shutdown_connections 2024-04-23T14:51:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.557+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3619279760 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c3bfb18060 0x55c3bfafc4c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.557+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3619279760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c3bfb08240 0x55c3bfb1d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.557+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3619279760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3bfafe370 0x55c3bfb1cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.557+0000 7f83c3ff1740 1 --2- 172.21.15.83:0/3619279760 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c3bfafd9b0 0x55c3bfb79ca0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.557+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3619279760 >> 172.21.15.83:0/3619279760 conn(0x55c3bfaefe60 msgr2=0x55c3bfb09880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.557+0000 7f839be00640 0 client.26081 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.559+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3619279760 shutdown_connections 2024-04-23T14:51:58.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.559+0000 7f83c3ff1740 1 -- 172.21.15.83:0/3619279760 wait complete. 2024-04-23T14:51:58.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.559+0000 7f83c3ff1740 20 client.26081 trim_cache size 0 max 16384 2024-04-23T14:51:58.564 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:51:58.564 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:51:58.564 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "3c3166d5-d451-4a5c-9ef9-dc8be43e661b", 2024-04-23T14:51:58.564 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:51:58.564 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:51:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.563+0000 7f83c1200640 1 -- 172.21.15.83:0/1030935699 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f83a00a3ea0 msgr2=0x7f83a00a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.563+0000 7f83c1200640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f83a00a3ea0 0x7f83a00a6360 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f83b4009bf0 tx=0x7f83b4005f80 comp rx=0 tx=0).stop 2024-04-23T14:51:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.563+0000 7f83c1200640 1 -- 172.21.15.83:0/1030935699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83bc109210 msgr2=0x7f83bc1a1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.563+0000 7f83c1200640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83bc109210 0x7f83bc1a1140 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00e9f0 tx=0x7f83ac00eeb0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.564+0000 7f83c1200640 1 -- 172.21.15.83:0/1030935699 shutdown_connections 2024-04-23T14:51:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.564+0000 7f83c1200640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f83bc115150 0x7f83bc1a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.564+0000 7f83c1200640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f83a00a3ea0 0x7f83a00a6360 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.564+0000 7f83c1200640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83bc109b30 0x7f83bc1a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.564+0000 7f83c1200640 1 --2- 172.21.15.83:0/1030935699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83bc109210 0x7f83bc1a1140 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.564+0000 7f83c1200640 1 -- 172.21.15.83:0/1030935699 >> 172.21.15.83:0/1030935699 conn(0x7f83bc06d110 msgr2=0x7f83bc070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.566+0000 7f83c1200640 1 -- 172.21.15.83:0/1030935699 shutdown_connections 2024-04-23T14:51:58.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.566+0000 7f83c1200640 1 -- 172.21.15.83:0/1030935699 wait complete. 2024-04-23T14:51:58.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:51:58.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:51:58.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.971+0000 7fc7e6400640 1 -- 172.21.15.83:0/2923147187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e0102bf0 msgr2=0x7fc7e0106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:58.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.971+0000 7fc7e6400640 1 --2- 172.21.15.83:0/2923147187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e0102bf0 0x7fc7e0106c30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009860 tx=0x7fc7cc02a360 comp rx=0 tx=0).stop 2024-04-23T14:51:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.976+0000 7fc7e6400640 1 -- 172.21.15.83:0/2923147187 shutdown_connections 2024-04-23T14:51:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.978+0000 7fc7e6400640 1 --2- 172.21.15.83:0/2923147187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e0107310 0x7fc7e010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.978+0000 7fc7e6400640 1 --2- 172.21.15.83:0/2923147187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e0102bf0 0x7fc7e0106c30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.978+0000 7fc7e6400640 1 --2- 172.21.15.83:0/2923147187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e0102240 0x7fc7e0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.978+0000 7fc7e6400640 1 -- 172.21.15.83:0/2923147187 >> 172.21.15.83:0/2923147187 conn(0x7fc7e0079450 msgr2=0x7fc7e0079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:58.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.980+0000 7fc7e6400640 1 -- 172.21.15.83:0/2923147187 shutdown_connections 2024-04-23T14:51:58.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.980+0000 7fc7e6400640 1 -- 172.21.15.83:0/2923147187 wait complete. 2024-04-23T14:51:58.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.982+0000 7fc7e6400640 1 Processor -- start 2024-04-23T14:51:58.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.982+0000 7fc7e6400640 1 -- start start 2024-04-23T14:51:58.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.984+0000 7fc7e6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e0102240 0x7fc7e01167b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.985+0000 7fc7e6400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e0107310 0x7fc7e0116cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.985+0000 7fc7e6400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e0111910 0x7fc7e0111dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:58.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.985+0000 7fc7e6400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0075b90 con 0x7fc7e0102240 2024-04-23T14:51:58.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.985+0000 7fc7e6400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0075a10 con 0x7fc7e0107310 2024-04-23T14:51:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.985+0000 7fc7e6400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0075d10 con 0x7fc7e0111910 2024-04-23T14:51:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.986+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e0111910 0x7fc7e0111dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.986+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e0111910 0x7fc7e0111dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41080/0 (socket says 172.21.15.83:41080) 2024-04-23T14:51:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.986+0000 7fc7dfe00640 1 -- 172.21.15.83:0/2654450141 learned_addr learned my addr 172.21.15.83:0/2654450141 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.986+0000 7fc7dea00640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e0107310 0x7fc7e0116cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.986+0000 7fc7dfe00640 1 -- 172.21.15.83:0/2654450141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e0107310 msgr2=0x7fc7e0116cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.986+0000 7fc7dfe00640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e0107310 0x7fc7e0116cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.986+0000 7fc7dfe00640 1 -- 172.21.15.83:0/2654450141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e0102240 msgr2=0x7fc7e01167b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.986+0000 7fc7df400640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e0102240 0x7fc7e01167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.987+0000 7fc7dfe00640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e0102240 0x7fc7e01167b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.987+0000 7fc7dfe00640 1 -- 172.21.15.83:0/2654450141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e0112680 con 0x7fc7e0111910 2024-04-23T14:51:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.987+0000 7fc7df400640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e0102240 0x7fc7e01167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:58.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.988+0000 7fc7dfe00640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e0111910 0x7fc7e0111dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009fd0 tx=0x7fc7d800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:58.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.989+0000 7fc7d7e00640 1 -- 172.21.15.83:0/2654450141 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8004b20 con 0x7fc7e0111910 2024-04-23T14:51:58.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.989+0000 7fc7d7e00640 1 -- 172.21.15.83:0/2654450141 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d8003e50 con 0x7fc7e0111910 2024-04-23T14:51:58.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.989+0000 7fc7d7e00640 1 -- 172.21.15.83:0/2654450141 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d800b630 con 0x7fc7e0111910 2024-04-23T14:51:58.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.989+0000 7fc7e6400640 1 -- 172.21.15.83:0/2654450141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e011c4c0 con 0x7fc7e0111910 2024-04-23T14:51:58.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.990+0000 7fc7e6400640 1 -- 172.21.15.83:0/2654450141 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e011c990 con 0x7fc7e0111910 2024-04-23T14:51:58.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.995+0000 7fc7e9373740 1 Processor -- start 2024-04-23T14:51:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.996+0000 7fc7d7e00640 1 -- 172.21.15.83:0/2654450141 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc7d801d020 con 0x7fc7e0111910 2024-04-23T14:51:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.996+0000 7fc7e9373740 1 -- start start 2024-04-23T14:51:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.999+0000 7fc7d7e00640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc7c40a3ea0 0x7fc7c40a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:58.999+0000 7fc7d7e00640 1 -- 172.21.15.83:0/2654450141 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc7d80d0640 con 0x7fc7e0111910 2024-04-23T14:51:59.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.000+0000 7fc7e9373740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693ac860 0x55a8693acc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.000+0000 7fc7e9373740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 0x55a8693b7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.000+0000 7fc7e9373740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693b7850 0x55a8693b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.000+0000 7fc7e9373740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a8693121f0 con 0x55a8693b7850 2024-04-23T14:51:59.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.000+0000 7fc7e9373740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a8692b8350 con 0x55a8693ad180 2024-04-23T14:51:59.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.000+0000 7fc7e9373740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a86930e1b0 con 0x55a8693ac860 2024-04-23T14:51:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.001+0000 7fc7dea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 0x55a8693b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.001+0000 7fc7dea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 0x55a8693b7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33348/0 (socket says 172.21.15.83:33348) 2024-04-23T14:51:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.001+0000 7fc7dea00640 1 -- 172.21.15.83:0/872333484 learned_addr learned my addr 172.21.15.83:0/872333484 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.001+0000 7fc7dea00640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc7c40a3ea0 0x7fc7c40a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.001+0000 7fc7dfe00640 1 --2- 172.21.15.83:0/872333484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693b7850 0x55a8693b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.002+0000 7fc7dea00640 1 -- 172.21.15.83:0/872333484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693ac860 msgr2=0x55a8693acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.002+0000 7fc7dea00640 1 --2- 172.21.15.83:0/872333484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693ac860 0x55a8693acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.002+0000 7fc7dea00640 1 -- 172.21.15.83:0/872333484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693b7850 msgr2=0x55a8693b9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.002+0000 7fc7dea00640 1 --2- 172.21.15.83:0/872333484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693b7850 0x55a8693b9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.002+0000 7fc7dea00640 1 -- 172.21.15.83:0/872333484 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86930cf50 con 0x55a8693ad180 2024-04-23T14:51:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.002+0000 7fc7dea00640 1 --2- 172.21.15.83:0/872333484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 0x55a8693b7310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0304c0 tx=0x7fc7cc030f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fc0f06528cb3921 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.003+0000 7fc7dea00640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc7c40a3ea0 0x7fc7c40a6360 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc000c00 tx=0x7fc7cc03f000 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:59.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.003+0000 7fc7d4c00640 1 -- 172.21.15.83:0/872333484 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc037040 con 0x55a8693ad180 2024-04-23T14:51:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.003+0000 7fc7d4c00640 1 -- 172.21.15.83:0/872333484 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7cc04f070 con 0x55a8693ad180 2024-04-23T14:51:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.003+0000 7fc7d4c00640 1 -- 172.21.15.83:0/872333484 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc037040 con 0x55a8693ad180 2024-04-23T14:51:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.004+0000 7fc7e9373740 1 -- 172.21.15.83:0/872333484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 msgr2=0x55a8693b7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.004+0000 7fc7e9373740 1 --2- 172.21.15.83:0/872333484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 0x55a8693b7310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0304c0 tx=0x7fc7cc030f90 comp rx=0 tx=0).stop 2024-04-23T14:51:59.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.004+0000 7fc7e9373740 1 -- 172.21.15.83:0/872333484 shutdown_connections 2024-04-23T14:51:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.004+0000 7fc7e9373740 1 --2- 172.21.15.83:0/872333484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693b7850 0x55a8693b9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.004+0000 7fc7e9373740 1 --2- 172.21.15.83:0/872333484 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 0x55a8693b7310 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.004+0000 7fc7e9373740 1 --2- 172.21.15.83:0/872333484 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693ac860 0x55a8693acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.004+0000 7fc7e9373740 1 -- 172.21.15.83:0/872333484 >> 172.21.15.83:0/872333484 conn(0x55a8693a8b80 msgr2=0x55a8693a8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.005+0000 7fc7e9373740 1 -- 172.21.15.83:0/872333484 shutdown_connections 2024-04-23T14:51:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.005+0000 7fc7e9373740 1 -- 172.21.15.83:0/872333484 wait complete. 2024-04-23T14:51:59.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.005+0000 7fc7e9373740 1 -- start start 2024-04-23T14:51:59.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.006+0000 7fc7e9373740 1 Processor -- start 2024-04-23T14:51:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.008+0000 7fc7e9373740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ac860 0x55a8693b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.008+0000 7fc7e9373740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693ad180 0x55a8693b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.008+0000 7fc7e9373740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693b7850 0x55a8693b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.008+0000 7fc7e9373740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a8693ba5e0 con 0x55a8693ad180 2024-04-23T14:51:59.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.008+0000 7fc7e9373740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a8692cb690 con 0x55a8693ac860 2024-04-23T14:51:59.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.008+0000 7fc7e9373740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a8693ba760 con 0x55a8693b7850 2024-04-23T14:51:59.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.008+0000 7fc7e9373740 10 client.0 _ll_register_callbacks cb 0x55a869307650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:59.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.009+0000 7fc7df400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ac860 0x55a8693b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:59.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.009+0000 7fc7df400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ac860 0x55a8693b4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33350/0 (socket says 172.21.15.83:33350) 2024-04-23T14:51:59.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.009+0000 7fc7df400640 1 -- 172.21.15.83:0/1563637010 learned_addr learned my addr 172.21.15.83:0/1563637010 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:59.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.009+0000 7fc7dea00640 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693ad180 0x55a8693b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:59.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.009+0000 7fc7dfe00640 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693b7850 0x55a8693b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:59.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.010+0000 7fc7df400640 1 -- 172.21.15.83:0/1563637010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693b7850 msgr2=0x55a8693b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:59.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.010+0000 7fc7df400640 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693b7850 0x55a8693b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.010+0000 7fc7df400640 1 -- 172.21.15.83:0/1563637010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693ad180 msgr2=0x55a8693b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.010+0000 7fc7df400640 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693ad180 0x55a8693b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.010+0000 7fc7df400640 1 -- 172.21.15.83:0/1563637010 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8692ca920 con 0x55a8693ac860 2024-04-23T14:51:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.010+0000 7fc7dea00640 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693ad180 0x55a8693b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:51:59.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.011+0000 7fc7df400640 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ac860 0x55a8693b4820 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000e3f0 tx=0x7fc7d000e8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eefc551c3497818 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.011+0000 7fc7c9600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:51:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.011+0000 7fc7c9600640 1 -- 172.21.15.83:0/1563637010 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0016070 con 0x55a8693ac860 2024-04-23T14:51:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.011+0000 7fc7c9600640 1 -- 172.21.15.83:0/1563637010 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc7d00108a0 con 0x55a8693ac860 2024-04-23T14:51:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.012+0000 7fc7e9373740 1 -- 172.21.15.83:0/1563637010 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7cc04f070 con 0x55a8693ac860 2024-04-23T14:51:59.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.012+0000 7fc7c9600640 1 -- 172.21.15.83:0/1563637010 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7d0020050 con 0x55a8693ac860 2024-04-23T14:51:59.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.012+0000 7fc7e9373740 10 client.26093 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:59.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.013+0000 7fc7e9373740 1 -- 172.21.15.83:0/1563637010 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8690ee7d0 con 0x55a8693ac860 2024-04-23T14:51:59.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.013+0000 7fc7c9600640 1 -- 172.21.15.83:0/1563637010 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7cc04f070 con 0x55a8693ac860 2024-04-23T14:51:59.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.013+0000 7fc7c9600640 1 -- 172.21.15.83:0/1563637010 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fc7d000f040 con 0x55a8693ac860 2024-04-23T14:51:59.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.014+0000 7fc7e9373740 10 client.26093 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:59.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.014+0000 7fc7e9373740 10 client.26093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:59.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.014+0000 7fc7e9373740 20 client.26093 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:59.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.014+0000 7fc7e9373740 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55a8693c7d40 0x55a8693ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.014+0000 7fc7e9373740 4 client.26093 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:51:59.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.014+0000 7fc7e9373740 1 -- 172.21.15.83:0/1563637010 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86931fc10 con 0x55a8693c7d40 2024-04-23T14:51:59.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.015+0000 7fc7dea00640 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55a8693c7d40 0x55a8693ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:59.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.016+0000 7fc7dea00640 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55a8693c7d40 0x55a8693ca1a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f1aed0ee5226a9 server_cookie=457bbb0ce67cc4fb in_seq=0 out_seq=0 2024-04-23T14:51:59.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.016+0000 7fc7c9600640 10 client.26093 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:59.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.017+0000 7fc7c9600640 1 -- 172.21.15.83:0/1563637010 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a86931fc10 con 0x55a8693c7d40 2024-04-23T14:51:59.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.017+0000 7fc7c9600640 10 client.26093 handle_command_reply: tid=0 2024-04-23T14:51:59.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.017+0000 7fc7e9373740 1 client.26093 shutdown 2024-04-23T14:51:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.019+0000 7fc7e9373740 1 -- 172.21.15.83:0/1563637010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ac860 msgr2=0x55a8693b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.019+0000 7fc7e9373740 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ac860 0x55a8693b4820 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc7d000e3f0 tx=0x7fc7d000e8b0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.019+0000 7fc7e9373740 1 -- 172.21.15.83:0/1563637010 shutdown_connections 2024-04-23T14:51:59.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.019+0000 7fc7e9373740 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55a8693c7d40 0x55a8693ca1a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.019+0000 7fc7e9373740 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693b7850 0x55a8693b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.019+0000 7fc7e9373740 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693ad180 0x55a8693b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.019+0000 7fc7e9373740 1 --2- 172.21.15.83:0/1563637010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ac860 0x55a8693b4820 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.019+0000 7fc7e9373740 1 -- 172.21.15.83:0/1563637010 >> 172.21.15.83:0/1563637010 conn(0x55a8693a8b80 msgr2=0x55a8693ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:59.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.019+0000 7fc7c9600640 0 client.26093 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:59.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.020+0000 7fc7e9373740 1 -- 172.21.15.83:0/1563637010 shutdown_connections 2024-04-23T14:51:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.021+0000 7fc7e9373740 1 -- 172.21.15.83:0/1563637010 wait complete. 2024-04-23T14:51:59.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.021+0000 7fc7e9373740 20 client.26093 trim_cache size 0 max 16384 2024-04-23T14:51:59.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.067+0000 7fc7e9373740 1 Processor -- start 2024-04-23T14:51:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.068+0000 7fc7e9373740 1 -- start start 2024-04-23T14:51:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.069+0000 7fc7e9373740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693ac860 0x55a8693c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.069+0000 7fc7e9373740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 0x55a8693ae700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.069+0000 7fc7e9373740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693b7850 0x55a8693cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.069+0000 7fc7e9373740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c0011140 con 0x55a8693b7850 2024-04-23T14:51:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.069+0000 7fc7e9373740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c000e9e0 con 0x55a8693ad180 2024-04-23T14:51:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.069+0000 7fc7e9373740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c000c7f0 con 0x55a8693ac860 2024-04-23T14:51:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.070+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693b7850 0x55a8693cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:59.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.070+0000 7fc7dea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 0x55a8693ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.070+0000 7fc7dea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 0x55a8693ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33354/0 (socket says 172.21.15.83:33354) 2024-04-23T14:51:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.070+0000 7fc7dea00640 1 -- 172.21.15.83:0/913689555 learned_addr learned my addr 172.21.15.83:0/913689555 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.070+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693b7850 0x55a8693cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51498/0 (socket says 172.21.15.83:51498) 2024-04-23T14:51:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.070+0000 7fc7dea00640 1 -- 172.21.15.83:0/913689555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693ac860 msgr2=0x55a8693c34d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:59.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.071+0000 7fc7dea00640 1 --2- 172.21.15.83:0/913689555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693ac860 0x55a8693c34d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.071+0000 7fc7dea00640 1 -- 172.21.15.83:0/913689555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693b7850 msgr2=0x55a8693cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:59.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.071+0000 7fc7dea00640 1 --2- 172.21.15.83:0/913689555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693b7850 0x55a8693cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.071+0000 7fc7dea00640 1 -- 172.21.15.83:0/913689555 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8690243f0 con 0x55a8693ad180 2024-04-23T14:51:59.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.071+0000 7fc7dfe00640 1 --2- 172.21.15.83:0/913689555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693b7850 0x55a8693cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:51:59.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.071+0000 7fc7dea00640 1 --2- 172.21.15.83:0/913689555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 0x55a8693ae700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc031000 tx=0x7fc7cc02c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe5b74a0d321f095 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:59.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.074+0000 7fc7cbe00640 1 -- 172.21.15.83:0/913689555 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc037040 con 0x55a8693ad180 2024-04-23T14:51:59.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.074+0000 7fc7cbe00640 1 -- 172.21.15.83:0/913689555 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7cc04f040 con 0x55a8693ad180 2024-04-23T14:51:59.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.074+0000 7fc7cbe00640 1 -- 172.21.15.83:0/913689555 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc05b6d0 con 0x55a8693ad180 2024-04-23T14:51:59.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.075+0000 7fc7e9373740 1 -- 172.21.15.83:0/913689555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 msgr2=0x55a8693ae700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:59.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.075+0000 7fc7e9373740 1 --2- 172.21.15.83:0/913689555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 0x55a8693ae700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc031000 tx=0x7fc7cc02c040 comp rx=0 tx=0).stop 2024-04-23T14:51:59.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.077+0000 7fc7e9373740 1 -- 172.21.15.83:0/913689555 shutdown_connections 2024-04-23T14:51:59.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.077+0000 7fc7e9373740 1 --2- 172.21.15.83:0/913689555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693b7850 0x55a8693cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.077+0000 7fc7e9373740 1 --2- 172.21.15.83:0/913689555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693ad180 0x55a8693ae700 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.077+0000 7fc7e9373740 1 --2- 172.21.15.83:0/913689555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693ac860 0x55a8693c34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.077+0000 7fc7e9373740 1 -- 172.21.15.83:0/913689555 >> 172.21.15.83:0/913689555 conn(0x55a8693a8b80 msgr2=0x55a8693adff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.078+0000 7fc7e9373740 1 -- 172.21.15.83:0/913689555 shutdown_connections 2024-04-23T14:51:59.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.078+0000 7fc7e9373740 1 -- 172.21.15.83:0/913689555 wait complete. 2024-04-23T14:51:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.079+0000 7fc7e9373740 1 -- start start 2024-04-23T14:51:59.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.079+0000 7fc7e9373740 1 Processor -- start 2024-04-23T14:51:59.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.080+0000 7fc7e9373740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693ac860 0x55a8693bb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.080+0000 7fc7e9373740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693ad180 0x55a8693bbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.080+0000 7fc7e9373740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693b7850 0x55a86942d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.080+0000 7fc7e9373740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c000cea0 con 0x55a8693ac860 2024-04-23T14:51:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.080+0000 7fc7e9373740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c000e310 con 0x55a8693b7850 2024-04-23T14:51:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.080+0000 7fc7e9373740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c000d570 con 0x55a8693ad180 2024-04-23T14:51:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.080+0000 7fc7e9373740 10 client.0 _ll_register_callbacks cb 0x7fc7d0009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:51:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.081+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693b7850 0x55a86942d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.081+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693b7850 0x55a86942d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:33368/0 (socket says 172.21.15.83:33368) 2024-04-23T14:51:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.081+0000 7fc7dfe00640 1 -- 172.21.15.83:0/2176554182 learned_addr learned my addr 172.21.15.83:0/2176554182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:51:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.081+0000 7fc7dfe00640 1 -- 172.21.15.83:0/2176554182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693ad180 msgr2=0x55a8693bbc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:51:59.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.082+0000 7fc7dfe00640 1 --2- 172.21.15.83:0/2176554182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693ad180 0x55a8693bbc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.082+0000 7fc7dfe00640 1 -- 172.21.15.83:0/2176554182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693ac860 msgr2=0x55a8693bb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:51:59.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.082+0000 7fc7dfe00640 1 --2- 172.21.15.83:0/2176554182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693ac860 0x55a8693bb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.082+0000 7fc7dfe00640 1 -- 172.21.15.83:0/2176554182 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a868fe40f0 con 0x55a8693b7850 2024-04-23T14:51:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.082+0000 7fc7dfe00640 1 --2- 172.21.15.83:0/2176554182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693b7850 0x55a86942d2e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8004420 tx=0x7fc7d809c240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7583aac7587534f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:51:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.082+0000 7fc7c9600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:51:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.083+0000 7fc7c9600640 1 -- 172.21.15.83:0/2176554182 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d801a040 con 0x55a8693b7850 2024-04-23T14:51:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.083+0000 7fc7c9600640 1 -- 172.21.15.83:0/2176554182 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc7d8003970 con 0x55a8693b7850 2024-04-23T14:51:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.083+0000 7fc7e9373740 1 -- 172.21.15.83:0/2176554182 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7cc04f040 con 0x55a8693b7850 2024-04-23T14:51:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.083+0000 7fc7c9600640 1 -- 172.21.15.83:0/2176554182 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7d809ccf0 con 0x55a8693b7850 2024-04-23T14:51:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.083+0000 7fc7e9373740 10 client.26099 fetch_fsmap learned FSMap version 26 2024-04-23T14:51:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.083+0000 7fc7e9373740 1 -- 172.21.15.83:0/2176554182 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8692d7810 con 0x55a8693b7850 2024-04-23T14:51:59.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.084+0000 7fc7c9600640 1 -- 172.21.15.83:0/2176554182 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7cc04f040 con 0x55a8693b7850 2024-04-23T14:51:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.085+0000 7fc7c9600640 1 -- 172.21.15.83:0/2176554182 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7fc7d800b040 con 0x55a8693b7850 2024-04-23T14:51:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.085+0000 7fc7e9373740 10 client.26099 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:51:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.085+0000 7fc7e9373740 10 client.26099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:51:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.085+0000 7fc7e9373740 20 client.26099 populate_metadata read hostname 'smithi083' 2024-04-23T14:51:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.085+0000 7fc7e9373740 1 --2- 172.21.15.83:0/2176554182 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55a86942edf0 0x55a869431250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:51:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.085+0000 7fc7e9373740 4 client.26099 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:51:59.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.085+0000 7fc7e9373740 1 -- 172.21.15.83:0/2176554182 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a869319a80 con 0x55a86942edf0 2024-04-23T14:51:59.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.086+0000 7fc7df400640 1 --2- 172.21.15.83:0/2176554182 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55a86942edf0 0x55a869431250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:51:59.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.087+0000 7fc7df400640 1 --2- 172.21.15.83:0/2176554182 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55a86942edf0 0x55a869431250 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b1b8cdccf0da5e server_cookie=236d4e4e2cab342f in_seq=0 out_seq=0 2024-04-23T14:51:59.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.087+0000 7fc7c9600640 10 client.26099 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:59.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.087+0000 7fc7c9600640 1 -- 172.21.15.83:0/2176554182 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55a869319a80 con 0x55a86942edf0 2024-04-23T14:51:59.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.087+0000 7fc7c9600640 10 client.26099 handle_command_reply: tid=0 2024-04-23T14:51:59.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.087+0000 7fc7e9373740 1 client.26099 shutdown 2024-04-23T14:51:59.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.087+0000 7fc7e9373740 1 -- 172.21.15.83:0/2176554182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693b7850 msgr2=0x55a86942d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.087+0000 7fc7e9373740 1 --2- 172.21.15.83:0/2176554182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693b7850 0x55a86942d2e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8004420 tx=0x7fc7d809c240 comp rx=0 tx=0).stop 2024-04-23T14:51:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.088+0000 7fc7e9373740 1 -- 172.21.15.83:0/2176554182 shutdown_connections 2024-04-23T14:51:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.088+0000 7fc7e9373740 1 --2- 172.21.15.83:0/2176554182 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55a86942edf0 0x55a869431250 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.088+0000 7fc7e9373740 1 --2- 172.21.15.83:0/2176554182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a8693b7850 0x55a86942d2e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.088+0000 7fc7e9373740 1 --2- 172.21.15.83:0/2176554182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a8693ad180 0x55a8693bbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.088+0000 7fc7e9373740 1 --2- 172.21.15.83:0/2176554182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8693ac860 0x55a8693bb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.088+0000 7fc7e9373740 1 -- 172.21.15.83:0/2176554182 >> 172.21.15.83:0/2176554182 conn(0x55a8693a8b80 msgr2=0x55a8693ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.089+0000 7fc7c9600640 0 client.26099 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:51:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.089+0000 7fc7e9373740 1 -- 172.21.15.83:0/2176554182 shutdown_connections 2024-04-23T14:51:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.089+0000 7fc7e9373740 1 -- 172.21.15.83:0/2176554182 wait complete. 2024-04-23T14:51:59.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.089+0000 7fc7e9373740 20 client.26099 trim_cache size 0 max 16384 2024-04-23T14:51:59.091 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:51:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (0 inodes in the stack)", 2024-04-23T14:51:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:51:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "3c3166d5-d451-4a5c-9ef9-dc8be43e661b": { 2024-04-23T14:51:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:51:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "3c3166d5-d451-4a5c-9ef9-dc8be43e661b", 2024-04-23T14:51:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:51:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:51:59.091 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:51:59.091 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:51:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.090+0000 7fc7e6400640 1 -- 172.21.15.83:0/2654450141 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc7c40a3ea0 msgr2=0x7fc7c40a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.090+0000 7fc7e6400640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc7c40a3ea0 0x7fc7c40a6360 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc000c00 tx=0x7fc7cc03f000 comp rx=0 tx=0).stop 2024-04-23T14:51:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.090+0000 7fc7e6400640 1 -- 172.21.15.83:0/2654450141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e0111910 msgr2=0x7fc7e0111dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:51:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.090+0000 7fc7e6400640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e0111910 0x7fc7e0111dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009fd0 tx=0x7fc7d800c800 comp rx=0 tx=0).stop 2024-04-23T14:51:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.090+0000 7fc7e6400640 1 -- 172.21.15.83:0/2654450141 shutdown_connections 2024-04-23T14:51:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.090+0000 7fc7e6400640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7e0111910 0x7fc7e0111dc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.090+0000 7fc7e6400640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc7e0107310 0x7fc7e0116cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.090+0000 7fc7e6400640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc7c40a3ea0 0x7fc7c40a6360 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc000c00 tx=0x7fc7cc03f000 comp rx=0 tx=0).stop 2024-04-23T14:51:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.090+0000 7fc7e6400640 1 --2- 172.21.15.83:0/2654450141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7e0102240 0x7fc7e01167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:51:59.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.090+0000 7fc7e6400640 1 -- 172.21.15.83:0/2654450141 >> 172.21.15.83:0/2654450141 conn(0x7fc7e0079450 msgr2=0x7fc7e00ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:51:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.091+0000 7fc7e6400640 1 -- 172.21.15.83:0/2654450141 shutdown_connections 2024-04-23T14:51:59.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:51:59.091+0000 7fc7e6400640 1 -- 172.21.15.83:0/2654450141 wait complete. 2024-04-23T14:51:59.129 INFO:tasks.cephfs.filesystem:scrub status for tag:3c3166d5-d451-4a5c-9ef9-dc8be43e661b - {'path': '/', 'tag': '3c3166d5-d451-4a5c-9ef9-dc8be43e661b', 'options': 'recursive,force'} 2024-04-23T14:51:59.415 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:59 smithi083 ceph-mon[27867]: pgmap v108: 97 pgs: 97 active+clean; 16 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 135 MiB/s wr, 19.40k op/s 2024-04-23T14:51:59.415 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:59 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:51:59.415 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:59 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:59.415 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:59 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:51:59.415 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:51:59 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:51:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:59 smithi203 ceph-mon[32800]: pgmap v108: 97 pgs: 97 active+clean; 16 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 135 MiB/s wr, 19.40k op/s 2024-04-23T14:51:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:59 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:51:59.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:59 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:59.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:59 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:51:59.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:51:59 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:51:59.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:59 smithi173 ceph-mon[32443]: pgmap v108: 97 pgs: 97 active+clean; 16 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 135 MiB/s wr, 19.40k op/s 2024-04-23T14:51:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:59 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:51:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:59 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:51:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:59 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:51:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:51:59 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:52:00.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:00 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:00.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:00 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:00.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:00 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:01.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:01 smithi203 ceph-mon[32800]: pgmap v109: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 167 MiB/s wr, 23.66k op/s 2024-04-23T14:52:01.760 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:01 smithi083 ceph-mon[27867]: pgmap v109: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 167 MiB/s wr, 23.66k op/s 2024-04-23T14:52:01.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:01 smithi173 ceph-mon[32443]: pgmap v109: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 167 MiB/s wr, 23.66k op/s 2024-04-23T14:52:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:03 smithi203 ceph-mon[32800]: pgmap v110: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 109 MiB/s wr, 15.17k op/s 2024-04-23T14:52:03.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:03 smithi083 ceph-mon[27867]: pgmap v110: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 109 MiB/s wr, 15.17k op/s 2024-04-23T14:52:03.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:03 smithi173 ceph-mon[32443]: pgmap v110: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 109 MiB/s wr, 15.17k op/s 2024-04-23T14:52:05.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:05 smithi203 ceph-mon[32800]: pgmap v111: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 94 MiB/s wr, 13.05k op/s 2024-04-23T14:52:05.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:05 smithi083 ceph-mon[27867]: pgmap v111: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 94 MiB/s wr, 13.05k op/s 2024-04-23T14:52:05.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:05 smithi173 ceph-mon[32443]: pgmap v111: 97 pgs: 97 active+clean; 16 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 94 MiB/s wr, 13.05k op/s 2024-04-23T14:52:06.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:06 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:52:06.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:06 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:06.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:06 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:52:06.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:06 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:06.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:06 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:52:06.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:06 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:06.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:06 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:52:06.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:06 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:06.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:06 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:52:06.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:06 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:06.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:06 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:52:06.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:06 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:07.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:07 smithi083 ceph-mon[27867]: pgmap v112: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 127 MiB/s wr, 17.87k op/s 2024-04-23T14:52:07.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:07 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:52:07.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:07 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:07.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:07 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:07.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:07 smithi173 ceph-mon[32443]: pgmap v112: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 127 MiB/s wr, 17.87k op/s 2024-04-23T14:52:07.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:07 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:52:07.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:07 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:07.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:07 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:07.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:07 smithi203 ceph-mon[32800]: pgmap v112: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 127 MiB/s wr, 17.87k op/s 2024-04-23T14:52:07.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:07 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:52:07.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:07 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:07.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:07 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:52:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:09 smithi083 ceph-mon[27867]: pgmap v113: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 127 MiB/s wr, 17.84k op/s 2024-04-23T14:52:09.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:09 smithi173 ceph-mon[32443]: pgmap v113: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 127 MiB/s wr, 17.84k op/s 2024-04-23T14:52:09.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:09 smithi203 ceph-mon[32800]: pgmap v113: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 127 MiB/s wr, 17.84k op/s 2024-04-23T14:52:11.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:11 smithi083 ceph-mon[27867]: pgmap v114: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 113 MiB/s wr, 15.90k op/s 2024-04-23T14:52:11.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:11 smithi173 ceph-mon[32443]: pgmap v114: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 113 MiB/s wr, 15.90k op/s 2024-04-23T14:52:11.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:11 smithi203 ceph-mon[32800]: pgmap v114: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 113 MiB/s wr, 15.90k op/s 2024-04-23T14:52:13.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:13 smithi173 ceph-mon[32443]: pgmap v115: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 75 MiB/s wr, 10.96k op/s 2024-04-23T14:52:13.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:13 smithi203 ceph-mon[32800]: pgmap v115: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 75 MiB/s wr, 10.96k op/s 2024-04-23T14:52:14.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:13 smithi083 ceph-mon[27867]: pgmap v115: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 75 MiB/s wr, 10.96k op/s 2024-04-23T14:52:15.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:15 smithi173 ceph-mon[32443]: pgmap v116: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 68 MiB/s wr, 9.97k op/s 2024-04-23T14:52:15.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:15 smithi203 ceph-mon[32800]: pgmap v116: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 68 MiB/s wr, 9.97k op/s 2024-04-23T14:52:16.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:15 smithi083 ceph-mon[27867]: pgmap v116: 97 pgs: 97 active+clean; 16 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 68 MiB/s wr, 9.97k op/s 2024-04-23T14:52:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:17 smithi173 ceph-mon[32443]: pgmap v117: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 102 MiB/s wr, 14.39k op/s 2024-04-23T14:52:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:17 smithi203 ceph-mon[32800]: pgmap v117: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 102 MiB/s wr, 14.39k op/s 2024-04-23T14:52:18.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:17 smithi083 ceph-mon[27867]: pgmap v117: 97 pgs: 97 active+clean; 16 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 102 MiB/s wr, 14.39k op/s 2024-04-23T14:52:19.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:19 smithi173 ceph-mon[32443]: pgmap v118: 97 pgs: 97 active+clean; 16 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 80 MiB/s wr, 11.20k op/s 2024-04-23T14:52:19.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:19 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:52:19.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:19 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:52:19.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:19 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:52:19.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:19 smithi203 ceph-mon[32800]: pgmap v118: 97 pgs: 97 active+clean; 16 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 80 MiB/s wr, 11.20k op/s 2024-04-23T14:52:19.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:19 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:52:19.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:19 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:52:19.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:19 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:52:20.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:19 smithi083 ceph-mon[27867]: pgmap v118: 97 pgs: 97 active+clean; 16 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 80 MiB/s wr, 11.20k op/s 2024-04-23T14:52:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:19 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:52:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:19 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:52:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:19 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:52:21.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:21 smithi173 ceph-mon[32443]: pgmap v119: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 76 MiB/s wr, 11.07k op/s 2024-04-23T14:52:21.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:21 smithi203 ceph-mon[32800]: pgmap v119: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 76 MiB/s wr, 11.07k op/s 2024-04-23T14:52:22.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:21 smithi083 ceph-mon[27867]: pgmap v119: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 76 MiB/s wr, 11.07k op/s 2024-04-23T14:52:23.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:23 smithi173 ceph-mon[32443]: pgmap v120: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 79 MiB/s wr, 11.77k op/s 2024-04-23T14:52:23.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:23 smithi203 ceph-mon[32800]: pgmap v120: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 79 MiB/s wr, 11.77k op/s 2024-04-23T14:52:24.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:23 smithi083 ceph-mon[27867]: pgmap v120: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 79 MiB/s wr, 11.77k op/s 2024-04-23T14:52:25.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:25 smithi173 ceph-mon[32443]: pgmap v121: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 MiB/s wr, 12.59k op/s 2024-04-23T14:52:25.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:25 smithi203 ceph-mon[32800]: pgmap v121: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 MiB/s wr, 12.59k op/s 2024-04-23T14:52:26.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:25 smithi083 ceph-mon[27867]: pgmap v121: 97 pgs: 97 active+clean; 17 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 MiB/s wr, 12.59k op/s 2024-04-23T14:52:27.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:27 smithi203 ceph-mon[32800]: pgmap v122: 97 pgs: 97 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 94 MiB/s wr, 14.80k op/s 2024-04-23T14:52:28.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:27 smithi083 ceph-mon[27867]: pgmap v122: 97 pgs: 97 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 94 MiB/s wr, 14.80k op/s 2024-04-23T14:52:28.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:27 smithi173 ceph-mon[32443]: pgmap v122: 97 pgs: 97 active+clean; 17 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 94 MiB/s wr, 14.80k op/s 2024-04-23T14:52:29.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:52:29.130 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:52:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.438+0000 7faa7bc00640 1 -- 172.21.15.83:0/4111059576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74109210 msgr2=0x7faa741095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.438+0000 7faa7bc00640 1 --2- 172.21.15.83:0/4111059576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74109210 0x7faa741095f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faa60009860 tx=0x7faa6002a340 comp rx=0 tx=0).stop 2024-04-23T14:52:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.439+0000 7faa7bc00640 1 -- 172.21.15.83:0/4111059576 shutdown_connections 2024-04-23T14:52:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.439+0000 7faa7bc00640 1 --2- 172.21.15.83:0/4111059576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa74115150 0x7faa74117540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.439+0000 7faa7bc00640 1 --2- 172.21.15.83:0/4111059576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74109b30 0x7faa74114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.439+0000 7faa7bc00640 1 --2- 172.21.15.83:0/4111059576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74109210 0x7faa741095f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.439+0000 7faa7bc00640 1 -- 172.21.15.83:0/4111059576 >> 172.21.15.83:0/4111059576 conn(0x7faa7406d110 msgr2=0x7faa7406d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.440+0000 7faa7bc00640 1 -- 172.21.15.83:0/4111059576 shutdown_connections 2024-04-23T14:52:29.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.440+0000 7faa7bc00640 1 -- 172.21.15.83:0/4111059576 wait complete. 2024-04-23T14:52:29.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.443+0000 7faa7bc00640 1 Processor -- start 2024-04-23T14:52:29.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.443+0000 7faa7bc00640 1 -- start start 2024-04-23T14:52:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.444+0000 7faa7bc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74109210 0x7faa741a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.444+0000 7faa7bc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74109b30 0x7faa741a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.444+0000 7faa7bc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa74115150 0x7faa741a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.444+0000 7faa7bc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa7411ce80 con 0x7faa74115150 2024-04-23T14:52:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.444+0000 7faa7bc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faa7411cd00 con 0x7faa74109b30 2024-04-23T14:52:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.444+0000 7faa7bc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa7411d000 con 0x7faa74109210 2024-04-23T14:52:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.445+0000 7faa79200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74109210 0x7faa741a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.445+0000 7faa79200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74109210 0x7faa741a3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:32788/0 (socket says 172.21.15.83:32788) 2024-04-23T14:52:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.445+0000 7faa79200640 1 -- 172.21.15.83:0/471674860 learned_addr learned my addr 172.21.15.83:0/471674860 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.445+0000 7faa73e00640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74109b30 0x7faa741a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.445+0000 7faa79200640 1 -- 172.21.15.83:0/471674860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74109b30 msgr2=0x7faa741a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.445+0000 7faa79200640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74109b30 0x7faa741a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.445+0000 7faa79200640 1 -- 172.21.15.83:0/471674860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa74115150 msgr2=0x7faa741a7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.445+0000 7faa79200640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa74115150 0x7faa741a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.445+0000 7faa79200640 1 -- 172.21.15.83:0/471674860 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa741a83a0 con 0x7faa74109210 2024-04-23T14:52:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.446+0000 7faa79200640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74109210 0x7faa741a3400 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faa60035040 tx=0x7faa60033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.447+0000 7faa71600640 1 -- 172.21.15.83:0/471674860 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60037040 con 0x7faa74109210 2024-04-23T14:52:29.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.447+0000 7faa71600640 1 -- 172.21.15.83:0/471674860 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa60040070 con 0x7faa74109210 2024-04-23T14:52:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.447+0000 7faa71600640 1 -- 172.21.15.83:0/471674860 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600039f0 con 0x7faa74109210 2024-04-23T14:52:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.447+0000 7faa7bc00640 1 -- 172.21.15.83:0/471674860 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa741a8630 con 0x7faa74109210 2024-04-23T14:52:29.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.447+0000 7faa7bc00640 1 -- 172.21.15.83:0/471674860 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa741aff20 con 0x7faa74109210 2024-04-23T14:52:29.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.455+0000 7faa71600640 1 -- 172.21.15.83:0/471674860 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa60003b90 con 0x7faa74109210 2024-04-23T14:52:29.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.455+0000 7faa7e987740 1 Processor -- start 2024-04-23T14:52:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.456+0000 7faa7e987740 1 -- start start 2024-04-23T14:52:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.456+0000 7faa7e987740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 0x5592eb78d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.456+0000 7faa7e987740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb78d990 0x5592eb797b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.456+0000 7faa7e987740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb798060 0x5592eb79a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.456+0000 7faa7e987740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5592eb6ea1f0 con 0x5592eb78d070 2024-04-23T14:52:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.456+0000 7faa7e987740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5592eb690350 con 0x5592eb78d990 2024-04-23T14:52:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.456+0000 7faa7e987740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5592eb6e61b0 con 0x5592eb798060 2024-04-23T14:52:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.457+0000 7faa73e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb798060 0x5592eb79a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.457+0000 7faa73e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb798060 0x5592eb79a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:32802/0 (socket says 172.21.15.83:32802) 2024-04-23T14:52:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.457+0000 7faa73e00640 1 -- 172.21.15.83:0/1572188968 learned_addr learned my addr 172.21.15.83:0/1572188968 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.457+0000 7faa79200640 1 --2- 172.21.15.83:0/1572188968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb78d990 0x5592eb797b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.460 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.457+0000 7faa79c00640 1 --2- 172.21.15.83:0/1572188968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 0x5592eb78d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.459+0000 7faa73e00640 1 -- 172.21.15.83:0/1572188968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb78d990 msgr2=0x5592eb797b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.459+0000 7faa73e00640 1 --2- 172.21.15.83:0/1572188968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb78d990 0x5592eb797b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.459+0000 7faa73e00640 1 -- 172.21.15.83:0/1572188968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 msgr2=0x5592eb78d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.459+0000 7faa73e00640 1 --2- 172.21.15.83:0/1572188968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 0x5592eb78d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.459+0000 7faa73e00640 1 -- 172.21.15.83:0/1572188968 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592eb6e4f50 con 0x5592eb798060 2024-04-23T14:52:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.460+0000 7faa71600640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faa5c0a3da0 0x7faa5c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.459+0000 7faa73e00640 1 --2- 172.21.15.83:0/1572188968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb798060 0x5592eb79a450 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faa68009ca0 tx=0x7faa68011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f77cff5280e8876 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:29.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.460+0000 7faa66000640 1 -- 172.21.15.83:0/1572188968 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68004230 con 0x5592eb798060 2024-04-23T14:52:29.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.460+0000 7faa66000640 1 -- 172.21.15.83:0/1572188968 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa680043d0 con 0x5592eb798060 2024-04-23T14:52:29.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.460+0000 7faa66000640 1 -- 172.21.15.83:0/1572188968 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa680035f0 con 0x5592eb798060 2024-04-23T14:52:29.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.461+0000 7faa71600640 1 -- 172.21.15.83:0/471674860 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7faa600f04c0 con 0x7faa74109210 2024-04-23T14:52:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.461+0000 7faa79c00640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faa5c0a3da0 0x7faa5c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.461+0000 7faa7e987740 1 -- 172.21.15.83:0/1572188968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb798060 msgr2=0x5592eb79a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.461+0000 7faa7e987740 1 --2- 172.21.15.83:0/1572188968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb798060 0x5592eb79a450 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faa68009ca0 tx=0x7faa68011810 comp rx=0 tx=0).stop 2024-04-23T14:52:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.462+0000 7faa7e987740 1 -- 172.21.15.83:0/1572188968 shutdown_connections 2024-04-23T14:52:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.462+0000 7faa7e987740 1 --2- 172.21.15.83:0/1572188968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb798060 0x5592eb79a450 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.462+0000 7faa7e987740 1 --2- 172.21.15.83:0/1572188968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb78d990 0x5592eb797b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.462+0000 7faa7e987740 1 --2- 172.21.15.83:0/1572188968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 0x5592eb78d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.462+0000 7faa7e987740 1 -- 172.21.15.83:0/1572188968 >> 172.21.15.83:0/1572188968 conn(0x5592eb789390 msgr2=0x5592eb789770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:29.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.462+0000 7faa7e987740 1 -- 172.21.15.83:0/1572188968 shutdown_connections 2024-04-23T14:52:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.462+0000 7faa7e987740 1 -- 172.21.15.83:0/1572188968 wait complete. 2024-04-23T14:52:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.463+0000 7faa7e987740 1 -- start start 2024-04-23T14:52:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.463+0000 7faa79c00640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faa5c0a3da0 0x7faa5c0a6260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faa6c009e40 tx=0x7faa6c00d040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:29.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.464+0000 7faa7e987740 1 Processor -- start 2024-04-23T14:52:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.465+0000 7faa7e987740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 0x5592eb794f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.465+0000 7faa7e987740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d990 0x5592eb7954d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.465+0000 7faa7e987740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb798060 0x5592eb795a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.465+0000 7faa7e987740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5592eb79ae50 con 0x5592eb78d070 2024-04-23T14:52:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.465+0000 7faa7e987740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5592eb6a3690 con 0x5592eb798060 2024-04-23T14:52:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.465+0000 7faa7e987740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5592eb79afd0 con 0x5592eb78d990 2024-04-23T14:52:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.465+0000 7faa7e987740 10 client.0 _ll_register_callbacks cb 0x5592eb6df650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:52:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.465+0000 7faa79c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb798060 0x5592eb795a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.465+0000 7faa79c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb798060 0x5592eb795a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38694/0 (socket says 172.21.15.83:38694) 2024-04-23T14:52:29.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.466+0000 7faa79c00640 1 -- 172.21.15.83:0/2533473993 learned_addr learned my addr 172.21.15.83:0/2533473993 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.466+0000 7faa79c00640 1 -- 172.21.15.83:0/2533473993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d990 msgr2=0x5592eb7954d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.466+0000 7faa79c00640 1 --2- 172.21.15.83:0/2533473993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d990 0x5592eb7954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.466+0000 7faa79c00640 1 -- 172.21.15.83:0/2533473993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 msgr2=0x5592eb794f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.466+0000 7faa79c00640 1 --2- 172.21.15.83:0/2533473993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 0x5592eb794f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.466+0000 7faa79c00640 1 -- 172.21.15.83:0/2533473993 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592eb6a2920 con 0x5592eb798060 2024-04-23T14:52:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.467+0000 7faa79c00640 1 --2- 172.21.15.83:0/2533473993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb798060 0x5592eb795a10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faa6c016e10 tx=0x7faa6c004bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac158760c294763 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.467+0000 7faa5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:52:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.467+0000 7faa5aa00640 1 -- 172.21.15.83:0/2533473993 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c025060 con 0x5592eb798060 2024-04-23T14:52:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.467+0000 7faa5aa00640 1 -- 172.21.15.83:0/2533473993 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7faa6c0205f0 con 0x5592eb798060 2024-04-23T14:52:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.468+0000 7faa7e987740 1 -- 172.21.15.83:0/2533473993 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa680043d0 con 0x5592eb798060 2024-04-23T14:52:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.470+0000 7faa5aa00640 1 -- 172.21.15.83:0/2533473993 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faa6c020830 con 0x5592eb798060 2024-04-23T14:52:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.470+0000 7faa7e987740 10 client.26111 fetch_fsmap learned FSMap version 26 2024-04-23T14:52:29.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.470+0000 7faa7e987740 1 -- 172.21.15.83:0/2533473993 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5592eb4c67d0 con 0x5592eb798060 2024-04-23T14:52:29.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.470+0000 7faa5aa00640 1 -- 172.21.15.83:0/2533473993 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa680043d0 con 0x5592eb798060 2024-04-23T14:52:29.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.470+0000 7faa5aa00640 1 -- 172.21.15.83:0/2533473993 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7faa6c029900 con 0x5592eb798060 2024-04-23T14:52:29.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.471+0000 7faa7e987740 10 client.26111 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:52:29.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.471+0000 7faa7e987740 10 client.26111 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:52:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.471+0000 7faa7e987740 20 client.26111 populate_metadata read hostname 'smithi083' 2024-04-23T14:52:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.471+0000 7faa7e987740 1 --2- 172.21.15.83:0/2533473993 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592eb7a83f0 0x5592eb7aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.471+0000 7faa7e987740 4 client.26111 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:52:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.471+0000 7faa7e987740 1 -- 172.21.15.83:0/2533473993 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592eb6f7c10 con 0x5592eb7a83f0 2024-04-23T14:52:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.472+0000 7faa79200640 1 --2- 172.21.15.83:0/2533473993 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592eb7a83f0 0x5592eb7aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.473+0000 7faa79200640 1 --2- 172.21.15.83:0/2533473993 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592eb7a83f0 0x5592eb7aa8b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=304042a1d4acdd9f server_cookie=2d8245cdd1614987 in_seq=0 out_seq=0 2024-04-23T14:52:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.473+0000 7faa5aa00640 10 client.26111 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:52:29.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.474+0000 7faa5aa00640 1 -- 172.21.15.83:0/2533473993 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592eb6f7c10 con 0x5592eb7a83f0 2024-04-23T14:52:29.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.474+0000 7faa5aa00640 10 client.26111 handle_command_reply: tid=0 2024-04-23T14:52:29.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.474+0000 7faa7e987740 1 client.26111 shutdown 2024-04-23T14:52:29.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.475+0000 7faa7e987740 1 -- 172.21.15.83:0/2533473993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb798060 msgr2=0x5592eb795a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.475+0000 7faa7e987740 1 --2- 172.21.15.83:0/2533473993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb798060 0x5592eb795a10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faa6c016e10 tx=0x7faa6c004bc0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.476+0000 7faa7e987740 1 -- 172.21.15.83:0/2533473993 shutdown_connections 2024-04-23T14:52:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.476+0000 7faa7e987740 1 --2- 172.21.15.83:0/2533473993 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592eb7a83f0 0x5592eb7aa8b0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.476+0000 7faa7e987740 1 --2- 172.21.15.83:0/2533473993 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb798060 0x5592eb795a10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.476+0000 7faa7e987740 1 --2- 172.21.15.83:0/2533473993 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d990 0x5592eb7954d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.476+0000 7faa7e987740 1 --2- 172.21.15.83:0/2533473993 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 0x5592eb794f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.476+0000 7faa5aa00640 0 client.26111 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:52:29.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.476+0000 7faa7e987740 1 -- 172.21.15.83:0/2533473993 >> 172.21.15.83:0/2533473993 conn(0x5592eb789390 msgr2=0x5592eb77fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:29.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.476+0000 7faa7e987740 1 -- 172.21.15.83:0/2533473993 shutdown_connections 2024-04-23T14:52:29.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.476+0000 7faa7e987740 1 -- 172.21.15.83:0/2533473993 wait complete. 2024-04-23T14:52:29.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.476+0000 7faa7e987740 20 client.26111 trim_cache size 0 max 16384 2024-04-23T14:52:29.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.507+0000 7faa7e987740 1 Processor -- start 2024-04-23T14:52:29.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.507+0000 7faa7e987740 1 -- start start 2024-04-23T14:52:29.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.507+0000 7faa7e987740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 0x5592eb793ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.507+0000 7faa7e987740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d990 0x5592eb7a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.507+0000 7faa7e987740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb798060 0x5592eb78f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.507+0000 7faa7e987740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa741b6c60 con 0x5592eb78d070 2024-04-23T14:52:29.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.507+0000 7faa7e987740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faa741b4500 con 0x5592eb798060 2024-04-23T14:52:29.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.507+0000 7faa7e987740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa741b2350 con 0x5592eb78d990 2024-04-23T14:52:29.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.508+0000 7faa73e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d990 0x5592eb7a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.508+0000 7faa73e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d990 0x5592eb7a3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:32820/0 (socket says 172.21.15.83:32820) 2024-04-23T14:52:29.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.508+0000 7faa73e00640 1 -- 172.21.15.83:0/948852367 learned_addr learned my addr 172.21.15.83:0/948852367 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:29.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.508+0000 7faa79200640 1 --2- 172.21.15.83:0/948852367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 0x5592eb793ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.509+0000 7faa79200640 1 -- 172.21.15.83:0/948852367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d990 msgr2=0x5592eb7a3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:29.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.509+0000 7faa79200640 1 --2- 172.21.15.83:0/948852367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d990 0x5592eb7a3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.509+0000 7faa79200640 1 -- 172.21.15.83:0/948852367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb798060 msgr2=0x5592eb78f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:29.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.509+0000 7faa79200640 1 --2- 172.21.15.83:0/948852367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb798060 0x5592eb78f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.509+0000 7faa79200640 1 -- 172.21.15.83:0/948852367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592eb3fc3f0 con 0x5592eb78d070 2024-04-23T14:52:29.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.509+0000 7faa79200640 1 --2- 172.21.15.83:0/948852367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 0x5592eb793ed0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7faa600edd30 tx=0x7faa600f29d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5955816513ef786d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:29.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.510+0000 7faa65600640 1 -- 172.21.15.83:0/948852367 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600b8820 con 0x5592eb78d070 2024-04-23T14:52:29.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.511+0000 7faa65600640 1 -- 172.21.15.83:0/948852367 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa600bcc80 con 0x5592eb78d070 2024-04-23T14:52:29.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.511+0000 7faa65600640 1 -- 172.21.15.83:0/948852367 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600b8820 con 0x5592eb78d070 2024-04-23T14:52:29.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.512+0000 7faa7e987740 1 -- 172.21.15.83:0/948852367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 msgr2=0x5592eb793ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:29.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.512+0000 7faa7e987740 1 --2- 172.21.15.83:0/948852367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 0x5592eb793ed0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7faa600edd30 tx=0x7faa600f29d0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.512+0000 7faa7e987740 1 -- 172.21.15.83:0/948852367 shutdown_connections 2024-04-23T14:52:29.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.512+0000 7faa7e987740 1 --2- 172.21.15.83:0/948852367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb798060 0x5592eb78f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.512+0000 7faa7e987740 1 --2- 172.21.15.83:0/948852367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d990 0x5592eb7a3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.512+0000 7faa7e987740 1 --2- 172.21.15.83:0/948852367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb78d070 0x5592eb793ed0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.512+0000 7faa7e987740 1 -- 172.21.15.83:0/948852367 >> 172.21.15.83:0/948852367 conn(0x5592eb789390 msgr2=0x5592eb7957e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:29.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.512+0000 7faa7e987740 1 -- 172.21.15.83:0/948852367 shutdown_connections 2024-04-23T14:52:29.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.513+0000 7faa7e987740 1 -- 172.21.15.83:0/948852367 wait complete. 2024-04-23T14:52:29.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.513+0000 7faa7e987740 1 -- start start 2024-04-23T14:52:29.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.514+0000 7faa7e987740 1 Processor -- start 2024-04-23T14:52:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.515+0000 7faa7e987740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d070 0x5592eb79bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.515+0000 7faa7e987740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb78d990 0x5592eb79c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.515+0000 7faa7e987740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb798060 0x5592eb79c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.515+0000 7faa79200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d070 0x5592eb79bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.515+0000 7faa79200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d070 0x5592eb79bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:32830/0 (socket says 172.21.15.83:32830) 2024-04-23T14:52:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.515+0000 7faa73e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb78d990 0x5592eb79c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa73e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb78d990 0x5592eb79c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38716/0 (socket says 172.21.15.83:38716) 2024-04-23T14:52:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa73e00640 1 -- 172.21.15.83:0/1464813659 learned_addr learned my addr 172.21.15.83:0/1464813659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa7e987740 1 -- 172.21.15.83:0/1464813659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa741b29c0 con 0x5592eb798060 2024-04-23T14:52:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa7e987740 1 -- 172.21.15.83:0/1464813659 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faa741b3e30 con 0x5592eb78d990 2024-04-23T14:52:29.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa7e987740 1 -- 172.21.15.83:0/1464813659 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faa741b3090 con 0x5592eb78d070 2024-04-23T14:52:29.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa79c00640 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb798060 0x5592eb79c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa7e987740 10 client.0 _ll_register_callbacks cb 0x7faa6c009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:52:29.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa73e00640 1 -- 172.21.15.83:0/1464813659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d070 msgr2=0x5592eb79bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:29.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa73e00640 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d070 0x5592eb79bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa73e00640 1 -- 172.21.15.83:0/1464813659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb798060 msgr2=0x5592eb79c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:29.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa73e00640 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb798060 0x5592eb79c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa73e00640 1 -- 172.21.15.83:0/1464813659 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592eb3bc0f0 con 0x5592eb78d990 2024-04-23T14:52:29.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.516+0000 7faa79c00640 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb798060 0x5592eb79c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:52:29.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.517+0000 7faa73e00640 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb78d990 0x5592eb79c0e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa68011850 tx=0x7faa68005710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=786cb2b7eee194f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:29.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.517+0000 7faa5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:52:29.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.517+0000 7faa5aa00640 1 -- 172.21.15.83:0/1464813659 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa680035f0 con 0x5592eb78d990 2024-04-23T14:52:29.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.517+0000 7faa5aa00640 1 -- 172.21.15.83:0/1464813659 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7faa68003790 con 0x5592eb78d990 2024-04-23T14:52:29.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.517+0000 7faa7e987740 1 -- 172.21.15.83:0/1464813659 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa600bcc80 con 0x5592eb78d990 2024-04-23T14:52:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.517+0000 7faa5aa00640 1 -- 172.21.15.83:0/1464813659 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faa680039b0 con 0x5592eb78d990 2024-04-23T14:52:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.518+0000 7faa7e987740 10 client.26114 fetch_fsmap learned FSMap version 26 2024-04-23T14:52:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.518+0000 7faa7e987740 1 -- 172.21.15.83:0/1464813659 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5592eb6af810 con 0x5592eb78d990 2024-04-23T14:52:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.518+0000 7faa5aa00640 1 -- 172.21.15.83:0/1464813659 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa600bcc80 con 0x5592eb78d990 2024-04-23T14:52:29.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.518+0000 7faa5aa00640 1 -- 172.21.15.83:0/1464813659 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7faa68018070 con 0x5592eb78d990 2024-04-23T14:52:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.518+0000 7faa7e987740 10 client.26114 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:52:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.518+0000 7faa7e987740 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:52:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.518+0000 7faa7e987740 20 client.26114 populate_metadata read hostname 'smithi083' 2024-04-23T14:52:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.518+0000 7faa7e987740 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592eb80eee0 0x5592eb811300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.518+0000 7faa7e987740 4 client.26114 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:52:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.518+0000 7faa7e987740 1 -- 172.21.15.83:0/1464813659 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5592eb6f1a80 con 0x5592eb80eee0 2024-04-23T14:52:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.519+0000 7faa79200640 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592eb80eee0 0x5592eb811300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.520+0000 7faa5aa00640 10 client.26114 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:52:29.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.520+0000 7faa79200640 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592eb80eee0 0x5592eb811300 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af84b761a653a23 server_cookie=c3b0428042fe1237 in_seq=0 out_seq=0 2024-04-23T14:52:29.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.520+0000 7faa5aa00640 1 -- 172.21.15.83:0/1464813659 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5592eb6f1a80 con 0x5592eb80eee0 2024-04-23T14:52:29.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.520+0000 7faa5aa00640 10 client.26114 handle_command_reply: tid=0 2024-04-23T14:52:29.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.520+0000 7faa7e987740 1 client.26114 shutdown 2024-04-23T14:52:29.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.522+0000 7faa7e987740 1 -- 172.21.15.83:0/1464813659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb78d990 msgr2=0x5592eb79c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:29.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.522+0000 7faa7e987740 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb78d990 0x5592eb79c0e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faa68011850 tx=0x7faa68005710 comp rx=0 tx=0).stop 2024-04-23T14:52:29.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.522+0000 7faa7e987740 1 -- 172.21.15.83:0/1464813659 shutdown_connections 2024-04-23T14:52:29.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.522+0000 7faa7e987740 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592eb80eee0 0x5592eb811300 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.522+0000 7faa7e987740 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592eb798060 0x5592eb79c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.522+0000 7faa7e987740 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592eb78d990 0x5592eb79c0e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.522+0000 7faa7e987740 1 --2- 172.21.15.83:0/1464813659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592eb78d070 0x5592eb79bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.522+0000 7faa5aa00640 0 client.26114 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:52:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.522+0000 7faa7e987740 1 -- 172.21.15.83:0/1464813659 >> 172.21.15.83:0/1464813659 conn(0x5592eb789390 msgr2=0x5592eb794f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.523+0000 7faa7e987740 1 -- 172.21.15.83:0/1464813659 shutdown_connections 2024-04-23T14:52:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.523+0000 7faa7e987740 1 -- 172.21.15.83:0/1464813659 wait complete. 2024-04-23T14:52:29.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.523+0000 7faa7e987740 20 client.26114 trim_cache size 0 max 16384 2024-04-23T14:52:29.525 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:52:29.525 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:52:29.525 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:52:29.525 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:52:29.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.524+0000 7faa7bc00640 1 -- 172.21.15.83:0/471674860 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faa5c0a3da0 msgr2=0x7faa5c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:29.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.524+0000 7faa7bc00640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faa5c0a3da0 0x7faa5c0a6260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faa6c009e40 tx=0x7faa6c00d040 comp rx=0 tx=0).stop 2024-04-23T14:52:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.524+0000 7faa7bc00640 1 -- 172.21.15.83:0/471674860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74109210 msgr2=0x7faa741a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.524+0000 7faa7bc00640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74109210 0x7faa741a3400 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faa60035040 tx=0x7faa60033d90 comp rx=0 tx=0).stop 2024-04-23T14:52:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.524+0000 7faa7bc00640 1 -- 172.21.15.83:0/471674860 shutdown_connections 2024-04-23T14:52:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.524+0000 7faa7bc00640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faa5c0a3da0 0x7faa5c0a6260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.524+0000 7faa7bc00640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa74115150 0x7faa741a7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.524+0000 7faa7bc00640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74109b30 0x7faa741a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.524+0000 7faa7bc00640 1 --2- 172.21.15.83:0/471674860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74109210 0x7faa741a3400 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.524+0000 7faa7bc00640 1 -- 172.21.15.83:0/471674860 >> 172.21.15.83:0/471674860 conn(0x7faa7406d110 msgr2=0x7faa74070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.525+0000 7faa7bc00640 1 -- 172.21.15.83:0/471674860 shutdown_connections 2024-04-23T14:52:29.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.525+0000 7faa7bc00640 1 -- 172.21.15.83:0/471674860 wait complete. 2024-04-23T14:52:29.552 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:52:29.553 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:52:29.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.852+0000 7fcd8ce00640 1 -- 172.21.15.83:0/1640595791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd88108fd0 msgr2=0x7fcd881154e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.852+0000 7fcd8ce00640 1 --2- 172.21.15.83:0/1640595791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd88108fd0 0x7fcd881154e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009cd0 tx=0x7fcd8002a8e0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.852+0000 7fcd8ce00640 1 -- 172.21.15.83:0/1640595791 shutdown_connections 2024-04-23T14:52:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.852+0000 7fcd8ce00640 1 --2- 172.21.15.83:0/1640595791 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd88108fd0 0x7fcd881154e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.852+0000 7fcd8ce00640 1 --2- 172.21.15.83:0/1640595791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd88108610 0x7fcd88108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.852+0000 7fcd8ce00640 1 --2- 172.21.15.83:0/1640595791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd8810e6b0 0x7fcd8810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.852+0000 7fcd8ce00640 1 -- 172.21.15.83:0/1640595791 >> 172.21.15.83:0/1640595791 conn(0x7fcd88075800 msgr2=0x7fcd88106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.852+0000 7fcd8ce00640 1 -- 172.21.15.83:0/1640595791 shutdown_connections 2024-04-23T14:52:29.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.852+0000 7fcd8ce00640 1 -- 172.21.15.83:0/1640595791 wait complete. 2024-04-23T14:52:29.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.854+0000 7fcd8ce00640 1 Processor -- start 2024-04-23T14:52:29.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.854+0000 7fcd8ce00640 1 -- start start 2024-04-23T14:52:29.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.855+0000 7fcd8ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd88108610 0x7fcd880824f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.856+0000 7fcd8ce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd88108fd0 0x7fcd88082a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.856+0000 7fcd8ce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8810e6b0 0x7fcd8807b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.856+0000 7fcd8ce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd88118a40 con 0x7fcd88108610 2024-04-23T14:52:29.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.856+0000 7fcd8ce00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcd881188c0 con 0x7fcd8810e6b0 2024-04-23T14:52:29.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.856+0000 7fcd8ce00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd88118bc0 con 0x7fcd88108fd0 2024-04-23T14:52:29.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.857+0000 7fcd85400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd88108fd0 0x7fcd88082a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.858+0000 7fcd86800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8810e6b0 0x7fcd8807b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.858+0000 7fcd85e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd88108610 0x7fcd880824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.858+0000 7fcd86800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8810e6b0 0x7fcd8807b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38740/0 (socket says 172.21.15.83:38740) 2024-04-23T14:52:29.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.858+0000 7fcd86800640 1 -- 172.21.15.83:0/4259650698 learned_addr learned my addr 172.21.15.83:0/4259650698 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:29.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.858+0000 7fcd86800640 1 -- 172.21.15.83:0/4259650698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd88108fd0 msgr2=0x7fcd88082a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.858+0000 7fcd86800640 1 --2- 172.21.15.83:0/4259650698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd88108fd0 0x7fcd88082a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.858+0000 7fcd86800640 1 -- 172.21.15.83:0/4259650698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd88108610 msgr2=0x7fcd880824f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.858+0000 7fcd86800640 1 --2- 172.21.15.83:0/4259650698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd88108610 0x7fcd880824f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:29.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.858+0000 7fcd86800640 1 -- 172.21.15.83:0/4259650698 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8807bf20 con 0x7fcd8810e6b0 2024-04-23T14:52:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.861+0000 7fcd86800640 1 --2- 172.21.15.83:0/4259650698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8810e6b0 0x7fcd8807b660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd80005e30 tx=0x7fcd80003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.861+0000 7fcd7ea00640 1 -- 172.21.15.83:0/4259650698 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80004240 con 0x7fcd8810e6b0 2024-04-23T14:52:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.861+0000 7fcd8ce00640 1 -- 172.21.15.83:0/4259650698 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8807c1b0 con 0x7fcd8810e6b0 2024-04-23T14:52:29.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.861+0000 7fcd8ce00640 1 -- 172.21.15.83:0/4259650698 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd880801f0 con 0x7fcd8810e6b0 2024-04-23T14:52:29.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.862+0000 7fcd7ea00640 1 -- 172.21.15.83:0/4259650698 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd800098b0 con 0x7fcd8810e6b0 2024-04-23T14:52:29.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.862+0000 7fcd7ea00640 1 -- 172.21.15.83:0/4259650698 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80030820 con 0x7fcd8810e6b0 2024-04-23T14:52:29.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.863+0000 7fcd8ce00640 1 -- 172.21.15.83:0/4259650698 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd68005290 con 0x7fcd8810e6b0 2024-04-23T14:52:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.867+0000 7fcd7ea00640 1 -- 172.21.15.83:0/4259650698 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcd800043e0 con 0x7fcd8810e6b0 2024-04-23T14:52:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.870+0000 7fcd7ea00640 1 --2- 172.21.15.83:0/4259650698 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcd6c0a3de0 0x7fcd6c0a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.870+0000 7fcd7ea00640 1 -- 172.21.15.83:0/4259650698 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcd800f16e0 con 0x7fcd8810e6b0 2024-04-23T14:52:29.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.871+0000 7fcd85e00640 1 --2- 172.21.15.83:0/4259650698 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcd6c0a3de0 0x7fcd6c0a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:29.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.873+0000 7fcd85e00640 1 --2- 172.21.15.83:0/4259650698 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcd6c0a3de0 0x7fcd6c0a62a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcd88109750 tx=0x7fcd74005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:29.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:29.875+0000 7fcd7ea00640 1 -- 172.21.15.83:0/4259650698 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd800bded0 con 0x7fcd8810e6b0 2024-04-23T14:52:29.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:29 smithi203 ceph-mon[32800]: pgmap v123: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 64 MiB/s wr, 11.34k op/s 2024-04-23T14:52:30.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:29 smithi083 ceph-mon[27867]: pgmap v123: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 64 MiB/s wr, 11.34k op/s 2024-04-23T14:52:30.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:29 smithi173 ceph-mon[32443]: pgmap v123: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 64 MiB/s wr, 11.34k op/s 2024-04-23T14:52:30.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.342+0000 7fcd8ce00640 1 -- 172.21.15.83:0/4259650698 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd68005580 con 0x7fcd8810e6b0 2024-04-23T14:52:30.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.344+0000 7fcd7ea00640 1 -- 172.21.15.83:0/4259650698 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11867 (secure 0 0 0) 0x7fcd800bd7d0 con 0x7fcd8810e6b0 2024-04-23T14:52:30.346 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:52:30.346 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:50:17.708313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:52:30.346 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 26 2024-04-23T14:52:30.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.348+0000 7fcd8ce00640 1 -- 172.21.15.83:0/4259650698 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcd6c0a3de0 msgr2=0x7fcd6c0a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:30.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.348+0000 7fcd8ce00640 1 --2- 172.21.15.83:0/4259650698 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcd6c0a3de0 0x7fcd6c0a62a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcd88109750 tx=0x7fcd74005f90 comp rx=0 tx=0).stop 2024-04-23T14:52:30.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.348+0000 7fcd8ce00640 1 -- 172.21.15.83:0/4259650698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8810e6b0 msgr2=0x7fcd8807b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:30.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.348+0000 7fcd8ce00640 1 --2- 172.21.15.83:0/4259650698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8810e6b0 0x7fcd8807b660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd80005e30 tx=0x7fcd80003f90 comp rx=0 tx=0).stop 2024-04-23T14:52:30.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.348+0000 7fcd8ce00640 1 -- 172.21.15.83:0/4259650698 shutdown_connections 2024-04-23T14:52:30.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.348+0000 7fcd8ce00640 1 --2- 172.21.15.83:0/4259650698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd8810e6b0 0x7fcd8807b660 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.348+0000 7fcd8ce00640 1 --2- 172.21.15.83:0/4259650698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd88108fd0 0x7fcd88082a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.348+0000 7fcd8ce00640 1 --2- 172.21.15.83:0/4259650698 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcd6c0a3de0 0x7fcd6c0a62a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.348+0000 7fcd8ce00640 1 --2- 172.21.15.83:0/4259650698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd88108610 0x7fcd880824f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.348+0000 7fcd8ce00640 1 -- 172.21.15.83:0/4259650698 >> 172.21.15.83:0/4259650698 conn(0x7fcd88075800 msgr2=0x7fcd88076560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:30.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.350+0000 7fcd8ce00640 1 -- 172.21.15.83:0/4259650698 shutdown_connections 2024-04-23T14:52:30.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.350+0000 7fcd8ce00640 1 -- 172.21.15.83:0/4259650698 wait complete. 2024-04-23T14:52:30.387 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:52:30.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:30 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/4259650698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:52:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.761+0000 7f0691400640 1 -- 172.21.15.83:0/3153535781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f068c109a80 msgr2=0x7f068c11d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.761+0000 7f0691400640 1 --2- 172.21.15.83:0/3153535781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f068c109a80 0x7f068c11d7f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0678009860 tx=0x7f067802a3d0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.762+0000 7f0691400640 1 -- 172.21.15.83:0/3153535781 shutdown_connections 2024-04-23T14:52:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.762+0000 7f0691400640 1 --2- 172.21.15.83:0/3153535781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f068c109a80 0x7f068c11d7f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.762+0000 7f0691400640 1 --2- 172.21.15.83:0/3153535781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f068c109000 0x7f068c109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.762+0000 7f0691400640 1 --2- 172.21.15.83:0/3153535781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f068c1106a0 0x7f068c110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.762+0000 7f0691400640 1 -- 172.21.15.83:0/3153535781 >> 172.21.15.83:0/3153535781 conn(0x7f068c06cf80 msgr2=0x7f068c06d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:30.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.762+0000 7f0691400640 1 -- 172.21.15.83:0/3153535781 shutdown_connections 2024-04-23T14:52:30.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.762+0000 7f0691400640 1 -- 172.21.15.83:0/3153535781 wait complete. 2024-04-23T14:52:30.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.763+0000 7f0691400640 1 Processor -- start 2024-04-23T14:52:30.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.764+0000 7f0691400640 1 -- start start 2024-04-23T14:52:30.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.764+0000 7f0691400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f068c109000 0x7f068c11d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.764+0000 7f0691400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f068c109a80 0x7f068c11d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.764+0000 7f0691400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f068c1106a0 0x7f068c116570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.764+0000 7f068b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f068c109a80 0x7f068c11d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.764+0000 7f0691400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f068c121090 con 0x7f068c109000 2024-04-23T14:52:30.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.764+0000 7f0691400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f068c120f10 con 0x7f068c109a80 2024-04-23T14:52:30.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.764+0000 7f0691400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f068c121210 con 0x7f068c1106a0 2024-04-23T14:52:30.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.765+0000 7f068be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f068c109000 0x7f068c11d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.765+0000 7f068be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f068c109000 0x7f068c11d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46116/0 (socket says 172.21.15.83:46116) 2024-04-23T14:52:30.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.765+0000 7f068be00640 1 -- 172.21.15.83:0/1120966952 learned_addr learned my addr 172.21.15.83:0/1120966952 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:30.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.765+0000 7f0690a00640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f068c1106a0 0x7f068c116570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.765+0000 7f068be00640 1 -- 172.21.15.83:0/1120966952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f068c1106a0 msgr2=0x7f068c116570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:30.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.765+0000 7f068be00640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f068c1106a0 0x7f068c116570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.765+0000 7f068be00640 1 -- 172.21.15.83:0/1120966952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f068c109a80 msgr2=0x7f068c11d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:30.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.765+0000 7f068be00640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f068c109a80 0x7f068c11d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.765+0000 7f068be00640 1 -- 172.21.15.83:0/1120966952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c116e30 con 0x7f068c109000 2024-04-23T14:52:30.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.766+0000 7f068be00640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f068c109000 0x7f068c11d310 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f068400efb0 tx=0x7f068400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:30.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.766+0000 7f0688c00640 1 -- 172.21.15.83:0/1120966952 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068401b070 con 0x7f068c109000 2024-04-23T14:52:30.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.766+0000 7f0688c00640 1 -- 172.21.15.83:0/1120966952 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068400ba50 con 0x7f068c109000 2024-04-23T14:52:30.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.766+0000 7f0688c00640 1 -- 172.21.15.83:0/1120966952 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0684016b90 con 0x7f068c109000 2024-04-23T14:52:30.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.767+0000 7f0691400640 1 -- 172.21.15.83:0/1120966952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068c117120 con 0x7f068c109000 2024-04-23T14:52:30.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.767+0000 7f0691400640 1 -- 172.21.15.83:0/1120966952 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f068c11b070 con 0x7f068c109000 2024-04-23T14:52:30.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.774+0000 7f06957a2740 1 Processor -- start 2024-04-23T14:52:30.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.774+0000 7f06957a2740 1 -- start start 2024-04-23T14:52:30.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.774+0000 7f0688c00640 1 -- 172.21.15.83:0/1120966952 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f068401e020 con 0x7f068c109000 2024-04-23T14:52:30.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.775+0000 7f06957a2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 0x55e8797acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.775+0000 7f06957a2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797ad460 0x55e8797b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.775+0000 7f06957a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797b7b30 0x55e8797b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.775+0000 7f06957a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e8797121d0 con 0x55e8797b7b30 2024-04-23T14:52:30.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.775+0000 7f06957a2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e8796b8450 con 0x55e8797acb40 2024-04-23T14:52:30.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.775+0000 7f06957a2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e87970e190 con 0x55e8797ad460 2024-04-23T14:52:30.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.775+0000 7f0690a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 0x55e8797acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.775+0000 7f0690a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 0x55e8797acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38784/0 (socket says 172.21.15.83:38784) 2024-04-23T14:52:30.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.775+0000 7f0690a00640 1 -- 172.21.15.83:0/1552485589 learned_addr learned my addr 172.21.15.83:0/1552485589 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:30.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.775+0000 7f068be00640 1 --2- 172.21.15.83:0/1552485589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797ad460 0x55e8797b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.776+0000 7f068b400640 1 --2- 172.21.15.83:0/1552485589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797b7b30 0x55e8797b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.776+0000 7f068be00640 1 -- 172.21.15.83:0/1552485589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 msgr2=0x55e8797acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:30.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.776+0000 7f068be00640 1 --2- 172.21.15.83:0/1552485589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 0x55e8797acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.776+0000 7f068be00640 1 -- 172.21.15.83:0/1552485589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797b7b30 msgr2=0x55e8797b9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:30.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.776+0000 7f068be00640 1 --2- 172.21.15.83:0/1552485589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797b7b30 0x55e8797b9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.776+0000 7f068be00640 1 -- 172.21.15.83:0/1552485589 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e879775310 con 0x55e8797ad460 2024-04-23T14:52:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.777+0000 7f068be00640 1 --2- 172.21.15.83:0/1552485589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797ad460 0x55e8797b75f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f06840cdf00 tx=0x7f06840d6cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=396033f1d064981b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.777+0000 7f0681200640 1 -- 172.21.15.83:0/1552485589 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06840df070 con 0x55e8797ad460 2024-04-23T14:52:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.777+0000 7f0681200640 1 -- 172.21.15.83:0/1552485589 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06840d8d30 con 0x55e8797ad460 2024-04-23T14:52:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.777+0000 7f0681200640 1 -- 172.21.15.83:0/1552485589 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06840db420 con 0x55e8797ad460 2024-04-23T14:52:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.778+0000 7f0688c00640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06700a3e60 0x7f06700a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.778+0000 7f0690a00640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06700a3e60 0x7f06700a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.779+0000 7f06957a2740 1 -- 172.21.15.83:0/1552485589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797ad460 msgr2=0x55e8797b75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.779+0000 7f06957a2740 1 --2- 172.21.15.83:0/1552485589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797ad460 0x55e8797b75f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f06840cdf00 tx=0x7f06840d6cb0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.779+0000 7f0688c00640 1 -- 172.21.15.83:0/1120966952 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f06840d0800 con 0x7f068c109000 2024-04-23T14:52:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.780+0000 7f06957a2740 1 -- 172.21.15.83:0/1552485589 shutdown_connections 2024-04-23T14:52:30.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.780+0000 7f0690a00640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06700a3e60 0x7f06700a6320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0678034000 tx=0x7f0678032fb0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.780+0000 7f06957a2740 1 --2- 172.21.15.83:0/1552485589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797b7b30 0x55e8797b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.780+0000 7f06957a2740 1 --2- 172.21.15.83:0/1552485589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797ad460 0x55e8797b75f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.780+0000 7f06957a2740 1 --2- 172.21.15.83:0/1552485589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 0x55e8797acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.780+0000 7f06957a2740 1 -- 172.21.15.83:0/1552485589 >> 172.21.15.83:0/1552485589 conn(0x55e8797a8e60 msgr2=0x55e8797a9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.781+0000 7f06957a2740 1 -- 172.21.15.83:0/1552485589 shutdown_connections 2024-04-23T14:52:30.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.781+0000 7f06957a2740 1 -- 172.21.15.83:0/1552485589 wait complete. 2024-04-23T14:52:30.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.781+0000 7f06957a2740 1 -- start start 2024-04-23T14:52:30.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.783+0000 7f06957a2740 1 Processor -- start 2024-04-23T14:52:30.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.785+0000 7f06957a2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797acb40 0x55e8797b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.785+0000 7f06957a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797ad460 0x55e8797b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.785+0000 7f06957a2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797b7b30 0x55e8797b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.786+0000 7f068be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797acb40 0x55e8797b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.786+0000 7f06957a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e8797ba9c0 con 0x55e8797ad460 2024-04-23T14:52:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.786+0000 7f06957a2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e8796cb810 con 0x55e8797b7b30 2024-04-23T14:52:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.786+0000 7f06957a2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e8797bab40 con 0x55e8797acb40 2024-04-23T14:52:30.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.786+0000 7f06957a2740 10 client.0 _ll_register_callbacks cb 0x55e879699140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:52:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.786+0000 7f068be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797acb40 0x55e8797b75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:32900/0 (socket says 172.21.15.83:32900) 2024-04-23T14:52:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.786+0000 7f068be00640 1 -- 172.21.15.83:0/216931127 learned_addr learned my addr 172.21.15.83:0/216931127 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.786+0000 7f0690a00640 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797b7b30 0x55e8797b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.787+0000 7f068b400640 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797ad460 0x55e8797b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.787+0000 7f068be00640 1 -- 172.21.15.83:0/216931127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797b7b30 msgr2=0x55e8797b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.787+0000 7f068be00640 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797b7b30 0x55e8797b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.787+0000 7f068be00640 1 -- 172.21.15.83:0/216931127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797ad460 msgr2=0x55e8797b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.787+0000 7f068be00640 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797ad460 0x55e8797b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.787+0000 7f068be00640 1 -- 172.21.15.83:0/216931127 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8796dc3f0 con 0x55e8797acb40 2024-04-23T14:52:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.787+0000 7f068b400640 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797ad460 0x55e8797b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:52:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.788+0000 7f068be00640 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797acb40 0x55e8797b75e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f068400ef80 tx=0x7f06840165f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=350c61922b82a3ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:30.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.788+0000 7f0675600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:52:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.788+0000 7f0675600640 1 -- 172.21.15.83:0/216931127 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06840df070 con 0x55e8797acb40 2024-04-23T14:52:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.791+0000 7f0675600640 1 -- 172.21.15.83:0/216931127 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f0684098dc0 con 0x55e8797acb40 2024-04-23T14:52:30.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.791+0000 7f06957a2740 1 -- 172.21.15.83:0/216931127 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06840d8d30 con 0x55e8797acb40 2024-04-23T14:52:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.792+0000 7f0675600640 1 -- 172.21.15.83:0/216931127 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f06840989f0 con 0x55e8797acb40 2024-04-23T14:52:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.792+0000 7f06957a2740 10 client.26080 fetch_fsmap learned FSMap version 26 2024-04-23T14:52:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.792+0000 7f06957a2740 1 -- 172.21.15.83:0/216931127 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8794243f0 con 0x55e8797acb40 2024-04-23T14:52:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.792+0000 7f0675600640 1 -- 172.21.15.83:0/216931127 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06840d8d30 con 0x55e8797acb40 2024-04-23T14:52:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.792+0000 7f0675600640 1 -- 172.21.15.83:0/216931127 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f068400b040 con 0x55e8797acb40 2024-04-23T14:52:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.792+0000 7f06957a2740 10 client.26080 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:52:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.792+0000 7f06957a2740 10 client.26080 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:52:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.793+0000 7f06957a2740 20 client.26080 populate_metadata read hostname 'smithi083' 2024-04-23T14:52:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.793+0000 7f06957a2740 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55e8797c7bf0 0x55e8797c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.793+0000 7f06957a2740 4 client.26080 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:52:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.793+0000 7f06957a2740 1 -- 172.21.15.83:0/216931127 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e87971fbf0 con 0x55e8797c7bf0 2024-04-23T14:52:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.794+0000 7f068b400640 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55e8797c7bf0 0x55e8797c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.795+0000 7f0675600640 10 client.26080 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:52:30.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.795+0000 7f068b400640 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55e8797c7bf0 0x55e8797c9fe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc4d394d12a8594f server_cookie=32b88532157fcdc7 in_seq=0 out_seq=0 2024-04-23T14:52:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.796+0000 7f0675600640 1 -- 172.21.15.83:0/216931127 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e87971fbf0 con 0x55e8797c7bf0 2024-04-23T14:52:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.796+0000 7f0675600640 10 client.26080 handle_command_reply: tid=0 2024-04-23T14:52:30.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.796+0000 7f06957a2740 1 client.26080 shutdown 2024-04-23T14:52:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.797+0000 7f06957a2740 1 -- 172.21.15.83:0/216931127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797acb40 msgr2=0x55e8797b75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:30.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.797+0000 7f06957a2740 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797acb40 0x55e8797b75e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f068400ef80 tx=0x7f06840165f0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.799+0000 7f06957a2740 1 -- 172.21.15.83:0/216931127 shutdown_connections 2024-04-23T14:52:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.799+0000 7f06957a2740 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55e8797c7bf0 0x55e8797c9fe0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.799+0000 7f06957a2740 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797b7b30 0x55e8797b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.799+0000 7f06957a2740 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797ad460 0x55e8797b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.799+0000 7f06957a2740 1 --2- 172.21.15.83:0/216931127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797acb40 0x55e8797b75e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.799+0000 7f06957a2740 1 -- 172.21.15.83:0/216931127 >> 172.21.15.83:0/216931127 conn(0x55e8797a8e60 msgr2=0x55e8797aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:30.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.799+0000 7f0675600640 0 client.26080 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:52:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.800+0000 7f06957a2740 1 -- 172.21.15.83:0/216931127 shutdown_connections 2024-04-23T14:52:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.800+0000 7f06957a2740 1 -- 172.21.15.83:0/216931127 wait complete. 2024-04-23T14:52:30.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.800+0000 7f06957a2740 20 client.26080 trim_cache size 0 max 16384 2024-04-23T14:52:30.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.836+0000 7f06957a2740 1 Processor -- start 2024-04-23T14:52:30.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.836+0000 7f06957a2740 1 -- start start 2024-04-23T14:52:30.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.836+0000 7f06957a2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 0x55e8797a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.836+0000 7f06957a2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797ad460 0x55e8797b7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.836+0000 7f06957a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797b7b30 0x55e8797cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.836+0000 7f06957a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f068c1c7a20 con 0x55e8797b7b30 2024-04-23T14:52:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.836+0000 7f06957a2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f068c1c52c0 con 0x55e8797acb40 2024-04-23T14:52:30.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.836+0000 7f06957a2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f068c1c3110 con 0x55e8797ad460 2024-04-23T14:52:30.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.837+0000 7f068be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 0x55e8797a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.837+0000 7f068be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 0x55e8797a7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38810/0 (socket says 172.21.15.83:38810) 2024-04-23T14:52:30.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.837+0000 7f068b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797ad460 0x55e8797b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.837+0000 7f068be00640 1 -- 172.21.15.83:0/2766168295 learned_addr learned my addr 172.21.15.83:0/2766168295 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:30.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.838+0000 7f0690a00640 1 --2- 172.21.15.83:0/2766168295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797b7b30 0x55e8797cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.838+0000 7f068b400640 1 -- 172.21.15.83:0/2766168295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 msgr2=0x55e8797a7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.838+0000 7f068b400640 1 --2- 172.21.15.83:0/2766168295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 0x55e8797a7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.838+0000 7f068b400640 1 -- 172.21.15.83:0/2766168295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797b7b30 msgr2=0x55e8797cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.838+0000 7f068b400640 1 --2- 172.21.15.83:0/2766168295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797b7b30 0x55e8797cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.838+0000 7f068b400640 1 -- 172.21.15.83:0/2766168295 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8794ee7d0 con 0x55e8797ad460 2024-04-23T14:52:30.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.839+0000 7f068be00640 1 --2- 172.21.15.83:0/2766168295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 0x55e8797a7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:52:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.839+0000 7f0690a00640 1 --2- 172.21.15.83:0/2766168295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797b7b30 0x55e8797cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:52:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.839+0000 7f068b400640 1 --2- 172.21.15.83:0/2766168295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797ad460 0x55e8797b7110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55e8797bac70 tx=0x7f067c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ae35fb27fb561c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:30.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.840+0000 7f0677e00640 1 -- 172.21.15.83:0/2766168295 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c0139f0 con 0x55e8797ad460 2024-04-23T14:52:30.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.840+0000 7f0677e00640 1 -- 172.21.15.83:0/2766168295 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f067c005500 con 0x55e8797ad460 2024-04-23T14:52:30.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.840+0000 7f0677e00640 1 -- 172.21.15.83:0/2766168295 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c01a890 con 0x55e8797ad460 2024-04-23T14:52:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.841+0000 7f06957a2740 1 -- 172.21.15.83:0/2766168295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797ad460 msgr2=0x55e8797b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:30.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.842+0000 7f06957a2740 1 --2- 172.21.15.83:0/2766168295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797ad460 0x55e8797b7110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55e8797bac70 tx=0x7f067c005f90 comp rx=0 tx=0).stop 2024-04-23T14:52:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.844+0000 7f06957a2740 1 -- 172.21.15.83:0/2766168295 shutdown_connections 2024-04-23T14:52:30.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.844+0000 7f06957a2740 1 --2- 172.21.15.83:0/2766168295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797b7b30 0x55e8797cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.844+0000 7f06957a2740 1 --2- 172.21.15.83:0/2766168295 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797ad460 0x55e8797b7110 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.844+0000 7f06957a2740 1 --2- 172.21.15.83:0/2766168295 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797acb40 0x55e8797a7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.844+0000 7f06957a2740 1 -- 172.21.15.83:0/2766168295 >> 172.21.15.83:0/2766168295 conn(0x55e8797a8e60 msgr2=0x55e8797a76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:30.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.847+0000 7f06957a2740 1 -- 172.21.15.83:0/2766168295 shutdown_connections 2024-04-23T14:52:30.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.847+0000 7f06957a2740 1 -- 172.21.15.83:0/2766168295 wait complete. 2024-04-23T14:52:30.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.847+0000 7f06957a2740 1 -- start start 2024-04-23T14:52:30.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.848+0000 7f06957a2740 1 Processor -- start 2024-04-23T14:52:30.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.848+0000 7f06957a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797acb40 0x55e8797cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.848+0000 7f06957a2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797ad460 0x55e8797cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.848+0000 7f06957a2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797b7b30 0x55e8797ca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.848+0000 7f06957a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f068c1c3780 con 0x55e8797acb40 2024-04-23T14:52:30.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.848+0000 7f06957a2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f068c1c4bf0 con 0x55e8797ad460 2024-04-23T14:52:30.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.848+0000 7f06957a2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f068c1c3e50 con 0x55e8797b7b30 2024-04-23T14:52:30.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.848+0000 7f06957a2740 10 client.0 _ll_register_callbacks cb 0x7f067c00b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:52:30.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.849+0000 7f0690a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797b7b30 0x55e8797ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.849+0000 7f068b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797ad460 0x55e8797cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.849+0000 7f0690a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797b7b30 0x55e8797ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:32916/0 (socket says 172.21.15.83:32916) 2024-04-23T14:52:30.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.849+0000 7f068b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797ad460 0x55e8797cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38814/0 (socket says 172.21.15.83:38814) 2024-04-23T14:52:30.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.849+0000 7f068b400640 1 -- 172.21.15.83:0/452445203 learned_addr learned my addr 172.21.15.83:0/452445203 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:30.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.850+0000 7f0690a00640 1 -- 172.21.15.83:0/452445203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797ad460 msgr2=0x55e8797cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:30.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.850+0000 7f0690a00640 1 --2- 172.21.15.83:0/452445203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797ad460 0x55e8797cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.850+0000 7f0690a00640 1 -- 172.21.15.83:0/452445203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797acb40 msgr2=0x55e8797cd010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:52:30.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.850+0000 7f0690a00640 1 --2- 172.21.15.83:0/452445203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797acb40 0x55e8797cd010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.850+0000 7f0690a00640 1 -- 172.21.15.83:0/452445203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8797ca990 con 0x55e8797b7b30 2024-04-23T14:52:30.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.850+0000 7f0690a00640 1 --2- 172.21.15.83:0/452445203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797b7b30 0x55e8797ca190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0678009780 tx=0x7f0678005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e003f5a047d9982b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:30.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.850+0000 7f0675600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:52:30.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.851+0000 7f0675600640 1 -- 172.21.15.83:0/452445203 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678058060 con 0x55e8797b7b30 2024-04-23T14:52:30.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.851+0000 7f0675600640 1 -- 172.21.15.83:0/452445203 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f06780054d0 con 0x55e8797b7b30 2024-04-23T14:52:30.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.852+0000 7f06957a2740 1 -- 172.21.15.83:0/452445203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f067c005500 con 0x55e8797b7b30 2024-04-23T14:52:30.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.852+0000 7f0675600640 1 -- 172.21.15.83:0/452445203 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f06780046c0 con 0x55e8797b7b30 2024-04-23T14:52:30.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.852+0000 7f06957a2740 10 client.26086 fetch_fsmap learned FSMap version 26 2024-04-23T14:52:30.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.852+0000 7f06957a2740 1 -- 172.21.15.83:0/452445203 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e87982e970 con 0x55e8797b7b30 2024-04-23T14:52:30.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.852+0000 7f0675600640 1 -- 172.21.15.83:0/452445203 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f067c005500 con 0x55e8797b7b30 2024-04-23T14:52:30.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.853+0000 7f0675600640 1 -- 172.21.15.83:0/452445203 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f0678053850 con 0x55e8797b7b30 2024-04-23T14:52:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.853+0000 7f06957a2740 10 client.26086 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T14:52:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.853+0000 7f06957a2740 10 client.26086 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:52:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.853+0000 7f06957a2740 20 client.26086 populate_metadata read hostname 'smithi083' 2024-04-23T14:52:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.853+0000 7f06957a2740 1 --2- 172.21.15.83:0/452445203 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55e87982f100 0x55e879831520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.853+0000 7f06957a2740 4 client.26086 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:52:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.853+0000 7f06957a2740 1 -- 172.21.15.83:0/452445203 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e879719a60 con 0x55e87982f100 2024-04-23T14:52:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.854+0000 7f068be00640 1 --2- 172.21.15.83:0/452445203 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55e87982f100 0x55e879831520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:30.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.855+0000 7f0675600640 10 client.26086 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:52:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.857+0000 7f068be00640 1 --2- 172.21.15.83:0/452445203 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55e87982f100 0x55e879831520 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5d2f56be293aadb server_cookie=b5af58683eefcf43 in_seq=0 out_seq=0 2024-04-23T14:52:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.857+0000 7f0675600640 1 -- 172.21.15.83:0/452445203 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e879719a60 con 0x55e87982f100 2024-04-23T14:52:30.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.858+0000 7f0675600640 10 client.26086 handle_command_reply: tid=0 2024-04-23T14:52:30.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.858+0000 7f06957a2740 1 client.26086 shutdown 2024-04-23T14:52:30.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.858+0000 7f06957a2740 1 -- 172.21.15.83:0/452445203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797b7b30 msgr2=0x55e8797ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:30.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.858+0000 7f06957a2740 1 --2- 172.21.15.83:0/452445203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797b7b30 0x55e8797ca190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0678009780 tx=0x7f0678005f90 comp rx=0 tx=0).stop 2024-04-23T14:52:30.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.859+0000 7f06957a2740 1 -- 172.21.15.83:0/452445203 shutdown_connections 2024-04-23T14:52:30.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.859+0000 7f06957a2740 1 --2- 172.21.15.83:0/452445203 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55e87982f100 0x55e879831520 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.859+0000 7f06957a2740 1 --2- 172.21.15.83:0/452445203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e8797b7b30 0x55e8797ca190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.859+0000 7f06957a2740 1 --2- 172.21.15.83:0/452445203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e8797ad460 0x55e8797cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.859+0000 7f06957a2740 1 --2- 172.21.15.83:0/452445203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8797acb40 0x55e8797cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.859+0000 7f0675600640 0 client.26086 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:52:30.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.859+0000 7f06957a2740 1 -- 172.21.15.83:0/452445203 >> 172.21.15.83:0/452445203 conn(0x55e8797a8e60 msgr2=0x55e8797c3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:30.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.859+0000 7f06957a2740 1 -- 172.21.15.83:0/452445203 shutdown_connections 2024-04-23T14:52:30.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.859+0000 7f06957a2740 1 -- 172.21.15.83:0/452445203 wait complete. 2024-04-23T14:52:30.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.860+0000 7f06957a2740 20 client.26086 trim_cache size 0 max 16384 2024-04-23T14:52:30.862 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:52:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.860+0000 7f0691400640 1 -- 172.21.15.83:0/1120966952 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06700a3e60 msgr2=0x7f06700a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.860+0000 7f0691400640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06700a3e60 0x7f06700a6320 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0678034000 tx=0x7f0678032fb0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.860+0000 7f0691400640 1 -- 172.21.15.83:0/1120966952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f068c109000 msgr2=0x7f068c11d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.861+0000 7f0691400640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f068c109000 0x7f068c11d310 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f068400efb0 tx=0x7f068400c800 comp rx=0 tx=0).stop 2024-04-23T14:52:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.861+0000 7f0691400640 1 -- 172.21.15.83:0/1120966952 shutdown_connections 2024-04-23T14:52:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.861+0000 7f0691400640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f068c1106a0 0x7f068c116570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.861+0000 7f0691400640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f068c109a80 0x7f068c11d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.861+0000 7f0691400640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06700a3e60 0x7f06700a6320 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.861+0000 7f0691400640 1 --2- 172.21.15.83:0/1120966952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f068c109000 0x7f068c11d310 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.861+0000 7f0691400640 1 -- 172.21.15.83:0/1120966952 >> 172.21.15.83:0/1120966952 conn(0x7f068c06cf80 msgr2=0x7f068c10d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.861+0000 7f0691400640 1 -- 172.21.15.83:0/1120966952 shutdown_connections 2024-04-23T14:52:30.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:30.861+0000 7f0691400640 1 -- 172.21.15.83:0/1120966952 wait complete. 2024-04-23T14:52:30.887 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:52:30.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:30 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/4259650698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:52:31.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:30 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/4259650698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:52:31.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:31 smithi203 ceph-mon[32800]: pgmap v124: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 61 MiB/s wr, 11.50k op/s 2024-04-23T14:52:32.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:31 smithi173 ceph-mon[32443]: pgmap v124: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 61 MiB/s wr, 11.50k op/s 2024-04-23T14:52:33.073 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:31 smithi083 ceph-mon[27867]: pgmap v124: 97 pgs: 97 active+clean; 17 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 61 MiB/s wr, 11.50k op/s 2024-04-23T14:52:33.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:33 smithi203 ceph-mon[32800]: pgmap v125: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 66 MiB/s wr, 12.67k op/s 2024-04-23T14:52:34.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:33 smithi083 ceph-mon[27867]: pgmap v125: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 66 MiB/s wr, 12.67k op/s 2024-04-23T14:52:34.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:33 smithi173 ceph-mon[32443]: pgmap v125: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 66 MiB/s wr, 12.67k op/s 2024-04-23T14:52:35.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:35 smithi203 ceph-mon[32800]: pgmap v126: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 59 MiB/s wr, 11.59k op/s 2024-04-23T14:52:35.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:35 smithi203 ceph-mon[32800]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:52:36.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:35 smithi083 ceph-mon[27867]: pgmap v126: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 59 MiB/s wr, 11.59k op/s 2024-04-23T14:52:36.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:35 smithi083 ceph-mon[27867]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:52:36.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:35 smithi173 ceph-mon[32443]: pgmap v126: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 59 MiB/s wr, 11.59k op/s 2024-04-23T14:52:36.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:35 smithi173 ceph-mon[32443]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:52:37.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:37 smithi203 ceph-mon[32800]: pgmap v127: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 74 MiB/s wr, 14.69k op/s 2024-04-23T14:52:38.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:37 smithi083 ceph-mon[27867]: pgmap v127: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 74 MiB/s wr, 14.69k op/s 2024-04-23T14:52:38.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:37 smithi173 ceph-mon[32443]: pgmap v127: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 74 MiB/s wr, 14.69k op/s 2024-04-23T14:52:39.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:38 smithi083 ceph-mon[27867]: pgmap v128: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 60 MiB/s wr, 12.01k op/s 2024-04-23T14:52:39.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:38 smithi173 ceph-mon[32443]: pgmap v128: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 60 MiB/s wr, 12.01k op/s 2024-04-23T14:52:39.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:38 smithi203 ceph-mon[32800]: pgmap v128: 97 pgs: 97 active+clean; 17 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 60 MiB/s wr, 12.01k op/s 2024-04-23T14:52:41.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:40 smithi203 ceph-mon[32800]: pgmap v129: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 59 MiB/s wr, 12.02k op/s 2024-04-23T14:52:41.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:40 smithi083 ceph-mon[27867]: pgmap v129: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 59 MiB/s wr, 12.02k op/s 2024-04-23T14:52:41.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:40 smithi173 ceph-mon[32443]: pgmap v129: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 59 MiB/s wr, 12.02k op/s 2024-04-23T14:52:43.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:42 smithi203 ceph-mon[32800]: pgmap v130: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 67 MiB/s wr, 13.78k op/s 2024-04-23T14:52:43.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:42 smithi083 ceph-mon[27867]: pgmap v130: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 67 MiB/s wr, 13.78k op/s 2024-04-23T14:52:43.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:42 smithi173 ceph-mon[32443]: pgmap v130: 97 pgs: 97 active+clean; 17 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 67 MiB/s wr, 13.78k op/s 2024-04-23T14:52:44.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:43 smithi203 ceph-mon[32800]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:52:44.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:43 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:52:44.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:43 smithi083 ceph-mon[27867]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:52:44.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:43 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:52:44.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:43 smithi173 ceph-mon[32443]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:52:44.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:43 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:52:45.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:44 smithi203 ceph-mon[32800]: pgmap v131: 97 pgs: 97 active+clean; 17 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 MiB/s wr, 11.50k op/s 2024-04-23T14:52:45.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:44 smithi083 ceph-mon[27867]: pgmap v131: 97 pgs: 97 active+clean; 17 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 MiB/s wr, 11.50k op/s 2024-04-23T14:52:45.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:44 smithi173 ceph-mon[32443]: pgmap v131: 97 pgs: 97 active+clean; 17 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 56 MiB/s wr, 11.50k op/s 2024-04-23T14:52:47.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:46 smithi203 ceph-mon[32800]: pgmap v132: 97 pgs: 97 active+clean; 17 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 MiB/s wr, 12.48k op/s 2024-04-23T14:52:47.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:46 smithi083 ceph-mon[27867]: pgmap v132: 97 pgs: 97 active+clean; 17 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 MiB/s wr, 12.48k op/s 2024-04-23T14:52:47.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:46 smithi173 ceph-mon[32443]: pgmap v132: 97 pgs: 97 active+clean; 17 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 60 MiB/s wr, 12.48k op/s 2024-04-23T14:52:49.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:48 smithi083 ceph-mon[27867]: pgmap v133: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 50 MiB/s wr, 10.77k op/s 2024-04-23T14:52:49.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:48 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:52:49.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:48 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:52:49.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:48 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:52:49.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:48 smithi173 ceph-mon[32443]: pgmap v133: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 50 MiB/s wr, 10.77k op/s 2024-04-23T14:52:49.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:48 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:52:49.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:48 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:52:49.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:48 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:52:49.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:48 smithi203 ceph-mon[32800]: pgmap v133: 97 pgs: 97 active+clean; 18 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 50 MiB/s wr, 10.77k op/s 2024-04-23T14:52:49.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:48 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:52:49.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:48 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:52:49.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:48 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:52:50.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:49 smithi083 ceph-mon[27867]: pgmap v134: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 MiB/s wr, 11.74k op/s 2024-04-23T14:52:50.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:49 smithi083 ceph-mon[27867]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-23T14:52:50.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:49 smithi173 ceph-mon[32443]: pgmap v134: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 MiB/s wr, 11.74k op/s 2024-04-23T14:52:50.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:49 smithi173 ceph-mon[32443]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-23T14:52:50.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:49 smithi203 ceph-mon[32800]: pgmap v134: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 54 MiB/s wr, 11.74k op/s 2024-04-23T14:52:50.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:49 smithi203 ceph-mon[32800]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-23T14:52:52.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:51 smithi083 ceph-mon[27867]: pgmap v135: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 54 MiB/s wr, 11.73k op/s 2024-04-23T14:52:52.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:51 smithi173 ceph-mon[32443]: pgmap v135: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 54 MiB/s wr, 11.73k op/s 2024-04-23T14:52:52.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:51 smithi203 ceph-mon[32800]: pgmap v135: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 54 MiB/s wr, 11.73k op/s 2024-04-23T14:52:54.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:54 smithi203 ceph-mon[32800]: pgmap v136: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 MiB/s wr, 9.59k op/s 2024-04-23T14:52:54.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:54 smithi083 ceph-mon[27867]: pgmap v136: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 MiB/s wr, 9.59k op/s 2024-04-23T14:52:54.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:54 smithi173 ceph-mon[32443]: pgmap v136: 97 pgs: 97 active+clean; 18 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 43 MiB/s wr, 9.59k op/s 2024-04-23T14:52:56.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:56 smithi203 ceph-mon[32800]: pgmap v137: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 MiB/s wr, 9.78k op/s 2024-04-23T14:52:56.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:56 smithi083 ceph-mon[27867]: pgmap v137: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 MiB/s wr, 9.78k op/s 2024-04-23T14:52:56.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:56 smithi173 ceph-mon[32443]: pgmap v137: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 MiB/s wr, 9.78k op/s 2024-04-23T14:52:58.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:52:58 smithi083 ceph-mon[27867]: pgmap v138: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 46 MiB/s wr, 10.03k op/s 2024-04-23T14:52:58.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:52:58 smithi173 ceph-mon[32443]: pgmap v138: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 46 MiB/s wr, 10.03k op/s 2024-04-23T14:52:58.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:52:58 smithi203 ceph-mon[32800]: pgmap v138: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 46 MiB/s wr, 10.03k op/s 2024-04-23T14:52:58.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.881+0000 7ff4d3e00640 1 -- 172.21.15.83:0/2189083750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d4102ac0 msgr2=0x7ff4d4106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:58.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.881+0000 7ff4d3e00640 1 --2- 172.21.15.83:0/2189083750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d4102ac0 0x7ff4d4106b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0009860 tx=0x7ff4c002a340 comp rx=0 tx=0).stop 2024-04-23T14:52:58.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.881+0000 7ff4d3e00640 1 -- 172.21.15.83:0/2189083750 shutdown_connections 2024-04-23T14:52:58.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.881+0000 7ff4d3e00640 1 --2- 172.21.15.83:0/2189083750 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d41071e0 0x7ff4d410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.881+0000 7ff4d3e00640 1 --2- 172.21.15.83:0/2189083750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d4102ac0 0x7ff4d4106b00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.881+0000 7ff4d3e00640 1 --2- 172.21.15.83:0/2189083750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d4102110 0x7ff4d41024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.881+0000 7ff4d3e00640 1 -- 172.21.15.83:0/2189083750 >> 172.21.15.83:0/2189083750 conn(0x7ff4d40fd9d0 msgr2=0x7ff4d40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:58.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.881+0000 7ff4d3e00640 1 -- 172.21.15.83:0/2189083750 shutdown_connections 2024-04-23T14:52:58.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.882+0000 7ff4d3e00640 1 -- 172.21.15.83:0/2189083750 wait complete. 2024-04-23T14:52:58.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.883+0000 7ff4d3e00640 1 Processor -- start 2024-04-23T14:52:58.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.884+0000 7ff4d3e00640 1 -- start start 2024-04-23T14:52:58.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.884+0000 7ff4d3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d4102110 0x7ff4d41a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.884+0000 7ff4d3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d41071e0 0x7ff4d41a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.884+0000 7ff4d3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d410dea0 0x7ff4d410e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.884+0000 7ff4d3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41122a0 con 0x7ff4d41071e0 2024-04-23T14:52:58.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.884+0000 7ff4d3e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4112120 con 0x7ff4d410dea0 2024-04-23T14:52:58.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.884+0000 7ff4d3e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4112420 con 0x7ff4d4102110 2024-04-23T14:52:58.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.885+0000 7ff4d3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d410dea0 0x7ff4d410e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.885+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d4102110 0x7ff4d41a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.885+0000 7ff4d3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d410dea0 0x7ff4d410e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:45066/0 (socket says 172.21.15.83:45066) 2024-04-23T14:52:58.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.885+0000 7ff4d3400640 1 -- 172.21.15.83:0/4121027625 learned_addr learned my addr 172.21.15.83:0/4121027625 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:58.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.886+0000 7ff4d2000640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d41071e0 0x7ff4d41a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.886+0000 7ff4d3400640 1 -- 172.21.15.83:0/4121027625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d4102110 msgr2=0x7ff4d41a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.886+0000 7ff4d3400640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d4102110 0x7ff4d41a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.886+0000 7ff4d3400640 1 -- 172.21.15.83:0/4121027625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d41071e0 msgr2=0x7ff4d41a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.886+0000 7ff4d3400640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d41071e0 0x7ff4d41a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.886+0000 7ff4d3400640 1 -- 172.21.15.83:0/4121027625 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d410ea80 con 0x7ff4d410dea0 2024-04-23T14:52:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.886+0000 7ff4d2000640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d41071e0 0x7ff4d41a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:52:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.887+0000 7ff4d3400640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d410dea0 0x7ff4d410e350 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00eda0 tx=0x7ff4cc00c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.887+0000 7ff4d8a00640 1 -- 172.21.15.83:0/4121027625 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc003b70 con 0x7ff4d410dea0 2024-04-23T14:52:58.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.887+0000 7ff4d8a00640 1 -- 172.21.15.83:0/4121027625 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc003d10 con 0x7ff4d410dea0 2024-04-23T14:52:58.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.887+0000 7ff4d8a00640 1 -- 172.21.15.83:0/4121027625 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc017910 con 0x7ff4d410dea0 2024-04-23T14:52:58.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.887+0000 7ff4d3e00640 1 -- 172.21.15.83:0/4121027625 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d41ab910 con 0x7ff4d410dea0 2024-04-23T14:52:58.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.887+0000 7ff4d3e00640 1 -- 172.21.15.83:0/4121027625 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d41abcc0 con 0x7ff4d410dea0 2024-04-23T14:52:58.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.895+0000 7ff4d8a00640 1 -- 172.21.15.83:0/4121027625 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff4cc017ab0 con 0x7ff4d410dea0 2024-04-23T14:52:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.897+0000 7ff4d8a00640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff4b80a3d60 0x7ff4b80a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.898+0000 7ff4d8a00640 1 -- 172.21.15.83:0/4121027625 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ff4cc0d1920 con 0x7ff4d410dea0 2024-04-23T14:52:58.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.903+0000 7ff4d2a00640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff4b80a3d60 0x7ff4b80a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.912+0000 7ff4d2a00640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff4b80a3d60 0x7ff4b80a6220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009bd0 tx=0x7ff4c4005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:58.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.914+0000 7ff4dcc7e740 1 Processor -- start 2024-04-23T14:52:58.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.915+0000 7ff4dcc7e740 1 -- start start 2024-04-23T14:52:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.915+0000 7ff4dcc7e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd89f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.915+0000 7ff4dcc7e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 0x56038cd945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.915+0000 7ff4dcc7e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd96f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.915+0000 7ff4dcc7e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56038ccef1d0 con 0x56038cd8a460 2024-04-23T14:52:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.915+0000 7ff4dcc7e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56038cc95450 con 0x56038cd94b30 2024-04-23T14:52:58.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.915+0000 7ff4dcc7e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56038cceb190 con 0x56038cd89b40 2024-04-23T14:52:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.916+0000 7ff4d3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd96f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.916+0000 7ff4d3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd96f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:45078/0 (socket says 172.21.15.83:45078) 2024-04-23T14:52:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.916+0000 7ff4d3400640 1 -- 172.21.15.83:0/3170336537 learned_addr learned my addr 172.21.15.83:0/3170336537 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.916+0000 7ff4d2a00640 1 --2- 172.21.15.83:0/3170336537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd89f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.917+0000 7ff4d3400640 1 -- 172.21.15.83:0/3170336537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 msgr2=0x56038cd89f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.917+0000 7ff4d3400640 1 --2- 172.21.15.83:0/3170336537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd89f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.917+0000 7ff4d3400640 1 -- 172.21.15.83:0/3170336537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 msgr2=0x56038cd945f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:52:58.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.917+0000 7ff4d3400640 1 --2- 172.21.15.83:0/3170336537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 0x56038cd945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.918+0000 7ff4d3400640 1 -- 172.21.15.83:0/3170336537 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56038cd52310 con 0x56038cd94b30 2024-04-23T14:52:58.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.918+0000 7ff4d3400640 1 --2- 172.21.15.83:0/3170336537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd96f20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0a6120 tx=0x7ff4cc09d450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea78e3cea38847d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:58.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.918+0000 7ff4bfe00640 1 -- 172.21.15.83:0/3170336537 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0a1d90 con 0x56038cd94b30 2024-04-23T14:52:58.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.919+0000 7ff4bfe00640 1 -- 172.21.15.83:0/3170336537 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc0a6650 con 0x56038cd94b30 2024-04-23T14:52:58.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.919+0000 7ff4bfe00640 1 -- 172.21.15.83:0/3170336537 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0bcba0 con 0x56038cd94b30 2024-04-23T14:52:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.921+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3170336537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 msgr2=0x56038cd96f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:58.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.921+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/3170336537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd96f20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0a6120 tx=0x7ff4cc09d450 comp rx=0 tx=0).stop 2024-04-23T14:52:58.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.924+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3170336537 shutdown_connections 2024-04-23T14:52:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.924+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/3170336537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd96f20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.924+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/3170336537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 0x56038cd945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.924+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/3170336537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd89f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.924+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3170336537 >> 172.21.15.83:0/3170336537 conn(0x56038cd85e60 msgr2=0x56038cd86240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.925+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3170336537 shutdown_connections 2024-04-23T14:52:58.926 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.926+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3170336537 wait complete. 2024-04-23T14:52:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.926+0000 7ff4dcc7e740 1 -- start start 2024-04-23T14:52:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.927+0000 7ff4dcc7e740 1 Processor -- start 2024-04-23T14:52:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.927+0000 7ff4dcc7e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd919d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.928+0000 7ff4dcc7e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 0x56038cd91f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.927 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.928+0000 7ff4dcc7e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd92450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.928+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd919d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.928+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd919d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:37984/0 (socket says 172.21.15.83:37984) 2024-04-23T14:52:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4d2a00640 1 -- 172.21.15.83:0/2089370480 learned_addr learned my addr 172.21.15.83:0/2089370480 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2089370480 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56038cda8970 con 0x56038cd8a460 2024-04-23T14:52:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2089370480 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56038cca8810 con 0x56038cd94b30 2024-04-23T14:52:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2089370480 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56038cda8af0 con 0x56038cd89b40 2024-04-23T14:52:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4d3400640 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd92450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4d2000640 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 0x56038cd91f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4d2a00640 1 -- 172.21.15.83:0/2089370480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 msgr2=0x56038cd92450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4d2a00640 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd92450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4d2a00640 1 -- 172.21.15.83:0/2089370480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 msgr2=0x56038cd91f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4d2a00640 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 0x56038cd91f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4d2a00640 1 -- 172.21.15.83:0/2089370480 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56038ccb93f0 con 0x56038cd89b40 2024-04-23T14:52:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4d3400640 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd92450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:52:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4d2000640 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 0x56038cd91f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:52:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.929+0000 7ff4dcc7e740 10 client.0 _ll_register_callbacks cb 0x56038cc76140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:52:58.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.930+0000 7ff4d2a00640 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd919d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff4c401ed00 tx=0x7ff4c4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1538708c03e883e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.930+0000 7ff4bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:52:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.930+0000 7ff4bcc00640 1 -- 172.21.15.83:0/2089370480 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4004020 con 0x56038cd89b40 2024-04-23T14:52:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.930+0000 7ff4bcc00640 1 -- 172.21.15.83:0/2089370480 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ff4c40041c0 con 0x56038cd89b40 2024-04-23T14:52:58.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.931+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2089370480 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4cc0a6650 con 0x56038cd89b40 2024-04-23T14:52:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.932+0000 7ff4bcc00640 1 -- 172.21.15.83:0/2089370480 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4c40043c0 con 0x56038cd89b40 2024-04-23T14:52:58.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.933+0000 7ff4dcc7e740 10 client.26098 fetch_fsmap learned FSMap version 28 2024-04-23T14:52:58.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.933+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2089370480 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56038ca013f0 con 0x56038cd89b40 2024-04-23T14:52:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.933+0000 7ff4bcc00640 1 -- 172.21.15.83:0/2089370480 <== mon.1 v2:172.21.15.203:3300/0 4 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7ff4c4025070 con 0x56038cd89b40 2024-04-23T14:52:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.933+0000 7ff4dcc7e740 10 client.26098 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T14:52:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.933+0000 7ff4dcc7e740 10 client.26098 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:52:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.933+0000 7ff4dcc7e740 20 client.26098 populate_metadata read hostname 'smithi083' 2024-04-23T14:52:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.933+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x56038cdb59a0 0x56038cdb7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.934+0000 7ff4dcc7e740 4 client.26098 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:52:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.934+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2089370480 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56038ccfcbf0 con 0x56038cdb59a0 2024-04-23T14:52:58.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.934+0000 7ff4d2000640 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x56038cdb59a0 0x56038cdb7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.937+0000 7ff4d2000640 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x56038cdb59a0 0x56038cdb7e30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99f9b7374af2c607 server_cookie=4da3ea29d2368461 in_seq=0 out_seq=0 2024-04-23T14:52:58.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.937+0000 7ff4bcc00640 10 client.26098 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:52:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.938+0000 7ff4bcc00640 1 -- 172.21.15.83:0/2089370480 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56038ccfcbf0 con 0x56038cdb59a0 2024-04-23T14:52:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.938+0000 7ff4bcc00640 10 client.26098 handle_command_reply: tid=0 2024-04-23T14:52:58.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.938+0000 7ff4dcc7e740 1 client.26098 shutdown 2024-04-23T14:52:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.938+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2089370480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 msgr2=0x56038cd919d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:58.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.939+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd919d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff4c401ed00 tx=0x7ff4c4004730 comp rx=0 tx=0).stop 2024-04-23T14:52:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.939+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2089370480 shutdown_connections 2024-04-23T14:52:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.939+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd92450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.939+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 0x56038cd91f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.939+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x56038cdb59a0 0x56038cdb7e30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.939+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/2089370480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd919d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.939+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2089370480 >> 172.21.15.83:0/2089370480 conn(0x56038cd85e60 msgr2=0x56038cd8bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.940+0000 7ff4bcc00640 0 client.26098 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:52:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.940+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2089370480 shutdown_connections 2024-04-23T14:52:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.940+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2089370480 wait complete. 2024-04-23T14:52:58.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.940+0000 7ff4dcc7e740 20 client.26098 trim_cache size 0 max 16384 2024-04-23T14:52:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.978+0000 7ff4dcc7e740 1 Processor -- start 2024-04-23T14:52:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.978+0000 7ff4dcc7e740 1 -- start start 2024-04-23T14:52:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.979+0000 7ff4dcc7e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd94210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.979+0000 7ff4dcc7e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 0x56038cd90b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.979+0000 7ff4dcc7e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd92270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.979+0000 7ff4dcc7e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41ae340 con 0x56038cd8a460 2024-04-23T14:52:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.979+0000 7ff4dcc7e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4073ef0 con 0x56038cd94b30 2024-04-23T14:52:58.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.979+0000 7ff4dcc7e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4071d40 con 0x56038cd89b40 2024-04-23T14:52:58.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.979+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd94210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.979+0000 7ff4d2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd94210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:37986/0 (socket says 172.21.15.83:37986) 2024-04-23T14:52:58.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.979+0000 7ff4d2a00640 1 -- 172.21.15.83:0/2607003460 learned_addr learned my addr 172.21.15.83:0/2607003460 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:58.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.979+0000 7ff4d2000640 1 --2- 172.21.15.83:0/2607003460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 0x56038cd90b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.980+0000 7ff4d3400640 1 --2- 172.21.15.83:0/2607003460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd92270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.980+0000 7ff4d2a00640 1 -- 172.21.15.83:0/2607003460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 msgr2=0x56038cd92270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:58.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.980+0000 7ff4d2a00640 1 --2- 172.21.15.83:0/2607003460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd92270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.980+0000 7ff4d2a00640 1 -- 172.21.15.83:0/2607003460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 msgr2=0x56038cd90b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:58.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.980+0000 7ff4d2a00640 1 --2- 172.21.15.83:0/2607003460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 0x56038cd90b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.980+0000 7ff4d2a00640 1 -- 172.21.15.83:0/2607003460 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56038cacb7d0 con 0x56038cd89b40 2024-04-23T14:52:58.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.981+0000 7ff4d2a00640 1 --2- 172.21.15.83:0/2607003460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd94210 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400a010 tx=0x7ff4c4020040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e00e4c4215b438 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:58.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.981+0000 7ff4bf400640 1 -- 172.21.15.83:0/2607003460 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400ee20 con 0x56038cd89b40 2024-04-23T14:52:58.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.982+0000 7ff4bf400640 1 -- 172.21.15.83:0/2607003460 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc0a6650 con 0x56038cd89b40 2024-04-23T14:52:58.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.982+0000 7ff4bf400640 1 -- 172.21.15.83:0/2607003460 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c40263f0 con 0x56038cd89b40 2024-04-23T14:52:58.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.983+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2607003460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 msgr2=0x56038cd94210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:58.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.983+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/2607003460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd94210 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400a010 tx=0x7ff4c4020040 comp rx=0 tx=0).stop 2024-04-23T14:52:58.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.983+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2607003460 shutdown_connections 2024-04-23T14:52:58.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.983+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/2607003460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd94b30 0x56038cd92270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.983+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/2607003460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd8a460 0x56038cd90b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.983+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/2607003460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd89b40 0x56038cd94210 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.983+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2607003460 >> 172.21.15.83:0/2607003460 conn(0x56038cd85e60 msgr2=0x56038cd8b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:58.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.983+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2607003460 shutdown_connections 2024-04-23T14:52:58.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.983+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/2607003460 wait complete. 2024-04-23T14:52:58.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.984+0000 7ff4dcc7e740 1 -- start start 2024-04-23T14:52:58.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.985+0000 7ff4dcc7e740 1 Processor -- start 2024-04-23T14:52:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.987+0000 7ff4dcc7e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd89b40 0x56038cda8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.987+0000 7ff4dcc7e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd8a460 0x56038cda9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.987+0000 7ff4dcc7e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd94b30 0x56038cda9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.987+0000 7ff4dcc7e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40723b0 con 0x56038cd94b30 2024-04-23T14:52:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.987+0000 7ff4dcc7e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4073820 con 0x56038cd89b40 2024-04-23T14:52:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.987+0000 7ff4dcc7e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4072a80 con 0x56038cd8a460 2024-04-23T14:52:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.988+0000 7ff4dcc7e740 10 client.0 _ll_register_callbacks cb 0x7ff4c4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:52:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.988+0000 7ff4d2000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd8a460 0x56038cda9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.988+0000 7ff4d2000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd8a460 0x56038cda9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:37992/0 (socket says 172.21.15.83:37992) 2024-04-23T14:52:58.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.988+0000 7ff4d2000640 1 -- 172.21.15.83:0/3867216906 learned_addr learned my addr 172.21.15.83:0/3867216906 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:58.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.988+0000 7ff4d2000640 1 -- 172.21.15.83:0/3867216906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd89b40 msgr2=0x56038cda8cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T14:52:58.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.988+0000 7ff4d2000640 1 --2- 172.21.15.83:0/3867216906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd89b40 0x56038cda8cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.988+0000 7ff4d2000640 1 -- 172.21.15.83:0/3867216906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd94b30 msgr2=0x56038cda9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:58.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.988+0000 7ff4d2000640 1 --2- 172.21.15.83:0/3867216906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd94b30 0x56038cda9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.988+0000 7ff4d2000640 1 -- 172.21.15.83:0/3867216906 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56038ce1a090 con 0x56038cd8a460 2024-04-23T14:52:58.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.989+0000 7ff4d2000640 1 --2- 172.21.15.83:0/3867216906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd8a460 0x56038cda9210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00304f0 tx=0x7ff4c0030d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=762e99a482195885 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:58.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.989+0000 7ff4bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:52:58.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.990+0000 7ff4bcc00640 1 -- 172.21.15.83:0/3867216906 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0037040 con 0x56038cd8a460 2024-04-23T14:52:58.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.991+0000 7ff4bcc00640 1 -- 172.21.15.83:0/3867216906 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ff4c0004380 con 0x56038cd8a460 2024-04-23T14:52:58.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.991+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3867216906 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4cc0a6650 con 0x56038cd8a460 2024-04-23T14:52:58.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.992+0000 7ff4bcc00640 1 -- 172.21.15.83:0/3867216906 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff4c0050050 con 0x56038cd8a460 2024-04-23T14:52:58.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.992+0000 7ff4dcc7e740 10 client.26104 fetch_fsmap learned FSMap version 28 2024-04-23T14:52:58.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.992+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3867216906 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56038ce1b330 con 0x56038cd8a460 2024-04-23T14:52:58.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.992+0000 7ff4bcc00640 1 -- 172.21.15.83:0/3867216906 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4cc0a6650 con 0x56038cd8a460 2024-04-23T14:52:58.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.992+0000 7ff4bcc00640 1 -- 172.21.15.83:0/3867216906 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7ff4c00046c0 con 0x56038cd8a460 2024-04-23T14:52:58.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.992+0000 7ff4dcc7e740 10 client.26104 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T14:52:58.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.992+0000 7ff4dcc7e740 10 client.26104 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:52:58.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.992+0000 7ff4dcc7e740 20 client.26104 populate_metadata read hostname 'smithi083' 2024-04-23T14:52:58.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.992+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/3867216906 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x56038ce1bad0 0x56038ce1def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:58.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.992+0000 7ff4dcc7e740 4 client.26104 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:52:58.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.992+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3867216906 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56038ccf6a60 con 0x56038ce1bad0 2024-04-23T14:52:58.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.995+0000 7ff4d2a00640 1 --2- 172.21.15.83:0/3867216906 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x56038ce1bad0 0x56038ce1def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:58.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.996+0000 7ff4bcc00640 10 client.26104 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:52:58.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.996+0000 7ff4d2a00640 1 --2- 172.21.15.83:0/3867216906 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x56038ce1bad0 0x56038ce1def0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=350c997fc83198b7 server_cookie=1165681d6b978486 in_seq=0 out_seq=0 2024-04-23T14:52:58.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.996+0000 7ff4bcc00640 1 -- 172.21.15.83:0/3867216906 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56038ccf6a60 con 0x56038ce1bad0 2024-04-23T14:52:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.996+0000 7ff4bcc00640 10 client.26104 handle_command_reply: tid=0 2024-04-23T14:52:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.996+0000 7ff4dcc7e740 1 client.26104 shutdown 2024-04-23T14:52:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.997+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3867216906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd8a460 msgr2=0x56038cda9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.997+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/3867216906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd8a460 0x56038cda9210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00304f0 tx=0x7ff4c0030d30 comp rx=0 tx=0).stop 2024-04-23T14:52:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.997+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3867216906 shutdown_connections 2024-04-23T14:52:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.997+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/3867216906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56038cd94b30 0x56038cda9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.997+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/3867216906 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x56038ce1bad0 0x56038ce1def0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.997+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/3867216906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56038cd8a460 0x56038cda9210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.997+0000 7ff4dcc7e740 1 --2- 172.21.15.83:0/3867216906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56038cd89b40 0x56038cda8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.997+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3867216906 >> 172.21.15.83:0/3867216906 conn(0x56038cd85e60 msgr2=0x56038cd846c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.997+0000 7ff4bcc00640 0 client.26104 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:52:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.997+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3867216906 shutdown_connections 2024-04-23T14:52:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.997+0000 7ff4dcc7e740 1 -- 172.21.15.83:0/3867216906 wait complete. 2024-04-23T14:52:58.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.997+0000 7ff4dcc7e740 20 client.26104 trim_cache size 0 max 16384 2024-04-23T14:52:58.999 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:52:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.999+0000 7ff4d3e00640 1 -- 172.21.15.83:0/4121027625 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff4b80a3d60 msgr2=0x7ff4b80a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.999+0000 7ff4d3e00640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff4b80a3d60 0x7ff4b80a6220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009bd0 tx=0x7ff4c4005f90 comp rx=0 tx=0).stop 2024-04-23T14:52:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.999+0000 7ff4d3e00640 1 -- 172.21.15.83:0/4121027625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d410dea0 msgr2=0x7ff4d410e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.999+0000 7ff4d3e00640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d410dea0 0x7ff4d410e350 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00eda0 tx=0x7ff4cc00c6e0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.999+0000 7ff4d3e00640 1 -- 172.21.15.83:0/4121027625 shutdown_connections 2024-04-23T14:52:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.999+0000 7ff4d3e00640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4d410dea0 0x7ff4d410e350 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.999+0000 7ff4d3e00640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff4b80a3d60 0x7ff4b80a6220 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.999+0000 7ff4d3e00640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4d41071e0 0x7ff4d41a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.999+0000 7ff4d3e00640 1 --2- 172.21.15.83:0/4121027625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4d4102110 0x7ff4d41a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:58.999+0000 7ff4d3e00640 1 -- 172.21.15.83:0/4121027625 >> 172.21.15.83:0/4121027625 conn(0x7ff4d40fd9d0 msgr2=0x7ff4d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.000+0000 7ff4d3e00640 1 -- 172.21.15.83:0/4121027625 shutdown_connections 2024-04-23T14:52:59.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.000+0000 7ff4d3e00640 1 -- 172.21.15.83:0/4121027625 wait complete. 2024-04-23T14:52:59.053 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:52:59.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.730+0000 7fde9ce00640 1 -- 172.21.15.83:0/2364813076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde98109270 msgr2=0x7fde98109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:59.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.730+0000 7fde9ce00640 1 --2- 172.21.15.83:0/2364813076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde98109270 0x7fde98109650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fde80009860 tx=0x7fde8002a340 comp rx=0 tx=0).stop 2024-04-23T14:52:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.733+0000 7fde9ce00640 1 -- 172.21.15.83:0/2364813076 shutdown_connections 2024-04-23T14:52:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.733+0000 7fde9ce00640 1 --2- 172.21.15.83:0/2364813076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde98069aa0 0x7fde981159f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.733+0000 7fde9ce00640 1 --2- 172.21.15.83:0/2364813076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde98109c20 0x7fde9810d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.733+0000 7fde9ce00640 1 --2- 172.21.15.83:0/2364813076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde98109270 0x7fde98109650 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.733+0000 7fde9ce00640 1 -- 172.21.15.83:0/2364813076 >> 172.21.15.83:0/2364813076 conn(0x7fde9806d3e0 msgr2=0x7fde9806d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:59.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.734+0000 7fde9ce00640 1 -- 172.21.15.83:0/2364813076 shutdown_connections 2024-04-23T14:52:59.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.735+0000 7fde9ce00640 1 -- 172.21.15.83:0/2364813076 wait complete. 2024-04-23T14:52:59.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.738+0000 7fde9ce00640 1 Processor -- start 2024-04-23T14:52:59.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.739+0000 7fde9ce00640 1 -- start start 2024-04-23T14:52:59.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.740+0000 7fde9ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde98069aa0 0x7fde981a3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.740+0000 7fde9ce00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde98109270 0x7fde981a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.740+0000 7fde9ce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde981a7cf0 0x7fde981a81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.740+0000 7fde9ce00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fde98118ab0 con 0x7fde98069aa0 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.740+0000 7fde9ce00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fde98118930 con 0x7fde98109270 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.740+0000 7fde9ce00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde98118c30 con 0x7fde981a7cf0 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.741+0000 7fde97e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde981a7cf0 0x7fde981a81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.741+0000 7fde97e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde981a7cf0 0x7fde981a81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38010/0 (socket says 172.21.15.83:38010) 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.741+0000 7fde97e00640 1 -- 172.21.15.83:0/3073392059 learned_addr learned my addr 172.21.15.83:0/3073392059 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.742+0000 7fde97e00640 1 -- 172.21.15.83:0/3073392059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde98109270 msgr2=0x7fde981a3970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.742+0000 7fde97e00640 1 --2- 172.21.15.83:0/3073392059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde98109270 0x7fde981a3970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.742+0000 7fde97e00640 1 -- 172.21.15.83:0/3073392059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde98069aa0 msgr2=0x7fde981a3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.742+0000 7fde97e00640 1 --2- 172.21.15.83:0/3073392059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde98069aa0 0x7fde981a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.742+0000 7fde97e00640 1 -- 172.21.15.83:0/3073392059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde981a87e0 con 0x7fde981a7cf0 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.742+0000 7fde97e00640 1 --2- 172.21.15.83:0/3073392059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde981a7cf0 0x7fde981a81a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fde9000db10 tx=0x7fde9000a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.743+0000 7fde94c00640 1 -- 172.21.15.83:0/3073392059 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9000ae60 con 0x7fde981a7cf0 2024-04-23T14:52:59.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.743+0000 7fde9ce00640 1 -- 172.21.15.83:0/3073392059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde981a8ad0 con 0x7fde981a7cf0 2024-04-23T14:52:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.743+0000 7fde94c00640 1 -- 172.21.15.83:0/3073392059 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde90005500 con 0x7fde981a7cf0 2024-04-23T14:52:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.743+0000 7fde94c00640 1 -- 172.21.15.83:0/3073392059 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9000e780 con 0x7fde981a7cf0 2024-04-23T14:52:59.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.744+0000 7fde9ce00640 1 -- 172.21.15.83:0/3073392059 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde981b02c0 con 0x7fde981a7cf0 2024-04-23T14:52:59.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.757+0000 7fdea123c740 1 Processor -- start 2024-04-23T14:52:59.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.759+0000 7fde94c00640 1 -- 172.21.15.83:0/3073392059 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fde9000e920 con 0x7fde981a7cf0 2024-04-23T14:52:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.761+0000 7fde94c00640 1 --2- 172.21.15.83:0/3073392059 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fde7c0a3da0 0x7fde7c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.762+0000 7fde94c00640 1 -- 172.21.15.83:0/3073392059 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fde900d1540 con 0x7fde981a7cf0 2024-04-23T14:52:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.762+0000 7fde8ea00640 1 --2- 172.21.15.83:0/3073392059 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fde7c0a3da0 0x7fde7c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.763+0000 7fdea123c740 1 -- start start 2024-04-23T14:52:59.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.763+0000 7fdea123c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 0x5603f569ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.763+0000 7fdea123c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569f460 0x5603f56a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.763+0000 7fdea123c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f56a9b30 0x5603f56abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.764+0000 7fdea123c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5603f56041d0 con 0x5603f56a9b30 2024-04-23T14:52:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.764+0000 7fdea123c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5603f55aa450 con 0x5603f569f460 2024-04-23T14:52:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.764+0000 7fdea123c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5603f5600190 con 0x5603f569eb40 2024-04-23T14:52:59.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.764+0000 7fde97400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 0x5603f569ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.764+0000 7fde97400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 0x5603f569ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38026/0 (socket says 172.21.15.83:38026) 2024-04-23T14:52:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.764+0000 7fde97e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f56a9b30 0x5603f56abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.764+0000 7fde97400640 1 -- 172.21.15.83:0/3880096330 learned_addr learned my addr 172.21.15.83:0/3880096330 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.764+0000 7fde8ea00640 1 --2- 172.21.15.83:0/3880096330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569f460 0x5603f56a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.765+0000 7fde97400640 1 -- 172.21.15.83:0/3880096330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569f460 msgr2=0x5603f56a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:59.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.765+0000 7fde97400640 1 --2- 172.21.15.83:0/3880096330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569f460 0x5603f56a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.765+0000 7fde97400640 1 -- 172.21.15.83:0/3880096330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f56a9b30 msgr2=0x5603f56abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:59.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.765+0000 7fde97400640 1 --2- 172.21.15.83:0/3880096330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f56a9b30 0x5603f56abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.765+0000 7fde97400640 1 -- 172.21.15.83:0/3880096330 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f5667310 con 0x5603f569eb40 2024-04-23T14:52:59.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.766+0000 7fde8ea00640 1 --2- 172.21.15.83:0/3073392059 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fde7c0a3da0 0x7fde7c0a6260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fde88005fd0 tx=0x7fde88013480 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:59.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.767+0000 7fde97400640 1 --2- 172.21.15.83:0/3880096330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 0x5603f569ef20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fde800095e0 tx=0x7fde80005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0cf991ab75d834d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.769+0000 7fde8cc00640 1 -- 172.21.15.83:0/3880096330 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80037040 con 0x5603f569eb40 2024-04-23T14:52:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.770+0000 7fde8cc00640 1 -- 172.21.15.83:0/3880096330 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde80040070 con 0x5603f569eb40 2024-04-23T14:52:59.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.770+0000 7fde8cc00640 1 -- 172.21.15.83:0/3880096330 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80003850 con 0x5603f569eb40 2024-04-23T14:52:59.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.771+0000 7fdea123c740 1 -- 172.21.15.83:0/3880096330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 msgr2=0x5603f569ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:59.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.771+0000 7fdea123c740 1 --2- 172.21.15.83:0/3880096330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 0x5603f569ef20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fde800095e0 tx=0x7fde80005c90 comp rx=0 tx=0).stop 2024-04-23T14:52:59.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.772+0000 7fdea123c740 1 -- 172.21.15.83:0/3880096330 shutdown_connections 2024-04-23T14:52:59.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.772+0000 7fdea123c740 1 --2- 172.21.15.83:0/3880096330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f56a9b30 0x5603f56abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.772+0000 7fdea123c740 1 --2- 172.21.15.83:0/3880096330 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569f460 0x5603f56a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.772+0000 7fdea123c740 1 --2- 172.21.15.83:0/3880096330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 0x5603f569ef20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.772+0000 7fdea123c740 1 -- 172.21.15.83:0/3880096330 >> 172.21.15.83:0/3880096330 conn(0x5603f569ae60 msgr2=0x5603f569b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:59.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.772+0000 7fdea123c740 1 -- 172.21.15.83:0/3880096330 shutdown_connections 2024-04-23T14:52:59.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.773+0000 7fdea123c740 1 -- 172.21.15.83:0/3880096330 wait complete. 2024-04-23T14:52:59.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.773+0000 7fdea123c740 1 -- start start 2024-04-23T14:52:59.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.774+0000 7fdea123c740 1 Processor -- start 2024-04-23T14:52:59.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.775+0000 7fdea123c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569eb40 0x5603f56a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.775+0000 7fdea123c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 0x5603f56a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.776+0000 7fdea123c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 0x5603f56a7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.776+0000 7fdea123c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5603f56ac960 con 0x5603f569f460 2024-04-23T14:52:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.776+0000 7fdea123c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5603f55bd810 con 0x5603f569eb40 2024-04-23T14:52:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.776+0000 7fdea123c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5603f56acae0 con 0x5603f56a9b30 2024-04-23T14:52:59.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.776+0000 7fde8ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 0x5603f56a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.776+0000 7fde97400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569eb40 0x5603f56a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.776+0000 7fde97e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 0x5603f56a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.776+0000 7fde97e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 0x5603f56a7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38036/0 (socket says 172.21.15.83:38036) 2024-04-23T14:52:59.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.776+0000 7fde97e00640 1 -- 172.21.15.83:0/2766342725 learned_addr learned my addr 172.21.15.83:0/2766342725 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:59.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.776+0000 7fdea123c740 10 client.0 _ll_register_callbacks cb 0x5603f558b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:52:59.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.777+0000 7fde97e00640 1 -- 172.21.15.83:0/2766342725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569eb40 msgr2=0x5603f56a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:59.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.777+0000 7fde97e00640 1 --2- 172.21.15.83:0/2766342725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569eb40 0x5603f56a95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.777+0000 7fde97e00640 1 -- 172.21.15.83:0/2766342725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 msgr2=0x5603f56a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:59.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.777+0000 7fde97e00640 1 --2- 172.21.15.83:0/2766342725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 0x5603f56a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.777+0000 7fde97e00640 1 -- 172.21.15.83:0/2766342725 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f55ce3f0 con 0x5603f56a9b30 2024-04-23T14:52:59.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.777+0000 7fde97e00640 1 --2- 172.21.15.83:0/2766342725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 0x5603f56a7010 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde900cedb0 tx=0x7fde9009ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a3193ce10f7d220 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:59.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.778+0000 7fde85600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:52:59.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.778+0000 7fde85600640 1 -- 172.21.15.83:0/2766342725 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde900bb070 con 0x5603f56a9b30 2024-04-23T14:52:59.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.778+0000 7fde85600640 1 -- 172.21.15.83:0/2766342725 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fde90004930 con 0x5603f56a9b30 2024-04-23T14:52:59.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.779+0000 7fde8ea00640 1 --2- 172.21.15.83:0/2766342725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 0x5603f56a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:52:59.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.780+0000 7fdea123c740 1 -- 172.21.15.83:0/2766342725 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde80040070 con 0x5603f56a9b30 2024-04-23T14:52:59.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.781+0000 7fde85600640 1 -- 172.21.15.83:0/2766342725 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde90068050 con 0x5603f56a9b30 2024-04-23T14:52:59.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.781+0000 7fdea123c740 10 client.26116 fetch_fsmap learned FSMap version 28 2024-04-23T14:52:59.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.782+0000 7fdea123c740 1 -- 172.21.15.83:0/2766342725 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603f53163f0 con 0x5603f56a9b30 2024-04-23T14:52:59.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.783+0000 7fde85600640 1 -- 172.21.15.83:0/2766342725 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde80040070 con 0x5603f56a9b30 2024-04-23T14:52:59.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.784+0000 7fde85600640 1 -- 172.21.15.83:0/2766342725 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7fde9000b040 con 0x5603f56a9b30 2024-04-23T14:52:59.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.785+0000 7fdea123c740 10 client.26116 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T14:52:59.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.785+0000 7fdea123c740 10 client.26116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:52:59.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.785+0000 7fdea123c740 20 client.26116 populate_metadata read hostname 'smithi083' 2024-04-23T14:52:59.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.785+0000 7fdea123c740 1 --2- 172.21.15.83:0/2766342725 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5603f56b9d70 0x5603f56bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.785+0000 7fdea123c740 4 client.26116 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:52:59.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.785+0000 7fdea123c740 1 -- 172.21.15.83:0/2766342725 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603f5611bf0 con 0x5603f56b9d70 2024-04-23T14:52:59.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.787+0000 7fde97400640 1 --2- 172.21.15.83:0/2766342725 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5603f56b9d70 0x5603f56bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.790+0000 7fde97400640 1 --2- 172.21.15.83:0/2766342725 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5603f56b9d70 0x5603f56bc1e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a809ee73a802b52 server_cookie=ce09bedb0b7ea023 in_seq=0 out_seq=0 2024-04-23T14:52:59.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.790+0000 7fde85600640 10 client.26116 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:52:59.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.792+0000 7fde85600640 1 -- 172.21.15.83:0/2766342725 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603f5611bf0 con 0x5603f56b9d70 2024-04-23T14:52:59.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.792+0000 7fde85600640 10 client.26116 handle_command_reply: tid=0 2024-04-23T14:52:59.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.792+0000 7fdea123c740 1 client.26116 shutdown 2024-04-23T14:52:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.794+0000 7fdea123c740 1 -- 172.21.15.83:0/2766342725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 msgr2=0x5603f56a7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.794+0000 7fdea123c740 1 --2- 172.21.15.83:0/2766342725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 0x5603f56a7010 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde900cedb0 tx=0x7fde9009ed70 comp rx=0 tx=0).stop 2024-04-23T14:52:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.794+0000 7fdea123c740 1 -- 172.21.15.83:0/2766342725 shutdown_connections 2024-04-23T14:52:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.794+0000 7fdea123c740 1 --2- 172.21.15.83:0/2766342725 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5603f56b9d70 0x5603f56bc1e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.794+0000 7fdea123c740 1 --2- 172.21.15.83:0/2766342725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 0x5603f56a7010 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.794+0000 7fdea123c740 1 --2- 172.21.15.83:0/2766342725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 0x5603f56a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.794+0000 7fdea123c740 1 --2- 172.21.15.83:0/2766342725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569eb40 0x5603f56a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.794+0000 7fde85600640 0 client.26116 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:52:59.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.794+0000 7fdea123c740 1 -- 172.21.15.83:0/2766342725 >> 172.21.15.83:0/2766342725 conn(0x5603f569ae60 msgr2=0x5603f56a0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:59.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.795+0000 7fdea123c740 1 -- 172.21.15.83:0/2766342725 shutdown_connections 2024-04-23T14:52:59.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.795+0000 7fdea123c740 1 -- 172.21.15.83:0/2766342725 wait complete. 2024-04-23T14:52:59.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.795+0000 7fdea123c740 20 client.26116 trim_cache size 0 max 16384 2024-04-23T14:52:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.838+0000 7fdea123c740 1 Processor -- start 2024-04-23T14:52:59.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.839+0000 7fdea123c740 1 -- start start 2024-04-23T14:52:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.840+0000 7fdea123c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569eb40 0x5603f56b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.840+0000 7fdea123c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 0x5603f56a9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.840+0000 7fdea123c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 0x5603f56c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.840+0000 7fdea123c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fde981baec0 con 0x5603f569f460 2024-04-23T14:52:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.840+0000 7fdea123c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fde981b8790 con 0x5603f569eb40 2024-04-23T14:52:59.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.840+0000 7fdea123c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde981b65e0 con 0x5603f56a9b30 2024-04-23T14:52:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.841+0000 7fde97e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 0x5603f56c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.841+0000 7fde8ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 0x5603f56a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.841+0000 7fde8ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 0x5603f56a9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45416/0 (socket says 172.21.15.83:45416) 2024-04-23T14:52:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.841+0000 7fde8ea00640 1 -- 172.21.15.83:0/1629285619 learned_addr learned my addr 172.21.15.83:0/1629285619 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.841+0000 7fde97e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 0x5603f56c1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38044/0 (socket says 172.21.15.83:38044) 2024-04-23T14:52:59.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.841+0000 7fde97400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569eb40 0x5603f56b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.842+0000 7fde97e00640 1 -- 172.21.15.83:0/1629285619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569eb40 msgr2=0x5603f56b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.842+0000 7fde97e00640 1 --2- 172.21.15.83:0/1629285619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569eb40 0x5603f56b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.842+0000 7fde97e00640 1 -- 172.21.15.83:0/1629285619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 msgr2=0x5603f56a9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.842+0000 7fde97e00640 1 --2- 172.21.15.83:0/1629285619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 0x5603f56a9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.842+0000 7fde97e00640 1 -- 172.21.15.83:0/1629285619 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f53e07d0 con 0x5603f56a9b30 2024-04-23T14:52:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.842+0000 7fde97e00640 1 --2- 172.21.15.83:0/1629285619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 0x5603f56c1ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fde9009c860 tx=0x7fde9000da80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4844d99b9f42ca2d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.843+0000 7fde87e00640 1 -- 172.21.15.83:0/1629285619 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde900bb070 con 0x5603f56a9b30 2024-04-23T14:52:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.843+0000 7fde87e00640 1 -- 172.21.15.83:0/1629285619 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde9009e860 con 0x5603f56a9b30 2024-04-23T14:52:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.843+0000 7fde87e00640 1 -- 172.21.15.83:0/1629285619 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde900a6e80 con 0x5603f56a9b30 2024-04-23T14:52:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.843+0000 7fde8ea00640 1 --2- 172.21.15.83:0/1629285619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 0x5603f56a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:52:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.844+0000 7fdea123c740 1 -- 172.21.15.83:0/1629285619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 msgr2=0x5603f56c1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.844+0000 7fdea123c740 1 --2- 172.21.15.83:0/1629285619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 0x5603f56c1ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fde9009c860 tx=0x7fde9000da80 comp rx=0 tx=0).stop 2024-04-23T14:52:59.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.844+0000 7fdea123c740 1 -- 172.21.15.83:0/1629285619 shutdown_connections 2024-04-23T14:52:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.844+0000 7fdea123c740 1 --2- 172.21.15.83:0/1629285619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f56a9b30 0x5603f56c1ab0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.844+0000 7fdea123c740 1 --2- 172.21.15.83:0/1629285619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f569f460 0x5603f56a9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.844+0000 7fdea123c740 1 --2- 172.21.15.83:0/1629285619 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569eb40 0x5603f56b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.844+0000 7fdea123c740 1 -- 172.21.15.83:0/1629285619 >> 172.21.15.83:0/1629285619 conn(0x5603f569ae60 msgr2=0x5603f56a1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.845+0000 7fdea123c740 1 -- 172.21.15.83:0/1629285619 shutdown_connections 2024-04-23T14:52:59.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.845+0000 7fdea123c740 1 -- 172.21.15.83:0/1629285619 wait complete. 2024-04-23T14:52:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.846+0000 7fdea123c740 1 -- start start 2024-04-23T14:52:59.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.846+0000 7fdea123c740 1 Processor -- start 2024-04-23T14:52:59.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.847+0000 7fdea123c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 0x5603f56b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.847+0000 7fdea123c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569f460 0x5603f56a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.847+0000 7fdea123c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f56a9b30 0x5603f56a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.848+0000 7fde97400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 0x5603f56b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.848+0000 7fde8ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569f460 0x5603f56a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.848+0000 7fde97400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 0x5603f56b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38060/0 (socket says 172.21.15.83:38060) 2024-04-23T14:52:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.848+0000 7fde8ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569f460 0x5603f56a3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:45152/0 (socket says 172.21.15.83:45152) 2024-04-23T14:52:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.848+0000 7fdea123c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fde981b6c50 con 0x5603f56a9b30 2024-04-23T14:52:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.848+0000 7fdea123c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fde981b80c0 con 0x5603f569f460 2024-04-23T14:52:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.848+0000 7fdea123c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fde981b7320 con 0x5603f569eb40 2024-04-23T14:52:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.848+0000 7fdea123c740 10 client.0 _ll_register_callbacks cb 0x7fde9009e1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:52:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.848+0000 7fde97400640 1 -- 172.21.15.83:0/1103279700 learned_addr learned my addr 172.21.15.83:0/1103279700 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:52:59.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.848+0000 7fde97e00640 1 --2- 172.21.15.83:0/1103279700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f56a9b30 0x5603f56a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.849+0000 7fde97400640 1 -- 172.21.15.83:0/1103279700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569f460 msgr2=0x5603f56a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.849+0000 7fde97400640 1 --2- 172.21.15.83:0/1103279700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569f460 0x5603f56a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.849+0000 7fde97400640 1 -- 172.21.15.83:0/1103279700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f56a9b30 msgr2=0x5603f56a44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:52:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.849+0000 7fde97400640 1 --2- 172.21.15.83:0/1103279700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f56a9b30 0x5603f56a44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.849+0000 7fde97400640 1 -- 172.21.15.83:0/1103279700 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f56aac20 con 0x5603f569eb40 2024-04-23T14:52:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.850+0000 7fde97400640 1 --2- 172.21.15.83:0/1103279700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 0x5603f56b2200 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fde8002a930 tx=0x7fde80030360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6472ac7eb76c4758 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:52:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.850+0000 7fde85600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:52:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.850+0000 7fde85600640 1 -- 172.21.15.83:0/1103279700 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80037040 con 0x5603f569eb40 2024-04-23T14:52:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.850+0000 7fdea123c740 1 -- 172.21.15.83:0/1103279700 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde9009e860 con 0x5603f569eb40 2024-04-23T14:52:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.851+0000 7fde85600640 1 -- 172.21.15.83:0/1103279700 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fde8004fbd0 con 0x5603f569eb40 2024-04-23T14:52:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.851+0000 7fde85600640 1 -- 172.21.15.83:0/1103279700 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde8004fdd0 con 0x5603f569eb40 2024-04-23T14:52:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.851+0000 7fdea123c740 10 client.26122 fetch_fsmap learned FSMap version 28 2024-04-23T14:52:59.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.851+0000 7fdea123c740 1 -- 172.21.15.83:0/1103279700 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603f5729000 con 0x5603f569eb40 2024-04-23T14:52:59.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.851+0000 7fde85600640 1 -- 172.21.15.83:0/1103279700 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde9009e860 con 0x5603f569eb40 2024-04-23T14:52:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.853+0000 7fde85600640 1 -- 172.21.15.83:0/1103279700 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7fde8003a070 con 0x5603f569eb40 2024-04-23T14:52:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.853+0000 7fdea123c740 10 client.26122 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T14:52:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.853+0000 7fdea123c740 10 client.26122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:52:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.853+0000 7fdea123c740 20 client.26122 populate_metadata read hostname 'smithi083' 2024-04-23T14:52:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.854+0000 7fdea123c740 1 --2- 172.21.15.83:0/1103279700 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5603f5729400 0x5603f572b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:52:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.854+0000 7fdea123c740 4 client.26122 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:52:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.854+0000 7fdea123c740 1 -- 172.21.15.83:0/1103279700 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603f560ba60 con 0x5603f5729400 2024-04-23T14:52:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.854+0000 7fde8ea00640 1 --2- 172.21.15.83:0/1103279700 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5603f5729400 0x5603f572b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:52:59.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.856+0000 7fde8ea00640 1 --2- 172.21.15.83:0/1103279700 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5603f5729400 0x5603f572b7f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3200cfc05e8c45bc server_cookie=4050433029964471 in_seq=0 out_seq=0 2024-04-23T14:52:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.858+0000 7fde85600640 10 client.26122 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:52:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.858+0000 7fde85600640 1 -- 172.21.15.83:0/1103279700 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603f560ba60 con 0x5603f5729400 2024-04-23T14:52:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.858+0000 7fde85600640 10 client.26122 handle_command_reply: tid=0 2024-04-23T14:52:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.858+0000 7fdea123c740 1 client.26122 shutdown 2024-04-23T14:52:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.859+0000 7fdea123c740 1 -- 172.21.15.83:0/1103279700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 msgr2=0x5603f56b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.859+0000 7fdea123c740 1 --2- 172.21.15.83:0/1103279700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 0x5603f56b2200 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fde8002a930 tx=0x7fde80030360 comp rx=0 tx=0).stop 2024-04-23T14:52:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.859+0000 7fdea123c740 1 -- 172.21.15.83:0/1103279700 shutdown_connections 2024-04-23T14:52:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.859+0000 7fdea123c740 1 --2- 172.21.15.83:0/1103279700 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5603f5729400 0x5603f572b7f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.859+0000 7fdea123c740 1 --2- 172.21.15.83:0/1103279700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603f56a9b30 0x5603f56a44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.859+0000 7fdea123c740 1 --2- 172.21.15.83:0/1103279700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5603f569f460 0x5603f56a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.859+0000 7fdea123c740 1 --2- 172.21.15.83:0/1103279700 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5603f569eb40 0x5603f56b2200 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.859+0000 7fdea123c740 1 -- 172.21.15.83:0/1103279700 >> 172.21.15.83:0/1103279700 conn(0x5603f569ae60 msgr2=0x5603f56996c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.859+0000 7fde85600640 0 client.26122 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:52:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.859+0000 7fdea123c740 1 -- 172.21.15.83:0/1103279700 shutdown_connections 2024-04-23T14:52:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.859+0000 7fdea123c740 1 -- 172.21.15.83:0/1103279700 wait complete. 2024-04-23T14:52:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.860+0000 7fdea123c740 20 client.26122 trim_cache size 0 max 16384 2024-04-23T14:52:59.860 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:52:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.861+0000 7fde9ce00640 1 -- 172.21.15.83:0/3073392059 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fde7c0a3da0 msgr2=0x7fde7c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.861+0000 7fde9ce00640 1 --2- 172.21.15.83:0/3073392059 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fde7c0a3da0 0x7fde7c0a6260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fde88005fd0 tx=0x7fde88013480 comp rx=0 tx=0).stop 2024-04-23T14:52:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.861+0000 7fde9ce00640 1 -- 172.21.15.83:0/3073392059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde981a7cf0 msgr2=0x7fde981a81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:52:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.861+0000 7fde9ce00640 1 --2- 172.21.15.83:0/3073392059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde981a7cf0 0x7fde981a81a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fde9000db10 tx=0x7fde9000a250 comp rx=0 tx=0).stop 2024-04-23T14:52:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.861+0000 7fde9ce00640 1 -- 172.21.15.83:0/3073392059 shutdown_connections 2024-04-23T14:52:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.861+0000 7fde9ce00640 1 --2- 172.21.15.83:0/3073392059 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde981a7cf0 0x7fde981a81a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.861+0000 7fde9ce00640 1 --2- 172.21.15.83:0/3073392059 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde98109270 0x7fde981a3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.861+0000 7fde9ce00640 1 --2- 172.21.15.83:0/3073392059 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fde7c0a3da0 0x7fde7c0a6260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.861+0000 7fde9ce00640 1 --2- 172.21.15.83:0/3073392059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fde98069aa0 0x7fde981a3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:52:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.861+0000 7fde9ce00640 1 -- 172.21.15.83:0/3073392059 >> 172.21.15.83:0/3073392059 conn(0x7fde9806d3e0 msgr2=0x7fde98071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:52:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.863+0000 7fde9ce00640 1 -- 172.21.15.83:0/3073392059 shutdown_connections 2024-04-23T14:52:59.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:52:59.863+0000 7fde9ce00640 1 -- 172.21.15.83:0/3073392059 wait complete. 2024-04-23T14:53:00.230 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:00 smithi083 ceph-mon[27867]: pgmap v139: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 MiB/s wr, 8.88k op/s 2024-04-23T14:53:00.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:00 smithi173 ceph-mon[32443]: pgmap v139: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 MiB/s wr, 8.88k op/s 2024-04-23T14:53:00.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:00 smithi203 ceph-mon[32800]: pgmap v139: 97 pgs: 97 active+clean; 18 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 42 MiB/s wr, 8.88k op/s 2024-04-23T14:53:00.903 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:53:00.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:53:00.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:53:01.967 INFO:tasks.workunit.client.0.smithi083.stdout:Wrote -1 instead of 4096 bytes. 2024-04-23T14:53:01.967 INFO:tasks.workunit.client.0.smithi083.stdout:Probably out of disk space 2024-04-23T14:53:01.968 INFO:tasks.workunit.client.0.smithi083.stderr:write: Resource temporarily unavailable 2024-04-23T14:53:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.098+0000 7f3763e00640 1 -- 172.21.15.83:0/275516198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3764102070 msgr2=0x7f3764102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.098+0000 7f3763e00640 1 --2- 172.21.15.83:0/275516198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3764102070 0x7f3764102450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3754009900 tx=0x7f375402a410 comp rx=0 tx=0).stop 2024-04-23T14:53:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.099+0000 7f3763e00640 1 -- 172.21.15.83:0/275516198 shutdown_connections 2024-04-23T14:53:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.099+0000 7f3763e00640 1 --2- 172.21.15.83:0/275516198 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3764107140 0x7f376410ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.099+0000 7f3763e00640 1 --2- 172.21.15.83:0/275516198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764102a20 0x7f3764106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.099+0000 7f3763e00640 1 --2- 172.21.15.83:0/275516198 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3764102070 0x7f3764102450 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.099+0000 7f3763e00640 1 -- 172.21.15.83:0/275516198 >> 172.21.15.83:0/275516198 conn(0x7f37640fd910 msgr2=0x7f37640ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:02.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.101+0000 7f3763e00640 1 -- 172.21.15.83:0/275516198 shutdown_connections 2024-04-23T14:53:02.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.101+0000 7f3763e00640 1 -- 172.21.15.83:0/275516198 wait complete. 2024-04-23T14:53:02.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.103+0000 7f3763e00640 1 Processor -- start 2024-04-23T14:53:02.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.103+0000 7f3763e00640 1 -- start start 2024-04-23T14:53:02.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.107+0000 7f3763e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764102070 0x7f3764116540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.107+0000 7f3763e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3764107140 0x7f3764116a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.107+0000 7f3763e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f376411c250 0x7f376411c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.107+0000 7f3763e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3764075fb0 con 0x7f3764102070 2024-04-23T14:53:02.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.107+0000 7f3763e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f3764075e30 con 0x7f376411c250 2024-04-23T14:53:02.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.107+0000 7f3763e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3764076130 con 0x7f3764107140 2024-04-23T14:53:02.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.108+0000 7f3762000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3764107140 0x7f3764116a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.108+0000 7f3762000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3764107140 0x7f3764116a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38078/0 (socket says 172.21.15.83:38078) 2024-04-23T14:53:02.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.108+0000 7f3762000640 1 -- 172.21.15.83:0/532752103 learned_addr learned my addr 172.21.15.83:0/532752103 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:02.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.108+0000 7f3763400640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f376411c250 0x7f376411c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.108+0000 7f3762a00640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764102070 0x7f3764116540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.108+0000 7f3763400640 1 -- 172.21.15.83:0/532752103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3764107140 msgr2=0x7f3764116a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.108+0000 7f3763400640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3764107140 0x7f3764116a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.109+0000 7f3763400640 1 -- 172.21.15.83:0/532752103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764102070 msgr2=0x7f3764116540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.109+0000 7f3763400640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764102070 0x7f3764116540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.109+0000 7f3763400640 1 -- 172.21.15.83:0/532752103 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37641116a0 con 0x7f376411c250 2024-04-23T14:53:02.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.109+0000 7f3763400640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f376411c250 0x7f376411c700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f375000e970 tx=0x7f375000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:02.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.109+0000 7f375b400640 1 -- 172.21.15.83:0/532752103 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375000ccc0 con 0x7f376411c250 2024-04-23T14:53:02.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.110+0000 7f375b400640 1 -- 172.21.15.83:0/532752103 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3750005020 con 0x7f376411c250 2024-04-23T14:53:02.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.110+0000 7f375b400640 1 -- 172.21.15.83:0/532752103 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750004a00 con 0x7f376411c250 2024-04-23T14:53:02.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.110+0000 7f3763e00640 1 -- 172.21.15.83:0/532752103 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3764111990 con 0x7f376411c250 2024-04-23T14:53:02.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.110+0000 7f3763e00640 1 -- 172.21.15.83:0/532752103 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3764111e30 con 0x7f376411c250 2024-04-23T14:53:02.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.120+0000 7f375b400640 1 -- 172.21.15.83:0/532752103 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f37500051d0 con 0x7f376411c250 2024-04-23T14:53:02.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.122+0000 7f375b400640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f37480a3da0 0x7f37480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.123+0000 7f375b400640 1 -- 172.21.15.83:0/532752103 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f37500d07f0 con 0x7f376411c250 2024-04-23T14:53:02.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.123+0000 7f3762a00640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f37480a3da0 0x7f37480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.126+0000 7f376c7fb740 1 Processor -- start 2024-04-23T14:53:02.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.128+0000 7f376c7fb740 1 -- start start 2024-04-23T14:53:02.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.129+0000 7f3762a00640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f37480a3da0 0x7f37480a6260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3754000c00 tx=0x7f3754005a20 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:02.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.130+0000 7f376c7fb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297be690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.130+0000 7f376c7fb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297bebd0 0x5613297c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.130+0000 7f376c7fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297c92a0 0x5613297cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.130+0000 7f376c7fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561329723e70 con 0x5613297c92a0 2024-04-23T14:53:02.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.130+0000 7f376c7fb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5613296ca380 con 0x5613297be2b0 2024-04-23T14:53:02.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.130+0000 7f376c7fb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56132971fe30 con 0x5613297bebd0 2024-04-23T14:53:02.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.130+0000 7f3762a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297be690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.130+0000 7f3762a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297be690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:45170/0 (socket says 172.21.15.83:45170) 2024-04-23T14:53:02.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.130+0000 7f3762a00640 1 -- 172.21.15.83:0/3819537934 learned_addr learned my addr 172.21.15.83:0/3819537934 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:02.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.131+0000 7f3763400640 1 --2- 172.21.15.83:0/3819537934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297c92a0 0x5613297cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.131+0000 7f3762a00640 1 -- 172.21.15.83:0/3819537934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297bebd0 msgr2=0x5613297c8d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:53:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.131+0000 7f3762a00640 1 --2- 172.21.15.83:0/3819537934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297bebd0 0x5613297c8d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.131+0000 7f3762a00640 1 -- 172.21.15.83:0/3819537934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297c92a0 msgr2=0x5613297cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.131+0000 7f3762a00640 1 --2- 172.21.15.83:0/3819537934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297c92a0 0x5613297cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.131+0000 7f3762a00640 1 -- 172.21.15.83:0/3819537934 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613294fe7e0 con 0x5613297be2b0 2024-04-23T14:53:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.131+0000 7f3763400640 1 --2- 172.21.15.83:0/3819537934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297c92a0 0x5613297cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:53:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.132+0000 7f3762a00640 1 --2- 172.21.15.83:0/3819537934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297be690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3754009900 tx=0x7f375403c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfeec903cbe68cef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.133+0000 7f374fe00640 1 -- 172.21.15.83:0/3819537934 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754035050 con 0x5613297be2b0 2024-04-23T14:53:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.133+0000 7f374fe00640 1 -- 172.21.15.83:0/3819537934 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375404d070 con 0x5613297be2b0 2024-04-23T14:53:02.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.133+0000 7f374fe00640 1 -- 172.21.15.83:0/3819537934 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754032a80 con 0x5613297be2b0 2024-04-23T14:53:02.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.134+0000 7f376c7fb740 1 -- 172.21.15.83:0/3819537934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 msgr2=0x5613297be690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.134+0000 7f376c7fb740 1 --2- 172.21.15.83:0/3819537934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297be690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3754009900 tx=0x7f375403c000 comp rx=0 tx=0).stop 2024-04-23T14:53:02.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.135+0000 7f376c7fb740 1 -- 172.21.15.83:0/3819537934 shutdown_connections 2024-04-23T14:53:02.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.135+0000 7f376c7fb740 1 --2- 172.21.15.83:0/3819537934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297c92a0 0x5613297cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.135+0000 7f376c7fb740 1 --2- 172.21.15.83:0/3819537934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297bebd0 0x5613297c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.135+0000 7f376c7fb740 1 --2- 172.21.15.83:0/3819537934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297be690 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.135+0000 7f376c7fb740 1 -- 172.21.15.83:0/3819537934 >> 172.21.15.83:0/3819537934 conn(0x5613297ba5d0 msgr2=0x5613297ba9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:02.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.136+0000 7f376c7fb740 1 -- 172.21.15.83:0/3819537934 shutdown_connections 2024-04-23T14:53:02.136 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.136+0000 7f376c7fb740 1 -- 172.21.15.83:0/3819537934 wait complete. 2024-04-23T14:53:02.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.137+0000 7f376c7fb740 1 -- start start 2024-04-23T14:53:02.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.137+0000 7f376c7fb740 1 Processor -- start 2024-04-23T14:53:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.138+0000 7f376c7fb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.138+0000 7f376c7fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297bebd0 0x5613297c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.138+0000 7f376c7fb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 0x5613297c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.138+0000 7f376c7fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5613297cc030 con 0x5613297bebd0 2024-04-23T14:53:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.138+0000 7f376c7fb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5613296dd660 con 0x5613297be2b0 2024-04-23T14:53:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.138+0000 7f376c7fb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5613297cc1b0 con 0x5613297c92a0 2024-04-23T14:53:02.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.138+0000 7f376c7fb740 10 client.0 _ll_register_callbacks cb 0x5613296cbaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.139+0000 7f3763400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 0x5613297c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.139+0000 7f3763400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 0x5613297c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38084/0 (socket says 172.21.15.83:38084) 2024-04-23T14:53:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.139+0000 7f3763400640 1 -- 172.21.15.83:0/2076582982 learned_addr learned my addr 172.21.15.83:0/2076582982 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.139+0000 7f3762a00640 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.140+0000 7f3762000640 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297bebd0 0x5613297c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.140+0000 7f3763400640 1 -- 172.21.15.83:0/2076582982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 msgr2=0x5613297c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.140+0000 7f3763400640 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.140+0000 7f3763400640 1 -- 172.21.15.83:0/2076582982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297bebd0 msgr2=0x5613297c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.140+0000 7f3763400640 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297bebd0 0x5613297c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.140+0000 7f3763400640 1 -- 172.21.15.83:0/2076582982 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56132955ef70 con 0x5613297c92a0 2024-04-23T14:53:02.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.141+0000 7f3763400640 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 0x5613297c6c20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3750099110 tx=0x7f375009c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e87194e6d00abeb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:02.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.142+0000 7f374cc00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:53:02.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.142+0000 7f374cc00640 1 -- 172.21.15.83:0/2076582982 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375001c040 con 0x5613297c92a0 2024-04-23T14:53:02.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.142+0000 7f376c7fb740 1 -- 172.21.15.83:0/2076582982 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375404d070 con 0x5613297c92a0 2024-04-23T14:53:02.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.142+0000 7f374cc00640 1 -- 172.21.15.83:0/2076582982 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3750098db0 con 0x5613297c92a0 2024-04-23T14:53:02.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.143+0000 7f374cc00640 1 -- 172.21.15.83:0/2076582982 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f37500a4f20 con 0x5613297c92a0 2024-04-23T14:53:02.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.143+0000 7f376c7fb740 10 client.26131 fetch_fsmap learned FSMap version 28 2024-04-23T14:53:02.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.144+0000 7f376c7fb740 1 -- 172.21.15.83:0/2076582982 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5613293f6170 con 0x5613297c92a0 2024-04-23T14:53:02.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.144+0000 7f374cc00640 1 -- 172.21.15.83:0/2076582982 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375404d070 con 0x5613297c92a0 2024-04-23T14:53:02.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.144+0000 7f374cc00640 1 -- 172.21.15.83:0/2076582982 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f375009bb90 con 0x5613297c92a0 2024-04-23T14:53:02.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.144+0000 7f376c7fb740 10 client.26131 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T14:53:02.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.144+0000 7f376c7fb740 10 client.26131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:53:02.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.144+0000 7f376c7fb740 20 client.26131 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:02.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.144+0000 7f376c7fb740 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613297eaa00 0x5613297ece20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.145+0000 7f376c7fb740 4 client.26131 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:53:02.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.145+0000 7f376c7fb740 1 -- 172.21.15.83:0/2076582982 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561329731890 con 0x5613297eaa00 2024-04-23T14:53:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.145+0000 7f3762a00640 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613297eaa00 0x5613297ece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.146+0000 7f3762a00640 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613297eaa00 0x5613297ece20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=697c1797fdf34e62 server_cookie=6a85095d5e414309 in_seq=0 out_seq=0 2024-04-23T14:53:02.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.146+0000 7f374cc00640 10 client.26131 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:02.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.147+0000 7f374cc00640 1 -- 172.21.15.83:0/2076582982 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561329731890 con 0x5613297eaa00 2024-04-23T14:53:02.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.147+0000 7f374cc00640 10 client.26131 handle_command_reply: tid=0 2024-04-23T14:53:02.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.147+0000 7f376c7fb740 1 client.26131 shutdown 2024-04-23T14:53:02.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.148+0000 7f376c7fb740 1 -- 172.21.15.83:0/2076582982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 msgr2=0x5613297c6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.148+0000 7f376c7fb740 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 0x5613297c6c20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3750099110 tx=0x7f375009c740 comp rx=0 tx=0).stop 2024-04-23T14:53:02.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.148+0000 7f376c7fb740 1 -- 172.21.15.83:0/2076582982 shutdown_connections 2024-04-23T14:53:02.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.148+0000 7f376c7fb740 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613297eaa00 0x5613297ece20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.148+0000 7f376c7fb740 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 0x5613297c6c20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.148+0000 7f376c7fb740 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297bebd0 0x5613297c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.148+0000 7f376c7fb740 1 --2- 172.21.15.83:0/2076582982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.148+0000 7f376c7fb740 1 -- 172.21.15.83:0/2076582982 >> 172.21.15.83:0/2076582982 conn(0x5613297ba5d0 msgr2=0x5613297c03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:02.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.148+0000 7f374cc00640 0 client.26131 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:02.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.149+0000 7f376c7fb740 1 -- 172.21.15.83:0/2076582982 shutdown_connections 2024-04-23T14:53:02.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.151+0000 7f376c7fb740 1 -- 172.21.15.83:0/2076582982 wait complete. 2024-04-23T14:53:02.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.152+0000 7f376c7fb740 20 client.26131 trim_cache size 0 max 16384 2024-04-23T14:53:02.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.191+0000 7f376c7fb740 1 Processor -- start 2024-04-23T14:53:02.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.192+0000 7f376c7fb740 1 -- start start 2024-04-23T14:53:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.192+0000 7f376c7fb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297d4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.192+0000 7f376c7fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297bebd0 0x5613297f2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.192+0000 7f376c7fb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 0x5613297f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.192+0000 7f376c7fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37641b6ae0 con 0x5613297bebd0 2024-04-23T14:53:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.192+0000 7f376c7fb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f37640737c0 con 0x5613297be2b0 2024-04-23T14:53:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.192+0000 7f376c7fb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3764071610 con 0x5613297c92a0 2024-04-23T14:53:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.193+0000 7f3763400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 0x5613297f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.193+0000 7f3763400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 0x5613297f2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:38100/0 (socket says 172.21.15.83:38100) 2024-04-23T14:53:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.193+0000 7f3762000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297bebd0 0x5613297f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.193+0000 7f3763400640 1 -- 172.21.15.83:0/1385558498 learned_addr learned my addr 172.21.15.83:0/1385558498 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.193+0000 7f3763400640 1 -- 172.21.15.83:0/1385558498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 msgr2=0x5613297d4f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T14:53:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.193+0000 7f3763400640 1 --2- 172.21.15.83:0/1385558498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297d4f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.193+0000 7f3763400640 1 -- 172.21.15.83:0/1385558498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297bebd0 msgr2=0x5613297f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.193+0000 7f3763400640 1 --2- 172.21.15.83:0/1385558498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297bebd0 0x5613297f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.193+0000 7f3763400640 1 -- 172.21.15.83:0/1385558498 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613297e9140 con 0x5613297c92a0 2024-04-23T14:53:02.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.194+0000 7f3763400640 1 --2- 172.21.15.83:0/1385558498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 0x5613297f2c50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f375009a9f0 tx=0x7f375009ce40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1fa9b8cf00d85a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:02.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.195+0000 7f374f400640 1 -- 172.21.15.83:0/1385558498 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375001c070 con 0x5613297c92a0 2024-04-23T14:53:02.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.195+0000 7f374f400640 1 -- 172.21.15.83:0/1385558498 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375007f020 con 0x5613297c92a0 2024-04-23T14:53:02.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.195+0000 7f374f400640 1 -- 172.21.15.83:0/1385558498 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375009aad0 con 0x5613297c92a0 2024-04-23T14:53:02.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.196+0000 7f376c7fb740 1 -- 172.21.15.83:0/1385558498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 msgr2=0x5613297f2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.196+0000 7f376c7fb740 1 --2- 172.21.15.83:0/1385558498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 0x5613297f2c50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f375009a9f0 tx=0x7f375009ce40 comp rx=0 tx=0).stop 2024-04-23T14:53:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.200+0000 7f376c7fb740 1 -- 172.21.15.83:0/1385558498 shutdown_connections 2024-04-23T14:53:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.200+0000 7f376c7fb740 1 --2- 172.21.15.83:0/1385558498 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297c92a0 0x5613297f2c50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.200+0000 7f376c7fb740 1 --2- 172.21.15.83:0/1385558498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297bebd0 0x5613297f2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.200+0000 7f376c7fb740 1 --2- 172.21.15.83:0/1385558498 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297be2b0 0x5613297d4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.200+0000 7f376c7fb740 1 -- 172.21.15.83:0/1385558498 >> 172.21.15.83:0/1385558498 conn(0x5613297ba5d0 msgr2=0x5613297b9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.201+0000 7f376c7fb740 1 -- 172.21.15.83:0/1385558498 shutdown_connections 2024-04-23T14:53:02.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.201+0000 7f376c7fb740 1 -- 172.21.15.83:0/1385558498 wait complete. 2024-04-23T14:53:02.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.202+0000 7f376c7fb740 1 -- start start 2024-04-23T14:53:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.202+0000 7f376c7fb740 1 Processor -- start 2024-04-23T14:53:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.203+0000 7f376c7fb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297be2b0 0x56132984a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.203+0000 7f376c7fb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297bebd0 0x56132984a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.203+0000 7f376c7fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297c92a0 0x5613297cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.203+0000 7f376c7fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3764071c80 con 0x5613297c92a0 2024-04-23T14:53:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.203+0000 7f376c7fb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f37640730f0 con 0x5613297bebd0 2024-04-23T14:53:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.203+0000 7f376c7fb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3764072350 con 0x5613297be2b0 2024-04-23T14:53:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.203+0000 7f376c7fb740 10 client.0 _ll_register_callbacks cb 0x7f3750098340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.203+0000 7f3762000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297bebd0 0x56132984a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.203+0000 7f3762000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297bebd0 0x56132984a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:45186/0 (socket says 172.21.15.83:45186) 2024-04-23T14:53:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.203+0000 7f3762000640 1 -- 172.21.15.83:0/1648675533 learned_addr learned my addr 172.21.15.83:0/1648675533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:02.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.203+0000 7f3762a00640 1 --2- 172.21.15.83:0/1648675533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297be2b0 0x56132984a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.204+0000 7f3762000640 1 -- 172.21.15.83:0/1648675533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297be2b0 msgr2=0x56132984a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.204+0000 7f3762000640 1 --2- 172.21.15.83:0/1648675533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297be2b0 0x56132984a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.204+0000 7f3762000640 1 -- 172.21.15.83:0/1648675533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297c92a0 msgr2=0x5613297cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.204+0000 7f3762000640 1 --2- 172.21.15.83:0/1648675533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297c92a0 0x5613297cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.204+0000 7f3762000640 1 -- 172.21.15.83:0/1648675533 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613297cdd80 con 0x5613297bebd0 2024-04-23T14:53:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.205+0000 7f3762000640 1 --2- 172.21.15.83:0/1648675533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297bebd0 0x56132984a840 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f375c005c40 tx=0x7f375c011e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65e55a3bd1b19c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:02.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.205+0000 7f374cc00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:53:02.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.206+0000 7f374cc00640 1 -- 172.21.15.83:0/1648675533 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c00fc00 con 0x5613297bebd0 2024-04-23T14:53:02.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.206+0000 7f376c7fb740 1 -- 172.21.15.83:0/1648675533 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375007f020 con 0x5613297bebd0 2024-04-23T14:53:02.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.206+0000 7f374cc00640 1 -- 172.21.15.83:0/1648675533 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f375c00fda0 con 0x5613297bebd0 2024-04-23T14:53:02.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.207+0000 7f374cc00640 1 -- 172.21.15.83:0/1648675533 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f375c004b20 con 0x5613297bebd0 2024-04-23T14:53:02.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.207+0000 7f376c7fb740 10 client.26180 fetch_fsmap learned FSMap version 28 2024-04-23T14:53:02.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.207+0000 7f376c7fb740 1 -- 172.21.15.83:0/1648675533 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561329851b10 con 0x5613297bebd0 2024-04-23T14:53:02.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.207+0000 7f374cc00640 1 -- 172.21.15.83:0/1648675533 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375007f020 con 0x5613297bebd0 2024-04-23T14:53:02.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.209+0000 7f374cc00640 1 -- 172.21.15.83:0/1648675533 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f375c0195d0 con 0x5613297bebd0 2024-04-23T14:53:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.209+0000 7f376c7fb740 10 client.26180 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T14:53:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.209+0000 7f376c7fb740 10 client.26180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:53:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.209+0000 7f376c7fb740 20 client.26180 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.209+0000 7f376c7fb740 1 --2- 172.21.15.83:0/1648675533 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613298522c0 0x5613298546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.209+0000 7f376c7fb740 4 client.26180 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:53:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.209+0000 7f376c7fb740 1 -- 172.21.15.83:0/1648675533 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56132972b700 con 0x5613298522c0 2024-04-23T14:53:02.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.209+0000 7f3762a00640 1 --2- 172.21.15.83:0/1648675533 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613298522c0 0x5613298546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.211+0000 7f3762a00640 1 --2- 172.21.15.83:0/1648675533 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613298522c0 0x5613298546e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc3ccfa607dd225 server_cookie=712e136fcb45aab7 in_seq=0 out_seq=0 2024-04-23T14:53:02.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.211+0000 7f374cc00640 10 client.26180 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.212+0000 7f374cc00640 1 -- 172.21.15.83:0/1648675533 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56132972b700 con 0x5613298522c0 2024-04-23T14:53:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.212+0000 7f374cc00640 10 client.26180 handle_command_reply: tid=0 2024-04-23T14:53:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.212+0000 7f376c7fb740 1 client.26180 shutdown 2024-04-23T14:53:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.213+0000 7f376c7fb740 1 -- 172.21.15.83:0/1648675533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297bebd0 msgr2=0x56132984a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.213 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.213+0000 7f376c7fb740 1 --2- 172.21.15.83:0/1648675533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297bebd0 0x56132984a840 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f375c005c40 tx=0x7f375c011e10 comp rx=0 tx=0).stop 2024-04-23T14:53:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.213+0000 7f376c7fb740 1 -- 172.21.15.83:0/1648675533 shutdown_connections 2024-04-23T14:53:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.213+0000 7f376c7fb740 1 --2- 172.21.15.83:0/1648675533 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613298522c0 0x5613298546e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.213+0000 7f376c7fb740 1 --2- 172.21.15.83:0/1648675533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613297c92a0 0x5613297cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.213+0000 7f376c7fb740 1 --2- 172.21.15.83:0/1648675533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613297bebd0 0x56132984a840 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.213+0000 7f376c7fb740 1 --2- 172.21.15.83:0/1648675533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613297be2b0 0x56132984a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.213+0000 7f376c7fb740 1 -- 172.21.15.83:0/1648675533 >> 172.21.15.83:0/1648675533 conn(0x5613297ba5d0 msgr2=0x5613297c5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.213+0000 7f374cc00640 0 client.26180 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.213+0000 7f376c7fb740 1 -- 172.21.15.83:0/1648675533 shutdown_connections 2024-04-23T14:53:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.213+0000 7f376c7fb740 1 -- 172.21.15.83:0/1648675533 wait complete. 2024-04-23T14:53:02.214 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.214+0000 7f376c7fb740 20 client.26180 trim_cache size 0 max 16384 2024-04-23T14:53:02.215 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:53:02.215 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:53:02.215 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "795da87d-4c0a-4542-b0c5-c25c5782569b", 2024-04-23T14:53:02.215 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:53:02.215 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:53:02.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.216+0000 7f3763e00640 1 -- 172.21.15.83:0/532752103 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f37480a3da0 msgr2=0x7f37480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.216+0000 7f3763e00640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f37480a3da0 0x7f37480a6260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3754000c00 tx=0x7f3754005a20 comp rx=0 tx=0).stop 2024-04-23T14:53:02.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.216+0000 7f3763e00640 1 -- 172.21.15.83:0/532752103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f376411c250 msgr2=0x7f376411c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.216+0000 7f3763e00640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f376411c250 0x7f376411c700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f375000e970 tx=0x7f375000ee30 comp rx=0 tx=0).stop 2024-04-23T14:53:02.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.216+0000 7f3763e00640 1 -- 172.21.15.83:0/532752103 shutdown_connections 2024-04-23T14:53:02.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.216+0000 7f3763e00640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f376411c250 0x7f376411c700 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.216+0000 7f3763e00640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3764107140 0x7f3764116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.216+0000 7f3763e00640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f37480a3da0 0x7f37480a6260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.216+0000 7f3763e00640 1 --2- 172.21.15.83:0/532752103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3764102070 0x7f3764116540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.216+0000 7f3763e00640 1 -- 172.21.15.83:0/532752103 >> 172.21.15.83:0/532752103 conn(0x7f37640fd910 msgr2=0x7f37640ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:02.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.216+0000 7f3763e00640 1 -- 172.21.15.83:0/532752103 shutdown_connections 2024-04-23T14:53:02.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.216+0000 7f3763e00640 1 -- 172.21.15.83:0/532752103 wait complete. 2024-04-23T14:53:02.242 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:02 smithi083 ceph-mon[27867]: pgmap v140: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 54 MiB/s wr, 11.36k op/s 2024-04-23T14:53:02.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:53:02.246 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:53:02.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.530+0000 7f29ed800640 1 -- 172.21.15.83:0/1667308975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e81044b0 msgr2=0x7f29e8104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.530 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.530+0000 7f29ed800640 1 --2- 172.21.15.83:0/1667308975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e81044b0 0x7f29e8104890 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009ca0 tx=0x7f29e002a8c0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.531+0000 7f29ed800640 1 -- 172.21.15.83:0/1667308975 shutdown_connections 2024-04-23T14:53:02.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.531+0000 7f29ed800640 1 --2- 172.21.15.83:0/1667308975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29e8109580 0x7f29e8111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.531+0000 7f29ed800640 1 --2- 172.21.15.83:0/1667308975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e8104e60 0x7f29e8108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.531+0000 7f29ed800640 1 --2- 172.21.15.83:0/1667308975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e81044b0 0x7f29e8104890 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.531+0000 7f29ed800640 1 -- 172.21.15.83:0/1667308975 >> 172.21.15.83:0/1667308975 conn(0x7f29e80ffd90 msgr2=0x7f29e81021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:02.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.531+0000 7f29ed800640 1 -- 172.21.15.83:0/1667308975 shutdown_connections 2024-04-23T14:53:02.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.531+0000 7f29ed800640 1 -- 172.21.15.83:0/1667308975 wait complete. 2024-04-23T14:53:02.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.533+0000 7f29ed800640 1 Processor -- start 2024-04-23T14:53:02.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.533+0000 7f29ed800640 1 -- start start 2024-04-23T14:53:02.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.533+0000 7f29ed800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e81044b0 0x7f29e81a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.533+0000 7f29ed800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e8109580 0x7f29e81a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.533+0000 7f29ed800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29e819a9b0 0x7f29e819ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.533+0000 7f29ed800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29e8114690 con 0x7f29e81044b0 2024-04-23T14:53:02.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.533+0000 7f29ed800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29e8114510 con 0x7f29e819a9b0 2024-04-23T14:53:02.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.533+0000 7f29ed800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f29e8114810 con 0x7f29e8109580 2024-04-23T14:53:02.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.534+0000 7f29e7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29e819a9b0 0x7f29e819ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.534+0000 7f29e6000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e8109580 0x7f29e81a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.534+0000 7f29e7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29e819a9b0 0x7f29e819ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:45208/0 (socket says 172.21.15.83:45208) 2024-04-23T14:53:02.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.534+0000 7f29e7400640 1 -- 172.21.15.83:0/2070836407 learned_addr learned my addr 172.21.15.83:0/2070836407 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:02.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.534+0000 7f29e6000640 1 -- 172.21.15.83:0/2070836407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29e819a9b0 msgr2=0x7f29e819ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.534+0000 7f29e6000640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29e819a9b0 0x7f29e819ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.534+0000 7f29e6000640 1 -- 172.21.15.83:0/2070836407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e81044b0 msgr2=0x7f29e81a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.534+0000 7f29e6a00640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e81044b0 0x7f29e81a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.535+0000 7f29e6000640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e81044b0 0x7f29e81a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.535+0000 7f29e6000640 1 -- 172.21.15.83:0/2070836407 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e819b4d0 con 0x7f29e8109580 2024-04-23T14:53:02.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.535+0000 7f29e6a00640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e81044b0 0x7f29e81a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:53:02.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.536+0000 7f29e6000640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e8109580 0x7f29e81a0d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c940 tx=0x7f29d800ce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:02.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.536+0000 7f29df400640 1 -- 172.21.15.83:0/2070836407 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8007bb0 con 0x7f29e8109580 2024-04-23T14:53:02.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.536+0000 7f29df400640 1 -- 172.21.15.83:0/2070836407 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d8007d50 con 0x7f29e8109580 2024-04-23T14:53:02.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.536+0000 7f29df400640 1 -- 172.21.15.83:0/2070836407 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d800f450 con 0x7f29e8109580 2024-04-23T14:53:02.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.536+0000 7f29ed800640 1 -- 172.21.15.83:0/2070836407 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e819b7c0 con 0x7f29e8109580 2024-04-23T14:53:02.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.536+0000 7f29ed800640 1 -- 172.21.15.83:0/2070836407 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e81a79c0 con 0x7f29e8109580 2024-04-23T14:53:02.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.540+0000 7f29df400640 1 -- 172.21.15.83:0/2070836407 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f29d8004ea0 con 0x7f29e8109580 2024-04-23T14:53:02.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.542+0000 7f29df400640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f29cc0a3e20 0x7f29cc0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.542+0000 7f29f05b7740 1 Processor -- start 2024-04-23T14:53:02.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.543+0000 7f29df400640 1 -- 172.21.15.83:0/2070836407 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f29d80d04d0 con 0x7f29e8109580 2024-04-23T14:53:02.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.543+0000 7f29f05b7740 1 -- start start 2024-04-23T14:53:02.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.543+0000 7f29f05b7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x5582637e4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.543+0000 7f29f05b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e4f60 0x5582637f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.543+0000 7f29f05b7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637f8060 0x5582637fa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.543+0000 7f29f05b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55826374a1f0 con 0x5582637e4f60 2024-04-23T14:53:02.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.543+0000 7f29f05b7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5582636f0350 con 0x5582637e45b0 2024-04-23T14:53:02.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.544+0000 7f29f05b7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5582637461b0 con 0x5582637f8060 2024-04-23T14:53:02.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.544+0000 7f29e6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e4f60 0x5582637f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.544+0000 7f29e6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e4f60 0x5582637f7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45510/0 (socket says 172.21.15.83:45510) 2024-04-23T14:53:02.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.544+0000 7f29e6000640 1 -- 172.21.15.83:0/3195493374 learned_addr learned my addr 172.21.15.83:0/3195493374 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:02.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.544+0000 7f29e6a00640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f29cc0a3e20 0x7f29cc0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.544+0000 7f29e7400640 1 --2- 172.21.15.83:0/3195493374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637f8060 0x5582637fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.544+0000 7f29e6000640 1 -- 172.21.15.83:0/3195493374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637f8060 msgr2=0x5582637fa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.544+0000 7f29e6000640 1 --2- 172.21.15.83:0/3195493374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637f8060 0x5582637fa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.544+0000 7f29e6000640 1 -- 172.21.15.83:0/3195493374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 msgr2=0x5582637e4990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T14:53:02.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.544+0000 7f29e6000640 1 --2- 172.21.15.83:0/3195493374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x5582637e4990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.544+0000 7f29e6000640 1 -- 172.21.15.83:0/3195493374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558263744f50 con 0x5582637e4f60 2024-04-23T14:53:02.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.545+0000 7f29e6000640 1 --2- 172.21.15.83:0/3195493374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e4f60 0x5582637f7b20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f29d80a4b60 tx=0x7f29d800a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b427ee6482972b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:02.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.545+0000 7f29d3e00640 1 -- 172.21.15.83:0/3195493374 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d801d040 con 0x5582637e4f60 2024-04-23T14:53:02.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.545+0000 7f29d3e00640 1 -- 172.21.15.83:0/3195493374 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d80a07d0 con 0x5582637e4f60 2024-04-23T14:53:02.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.545+0000 7f29d3e00640 1 -- 172.21.15.83:0/3195493374 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d809b870 con 0x5582637e4f60 2024-04-23T14:53:02.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.546+0000 7f29f05b7740 1 -- 172.21.15.83:0/3195493374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e4f60 msgr2=0x5582637f7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.546+0000 7f29f05b7740 1 --2- 172.21.15.83:0/3195493374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e4f60 0x5582637f7b20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f29d80a4b60 tx=0x7f29d800a520 comp rx=0 tx=0).stop 2024-04-23T14:53:02.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.546+0000 7f29f05b7740 1 -- 172.21.15.83:0/3195493374 shutdown_connections 2024-04-23T14:53:02.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.546+0000 7f29f05b7740 1 --2- 172.21.15.83:0/3195493374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637f8060 0x5582637fa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.546+0000 7f29f05b7740 1 --2- 172.21.15.83:0/3195493374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e4f60 0x5582637f7b20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.546+0000 7f29f05b7740 1 --2- 172.21.15.83:0/3195493374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x5582637e4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.546+0000 7f29f05b7740 1 -- 172.21.15.83:0/3195493374 >> 172.21.15.83:0/3195493374 conn(0x5582637e0b80 msgr2=0x5582637e0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:02.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.547+0000 7f29e6a00640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f29cc0a3e20 0x7f29cc0a62e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f29e819baf0 tx=0x7f29e0000f40 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:02.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.547+0000 7f29f05b7740 1 -- 172.21.15.83:0/3195493374 shutdown_connections 2024-04-23T14:53:02.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.547+0000 7f29f05b7740 1 -- 172.21.15.83:0/3195493374 wait complete. 2024-04-23T14:53:02.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.548+0000 7f29f05b7740 1 -- start start 2024-04-23T14:53:02.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.548+0000 7f29f05b7740 1 Processor -- start 2024-04-23T14:53:02.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.549+0000 7f29f05b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e45b0 0x5582637f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.549+0000 7f29f05b7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637e4f60 0x5582637f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.550+0000 7f29f05b7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637f8060 0x5582637f2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.550+0000 7f29e6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e45b0 0x5582637f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.550+0000 7f29e6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e45b0 0x5582637f1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:45522/0 (socket says 172.21.15.83:45522) 2024-04-23T14:53:02.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.550+0000 7f29e7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637f8060 0x5582637f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.550+0000 7f29e7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637f8060 0x5582637f2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:45226/0 (socket says 172.21.15.83:45226) 2024-04-23T14:53:02.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.550+0000 7f29e6a00640 1 -- 172.21.15.83:0/2505142431 learned_addr learned my addr 172.21.15.83:0/2505142431 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:02.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.550+0000 7f29e6000640 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637e4f60 0x5582637f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.550+0000 7f29f05b7740 1 -- 172.21.15.83:0/2505142431 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5582637e76b0 con 0x5582637e45b0 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.550+0000 7f29f05b7740 1 -- 172.21.15.83:0/2505142431 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558263703690 con 0x5582637f8060 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.550+0000 7f29f05b7740 1 -- 172.21.15.83:0/2505142431 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5582637e7830 con 0x5582637e4f60 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.551+0000 7f29f05b7740 10 client.0 _ll_register_callbacks cb 0x55826373f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.551+0000 7f29e7400640 1 -- 172.21.15.83:0/2505142431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637e4f60 msgr2=0x5582637f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.551+0000 7f29e7400640 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637e4f60 0x5582637f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.551+0000 7f29e7400640 1 -- 172.21.15.83:0/2505142431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e45b0 msgr2=0x5582637f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.551+0000 7f29e7400640 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e45b0 0x5582637f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.551+0000 7f29e7400640 1 -- 172.21.15.83:0/2505142431 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558263702920 con 0x5582637f8060 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.551+0000 7f29e7400640 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637f8060 0x5582637f2290 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x5582637e7190 tx=0x7f29d4013b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b381bf884886fc47 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.551+0000 7f29d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.551+0000 7f29e6a00640 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e45b0 0x5582637f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.552+0000 7f29d0c00640 1 -- 172.21.15.83:0/2505142431 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4020060 con 0x5582637f8060 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.552+0000 7f29d0c00640 1 -- 172.21.15.83:0/2505142431 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f29d4011d50 con 0x5582637f8060 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.552+0000 7f29f05b7740 1 -- 172.21.15.83:0/2505142431 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d80a07d0 con 0x5582637f8060 2024-04-23T14:53:02.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.552+0000 7f29e6000640 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637e4f60 0x5582637f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:53:02.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.552+0000 7f29d0c00640 1 -- 172.21.15.83:0/2505142431 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29d4005020 con 0x5582637f8060 2024-04-23T14:53:02.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.552+0000 7f29f05b7740 10 client.26189 fetch_fsmap learned FSMap version 28 2024-04-23T14:53:02.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.552+0000 7f29f05b7740 1 -- 172.21.15.83:0/2505142431 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5582635267d0 con 0x5582637f8060 2024-04-23T14:53:02.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.552+0000 7f29d0c00640 1 -- 172.21.15.83:0/2505142431 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d80a07d0 con 0x5582637f8060 2024-04-23T14:53:02.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.553+0000 7f29d0c00640 1 -- 172.21.15.83:0/2505142431 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f29d401a030 con 0x5582637f8060 2024-04-23T14:53:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.553+0000 7f29f05b7740 10 client.26189 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T14:53:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.553+0000 7f29f05b7740 10 client.26189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:53:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.553+0000 7f29f05b7740 20 client.26189 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.553+0000 7f29f05b7740 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558263810d00 0x558263813180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.553+0000 7f29f05b7740 4 client.26189 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:53:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.553+0000 7f29f05b7740 1 -- 172.21.15.83:0/2505142431 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558263757c10 con 0x558263810d00 2024-04-23T14:53:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.553+0000 7f29e6a00640 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558263810d00 0x558263813180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.554+0000 7f29e6a00640 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558263810d00 0x558263813180 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cca2def8ff31415f server_cookie=8883a9dc55fd052a in_seq=0 out_seq=0 2024-04-23T14:53:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.554+0000 7f29d0c00640 10 client.26189 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.555+0000 7f29d0c00640 1 -- 172.21.15.83:0/2505142431 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558263757c10 con 0x558263810d00 2024-04-23T14:53:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.555+0000 7f29d0c00640 10 client.26189 handle_command_reply: tid=0 2024-04-23T14:53:02.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.555+0000 7f29f05b7740 1 client.26189 shutdown 2024-04-23T14:53:02.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.559+0000 7f29f05b7740 1 -- 172.21.15.83:0/2505142431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637f8060 msgr2=0x5582637f2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.559+0000 7f29f05b7740 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637f8060 0x5582637f2290 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x5582637e7190 tx=0x7f29d4013b70 comp rx=0 tx=0).stop 2024-04-23T14:53:02.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.560+0000 7f29f05b7740 1 -- 172.21.15.83:0/2505142431 shutdown_connections 2024-04-23T14:53:02.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.560+0000 7f29f05b7740 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558263810d00 0x558263813180 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.560+0000 7f29f05b7740 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637f8060 0x5582637f2290 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.560+0000 7f29d0c00640 0 client.26189 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:02.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.560+0000 7f29f05b7740 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637e4f60 0x5582637f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.560+0000 7f29f05b7740 1 --2- 172.21.15.83:0/2505142431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e45b0 0x5582637f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.560+0000 7f29f05b7740 1 -- 172.21.15.83:0/2505142431 >> 172.21.15.83:0/2505142431 conn(0x5582637e0b80 msgr2=0x5582637e69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:02.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.560+0000 7f29f05b7740 1 -- 172.21.15.83:0/2505142431 shutdown_connections 2024-04-23T14:53:02.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.560+0000 7f29f05b7740 1 -- 172.21.15.83:0/2505142431 wait complete. 2024-04-23T14:53:02.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.561+0000 7f29f05b7740 20 client.26189 trim_cache size 0 max 16384 2024-04-23T14:53:02.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:02 smithi173 ceph-mon[32443]: pgmap v140: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 54 MiB/s wr, 11.36k op/s 2024-04-23T14:53:02.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.598+0000 7f29f05b7740 1 Processor -- start 2024-04-23T14:53:02.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.599+0000 7f29f05b7740 1 -- start start 2024-04-23T14:53:02.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.599+0000 7f29f05b7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x5582637e6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.599+0000 7f29f05b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e4f60 0x558263818a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.599+0000 7f29f05b7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637f8060 0x558263818f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.599+0000 7f29f05b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29e81ae520 con 0x5582637e4f60 2024-04-23T14:53:02.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.599+0000 7f29f05b7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29e81abdf0 con 0x5582637e45b0 2024-04-23T14:53:02.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.599+0000 7f29f05b7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f29e81a9c40 con 0x5582637f8060 2024-04-23T14:53:02.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.600+0000 7f29e6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x5582637e6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.600+0000 7f29e6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x5582637e6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:45232/0 (socket says 172.21.15.83:45232) 2024-04-23T14:53:02.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.600+0000 7f29e6a00640 1 -- 172.21.15.83:0/925493606 learned_addr learned my addr 172.21.15.83:0/925493606 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:02.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.600+0000 7f29e7400640 1 --2- 172.21.15.83:0/925493606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637f8060 0x558263818f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.600+0000 7f29e6a00640 1 -- 172.21.15.83:0/925493606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637f8060 msgr2=0x558263818f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.600+0000 7f29e6a00640 1 --2- 172.21.15.83:0/925493606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637f8060 0x558263818f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.600+0000 7f29e6a00640 1 -- 172.21.15.83:0/925493606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e4f60 msgr2=0x558263818a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.600+0000 7f29e6a00640 1 --2- 172.21.15.83:0/925493606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e4f60 0x558263818a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.600+0000 7f29e6a00640 1 -- 172.21.15.83:0/925493606 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826345c3f0 con 0x5582637e45b0 2024-04-23T14:53:02.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.601+0000 7f29e6a00640 1 --2- 172.21.15.83:0/925493606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x5582637e6800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5582638139c0 tx=0x7f29e003d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffe51897b3d2bc8f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.601+0000 7f29d3400640 1 -- 172.21.15.83:0/925493606 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0031820 con 0x5582637e45b0 2024-04-23T14:53:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.601+0000 7f29d3400640 1 -- 172.21.15.83:0/925493606 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e0036050 con 0x5582637e45b0 2024-04-23T14:53:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.602+0000 7f29d3400640 1 -- 172.21.15.83:0/925493606 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0031820 con 0x5582637e45b0 2024-04-23T14:53:02.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.603+0000 7f29f05b7740 1 -- 172.21.15.83:0/925493606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 msgr2=0x5582637e6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.603+0000 7f29f05b7740 1 --2- 172.21.15.83:0/925493606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x5582637e6800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5582638139c0 tx=0x7f29e003d000 comp rx=0 tx=0).stop 2024-04-23T14:53:02.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.603+0000 7f29f05b7740 1 -- 172.21.15.83:0/925493606 shutdown_connections 2024-04-23T14:53:02.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.603+0000 7f29f05b7740 1 --2- 172.21.15.83:0/925493606 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637f8060 0x558263818f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.603+0000 7f29f05b7740 1 --2- 172.21.15.83:0/925493606 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637e4f60 0x558263818a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.603+0000 7f29f05b7740 1 --2- 172.21.15.83:0/925493606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x5582637e6800 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.603+0000 7f29f05b7740 1 -- 172.21.15.83:0/925493606 >> 172.21.15.83:0/925493606 conn(0x5582637e0b80 msgr2=0x5582637e60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:02.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.603+0000 7f29f05b7740 1 -- 172.21.15.83:0/925493606 shutdown_connections 2024-04-23T14:53:02.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.603+0000 7f29f05b7740 1 -- 172.21.15.83:0/925493606 wait complete. 2024-04-23T14:53:02.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.604+0000 7f29f05b7740 1 -- start start 2024-04-23T14:53:02.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.604+0000 7f29f05b7740 1 Processor -- start 2024-04-23T14:53:02.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.605+0000 7f29f05b7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x55826381f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.605+0000 7f29f05b7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637e4f60 0x55826381ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.605+0000 7f29f05b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637f8060 0x558263820440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.605+0000 7f29f05b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29e81aa2b0 con 0x5582637f8060 2024-04-23T14:53:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.605+0000 7f29f05b7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29e81ab720 con 0x5582637e45b0 2024-04-23T14:53:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.605+0000 7f29f05b7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f29e81aa980 con 0x5582637e4f60 2024-04-23T14:53:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.605+0000 7f29f05b7740 10 client.0 _ll_register_callbacks cb 0x7f29d4009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.605+0000 7f29e6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x55826381f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.605+0000 7f29e6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x55826381f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:45248/0 (socket says 172.21.15.83:45248) 2024-04-23T14:53:02.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.605+0000 7f29e6a00640 1 -- 172.21.15.83:0/3854775917 learned_addr learned my addr 172.21.15.83:0/3854775917 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:02.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.605+0000 7f29e6000640 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637e4f60 0x55826381ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.606+0000 7f29e7400640 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637f8060 0x558263820440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.606+0000 7f29e6a00640 1 -- 172.21.15.83:0/3854775917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637e4f60 msgr2=0x55826381ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.606+0000 7f29e6a00640 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637e4f60 0x55826381ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.606+0000 7f29e6a00640 1 -- 172.21.15.83:0/3854775917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637f8060 msgr2=0x558263820440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:02.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.606+0000 7f29e6a00640 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637f8060 0x558263820440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.606+0000 7f29e6a00640 1 -- 172.21.15.83:0/3854775917 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55826341c0f0 con 0x5582637e45b0 2024-04-23T14:53:02.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.606+0000 7f29e6000640 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637e4f60 0x55826381ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.606+0000 7f29e6a00640 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x55826381f9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f29e002a900 tx=0x7f29e00099f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa2b8dcb7a48ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.606+0000 7f29d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:53:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.607+0000 7f29d0c00640 1 -- 172.21.15.83:0/3854775917 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0004b40 con 0x5582637e45b0 2024-04-23T14:53:02.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.607+0000 7f29f05b7740 1 -- 172.21.15.83:0/3854775917 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29e0036050 con 0x5582637e45b0 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.607+0000 7f29d0c00640 1 -- 172.21.15.83:0/3854775917 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f29e0004ce0 con 0x5582637e45b0 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.607+0000 7f29d0c00640 1 -- 172.21.15.83:0/3854775917 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29e00042d0 con 0x5582637e45b0 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.607+0000 7f29f05b7740 10 client.26195 fetch_fsmap learned FSMap version 28 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.607+0000 7f29f05b7740 1 -- 172.21.15.83:0/3854775917 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55826370f810 con 0x5582637e45b0 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.609+0000 7f29d0c00640 1 -- 172.21.15.83:0/3854775917 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29e0036050 con 0x5582637e45b0 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.609+0000 7f29d0c00640 1 -- 172.21.15.83:0/3854775917 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f29e002f610 con 0x5582637e45b0 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.609+0000 7f29f05b7740 10 client.26195 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.609+0000 7f29f05b7740 10 client.26195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.609+0000 7f29f05b7740 20 client.26195 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.610+0000 7f29f05b7740 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558263877260 0x5582638796a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.610+0000 7f29f05b7740 4 client.26195 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.610+0000 7f29f05b7740 1 -- 172.21.15.83:0/3854775917 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558263751a80 con 0x558263877260 2024-04-23T14:53:02.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.611+0000 7f29e6000640 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558263877260 0x5582638796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.612+0000 7f29e6000640 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558263877260 0x5582638796a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc7ebdb057b6f50f server_cookie=93a84c6c4dc3ffde in_seq=0 out_seq=0 2024-04-23T14:53:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.612+0000 7f29d0c00640 10 client.26195 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.612+0000 7f29d0c00640 1 -- 172.21.15.83:0/3854775917 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x558263751a80 con 0x558263877260 2024-04-23T14:53:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.612+0000 7f29d0c00640 10 client.26195 handle_command_reply: tid=0 2024-04-23T14:53:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.612+0000 7f29f05b7740 1 client.26195 shutdown 2024-04-23T14:53:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.612+0000 7f29f05b7740 1 -- 172.21.15.83:0/3854775917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 msgr2=0x55826381f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.612+0000 7f29f05b7740 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x55826381f9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f29e002a900 tx=0x7f29e00099f0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.613+0000 7f29f05b7740 1 -- 172.21.15.83:0/3854775917 shutdown_connections 2024-04-23T14:53:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.613+0000 7f29f05b7740 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x558263877260 0x5582638796a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.613+0000 7f29f05b7740 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582637f8060 0x558263820440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.613+0000 7f29f05b7740 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5582637e4f60 0x55826381ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.613+0000 7f29f05b7740 1 --2- 172.21.15.83:0/3854775917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5582637e45b0 0x55826381f9c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.613+0000 7f29f05b7740 1 -- 172.21.15.83:0/3854775917 >> 172.21.15.83:0/3854775917 conn(0x5582637e0b80 msgr2=0x5582637e6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:02.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.613+0000 7f29d0c00640 0 client.26195 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:02.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.614+0000 7f29f05b7740 1 -- 172.21.15.83:0/3854775917 shutdown_connections 2024-04-23T14:53:02.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.614+0000 7f29f05b7740 1 -- 172.21.15.83:0/3854775917 wait complete. 2024-04-23T14:53:02.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.614+0000 7f29f05b7740 20 client.26195 trim_cache size 0 max 16384 2024-04-23T14:53:02.616 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:53:02.616 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "scrub active (0 inodes in the stack)", 2024-04-23T14:53:02.616 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:53:02.616 INFO:teuthology.orchestra.run.smithi083.stdout: "795da87d-4c0a-4542-b0c5-c25c5782569b": { 2024-04-23T14:53:02.616 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:53:02.616 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "795da87d-4c0a-4542-b0c5-c25c5782569b", 2024-04-23T14:53:02.616 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:53:02.616 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:53:02.616 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:53:02.617 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:53:02.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.615+0000 7f29ed800640 1 -- 172.21.15.83:0/2070836407 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f29cc0a3e20 msgr2=0x7f29cc0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.615+0000 7f29ed800640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f29cc0a3e20 0x7f29cc0a62e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f29e819baf0 tx=0x7f29e0000f40 comp rx=0 tx=0).stop 2024-04-23T14:53:02.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.615+0000 7f29ed800640 1 -- 172.21.15.83:0/2070836407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e8109580 msgr2=0x7f29e81a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:02.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.615+0000 7f29ed800640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e8109580 0x7f29e81a0d40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c940 tx=0x7f29d800ce00 comp rx=0 tx=0).stop 2024-04-23T14:53:02.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.616+0000 7f29ed800640 1 -- 172.21.15.83:0/2070836407 shutdown_connections 2024-04-23T14:53:02.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.616+0000 7f29ed800640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29e819a9b0 0x7f29e819ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.616+0000 7f29ed800640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e8109580 0x7f29e81a0d40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.616+0000 7f29ed800640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f29cc0a3e20 0x7f29cc0a62e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.616+0000 7f29ed800640 1 --2- 172.21.15.83:0/2070836407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29e81044b0 0x7f29e81a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:02.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.616+0000 7f29ed800640 1 -- 172.21.15.83:0/2070836407 >> 172.21.15.83:0/2070836407 conn(0x7f29e80ffd90 msgr2=0x7f29e81020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:02.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.617+0000 7f29ed800640 1 -- 172.21.15.83:0/2070836407 shutdown_connections 2024-04-23T14:53:02.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:02.617+0000 7f29ed800640 1 -- 172.21.15.83:0/2070836407 wait complete. 2024-04-23T14:53:02.647 INFO:tasks.cephfs.filesystem:scrub status for tag:795da87d-4c0a-4542-b0c5-c25c5782569b - {'path': '/', 'tag': '795da87d-4c0a-4542-b0c5-c25c5782569b', 'options': 'recursive,force'} 2024-04-23T14:53:02.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:02 smithi203 ceph-mon[32800]: pgmap v140: 97 pgs: 97 active+clean; 18 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 54 MiB/s wr, 11.36k op/s 2024-04-23T14:53:02.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T14:53:02.801 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-04-23T14:53:02.801 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-23T14:53:03.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:03 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:53:03.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:03 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:03.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:03 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:53:03.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:03 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:53:03.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:03 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:03.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:03 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:53:03.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:03 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:03.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:03 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:53:03.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:03 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:53:03.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:03 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:03 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:53:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:03 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:03 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:53:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:03 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:53:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:03 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:04.146 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi083 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d44fddd910b5e94da0f2357af4be37d7bd495ae0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2024-04-23T14:53:04.411 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=0fd6da3dc7c344d199b1a9e1c0a3971b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_d44fddd910b5e94da0f2357af4be37d7bd495ae0/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/ffsb.sh) on smithi083 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d44fddd910b5e94da0f2357af4be37d7bd495ae0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh' 2024-04-23T14:53:04.413 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-23T14:53:04.425 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-23T14:53:04.425 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T14:53:04.425 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi083.front.sepia.ceph.com 2024-04-23T14:53:04.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-23T14:53:04.503 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:04 smithi083 ceph-mon[27867]: pgmap v141: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 56 MiB/s wr, 11.53k op/s 2024-04-23T14:53:04.503 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:04 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:53:04.503 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:04 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:04.503 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:04 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:53:04.504 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:04 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:04.504 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:04 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:53:04.504 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:04 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:04.504 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:04 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:04.519 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-23T14:53:04.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:04 smithi173 ceph-mon[32443]: pgmap v141: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 56 MiB/s wr, 11.53k op/s 2024-04-23T14:53:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:04 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:53:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:04 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:04 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:53:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:04 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:04 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:53:04.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:04 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:04.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:04 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:04.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:04 smithi203 ceph-mon[32800]: pgmap v141: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 56 MiB/s wr, 11.53k op/s 2024-04-23T14:53:04.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:04 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:53:04.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:04 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:04.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:04 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:53:04.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:04 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:04.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:04 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:53:04.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:04 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:04.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:04 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:06.571 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:06 smithi083 ceph-mon[27867]: pgmap v142: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 53 MiB/s wr, 10.71k op/s 2024-04-23T14:53:06.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:06 smithi173 ceph-mon[32443]: pgmap v142: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 53 MiB/s wr, 10.71k op/s 2024-04-23T14:53:06.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:06 smithi203 ceph-mon[32800]: pgmap v142: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 53 MiB/s wr, 10.71k op/s 2024-04-23T14:53:08.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:08 smithi173 ceph-mon[32443]: pgmap v143: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 55 MiB/s wr, 11.09k op/s 2024-04-23T14:53:08.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:08 smithi203 ceph-mon[32800]: pgmap v143: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 55 MiB/s wr, 11.09k op/s 2024-04-23T14:53:08.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:08 smithi083 ceph-mon[27867]: pgmap v143: 97 pgs: 97 active+clean; 18 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 55 MiB/s wr, 11.09k op/s 2024-04-23T14:53:10.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:10 smithi173 ceph-mon[32443]: pgmap v144: 97 pgs: 97 active+clean; 18 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 53 MiB/s wr, 10.44k op/s 2024-04-23T14:53:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:10 smithi203 ceph-mon[32800]: pgmap v144: 97 pgs: 97 active+clean; 18 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 53 MiB/s wr, 10.44k op/s 2024-04-23T14:53:10.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:10 smithi083 ceph-mon[27867]: pgmap v144: 97 pgs: 97 active+clean; 18 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 53 MiB/s wr, 10.44k op/s 2024-04-23T14:53:12.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:12 smithi173 ceph-mon[32443]: pgmap v145: 97 pgs: 97 active+clean; 18 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 54 MiB/s wr, 10.48k op/s 2024-04-23T14:53:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:12 smithi203 ceph-mon[32800]: pgmap v145: 97 pgs: 97 active+clean; 18 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 54 MiB/s wr, 10.48k op/s 2024-04-23T14:53:12.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:12 smithi083 ceph-mon[27867]: pgmap v145: 97 pgs: 97 active+clean; 18 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 54 MiB/s wr, 10.48k op/s 2024-04-23T14:53:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:14 smithi083 ceph-mon[27867]: pgmap v146: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 43 MiB/s wr, 8.14k op/s 2024-04-23T14:53:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:14 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:53:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:14 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:53:14.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:14 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:53:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:14 smithi203 ceph-mon[32800]: pgmap v146: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 43 MiB/s wr, 8.14k op/s 2024-04-23T14:53:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:14 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:53:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:14 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:53:14.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:14 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:53:14.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:14 smithi173 ceph-mon[32443]: pgmap v146: 97 pgs: 97 active+clean; 19 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 43 MiB/s wr, 8.14k op/s 2024-04-23T14:53:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:14 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:53:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:14 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:53:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:14 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:53:16.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:16 smithi203 ceph-mon[32800]: pgmap v147: 97 pgs: 97 active+clean; 19 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 7.01k op/s 2024-04-23T14:53:16.760 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:16 smithi083 ceph-mon[27867]: pgmap v147: 97 pgs: 97 active+clean; 19 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 7.01k op/s 2024-04-23T14:53:16.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:16 smithi173 ceph-mon[32443]: pgmap v147: 97 pgs: 97 active+clean; 19 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 37 MiB/s wr, 7.01k op/s 2024-04-23T14:53:18.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:18 smithi203 ceph-mon[32800]: pgmap v148: 97 pgs: 97 active+clean; 19 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 57 MiB/s wr, 10.77k op/s 2024-04-23T14:53:18.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:18 smithi083 ceph-mon[27867]: pgmap v148: 97 pgs: 97 active+clean; 19 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 57 MiB/s wr, 10.77k op/s 2024-04-23T14:53:18.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:18 smithi173 ceph-mon[32443]: pgmap v148: 97 pgs: 97 active+clean; 19 GiB data, 26 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 57 MiB/s wr, 10.77k op/s 2024-04-23T14:53:20.627 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:20 smithi173 ceph-mon[32443]: pgmap v149: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 58 MiB/s wr, 10.96k op/s 2024-04-23T14:53:20.627 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:20 smithi173 ceph-mon[32443]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:53:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:20 smithi203 ceph-mon[32800]: pgmap v149: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 58 MiB/s wr, 10.96k op/s 2024-04-23T14:53:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:20 smithi203 ceph-mon[32800]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:53:20.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:20 smithi083 ceph-mon[27867]: pgmap v149: 97 pgs: 97 active+clean; 19 GiB data, 27 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 58 MiB/s wr, 10.96k op/s 2024-04-23T14:53:20.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:20 smithi083 ceph-mon[27867]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T14:53:22.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:22 smithi203 ceph-mon[32800]: pgmap v150: 97 pgs: 97 active+clean; 19 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 79 MiB/s wr, 15.19k op/s 2024-04-23T14:53:22.760 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:22 smithi083 ceph-mon[27867]: pgmap v150: 97 pgs: 97 active+clean; 19 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 79 MiB/s wr, 15.19k op/s 2024-04-23T14:53:22.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:22 smithi173 ceph-mon[32443]: pgmap v150: 97 pgs: 97 active+clean; 19 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 79 MiB/s wr, 15.19k op/s 2024-04-23T14:53:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:24 smithi203 ceph-mon[32800]: pgmap v151: 97 pgs: 97 active+clean; 19 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 81 MiB/s wr, 15.59k op/s 2024-04-23T14:53:24.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:24 smithi083 ceph-mon[27867]: pgmap v151: 97 pgs: 97 active+clean; 19 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 81 MiB/s wr, 15.59k op/s 2024-04-23T14:53:24.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:24 smithi173 ceph-mon[32443]: pgmap v151: 97 pgs: 97 active+clean; 19 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 81 MiB/s wr, 15.59k op/s 2024-04-23T14:53:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:26 smithi203 ceph-mon[32800]: pgmap v152: 97 pgs: 97 active+clean; 19 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 81 MiB/s wr, 15.54k op/s 2024-04-23T14:53:26.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:26 smithi083 ceph-mon[27867]: pgmap v152: 97 pgs: 97 active+clean; 19 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 81 MiB/s wr, 15.54k op/s 2024-04-23T14:53:26.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:26 smithi173 ceph-mon[32443]: pgmap v152: 97 pgs: 97 active+clean; 19 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 81 MiB/s wr, 15.54k op/s 2024-04-23T14:53:28.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:28 smithi203 ceph-mon[32800]: pgmap v153: 97 pgs: 97 active+clean; 20 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 103 MiB/s wr, 19.45k op/s 2024-04-23T14:53:28.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:28 smithi083 ceph-mon[27867]: pgmap v153: 97 pgs: 97 active+clean; 20 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 103 MiB/s wr, 19.45k op/s 2024-04-23T14:53:28.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:28 smithi173 ceph-mon[32443]: pgmap v153: 97 pgs: 97 active+clean; 20 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 103 MiB/s wr, 19.45k op/s 2024-04-23T14:53:30.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:30 smithi203 ceph-mon[32800]: pgmap v154: 97 pgs: 97 active+clean; 20 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 85 MiB/s wr, 15.98k op/s 2024-04-23T14:53:30.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:30 smithi083 ceph-mon[27867]: pgmap v154: 97 pgs: 97 active+clean; 20 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 85 MiB/s wr, 15.98k op/s 2024-04-23T14:53:30.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:30 smithi173 ceph-mon[32443]: pgmap v154: 97 pgs: 97 active+clean; 20 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 85 MiB/s wr, 15.98k op/s 2024-04-23T14:53:32.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:53:32.648 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:53:32.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:32 smithi203 ceph-mon[32800]: pgmap v155: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 94 MiB/s wr, 17.63k op/s 2024-04-23T14:53:32.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:32 smithi083 ceph-mon[27867]: pgmap v155: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 94 MiB/s wr, 17.63k op/s 2024-04-23T14:53:32.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:32 smithi173 ceph-mon[32443]: pgmap v155: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 94 MiB/s wr, 17.63k op/s 2024-04-23T14:53:34.694 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:34 smithi083 ceph-mon[27867]: pgmap v156: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 MiB/s wr, 12.36k op/s 2024-04-23T14:53:34.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:34 smithi203 ceph-mon[32800]: pgmap v156: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 MiB/s wr, 12.36k op/s 2024-04-23T14:53:34.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:34 smithi173 ceph-mon[32443]: pgmap v156: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 MiB/s wr, 12.36k op/s 2024-04-23T14:53:35.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.813+0000 7f8971600640 1 -- 172.21.15.83:0/1093449125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c10d500 msgr2=0x7f896c10d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:35.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.813+0000 7f8971600640 1 --2- 172.21.15.83:0/1093449125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c10d500 0x7f896c10d8e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f895c009900 tx=0x7f895c02a410 comp rx=0 tx=0).stop 2024-04-23T14:53:35.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.813+0000 7f8971600640 1 -- 172.21.15.83:0/1093449125 shutdown_connections 2024-04-23T14:53:35.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.813+0000 7f8971600640 1 --2- 172.21.15.83:0/1093449125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f896c108f40 0x7f896c1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.813+0000 7f8971600640 1 --2- 172.21.15.83:0/1093449125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f896c108580 0x7f896c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.813+0000 7f8971600640 1 --2- 172.21.15.83:0/1093449125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c10d500 0x7f896c10d8e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.813+0000 7f8971600640 1 -- 172.21.15.83:0/1093449125 >> 172.21.15.83:0/1093449125 conn(0x7f896c077e20 msgr2=0x7f896c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:35.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.813+0000 7f8971600640 1 -- 172.21.15.83:0/1093449125 shutdown_connections 2024-04-23T14:53:35.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.813+0000 7f8971600640 1 -- 172.21.15.83:0/1093449125 wait complete. 2024-04-23T14:53:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.814+0000 7f8971600640 1 Processor -- start 2024-04-23T14:53:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.815+0000 7f8971600640 1 -- start start 2024-04-23T14:53:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.815+0000 7f8971600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f896c108580 0x7f896c1a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.815+0000 7f8971600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f896c108f40 0x7f896c1a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.815+0000 7f8971600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c10d500 0x7f896c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.815+0000 7f8971600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1188c0 con 0x7f896c108580 2024-04-23T14:53:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.815+0000 7f8971600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f896c118740 con 0x7f896c10d500 2024-04-23T14:53:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.815+0000 7f8971600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f896c118a40 con 0x7f896c108f40 2024-04-23T14:53:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.816+0000 7f896be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f896c108580 0x7f896c1a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.816+0000 7f896be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f896c108580 0x7f896c1a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51792/0 (socket says 172.21.15.83:51792) 2024-04-23T14:53:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.816+0000 7f896b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f896c108f40 0x7f896c1a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.816+0000 7f896be00640 1 -- 172.21.15.83:0/3530241583 learned_addr learned my addr 172.21.15.83:0/3530241583 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:35.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.816+0000 7f8970c00640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c10d500 0x7f896c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.816+0000 7f896b400640 1 -- 172.21.15.83:0/3530241583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c10d500 msgr2=0x7f896c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:35.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.816+0000 7f896b400640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c10d500 0x7f896c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.816+0000 7f896b400640 1 -- 172.21.15.83:0/3530241583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f896c108580 msgr2=0x7f896c1a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:35.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.816+0000 7f896b400640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f896c108580 0x7f896c1a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.816+0000 7f896b400640 1 -- 172.21.15.83:0/3530241583 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896c19f3e0 con 0x7f896c108f40 2024-04-23T14:53:35.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.816+0000 7f896be00640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f896c108580 0x7f896c1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:35.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.817+0000 7f896b400640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f896c108f40 0x7f896c1a4fa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f895800e9f0 tx=0x7f895800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:35.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.817+0000 7f8968c00640 1 -- 172.21.15.83:0/3530241583 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895800ccc0 con 0x7f896c108f40 2024-04-23T14:53:35.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.817+0000 7f8968c00640 1 -- 172.21.15.83:0/3530241583 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8958010910 con 0x7f896c108f40 2024-04-23T14:53:35.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.817+0000 7f8968c00640 1 -- 172.21.15.83:0/3530241583 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958018960 con 0x7f896c108f40 2024-04-23T14:53:35.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.817+0000 7f8971600640 1 -- 172.21.15.83:0/3530241583 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896c19f6d0 con 0x7f896c108f40 2024-04-23T14:53:35.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.817+0000 7f8971600640 1 -- 172.21.15.83:0/3530241583 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896c1ab9e0 con 0x7f896c108f40 2024-04-23T14:53:35.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.821+0000 7f89759ba740 1 Processor -- start 2024-04-23T14:53:35.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.821+0000 7f89759ba740 1 -- start start 2024-04-23T14:53:35.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.821+0000 7f8968c00640 1 -- 172.21.15.83:0/3530241583 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8958010ac0 con 0x7f896c108f40 2024-04-23T14:53:35.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.821+0000 7f89759ba740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eaca30 0x562850eace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.821+0000 7f89759ba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850ead350 0x562850eb74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.821+0000 7f89759ba740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eb7a20 0x562850eb9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.821+0000 7f89759ba740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562850e121f0 con 0x562850eb7a20 2024-04-23T14:53:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.821+0000 7f89759ba740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562850db8350 con 0x562850eaca30 2024-04-23T14:53:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.821+0000 7f89759ba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562850e0e1b0 con 0x562850ead350 2024-04-23T14:53:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.821+0000 7f8970c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eaca30 0x562850eace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.821+0000 7f896b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eb7a20 0x562850eb9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.822+0000 7f8970c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eaca30 0x562850eace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37016/0 (socket says 172.21.15.83:37016) 2024-04-23T14:53:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.822+0000 7f8970c00640 1 -- 172.21.15.83:0/624593783 learned_addr learned my addr 172.21.15.83:0/624593783 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.822+0000 7f896be00640 1 --2- 172.21.15.83:0/624593783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850ead350 0x562850eb74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.822+0000 7f8970c00640 1 -- 172.21.15.83:0/624593783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850ead350 msgr2=0x562850eb74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.822+0000 7f8970c00640 1 --2- 172.21.15.83:0/624593783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850ead350 0x562850eb74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.822+0000 7f8970c00640 1 -- 172.21.15.83:0/624593783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eb7a20 msgr2=0x562850eb9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.822+0000 7f8970c00640 1 --2- 172.21.15.83:0/624593783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eb7a20 0x562850eb9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.822+0000 7f8970c00640 1 -- 172.21.15.83:0/624593783 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562850e0cf50 con 0x562850eaca30 2024-04-23T14:53:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.822+0000 7f8970c00640 1 --2- 172.21.15.83:0/624593783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eaca30 0x562850eace10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8964009fc0 tx=0x7f89640118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fcc5c7fc469a5dd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.823+0000 7f8961200640 1 -- 172.21.15.83:0/624593783 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964004d20 con 0x562850eaca30 2024-04-23T14:53:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.823+0000 7f8961200640 1 -- 172.21.15.83:0/624593783 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8964004050 con 0x562850eaca30 2024-04-23T14:53:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.823+0000 7f8961200640 1 -- 172.21.15.83:0/624593783 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964005280 con 0x562850eaca30 2024-04-23T14:53:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.823+0000 7f8968c00640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f89500a3da0 0x7f89500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.823+0000 7f8968c00640 1 -- 172.21.15.83:0/3530241583 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f89580d0b30 con 0x7f896c108f40 2024-04-23T14:53:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.823+0000 7f896be00640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f89500a3da0 0x7f89500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.824+0000 7f89759ba740 1 -- 172.21.15.83:0/624593783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eaca30 msgr2=0x562850eace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.824+0000 7f89759ba740 1 --2- 172.21.15.83:0/624593783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eaca30 0x562850eace10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8964009fc0 tx=0x7f89640118d0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.824+0000 7f89759ba740 1 -- 172.21.15.83:0/624593783 shutdown_connections 2024-04-23T14:53:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.824+0000 7f89759ba740 1 --2- 172.21.15.83:0/624593783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eb7a20 0x562850eb9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.824+0000 7f89759ba740 1 --2- 172.21.15.83:0/624593783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850ead350 0x562850eb74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.824+0000 7f89759ba740 1 --2- 172.21.15.83:0/624593783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eaca30 0x562850eace10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.824+0000 7f89759ba740 1 -- 172.21.15.83:0/624593783 >> 172.21.15.83:0/624593783 conn(0x562850ea8d50 msgr2=0x562850ea9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.824+0000 7f89759ba740 1 -- 172.21.15.83:0/624593783 shutdown_connections 2024-04-23T14:53:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.824+0000 7f89759ba740 1 -- 172.21.15.83:0/624593783 wait complete. 2024-04-23T14:53:35.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.824+0000 7f896be00640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f89500a3da0 0x7f89500a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x562850eadff0 tx=0x7f895c0058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.825+0000 7f89759ba740 1 -- start start 2024-04-23T14:53:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.825+0000 7f89759ba740 1 Processor -- start 2024-04-23T14:53:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.825+0000 7f89759ba740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 0x562850eb4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.825+0000 7f89759ba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850ead350 0x562850eb4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.825+0000 7f89759ba740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eb7a20 0x562850eb53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.825+0000 7f89759ba740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562850eba810 con 0x562850eaca30 2024-04-23T14:53:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.825+0000 7f89759ba740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562850dcb690 con 0x562850eb7a20 2024-04-23T14:53:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.825+0000 7f89759ba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562850eba990 con 0x562850ead350 2024-04-23T14:53:35.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.825+0000 7f89759ba740 10 client.0 _ll_register_callbacks cb 0x562850e07650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.826+0000 7f896be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 0x562850eb4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.826+0000 7f896b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850ead350 0x562850eb4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.826+0000 7f896be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 0x562850eb4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51808/0 (socket says 172.21.15.83:51808) 2024-04-23T14:53:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.826+0000 7f8970c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eb7a20 0x562850eb53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.826+0000 7f896be00640 1 -- 172.21.15.83:0/1861230646 learned_addr learned my addr 172.21.15.83:0/1861230646 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.826+0000 7f8970c00640 1 -- 172.21.15.83:0/1861230646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850ead350 msgr2=0x562850eb4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.826+0000 7f8970c00640 1 --2- 172.21.15.83:0/1861230646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850ead350 0x562850eb4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.826+0000 7f8970c00640 1 -- 172.21.15.83:0/1861230646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 msgr2=0x562850eb4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.826+0000 7f8970c00640 1 --2- 172.21.15.83:0/1861230646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 0x562850eb4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.826+0000 7f8970c00640 1 -- 172.21.15.83:0/1861230646 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562850dca920 con 0x562850eb7a20 2024-04-23T14:53:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.826+0000 7f8970c00640 1 --2- 172.21.15.83:0/1861230646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eb7a20 0x562850eb53e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f896401b040 tx=0x7f89640048b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6957143882867621 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.826+0000 7f8955600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:53:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.827+0000 7f8955600640 1 -- 172.21.15.83:0/1861230646 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964029020 con 0x562850eb7a20 2024-04-23T14:53:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.827+0000 7f89759ba740 1 -- 172.21.15.83:0/1861230646 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8964004050 con 0x562850eb7a20 2024-04-23T14:53:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.827+0000 7f8955600640 1 -- 172.21.15.83:0/1861230646 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f8964003c30 con 0x562850eb7a20 2024-04-23T14:53:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.827+0000 7f8955600640 1 -- 172.21.15.83:0/1861230646 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8964034050 con 0x562850eb7a20 2024-04-23T14:53:35.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.827+0000 7f89759ba740 10 client.26207 fetch_fsmap learned FSMap version 29 2024-04-23T14:53:35.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.827+0000 7f89759ba740 1 -- 172.21.15.83:0/1861230646 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562850bee7d0 con 0x562850eb7a20 2024-04-23T14:53:35.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.828+0000 7f8955600640 1 -- 172.21.15.83:0/1861230646 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8964004050 con 0x562850eb7a20 2024-04-23T14:53:35.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.828+0000 7f8955600640 1 -- 172.21.15.83:0/1861230646 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f8964025470 con 0x562850eb7a20 2024-04-23T14:53:35.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.828+0000 7f89759ba740 10 client.26207 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:53:35.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.828+0000 7f89759ba740 10 client.26207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:53:35.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.828+0000 7f89759ba740 20 client.26207 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:35.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.828+0000 7f89759ba740 1 --2- 172.21.15.83:0/1861230646 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562850ec7cf0 0x562850eca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.828+0000 7f89759ba740 4 client.26207 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:53:35.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.828+0000 7f89759ba740 1 -- 172.21.15.83:0/1861230646 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562850e1fc10 con 0x562850ec7cf0 2024-04-23T14:53:35.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.829+0000 7f896be00640 1 --2- 172.21.15.83:0/1861230646 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562850ec7cf0 0x562850eca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.830+0000 7f896be00640 1 --2- 172.21.15.83:0/1861230646 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562850ec7cf0 0x562850eca0e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f81041a2064caf6a server_cookie=90a62641d59d34ff in_seq=0 out_seq=0 2024-04-23T14:53:35.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.830+0000 7f8955600640 10 client.26207 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f8955600640 1 -- 172.21.15.83:0/1861230646 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562850e1fc10 con 0x562850ec7cf0 2024-04-23T14:53:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f8955600640 10 client.26207 handle_command_reply: tid=0 2024-04-23T14:53:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f89759ba740 1 client.26207 shutdown 2024-04-23T14:53:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f89759ba740 1 -- 172.21.15.83:0/1861230646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eb7a20 msgr2=0x562850eb53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f89759ba740 1 --2- 172.21.15.83:0/1861230646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eb7a20 0x562850eb53e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f896401b040 tx=0x7f89640048b0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f89759ba740 1 -- 172.21.15.83:0/1861230646 shutdown_connections 2024-04-23T14:53:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f89759ba740 1 --2- 172.21.15.83:0/1861230646 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562850ec7cf0 0x562850eca0e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f89759ba740 1 --2- 172.21.15.83:0/1861230646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850eb7a20 0x562850eb53e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f89759ba740 1 --2- 172.21.15.83:0/1861230646 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850ead350 0x562850eb4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f89759ba740 1 --2- 172.21.15.83:0/1861230646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 0x562850eb4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f8955600640 0 client.26207 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:35.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f89759ba740 1 -- 172.21.15.83:0/1861230646 >> 172.21.15.83:0/1861230646 conn(0x562850ea8d50 msgr2=0x562850eaeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:35.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f89759ba740 1 -- 172.21.15.83:0/1861230646 shutdown_connections 2024-04-23T14:53:35.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f89759ba740 1 -- 172.21.15.83:0/1861230646 wait complete. 2024-04-23T14:53:35.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.831+0000 7f89759ba740 20 client.26207 trim_cache size 0 max 16384 2024-04-23T14:53:35.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.851+0000 7f89759ba740 1 Processor -- start 2024-04-23T14:53:35.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.851+0000 7f89759ba740 1 -- start start 2024-04-23T14:53:35.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.851+0000 7f89759ba740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 0x562850ec36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.851+0000 7f89759ba740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850ead350 0x562850eb5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.851+0000 7f89759ba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850eb7a20 0x562850ecf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.851+0000 7f89759ba740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1b69c0 con 0x562850eaca30 2024-04-23T14:53:35.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.851+0000 7f89759ba740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1b4290 con 0x562850ead350 2024-04-23T14:53:35.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.851+0000 7f89759ba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1b20e0 con 0x562850eb7a20 2024-04-23T14:53:35.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.852+0000 7f896be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 0x562850ec36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.852+0000 7f896be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 0x562850ec36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51816/0 (socket says 172.21.15.83:51816) 2024-04-23T14:53:35.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.852+0000 7f896be00640 1 -- 172.21.15.83:0/894358011 learned_addr learned my addr 172.21.15.83:0/894358011 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:35.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.852+0000 7f8970c00640 1 --2- 172.21.15.83:0/894358011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850eb7a20 0x562850ecf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.852+0000 7f896b400640 1 --2- 172.21.15.83:0/894358011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850ead350 0x562850eb5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.852+0000 7f896be00640 1 -- 172.21.15.83:0/894358011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850eb7a20 msgr2=0x562850ecf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:35.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.852+0000 7f896be00640 1 --2- 172.21.15.83:0/894358011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850eb7a20 0x562850ecf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.852+0000 7f896be00640 1 -- 172.21.15.83:0/894358011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850ead350 msgr2=0x562850eb5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:35.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.852+0000 7f896be00640 1 --2- 172.21.15.83:0/894358011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850ead350 0x562850eb5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.852+0000 7f896be00640 1 -- 172.21.15.83:0/894358011 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562850ead180 con 0x562850eaca30 2024-04-23T14:53:35.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.852+0000 7f896be00640 1 --2- 172.21.15.83:0/894358011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 0x562850ec36d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f895c009230 tx=0x7f895c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d9c4f3cabf99e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:35.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.853+0000 7f8957e00640 1 -- 172.21.15.83:0/894358011 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c05c070 con 0x562850eaca30 2024-04-23T14:53:35.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.853+0000 7f8957e00640 1 -- 172.21.15.83:0/894358011 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c0565b0 con 0x562850eaca30 2024-04-23T14:53:35.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.853+0000 7f8957e00640 1 -- 172.21.15.83:0/894358011 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c035050 con 0x562850eaca30 2024-04-23T14:53:35.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.853+0000 7f89759ba740 1 -- 172.21.15.83:0/894358011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 msgr2=0x562850ec36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:35.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.853+0000 7f89759ba740 1 --2- 172.21.15.83:0/894358011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 0x562850ec36d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f895c009230 tx=0x7f895c041000 comp rx=0 tx=0).stop 2024-04-23T14:53:35.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.854+0000 7f89759ba740 1 -- 172.21.15.83:0/894358011 shutdown_connections 2024-04-23T14:53:35.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.854+0000 7f89759ba740 1 --2- 172.21.15.83:0/894358011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850eb7a20 0x562850ecf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.854+0000 7f89759ba740 1 --2- 172.21.15.83:0/894358011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850ead350 0x562850eb5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.854+0000 7f89759ba740 1 --2- 172.21.15.83:0/894358011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eaca30 0x562850ec36d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.854+0000 7f89759ba740 1 -- 172.21.15.83:0/894358011 >> 172.21.15.83:0/894358011 conn(0x562850ea8d50 msgr2=0x562850eb4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:35.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.854+0000 7f89759ba740 1 -- 172.21.15.83:0/894358011 shutdown_connections 2024-04-23T14:53:35.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.854+0000 7f89759ba740 1 -- 172.21.15.83:0/894358011 wait complete. 2024-04-23T14:53:35.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.854+0000 7f89759ba740 1 -- start start 2024-04-23T14:53:35.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.854+0000 7f89759ba740 1 Processor -- start 2024-04-23T14:53:35.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.855+0000 7f89759ba740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850eaca30 0x562850ebff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.855+0000 7f89759ba740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850ead350 0x562850ebb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.855+0000 7f89759ba740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eb7a20 0x562850ebb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.855+0000 7f89759ba740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1b2750 con 0x562850eb7a20 2024-04-23T14:53:35.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.855+0000 7f89759ba740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1b3bc0 con 0x562850ead350 2024-04-23T14:53:35.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.855+0000 7f89759ba740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f896c1b2e20 con 0x562850eaca30 2024-04-23T14:53:35.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.855+0000 7f89759ba740 10 client.0 _ll_register_callbacks cb 0x7f89580d44b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:35.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.855+0000 7f896be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850eaca30 0x562850ebff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.855+0000 7f896b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850ead350 0x562850ebb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.855+0000 7f896be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850eaca30 0x562850ebff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55186/0 (socket says 172.21.15.83:55186) 2024-04-23T14:53:35.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.855+0000 7f896be00640 1 -- 172.21.15.83:0/966883210 learned_addr learned my addr 172.21.15.83:0/966883210 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:35.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.855+0000 7f8970c00640 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eb7a20 0x562850ebb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f896b400640 1 -- 172.21.15.83:0/966883210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850eaca30 msgr2=0x562850ebff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:35.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f896b400640 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850eaca30 0x562850ebff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f896b400640 1 -- 172.21.15.83:0/966883210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eb7a20 msgr2=0x562850ebb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:35.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f896b400640 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eb7a20 0x562850ebb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f896b400640 1 -- 172.21.15.83:0/966883210 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562850b243f0 con 0x562850ead350 2024-04-23T14:53:35.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f896be00640 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850eaca30 0x562850ebff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:35.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f8970c00640 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eb7a20 0x562850ebb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:53:35.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f896b400640 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850ead350 0x562850ebb270 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89580ce230 tx=0x7f8958009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4bb3f8e8faa17dc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:35.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f8955600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:53:35.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f8955600640 1 -- 172.21.15.83:0/966883210 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89580ce6c0 con 0x562850ead350 2024-04-23T14:53:35.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f8955600640 1 -- 172.21.15.83:0/966883210 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f895809c180 con 0x562850ead350 2024-04-23T14:53:35.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f89759ba740 1 -- 172.21.15.83:0/966883210 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f895c0565b0 con 0x562850ead350 2024-04-23T14:53:35.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f8955600640 1 -- 172.21.15.83:0/966883210 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f895809cb30 con 0x562850ead350 2024-04-23T14:53:35.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f89759ba740 10 client.26213 fetch_fsmap learned FSMap version 29 2024-04-23T14:53:35.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.856+0000 7f89759ba740 1 -- 172.21.15.83:0/966883210 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562850ae40f0 con 0x562850ead350 2024-04-23T14:53:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.857+0000 7f8955600640 1 -- 172.21.15.83:0/966883210 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c0565b0 con 0x562850ead350 2024-04-23T14:53:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.857+0000 7f8955600640 1 -- 172.21.15.83:0/966883210 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f8958098880 con 0x562850ead350 2024-04-23T14:53:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.857+0000 7f89759ba740 10 client.26213 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:53:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.857+0000 7f89759ba740 10 client.26213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:53:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.857+0000 7f89759ba740 20 client.26213 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.857+0000 7f89759ba740 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562850f2e510 0x562850f30930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.857+0000 7f89759ba740 4 client.26213 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:53:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.857+0000 7f89759ba740 1 -- 172.21.15.83:0/966883210 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562850e19a80 con 0x562850f2e510 2024-04-23T14:53:35.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.857+0000 7f896be00640 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562850f2e510 0x562850f30930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:35.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f896be00640 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562850f2e510 0x562850f30930 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29c4352379e934e7 server_cookie=50191118fb0014fc in_seq=0 out_seq=0 2024-04-23T14:53:35.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f8955600640 10 client.26213 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:35.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f8955600640 1 -- 172.21.15.83:0/966883210 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562850e19a80 con 0x562850f2e510 2024-04-23T14:53:35.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f8955600640 10 client.26213 handle_command_reply: tid=0 2024-04-23T14:53:35.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f89759ba740 1 client.26213 shutdown 2024-04-23T14:53:35.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f89759ba740 1 -- 172.21.15.83:0/966883210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850ead350 msgr2=0x562850ebb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:35.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f89759ba740 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850ead350 0x562850ebb270 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89580ce230 tx=0x7f8958009700 comp rx=0 tx=0).stop 2024-04-23T14:53:35.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f89759ba740 1 -- 172.21.15.83:0/966883210 shutdown_connections 2024-04-23T14:53:35.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f89759ba740 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562850f2e510 0x562850f30930 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f89759ba740 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562850eb7a20 0x562850ebb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f89759ba740 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562850ead350 0x562850ebb270 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f89759ba740 1 --2- 172.21.15.83:0/966883210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562850eaca30 0x562850ebff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f8955600640 0 client.26213 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:35.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f89759ba740 1 -- 172.21.15.83:0/966883210 >> 172.21.15.83:0/966883210 conn(0x562850ea8d50 msgr2=0x562850eaeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:35.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f89759ba740 1 -- 172.21.15.83:0/966883210 shutdown_connections 2024-04-23T14:53:35.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f89759ba740 1 -- 172.21.15.83:0/966883210 wait complete. 2024-04-23T14:53:35.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.858+0000 7f89759ba740 20 client.26213 trim_cache size 0 max 16384 2024-04-23T14:53:35.867 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:53:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:53:35.867 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:53:35.867 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:53:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.859+0000 7f8971600640 1 -- 172.21.15.83:0/3530241583 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f89500a3da0 msgr2=0x7f89500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.859+0000 7f8971600640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f89500a3da0 0x7f89500a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x562850eadff0 tx=0x7f895c0058d0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.859+0000 7f8971600640 1 -- 172.21.15.83:0/3530241583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f896c108f40 msgr2=0x7f896c1a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:35.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.859+0000 7f8971600640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f896c108f40 0x7f896c1a4fa0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f895800e9f0 tx=0x7f895800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.859+0000 7f8971600640 1 -- 172.21.15.83:0/3530241583 shutdown_connections 2024-04-23T14:53:35.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.859+0000 7f8971600640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f896c10d500 0x7f896c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.859+0000 7f8971600640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f896c108f40 0x7f896c1a4fa0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.859+0000 7f8971600640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f89500a3da0 0x7f89500a6260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.859+0000 7f8971600640 1 --2- 172.21.15.83:0/3530241583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f896c108580 0x7f896c1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:35.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.859+0000 7f8971600640 1 -- 172.21.15.83:0/3530241583 >> 172.21.15.83:0/3530241583 conn(0x7f896c077e20 msgr2=0x7f896c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:35.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.859+0000 7f8971600640 1 -- 172.21.15.83:0/3530241583 shutdown_connections 2024-04-23T14:53:35.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:35.859+0000 7f8971600640 1 -- 172.21.15.83:0/3530241583 wait complete. 2024-04-23T14:53:35.877 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:53:35.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:53:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.067+0000 7f5270a00640 1 -- 172.21.15.83:0/1161096304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f526c102090 msgr2=0x7f526c102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.067+0000 7f5270a00640 1 --2- 172.21.15.83:0/1161096304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f526c102090 0x7f526c102470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f525c009900 tx=0x7f525c02a3f0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.067+0000 7f5270a00640 1 -- 172.21.15.83:0/1161096304 shutdown_connections 2024-04-23T14:53:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.067+0000 7f5270a00640 1 --2- 172.21.15.83:0/1161096304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f526c107160 0x7f526c10ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.067+0000 7f5270a00640 1 --2- 172.21.15.83:0/1161096304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c102a40 0x7f526c106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.067+0000 7f5270a00640 1 --2- 172.21.15.83:0/1161096304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f526c102090 0x7f526c102470 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.067+0000 7f5270a00640 1 -- 172.21.15.83:0/1161096304 >> 172.21.15.83:0/1161096304 conn(0x7f526c0fd930 msgr2=0x7f526c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.067+0000 7f5270a00640 1 -- 172.21.15.83:0/1161096304 shutdown_connections 2024-04-23T14:53:36.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.067+0000 7f5270a00640 1 -- 172.21.15.83:0/1161096304 wait complete. 2024-04-23T14:53:36.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.068+0000 7f5270a00640 1 Processor -- start 2024-04-23T14:53:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.068+0000 7f5270a00640 1 -- start start 2024-04-23T14:53:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.069+0000 7f5270a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f526c102090 0x7f526c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.069+0000 7f5270a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c107160 0x7f526c1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.069+0000 7f5270a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f526c19a7f0 0x7f526c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.069+0000 7f5270a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f526c112270 con 0x7f526c107160 2024-04-23T14:53:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.069+0000 7f5270a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1120f0 con 0x7f526c102090 2024-04-23T14:53:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.069+0000 7f5270a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1123f0 con 0x7f526c19a7f0 2024-04-23T14:53:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.069+0000 7f526aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c107160 0x7f526c1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.069+0000 7f526b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f526c102090 0x7f526c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.069+0000 7f526aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c107160 0x7f526c1a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51840/0 (socket says 172.21.15.83:51840) 2024-04-23T14:53:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.069+0000 7f526aa00640 1 -- 172.21.15.83:0/1275916660 learned_addr learned my addr 172.21.15.83:0/1275916660 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.069+0000 7f526b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f526c102090 0x7f526c1a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37084/0 (socket says 172.21.15.83:37084) 2024-04-23T14:53:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.069+0000 7f526be00640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f526c19a7f0 0x7f526c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.070+0000 7f526aa00640 1 -- 172.21.15.83:0/1275916660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f526c19a7f0 msgr2=0x7f526c19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.070+0000 7f526aa00640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f526c19a7f0 0x7f526c19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.070+0000 7f526aa00640 1 -- 172.21.15.83:0/1275916660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f526c102090 msgr2=0x7f526c1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.070+0000 7f526aa00640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f526c102090 0x7f526c1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.070+0000 7f526aa00640 1 -- 172.21.15.83:0/1275916660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c19b380 con 0x7f526c107160 2024-04-23T14:53:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.070+0000 7f526aa00640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c107160 0x7f526c1a0b20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f525800e9a0 tx=0x7f525800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.070+0000 7f5263e00640 1 -- 172.21.15.83:0/1275916660 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525800cd80 con 0x7f526c107160 2024-04-23T14:53:36.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.070+0000 7f5263e00640 1 -- 172.21.15.83:0/1275916660 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52580035f0 con 0x7f526c107160 2024-04-23T14:53:36.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.070+0000 7f5270a00640 1 -- 172.21.15.83:0/1275916660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526c19b670 con 0x7f526c107160 2024-04-23T14:53:36.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.070+0000 7f5263e00640 1 -- 172.21.15.83:0/1275916660 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258010670 con 0x7f526c107160 2024-04-23T14:53:36.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.071+0000 7f5270a00640 1 -- 172.21.15.83:0/1275916660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526c1a7800 con 0x7f526c107160 2024-04-23T14:53:36.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.072+0000 7f5270a00640 1 -- 172.21.15.83:0/1275916660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526c103560 con 0x7f526c107160 2024-04-23T14:53:36.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.075+0000 7f5263e00640 1 -- 172.21.15.83:0/1275916660 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f52580037a0 con 0x7f526c107160 2024-04-23T14:53:36.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.076+0000 7f5263e00640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f52500a3da0 0x7f52500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.076+0000 7f5263e00640 1 -- 172.21.15.83:0/1275916660 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f52580d0a30 con 0x7f526c107160 2024-04-23T14:53:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.076+0000 7f526b400640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f52500a3da0 0x7f52500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.077+0000 7f526b400640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f52500a3da0 0x7f52500a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f525c000c00 tx=0x7f525c005b50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.081+0000 7f5263e00640 1 -- 172.21.15.83:0/1275916660 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f526c19b670 con 0x7f526c107160 2024-04-23T14:53:36.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.328+0000 7f5270a00640 1 -- 172.21.15.83:0/1275916660 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f526c102b10 con 0x7f526c107160 2024-04-23T14:53:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.329+0000 7f5263e00640 1 -- 172.21.15.83:0/1275916660 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7f525809d080 con 0x7f526c107160 2024-04-23T14:53:36.330 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:53:36.330 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:53:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:53:36.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.332+0000 7f5270a00640 1 -- 172.21.15.83:0/1275916660 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f52500a3da0 msgr2=0x7f52500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.332+0000 7f5270a00640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f52500a3da0 0x7f52500a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f525c000c00 tx=0x7f525c005b50 comp rx=0 tx=0).stop 2024-04-23T14:53:36.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.332+0000 7f5270a00640 1 -- 172.21.15.83:0/1275916660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c107160 msgr2=0x7f526c1a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.332+0000 7f5270a00640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c107160 0x7f526c1a0b20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f525800e9a0 tx=0x7f525800ee60 comp rx=0 tx=0).stop 2024-04-23T14:53:36.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.332+0000 7f5270a00640 1 -- 172.21.15.83:0/1275916660 shutdown_connections 2024-04-23T14:53:36.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.332+0000 7f5270a00640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f526c19a7f0 0x7f526c19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.332+0000 7f5270a00640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f52500a3da0 0x7f52500a6260 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.332+0000 7f5270a00640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f526c107160 0x7f526c1a0b20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.332+0000 7f5270a00640 1 --2- 172.21.15.83:0/1275916660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f526c102090 0x7f526c1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.332+0000 7f5270a00640 1 -- 172.21.15.83:0/1275916660 >> 172.21.15.83:0/1275916660 conn(0x7f526c0fd930 msgr2=0x7f526c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.333+0000 7f5270a00640 1 -- 172.21.15.83:0/1275916660 shutdown_connections 2024-04-23T14:53:36.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.333+0000 7f5270a00640 1 -- 172.21.15.83:0/1275916660 wait complete. 2024-04-23T14:53:36.354 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:53:36.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.551+0000 7f0f09600640 1 -- 172.21.15.83:0/3833507812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f0410c4d0 msgr2=0x7f0f0410e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.551+0000 7f0f09600640 1 --2- 172.21.15.83:0/3833507812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f0410c4d0 0x7f0f0410e920 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009ca0 tx=0x7f0efc02a950 comp rx=0 tx=0).stop 2024-04-23T14:53:36.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.551+0000 7f0f09600640 1 -- 172.21.15.83:0/3833507812 shutdown_connections 2024-04-23T14:53:36.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.551+0000 7f0f09600640 1 --2- 172.21.15.83:0/3833507812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f0410c4d0 0x7f0f0410e920 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.551+0000 7f0f09600640 1 --2- 172.21.15.83:0/3833507812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f04103a20 0x7f0f0410bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.551+0000 7f0f09600640 1 --2- 172.21.15.83:0/3833507812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f04103100 0x7f0f041034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.551+0000 7f0f09600640 1 -- 172.21.15.83:0/3833507812 >> 172.21.15.83:0/3833507812 conn(0x7f0f040fc880 msgr2=0x7f0f040feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.551+0000 7f0f09600640 1 -- 172.21.15.83:0/3833507812 shutdown_connections 2024-04-23T14:53:36.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.551+0000 7f0f09600640 1 -- 172.21.15.83:0/3833507812 wait complete. 2024-04-23T14:53:36.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.553+0000 7f0f09600640 1 Processor -- start 2024-04-23T14:53:36.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.556+0000 7f0f09600640 1 -- start start 2024-04-23T14:53:36.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.556+0000 7f0f09600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f04103100 0x7f0f04198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.556+0000 7f0f09600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f04103a20 0x7f0f04198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.556+0000 7f0f09600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f0410c4d0 0x7f0f0419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.556+0000 7f0f09600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f04112280 con 0x7f0f0410c4d0 2024-04-23T14:53:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.556+0000 7f0f09600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0f04112100 con 0x7f0f04103a20 2024-04-23T14:53:36.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.556+0000 7f0f09600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0f04112400 con 0x7f0f04103100 2024-04-23T14:53:36.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.557+0000 7f0f03e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f04103100 0x7f0f04198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.557+0000 7f0f03e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f04103100 0x7f0f04198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55216/0 (socket says 172.21.15.83:55216) 2024-04-23T14:53:36.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.557+0000 7f0f08c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f0410c4d0 0x7f0f0419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.557+0000 7f0f03e00640 1 -- 172.21.15.83:0/1862692194 learned_addr learned my addr 172.21.15.83:0/1862692194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:36.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.557+0000 7f0f03400640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f04103a20 0x7f0f04198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.557+0000 7f0f03e00640 1 -- 172.21.15.83:0/1862692194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f04103a20 msgr2=0x7f0f04198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.557+0000 7f0f03e00640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f04103a20 0x7f0f04198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.557+0000 7f0f03e00640 1 -- 172.21.15.83:0/1862692194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f0410c4d0 msgr2=0x7f0f0419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.557+0000 7f0f03e00640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f0410c4d0 0x7f0f0419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.557+0000 7f0f03e00640 1 -- 172.21.15.83:0/1862692194 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0419d700 con 0x7f0f04103100 2024-04-23T14:53:36.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.557+0000 7f0f03400640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f04103a20 0x7f0f04198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:53:36.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.558+0000 7f0f03e00640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f04103100 0x7f0f04198760 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400ea60 tx=0x7f0ef400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.558+0000 7f0f00c00640 1 -- 172.21.15.83:0/1862692194 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef400cde0 con 0x7f0f04103100 2024-04-23T14:53:36.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.558+0000 7f0f09600640 1 -- 172.21.15.83:0/1862692194 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0419d9f0 con 0x7f0f04103100 2024-04-23T14:53:36.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.558+0000 7f0f00c00640 1 -- 172.21.15.83:0/1862692194 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef40137d0 con 0x7f0f04103100 2024-04-23T14:53:36.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.558+0000 7f0f00c00640 1 -- 172.21.15.83:0/1862692194 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef4003950 con 0x7f0f04103100 2024-04-23T14:53:36.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.558+0000 7f0f09600640 1 -- 172.21.15.83:0/1862692194 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f04100c80 con 0x7f0f04103100 2024-04-23T14:53:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.562+0000 7f0f0dbc6740 1 Processor -- start 2024-04-23T14:53:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.562+0000 7f0f0dbc6740 1 -- start start 2024-04-23T14:53:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.563+0000 7f0f0dbc6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 0x55755c71af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.563+0000 7f0f0dbc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 0x55755c7255f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.563+0000 7f0f0dbc6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c725b30 0x55755c727f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.563+0000 7f0f0dbc6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55755c6801d0 con 0x55755c725b30 2024-04-23T14:53:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.563+0000 7f0f0dbc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55755c626450 con 0x55755c71b460 2024-04-23T14:53:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.563+0000 7f0f0dbc6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55755c67c190 con 0x55755c71ab40 2024-04-23T14:53:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.563+0000 7f0f03400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c725b30 0x55755c727f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.563+0000 7f0f08c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 0x55755c71af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.563+0000 7f0f08c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 0x55755c71af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55232/0 (socket says 172.21.15.83:55232) 2024-04-23T14:53:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.563+0000 7f0f08c00640 1 -- 172.21.15.83:0/59093632 learned_addr learned my addr 172.21.15.83:0/59093632 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:36.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.563+0000 7f0f00c00640 1 -- 172.21.15.83:0/1862692194 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ef4022020 con 0x7f0f04103100 2024-04-23T14:53:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.564+0000 7f0f03400640 1 -- 172.21.15.83:0/59093632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 msgr2=0x55755c71af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.564+0000 7f0f03400640 1 --2- 172.21.15.83:0/59093632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 0x55755c71af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.564+0000 7f0f03e00640 1 --2- 172.21.15.83:0/59093632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 0x55755c7255f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.564+0000 7f0f03400640 1 -- 172.21.15.83:0/59093632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 msgr2=0x55755c7255f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.564+0000 7f0f03400640 1 --2- 172.21.15.83:0/59093632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 0x55755c7255f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.564+0000 7f0f03400640 1 -- 172.21.15.83:0/59093632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55755c6e3310 con 0x55755c725b30 2024-04-23T14:53:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.564+0000 7f0f03e00640 1 --2- 172.21.15.83:0/59093632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 0x55755c7255f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:53:36.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.564+0000 7f0f03400640 1 --2- 172.21.15.83:0/59093632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c725b30 0x55755c727f20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009b80 tx=0x7f0ef00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f8f7ea4cd8655 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.565+0000 7f0ef9200640 1 -- 172.21.15.83:0/59093632 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0012450 con 0x55755c725b30 2024-04-23T14:53:36.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.565+0000 7f0ef9200640 1 -- 172.21.15.83:0/59093632 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef0012ad0 con 0x55755c725b30 2024-04-23T14:53:36.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.565+0000 7f0ef9200640 1 -- 172.21.15.83:0/59093632 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000fab0 con 0x55755c725b30 2024-04-23T14:53:36.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.565+0000 7f0f00c00640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0ee80a3f20 0x7f0ee80a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.566+0000 7f0f00c00640 1 -- 172.21.15.83:0/1862692194 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f0ef40036f0 con 0x7f0f04103100 2024-04-23T14:53:36.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.566+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/59093632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c725b30 msgr2=0x55755c727f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.566+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/59093632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c725b30 0x55755c727f20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009b80 tx=0x7f0ef00117b0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.566+0000 7f0f08c00640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0ee80a3f20 0x7f0ee80a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.567+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/59093632 shutdown_connections 2024-04-23T14:53:36.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.567+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/59093632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c725b30 0x55755c727f20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.567+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/59093632 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 0x55755c7255f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.567+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/59093632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 0x55755c71af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.567+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/59093632 >> 172.21.15.83:0/59093632 conn(0x55755c716e60 msgr2=0x55755c717240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.567+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/59093632 shutdown_connections 2024-04-23T14:53:36.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.567+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/59093632 wait complete. 2024-04-23T14:53:36.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.568+0000 7f0f0dbc6740 1 -- start start 2024-04-23T14:53:36.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.568+0000 7f0f0dbc6740 1 Processor -- start 2024-04-23T14:53:36.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.568+0000 7f0f08c00640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0ee80a3f20 0x7f0ee80a63e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009a60 tx=0x7f0efc005d80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.569+0000 7f0f0dbc6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 0x55755c725530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.570+0000 7f0f0dbc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 0x55755c722890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.570+0000 7f0f0dbc6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c725b30 0x55755c722f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.570+0000 7f0f03e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 0x55755c725530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.570+0000 7f0f03e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 0x55755c725530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55244/0 (socket says 172.21.15.83:55244) 2024-04-23T14:53:36.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.570+0000 7f0f03e00640 1 -- 172.21.15.83:0/673027383 learned_addr learned my addr 172.21.15.83:0/673027383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:36.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.570+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/673027383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55755c7289c0 con 0x55755c725b30 2024-04-23T14:53:36.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.570+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/673027383 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55755c639810 con 0x55755c71b460 2024-04-23T14:53:36.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.570+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/673027383 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55755c728b40 con 0x55755c71ab40 2024-04-23T14:53:36.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.570+0000 7f0f03400640 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 0x55755c722890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.570+0000 7f0f0dbc6740 10 client.0 _ll_register_callbacks cb 0x55755c607140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:36.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.570+0000 7f0f08c00640 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c725b30 0x55755c722f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.571+0000 7f0f03e00640 1 -- 172.21.15.83:0/673027383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 msgr2=0x55755c722890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.571+0000 7f0f03e00640 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 0x55755c722890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.571+0000 7f0f03e00640 1 -- 172.21.15.83:0/673027383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c725b30 msgr2=0x55755c722f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.571+0000 7f0f03e00640 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c725b30 0x55755c722f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.571+0000 7f0f03e00640 1 -- 172.21.15.83:0/673027383 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55755c64a3f0 con 0x55755c71ab40 2024-04-23T14:53:36.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.571+0000 7f0f03e00640 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 0x55755c725530 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40a03a0 tx=0x7f0ef40a0000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f784f93317f8f1c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.571+0000 7f0eed600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:53:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.572+0000 7f0eed600640 1 -- 172.21.15.83:0/673027383 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef408b070 con 0x55755c71ab40 2024-04-23T14:53:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.572+0000 7f0eed600640 1 -- 172.21.15.83:0/673027383 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f0ef409cd10 con 0x55755c71ab40 2024-04-23T14:53:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.572+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/673027383 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef0012ad0 con 0x55755c71ab40 2024-04-23T14:53:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.572+0000 7f0eed600640 1 -- 172.21.15.83:0/673027383 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ef40a4a50 con 0x55755c71ab40 2024-04-23T14:53:36.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.572+0000 7f0f0dbc6740 10 client.26188 fetch_fsmap learned FSMap version 29 2024-04-23T14:53:36.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.572+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/673027383 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55755c3923f0 con 0x55755c71ab40 2024-04-23T14:53:36.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.572+0000 7f0eed600640 1 -- 172.21.15.83:0/673027383 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef0012ad0 con 0x55755c71ab40 2024-04-23T14:53:36.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.573+0000 7f0eed600640 1 -- 172.21.15.83:0/673027383 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0ef40b9070 con 0x55755c71ab40 2024-04-23T14:53:36.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.573+0000 7f0f0dbc6740 10 client.26188 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:53:36.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.573+0000 7f0f0dbc6740 10 client.26188 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:53:36.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.573+0000 7f0f0dbc6740 20 client.26188 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:36.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.573+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55755c735c60 0x55755c7380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.573+0000 7f0f0dbc6740 4 client.26188 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:53:36.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.573+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/673027383 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55755c68dbf0 con 0x55755c735c60 2024-04-23T14:53:36.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.573+0000 7f0f03400640 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55755c735c60 0x55755c7380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.574+0000 7f0eed600640 10 client.26188 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:53:36.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.575+0000 7f0f03400640 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55755c735c60 0x55755c7380f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66c0f47870e205fa server_cookie=7fe2924f685ec4e7 in_seq=0 out_seq=0 2024-04-23T14:53:36.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.576+0000 7f0eed600640 1 -- 172.21.15.83:0/673027383 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55755c68dbf0 con 0x55755c735c60 2024-04-23T14:53:36.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.576+0000 7f0eed600640 10 client.26188 handle_command_reply: tid=0 2024-04-23T14:53:36.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.576+0000 7f0f0dbc6740 1 client.26188 shutdown 2024-04-23T14:53:36.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.576+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/673027383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 msgr2=0x55755c725530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.576+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 0x55755c725530 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40a03a0 tx=0x7f0ef40a0000 comp rx=0 tx=0).stop 2024-04-23T14:53:36.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.576+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/673027383 shutdown_connections 2024-04-23T14:53:36.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.577+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55755c735c60 0x55755c7380f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.577+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c725b30 0x55755c722f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.577+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 0x55755c722890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.577+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/673027383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c71ab40 0x55755c725530 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.577+0000 7f0eed600640 0 client.26188 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:53:36.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.577+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/673027383 >> 172.21.15.83:0/673027383 conn(0x55755c716e60 msgr2=0x55755c71cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.577+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/673027383 shutdown_connections 2024-04-23T14:53:36.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.577+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/673027383 wait complete. 2024-04-23T14:53:36.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.577+0000 7f0f0dbc6740 20 client.26188 trim_cache size 0 max 16384 2024-04-23T14:53:36.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.599+0000 7f0f0dbc6740 1 Processor -- start 2024-04-23T14:53:36.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.599+0000 7f0f0dbc6740 1 -- start start 2024-04-23T14:53:36.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.600+0000 7f0f0dbc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71ab40 0x55755c722170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.600+0000 7f0f0dbc6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71b460 0x55755c715dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.600+0000 7f0f0dbc6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c725b30 0x55755c7250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.600+0000 7f0f0dbc6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041ae1c0 con 0x55755c71b460 2024-04-23T14:53:36.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.600+0000 7f0f0dbc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041aba90 con 0x55755c71ab40 2024-04-23T14:53:36.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.600+0000 7f0f0dbc6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041a98e0 con 0x55755c725b30 2024-04-23T14:53:36.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.600+0000 7f0f08c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c725b30 0x55755c7250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.601 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.600+0000 7f0f03400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71b460 0x55755c715dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.601+0000 7f0f08c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c725b30 0x55755c7250d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55246/0 (socket says 172.21.15.83:55246) 2024-04-23T14:53:36.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.601+0000 7f0f08c00640 1 -- 172.21.15.83:0/571523761 learned_addr learned my addr 172.21.15.83:0/571523761 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:36.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.601+0000 7f0f03e00640 1 --2- 172.21.15.83:0/571523761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71ab40 0x55755c722170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.601+0000 7f0f03400640 1 -- 172.21.15.83:0/571523761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c725b30 msgr2=0x55755c7250d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.604+0000 7f0f03400640 1 --2- 172.21.15.83:0/571523761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c725b30 0x55755c7250d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.604+0000 7f0f03400640 1 -- 172.21.15.83:0/571523761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71ab40 msgr2=0x55755c722170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.604+0000 7f0f03400640 1 --2- 172.21.15.83:0/571523761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71ab40 0x55755c722170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.604+0000 7f0f03400640 1 -- 172.21.15.83:0/571523761 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55755c45c7d0 con 0x55755c71b460 2024-04-23T14:53:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.604+0000 7f0f08c00640 1 --2- 172.21.15.83:0/571523761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c725b30 0x55755c7250d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:36.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.604+0000 7f0f03e00640 1 --2- 172.21.15.83:0/571523761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71ab40 0x55755c722170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:36.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.604+0000 7f0f03400640 1 --2- 172.21.15.83:0/571523761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71b460 0x55755c715dd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0011cb0 tx=0x7f0ef00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c51743ede0835e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.605+0000 7f0eefe00640 1 -- 172.21.15.83:0/571523761 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0009db0 con 0x55755c71b460 2024-04-23T14:53:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.605+0000 7f0eefe00640 1 -- 172.21.15.83:0/571523761 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef0005520 con 0x55755c71b460 2024-04-23T14:53:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.605+0000 7f0eefe00640 1 -- 172.21.15.83:0/571523761 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000f520 con 0x55755c71b460 2024-04-23T14:53:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.606+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/571523761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71b460 msgr2=0x55755c715dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.606+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/571523761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71b460 0x55755c715dd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0011cb0 tx=0x7f0ef00058b0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.606+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/571523761 shutdown_connections 2024-04-23T14:53:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.606+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/571523761 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c725b30 0x55755c7250d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.606+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/571523761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71b460 0x55755c715dd0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.606+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/571523761 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71ab40 0x55755c722170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.606+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/571523761 >> 172.21.15.83:0/571523761 conn(0x55755c716e60 msgr2=0x55755c7156c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.606+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/571523761 shutdown_connections 2024-04-23T14:53:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.606+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/571523761 wait complete. 2024-04-23T14:53:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.606+0000 7f0f0dbc6740 1 -- start start 2024-04-23T14:53:36.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.607+0000 7f0f0dbc6740 1 Processor -- start 2024-04-23T14:53:36.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.607+0000 7f0f0dbc6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71ab40 0x55755c72e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.607+0000 7f0f0dbc6740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 0x55755c71fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.607+0000 7f0f0dbc6740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c725b30 0x55755c720050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.607+0000 7f0f0dbc6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041a9f50 con 0x55755c71ab40 2024-04-23T14:53:36.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.607+0000 7f0f0dbc6740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041ab3c0 con 0x55755c71b460 2024-04-23T14:53:36.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.607+0000 7f0f0dbc6740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041aa620 con 0x55755c725b30 2024-04-23T14:53:36.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.607+0000 7f0f0dbc6740 10 client.0 _ll_register_callbacks cb 0x7f0ef4009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:36.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.607+0000 7f0f03e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71ab40 0x55755c72e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.607+0000 7f0f03e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71ab40 0x55755c72e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51876/0 (socket says 172.21.15.83:51876) 2024-04-23T14:53:36.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.607+0000 7f0f03e00640 1 -- 172.21.15.83:0/2872063482 learned_addr learned my addr 172.21.15.83:0/2872063482 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:36.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.608+0000 7f0f08c00640 1 --2- 172.21.15.83:0/2872063482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c725b30 0x55755c720050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.608+0000 7f0f03e00640 1 -- 172.21.15.83:0/2872063482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c725b30 msgr2=0x55755c720050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.608+0000 7f0f03e00640 1 --2- 172.21.15.83:0/2872063482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c725b30 0x55755c720050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.608+0000 7f0f03e00640 1 -- 172.21.15.83:0/2872063482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 msgr2=0x55755c71fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.608+0000 7f0f03e00640 1 --2- 172.21.15.83:0/2872063482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 0x55755c71fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.608+0000 7f0f03e00640 1 -- 172.21.15.83:0/2872063482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55755c720830 con 0x55755c71ab40 2024-04-23T14:53:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.608+0000 7f0f03e00640 1 --2- 172.21.15.83:0/2872063482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71ab40 0x55755c72e180 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40a09e0 tx=0x7f0ef40a0130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85351edeb0291bff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.608+0000 7f0eed600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:53:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.608+0000 7f0eed600640 1 -- 172.21.15.83:0/2872063482 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef40a0470 con 0x55755c71ab40 2024-04-23T14:53:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.608+0000 7f0eed600640 1 -- 172.21.15.83:0/2872063482 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f0ef409faf0 con 0x55755c71ab40 2024-04-23T14:53:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.609+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/2872063482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef0005520 con 0x55755c71ab40 2024-04-23T14:53:36.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.610+0000 7f0eed600640 1 -- 172.21.15.83:0/2872063482 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ef40a9310 con 0x55755c71ab40 2024-04-23T14:53:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.610+0000 7f0f0dbc6740 10 client.16023 fetch_fsmap learned FSMap version 29 2024-04-23T14:53:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.610+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/2872063482 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55755c79c290 con 0x55755c71ab40 2024-04-23T14:53:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.610+0000 7f0eed600640 1 -- 172.21.15.83:0/2872063482 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef0005520 con 0x55755c71ab40 2024-04-23T14:53:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.610+0000 7f0eed600640 1 -- 172.21.15.83:0/2872063482 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0ef40b9040 con 0x55755c71ab40 2024-04-23T14:53:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.610+0000 7f0f0dbc6740 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:53:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.610+0000 7f0f0dbc6740 10 client.16023 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:53:36.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.610+0000 7f0f0dbc6740 20 client.16023 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:36.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.610+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/2872063482 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55755c79c9a0 0x55755c79edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.610+0000 7f0f0dbc6740 4 client.16023 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:53:36.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.610+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/2872063482 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55755c687a60 con 0x55755c79c9a0 2024-04-23T14:53:36.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.611+0000 7f0f03400640 1 --2- 172.21.15.83:0/2872063482 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55755c79c9a0 0x55755c79edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.616+0000 7f0f03400640 1 --2- 172.21.15.83:0/2872063482 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55755c79c9a0 0x55755c79edd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39f06db347a34f1e server_cookie=b57af56e612544c9 in_seq=0 out_seq=0 2024-04-23T14:53:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.616+0000 7f0eed600640 10 client.16023 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:53:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.617+0000 7f0eed600640 1 -- 172.21.15.83:0/2872063482 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55755c687a60 con 0x55755c79c9a0 2024-04-23T14:53:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.617+0000 7f0eed600640 10 client.16023 handle_command_reply: tid=0 2024-04-23T14:53:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.617+0000 7f0f0dbc6740 1 client.16023 shutdown 2024-04-23T14:53:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.618+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/2872063482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71ab40 msgr2=0x55755c72e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.618+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/2872063482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71ab40 0x55755c72e180 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0ef40a09e0 tx=0x7f0ef40a0130 comp rx=0 tx=0).stop 2024-04-23T14:53:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.618+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/2872063482 shutdown_connections 2024-04-23T14:53:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.618+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/2872063482 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55755c79c9a0 0x55755c79edd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.618+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/2872063482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55755c725b30 0x55755c720050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.618+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/2872063482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55755c71b460 0x55755c71fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.618+0000 7f0f0dbc6740 1 --2- 172.21.15.83:0/2872063482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55755c71ab40 0x55755c72e180 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.618+0000 7f0eed600640 0 client.16023 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:53:36.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.618+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/2872063482 >> 172.21.15.83:0/2872063482 conn(0x55755c716e60 msgr2=0x55755c715f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.618+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/2872063482 shutdown_connections 2024-04-23T14:53:36.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.618+0000 7f0f0dbc6740 1 -- 172.21.15.83:0/2872063482 wait complete. 2024-04-23T14:53:36.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.618+0000 7f0f0dbc6740 20 client.16023 trim_cache size 0 max 16384 2024-04-23T14:53:36.621 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:53:36.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.619+0000 7f0f09600640 1 -- 172.21.15.83:0/1862692194 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0ee80a3f20 msgr2=0x7f0ee80a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.619+0000 7f0f09600640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0ee80a3f20 0x7f0ee80a63e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009a60 tx=0x7f0efc005d80 comp rx=0 tx=0).stop 2024-04-23T14:53:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.619+0000 7f0f09600640 1 -- 172.21.15.83:0/1862692194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f04103100 msgr2=0x7f0f04198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.619+0000 7f0f09600640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f04103100 0x7f0f04198760 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0ef400ea60 tx=0x7f0ef400ef20 comp rx=0 tx=0).stop 2024-04-23T14:53:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.619+0000 7f0f09600640 1 -- 172.21.15.83:0/1862692194 shutdown_connections 2024-04-23T14:53:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.619+0000 7f0f09600640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0ee80a3f20 0x7f0ee80a63e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.619+0000 7f0f09600640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f0410c4d0 0x7f0f0419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.619+0000 7f0f09600640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f04103a20 0x7f0f04198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.619+0000 7f0f09600640 1 --2- 172.21.15.83:0/1862692194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f04103100 0x7f0f04198760 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.619+0000 7f0f09600640 1 -- 172.21.15.83:0/1862692194 >> 172.21.15.83:0/1862692194 conn(0x7f0f040fc880 msgr2=0x7f0f040fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.620+0000 7f0f09600640 1 -- 172.21.15.83:0/1862692194 shutdown_connections 2024-04-23T14:53:36.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.620+0000 7f0f09600640 1 -- 172.21.15.83:0/1862692194 wait complete. 2024-04-23T14:53:36.635 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:36 smithi083 ceph-mon[27867]: pgmap v157: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 64 MiB/s wr, 11.59k op/s 2024-04-23T14:53:36.636 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:36 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1275916660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:53:36.638 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:53:36.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:36 smithi203 ceph-mon[32800]: pgmap v157: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 64 MiB/s wr, 11.59k op/s 2024-04-23T14:53:36.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:36 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1275916660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:53:36.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:36 smithi173 ceph-mon[32443]: pgmap v157: 97 pgs: 97 active+clean; 20 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 64 MiB/s wr, 11.59k op/s 2024-04-23T14:53:36.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:36 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1275916660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:53:36.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.830+0000 7f5033e00640 1 -- 172.21.15.83:0/1760198714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503410d500 msgr2=0x7f503410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.830+0000 7f5033e00640 1 --2- 172.21.15.83:0/1760198714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503410d500 0x7f503410d8e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5024009890 tx=0x7f502402a380 comp rx=0 tx=0).stop 2024-04-23T14:53:36.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.831+0000 7f5033e00640 1 -- 172.21.15.83:0/1760198714 shutdown_connections 2024-04-23T14:53:36.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.831+0000 7f5033e00640 1 --2- 172.21.15.83:0/1760198714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5034108f40 0x7f50341153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.831+0000 7f5033e00640 1 --2- 172.21.15.83:0/1760198714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5034108580 0x7f5034108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.831+0000 7f5033e00640 1 --2- 172.21.15.83:0/1760198714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f503410d500 0x7f503410d8e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.831+0000 7f5033e00640 1 -- 172.21.15.83:0/1760198714 >> 172.21.15.83:0/1760198714 conn(0x7f5034077e20 msgr2=0x7f5034106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.831+0000 7f5033e00640 1 -- 172.21.15.83:0/1760198714 shutdown_connections 2024-04-23T14:53:36.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.831+0000 7f5033e00640 1 -- 172.21.15.83:0/1760198714 wait complete. 2024-04-23T14:53:36.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.832+0000 7f5033e00640 1 Processor -- start 2024-04-23T14:53:36.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.832+0000 7f5033e00640 1 -- start start 2024-04-23T14:53:36.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.832+0000 7f5033e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5034108580 0x7f503419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.832+0000 7f5033e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5034108f40 0x7f503419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.832+0000 7f5033e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f503410d500 0x7f50341a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.832+0000 7f5033e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50341188b0 con 0x7f5034108f40 2024-04-23T14:53:36.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.832+0000 7f5033e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5034118730 con 0x7f5034108580 2024-04-23T14:53:36.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.832+0000 7f5033e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5034118a30 con 0x7f503410d500 2024-04-23T14:53:36.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.833+0000 7f5032000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5034108f40 0x7f503419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.833+0000 7f5033400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f503410d500 0x7f50341a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.833+0000 7f5033400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f503410d500 0x7f50341a3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55276/0 (socket says 172.21.15.83:55276) 2024-04-23T14:53:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.833+0000 7f5033400640 1 -- 172.21.15.83:0/1950006425 learned_addr learned my addr 172.21.15.83:0/1950006425 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.833+0000 7f5032000640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5034108f40 0x7f503419f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51892/0 (socket says 172.21.15.83:51892) 2024-04-23T14:53:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.833+0000 7f5032000640 1 -- 172.21.15.83:0/1950006425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f503410d500 msgr2=0x7f50341a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.833+0000 7f5032000640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f503410d500 0x7f50341a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.833+0000 7f5032000640 1 -- 172.21.15.83:0/1950006425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5034108580 msgr2=0x7f503419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.833+0000 7f5032000640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5034108580 0x7f503419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.833+0000 7f5032000640 1 -- 172.21.15.83:0/1950006425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50341a3d50 con 0x7f5034108f40 2024-04-23T14:53:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.834+0000 7f5032000640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5034108f40 0x7f503419f2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f502000e790 tx=0x7f502000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.834+0000 7f502b400640 1 -- 172.21.15.83:0/1950006425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502000eea0 con 0x7f5034108f40 2024-04-23T14:53:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.834+0000 7f502b400640 1 -- 172.21.15.83:0/1950006425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502000ccc0 con 0x7f5034108f40 2024-04-23T14:53:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.834+0000 7f5033e00640 1 -- 172.21.15.83:0/1950006425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50341a4040 con 0x7f5034108f40 2024-04-23T14:53:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.834+0000 7f502b400640 1 -- 172.21.15.83:0/1950006425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020018780 con 0x7f5034108f40 2024-04-23T14:53:36.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.834+0000 7f5033e00640 1 -- 172.21.15.83:0/1950006425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50341ab8d0 con 0x7f5034108f40 2024-04-23T14:53:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.837+0000 7f503ca6c740 1 Processor -- start 2024-04-23T14:53:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.838+0000 7f503ca6c740 1 -- start start 2024-04-23T14:53:36.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.838+0000 7f503ca6c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.838+0000 7f503ca6c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c7460 0x55b7bc9d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.838+0000 7f503ca6c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9d3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.838+0000 7f503ca6c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b7bc92c1d0 con 0x55b7bc9c7460 2024-04-23T14:53:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.838+0000 7f503ca6c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b7bc8d2450 con 0x55b7bc9c6b40 2024-04-23T14:53:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.838+0000 7f503ca6c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b7bc928190 con 0x55b7bc9d1b30 2024-04-23T14:53:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.838+0000 7f5032a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c7460 0x55b7bc9d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.838+0000 7f5033400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.838+0000 7f5033400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37158/0 (socket says 172.21.15.83:37158) 2024-04-23T14:53:36.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.838+0000 7f5033400640 1 -- 172.21.15.83:0/1727174114 learned_addr learned my addr 172.21.15.83:0/1727174114 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:36.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.839+0000 7f5033400640 1 -- 172.21.15.83:0/1727174114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 msgr2=0x55b7bc9d3f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:53:36.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.839+0000 7f5033400640 1 --2- 172.21.15.83:0/1727174114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9d3f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.839+0000 7f5033400640 1 -- 172.21.15.83:0/1727174114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c7460 msgr2=0x55b7bc9d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.839+0000 7f5033400640 1 --2- 172.21.15.83:0/1727174114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c7460 0x55b7bc9d15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.839+0000 7f5033400640 1 -- 172.21.15.83:0/1727174114 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7bc98f310 con 0x55b7bc9c6b40 2024-04-23T14:53:36.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.839+0000 7f5033400640 1 --2- 172.21.15.83:0/1727174114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9c6f20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f502c011570 tx=0x7f502c011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde01175abb79021 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.839+0000 7f501fe00640 1 -- 172.21.15.83:0/1727174114 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c01e070 con 0x55b7bc9c6b40 2024-04-23T14:53:36.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.840+0000 7f502b400640 1 -- 172.21.15.83:0/1950006425 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5020010430 con 0x7f5034108f40 2024-04-23T14:53:36.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.840+0000 7f501fe00640 1 -- 172.21.15.83:0/1727174114 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502c00faa0 con 0x55b7bc9c6b40 2024-04-23T14:53:36.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.840+0000 7f501fe00640 1 -- 172.21.15.83:0/1727174114 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c019d10 con 0x55b7bc9c6b40 2024-04-23T14:53:36.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.840+0000 7f503ca6c740 1 -- 172.21.15.83:0/1727174114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 msgr2=0x55b7bc9c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.840+0000 7f503ca6c740 1 --2- 172.21.15.83:0/1727174114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9c6f20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f502c011570 tx=0x7f502c011a30 comp rx=0 tx=0).stop 2024-04-23T14:53:36.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.841+0000 7f503ca6c740 1 -- 172.21.15.83:0/1727174114 shutdown_connections 2024-04-23T14:53:36.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.841+0000 7f503ca6c740 1 --2- 172.21.15.83:0/1727174114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9d3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.841+0000 7f503ca6c740 1 --2- 172.21.15.83:0/1727174114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c7460 0x55b7bc9d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.841+0000 7f503ca6c740 1 --2- 172.21.15.83:0/1727174114 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9c6f20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.841+0000 7f503ca6c740 1 -- 172.21.15.83:0/1727174114 >> 172.21.15.83:0/1727174114 conn(0x55b7bc9c2e60 msgr2=0x55b7bc9c3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.841+0000 7f503ca6c740 1 -- 172.21.15.83:0/1727174114 shutdown_connections 2024-04-23T14:53:36.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.841+0000 7f503ca6c740 1 -- 172.21.15.83:0/1727174114 wait complete. 2024-04-23T14:53:36.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.841+0000 7f502b400640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f50180a3c30 0x7f50180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.841+0000 7f502b400640 1 -- 172.21.15.83:0/1950006425 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f50200d0a40 con 0x7f5034108f40 2024-04-23T14:53:36.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.841+0000 7f503ca6c740 1 -- start start 2024-04-23T14:53:36.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.841+0000 7f5032a00640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f50180a3c30 0x7f50180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.842+0000 7f503ca6c740 1 Processor -- start 2024-04-23T14:53:36.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.842+0000 7f503ca6c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.842+0000 7f503ca6c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c7460 0x55b7bc9ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.842+0000 7f503ca6c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.842+0000 7f503ca6c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b7bc9d4a10 con 0x55b7bc9d1b30 2024-04-23T14:53:36.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.842+0000 7f503ca6c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b7bc8e5810 con 0x55b7bc9c7460 2024-04-23T14:53:36.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.842+0000 7f503ca6c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b7bc9d4b90 con 0x55b7bc9c6b40 2024-04-23T14:53:36.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.842+0000 7f5032a00640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f50180a3c30 0x7f50180a60f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55b7bc9c8100 tx=0x7f502402c040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.842+0000 7f503ca6c740 10 client.0 _ll_register_callbacks cb 0x55b7bc8b3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:36.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.843+0000 7f5032a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.843+0000 7f5032a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9d15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55292/0 (socket says 172.21.15.83:55292) 2024-04-23T14:53:36.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.843+0000 7f5032a00640 1 -- 172.21.15.83:0/826094881 learned_addr learned my addr 172.21.15.83:0/826094881 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:36.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.843+0000 7f5033400640 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.843+0000 7f5032000640 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c7460 0x55b7bc9ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.843+0000 7f5033400640 1 -- 172.21.15.83:0/826094881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9c6b40 msgr2=0x55b7bc9d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.843+0000 7f5033400640 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.843+0000 7f5033400640 1 -- 172.21.15.83:0/826094881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c7460 msgr2=0x55b7bc9ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.843+0000 7f5033400640 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c7460 0x55b7bc9ce8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.843+0000 7f5033400640 1 -- 172.21.15.83:0/826094881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7bc8f63f0 con 0x55b7bc9d1b30 2024-04-23T14:53:36.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.843+0000 7f5032000640 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c7460 0x55b7bc9ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:36.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.844+0000 7f5033400640 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9cefb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f502c01c040 tx=0x7f502c00f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51134a184e7f4e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.844+0000 7f501cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:53:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.844+0000 7f501cc00640 1 -- 172.21.15.83:0/826094881 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c01e070 con 0x55b7bc9d1b30 2024-04-23T14:53:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.844+0000 7f501cc00640 1 -- 172.21.15.83:0/826094881 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f502c004c50 con 0x55b7bc9d1b30 2024-04-23T14:53:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.844+0000 7f503ca6c740 1 -- 172.21.15.83:0/826094881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f502c00faa0 con 0x55b7bc9d1b30 2024-04-23T14:53:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.844+0000 7f501cc00640 1 -- 172.21.15.83:0/826094881 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f502c02f050 con 0x55b7bc9d1b30 2024-04-23T14:53:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.844+0000 7f503ca6c740 10 client.16035 fetch_fsmap learned FSMap version 29 2024-04-23T14:53:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.844+0000 7f503ca6c740 1 -- 172.21.15.83:0/826094881 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b7bc63e3f0 con 0x55b7bc9d1b30 2024-04-23T14:53:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.845+0000 7f501cc00640 1 -- 172.21.15.83:0/826094881 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502c00faa0 con 0x55b7bc9d1b30 2024-04-23T14:53:36.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.845+0000 7f501cc00640 1 -- 172.21.15.83:0/826094881 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f502c02a030 con 0x55b7bc9d1b30 2024-04-23T14:53:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.845+0000 7f503ca6c740 10 client.16035 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:53:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.845+0000 7f503ca6c740 10 client.16035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:53:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.845+0000 7f503ca6c740 20 client.16035 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.845+0000 7f503ca6c740 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55b7bc9e1d90 0x55b7bc9e4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.845+0000 7f503ca6c740 4 client.16035 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:53:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.845+0000 7f503ca6c740 1 -- 172.21.15.83:0/826094881 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7bc939bf0 con 0x55b7bc9e1d90 2024-04-23T14:53:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.845+0000 7f5032a00640 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55b7bc9e1d90 0x55b7bc9e4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.846+0000 7f5032a00640 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55b7bc9e1d90 0x55b7bc9e4180 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff18405725f33798 server_cookie=1b806dfc2b685e5a in_seq=0 out_seq=0 2024-04-23T14:53:36.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.846+0000 7f501cc00640 10 client.16035 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:53:36.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.847+0000 7f501cc00640 1 -- 172.21.15.83:0/826094881 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7bc939bf0 con 0x55b7bc9e1d90 2024-04-23T14:53:36.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.847+0000 7f501cc00640 10 client.16035 handle_command_reply: tid=0 2024-04-23T14:53:36.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.847+0000 7f503ca6c740 1 client.16035 shutdown 2024-04-23T14:53:36.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.847+0000 7f503ca6c740 1 -- 172.21.15.83:0/826094881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9d1b30 msgr2=0x55b7bc9cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.847+0000 7f503ca6c740 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9cefb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f502c01c040 tx=0x7f502c00f870 comp rx=0 tx=0).stop 2024-04-23T14:53:36.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.847+0000 7f503ca6c740 1 -- 172.21.15.83:0/826094881 shutdown_connections 2024-04-23T14:53:36.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.847+0000 7f503ca6c740 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9cefb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.847+0000 7f503ca6c740 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c7460 0x55b7bc9ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.847+0000 7f503ca6c740 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55b7bc9e1d90 0x55b7bc9e4180 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.847+0000 7f503ca6c740 1 --2- 172.21.15.83:0/826094881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.847+0000 7f501cc00640 0 client.16035 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:53:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.847+0000 7f503ca6c740 1 -- 172.21.15.83:0/826094881 >> 172.21.15.83:0/826094881 conn(0x55b7bc9c2e60 msgr2=0x55b7bc9c8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.848+0000 7f503ca6c740 1 -- 172.21.15.83:0/826094881 shutdown_connections 2024-04-23T14:53:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.848+0000 7f503ca6c740 1 -- 172.21.15.83:0/826094881 wait complete. 2024-04-23T14:53:36.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.848+0000 7f503ca6c740 20 client.16035 trim_cache size 0 max 16384 2024-04-23T14:53:36.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.866+0000 7f503ca6c740 1 Processor -- start 2024-04-23T14:53:36.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.866+0000 7f503ca6c740 1 -- start start 2024-04-23T14:53:36.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.867+0000 7f503ca6c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9dd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.867+0000 7f503ca6c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c7460 0x55b7bc9d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.867+0000 7f503ca6c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9e98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.867+0000 7f503ca6c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50341b6b10 con 0x55b7bc9c7460 2024-04-23T14:53:36.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.867+0000 7f503ca6c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f50341b43e0 con 0x55b7bc9c6b40 2024-04-23T14:53:36.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.867+0000 7f503ca6c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f50341b2230 con 0x55b7bc9d1b30 2024-04-23T14:53:36.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.869+0000 7f5032a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9dd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.869+0000 7f5032a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9dd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37190/0 (socket says 172.21.15.83:37190) 2024-04-23T14:53:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.869+0000 7f5032a00640 1 -- 172.21.15.83:0/3275278085 learned_addr learned my addr 172.21.15.83:0/3275278085 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.873+0000 7f5032a00640 1 -- 172.21.15.83:0/3275278085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 msgr2=0x55b7bc9e98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.871+0000 7f5033400640 1 --2- 172.21.15.83:0/3275278085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9e98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.871+0000 7f5032000640 1 --2- 172.21.15.83:0/3275278085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c7460 0x55b7bc9d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.873+0000 7f5032a00640 1 --2- 172.21.15.83:0/3275278085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9e98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.873+0000 7f5032a00640 1 -- 172.21.15.83:0/3275278085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c7460 msgr2=0x55b7bc9d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.873+0000 7f5032a00640 1 --2- 172.21.15.83:0/3275278085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c7460 0x55b7bc9d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.873+0000 7f5032a00640 1 -- 172.21.15.83:0/3275278085 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7bc7087d0 con 0x55b7bc9c6b40 2024-04-23T14:53:36.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.873+0000 7f5033400640 1 --2- 172.21.15.83:0/3275278085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9e98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:53:36.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.874+0000 7f5032000640 1 --2- 172.21.15.83:0/3275278085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c7460 0x55b7bc9d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:53:36.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.875+0000 7f5032a00640 1 --2- 172.21.15.83:0/3275278085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9dd930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f503406b340 tx=0x7f502403b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7ca71dcf8cc6eb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.877+0000 7f501f400640 1 -- 172.21.15.83:0/3275278085 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024037040 con 0x55b7bc9c6b40 2024-04-23T14:53:36.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.877+0000 7f501f400640 1 -- 172.21.15.83:0/3275278085 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5024053070 con 0x55b7bc9c6b40 2024-04-23T14:53:36.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.878+0000 7f503ca6c740 1 -- 172.21.15.83:0/3275278085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 msgr2=0x55b7bc9dd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.878+0000 7f503ca6c740 1 --2- 172.21.15.83:0/3275278085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9dd930 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f503406b340 tx=0x7f502403b000 comp rx=0 tx=0).stop 2024-04-23T14:53:36.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.878+0000 7f503ca6c740 1 -- 172.21.15.83:0/3275278085 shutdown_connections 2024-04-23T14:53:36.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.878+0000 7f503ca6c740 1 --2- 172.21.15.83:0/3275278085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9e98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.878+0000 7f503ca6c740 1 --2- 172.21.15.83:0/3275278085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c7460 0x55b7bc9d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.878+0000 7f503ca6c740 1 --2- 172.21.15.83:0/3275278085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9dd930 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.878+0000 7f503ca6c740 1 -- 172.21.15.83:0/3275278085 >> 172.21.15.83:0/3275278085 conn(0x55b7bc9c2e60 msgr2=0x55b7bc9cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.885+0000 7f503ca6c740 1 -- 172.21.15.83:0/3275278085 shutdown_connections 2024-04-23T14:53:36.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.885+0000 7f503ca6c740 1 -- 172.21.15.83:0/3275278085 wait complete. 2024-04-23T14:53:36.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.886+0000 7f503ca6c740 1 -- start start 2024-04-23T14:53:36.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.886+0000 7f503ca6c740 1 Processor -- start 2024-04-23T14:53:36.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.888+0000 7f503ca6c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9d9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.888+0000 7f503ca6c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c7460 0x55b7bc9da380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.888+0000 7f503ca6c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.888+0000 7f503ca6c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50341b28a0 con 0x55b7bc9c6b40 2024-04-23T14:53:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.888+0000 7f503ca6c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f50341b3d10 con 0x55b7bc9c7460 2024-04-23T14:53:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.888+0000 7f503ca6c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f50341b2f70 con 0x55b7bc9d1b30 2024-04-23T14:53:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.888+0000 7f503ca6c740 10 client.0 _ll_register_callbacks cb 0x7f502c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:36.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.890+0000 7f5032a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9d9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.890+0000 7f5032a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9d9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51920/0 (socket says 172.21.15.83:51920) 2024-04-23T14:53:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.890+0000 7f5032000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c7460 0x55b7bc9da380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.890+0000 7f5032000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c7460 0x55b7bc9da380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37194/0 (socket says 172.21.15.83:37194) 2024-04-23T14:53:36.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.890+0000 7f5032000640 1 -- 172.21.15.83:0/297862865 learned_addr learned my addr 172.21.15.83:0/297862865 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:36.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.890+0000 7f5033400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.891+0000 7f5032a00640 1 -- 172.21.15.83:0/297862865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 msgr2=0x55b7bc9cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.891+0000 7f5032a00640 1 --2- 172.21.15.83:0/297862865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.891+0000 7f5032a00640 1 -- 172.21.15.83:0/297862865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c7460 msgr2=0x55b7bc9da380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:36.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.891+0000 7f5032a00640 1 --2- 172.21.15.83:0/297862865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c7460 0x55b7bc9da380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.891+0000 7f5032a00640 1 -- 172.21.15.83:0/297862865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7bc9cc520 con 0x55b7bc9c6b40 2024-04-23T14:53:36.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.892+0000 7f5032a00640 1 --2- 172.21.15.83:0/297862865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9d9e40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5024005ad0 tx=0x7f502402aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4daf43caa45b8e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:36.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.892+0000 7f501cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:53:36.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.893+0000 7f501cc00640 1 -- 172.21.15.83:0/297862865 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024037040 con 0x55b7bc9c6b40 2024-04-23T14:53:36.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.893+0000 7f501cc00640 1 -- 172.21.15.83:0/297862865 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f502404e150 con 0x55b7bc9c6b40 2024-04-23T14:53:36.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.894+0000 7f503ca6c740 1 -- 172.21.15.83:0/297862865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5024053070 con 0x55b7bc9c6b40 2024-04-23T14:53:36.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.898+0000 7f501cc00640 1 -- 172.21.15.83:0/297862865 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5024003e60 con 0x55b7bc9c6b40 2024-04-23T14:53:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.898+0000 7f503ca6c740 10 client.16038 fetch_fsmap learned FSMap version 29 2024-04-23T14:53:36.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.898+0000 7f503ca6c740 1 -- 172.21.15.83:0/297862865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b7bc9d2ee0 con 0x55b7bc9c6b40 2024-04-23T14:53:36.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.900+0000 7f501cc00640 1 -- 172.21.15.83:0/297862865 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5024053070 con 0x55b7bc9c6b40 2024-04-23T14:53:36.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.900+0000 7f501cc00640 1 -- 172.21.15.83:0/297862865 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f5024057730 con 0x55b7bc9c6b40 2024-04-23T14:53:36.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.901+0000 7f503ca6c740 10 client.16038 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:53:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.901+0000 7f503ca6c740 10 client.16038 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:53:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.901+0000 7f503ca6c740 20 client.16038 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:36.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.901+0000 7f503ca6c740 1 --2- 172.21.15.83:0/297862865 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55b7bca6a480 0x55b7bca6c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:36.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.901+0000 7f503ca6c740 4 client.16038 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:53:36.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.901+0000 7f503ca6c740 1 -- 172.21.15.83:0/297862865 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7bc933a60 con 0x55b7bca6a480 2024-04-23T14:53:36.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.902+0000 7f5032000640 1 --2- 172.21.15.83:0/297862865 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55b7bca6a480 0x55b7bca6c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.904+0000 7f5032000640 1 --2- 172.21.15.83:0/297862865 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55b7bca6a480 0x55b7bca6c8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63338c3e0dabf1ec server_cookie=2e6600b21bd7fc58 in_seq=0 out_seq=0 2024-04-23T14:53:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.904+0000 7f501cc00640 10 client.16038 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:53:36.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.906+0000 7f501cc00640 1 -- 172.21.15.83:0/297862865 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7bc933a60 con 0x55b7bca6a480 2024-04-23T14:53:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.906+0000 7f501cc00640 10 client.16038 handle_command_reply: tid=0 2024-04-23T14:53:36.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.906+0000 7f503ca6c740 1 client.16038 shutdown 2024-04-23T14:53:36.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.907+0000 7f503ca6c740 1 -- 172.21.15.83:0/297862865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c6b40 msgr2=0x55b7bc9d9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.907+0000 7f503ca6c740 1 --2- 172.21.15.83:0/297862865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9d9e40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5024005ad0 tx=0x7f502402aac0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.907+0000 7f503ca6c740 1 -- 172.21.15.83:0/297862865 shutdown_connections 2024-04-23T14:53:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.907+0000 7f503ca6c740 1 --2- 172.21.15.83:0/297862865 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55b7bca6a480 0x55b7bca6c8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.907+0000 7f503ca6c740 1 --2- 172.21.15.83:0/297862865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b7bc9d1b30 0x55b7bc9cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.907+0000 7f503ca6c740 1 --2- 172.21.15.83:0/297862865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7bc9c7460 0x55b7bc9da380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.907+0000 7f503ca6c740 1 --2- 172.21.15.83:0/297862865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b7bc9c6b40 0x55b7bc9d9e40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.907+0000 7f501cc00640 0 client.16038 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:53:36.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.907+0000 7f503ca6c740 1 -- 172.21.15.83:0/297862865 >> 172.21.15.83:0/297862865 conn(0x55b7bc9c2e60 msgr2=0x55b7bc9c1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.909+0000 7f503ca6c740 1 -- 172.21.15.83:0/297862865 shutdown_connections 2024-04-23T14:53:36.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.910+0000 7f503ca6c740 1 -- 172.21.15.83:0/297862865 wait complete. 2024-04-23T14:53:36.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.911+0000 7f503ca6c740 20 client.16038 trim_cache size 0 max 16384 2024-04-23T14:53:36.911 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:53:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.913+0000 7f5033e00640 1 -- 172.21.15.83:0/1950006425 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f50180a3c30 msgr2=0x7f50180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.913+0000 7f5033e00640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f50180a3c30 0x7f50180a60f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55b7bc9c8100 tx=0x7f502402c040 comp rx=0 tx=0).stop 2024-04-23T14:53:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.913+0000 7f5033e00640 1 -- 172.21.15.83:0/1950006425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5034108f40 msgr2=0x7f503419f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:36.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.913+0000 7f5033e00640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5034108f40 0x7f503419f2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f502000e790 tx=0x7f502000ec50 comp rx=0 tx=0).stop 2024-04-23T14:53:36.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.913+0000 7f5033e00640 1 -- 172.21.15.83:0/1950006425 shutdown_connections 2024-04-23T14:53:36.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.913+0000 7f5033e00640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f503410d500 0x7f50341a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.913+0000 7f5033e00640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f50180a3c30 0x7f50180a60f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.913+0000 7f5033e00640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5034108f40 0x7f503419f2f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.913+0000 7f5033e00640 1 --2- 172.21.15.83:0/1950006425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5034108580 0x7f503419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:36.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.913+0000 7f5033e00640 1 -- 172.21.15.83:0/1950006425 >> 172.21.15.83:0/1950006425 conn(0x7f5034077e20 msgr2=0x7f50340783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.916+0000 7f5033e00640 1 -- 172.21.15.83:0/1950006425 shutdown_connections 2024-04-23T14:53:36.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:36.916+0000 7f5033e00640 1 -- 172.21.15.83:0/1950006425 wait complete. 2024-04-23T14:53:36.956 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:53:37.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.275+0000 7f59c3200640 1 -- 172.21.15.83:0/3461001817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc100780 msgr2=0x7f59bc100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:37.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.275+0000 7f59c3200640 1 --2- 172.21.15.83:0/3461001817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc100780 0x7f59bc100b60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009ca0 tx=0x7f59b402a8e0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.275+0000 7f59c3200640 1 -- 172.21.15.83:0/3461001817 shutdown_connections 2024-04-23T14:53:37.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.275+0000 7f59c3200640 1 --2- 172.21.15.83:0/3461001817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59bc107390 0x7f59bc10ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.275+0000 7f59c3200640 1 --2- 172.21.15.83:0/3461001817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59bc101130 0x7f59bc106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.275+0000 7f59c3200640 1 --2- 172.21.15.83:0/3461001817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc100780 0x7f59bc100b60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.275+0000 7f59c3200640 1 -- 172.21.15.83:0/3461001817 >> 172.21.15.83:0/3461001817 conn(0x7f59bc0774a0 msgr2=0x7f59bc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:37.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.277+0000 7f59c3200640 1 -- 172.21.15.83:0/3461001817 shutdown_connections 2024-04-23T14:53:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.277+0000 7f59c3200640 1 -- 172.21.15.83:0/3461001817 wait complete. 2024-04-23T14:53:37.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.278+0000 7f59c3200640 1 Processor -- start 2024-04-23T14:53:37.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.278+0000 7f59c3200640 1 -- start start 2024-04-23T14:53:37.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.279+0000 7f59c3200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc100780 0x7f59bc0737d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.279+0000 7f59c3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59bc107390 0x7f59bc06c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.279+0000 7f59c3200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59bc073e20 0x7f59bc06cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.279+0000 7f59c3200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1124a0 con 0x7f59bc107390 2024-04-23T14:53:37.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.279+0000 7f59c3200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc112320 con 0x7f59bc073e20 2024-04-23T14:53:37.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.279+0000 7f59c3200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc112620 con 0x7f59bc100780 2024-04-23T14:53:37.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.279+0000 7f59bbe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc100780 0x7f59bc0737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:37.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.279+0000 7f59bbe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc100780 0x7f59bc0737d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55338/0 (socket says 172.21.15.83:55338) 2024-04-23T14:53:37.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.279+0000 7f59bbe00640 1 -- 172.21.15.83:0/3308563176 learned_addr learned my addr 172.21.15.83:0/3308563176 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:37.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.279+0000 7f59bb400640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59bc107390 0x7f59bc06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:37.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.280+0000 7f59bbe00640 1 -- 172.21.15.83:0/3308563176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59bc073e20 msgr2=0x7f59bc06cf20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T14:53:37.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.280+0000 7f59bbe00640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59bc073e20 0x7f59bc06cf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.280+0000 7f59bbe00640 1 -- 172.21.15.83:0/3308563176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59bc107390 msgr2=0x7f59bc06c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:37.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.280+0000 7f59bbe00640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59bc107390 0x7f59bc06c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.280+0000 7f59bbe00640 1 -- 172.21.15.83:0/3308563176 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc0712f0 con 0x7f59bc100780 2024-04-23T14:53:37.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.280+0000 7f59bbe00640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc100780 0x7f59bc0737d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f59b4005e30 tx=0x7f59b4003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:37.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.281+0000 7f59b8c00640 1 -- 172.21.15.83:0/3308563176 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4003d60 con 0x7f59bc100780 2024-04-23T14:53:37.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.281+0000 7f59b8c00640 1 -- 172.21.15.83:0/3308563176 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b4004520 con 0x7f59bc100780 2024-04-23T14:53:37.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.281+0000 7f59b8c00640 1 -- 172.21.15.83:0/3308563176 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b4030d20 con 0x7f59bc100780 2024-04-23T14:53:37.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.281+0000 7f59c3200640 1 -- 172.21.15.83:0/3308563176 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc071580 con 0x7f59bc100780 2024-04-23T14:53:37.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.281+0000 7f59c3200640 1 -- 172.21.15.83:0/3308563176 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc071a80 con 0x7f59bc100780 2024-04-23T14:53:37.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.288+0000 7f59c6014740 1 Processor -- start 2024-04-23T14:53:37.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.288+0000 7f59c6014740 1 -- start start 2024-04-23T14:53:37.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.289+0000 7f59c6014740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a3ca10 0x55b663a3cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.289+0000 7f59c6014740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38270 0x55b663a386f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.289+0000 7f59c6014740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38c30 0x55b663a44960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.289+0000 7f59c6014740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b6639941d0 con 0x55b663a38c30 2024-04-23T14:53:37.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.289+0000 7f59c6014740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b66393a450 con 0x55b663a38270 2024-04-23T14:53:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.289+0000 7f59c6014740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b663990190 con 0x55b663a3ca10 2024-04-23T14:53:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.290+0000 7f59c1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a3ca10 0x55b663a3cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.290+0000 7f59c1200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a3ca10 0x55b663a3cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55352/0 (socket says 172.21.15.83:55352) 2024-04-23T14:53:37.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.290+0000 7f59c1200640 1 -- 172.21.15.83:0/173759105 learned_addr learned my addr 172.21.15.83:0/173759105 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:37.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.291+0000 7f59c1200640 1 -- 172.21.15.83:0/173759105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38270 msgr2=0x55b663a386f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:53:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.291+0000 7f59c1200640 1 --2- 172.21.15.83:0/173759105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38270 0x55b663a386f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.291+0000 7f59c1200640 1 -- 172.21.15.83:0/173759105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38c30 msgr2=0x55b663a44960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:53:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.291+0000 7f59c1200640 1 --2- 172.21.15.83:0/173759105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38c30 0x55b663a44960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.291+0000 7f59c1200640 1 -- 172.21.15.83:0/173759105 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6639f7310 con 0x55b663a3ca10 2024-04-23T14:53:37.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.292+0000 7f59c1200640 1 --2- 172.21.15.83:0/173759105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a3ca10 0x55b663a3cdf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009bd0 tx=0x7f59a800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=135fa6030b99db64 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:37.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.293+0000 7f59b8c00640 1 -- 172.21.15.83:0/3308563176 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f59b40046d0 con 0x7f59bc100780 2024-04-23T14:53:37.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.293+0000 7f59b1600640 1 -- 172.21.15.83:0/173759105 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8005020 con 0x55b663a3ca10 2024-04-23T14:53:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.293+0000 7f59b1600640 1 -- 172.21.15.83:0/173759105 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a80047a0 con 0x55b663a3ca10 2024-04-23T14:53:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.293+0000 7f59b1600640 1 -- 172.21.15.83:0/173759105 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8010690 con 0x55b663a3ca10 2024-04-23T14:53:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.294+0000 7f59c6014740 1 -- 172.21.15.83:0/173759105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a3ca10 msgr2=0x55b663a3cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.294+0000 7f59c6014740 1 --2- 172.21.15.83:0/173759105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a3ca10 0x55b663a3cdf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009bd0 tx=0x7f59a800e720 comp rx=0 tx=0).stop 2024-04-23T14:53:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.295+0000 7f59b8c00640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f59a00a3ef0 0x7f59a00a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.295+0000 7f59b8c00640 1 -- 172.21.15.83:0/3308563176 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f59b40f1a90 con 0x7f59bc100780 2024-04-23T14:53:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.297+0000 7f59c6014740 1 -- 172.21.15.83:0/173759105 shutdown_connections 2024-04-23T14:53:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.297+0000 7f59c6014740 1 --2- 172.21.15.83:0/173759105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38c30 0x55b663a44960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.297+0000 7f59c6014740 1 --2- 172.21.15.83:0/173759105 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38270 0x55b663a386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.297+0000 7f59c6014740 1 --2- 172.21.15.83:0/173759105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a3ca10 0x55b663a3cdf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.297+0000 7f59c6014740 1 -- 172.21.15.83:0/173759105 >> 172.21.15.83:0/173759105 conn(0x55b663a2a720 msgr2=0x55b663a2ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.298+0000 7f59c6014740 1 -- 172.21.15.83:0/173759105 shutdown_connections 2024-04-23T14:53:37.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.298+0000 7f59c6014740 1 -- 172.21.15.83:0/173759105 wait complete. 2024-04-23T14:53:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.299+0000 7f59c6014740 1 -- start start 2024-04-23T14:53:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.299+0000 7f59c6014740 1 Processor -- start 2024-04-23T14:53:37.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.301+0000 7f59c1200640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f59a00a3ef0 0x7f59a00a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.302+0000 7f59c6014740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38270 0x55b663a448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.302+0000 7f59c6014740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38c30 0x55b663a41b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.302+0000 7f59c6014740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a3ca10 0x55b663a421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.302+0000 7f59bb400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38c30 0x55b663a41b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.302+0000 7f59bb400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38c30 0x55b663a41b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37222/0 (socket says 172.21.15.83:37222) 2024-04-23T14:53:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.302+0000 7f59bbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38270 0x55b663a448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:37.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.302+0000 7f59bbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38270 0x55b663a448a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51952/0 (socket says 172.21.15.83:51952) 2024-04-23T14:53:37.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.304+0000 7f59bb400640 1 -- 172.21.15.83:0/2065433425 learned_addr learned my addr 172.21.15.83:0/2065433425 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:37.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.304+0000 7f59c6014740 1 -- 172.21.15.83:0/2065433425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b663a44fa0 con 0x55b663a38270 2024-04-23T14:53:37.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.304+0000 7f59c1200640 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a3ca10 0x55b663a421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:37.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.305+0000 7f59c6014740 1 -- 172.21.15.83:0/2065433425 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55b66394d810 con 0x55b663a38c30 2024-04-23T14:53:37.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.305+0000 7f59c6014740 1 -- 172.21.15.83:0/2065433425 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55b663a45120 con 0x55b663a3ca10 2024-04-23T14:53:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.306+0000 7f59bbe00640 1 -- 172.21.15.83:0/2065433425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a3ca10 msgr2=0x55b663a421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.306+0000 7f59bbe00640 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a3ca10 0x55b663a421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.306+0000 7f59bbe00640 1 -- 172.21.15.83:0/2065433425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38c30 msgr2=0x55b663a41b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.306+0000 7f59bbe00640 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38c30 0x55b663a41b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.306+0000 7f59bbe00640 1 -- 172.21.15.83:0/2065433425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b66395e3f0 con 0x55b663a38270 2024-04-23T14:53:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.307+0000 7f59bb400640 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38c30 0x55b663a41b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:53:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.307+0000 7f59c6014740 10 client.0 _ll_register_callbacks cb 0x55b66391b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.307+0000 7f59c6014740 1 -- 172.21.15.83:0/2065433425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59a80047a0 con 0x55b663a38270 2024-04-23T14:53:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.307+0000 7f59bbe00640 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38270 0x55b663a448a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f59b40ef2a0 tx=0x7f59b40be1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b73ae3961d2bfa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:37.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.307+0000 7f59a6000640 10 client.16041 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:53:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.308+0000 7f59a6000640 1 -- 172.21.15.83:0/2065433425 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40c1ea0 con 0x55b663a38270 2024-04-23T14:53:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.308+0000 7f59c1200640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f59a00a3ef0 0x7f59a00a63b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009bd0 tx=0x7f59a8009cc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:37.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.309+0000 7f59a6000640 1 -- 172.21.15.83:0/2065433425 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f59b40ba160 con 0x55b663a38270 2024-04-23T14:53:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.310+0000 7f59a6000640 1 -- 172.21.15.83:0/2065433425 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f59b40c6760 con 0x55b663a38270 2024-04-23T14:53:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.310+0000 7f59c6014740 10 client.16041 fetch_fsmap learned FSMap version 29 2024-04-23T14:53:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.310+0000 7f59c6014740 1 -- 172.21.15.83:0/2065433425 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55b6636a63f0 con 0x55b663a38270 2024-04-23T14:53:37.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.310+0000 7f59a6000640 1 -- 172.21.15.83:0/2065433425 <== mon.0 v2:172.21.15.83:3300/0 4 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f59b40ef410 con 0x55b663a38270 2024-04-23T14:53:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.311+0000 7f59c6014740 10 client.16041 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:53:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.311+0000 7f59c6014740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:53:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.311+0000 7f59c6014740 20 client.16041 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.311+0000 7f59c6014740 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55b663a522d0 0x55b663a54700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.311+0000 7f59c6014740 4 client.16041 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:53:37.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.311+0000 7f59c6014740 1 -- 172.21.15.83:0/2065433425 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6639a1bf0 con 0x55b663a522d0 2024-04-23T14:53:37.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.311+0000 7f59bb400640 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55b663a522d0 0x55b663a54700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:37.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.313+0000 7f59bb400640 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55b663a522d0 0x55b663a54700 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3a22cb269e4e23 server_cookie=939060b30939efa6 in_seq=0 out_seq=0 2024-04-23T14:53:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.314+0000 7f59a6000640 10 client.16041 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.314+0000 7f59a6000640 1 -- 172.21.15.83:0/2065433425 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6639a1bf0 con 0x55b663a522d0 2024-04-23T14:53:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.314+0000 7f59a6000640 10 client.16041 handle_command_reply: tid=0 2024-04-23T14:53:37.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.314+0000 7f59c6014740 1 client.16041 shutdown 2024-04-23T14:53:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.320+0000 7f59c6014740 1 -- 172.21.15.83:0/2065433425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38270 msgr2=0x55b663a448a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:37.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.320+0000 7f59c6014740 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38270 0x55b663a448a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f59b40ef2a0 tx=0x7f59b40be1f0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.321+0000 7f59c6014740 1 -- 172.21.15.83:0/2065433425 shutdown_connections 2024-04-23T14:53:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.321+0000 7f59c6014740 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55b663a522d0 0x55b663a54700 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.321+0000 7f59a6000640 0 client.16041 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.321+0000 7f59c6014740 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a3ca10 0x55b663a421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.321+0000 7f59c6014740 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38c30 0x55b663a41b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.321+0000 7f59c6014740 1 --2- 172.21.15.83:0/2065433425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38270 0x55b663a448a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.321+0000 7f59c6014740 1 -- 172.21.15.83:0/2065433425 >> 172.21.15.83:0/2065433425 conn(0x55b663a2a720 msgr2=0x55b663a37af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:37.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.322+0000 7f59c6014740 1 -- 172.21.15.83:0/2065433425 shutdown_connections 2024-04-23T14:53:37.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.324+0000 7f59c6014740 1 -- 172.21.15.83:0/2065433425 wait complete. 2024-04-23T14:53:37.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.324+0000 7f59c6014740 20 client.16041 trim_cache size 0 max 16384 2024-04-23T14:53:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.345+0000 7f59c6014740 1 Processor -- start 2024-04-23T14:53:37.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.345+0000 7f59c6014740 1 -- start start 2024-04-23T14:53:37.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.346+0000 7f59c6014740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38270 0x55b663a386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.346+0000 7f59c6014740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a38c30 0x55b663a2dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.346+0000 7f59c6014740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a3ca10 0x55b663a404a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.346+0000 7f59c6014740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b6ad0 con 0x55b663a3ca10 2024-04-23T14:53:37.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.346+0000 7f59c6014740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b43a0 con 0x55b663a38270 2024-04-23T14:53:37.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.346+0000 7f59c6014740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b21f0 con 0x55b663a38c30 2024-04-23T14:53:37.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.346+0000 7f59bbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38270 0x55b663a386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:37.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.346+0000 7f59bbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38270 0x55b663a386d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37226/0 (socket says 172.21.15.83:37226) 2024-04-23T14:53:37.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.346+0000 7f59bbe00640 1 -- 172.21.15.83:0/125209804 learned_addr learned my addr 172.21.15.83:0/125209804 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:37.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.347+0000 7f59bbe00640 1 -- 172.21.15.83:0/125209804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a38c30 msgr2=0x55b663a2dfb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T14:53:37.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.347+0000 7f59bbe00640 1 --2- 172.21.15.83:0/125209804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a38c30 0x55b663a2dfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.347+0000 7f59bbe00640 1 -- 172.21.15.83:0/125209804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a3ca10 msgr2=0x55b663a404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.347+0000 7f59bbe00640 1 --2- 172.21.15.83:0/125209804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a3ca10 0x55b663a404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.347+0000 7f59bbe00640 1 -- 172.21.15.83:0/125209804 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6637707d0 con 0x55b663a38270 2024-04-23T14:53:37.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.347+0000 7f59bbe00640 1 --2- 172.21.15.83:0/125209804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38270 0x55b663a386d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f59b40bd7f0 tx=0x7f59b40de050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ea2b6b85e89fe61 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:37.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.347+0000 7f59b0c00640 1 -- 172.21.15.83:0/125209804 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40c1ea0 con 0x55b663a38270 2024-04-23T14:53:37.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.347+0000 7f59b0c00640 1 -- 172.21.15.83:0/125209804 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a80047a0 con 0x55b663a38270 2024-04-23T14:53:37.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.347+0000 7f59b0c00640 1 -- 172.21.15.83:0/125209804 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40b9840 con 0x55b663a38270 2024-04-23T14:53:37.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.348+0000 7f59c6014740 1 -- 172.21.15.83:0/125209804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38270 msgr2=0x55b663a386d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:37.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.348+0000 7f59c6014740 1 --2- 172.21.15.83:0/125209804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38270 0x55b663a386d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f59b40bd7f0 tx=0x7f59b40de050 comp rx=0 tx=0).stop 2024-04-23T14:53:37.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.348+0000 7f59c6014740 1 -- 172.21.15.83:0/125209804 shutdown_connections 2024-04-23T14:53:37.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.348+0000 7f59c6014740 1 --2- 172.21.15.83:0/125209804 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a3ca10 0x55b663a404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.348+0000 7f59c6014740 1 --2- 172.21.15.83:0/125209804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a38c30 0x55b663a2dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.348+0000 7f59c6014740 1 --2- 172.21.15.83:0/125209804 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a38270 0x55b663a386d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.348+0000 7f59c6014740 1 -- 172.21.15.83:0/125209804 >> 172.21.15.83:0/125209804 conn(0x55b663a2a720 msgr2=0x55b663a3ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:37.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.349+0000 7f59c6014740 1 -- 172.21.15.83:0/125209804 shutdown_connections 2024-04-23T14:53:37.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.350+0000 7f59c6014740 1 -- 172.21.15.83:0/125209804 wait complete. 2024-04-23T14:53:37.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.350+0000 7f59c6014740 1 -- start start 2024-04-23T14:53:37.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.350+0000 7f59c6014740 1 Processor -- start 2024-04-23T14:53:37.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.351+0000 7f59c6014740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a38270 0x55b663a4a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.351+0000 7f59c6014740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38c30 0x55b663a4a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.351+0000 7f59c6014740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a3ca10 0x55b663a4d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.351+0000 7f59c6014740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b2860 con 0x55b663a38c30 2024-04-23T14:53:37.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.351+0000 7f59c6014740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b3cd0 con 0x55b663a3ca10 2024-04-23T14:53:37.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.351+0000 7f59c6014740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc1b2f30 con 0x55b663a38270 2024-04-23T14:53:37.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.351+0000 7f59c6014740 10 client.0 _ll_register_callbacks cb 0x7f59b40095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:37.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.351+0000 7f59c1200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a3ca10 0x55b663a4d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:37.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.352+0000 7f59c1200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a3ca10 0x55b663a4d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37238/0 (socket says 172.21.15.83:37238) 2024-04-23T14:53:37.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.352+0000 7f59c1200640 1 -- 172.21.15.83:0/3682767920 learned_addr learned my addr 172.21.15.83:0/3682767920 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:37.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.352+0000 7f59bbe00640 1 --2- 172.21.15.83:0/3682767920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a38270 0x55b663a4a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:37.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.353+0000 7f59c1200640 1 -- 172.21.15.83:0/3682767920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a38270 msgr2=0x55b663a4a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:37.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.353+0000 7f59c1200640 1 --2- 172.21.15.83:0/3682767920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a38270 0x55b663a4a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.353+0000 7f59c1200640 1 -- 172.21.15.83:0/3682767920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38c30 msgr2=0x55b663a4a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:37.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.353+0000 7f59c1200640 1 --2- 172.21.15.83:0/3682767920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38c30 0x55b663a4a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.353+0000 7f59c1200640 1 -- 172.21.15.83:0/3682767920 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b663a4de70 con 0x55b663a3ca10 2024-04-23T14:53:37.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.354+0000 7f59c1200640 1 --2- 172.21.15.83:0/3682767920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a3ca10 0x55b663a4d640 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f59a8004280 tx=0x7f59a800e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df36bc9108546c03 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:37.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.354+0000 7f59a6000640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:53:37.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.355+0000 7f59a6000640 1 -- 172.21.15.83:0/3682767920 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a802b070 con 0x55b663a3ca10 2024-04-23T14:53:37.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.355+0000 7f59a6000640 1 -- 172.21.15.83:0/3682767920 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f59a80056a0 con 0x55b663a3ca10 2024-04-23T14:53:37.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.355+0000 7f59c6014740 1 -- 172.21.15.83:0/3682767920 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59a80047a0 con 0x55b663a3ca10 2024-04-23T14:53:37.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.355+0000 7f59a6000640 1 -- 172.21.15.83:0/3682767920 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f59a803d050 con 0x55b663a3ca10 2024-04-23T14:53:37.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.355+0000 7f59c6014740 10 client.26255 fetch_fsmap learned FSMap version 29 2024-04-23T14:53:37.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.355+0000 7f59c6014740 1 -- 172.21.15.83:0/3682767920 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b663a59c90 con 0x55b663a3ca10 2024-04-23T14:53:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.356+0000 7f59a6000640 1 -- 172.21.15.83:0/3682767920 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a80047a0 con 0x55b663a3ca10 2024-04-23T14:53:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.356+0000 7f59a6000640 1 -- 172.21.15.83:0/3682767920 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f59a802f7a0 con 0x55b663a3ca10 2024-04-23T14:53:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.356+0000 7f59c6014740 10 client.26255 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:53:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.357+0000 7f59c6014740 10 client.26255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:53:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.357+0000 7f59c6014740 20 client.26255 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.357+0000 7f59c6014740 1 --2- 172.21.15.83:0/3682767920 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55b663ab98e0 0x55b663abbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:37.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.357+0000 7f59c6014740 4 client.26255 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:53:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.357+0000 7f59c6014740 1 -- 172.21.15.83:0/3682767920 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b66399ba60 con 0x55b663ab98e0 2024-04-23T14:53:37.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.357+0000 7f59bbe00640 1 --2- 172.21.15.83:0/3682767920 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55b663ab98e0 0x55b663abbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:37.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.359+0000 7f59bbe00640 1 --2- 172.21.15.83:0/3682767920 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55b663ab98e0 0x55b663abbcd0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8836ea5179c9c753 server_cookie=48f881d34245d170 in_seq=0 out_seq=0 2024-04-23T14:53:37.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.360+0000 7f59a6000640 10 client.26255 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:37.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.360+0000 7f59a6000640 1 -- 172.21.15.83:0/3682767920 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b66399ba60 con 0x55b663ab98e0 2024-04-23T14:53:37.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.360+0000 7f59a6000640 10 client.26255 handle_command_reply: tid=0 2024-04-23T14:53:37.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.360+0000 7f59c6014740 1 client.26255 shutdown 2024-04-23T14:53:37.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.361+0000 7f59c6014740 1 -- 172.21.15.83:0/3682767920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a3ca10 msgr2=0x55b663a4d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:37.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.361+0000 7f59c6014740 1 --2- 172.21.15.83:0/3682767920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a3ca10 0x55b663a4d640 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f59a8004280 tx=0x7f59a800e720 comp rx=0 tx=0).stop 2024-04-23T14:53:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.362+0000 7f59c6014740 1 -- 172.21.15.83:0/3682767920 shutdown_connections 2024-04-23T14:53:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.362+0000 7f59c6014740 1 --2- 172.21.15.83:0/3682767920 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55b663ab98e0 0x55b663abbcd0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.362+0000 7f59c6014740 1 --2- 172.21.15.83:0/3682767920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b663a3ca10 0x55b663a4d640 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.362+0000 7f59c6014740 1 --2- 172.21.15.83:0/3682767920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b663a38c30 0x55b663a4a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.362+0000 7f59c6014740 1 --2- 172.21.15.83:0/3682767920 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b663a38270 0x55b663a4a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.362+0000 7f59c6014740 1 -- 172.21.15.83:0/3682767920 >> 172.21.15.83:0/3682767920 conn(0x55b663a2a720 msgr2=0x55b663a403b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:37.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.362+0000 7f59a6000640 0 client.26255 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:37.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.363+0000 7f59c6014740 1 -- 172.21.15.83:0/3682767920 shutdown_connections 2024-04-23T14:53:37.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.363+0000 7f59c6014740 1 -- 172.21.15.83:0/3682767920 wait complete. 2024-04-23T14:53:37.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.364+0000 7f59c6014740 20 client.26255 trim_cache size 0 max 16384 2024-04-23T14:53:37.364 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:53:37.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.365+0000 7f59c3200640 1 -- 172.21.15.83:0/3308563176 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f59a00a3ef0 msgr2=0x7f59a00a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:37.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.365+0000 7f59c3200640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f59a00a3ef0 0x7f59a00a63b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009bd0 tx=0x7f59a8009cc0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.365+0000 7f59c3200640 1 -- 172.21.15.83:0/3308563176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc100780 msgr2=0x7f59bc0737d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:37.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.365+0000 7f59c3200640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc100780 0x7f59bc0737d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f59b4005e30 tx=0x7f59b4003970 comp rx=0 tx=0).stop 2024-04-23T14:53:37.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.365+0000 7f59c3200640 1 -- 172.21.15.83:0/3308563176 shutdown_connections 2024-04-23T14:53:37.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.365+0000 7f59c3200640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59bc073e20 0x7f59bc06cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.365+0000 7f59c3200640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f59a00a3ef0 0x7f59a00a63b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.365+0000 7f59c3200640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59bc107390 0x7f59bc06c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.365+0000 7f59c3200640 1 --2- 172.21.15.83:0/3308563176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59bc100780 0x7f59bc0737d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:37.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.365+0000 7f59c3200640 1 -- 172.21.15.83:0/3308563176 >> 172.21.15.83:0/3308563176 conn(0x7f59bc0774a0 msgr2=0x7f59bc1054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:37.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.367+0000 7f59c3200640 1 -- 172.21.15.83:0/3308563176 shutdown_connections 2024-04-23T14:53:37.367 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:37.367+0000 7f59c3200640 1 -- 172.21.15.83:0/3308563176 wait complete. 2024-04-23T14:53:38.398 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:53:38.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:53:38.399 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:53:38.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:38 smithi203 ceph-mon[32800]: pgmap v158: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 71 MiB/s wr, 12.96k op/s 2024-04-23T14:53:38.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:38 smithi083 ceph-mon[27867]: pgmap v158: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 71 MiB/s wr, 12.96k op/s 2024-04-23T14:53:38.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:38 smithi173 ceph-mon[32443]: pgmap v158: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 71 MiB/s wr, 12.96k op/s 2024-04-23T14:53:39.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.993+0000 7fc135a00640 1 -- 172.21.15.83:0/1547537649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1301044b0 msgr2=0x7fc130104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:39.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.993+0000 7fc135a00640 1 --2- 172.21.15.83:0/1547537649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1301044b0 0x7fc130104890 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc120009ca0 tx=0x7fc12002a8c0 comp rx=0 tx=0).stop 2024-04-23T14:53:39.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.993+0000 7fc135a00640 1 -- 172.21.15.83:0/1547537649 shutdown_connections 2024-04-23T14:53:39.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.993+0000 7fc135a00640 1 --2- 172.21.15.83:0/1547537649 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc130109580 0x7fc130111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:39.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.993+0000 7fc135a00640 1 --2- 172.21.15.83:0/1547537649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc130104e60 0x7fc130108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:39.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.993+0000 7fc135a00640 1 --2- 172.21.15.83:0/1547537649 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc1301044b0 0x7fc130104890 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:39.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.993+0000 7fc135a00640 1 -- 172.21.15.83:0/1547537649 >> 172.21.15.83:0/1547537649 conn(0x7fc1300ffd90 msgr2=0x7fc1301021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:39.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.993+0000 7fc135a00640 1 -- 172.21.15.83:0/1547537649 shutdown_connections 2024-04-23T14:53:39.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.993+0000 7fc135a00640 1 -- 172.21.15.83:0/1547537649 wait complete. 2024-04-23T14:53:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.994+0000 7fc135a00640 1 Processor -- start 2024-04-23T14:53:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.994+0000 7fc135a00640 1 -- start start 2024-04-23T14:53:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.995+0000 7fc135a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301044b0 0x7fc1301a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.995+0000 7fc135a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc130109580 0x7fc1301a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.995+0000 7fc135a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc13019a9e0 0x7fc13019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:39.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.995+0000 7fc135a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc130114640 con 0x7fc1301044b0 2024-04-23T14:53:39.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.995+0000 7fc135a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301144c0 con 0x7fc13019a9e0 2024-04-23T14:53:39.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.995+0000 7fc135a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301147c0 con 0x7fc130109580 2024-04-23T14:53:39.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.995+0000 7fc12ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301044b0 0x7fc1301a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:39.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.995+0000 7fc12ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301044b0 0x7fc1301a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51988/0 (socket says 172.21.15.83:51988) 2024-04-23T14:53:39.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.995+0000 7fc12ea00640 1 -- 172.21.15.83:0/2552528249 learned_addr learned my addr 172.21.15.83:0/2552528249 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:39.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.995+0000 7fc12e000640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc130109580 0x7fc1301a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:39.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.995+0000 7fc12f400640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc13019a9e0 0x7fc13019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:39.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.996+0000 7fc12e000640 1 -- 172.21.15.83:0/2552528249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc13019a9e0 msgr2=0x7fc13019ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:39.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.996+0000 7fc12e000640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc13019a9e0 0x7fc13019ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:39.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.996+0000 7fc12e000640 1 -- 172.21.15.83:0/2552528249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301044b0 msgr2=0x7fc1301a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:39.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.996+0000 7fc12e000640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301044b0 0x7fc1301a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:39.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.996+0000 7fc12e000640 1 -- 172.21.15.83:0/2552528249 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13019b500 con 0x7fc130109580 2024-04-23T14:53:39.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.996+0000 7fc12ea00640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301044b0 0x7fc1301a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:39.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.996+0000 7fc12e000640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc130109580 0x7fc1301a0d20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc11800e9c0 tx=0x7fc11800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:39.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.996+0000 7fc127400640 1 -- 172.21.15.83:0/2552528249 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11800ccc0 con 0x7fc130109580 2024-04-23T14:53:39.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.996+0000 7fc127400640 1 -- 172.21.15.83:0/2552528249 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc118010910 con 0x7fc130109580 2024-04-23T14:53:40.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.996+0000 7fc135a00640 1 -- 172.21.15.83:0/2552528249 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13019b7f0 con 0x7fc130109580 2024-04-23T14:53:40.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.996+0000 7fc127400640 1 -- 172.21.15.83:0/2552528249 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1180188c0 con 0x7fc130109580 2024-04-23T14:53:40.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:39.997+0000 7fc135a00640 1 -- 172.21.15.83:0/2552528249 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1301a7a40 con 0x7fc130109580 2024-04-23T14:53:40.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.000+0000 7fc138775740 1 Processor -- start 2024-04-23T14:53:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.000+0000 7fc138775740 1 -- start start 2024-04-23T14:53:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.000+0000 7fc127400640 1 -- 172.21.15.83:0/2552528249 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc118010ac0 con 0x7fc130109580 2024-04-23T14:53:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.000+0000 7fc138775740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 0x55ae8578e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.000+0000 7fc138775740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e9b0 0x55ae85798b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.000+0000 7fc138775740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae85799080 0x55ae8579b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.000+0000 7fc138775740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ae856f3e70 con 0x55ae8578e090 2024-04-23T14:53:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.000+0000 7fc138775740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ae8569a380 con 0x55ae8578e9b0 2024-04-23T14:53:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.000+0000 7fc138775740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ae856efe30 con 0x55ae85799080 2024-04-23T14:53:40.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.000+0000 7fc12f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 0x55ae8578e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.000+0000 7fc12f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 0x55ae8578e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51990/0 (socket says 172.21.15.83:51990) 2024-04-23T14:53:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.000+0000 7fc12f400640 1 -- 172.21.15.83:0/1647601852 learned_addr learned my addr 172.21.15.83:0/1647601852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.001+0000 7fc12ea00640 1 --2- 172.21.15.83:0/1647601852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e9b0 0x55ae85798b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.001+0000 7fc12e000640 1 --2- 172.21.15.83:0/1647601852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae85799080 0x55ae8579b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.001+0000 7fc12f400640 1 -- 172.21.15.83:0/1647601852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae85799080 msgr2=0x55ae8579b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.001+0000 7fc12f400640 1 --2- 172.21.15.83:0/1647601852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae85799080 0x55ae8579b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.001+0000 7fc12f400640 1 -- 172.21.15.83:0/1647601852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e9b0 msgr2=0x55ae85798b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.001+0000 7fc12f400640 1 --2- 172.21.15.83:0/1647601852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e9b0 0x55ae85798b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.001+0000 7fc12f400640 1 -- 172.21.15.83:0/1647601852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae854ce7e0 con 0x55ae8578e090 2024-04-23T14:53:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.001+0000 7fc12ea00640 1 --2- 172.21.15.83:0/1647601852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e9b0 0x55ae85798b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:53:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.001+0000 7fc12f400640 1 --2- 172.21.15.83:0/1647601852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 0x55ae8578e470 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc128009c20 tx=0x7fc128011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba677d53d70f02ad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.001+0000 7fc11fe00640 1 -- 172.21.15.83:0/1647601852 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12801e070 con 0x55ae8578e090 2024-04-23T14:53:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.002+0000 7fc11fe00640 1 -- 172.21.15.83:0/1647601852 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc128019b10 con 0x55ae8578e090 2024-04-23T14:53:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.002+0000 7fc11fe00640 1 -- 172.21.15.83:0/1647601852 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12800eaa0 con 0x55ae8578e090 2024-04-23T14:53:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.002+0000 7fc127400640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1140a3da0 0x7fc1140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.002+0000 7fc127400640 1 -- 172.21.15.83:0/2552528249 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc1180d0b40 con 0x7fc130109580 2024-04-23T14:53:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.002+0000 7fc12ea00640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1140a3da0 0x7fc1140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.002+0000 7fc138775740 1 -- 172.21.15.83:0/1647601852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 msgr2=0x55ae8578e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.002+0000 7fc138775740 1 --2- 172.21.15.83:0/1647601852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 0x55ae8578e470 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc128009c20 tx=0x7fc128011810 comp rx=0 tx=0).stop 2024-04-23T14:53:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.002+0000 7fc138775740 1 -- 172.21.15.83:0/1647601852 shutdown_connections 2024-04-23T14:53:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.002+0000 7fc138775740 1 --2- 172.21.15.83:0/1647601852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae85799080 0x55ae8579b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.002+0000 7fc138775740 1 --2- 172.21.15.83:0/1647601852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e9b0 0x55ae85798b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.002+0000 7fc138775740 1 --2- 172.21.15.83:0/1647601852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 0x55ae8578e470 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.002+0000 7fc138775740 1 -- 172.21.15.83:0/1647601852 >> 172.21.15.83:0/1647601852 conn(0x55ae8578a3b0 msgr2=0x55ae8578a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.003+0000 7fc138775740 1 -- 172.21.15.83:0/1647601852 shutdown_connections 2024-04-23T14:53:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.003+0000 7fc12ea00640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1140a3da0 0x7fc1140a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55ae8578f650 tx=0x7fc12003d040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.005+0000 7fc138775740 1 -- 172.21.15.83:0/1647601852 wait complete. 2024-04-23T14:53:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.005+0000 7fc138775740 1 -- start start 2024-04-23T14:53:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.005+0000 7fc138775740 1 Processor -- start 2024-04-23T14:53:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc138775740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 0x55ae85796020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc138775740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae85796560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc138775740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 0x55ae85796aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc138775740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ae8579be70 con 0x55ae85799080 2024-04-23T14:53:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc138775740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ae856ad660 con 0x55ae8578e090 2024-04-23T14:53:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc138775740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ae8579bff0 con 0x55ae8578e9b0 2024-04-23T14:53:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc138775740 10 client.0 _ll_register_callbacks cb 0x55ae8569baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc12ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 0x55ae85796020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc12ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 0x55ae85796020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37266/0 (socket says 172.21.15.83:37266) 2024-04-23T14:53:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc12ea00640 1 -- 172.21.15.83:0/959736185 learned_addr learned my addr 172.21.15.83:0/959736185 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:40.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc12e000640 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae85796560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc12f400640 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 0x55ae85796aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc12ea00640 1 -- 172.21.15.83:0/959736185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 msgr2=0x55ae85796560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc12ea00640 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae85796560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc12ea00640 1 -- 172.21.15.83:0/959736185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 msgr2=0x55ae85796aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc12ea00640 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 0x55ae85796aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc12ea00640 1 -- 172.21.15.83:0/959736185 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae8552ef70 con 0x55ae8578e090 2024-04-23T14:53:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc12e000640 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae85796560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:40.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.006+0000 7fc12f400640 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 0x55ae85796aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:40.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.007+0000 7fc12ea00640 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 0x55ae85796020 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc12002ade0 tx=0x7fc12002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16a5350595e37cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:40.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.007+0000 7fc11cc00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:53:40.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.007+0000 7fc11cc00640 1 -- 172.21.15.83:0/959736185 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120056060 con 0x55ae8578e090 2024-04-23T14:53:40.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.007+0000 7fc11cc00640 1 -- 172.21.15.83:0/959736185 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc1200307d0 con 0x55ae8578e090 2024-04-23T14:53:40.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.007+0000 7fc138775740 1 -- 172.21.15.83:0/959736185 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc128019b10 con 0x55ae8578e090 2024-04-23T14:53:40.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.007+0000 7fc11cc00640 1 -- 172.21.15.83:0/959736185 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc1200309d0 con 0x55ae8578e090 2024-04-23T14:53:40.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.007+0000 7fc138775740 10 client.26267 fetch_fsmap learned FSMap version 29 2024-04-23T14:53:40.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.007+0000 7fc138775740 1 -- 172.21.15.83:0/959736185 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae853c6170 con 0x55ae8578e090 2024-04-23T14:53:40.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.008+0000 7fc11cc00640 1 -- 172.21.15.83:0/959736185 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc128019b10 con 0x55ae8578e090 2024-04-23T14:53:40.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.008+0000 7fc11cc00640 1 -- 172.21.15.83:0/959736185 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fc120046b90 con 0x55ae8578e090 2024-04-23T14:53:40.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.008+0000 7fc138775740 10 client.26267 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:53:40.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.008+0000 7fc138775740 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:53:40.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.008+0000 7fc138775740 20 client.26267 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:40.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.008+0000 7fc138775740 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae857a9450 0x55ae857ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.008+0000 7fc138775740 4 client.26267 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:53:40.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.008+0000 7fc138775740 1 -- 172.21.15.83:0/959736185 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae85701890 con 0x55ae857a9450 2024-04-23T14:53:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.008+0000 7fc12e000640 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae857a9450 0x55ae857ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.009+0000 7fc12e000640 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae857a9450 0x55ae857ab8f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c598dda457b9875 server_cookie=b84740bababb3145 in_seq=0 out_seq=0 2024-04-23T14:53:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.009+0000 7fc11cc00640 10 client.26267 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc11cc00640 1 -- 172.21.15.83:0/959736185 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae85701890 con 0x55ae857a9450 2024-04-23T14:53:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc11cc00640 10 client.26267 handle_command_reply: tid=0 2024-04-23T14:53:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc138775740 1 client.26267 shutdown 2024-04-23T14:53:40.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc138775740 1 -- 172.21.15.83:0/959736185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 msgr2=0x55ae85796020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc138775740 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 0x55ae85796020 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc12002ade0 tx=0x7fc12002a8c0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc138775740 1 -- 172.21.15.83:0/959736185 shutdown_connections 2024-04-23T14:53:40.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc138775740 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae857a9450 0x55ae857ab8f0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc138775740 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 0x55ae85796aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc138775740 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae85796560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc138775740 1 --2- 172.21.15.83:0/959736185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 0x55ae85796020 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc11cc00640 0 client.26267 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:40.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc138775740 1 -- 172.21.15.83:0/959736185 >> 172.21.15.83:0/959736185 conn(0x55ae8578a3b0 msgr2=0x55ae857900b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:40.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc138775740 1 -- 172.21.15.83:0/959736185 shutdown_connections 2024-04-23T14:53:40.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc138775740 1 -- 172.21.15.83:0/959736185 wait complete. 2024-04-23T14:53:40.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.010+0000 7fc138775740 20 client.26267 trim_cache size 0 max 16384 2024-04-23T14:53:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.029+0000 7fc138775740 1 Processor -- start 2024-04-23T14:53:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.030+0000 7fc138775740 1 -- start start 2024-04-23T14:53:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.030+0000 7fc138775740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 0x55ae85789c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.030+0000 7fc138775740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae857a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.030+0000 7fc138775740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae85799080 0x55ae857966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.030+0000 7fc138775740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301b28a0 con 0x55ae8578e090 2024-04-23T14:53:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.030+0000 7fc138775740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301b0170 con 0x55ae85799080 2024-04-23T14:53:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.030+0000 7fc138775740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301adfc0 con 0x55ae8578e9b0 2024-04-23T14:53:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.030+0000 7fc12ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 0x55ae85789c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.030+0000 7fc12ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 0x55ae85789c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52006/0 (socket says 172.21.15.83:52006) 2024-04-23T14:53:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.030+0000 7fc12ea00640 1 -- 172.21.15.83:0/3721812062 learned_addr learned my addr 172.21.15.83:0/3721812062 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:40.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.030+0000 7fc12f400640 1 --2- 172.21.15.83:0/3721812062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae85799080 0x55ae857966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.030+0000 7fc12e000640 1 --2- 172.21.15.83:0/3721812062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae857a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.031+0000 7fc12ea00640 1 -- 172.21.15.83:0/3721812062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 msgr2=0x55ae857a4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.031+0000 7fc12ea00640 1 --2- 172.21.15.83:0/3721812062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae857a4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.031+0000 7fc12ea00640 1 -- 172.21.15.83:0/3721812062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae85799080 msgr2=0x55ae857966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.031+0000 7fc12ea00640 1 --2- 172.21.15.83:0/3721812062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae85799080 0x55ae857966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.031+0000 7fc12ea00640 1 -- 172.21.15.83:0/3721812062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae85785d10 con 0x55ae8578e090 2024-04-23T14:53:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.031+0000 7fc12ea00640 1 --2- 172.21.15.83:0/3721812062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 0x55ae85789c90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc120009ba0 tx=0x7fc120009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88556e33204a4ca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:40.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.031+0000 7fc11f400640 1 -- 172.21.15.83:0/3721812062 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120056070 con 0x55ae8578e090 2024-04-23T14:53:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.031+0000 7fc11f400640 1 -- 172.21.15.83:0/3721812062 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc120036040 con 0x55ae8578e090 2024-04-23T14:53:40.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.031+0000 7fc11f400640 1 -- 172.21.15.83:0/3721812062 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120046410 con 0x55ae8578e090 2024-04-23T14:53:40.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.032+0000 7fc138775740 1 -- 172.21.15.83:0/3721812062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 msgr2=0x55ae85789c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.032+0000 7fc138775740 1 --2- 172.21.15.83:0/3721812062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 0x55ae85789c90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc120009ba0 tx=0x7fc120009360 comp rx=0 tx=0).stop 2024-04-23T14:53:40.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.032+0000 7fc138775740 1 -- 172.21.15.83:0/3721812062 shutdown_connections 2024-04-23T14:53:40.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.032+0000 7fc138775740 1 --2- 172.21.15.83:0/3721812062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae85799080 0x55ae857966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.032+0000 7fc138775740 1 --2- 172.21.15.83:0/3721812062 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae857a4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.032+0000 7fc138775740 1 --2- 172.21.15.83:0/3721812062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae8578e090 0x55ae85789c90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.032+0000 7fc138775740 1 -- 172.21.15.83:0/3721812062 >> 172.21.15.83:0/3721812062 conn(0x55ae8578a3b0 msgr2=0x55ae85795fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.032+0000 7fc138775740 1 -- 172.21.15.83:0/3721812062 shutdown_connections 2024-04-23T14:53:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.032+0000 7fc138775740 1 -- 172.21.15.83:0/3721812062 wait complete. 2024-04-23T14:53:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.032+0000 7fc138775740 1 -- start start 2024-04-23T14:53:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.032+0000 7fc138775740 1 Processor -- start 2024-04-23T14:53:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.033+0000 7fc138775740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 0x55ae85792f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.033+0000 7fc138775740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae85793440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.033+0000 7fc138775740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 0x55ae85793980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.033+0000 7fc138775740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301ae630 con 0x55ae85799080 2024-04-23T14:53:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.033+0000 7fc138775740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301afaa0 con 0x55ae8578e090 2024-04-23T14:53:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.033+0000 7fc138775740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc1301aed00 con 0x55ae8578e9b0 2024-04-23T14:53:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.033+0000 7fc138775740 10 client.0 _ll_register_callbacks cb 0x7fc128005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.033+0000 7fc12f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 0x55ae85793980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.033+0000 7fc12f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 0x55ae85793980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52010/0 (socket says 172.21.15.83:52010) 2024-04-23T14:53:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.033+0000 7fc12f400640 1 -- 172.21.15.83:0/2077977614 learned_addr learned my addr 172.21.15.83:0/2077977614 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.033+0000 7fc12ea00640 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 0x55ae85792f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.033+0000 7fc12e000640 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae85793440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.034+0000 7fc12f400640 1 -- 172.21.15.83:0/2077977614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 msgr2=0x55ae85793440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.034+0000 7fc12f400640 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae85793440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.034+0000 7fc12f400640 1 -- 172.21.15.83:0/2077977614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 msgr2=0x55ae85792f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.034+0000 7fc12f400640 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 0x55ae85792f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.034+0000 7fc12f400640 1 -- 172.21.15.83:0/2077977614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae85817440 con 0x55ae85799080 2024-04-23T14:53:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.034+0000 7fc12ea00640 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 0x55ae85792f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.034+0000 7fc12f400640 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 0x55ae85793980 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc1280058c0 tx=0x7fc128009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2f86f661cdd867 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.034+0000 7fc11cc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:53:40.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.034+0000 7fc11cc00640 1 -- 172.21.15.83:0/2077977614 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12801e040 con 0x55ae85799080 2024-04-23T14:53:40.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.034+0000 7fc11cc00640 1 -- 172.21.15.83:0/2077977614 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc128004880 con 0x55ae85799080 2024-04-23T14:53:40.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.034+0000 7fc138775740 1 -- 172.21.15.83:0/2077977614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc120036040 con 0x55ae85799080 2024-04-23T14:53:40.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.035+0000 7fc11cc00640 1 -- 172.21.15.83:0/2077977614 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc128034050 con 0x55ae85799080 2024-04-23T14:53:40.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.035+0000 7fc138775740 10 client.16059 fetch_fsmap learned FSMap version 29 2024-04-23T14:53:40.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.035+0000 7fc138775740 1 -- 172.21.15.83:0/2077977614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae85818760 con 0x55ae85799080 2024-04-23T14:53:40.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.035+0000 7fc11cc00640 1 -- 172.21.15.83:0/2077977614 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc120036040 con 0x55ae85799080 2024-04-23T14:53:40.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.035+0000 7fc11cc00640 1 -- 172.21.15.83:0/2077977614 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fc128025480 con 0x55ae85799080 2024-04-23T14:53:40.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.035+0000 7fc138775740 10 client.16059 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:53:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.035+0000 7fc138775740 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:53:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.035+0000 7fc138775740 20 client.16059 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.035+0000 7fc138775740 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae85818d40 0x55ae8581b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.035+0000 7fc138775740 4 client.16059 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:53:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.035+0000 7fc138775740 1 -- 172.21.15.83:0/2077977614 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ae856fb700 con 0x55ae85818d40 2024-04-23T14:53:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.036+0000 7fc12ea00640 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae85818d40 0x55ae8581b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.036+0000 7fc12ea00640 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae85818d40 0x55ae8581b160 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a02c6c6f086ad73 server_cookie=7ad152dc9deb12d in_seq=0 out_seq=0 2024-04-23T14:53:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.036+0000 7fc11cc00640 10 client.16059 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:40.044 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.037+0000 7fc11cc00640 1 -- 172.21.15.83:0/2077977614 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ae856fb700 con 0x55ae85818d40 2024-04-23T14:53:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.037+0000 7fc11cc00640 10 client.16059 handle_command_reply: tid=0 2024-04-23T14:53:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.037+0000 7fc138775740 1 client.16059 shutdown 2024-04-23T14:53:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.037+0000 7fc138775740 1 -- 172.21.15.83:0/2077977614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 msgr2=0x55ae85793980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.037+0000 7fc138775740 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 0x55ae85793980 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc1280058c0 tx=0x7fc128009f40 comp rx=0 tx=0).stop 2024-04-23T14:53:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.038+0000 7fc138775740 1 -- 172.21.15.83:0/2077977614 shutdown_connections 2024-04-23T14:53:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.038+0000 7fc138775740 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ae85818d40 0x55ae8581b160 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.038+0000 7fc138775740 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ae85799080 0x55ae85793980 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.038+0000 7fc138775740 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ae8578e9b0 0x55ae85793440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.045 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.038+0000 7fc138775740 1 --2- 172.21.15.83:0/2077977614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ae8578e090 0x55ae85792f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.038+0000 7fc138775740 1 -- 172.21.15.83:0/2077977614 >> 172.21.15.83:0/2077977614 conn(0x55ae8578a3b0 msgr2=0x55ae85796870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:40.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.038+0000 7fc11cc00640 0 client.16059 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:40.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.038+0000 7fc138775740 1 -- 172.21.15.83:0/2077977614 shutdown_connections 2024-04-23T14:53:40.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.038+0000 7fc138775740 1 -- 172.21.15.83:0/2077977614 wait complete. 2024-04-23T14:53:40.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.038+0000 7fc138775740 20 client.16059 trim_cache size 0 max 16384 2024-04-23T14:53:40.046 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:53:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:53:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "9c49bdae-8161-4000-bd27-d1d1964c7f43", 2024-04-23T14:53:40.047 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:53:40.047 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:53:40.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.039+0000 7fc135a00640 1 -- 172.21.15.83:0/2552528249 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1140a3da0 msgr2=0x7fc1140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.039+0000 7fc135a00640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1140a3da0 0x7fc1140a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55ae8578f650 tx=0x7fc12003d040 comp rx=0 tx=0).stop 2024-04-23T14:53:40.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.039+0000 7fc135a00640 1 -- 172.21.15.83:0/2552528249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc130109580 msgr2=0x7fc1301a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.039+0000 7fc135a00640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc130109580 0x7fc1301a0d20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc11800e9c0 tx=0x7fc11800ee80 comp rx=0 tx=0).stop 2024-04-23T14:53:40.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.039+0000 7fc135a00640 1 -- 172.21.15.83:0/2552528249 shutdown_connections 2024-04-23T14:53:40.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.039+0000 7fc135a00640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc13019a9e0 0x7fc13019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.039+0000 7fc135a00640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc130109580 0x7fc1301a0d20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.039+0000 7fc135a00640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc1140a3da0 0x7fc1140a6260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.039+0000 7fc135a00640 1 --2- 172.21.15.83:0/2552528249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301044b0 0x7fc1301a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.039+0000 7fc135a00640 1 -- 172.21.15.83:0/2552528249 >> 172.21.15.83:0/2552528249 conn(0x7fc1300ffd90 msgr2=0x7fc1301010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:40.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.039+0000 7fc135a00640 1 -- 172.21.15.83:0/2552528249 shutdown_connections 2024-04-23T14:53:40.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.039+0000 7fc135a00640 1 -- 172.21.15.83:0/2552528249 wait complete. 2024-04-23T14:53:40.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:53:40.059 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:53:40.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.257+0000 7f0e34c00640 1 -- 172.21.15.83:0/2991730826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3006a660 msgr2=0x7f0e3006aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.257+0000 7f0e34c00640 1 --2- 172.21.15.83:0/2991730826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3006a660 0x7f0e3006aa40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009900 tx=0x7f0e2002a3f0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.257+0000 7f0e34c00640 1 -- 172.21.15.83:0/2991730826 shutdown_connections 2024-04-23T14:53:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.257+0000 7f0e34c00640 1 --2- 172.21.15.83:0/2991730826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e3010ea10 0x7f0e30110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.257+0000 7f0e34c00640 1 --2- 172.21.15.83:0/2991730826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3006af80 0x7f0e3010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.257+0000 7f0e34c00640 1 --2- 172.21.15.83:0/2991730826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3006a660 0x7f0e3006aa40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.257+0000 7f0e34c00640 1 -- 172.21.15.83:0/2991730826 >> 172.21.15.83:0/2991730826 conn(0x7f0e30077f60 msgr2=0x7f0e30078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.258+0000 7f0e34c00640 1 -- 172.21.15.83:0/2991730826 shutdown_connections 2024-04-23T14:53:40.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.258+0000 7f0e34c00640 1 -- 172.21.15.83:0/2991730826 wait complete. 2024-04-23T14:53:40.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.258+0000 7f0e34c00640 1 Processor -- start 2024-04-23T14:53:40.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.259+0000 7f0e34c00640 1 -- start start 2024-04-23T14:53:40.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.259+0000 7f0e34c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e3006a660 0x7f0e301a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.259+0000 7f0e34c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3006af80 0x7f0e301a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.259+0000 7f0e34c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3010ea10 0x7f0e3019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.259+0000 7f0e34c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e30114630 con 0x7f0e3010ea10 2024-04-23T14:53:40.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.259+0000 7f0e34c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e301144b0 con 0x7f0e3006af80 2024-04-23T14:53:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.259+0000 7f0e34c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0e301147b0 con 0x7f0e3006a660 2024-04-23T14:53:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.259+0000 7f0e2e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3010ea10 0x7f0e3019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.259+0000 7f0e2de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e3006a660 0x7f0e301a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.259+0000 7f0e2de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e3006a660 0x7f0e301a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:55468/0 (socket says 172.21.15.83:55468) 2024-04-23T14:53:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.259+0000 7f0e2de00640 1 -- 172.21.15.83:0/883272134 learned_addr learned my addr 172.21.15.83:0/883272134 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.259+0000 7f0e27e00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3006af80 0x7f0e301a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.260+0000 7f0e2de00640 1 -- 172.21.15.83:0/883272134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3006af80 msgr2=0x7f0e301a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.260+0000 7f0e2de00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3006af80 0x7f0e301a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.260+0000 7f0e2de00640 1 -- 172.21.15.83:0/883272134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3010ea10 msgr2=0x7f0e3019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.260+0000 7f0e2de00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3010ea10 0x7f0e3019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.260+0000 7f0e2de00640 1 -- 172.21.15.83:0/883272134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3019b0d0 con 0x7f0e3006a660 2024-04-23T14:53:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.260+0000 7f0e2e800640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3010ea10 0x7f0e3019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.260+0000 7f0e2de00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e3006a660 0x7f0e301a07a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0e20004770 tx=0x7f0e20003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.261+0000 7f0e26a00640 1 -- 172.21.15.83:0/883272134 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2003f070 con 0x7f0e3006a660 2024-04-23T14:53:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.261+0000 7f0e26a00640 1 -- 172.21.15.83:0/883272134 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e20031ca0 con 0x7f0e3006a660 2024-04-23T14:53:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.261+0000 7f0e26a00640 1 -- 172.21.15.83:0/883272134 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20035030 con 0x7f0e3006a660 2024-04-23T14:53:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.261+0000 7f0e34c00640 1 -- 172.21.15.83:0/883272134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3019b360 con 0x7f0e3006a660 2024-04-23T14:53:40.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.261+0000 7f0e34c00640 1 -- 172.21.15.83:0/883272134 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e301a7660 con 0x7f0e3006a660 2024-04-23T14:53:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.264+0000 7f0e379f9740 1 Processor -- start 2024-04-23T14:53:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.264+0000 7f0e379f9740 1 -- start start 2024-04-23T14:53:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.264+0000 7f0e379f9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af858c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.264+0000 7f0e379f9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af859180 0x55d1af863310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.264+0000 7f0e379f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 0x55d1af865c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.264+0000 7f0e379f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d1af7be1f0 con 0x55d1af863850 2024-04-23T14:53:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.264+0000 7f0e379f9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d1af764350 con 0x55d1af858860 2024-04-23T14:53:40.265 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.264+0000 7f0e379f9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d1af7ba1b0 con 0x55d1af859180 2024-04-23T14:53:40.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.265+0000 7f0e27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 0x55d1af865c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.265+0000 7f0e2e800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af858c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.265+0000 7f0e27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 0x55d1af865c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52040/0 (socket says 172.21.15.83:52040) 2024-04-23T14:53:40.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.265+0000 7f0e27e00640 1 -- 172.21.15.83:0/3236008908 learned_addr learned my addr 172.21.15.83:0/3236008908 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:40.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.265+0000 7f0e2e800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af858c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37318/0 (socket says 172.21.15.83:37318) 2024-04-23T14:53:40.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.265+0000 7f0e27e00640 1 -- 172.21.15.83:0/3236008908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af859180 msgr2=0x55d1af863310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:53:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.265+0000 7f0e27e00640 1 --2- 172.21.15.83:0/3236008908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af859180 0x55d1af863310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.265+0000 7f0e27e00640 1 -- 172.21.15.83:0/3236008908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 msgr2=0x55d1af858c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.265+0000 7f0e27e00640 1 --2- 172.21.15.83:0/3236008908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af858c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.265+0000 7f0e27e00640 1 -- 172.21.15.83:0/3236008908 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1af7b8f50 con 0x55d1af863850 2024-04-23T14:53:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.265+0000 7f0e26a00640 1 -- 172.21.15.83:0/883272134 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0e20042070 con 0x7f0e3006a660 2024-04-23T14:53:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.266+0000 7f0e27e00640 1 --2- 172.21.15.83:0/3236008908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 0x55d1af865c40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009c60 tx=0x7f0e18011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511815fdc438af34 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:40.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.266+0000 7f0e1f000640 1 -- 172.21.15.83:0/3236008908 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e18004230 con 0x55d1af863850 2024-04-23T14:53:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.266+0000 7f0e1f000640 1 -- 172.21.15.83:0/3236008908 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e18009e10 con 0x55d1af863850 2024-04-23T14:53:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.266+0000 7f0e1f000640 1 -- 172.21.15.83:0/3236008908 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e180198a0 con 0x55d1af863850 2024-04-23T14:53:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e379f9740 1 -- 172.21.15.83:0/3236008908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 msgr2=0x55d1af865c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e379f9740 1 --2- 172.21.15.83:0/3236008908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 0x55d1af865c40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009c60 tx=0x7f0e18011810 comp rx=0 tx=0).stop 2024-04-23T14:53:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e379f9740 1 -- 172.21.15.83:0/3236008908 shutdown_connections 2024-04-23T14:53:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e379f9740 1 --2- 172.21.15.83:0/3236008908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 0x55d1af865c40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e379f9740 1 --2- 172.21.15.83:0/3236008908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af859180 0x55d1af863310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e379f9740 1 --2- 172.21.15.83:0/3236008908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af858c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e379f9740 1 -- 172.21.15.83:0/3236008908 >> 172.21.15.83:0/3236008908 conn(0x55d1af854b80 msgr2=0x55d1af854f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e379f9740 1 -- 172.21.15.83:0/3236008908 shutdown_connections 2024-04-23T14:53:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e379f9740 1 -- 172.21.15.83:0/3236008908 wait complete. 2024-04-23T14:53:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e26a00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0e140a3cb0 0x7f0e140a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e26a00640 1 -- 172.21.15.83:0/883272134 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f0e200f0cf0 con 0x7f0e3006a660 2024-04-23T14:53:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e27e00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0e140a3cb0 0x7f0e140a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.267+0000 7f0e379f9740 1 -- start start 2024-04-23T14:53:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.268+0000 7f0e379f9740 1 Processor -- start 2024-04-23T14:53:40.274 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.268+0000 7f0e27e00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0e140a3cb0 0x7f0e140a6170 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e18003fa0 tx=0x7f0e180193e0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.268+0000 7f0e379f9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af860800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.268+0000 7f0e379f9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af859180 0x55d1af860d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.268+0000 7f0e379f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 0x55d1af861280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.268+0000 7f0e379f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d1af866640 con 0x55d1af863850 2024-04-23T14:53:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.268+0000 7f0e379f9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d1af777690 con 0x55d1af858860 2024-04-23T14:53:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.268+0000 7f0e379f9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d1af8667c0 con 0x55d1af859180 2024-04-23T14:53:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e379f9740 10 client.0 _ll_register_callbacks cb 0x55d1af7b3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:40.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e2e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 0x55d1af861280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e2de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af860800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e2de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af860800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37320/0 (socket says 172.21.15.83:37320) 2024-04-23T14:53:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e2e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 0x55d1af861280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52042/0 (socket says 172.21.15.83:52042) 2024-04-23T14:53:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e2de00640 1 -- 172.21.15.83:0/2767079222 learned_addr learned my addr 172.21.15.83:0/2767079222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e27e00640 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af859180 0x55d1af860d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e2de00640 1 -- 172.21.15.83:0/2767079222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af859180 msgr2=0x55d1af860d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e2de00640 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af859180 0x55d1af860d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e2de00640 1 -- 172.21.15.83:0/2767079222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 msgr2=0x55d1af861280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e2de00640 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 0x55d1af861280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e2de00640 1 -- 172.21.15.83:0/2767079222 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1af776920 con 0x55d1af858860 2024-04-23T14:53:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.269+0000 7f0e2e800640 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 0x55d1af861280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.270+0000 7f0e2de00640 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af860800 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e200eea50 tx=0x7f0e200bcca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9320b6c144fc3ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.270+0000 7f0e13400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:53:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.270+0000 7f0e13400640 1 -- 172.21.15.83:0/2767079222 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e200db060 con 0x55d1af858860 2024-04-23T14:53:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.270+0000 7f0e13400640 1 -- 172.21.15.83:0/2767079222 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f0e200b9480 con 0x55d1af858860 2024-04-23T14:53:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.270+0000 7f0e379f9740 1 -- 172.21.15.83:0/2767079222 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e18009e10 con 0x55d1af858860 2024-04-23T14:53:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.270+0000 7f0e13400640 1 -- 172.21.15.83:0/2767079222 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0e200ae050 con 0x55d1af858860 2024-04-23T14:53:40.277 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.270+0000 7f0e379f9740 10 client.26285 fetch_fsmap learned FSMap version 29 2024-04-23T14:53:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.270+0000 7f0e379f9740 1 -- 172.21.15.83:0/2767079222 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d1af59a7d0 con 0x55d1af858860 2024-04-23T14:53:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.270+0000 7f0e13400640 1 -- 172.21.15.83:0/2767079222 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e18009e10 con 0x55d1af858860 2024-04-23T14:53:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.271+0000 7f0e13400640 1 -- 172.21.15.83:0/2767079222 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0e2003b4f0 con 0x55d1af858860 2024-04-23T14:53:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.271+0000 7f0e379f9740 10 client.26285 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:53:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.271+0000 7f0e379f9740 10 client.26285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:53:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.271+0000 7f0e379f9740 20 client.26285 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.271+0000 7f0e379f9740 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d1af873bd0 0x55d1af876020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.278 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.271+0000 7f0e379f9740 4 client.26285 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:53:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.271+0000 7f0e379f9740 1 -- 172.21.15.83:0/2767079222 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1af7cbc10 con 0x55d1af873bd0 2024-04-23T14:53:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.271+0000 7f0e27e00640 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d1af873bd0 0x55d1af876020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.272+0000 7f0e27e00640 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d1af873bd0 0x55d1af876020 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce3244f553def3c server_cookie=35a0afb0ba5b1679 in_seq=0 out_seq=0 2024-04-23T14:53:40.279 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.272+0000 7f0e13400640 10 client.26285 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:40.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e13400640 1 -- 172.21.15.83:0/2767079222 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1af7cbc10 con 0x55d1af873bd0 2024-04-23T14:53:40.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e13400640 10 client.26285 handle_command_reply: tid=0 2024-04-23T14:53:40.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e379f9740 1 client.26285 shutdown 2024-04-23T14:53:40.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e379f9740 1 -- 172.21.15.83:0/2767079222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 msgr2=0x55d1af860800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.280 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e379f9740 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af860800 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e200eea50 tx=0x7f0e200bcca0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e379f9740 1 -- 172.21.15.83:0/2767079222 shutdown_connections 2024-04-23T14:53:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e379f9740 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d1af873bd0 0x55d1af876020 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e379f9740 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af863850 0x55d1af861280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e13400640 0 client.26285 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e379f9740 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af859180 0x55d1af860d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e379f9740 1 --2- 172.21.15.83:0/2767079222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af860800 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e379f9740 1 -- 172.21.15.83:0/2767079222 >> 172.21.15.83:0/2767079222 conn(0x55d1af854b80 msgr2=0x55d1af85a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:40.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e379f9740 1 -- 172.21.15.83:0/2767079222 shutdown_connections 2024-04-23T14:53:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.273+0000 7f0e379f9740 1 -- 172.21.15.83:0/2767079222 wait complete. 2024-04-23T14:53:40.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.274+0000 7f0e379f9740 20 client.26285 trim_cache size 0 max 16384 2024-04-23T14:53:40.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.292+0000 7f0e379f9740 1 Processor -- start 2024-04-23T14:53:40.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.293+0000 7f0e379f9740 1 -- start start 2024-04-23T14:53:40.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.293+0000 7f0e379f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af858860 0x55d1af85f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.293+0000 7f0e379f9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af859180 0x55d1af85bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.293+0000 7f0e379f9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af863850 0x55d1af85b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.293+0000 7f0e379f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e301b27a0 con 0x55d1af858860 2024-04-23T14:53:40.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.293+0000 7f0e379f9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e301b0070 con 0x55d1af859180 2024-04-23T14:53:40.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.293+0000 7f0e379f9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0e301adec0 con 0x55d1af863850 2024-04-23T14:53:40.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.293+0000 7f0e2de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af858860 0x55d1af85f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.293+0000 7f0e2de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af858860 0x55d1af85f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52058/0 (socket says 172.21.15.83:52058) 2024-04-23T14:53:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.293+0000 7f0e2de00640 1 -- 172.21.15.83:0/1694911403 learned_addr learned my addr 172.21.15.83:0/1694911403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.293+0000 7f0e2e800640 1 --2- 172.21.15.83:0/1694911403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af863850 0x55d1af85b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.294+0000 7f0e27e00640 1 --2- 172.21.15.83:0/1694911403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af859180 0x55d1af85bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.294+0000 7f0e2de00640 1 -- 172.21.15.83:0/1694911403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af863850 msgr2=0x55d1af85b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.294+0000 7f0e2de00640 1 --2- 172.21.15.83:0/1694911403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af863850 0x55d1af85b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.294+0000 7f0e2de00640 1 -- 172.21.15.83:0/1694911403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af859180 msgr2=0x55d1af85bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.294+0000 7f0e2de00640 1 --2- 172.21.15.83:0/1694911403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af859180 0x55d1af85bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.296 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.294+0000 7f0e2de00640 1 -- 172.21.15.83:0/1694911403 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1af858fb0 con 0x55d1af858860 2024-04-23T14:53:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.294+0000 7f0e2e800640 1 --2- 172.21.15.83:0/1694911403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af863850 0x55d1af85b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:53:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.294+0000 7f0e27e00640 1 --2- 172.21.15.83:0/1694911403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af859180 0x55d1af85bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:53:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.294+0000 7f0e2de00640 1 --2- 172.21.15.83:0/1694911403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af858860 0x55d1af85f740 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0e200bcda0 tx=0x7f0e200bb320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766afe057aebffcd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.294+0000 7f0e1e600640 1 -- 172.21.15.83:0/1694911403 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20049040 con 0x55d1af858860 2024-04-23T14:53:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.294+0000 7f0e1e600640 1 -- 172.21.15.83:0/1694911403 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e200c2630 con 0x55d1af858860 2024-04-23T14:53:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.294+0000 7f0e1e600640 1 -- 172.21.15.83:0/1694911403 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e200dbe60 con 0x55d1af858860 2024-04-23T14:53:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.295+0000 7f0e379f9740 1 -- 172.21.15.83:0/1694911403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af858860 msgr2=0x55d1af85f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.295+0000 7f0e379f9740 1 --2- 172.21.15.83:0/1694911403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af858860 0x55d1af85f740 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0e200bcda0 tx=0x7f0e200bb320 comp rx=0 tx=0).stop 2024-04-23T14:53:40.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.295+0000 7f0e379f9740 1 -- 172.21.15.83:0/1694911403 shutdown_connections 2024-04-23T14:53:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.295+0000 7f0e379f9740 1 --2- 172.21.15.83:0/1694911403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af863850 0x55d1af85b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.295+0000 7f0e379f9740 1 --2- 172.21.15.83:0/1694911403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af859180 0x55d1af85bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.295+0000 7f0e379f9740 1 --2- 172.21.15.83:0/1694911403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af858860 0x55d1af85f740 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.295+0000 7f0e379f9740 1 -- 172.21.15.83:0/1694911403 >> 172.21.15.83:0/1694911403 conn(0x55d1af854b80 msgr2=0x55d1af853c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.295+0000 7f0e379f9740 1 -- 172.21.15.83:0/1694911403 shutdown_connections 2024-04-23T14:53:40.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.295+0000 7f0e379f9740 1 -- 172.21.15.83:0/1694911403 wait complete. 2024-04-23T14:53:40.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.295+0000 7f0e379f9740 1 -- start start 2024-04-23T14:53:40.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e379f9740 1 Processor -- start 2024-04-23T14:53:40.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e379f9740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af86ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e379f9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af859180 0x55d1af86bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e379f9740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af863850 0x55d1af85d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e379f9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e301ae530 con 0x55d1af859180 2024-04-23T14:53:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e379f9740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0e301af9a0 con 0x55d1af858860 2024-04-23T14:53:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e379f9740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0e301aec00 con 0x55d1af863850 2024-04-23T14:53:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e379f9740 10 client.0 _ll_register_callbacks cb 0x7f0e200b8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:53:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af859180 0x55d1af86bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e2de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af86ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e2de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af86ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37344/0 (socket says 172.21.15.83:37344) 2024-04-23T14:53:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e27e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af859180 0x55d1af86bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52070/0 (socket says 172.21.15.83:52070) 2024-04-23T14:53:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e2de00640 1 -- 172.21.15.83:0/1810726567 learned_addr learned my addr 172.21.15.83:0/1810726567 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:53:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.296+0000 7f0e2e800640 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af863850 0x55d1af85d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.297+0000 7f0e2de00640 1 -- 172.21.15.83:0/1810726567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af863850 msgr2=0x55d1af85d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.297+0000 7f0e2de00640 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af863850 0x55d1af85d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.297+0000 7f0e2de00640 1 -- 172.21.15.83:0/1810726567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af859180 msgr2=0x55d1af86bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:53:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.297+0000 7f0e2de00640 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af859180 0x55d1af86bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.297+0000 7f0e2de00640 1 -- 172.21.15.83:0/1810726567 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1af4d03f0 con 0x55d1af858860 2024-04-23T14:53:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.297+0000 7f0e27e00640 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af859180 0x55d1af86bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:53:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.297+0000 7f0e2de00640 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af86ba40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0e200dd210 tx=0x7f0e2003b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e070538a5b5c90f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:53:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.297+0000 7f0e13400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:53:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.297+0000 7f0e13400640 1 -- 172.21.15.83:0/1810726567 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e200b6070 con 0x55d1af858860 2024-04-23T14:53:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.297+0000 7f0e13400640 1 -- 172.21.15.83:0/1810726567 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f0e200b9480 con 0x55d1af858860 2024-04-23T14:53:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.297+0000 7f0e379f9740 1 -- 172.21.15.83:0/1810726567 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e200c2630 con 0x55d1af858860 2024-04-23T14:53:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.298+0000 7f0e13400640 1 -- 172.21.15.83:0/1810726567 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0e20049070 con 0x55d1af858860 2024-04-23T14:53:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.298+0000 7f0e379f9740 10 client.26288 fetch_fsmap learned FSMap version 29 2024-04-23T14:53:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.298+0000 7f0e379f9740 1 -- 172.21.15.83:0/1810726567 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d1af4900f0 con 0x55d1af858860 2024-04-23T14:53:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.298+0000 7f0e13400640 1 -- 172.21.15.83:0/1810726567 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e200c2630 con 0x55d1af858860 2024-04-23T14:53:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.298+0000 7f0e13400640 1 -- 172.21.15.83:0/1810726567 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0e200eeba0 con 0x55d1af858860 2024-04-23T14:53:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.298+0000 7f0e379f9740 10 client.26288 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:53:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.298+0000 7f0e379f9740 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:53:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.298+0000 7f0e379f9740 20 client.26288 populate_metadata read hostname 'smithi083' 2024-04-23T14:53:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.298+0000 7f0e379f9740 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d1af8d9cb0 0x55d1af8dc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:53:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.298+0000 7f0e379f9740 4 client.26288 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:53:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.298+0000 7f0e379f9740 1 -- 172.21.15.83:0/1810726567 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1af7c5a80 con 0x55d1af8d9cb0 2024-04-23T14:53:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.298+0000 7f0e27e00640 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d1af8d9cb0 0x55d1af8dc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:53:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e27e00640 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d1af8d9cb0 0x55d1af8dc0a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44de5f09b2eaa27c server_cookie=ec03fdc6c407b31b in_seq=0 out_seq=0 2024-04-23T14:53:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e13400640 10 client.26288 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e13400640 1 -- 172.21.15.83:0/1810726567 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d1af7c5a80 con 0x55d1af8d9cb0 2024-04-23T14:53:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e13400640 10 client.26288 handle_command_reply: tid=0 2024-04-23T14:53:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e379f9740 1 client.26288 shutdown 2024-04-23T14:53:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e379f9740 1 -- 172.21.15.83:0/1810726567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 msgr2=0x55d1af86ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e379f9740 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af86ba40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0e200dd210 tx=0x7f0e2003b460 comp rx=0 tx=0).stop 2024-04-23T14:53:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e379f9740 1 -- 172.21.15.83:0/1810726567 shutdown_connections 2024-04-23T14:53:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e379f9740 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d1af8d9cb0 0x55d1af8dc0a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e379f9740 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d1af863850 0x55d1af85d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e379f9740 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d1af859180 0x55d1af86bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e13400640 0 client.26288 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:53:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e379f9740 1 --2- 172.21.15.83:0/1810726567 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d1af858860 0x55d1af86ba40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.299+0000 7f0e379f9740 1 -- 172.21.15.83:0/1810726567 >> 172.21.15.83:0/1810726567 conn(0x55d1af854b80 msgr2=0x55d1af862010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e379f9740 1 -- 172.21.15.83:0/1810726567 shutdown_connections 2024-04-23T14:53:40.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e379f9740 1 -- 172.21.15.83:0/1810726567 wait complete. 2024-04-23T14:53:40.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e379f9740 20 client.26288 trim_cache size 0 max 16384 2024-04-23T14:53:40.309 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:53:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:53:40.309 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:53:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: "9c49bdae-8161-4000-bd27-d1d1964c7f43": { 2024-04-23T14:53:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:53:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "9c49bdae-8161-4000-bd27-d1d1964c7f43", 2024-04-23T14:53:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:53:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:53:40.310 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:53:40.310 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:53:40.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e34c00640 1 -- 172.21.15.83:0/883272134 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0e140a3cb0 msgr2=0x7f0e140a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e34c00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0e140a3cb0 0x7f0e140a6170 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e18003fa0 tx=0x7f0e180193e0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e34c00640 1 -- 172.21.15.83:0/883272134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e3006a660 msgr2=0x7f0e301a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:53:40.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e34c00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e3006a660 0x7f0e301a07a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0e20004770 tx=0x7f0e20003c10 comp rx=0 tx=0).stop 2024-04-23T14:53:40.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e34c00640 1 -- 172.21.15.83:0/883272134 shutdown_connections 2024-04-23T14:53:40.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e34c00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0e140a3cb0 0x7f0e140a6170 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e34c00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e3010ea10 0x7f0e3019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e34c00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3006af80 0x7f0e301a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e34c00640 1 --2- 172.21.15.83:0/883272134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e3006a660 0x7f0e301a07a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:53:40.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.300+0000 7f0e34c00640 1 -- 172.21.15.83:0/883272134 >> 172.21.15.83:0/883272134 conn(0x7f0e30077f60 msgr2=0x7f0e3010fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:53:40.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.301+0000 7f0e34c00640 1 -- 172.21.15.83:0/883272134 shutdown_connections 2024-04-23T14:53:40.312 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:53:40.301+0000 7f0e34c00640 1 -- 172.21.15.83:0/883272134 wait complete. 2024-04-23T14:53:40.317 INFO:tasks.cephfs.filesystem:scrub status for tag:9c49bdae-8161-4000-bd27-d1d1964c7f43 - {'path': '/', 'tag': '9c49bdae-8161-4000-bd27-d1d1964c7f43', 'options': 'recursive,force'} 2024-04-23T14:53:40.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:40 smithi203 ceph-mon[32800]: pgmap v159: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 MiB/s wr, 8.60k op/s 2024-04-23T14:53:40.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:40 smithi083 ceph-mon[27867]: pgmap v159: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 MiB/s wr, 8.60k op/s 2024-04-23T14:53:40.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:40 smithi173 ceph-mon[32443]: pgmap v159: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 46 MiB/s wr, 8.60k op/s 2024-04-23T14:53:41.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:41 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:53:41.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:41 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:41 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:53:41.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:41 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:53:41.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:41 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:41 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:53:41.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:41 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:41 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:53:41.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:41 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:41 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:53:41.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:41 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:41 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:41 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:53:41.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:41 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:41 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:53:41.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:41 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:53:41.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:41 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:41 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:53:41.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:41 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:41 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:53:41.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:41 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:41 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:53:41.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:41 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:41 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:41 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:53:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:41 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:41 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:53:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:41 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:53:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:41 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:41 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:53:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:41 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:41 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:53:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:41 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:41 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:53:41.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:41 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:41.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:41 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:53:42.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:42 smithi203 ceph-mon[32800]: pgmap v160: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 46 MiB/s wr, 8.75k op/s 2024-04-23T14:53:42.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:42 smithi083 ceph-mon[27867]: pgmap v160: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 46 MiB/s wr, 8.75k op/s 2024-04-23T14:53:42.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:42 smithi173 ceph-mon[32443]: pgmap v160: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 46 MiB/s wr, 8.75k op/s 2024-04-23T14:53:44.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:44 smithi203 ceph-mon[32800]: pgmap v161: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 29 MiB/s wr, 5.75k op/s 2024-04-23T14:53:44.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:44 smithi083 ceph-mon[27867]: pgmap v161: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 29 MiB/s wr, 5.75k op/s 2024-04-23T14:53:44.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:44 smithi173 ceph-mon[32443]: pgmap v161: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 29 MiB/s wr, 5.75k op/s 2024-04-23T14:53:46.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:46 smithi203 ceph-mon[32800]: pgmap v162: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 28 MiB/s wr, 5.45k op/s 2024-04-23T14:53:46.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:46 smithi083 ceph-mon[27867]: pgmap v162: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 28 MiB/s wr, 5.45k op/s 2024-04-23T14:53:46.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:46 smithi173 ceph-mon[32443]: pgmap v162: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 28 MiB/s wr, 5.45k op/s 2024-04-23T14:53:48.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:48 smithi203 ceph-mon[32800]: pgmap v163: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 21 MiB/s wr, 4.25k op/s 2024-04-23T14:53:48.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:48 smithi083 ceph-mon[27867]: pgmap v163: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 21 MiB/s wr, 4.25k op/s 2024-04-23T14:53:48.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:48 smithi173 ceph-mon[32443]: pgmap v163: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 21 MiB/s wr, 4.25k op/s 2024-04-23T14:53:49.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:49 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:53:49.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:49 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:53:49.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:49 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:53:50.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:50 smithi083 ceph-mon[27867]: pgmap v164: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.8 MiB/s wr, 1.65k op/s 2024-04-23T14:53:50.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:50 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:53:50.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:50 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:53:50.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:50 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:53:50.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:50 smithi173 ceph-mon[32443]: pgmap v164: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.8 MiB/s wr, 1.65k op/s 2024-04-23T14:53:50.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:50 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:53:50.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:50 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:53:50.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:50 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:53:50.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:50 smithi203 ceph-mon[32800]: pgmap v164: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.8 MiB/s wr, 1.65k op/s 2024-04-23T14:53:50.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:50 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:53:50.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:50 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:53:50.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:50 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:53:51.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:51 smithi083 ceph-mon[27867]: pgmap v165: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.5 MiB/s wr, 1.38k op/s 2024-04-23T14:53:51.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:51 smithi083 ceph-mon[27867]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-23T14:53:51.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:51 smithi083 ceph-mon[27867]: Cluster is now healthy 2024-04-23T14:53:51.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:51 smithi173 ceph-mon[32443]: pgmap v165: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.5 MiB/s wr, 1.38k op/s 2024-04-23T14:53:51.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:51 smithi173 ceph-mon[32443]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-23T14:53:51.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:51 smithi173 ceph-mon[32443]: Cluster is now healthy 2024-04-23T14:53:51.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:51 smithi203 ceph-mon[32800]: pgmap v165: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 6.5 MiB/s wr, 1.38k op/s 2024-04-23T14:53:51.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:51 smithi203 ceph-mon[32800]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-23T14:53:51.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:51 smithi203 ceph-mon[32800]: Cluster is now healthy 2024-04-23T14:53:53.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:53 smithi083 ceph-mon[27867]: pgmap v166: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 214 KiB/s wr, 50 op/s 2024-04-23T14:53:53.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:53 smithi173 ceph-mon[32443]: pgmap v166: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 214 KiB/s wr, 50 op/s 2024-04-23T14:53:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:53 smithi203 ceph-mon[32800]: pgmap v166: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 214 KiB/s wr, 50 op/s 2024-04-23T14:53:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:54 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:53:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:54 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:53:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:54 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:53:54.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:54 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:53:54.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:54 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:53:54.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:54 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:53:54.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:54 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:53:54.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:54 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:53:54.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:54 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:53:55.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:55 smithi083 ceph-mon[27867]: pgmap v167: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5 op/s 2024-04-23T14:53:55.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:55 smithi173 ceph-mon[32443]: pgmap v167: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5 op/s 2024-04-23T14:53:55.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:55 smithi203 ceph-mon[32800]: pgmap v167: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5 op/s 2024-04-23T14:53:57.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:57 smithi083 ceph-mon[27867]: pgmap v168: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-04-23T14:53:57.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:57 smithi173 ceph-mon[32443]: pgmap v168: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-04-23T14:53:57.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:57 smithi203 ceph-mon[32800]: pgmap v168: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-04-23T14:53:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:53:59 smithi173 ceph-mon[32443]: pgmap v169: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-04-23T14:53:59.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:53:59 smithi203 ceph-mon[32800]: pgmap v169: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-04-23T14:54:00.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:53:59 smithi083 ceph-mon[27867]: pgmap v169: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-04-23T14:54:01.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:01 smithi173 ceph-mon[32443]: pgmap v170: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-04-23T14:54:01.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:01 smithi203 ceph-mon[32800]: pgmap v170: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-04-23T14:54:02.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:01 smithi083 ceph-mon[27867]: pgmap v170: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.3 KiB/s wr, 6 op/s 2024-04-23T14:54:03.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:03 smithi173 ceph-mon[32443]: pgmap v171: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-04-23T14:54:03.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:03 smithi203 ceph-mon[32800]: pgmap v171: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-04-23T14:54:04.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:03 smithi083 ceph-mon[27867]: pgmap v171: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2024-04-23T14:54:05.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:05 smithi173 ceph-mon[32443]: pgmap v172: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-04-23T14:54:05.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:05 smithi203 ceph-mon[32800]: pgmap v172: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-04-23T14:54:06.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:05 smithi083 ceph-mon[27867]: pgmap v172: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-04-23T14:54:07.810 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:07 smithi083 ceph-mon[27867]: pgmap v173: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-23T14:54:07.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:07 smithi173 ceph-mon[32443]: pgmap v173: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-23T14:54:07.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:07 smithi203 ceph-mon[32800]: pgmap v173: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-23T14:54:09.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:09 smithi173 ceph-mon[32443]: pgmap v174: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-23T14:54:09.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:09 smithi203 ceph-mon[32800]: pgmap v174: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-23T14:54:10.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:09 smithi083 ceph-mon[27867]: pgmap v174: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-23T14:54:10.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:54:10.318 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:54:10.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.539+0000 7ffb1bc00640 1 -- 172.21.15.83:0/500992178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb140743a0 msgr2=0x7ffb14074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:10.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.539+0000 7ffb1bc00640 1 --2- 172.21.15.83:0/500992178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb140743a0 0x7ffb14074780 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009900 tx=0x7ffb0802a480 comp rx=0 tx=0).stop 2024-04-23T14:54:10.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.540+0000 7ffb1bc00640 1 -- 172.21.15.83:0/500992178 shutdown_connections 2024-04-23T14:54:10.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.540+0000 7ffb1bc00640 1 --2- 172.21.15.83:0/500992178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb14113030 0x7ffb14115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.540+0000 7ffb1bc00640 1 --2- 172.21.15.83:0/500992178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cc0 0x7ffb14112af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.540+0000 7ffb1bc00640 1 --2- 172.21.15.83:0/500992178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb140743a0 0x7ffb14074780 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.540+0000 7ffb1bc00640 1 -- 172.21.15.83:0/500992178 >> 172.21.15.83:0/500992178 conn(0x7ffb140fdb10 msgr2=0x7ffb140fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:10.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.540+0000 7ffb1bc00640 1 -- 172.21.15.83:0/500992178 shutdown_connections 2024-04-23T14:54:10.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.540+0000 7ffb1bc00640 1 -- 172.21.15.83:0/500992178 wait complete. 2024-04-23T14:54:10.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.541+0000 7ffb1bc00640 1 Processor -- start 2024-04-23T14:54:10.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.541+0000 7ffb1bc00640 1 -- start start 2024-04-23T14:54:10.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.542+0000 7ffb1bc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb140743a0 0x7ffb1419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.542+0000 7ffb1bc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cc0 0x7ffb1419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.542+0000 7ffb1bc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb14113030 0x7ffb141a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.542+0000 7ffb1bc00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14118a80 con 0x7ffb14074cc0 2024-04-23T14:54:10.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.542+0000 7ffb1bc00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14118900 con 0x7ffb140743a0 2024-04-23T14:54:10.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.542+0000 7ffb1bc00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ffb14118c00 con 0x7ffb14113030 2024-04-23T14:54:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.542+0000 7ffb13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cc0 0x7ffb1419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.542+0000 7ffb19c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb14113030 0x7ffb141a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.542+0000 7ffb19200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb140743a0 0x7ffb1419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.542+0000 7ffb13e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cc0 0x7ffb1419f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47412/0 (socket says 172.21.15.83:47412) 2024-04-23T14:54:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.542+0000 7ffb13e00640 1 -- 172.21.15.83:0/3357957461 learned_addr learned my addr 172.21.15.83:0/3357957461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.542+0000 7ffb19c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb14113030 0x7ffb141a3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50644/0 (socket says 172.21.15.83:50644) 2024-04-23T14:54:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.543+0000 7ffb19c00640 1 -- 172.21.15.83:0/3357957461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb140743a0 msgr2=0x7ffb1419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:10.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.543+0000 7ffb19c00640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb140743a0 0x7ffb1419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.543+0000 7ffb19c00640 1 -- 172.21.15.83:0/3357957461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cc0 msgr2=0x7ffb1419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.543+0000 7ffb19c00640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cc0 0x7ffb1419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.543+0000 7ffb19c00640 1 -- 172.21.15.83:0/3357957461 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb141a3f70 con 0x7ffb14113030 2024-04-23T14:54:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.543+0000 7ffb19200640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb140743a0 0x7ffb1419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.543+0000 7ffb19c00640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb14113030 0x7ffb141a3840 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c005fd0 tx=0x7ffb0c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.543+0000 7ffb11600640 1 -- 172.21.15.83:0/3357957461 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c00cdf0 con 0x7ffb14113030 2024-04-23T14:54:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.543+0000 7ffb11600640 1 -- 172.21.15.83:0/3357957461 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb0c005020 con 0x7ffb14113030 2024-04-23T14:54:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.543+0000 7ffb11600640 1 -- 172.21.15.83:0/3357957461 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0106d0 con 0x7ffb14113030 2024-04-23T14:54:10.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.543+0000 7ffb1bc00640 1 -- 172.21.15.83:0/3357957461 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb141a4260 con 0x7ffb14113030 2024-04-23T14:54:10.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.544+0000 7ffb1bc00640 1 -- 172.21.15.83:0/3357957461 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb141abaf0 con 0x7ffb14113030 2024-04-23T14:54:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.547+0000 7ffb1ea2f740 1 Processor -- start 2024-04-23T14:54:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.547+0000 7ffb11600640 1 -- 172.21.15.83:0/3357957461 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffb0c0054d0 con 0x7ffb14113030 2024-04-23T14:54:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.547+0000 7ffb1ea2f740 1 -- start start 2024-04-23T14:54:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb1ea2f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f051860 0x56308f051c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb1ea2f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f052180 0x56308f05c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb1ea2f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f05ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb1ea2f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56308efb71f0 con 0x56308f052180 2024-04-23T14:54:10.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb1ea2f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56308ef5d350 con 0x56308f051860 2024-04-23T14:54:10.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb1ea2f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56308efb31b0 con 0x56308f05c850 2024-04-23T14:54:10.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb13e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f051860 0x56308f051c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb19200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f052180 0x56308f05c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb13e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f051860 0x56308f051c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54812/0 (socket says 172.21.15.83:54812) 2024-04-23T14:54:10.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb13e00640 1 -- 172.21.15.83:0/875638210 learned_addr learned my addr 172.21.15.83:0/875638210 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:10.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb13e00640 1 --2- 172.21.15.83:0/875638210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f05ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb19200640 1 -- 172.21.15.83:0/875638210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 msgr2=0x56308f05ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:10.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb19200640 1 --2- 172.21.15.83:0/875638210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f05ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb19200640 1 -- 172.21.15.83:0/875638210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f051860 msgr2=0x56308f051c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:10.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb19200640 1 --2- 172.21.15.83:0/875638210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f051860 0x56308f051c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.548+0000 7ffb19200640 1 -- 172.21.15.83:0/875638210 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308efb1f50 con 0x56308f052180 2024-04-23T14:54:10.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.549+0000 7ffb19200640 1 --2- 172.21.15.83:0/875638210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f052180 0x56308f05c310 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffb0802aa70 tx=0x7ffb08005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7f797b6c06b481 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:10.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.549+0000 7ffb11600640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ffafc0a3da0 0x7ffafc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.549+0000 7ffb11600640 1 -- 172.21.15.83:0/3357957461 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ffb0c0d0d60 con 0x7ffb14113030 2024-04-23T14:54:10.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.549+0000 7ffb13e00640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ffafc0a3da0 0x7ffafc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.549+0000 7ffb02000640 1 -- 172.21.15.83:0/875638210 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08035040 con 0x56308f052180 2024-04-23T14:54:10.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.549+0000 7ffb02000640 1 -- 172.21.15.83:0/875638210 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb08004760 con 0x56308f052180 2024-04-23T14:54:10.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.549+0000 7ffb02000640 1 -- 172.21.15.83:0/875638210 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb080038e0 con 0x56308f052180 2024-04-23T14:54:10.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.550+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/875638210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f052180 msgr2=0x56308f05c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:10.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.550+0000 7ffb13e00640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ffafc0a3da0 0x7ffafc0a6260 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x56308f04ed10 tx=0x7ffb04015040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:10.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.550+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/875638210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f052180 0x56308f05c310 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ffb0802aa70 tx=0x7ffb08005de0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.550+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/875638210 shutdown_connections 2024-04-23T14:54:10.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.550+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/875638210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f05ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.550+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/875638210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f052180 0x56308f05c310 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.550+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/875638210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f051860 0x56308f051c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.550+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/875638210 >> 172.21.15.83:0/875638210 conn(0x56308f04db80 msgr2=0x56308f04df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:10.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.550+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/875638210 shutdown_connections 2024-04-23T14:54:10.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.550+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/875638210 wait complete. 2024-04-23T14:54:10.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.551+0000 7ffb1ea2f740 1 -- start start 2024-04-23T14:54:10.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.551+0000 7ffb1ea2f740 1 Processor -- start 2024-04-23T14:54:10.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.551+0000 7ffb1ea2f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f059810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.551+0000 7ffb1ea2f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f059d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.551+0000 7ffb1ea2f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f05a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.551+0000 7ffb1ea2f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56308f05f5e0 con 0x56308f051860 2024-04-23T14:54:10.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.551+0000 7ffb1ea2f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56308ef70690 con 0x56308f052180 2024-04-23T14:54:10.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.551+0000 7ffb1ea2f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56308f05f760 con 0x56308f05c850 2024-04-23T14:54:10.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.551+0000 7ffb1ea2f740 10 client.0 _ll_register_callbacks cb 0x56308efac650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:10.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffb19200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f059810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffb19200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f059810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47428/0 (socket says 172.21.15.83:47428) 2024-04-23T14:54:10.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffb13e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f059d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffb19200640 1 -- 172.21.15.83:0/3396180229 learned_addr learned my addr 172.21.15.83:0/3396180229 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:10.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffb19c00640 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f05a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffb13e00640 1 -- 172.21.15.83:0/3396180229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 msgr2=0x56308f05a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:10.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffb13e00640 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f05a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffb13e00640 1 -- 172.21.15.83:0/3396180229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 msgr2=0x56308f059810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:10.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffb13e00640 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f059810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffb13e00640 1 -- 172.21.15.83:0/3396180229 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308ef6f920 con 0x56308f052180 2024-04-23T14:54:10.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffb19c00640 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f05a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:10.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffb13e00640 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f059d50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb04004580 tx=0x7ffb04005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=851a42ea69d769b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:10.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.552+0000 7ffaf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:54:10.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.553+0000 7ffaf6a00640 1 -- 172.21.15.83:0/3396180229 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04026070 con 0x56308f052180 2024-04-23T14:54:10.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.553+0000 7ffaf6a00640 1 -- 172.21.15.83:0/3396180229 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ffb04005220 con 0x56308f052180 2024-04-23T14:54:10.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.553+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3396180229 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb08004760 con 0x56308f052180 2024-04-23T14:54:10.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.553+0000 7ffaf6a00640 1 -- 172.21.15.83:0/3396180229 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb04005420 con 0x56308f052180 2024-04-23T14:54:10.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.553+0000 7ffb1ea2f740 10 client.26300 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:10.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.553+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3396180229 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56308ed937d0 con 0x56308f052180 2024-04-23T14:54:10.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.553+0000 7ffaf6a00640 1 -- 172.21.15.83:0/3396180229 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb08004760 con 0x56308f052180 2024-04-23T14:54:10.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.554+0000 7ffaf6a00640 1 -- 172.21.15.83:0/3396180229 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ffb0400fb70 con 0x56308f052180 2024-04-23T14:54:10.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.554+0000 7ffb1ea2f740 10 client.26300 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:10.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.554+0000 7ffb1ea2f740 10 client.26300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:10.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.554+0000 7ffb1ea2f740 20 client.26300 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:10.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.554+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56308f06cb50 0x56308f06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.554+0000 7ffb1ea2f740 4 client.26300 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:54:10.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.554+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3396180229 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56308efc4c10 con 0x56308f06cb50 2024-04-23T14:54:10.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.554+0000 7ffb19200640 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56308f06cb50 0x56308f06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.555+0000 7ffb19200640 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56308f06cb50 0x56308f06eff0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=974d9d1f73d8f059 server_cookie=9d10dcac07ce3a5f in_seq=0 out_seq=0 2024-04-23T14:54:10.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.555+0000 7ffaf6a00640 10 client.26300 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:10.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffaf6a00640 1 -- 172.21.15.83:0/3396180229 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56308efc4c10 con 0x56308f06cb50 2024-04-23T14:54:10.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffaf6a00640 10 client.26300 handle_command_reply: tid=0 2024-04-23T14:54:10.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffb1ea2f740 1 client.26300 shutdown 2024-04-23T14:54:10.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3396180229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 msgr2=0x56308f059d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:10.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f059d50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb04004580 tx=0x7ffb04005f90 comp rx=0 tx=0).stop 2024-04-23T14:54:10.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3396180229 shutdown_connections 2024-04-23T14:54:10.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56308f06cb50 0x56308f06eff0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f05a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f059d50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/3396180229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f059810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffaf6a00640 0 client.26300 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:10.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3396180229 >> 172.21.15.83:0/3396180229 conn(0x56308f04db80 msgr2=0x56308f053980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:10.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3396180229 shutdown_connections 2024-04-23T14:54:10.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3396180229 wait complete. 2024-04-23T14:54:10.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.556+0000 7ffb1ea2f740 20 client.26300 trim_cache size 0 max 16384 2024-04-23T14:54:10.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.575+0000 7ffb1ea2f740 1 Processor -- start 2024-04-23T14:54:10.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.575+0000 7ffb1ea2f740 1 -- start start 2024-04-23T14:54:10.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.575+0000 7ffb1ea2f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f058750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.575+0000 7ffb1ea2f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f0684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.575+0000 7ffb1ea2f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f0748c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.575+0000 7ffb1ea2f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141b6c10 con 0x56308f051860 2024-04-23T14:54:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.575+0000 7ffb1ea2f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141b44e0 con 0x56308f052180 2024-04-23T14:54:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.575+0000 7ffb1ea2f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141b2330 con 0x56308f05c850 2024-04-23T14:54:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.576+0000 7ffb19200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f058750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.576+0000 7ffb13e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f0684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.576+0000 7ffb19200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f058750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47430/0 (socket says 172.21.15.83:47430) 2024-04-23T14:54:10.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.576+0000 7ffb19200640 1 -- 172.21.15.83:0/3933904204 learned_addr learned my addr 172.21.15.83:0/3933904204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.576+0000 7ffb19c00640 1 --2- 172.21.15.83:0/3933904204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f0748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.576+0000 7ffb13e00640 1 -- 172.21.15.83:0/3933904204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 msgr2=0x56308f0748c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.576+0000 7ffb13e00640 1 --2- 172.21.15.83:0/3933904204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f0748c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.576+0000 7ffb13e00640 1 -- 172.21.15.83:0/3933904204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 msgr2=0x56308f058750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.576+0000 7ffb13e00640 1 --2- 172.21.15.83:0/3933904204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f058750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.576+0000 7ffb13e00640 1 -- 172.21.15.83:0/3933904204 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308ecc93f0 con 0x56308f052180 2024-04-23T14:54:10.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.576+0000 7ffb19200640 1 --2- 172.21.15.83:0/3933904204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f058750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.576+0000 7ffb13e00640 1 --2- 172.21.15.83:0/3933904204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f0684a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffb04004550 tx=0x7ffb04003ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23867eb817bd081 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:10.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.577+0000 7ffb01600640 1 -- 172.21.15.83:0/3933904204 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04026070 con 0x56308f052180 2024-04-23T14:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.577+0000 7ffb01600640 1 -- 172.21.15.83:0/3933904204 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb04038070 con 0x56308f052180 2024-04-23T14:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.577+0000 7ffb01600640 1 -- 172.21.15.83:0/3933904204 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0402a400 con 0x56308f052180 2024-04-23T14:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.577+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3933904204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 msgr2=0x56308f0684a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.577+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/3933904204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f0684a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffb04004550 tx=0x7ffb04003ed0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.578+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3933904204 shutdown_connections 2024-04-23T14:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.578+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/3933904204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f0748c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.578+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/3933904204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f0684a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.578+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/3933904204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f058750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.578+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3933904204 >> 172.21.15.83:0/3933904204 conn(0x56308f04db80 msgr2=0x56308f05a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.578+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3933904204 shutdown_connections 2024-04-23T14:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.578+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/3933904204 wait complete. 2024-04-23T14:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.578+0000 7ffb1ea2f740 1 -- start start 2024-04-23T14:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.578+0000 7ffb1ea2f740 1 Processor -- start 2024-04-23T14:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb1ea2f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f060ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb1ea2f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f061000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb1ea2f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f0d2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb1ea2f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141b29a0 con 0x56308f051860 2024-04-23T14:54:10.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb1ea2f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141b3e10 con 0x56308f052180 2024-04-23T14:54:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb1ea2f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ffb141b3070 con 0x56308f05c850 2024-04-23T14:54:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb1ea2f740 10 client.0 _ll_register_callbacks cb 0x7ffb04004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb19200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f060ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb13e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f061000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb19200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f060ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47434/0 (socket says 172.21.15.83:47434) 2024-04-23T14:54:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb19200640 1 -- 172.21.15.83:0/2921865587 learned_addr learned my addr 172.21.15.83:0/2921865587 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb19c00640 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f0d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb13e00640 1 -- 172.21.15.83:0/2921865587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 msgr2=0x56308f0d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb13e00640 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f0d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb13e00640 1 -- 172.21.15.83:0/2921865587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 msgr2=0x56308f060ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb13e00640 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f060ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb13e00640 1 -- 172.21.15.83:0/2921865587 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308ec890f0 con 0x56308f052180 2024-04-23T14:54:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.579+0000 7ffb19200640 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f060ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.580+0000 7ffb13e00640 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f061000 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x56308f05f750 tx=0x7ffb04033890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95da3aecfd3eb4c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.580+0000 7ffaf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:54:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.580+0000 7ffaf6a00640 1 -- 172.21.15.83:0/2921865587 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04026040 con 0x56308f052180 2024-04-23T14:54:10.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.580+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/2921865587 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb04038070 con 0x56308f052180 2024-04-23T14:54:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.580+0000 7ffaf6a00640 1 -- 172.21.15.83:0/2921865587 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7ffb0400fb70 con 0x56308f052180 2024-04-23T14:54:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.580+0000 7ffaf6a00640 1 -- 172.21.15.83:0/2921865587 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb0403d460 con 0x56308f052180 2024-04-23T14:54:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.580+0000 7ffb1ea2f740 10 client.26306 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.580+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/2921865587 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56308ef7c810 con 0x56308f052180 2024-04-23T14:54:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.581+0000 7ffaf6a00640 1 -- 172.21.15.83:0/2921865587 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb04038070 con 0x56308f052180 2024-04-23T14:54:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.581+0000 7ffaf6a00640 1 -- 172.21.15.83:0/2921865587 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ffb04022570 con 0x56308f052180 2024-04-23T14:54:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.581+0000 7ffb1ea2f740 10 client.26306 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:10.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.581+0000 7ffb1ea2f740 10 client.26306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.581+0000 7ffb1ea2f740 20 client.26306 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.581+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56308f0d3b40 0x56308f0d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.581+0000 7ffb1ea2f740 4 client.26306 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:54:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.581+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/2921865587 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56308efbea80 con 0x56308f0d3b40 2024-04-23T14:54:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.581+0000 7ffb19200640 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56308f0d3b40 0x56308f0d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb19200640 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56308f0d3b40 0x56308f0d5f30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a2a27f042a11084 server_cookie=bf96fc7924d9f956 in_seq=0 out_seq=0 2024-04-23T14:54:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffaf6a00640 10 client.26306 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffaf6a00640 1 -- 172.21.15.83:0/2921865587 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56308efbea80 con 0x56308f0d3b40 2024-04-23T14:54:10.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffaf6a00640 10 client.26306 handle_command_reply: tid=0 2024-04-23T14:54:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb1ea2f740 1 client.26306 shutdown 2024-04-23T14:54:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/2921865587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 msgr2=0x56308f061000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f061000 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x56308f05f750 tx=0x7ffb04033890 comp rx=0 tx=0).stop 2024-04-23T14:54:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/2921865587 shutdown_connections 2024-04-23T14:54:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56308f0d3b40 0x56308f0d5f30 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56308f05c850 0x56308f0d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56308f052180 0x56308f061000 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb1ea2f740 1 --2- 172.21.15.83:0/2921865587 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56308f051860 0x56308f060ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffaf6a00640 0 client.26306 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/2921865587 >> 172.21.15.83:0/2921865587 conn(0x56308f04db80 msgr2=0x56308f052fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/2921865587 shutdown_connections 2024-04-23T14:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb1ea2f740 1 -- 172.21.15.83:0/2921865587 wait complete. 2024-04-23T14:54:10.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.582+0000 7ffb1ea2f740 20 client.26306 trim_cache size 0 max 16384 2024-04-23T14:54:10.591 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:54:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:54:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:54:10.592 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:54:10.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.583+0000 7ffb1bc00640 1 -- 172.21.15.83:0/3357957461 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ffafc0a3da0 msgr2=0x7ffafc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.583+0000 7ffb1bc00640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ffafc0a3da0 0x7ffafc0a6260 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x56308f04ed10 tx=0x7ffb04015040 comp rx=0 tx=0).stop 2024-04-23T14:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.583+0000 7ffb1bc00640 1 -- 172.21.15.83:0/3357957461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb14113030 msgr2=0x7ffb141a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.583+0000 7ffb1bc00640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb14113030 0x7ffb141a3840 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c005fd0 tx=0x7ffb0c00ef10 comp rx=0 tx=0).stop 2024-04-23T14:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.583+0000 7ffb1bc00640 1 -- 172.21.15.83:0/3357957461 shutdown_connections 2024-04-23T14:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.583+0000 7ffb1bc00640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb14113030 0x7ffb141a3840 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.583+0000 7ffb1bc00640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ffafc0a3da0 0x7ffafc0a6260 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.583+0000 7ffb1bc00640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb14074cc0 0x7ffb1419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.583+0000 7ffb1bc00640 1 --2- 172.21.15.83:0/3357957461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffb140743a0 0x7ffb1419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.583+0000 7ffb1bc00640 1 -- 172.21.15.83:0/3357957461 >> 172.21.15.83:0/3357957461 conn(0x7ffb140fdb10 msgr2=0x7ffb140ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:10.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.583+0000 7ffb1bc00640 1 -- 172.21.15.83:0/3357957461 shutdown_connections 2024-04-23T14:54:10.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.583+0000 7ffb1bc00640 1 -- 172.21.15.83:0/3357957461 wait complete. 2024-04-23T14:54:10.601 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:54:10.602 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:54:10.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.794+0000 7fd33fe00640 1 -- 172.21.15.83:0/2647261566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd338109520 msgr2=0x7fd3381110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:10.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.794+0000 7fd33fe00640 1 --2- 172.21.15.83:0/2647261566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd338109520 0x7fd3381110f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd330009be0 tx=0x7fd33002a6c0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.794+0000 7fd33fe00640 1 -- 172.21.15.83:0/2647261566 shutdown_connections 2024-04-23T14:54:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.794+0000 7fd33fe00640 1 --2- 172.21.15.83:0/2647261566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd338109520 0x7fd3381110f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.794+0000 7fd33fe00640 1 --2- 172.21.15.83:0/2647261566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd338104e00 0x7fd338108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.794+0000 7fd33fe00640 1 --2- 172.21.15.83:0/2647261566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd338104450 0x7fd338104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.794+0000 7fd33fe00640 1 -- 172.21.15.83:0/2647261566 >> 172.21.15.83:0/2647261566 conn(0x7fd3380ffcf0 msgr2=0x7fd338102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.794+0000 7fd33fe00640 1 -- 172.21.15.83:0/2647261566 shutdown_connections 2024-04-23T14:54:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.794+0000 7fd33fe00640 1 -- 172.21.15.83:0/2647261566 wait complete. 2024-04-23T14:54:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.795+0000 7fd33fe00640 1 Processor -- start 2024-04-23T14:54:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.795+0000 7fd33fe00640 1 -- start start 2024-04-23T14:54:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.796+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd338104450 0x7fd3381a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.796+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd338109520 0x7fd3381a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.796+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd33819a940 0x7fd33819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.796+0000 7fd33fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3381145c0 con 0x7fd33819a940 2024-04-23T14:54:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.796+0000 7fd33fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd338114440 con 0x7fd338109520 2024-04-23T14:54:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.796+0000 7fd33fe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd338114740 con 0x7fd338104450 2024-04-23T14:54:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.796+0000 7fd33d400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd338104450 0x7fd3381a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.796+0000 7fd33d400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd338104450 0x7fd3381a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50700/0 (socket says 172.21.15.83:50700) 2024-04-23T14:54:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.796+0000 7fd33d400640 1 -- 172.21.15.83:0/2004641004 learned_addr learned my addr 172.21.15.83:0/2004641004 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.796+0000 7fd33de00640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd33819a940 0x7fd33819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.796+0000 7fd33ca00640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd338109520 0x7fd3381a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.797+0000 7fd33de00640 1 -- 172.21.15.83:0/2004641004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd338104450 msgr2=0x7fd3381a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.797+0000 7fd33de00640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd338104450 0x7fd3381a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.797+0000 7fd33de00640 1 -- 172.21.15.83:0/2004641004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd338109520 msgr2=0x7fd3381a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.797+0000 7fd33de00640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd338109520 0x7fd3381a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.797+0000 7fd33de00640 1 -- 172.21.15.83:0/2004641004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33819b460 con 0x7fd33819a940 2024-04-23T14:54:10.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.797+0000 7fd33d400640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd338104450 0x7fd3381a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:10.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.797+0000 7fd33ca00640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd338109520 0x7fd3381a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:10.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.797+0000 7fd33de00640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd33819a940 0x7fd33819adf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd330009b30 tx=0x7fd3300305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:10.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.797+0000 7fd336000640 1 -- 172.21.15.83:0/2004641004 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330041020 con 0x7fd33819a940 2024-04-23T14:54:10.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.797+0000 7fd33fe00640 1 -- 172.21.15.83:0/2004641004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33819b6f0 con 0x7fd33819a940 2024-04-23T14:54:10.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.797+0000 7fd336000640 1 -- 172.21.15.83:0/2004641004 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd330003e50 con 0x7fd33819a940 2024-04-23T14:54:10.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.797+0000 7fd336000640 1 -- 172.21.15.83:0/2004641004 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330036040 con 0x7fd33819a940 2024-04-23T14:54:10.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.798+0000 7fd33fe00640 1 -- 172.21.15.83:0/2004641004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3381a78f0 con 0x7fd33819a940 2024-04-23T14:54:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.799+0000 7fd33fe00640 1 -- 172.21.15.83:0/2004641004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd338069270 con 0x7fd33819a940 2024-04-23T14:54:10.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.801+0000 7fd336000640 1 -- 172.21.15.83:0/2004641004 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd330044020 con 0x7fd33819a940 2024-04-23T14:54:10.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.803+0000 7fd336000640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd3200a3d20 0x7fd3200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:10.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.803+0000 7fd336000640 1 -- 172.21.15.83:0/2004641004 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd3300f1330 con 0x7fd33819a940 2024-04-23T14:54:10.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.803+0000 7fd33d400640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd3200a3d20 0x7fd3200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:10.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.804+0000 7fd33d400640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd3200a3d20 0x7fd3200a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd33819ba80 tx=0x7fd32c009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:10.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:10.808+0000 7fd336000640 1 -- 172.21.15.83:0/2004641004 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd33004b020 con 0x7fd33819a940 2024-04-23T14:54:11.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.055+0000 7fd33fe00640 1 -- 172.21.15.83:0/2004641004 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd33810c750 con 0x7fd33819a940 2024-04-23T14:54:11.057 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.057+0000 7fd336000640 1 -- 172.21.15.83:0/2004641004 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7fd3300bd980 con 0x7fd33819a940 2024-04-23T14:54:11.057 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:54:11.058 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:54:11.058 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:54:11.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.060+0000 7fd33fe00640 1 -- 172.21.15.83:0/2004641004 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd3200a3d20 msgr2=0x7fd3200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.060+0000 7fd33fe00640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd3200a3d20 0x7fd3200a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd33819ba80 tx=0x7fd32c009260 comp rx=0 tx=0).stop 2024-04-23T14:54:11.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.060+0000 7fd33fe00640 1 -- 172.21.15.83:0/2004641004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd33819a940 msgr2=0x7fd33819adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.060+0000 7fd33fe00640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd33819a940 0x7fd33819adf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd330009b30 tx=0x7fd3300305d0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.060+0000 7fd33fe00640 1 -- 172.21.15.83:0/2004641004 shutdown_connections 2024-04-23T14:54:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.060+0000 7fd33fe00640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd3200a3d20 0x7fd3200a61e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.060+0000 7fd33fe00640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd33819a940 0x7fd33819adf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.060+0000 7fd33fe00640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd338109520 0x7fd3381a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.060+0000 7fd33fe00640 1 --2- 172.21.15.83:0/2004641004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd338104450 0x7fd3381a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.060+0000 7fd33fe00640 1 -- 172.21.15.83:0/2004641004 >> 172.21.15.83:0/2004641004 conn(0x7fd3380ffcf0 msgr2=0x7fd338101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.060+0000 7fd33fe00640 1 -- 172.21.15.83:0/2004641004 shutdown_connections 2024-04-23T14:54:11.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.060+0000 7fd33fe00640 1 -- 172.21.15.83:0/2004641004 wait complete. 2024-04-23T14:54:11.080 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:54:11.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.283+0000 7f7bcbe00640 1 -- 172.21.15.83:0/1644372905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc4109520 msgr2=0x7f7bc41110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.283+0000 7f7bcbe00640 1 --2- 172.21.15.83:0/1644372905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc4109520 0x7f7bc41110f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009ca0 tx=0x7f7bbc02a8b0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.283+0000 7f7bcbe00640 1 -- 172.21.15.83:0/1644372905 shutdown_connections 2024-04-23T14:54:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.283+0000 7f7bcbe00640 1 --2- 172.21.15.83:0/1644372905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc4109520 0x7f7bc41110f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.283+0000 7f7bcbe00640 1 --2- 172.21.15.83:0/1644372905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4104e00 0x7f7bc4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.283+0000 7f7bcbe00640 1 --2- 172.21.15.83:0/1644372905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bc4104450 0x7f7bc4104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.283+0000 7f7bcbe00640 1 -- 172.21.15.83:0/1644372905 >> 172.21.15.83:0/1644372905 conn(0x7f7bc40ffc70 msgr2=0x7f7bc4102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.284 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.283+0000 7f7bcbe00640 1 -- 172.21.15.83:0/1644372905 shutdown_connections 2024-04-23T14:54:11.285 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.283+0000 7f7bcbe00640 1 -- 172.21.15.83:0/1644372905 wait complete. 2024-04-23T14:54:11.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.284+0000 7f7bcbe00640 1 Processor -- start 2024-04-23T14:54:11.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.284+0000 7f7bcbe00640 1 -- start start 2024-04-23T14:54:11.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.285+0000 7f7bcbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bc4104450 0x7f7bc41a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.285+0000 7f7bcbe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4109520 0x7f7bc41a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.285+0000 7f7bcbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc419a8d0 0x7f7bc419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.285+0000 7f7bcbe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc4114610 con 0x7f7bc419a8d0 2024-04-23T14:54:11.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.285+0000 7f7bcbe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc4114490 con 0x7f7bc4104450 2024-04-23T14:54:11.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.285+0000 7f7bcbe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc4114790 con 0x7f7bc4109520 2024-04-23T14:54:11.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.285+0000 7f7bc9400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bc4104450 0x7f7bc41a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.285+0000 7f7bc8a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4109520 0x7f7bc41a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.285+0000 7f7bc9400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bc4104450 0x7f7bc41a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54866/0 (socket says 172.21.15.83:54866) 2024-04-23T14:54:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.285+0000 7f7bc9400640 1 -- 172.21.15.83:0/888053346 learned_addr learned my addr 172.21.15.83:0/888053346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.285+0000 7f7bc9e00640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc419a8d0 0x7f7bc419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.286+0000 7f7bc8a00640 1 -- 172.21.15.83:0/888053346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bc4104450 msgr2=0x7f7bc41a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.286+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bc4104450 0x7f7bc41a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.286+0000 7f7bc8a00640 1 -- 172.21.15.83:0/888053346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc419a8d0 msgr2=0x7f7bc419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.286+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc419a8d0 0x7f7bc419ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.286+0000 7f7bc8a00640 1 -- 172.21.15.83:0/888053346 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc419b3f0 con 0x7f7bc4109520 2024-04-23T14:54:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.286+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4109520 0x7f7bc41a0c60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000e9c0 tx=0x7f7bb000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.286+0000 7f7bc2000640 1 -- 172.21.15.83:0/888053346 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb000cd80 con 0x7f7bc4109520 2024-04-23T14:54:11.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.286+0000 7f7bcbe00640 1 -- 172.21.15.83:0/888053346 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc419b6e0 con 0x7f7bc4109520 2024-04-23T14:54:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.286+0000 7f7bc2000640 1 -- 172.21.15.83:0/888053346 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bb00035f0 con 0x7f7bc4109520 2024-04-23T14:54:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.286+0000 7f7bc2000640 1 -- 172.21.15.83:0/888053346 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0010670 con 0x7f7bc4109520 2024-04-23T14:54:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.286+0000 7f7bcbe00640 1 -- 172.21.15.83:0/888053346 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc41a7980 con 0x7f7bc4109520 2024-04-23T14:54:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.289+0000 7f7bcebcd740 1 Processor -- start 2024-04-23T14:54:11.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.290+0000 7f7bcebcd740 1 -- start start 2024-04-23T14:54:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.290+0000 7f7bcebcd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805cffd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.290+0000 7f7bcebcd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 0x556805d0a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.290+0000 7f7bcebcd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d0a940 0x556805d0cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.290+0000 7f7bcebcd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556805c651d0 con 0x556805cff950 2024-04-23T14:54:11.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.290+0000 7f7bcebcd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556805c0b450 con 0x556805d0a940 2024-04-23T14:54:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.290+0000 7f7bcebcd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556805c61190 con 0x556805d00270 2024-04-23T14:54:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.290+0000 7f7bc2000640 1 -- 172.21.15.83:0/888053346 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7bb00037a0 con 0x7f7bc4109520 2024-04-23T14:54:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.290+0000 7f7bc9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805cffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.290+0000 7f7bc9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805cffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47468/0 (socket says 172.21.15.83:47468) 2024-04-23T14:54:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.290+0000 7f7bc9e00640 1 -- 172.21.15.83:0/2010303696 learned_addr learned my addr 172.21.15.83:0/2010303696 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.290+0000 7f7bc9400640 1 --2- 172.21.15.83:0/2010303696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 0x556805d0a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.291+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/2010303696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d0a940 0x556805d0cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.291+0000 7f7bc9e00640 1 -- 172.21.15.83:0/2010303696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 msgr2=0x556805d0a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.291+0000 7f7bc9e00640 1 --2- 172.21.15.83:0/2010303696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 0x556805d0a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.291+0000 7f7bc9e00640 1 -- 172.21.15.83:0/2010303696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d0a940 msgr2=0x556805d0cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.291+0000 7f7bc9e00640 1 --2- 172.21.15.83:0/2010303696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d0a940 0x556805d0cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.291+0000 7f7bc9e00640 1 -- 172.21.15.83:0/2010303696 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556805cc8310 con 0x556805cff950 2024-04-23T14:54:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.291+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/2010303696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d0a940 0x556805d0cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:54:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.291+0000 7f7bc9400640 1 --2- 172.21.15.83:0/2010303696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 0x556805d0a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.291+0000 7f7bc9e00640 1 --2- 172.21.15.83:0/2010303696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805cffd30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009ca0 tx=0x7f7bbc0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bca23cab8c100e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.291+0000 7f7bb6600640 1 -- 172.21.15.83:0/2010303696 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc030760 con 0x556805cff950 2024-04-23T14:54:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.291+0000 7f7bb6600640 1 -- 172.21.15.83:0/2010303696 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bbc036030 con 0x556805cff950 2024-04-23T14:54:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.292+0000 7f7bb6600640 1 -- 172.21.15.83:0/2010303696 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc004360 con 0x556805cff950 2024-04-23T14:54:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.292+0000 7f7bc2000640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7bac0a3d20 0x7f7bac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.292+0000 7f7bc2000640 1 -- 172.21.15.83:0/888053346 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7bb00d0a80 con 0x7f7bc4109520 2024-04-23T14:54:11.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.292+0000 7f7bc9400640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7bac0a3d20 0x7f7bac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.293+0000 7f7bcebcd740 1 -- 172.21.15.83:0/2010303696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 msgr2=0x556805cffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.293+0000 7f7bc9400640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7bac0a3d20 0x7f7bac0a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x556805d00f10 tx=0x7f7bb8005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.293+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/2010303696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805cffd30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009ca0 tx=0x7f7bbc0093e0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.293+0000 7f7bcebcd740 1 -- 172.21.15.83:0/2010303696 shutdown_connections 2024-04-23T14:54:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.293+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/2010303696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d0a940 0x556805d0cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.293+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/2010303696 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 0x556805d0a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.293+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/2010303696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805cffd30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.293+0000 7f7bcebcd740 1 -- 172.21.15.83:0/2010303696 >> 172.21.15.83:0/2010303696 conn(0x556805cfbc70 msgr2=0x556805cfc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.294+0000 7f7bcebcd740 1 -- 172.21.15.83:0/2010303696 shutdown_connections 2024-04-23T14:54:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.294+0000 7f7bcebcd740 1 -- 172.21.15.83:0/2010303696 wait complete. 2024-04-23T14:54:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.294+0000 7f7bcebcd740 1 -- start start 2024-04-23T14:54:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.294+0000 7f7bcebcd740 1 Processor -- start 2024-04-23T14:54:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bcebcd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805d0a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bcebcd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 0x556805d076e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bcebcd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d0a940 0x556805d07e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bcebcd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556805d0d770 con 0x556805cff950 2024-04-23T14:54:11.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bcebcd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556805c1e810 con 0x556805d00270 2024-04-23T14:54:11.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bcebcd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556805d0d8f0 con 0x556805d0a940 2024-04-23T14:54:11.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bcebcd740 10 client.0 _ll_register_callbacks cb 0x556805bec140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:11.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bc9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805d0a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.302 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bc8a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 0x556805d076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bc9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805d0a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47478/0 (socket says 172.21.15.83:47478) 2024-04-23T14:54:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bc9e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d0a940 0x556805d07e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bc8a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 0x556805d076e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54896/0 (socket says 172.21.15.83:54896) 2024-04-23T14:54:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bc9400640 1 -- 172.21.15.83:0/794338975 learned_addr learned my addr 172.21.15.83:0/794338975 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bc8a00640 1 -- 172.21.15.83:0/794338975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d0a940 msgr2=0x556805d07e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/794338975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d0a940 0x556805d07e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bc8a00640 1 -- 172.21.15.83:0/794338975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 msgr2=0x556805d0a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/794338975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805d0a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.295+0000 7f7bc8a00640 1 -- 172.21.15.83:0/794338975 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556805c2f3f0 con 0x556805d00270 2024-04-23T14:54:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.296+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/794338975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 0x556805d076e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00ce180 tx=0x7f7bb009c600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99f638620008dbcc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.296+0000 7f7baaa00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:54:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.296+0000 7f7baaa00640 1 -- 172.21.15.83:0/794338975 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb009c9d0 con 0x556805d00270 2024-04-23T14:54:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.296+0000 7f7baaa00640 1 -- 172.21.15.83:0/794338975 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7bb0099040 con 0x556805d00270 2024-04-23T14:54:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.296+0000 7f7bcebcd740 1 -- 172.21.15.83:0/794338975 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bbc036030 con 0x556805d00270 2024-04-23T14:54:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.296+0000 7f7baaa00640 1 -- 172.21.15.83:0/794338975 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7bb00a0d80 con 0x556805d00270 2024-04-23T14:54:11.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.296+0000 7f7bcebcd740 10 client.26321 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:11.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.296+0000 7f7bcebcd740 1 -- 172.21.15.83:0/794338975 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5568059773f0 con 0x556805d00270 2024-04-23T14:54:11.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.297+0000 7f7baaa00640 1 -- 172.21.15.83:0/794338975 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bbc036030 con 0x556805d00270 2024-04-23T14:54:11.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.297+0000 7f7baaa00640 1 -- 172.21.15.83:0/794338975 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f7bb00870b0 con 0x556805d00270 2024-04-23T14:54:11.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.297+0000 7f7bcebcd740 10 client.26321 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:11.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.297+0000 7f7bcebcd740 10 client.26321 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:54:11.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.297+0000 7f7bcebcd740 20 client.26321 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:11.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.297+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/794338975 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x556805d1aad0 0x556805d1cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.297+0000 7f7bcebcd740 4 client.26321 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:54:11.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.297+0000 7f7bcebcd740 1 -- 172.21.15.83:0/794338975 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556805c72bf0 con 0x556805d1aad0 2024-04-23T14:54:11.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.297+0000 7f7bc9400640 1 --2- 172.21.15.83:0/794338975 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x556805d1aad0 0x556805d1cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.298+0000 7f7bc9400640 1 --2- 172.21.15.83:0/794338975 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x556805d1aad0 0x556805d1cec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a471f981dba219cd server_cookie=a01d1efc2f489a45 in_seq=0 out_seq=0 2024-04-23T14:54:11.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.298+0000 7f7baaa00640 10 client.26321 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:54:11.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7baaa00640 1 -- 172.21.15.83:0/794338975 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556805c72bf0 con 0x556805d1aad0 2024-04-23T14:54:11.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7baaa00640 10 client.26321 handle_command_reply: tid=0 2024-04-23T14:54:11.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7bcebcd740 1 client.26321 shutdown 2024-04-23T14:54:11.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7bcebcd740 1 -- 172.21.15.83:0/794338975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 msgr2=0x556805d076e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/794338975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 0x556805d076e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00ce180 tx=0x7f7bb009c600 comp rx=0 tx=0).stop 2024-04-23T14:54:11.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7bcebcd740 1 -- 172.21.15.83:0/794338975 shutdown_connections 2024-04-23T14:54:11.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/794338975 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x556805d1aad0 0x556805d1cec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/794338975 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d0a940 0x556805d07e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/794338975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 0x556805d076e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/794338975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805d0a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7baaa00640 0 client.26321 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:54:11.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7bcebcd740 1 -- 172.21.15.83:0/794338975 >> 172.21.15.83:0/794338975 conn(0x556805cfbc70 msgr2=0x556805d01a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7bcebcd740 1 -- 172.21.15.83:0/794338975 shutdown_connections 2024-04-23T14:54:11.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.299+0000 7f7bcebcd740 1 -- 172.21.15.83:0/794338975 wait complete. 2024-04-23T14:54:11.309 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.300+0000 7f7bcebcd740 20 client.26321 trim_cache size 0 max 16384 2024-04-23T14:54:11.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.314+0000 7f7bcebcd740 1 Processor -- start 2024-04-23T14:54:11.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.314+0000 7f7bcebcd740 1 -- start start 2024-04-23T14:54:11.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.315+0000 7f7bcebcd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805cff950 0x556805d16630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.315+0000 7f7bcebcd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 0x556805d09f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.315+0000 7f7bcebcd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805d0a940 0x556805d22740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.315+0000 7f7bcebcd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41b26e0 con 0x556805d0a940 2024-04-23T14:54:11.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.315+0000 7f7bcebcd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41affb0 con 0x556805cff950 2024-04-23T14:54:11.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.315+0000 7f7bcebcd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41ade20 con 0x556805d00270 2024-04-23T14:54:11.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.315+0000 7f7bc9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805d0a940 0x556805d22740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.315+0000 7f7bc8a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 0x556805d09f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.315+0000 7f7bc9400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805cff950 0x556805d16630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.315+0000 7f7bc8a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 0x556805d09f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50748/0 (socket says 172.21.15.83:50748) 2024-04-23T14:54:11.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.315+0000 7f7bc8a00640 1 -- 172.21.15.83:0/770776633 learned_addr learned my addr 172.21.15.83:0/770776633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.316+0000 7f7bc8a00640 1 -- 172.21.15.83:0/770776633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805cff950 msgr2=0x556805d16630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.316+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/770776633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805cff950 0x556805d16630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.316+0000 7f7bc8a00640 1 -- 172.21.15.83:0/770776633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805d0a940 msgr2=0x556805d22740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.316+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/770776633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805d0a940 0x556805d22740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.316+0000 7f7bc8a00640 1 -- 172.21.15.83:0/770776633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556805a417d0 con 0x556805d00270 2024-04-23T14:54:11.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.316+0000 7f7bc9e00640 1 --2- 172.21.15.83:0/770776633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805d0a940 0x556805d22740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:11.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.316+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/770776633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 0x556805d09f20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7bb009c960 tx=0x7f7bb0005e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2799954f09cd41d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.316+0000 7f7bb5c00640 1 -- 172.21.15.83:0/770776633 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb009c9d0 con 0x556805d00270 2024-04-23T14:54:11.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.316+0000 7f7bb5c00640 1 -- 172.21.15.83:0/770776633 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bb00a5640 con 0x556805d00270 2024-04-23T14:54:11.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.316+0000 7f7bb5c00640 1 -- 172.21.15.83:0/770776633 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00991b0 con 0x556805d00270 2024-04-23T14:54:11.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.317+0000 7f7bcebcd740 1 -- 172.21.15.83:0/770776633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 msgr2=0x556805d09f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.317+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/770776633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 0x556805d09f20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7bb009c960 tx=0x7f7bb0005e30 comp rx=0 tx=0).stop 2024-04-23T14:54:11.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.317+0000 7f7bcebcd740 1 -- 172.21.15.83:0/770776633 shutdown_connections 2024-04-23T14:54:11.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.317+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/770776633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805d0a940 0x556805d22740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.317+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/770776633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d00270 0x556805d09f20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.317+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/770776633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805cff950 0x556805d16630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.317+0000 7f7bcebcd740 1 -- 172.21.15.83:0/770776633 >> 172.21.15.83:0/770776633 conn(0x556805cfbc70 msgr2=0x556805d010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.317+0000 7f7bcebcd740 1 -- 172.21.15.83:0/770776633 shutdown_connections 2024-04-23T14:54:11.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.317+0000 7f7bcebcd740 1 -- 172.21.15.83:0/770776633 wait complete. 2024-04-23T14:54:11.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.317+0000 7f7bcebcd740 1 -- start start 2024-04-23T14:54:11.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.317+0000 7f7bcebcd740 1 Processor -- start 2024-04-23T14:54:11.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.318+0000 7f7bcebcd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805d13060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.318+0000 7f7bcebcd740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 0x556805d04e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.318+0000 7f7bcebcd740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d0a940 0x556805d0ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.318+0000 7f7bcebcd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41ae470 con 0x556805cff950 2024-04-23T14:54:11.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.318+0000 7f7bcebcd740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41af8e0 con 0x556805d00270 2024-04-23T14:54:11.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.318+0000 7f7bcebcd740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc41aeb40 con 0x556805d0a940 2024-04-23T14:54:11.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.318+0000 7f7bcebcd740 10 client.0 _ll_register_callbacks cb 0x7f7bb0003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:11.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.318+0000 7f7bc9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805d13060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.318+0000 7f7bc9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805d13060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47494/0 (socket says 172.21.15.83:47494) 2024-04-23T14:54:11.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.318+0000 7f7bc9400640 1 -- 172.21.15.83:0/1159412197 learned_addr learned my addr 172.21.15.83:0/1159412197 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.318+0000 7f7bc9e00640 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d0a940 0x556805d0ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.318+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 0x556805d04e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.319+0000 7f7bc9400640 1 -- 172.21.15.83:0/1159412197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d0a940 msgr2=0x556805d0ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.319+0000 7f7bc9400640 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d0a940 0x556805d0ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.319+0000 7f7bc9400640 1 -- 172.21.15.83:0/1159412197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 msgr2=0x556805d04e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.319+0000 7f7bc9400640 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 0x556805d04e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.319+0000 7f7bc9400640 1 -- 172.21.15.83:0/1159412197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556805d0bf70 con 0x556805cff950 2024-04-23T14:54:11.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.319+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 0x556805d04e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:11.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.319+0000 7f7bc9400640 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805d13060 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x556805d1d6b0 tx=0x7f7bb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91151281b8c5cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.319+0000 7f7baaa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:54:11.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.319+0000 7f7baaa00640 1 -- 172.21.15.83:0/1159412197 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8004020 con 0x556805cff950 2024-04-23T14:54:11.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.319+0000 7f7bcebcd740 1 -- 172.21.15.83:0/1159412197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bb00a5640 con 0x556805cff950 2024-04-23T14:54:11.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.319+0000 7f7baaa00640 1 -- 172.21.15.83:0/1159412197 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7bb80041c0 con 0x556805cff950 2024-04-23T14:54:11.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.320+0000 7f7baaa00640 1 -- 172.21.15.83:0/1159412197 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7bb803b050 con 0x556805cff950 2024-04-23T14:54:11.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.320+0000 7f7bcebcd740 10 client.16119 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:11.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.320+0000 7f7bcebcd740 1 -- 172.21.15.83:0/1159412197 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556805d81600 con 0x556805cff950 2024-04-23T14:54:11.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.320+0000 7f7baaa00640 1 -- 172.21.15.83:0/1159412197 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bb00a5640 con 0x556805cff950 2024-04-23T14:54:11.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.320+0000 7f7baaa00640 1 -- 172.21.15.83:0/1159412197 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f7bb80039f0 con 0x556805cff950 2024-04-23T14:54:11.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.320+0000 7f7bcebcd740 10 client.16119 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:11.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.320+0000 7f7bcebcd740 10 client.16119 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:54:11.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.320+0000 7f7bcebcd740 20 client.16119 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:11.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.320+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x556805d81d90 0x556805d84180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.320+0000 7f7bcebcd740 4 client.16119 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:54:11.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.320+0000 7f7bcebcd740 1 -- 172.21.15.83:0/1159412197 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556805c6ca60 con 0x556805d81d90 2024-04-23T14:54:11.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.321+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x556805d81d90 0x556805d84180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.321+0000 7f7bc8a00640 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x556805d81d90 0x556805d84180 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89b9d538e7d6264b server_cookie=6b7ee0167f5a0083 in_seq=0 out_seq=0 2024-04-23T14:54:11.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.321+0000 7f7baaa00640 10 client.16119 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:54:11.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.321+0000 7f7baaa00640 1 -- 172.21.15.83:0/1159412197 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556805c6ca60 con 0x556805d81d90 2024-04-23T14:54:11.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.321+0000 7f7baaa00640 10 client.16119 handle_command_reply: tid=0 2024-04-23T14:54:11.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.321+0000 7f7bcebcd740 1 client.16119 shutdown 2024-04-23T14:54:11.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.322+0000 7f7bcebcd740 1 -- 172.21.15.83:0/1159412197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 msgr2=0x556805d13060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.322+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805d13060 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x556805d1d6b0 tx=0x7f7bb8017000 comp rx=0 tx=0).stop 2024-04-23T14:54:11.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.322+0000 7f7bcebcd740 1 -- 172.21.15.83:0/1159412197 shutdown_connections 2024-04-23T14:54:11.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.322+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x556805d81d90 0x556805d84180 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.322+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556805d0a940 0x556805d0ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.322+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556805d00270 0x556805d04e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.322+0000 7f7bcebcd740 1 --2- 172.21.15.83:0/1159412197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556805cff950 0x556805d13060 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.322+0000 7f7baaa00640 0 client.16119 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:54:11.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.322+0000 7f7bcebcd740 1 -- 172.21.15.83:0/1159412197 >> 172.21.15.83:0/1159412197 conn(0x556805cfbc70 msgr2=0x556805d15eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.322+0000 7f7bcebcd740 1 -- 172.21.15.83:0/1159412197 shutdown_connections 2024-04-23T14:54:11.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.322+0000 7f7bcebcd740 1 -- 172.21.15.83:0/1159412197 wait complete. 2024-04-23T14:54:11.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.322+0000 7f7bcebcd740 20 client.16119 trim_cache size 0 max 16384 2024-04-23T14:54:11.332 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:54:11.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.323+0000 7f7bcbe00640 1 -- 172.21.15.83:0/888053346 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7bac0a3d20 msgr2=0x7f7bac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.323+0000 7f7bcbe00640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7bac0a3d20 0x7f7bac0a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x556805d00f10 tx=0x7f7bb8005f90 comp rx=0 tx=0).stop 2024-04-23T14:54:11.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.323+0000 7f7bcbe00640 1 -- 172.21.15.83:0/888053346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4109520 msgr2=0x7f7bc41a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.323+0000 7f7bcbe00640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4109520 0x7f7bc41a0c60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7bb000e9c0 tx=0x7f7bb000ee80 comp rx=0 tx=0).stop 2024-04-23T14:54:11.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.323+0000 7f7bcbe00640 1 -- 172.21.15.83:0/888053346 shutdown_connections 2024-04-23T14:54:11.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.323+0000 7f7bcbe00640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7bac0a3d20 0x7f7bac0a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.323+0000 7f7bcbe00640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc419a8d0 0x7f7bc419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.323+0000 7f7bcbe00640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7bc4109520 0x7f7bc41a0c60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.323+0000 7f7bcbe00640 1 --2- 172.21.15.83:0/888053346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7bc4104450 0x7f7bc41a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.323+0000 7f7bcbe00640 1 -- 172.21.15.83:0/888053346 >> 172.21.15.83:0/888053346 conn(0x7f7bc40ffc70 msgr2=0x7f7bc41007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.323+0000 7f7bcbe00640 1 -- 172.21.15.83:0/888053346 shutdown_connections 2024-04-23T14:54:11.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.323+0000 7f7bcbe00640 1 -- 172.21.15.83:0/888053346 wait complete. 2024-04-23T14:54:11.340 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:54:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.535+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1660448820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d64076530 msgr2=0x7f1d64076910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.535+0000 7f1d6ac00640 1 --2- 172.21.15.83:0/1660448820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d64076530 0x7f1d64076910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009930 tx=0x7f1d5802a420 comp rx=0 tx=0).stop 2024-04-23T14:54:11.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.535+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1660448820 shutdown_connections 2024-04-23T14:54:11.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.535+0000 7f1d6ac00640 1 --2- 172.21.15.83:0/1660448820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d64112d40 0x7f1d64115130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.536+0000 7f1d6ac00640 1 --2- 172.21.15.83:0/1660448820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64076e50 0x7f1d64112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.536+0000 7f1d6ac00640 1 --2- 172.21.15.83:0/1660448820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d64076530 0x7f1d64076910 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.536+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1660448820 >> 172.21.15.83:0/1660448820 conn(0x7f1d640ffb70 msgr2=0x7f1d64101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.536+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1660448820 shutdown_connections 2024-04-23T14:54:11.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.536+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1660448820 wait complete. 2024-04-23T14:54:11.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.537+0000 7f1d6ac00640 1 Processor -- start 2024-04-23T14:54:11.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.537+0000 7f1d6ac00640 1 -- start start 2024-04-23T14:54:11.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.537+0000 7f1d6ac00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d64076530 0x7f1d6419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.537+0000 7f1d6ac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64076e50 0x7f1d6419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d6ac00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d64112d40 0x7f1d641a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d6ac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64118890 con 0x7f1d64076e50 2024-04-23T14:54:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d6ac00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64118710 con 0x7f1d64076530 2024-04-23T14:54:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d6ac00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64118a10 con 0x7f1d64112d40 2024-04-23T14:54:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d69800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d64076530 0x7f1d6419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d69800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d64076530 0x7f1d6419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54934/0 (socket says 172.21.15.83:54934) 2024-04-23T14:54:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d69800640 1 -- 172.21.15.83:0/1817707731 learned_addr learned my addr 172.21.15.83:0/1817707731 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d6a200640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d64112d40 0x7f1d641a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d68e00640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64076e50 0x7f1d6419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d69800640 1 -- 172.21.15.83:0/1817707731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d64112d40 msgr2=0x7f1d641a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d69800640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d64112d40 0x7f1d641a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d69800640 1 -- 172.21.15.83:0/1817707731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64076e50 msgr2=0x7f1d6419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d69800640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64076e50 0x7f1d6419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d69800640 1 -- 172.21.15.83:0/1817707731 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d641a3dc0 con 0x7f1d64076530 2024-04-23T14:54:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.538+0000 7f1d6a200640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d64112d40 0x7f1d641a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:11.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.539+0000 7f1d69800640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d64076530 0x7f1d6419edd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009230 tx=0x7f1d58004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.541+0000 7f1d62000640 1 -- 172.21.15.83:0/1817707731 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5803f030 con 0x7f1d64076530 2024-04-23T14:54:11.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.541+0000 7f1d62000640 1 -- 172.21.15.83:0/1817707731 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d58004440 con 0x7f1d64076530 2024-04-23T14:54:11.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.541+0000 7f1d62000640 1 -- 172.21.15.83:0/1817707731 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58035070 con 0x7f1d64076530 2024-04-23T14:54:11.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.541+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1817707731 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d641a4050 con 0x7f1d64076530 2024-04-23T14:54:11.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.542+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1817707731 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d641ab940 con 0x7f1d64076530 2024-04-23T14:54:11.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.545+0000 7f1d62000640 1 -- 172.21.15.83:0/1817707731 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1d58042030 con 0x7f1d64076530 2024-04-23T14:54:11.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.545+0000 7f1d6f011740 1 Processor -- start 2024-04-23T14:54:11.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.545+0000 7f1d6f011740 1 -- start start 2024-04-23T14:54:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.545+0000 7f1d6f011740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b0ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.545+0000 7f1d6f011740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b1a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.545+0000 7f1d6f011740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b1cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.545+0000 7f1d6f011740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ff49a751d0 con 0x55ff49b1ab30 2024-04-23T14:54:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.545+0000 7f1d6f011740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ff49a1b450 con 0x55ff49b0fb40 2024-04-23T14:54:11.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.545+0000 7f1d6f011740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ff49a71190 con 0x55ff49b10460 2024-04-23T14:54:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d6a200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b0ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d68e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b1cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d6a200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b0ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54946/0 (socket says 172.21.15.83:54946) 2024-04-23T14:54:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d68e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b1cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47512/0 (socket says 172.21.15.83:47512) 2024-04-23T14:54:11.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d6a200640 1 -- 172.21.15.83:0/2558816981 learned_addr learned my addr 172.21.15.83:0/2558816981 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d69800640 1 --2- 172.21.15.83:0/2558816981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b1a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d6a200640 1 -- 172.21.15.83:0/2558816981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 msgr2=0x55ff49b1a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d6a200640 1 --2- 172.21.15.83:0/2558816981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b1a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d6a200640 1 -- 172.21.15.83:0/2558816981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 msgr2=0x55ff49b1cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d6a200640 1 --2- 172.21.15.83:0/2558816981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b1cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d6a200640 1 -- 172.21.15.83:0/2558816981 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff49ad8310 con 0x55ff49b0fb40 2024-04-23T14:54:11.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d68e00640 1 --2- 172.21.15.83:0/2558816981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b1cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:11.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.546+0000 7f1d62000640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1d4c0a3d20 0x7f1d4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.547+0000 7f1d6a200640 1 --2- 172.21.15.83:0/2558816981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b0ff20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c011760 tx=0x7f1d5c011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efbd7445bf46f4a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.547+0000 7f1d62000640 1 -- 172.21.15.83:0/1817707731 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1d580f0b50 con 0x7f1d64076530 2024-04-23T14:54:11.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.547+0000 7f1d69800640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1d4c0a3d20 0x7f1d4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.547+0000 7f1d56a00640 1 -- 172.21.15.83:0/2558816981 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c004d20 con 0x55ff49b0fb40 2024-04-23T14:54:11.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.547+0000 7f1d56a00640 1 -- 172.21.15.83:0/2558816981 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d5c00f9a0 con 0x55ff49b0fb40 2024-04-23T14:54:11.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.547+0000 7f1d56a00640 1 -- 172.21.15.83:0/2558816981 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c004590 con 0x55ff49b0fb40 2024-04-23T14:54:11.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.548+0000 7f1d6f011740 1 -- 172.21.15.83:0/2558816981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 msgr2=0x55ff49b0ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.548+0000 7f1d69800640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1d4c0a3d20 0x7f1d4c0a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1d580ee360 tx=0x7f1d580f46a0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.548+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2558816981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b0ff20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c011760 tx=0x7f1d5c011c20 comp rx=0 tx=0).stop 2024-04-23T14:54:11.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.548+0000 7f1d6f011740 1 -- 172.21.15.83:0/2558816981 shutdown_connections 2024-04-23T14:54:11.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.548+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2558816981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b1cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.548+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2558816981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b1a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.548+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2558816981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b0ff20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.548+0000 7f1d6f011740 1 -- 172.21.15.83:0/2558816981 >> 172.21.15.83:0/2558816981 conn(0x55ff49b0be60 msgr2=0x55ff49b0c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.548+0000 7f1d6f011740 1 -- 172.21.15.83:0/2558816981 shutdown_connections 2024-04-23T14:54:11.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.548+0000 7f1d6f011740 1 -- 172.21.15.83:0/2558816981 wait complete. 2024-04-23T14:54:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.548+0000 7f1d6f011740 1 -- start start 2024-04-23T14:54:11.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d6f011740 1 Processor -- start 2024-04-23T14:54:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d6f011740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b1a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d6f011740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d6f011740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b17fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d6f011740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ff49b1d9c0 con 0x55ff49b1ab30 2024-04-23T14:54:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d6f011740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ff49a2e810 con 0x55ff49b0fb40 2024-04-23T14:54:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d6f011740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ff49b1db40 con 0x55ff49b10460 2024-04-23T14:54:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d6f011740 10 client.0 _ll_register_callbacks cb 0x55ff499fc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:11.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d6a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b17fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d68e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d6a200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b17fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47514/0 (socket says 172.21.15.83:47514) 2024-04-23T14:54:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d6a200640 1 -- 172.21.15.83:0/2217378631 learned_addr learned my addr 172.21.15.83:0/2217378631 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.549+0000 7f1d6a200640 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b1a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.550+0000 7f1d68e00640 1 -- 172.21.15.83:0/2217378631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 msgr2=0x55ff49b1a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.550+0000 7f1d68e00640 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b1a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.550+0000 7f1d68e00640 1 -- 172.21.15.83:0/2217378631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 msgr2=0x55ff49b17fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.550+0000 7f1d68e00640 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b17fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.550+0000 7f1d68e00640 1 -- 172.21.15.83:0/2217378631 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff49a3f3f0 con 0x55ff49b10460 2024-04-23T14:54:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.550+0000 7f1d6a200640 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b17fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.550+0000 7f1d68e00640 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b178b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1d64079210 tx=0x7f1d50012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79f73aecc4a0569 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.550+0000 7f1d4b400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:54:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.550+0000 7f1d4b400640 1 -- 172.21.15.83:0/2217378631 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50004030 con 0x55ff49b10460 2024-04-23T14:54:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.550+0000 7f1d4b400640 1 -- 172.21.15.83:0/2217378631 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1d500041d0 con 0x55ff49b10460 2024-04-23T14:54:11.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.550+0000 7f1d6f011740 1 -- 172.21.15.83:0/2217378631 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d5c00f9a0 con 0x55ff49b10460 2024-04-23T14:54:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.551+0000 7f1d4b400640 1 -- 172.21.15.83:0/2217378631 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1d500043d0 con 0x55ff49b10460 2024-04-23T14:54:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.551+0000 7f1d6f011740 10 client.26308 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.551+0000 7f1d6f011740 1 -- 172.21.15.83:0/2217378631 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff497873f0 con 0x55ff49b10460 2024-04-23T14:54:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.551+0000 7f1d4b400640 1 -- 172.21.15.83:0/2217378631 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d5c00f9a0 con 0x55ff49b10460 2024-04-23T14:54:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.551+0000 7f1d4b400640 1 -- 172.21.15.83:0/2217378631 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f1d50003bc0 con 0x55ff49b10460 2024-04-23T14:54:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.551+0000 7f1d6f011740 10 client.26308 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.551+0000 7f1d6f011740 10 client.26308 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:54:11.563 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.551+0000 7f1d6f011740 20 client.26308 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:11.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.551+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ff49b2ae70 0x55ff49b2d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.551+0000 7f1d6f011740 4 client.26308 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:54:11.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.551+0000 7f1d6f011740 1 -- 172.21.15.83:0/2217378631 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff49a82bf0 con 0x55ff49b2ae70 2024-04-23T14:54:11.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.552+0000 7f1d6a200640 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ff49b2ae70 0x55ff49b2d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.552+0000 7f1d6a200640 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ff49b2ae70 0x55ff49b2d290 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=befaf8eea1f1132d server_cookie=5ff2f0bb0d6af1b8 in_seq=0 out_seq=0 2024-04-23T14:54:11.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.552+0000 7f1d4b400640 10 client.26308 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:54:11.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d4b400640 1 -- 172.21.15.83:0/2217378631 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff49a82bf0 con 0x55ff49b2ae70 2024-04-23T14:54:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d4b400640 10 client.26308 handle_command_reply: tid=0 2024-04-23T14:54:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d6f011740 1 client.26308 shutdown 2024-04-23T14:54:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d6f011740 1 -- 172.21.15.83:0/2217378631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 msgr2=0x55ff49b178b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b178b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1d64079210 tx=0x7f1d50012b60 comp rx=0 tx=0).stop 2024-04-23T14:54:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d6f011740 1 -- 172.21.15.83:0/2217378631 shutdown_connections 2024-04-23T14:54:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b17fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ff49b2ae70 0x55ff49b2d290 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b178b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d4b400640 0 client.26308 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:54:11.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2217378631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b1a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d6f011740 1 -- 172.21.15.83:0/2217378631 >> 172.21.15.83:0/2217378631 conn(0x55ff49b0be60 msgr2=0x55ff49b11c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d6f011740 1 -- 172.21.15.83:0/2217378631 shutdown_connections 2024-04-23T14:54:11.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d6f011740 1 -- 172.21.15.83:0/2217378631 wait complete. 2024-04-23T14:54:11.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.553+0000 7f1d6f011740 20 client.26308 trim_cache size 0 max 16384 2024-04-23T14:54:11.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.568+0000 7f1d6f011740 1 Processor -- start 2024-04-23T14:54:11.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.568+0000 7f1d6f011740 1 -- start start 2024-04-23T14:54:11.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.569+0000 7f1d6f011740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b268b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.569+0000 7f1d6f011740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b1a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.569+0000 7f1d6f011740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b32b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.569+0000 7f1d6f011740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641b6a10 con 0x55ff49b1ab30 2024-04-23T14:54:11.570 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.569+0000 7f1d6f011740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641b42e0 con 0x55ff49b0fb40 2024-04-23T14:54:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.569+0000 7f1d6f011740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641b2130 con 0x55ff49b10460 2024-04-23T14:54:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.569+0000 7f1d6a200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b268b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.569+0000 7f1d6a200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b268b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54972/0 (socket says 172.21.15.83:54972) 2024-04-23T14:54:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.569+0000 7f1d6a200640 1 -- 172.21.15.83:0/1799318194 learned_addr learned my addr 172.21.15.83:0/1799318194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.569+0000 7f1d6a200640 1 --2- 172.21.15.83:0/1799318194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b32b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.569+0000 7f1d68e00640 1 --2- 172.21.15.83:0/1799318194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b1a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.570+0000 7f1d6a200640 1 -- 172.21.15.83:0/1799318194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 msgr2=0x55ff49b1a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.570+0000 7f1d6a200640 1 --2- 172.21.15.83:0/1799318194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b1a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.570+0000 7f1d6a200640 1 -- 172.21.15.83:0/1799318194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 msgr2=0x55ff49b32b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.570+0000 7f1d6a200640 1 --2- 172.21.15.83:0/1799318194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b32b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.570+0000 7f1d6a200640 1 -- 172.21.15.83:0/1799318194 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff498517d0 con 0x55ff49b0fb40 2024-04-23T14:54:11.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.570+0000 7f1d6a200640 1 --2- 172.21.15.83:0/1799318194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b268b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c005fd0 tx=0x7f1d5c01b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19f3e2644d593ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.570+0000 7f1d56000640 1 -- 172.21.15.83:0/1799318194 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c01e040 con 0x55ff49b0fb40 2024-04-23T14:54:11.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.570+0000 7f1d56000640 1 -- 172.21.15.83:0/1799318194 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d5c00f9a0 con 0x55ff49b0fb40 2024-04-23T14:54:11.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.570+0000 7f1d56000640 1 -- 172.21.15.83:0/1799318194 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c010690 con 0x55ff49b0fb40 2024-04-23T14:54:11.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.571+0000 7f1d6f011740 1 -- 172.21.15.83:0/1799318194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 msgr2=0x55ff49b268b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.571+0000 7f1d6f011740 1 --2- 172.21.15.83:0/1799318194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b268b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c005fd0 tx=0x7f1d5c01b040 comp rx=0 tx=0).stop 2024-04-23T14:54:11.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.571+0000 7f1d6f011740 1 -- 172.21.15.83:0/1799318194 shutdown_connections 2024-04-23T14:54:11.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.571+0000 7f1d6f011740 1 --2- 172.21.15.83:0/1799318194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b1ab30 0x55ff49b32b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.571+0000 7f1d6f011740 1 --2- 172.21.15.83:0/1799318194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b10460 0x55ff49b1a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.571+0000 7f1d6f011740 1 --2- 172.21.15.83:0/1799318194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b268b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.571+0000 7f1d6f011740 1 -- 172.21.15.83:0/1799318194 >> 172.21.15.83:0/1799318194 conn(0x55ff49b0be60 msgr2=0x55ff49b12b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.571+0000 7f1d6f011740 1 -- 172.21.15.83:0/1799318194 shutdown_connections 2024-04-23T14:54:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.571+0000 7f1d6f011740 1 -- 172.21.15.83:0/1799318194 wait complete. 2024-04-23T14:54:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.571+0000 7f1d6f011740 1 -- start start 2024-04-23T14:54:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d6f011740 1 Processor -- start 2024-04-23T14:54:11.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d6f011740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d6f011740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b10460 0x55ff49b1b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d6f011740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b1ab30 0x55ff49b1bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d6f011740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641b27a0 con 0x55ff49b10460 2024-04-23T14:54:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d6f011740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641b3c10 con 0x55ff49b0fb40 2024-04-23T14:54:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d6f011740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1d641b2e70 con 0x55ff49b1ab30 2024-04-23T14:54:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d6f011740 10 client.0 _ll_register_callbacks cb 0x7f1d50009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:11.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d6a200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d68e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b10460 0x55ff49b1b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d6a200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b232a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54986/0 (socket says 172.21.15.83:54986) 2024-04-23T14:54:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d6a200640 1 -- 172.21.15.83:0/2571707576 learned_addr learned my addr 172.21.15.83:0/2571707576 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.572+0000 7f1d6a200640 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b1ab30 0x55ff49b1bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d68e00640 1 -- 172.21.15.83:0/2571707576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b1ab30 msgr2=0x55ff49b1bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d68e00640 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b1ab30 0x55ff49b1bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d68e00640 1 -- 172.21.15.83:0/2571707576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 msgr2=0x55ff49b232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d68e00640 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d68e00640 1 -- 172.21.15.83:0/2571707576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff49b1c510 con 0x55ff49b10460 2024-04-23T14:54:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d6a200640 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b1ab30 0x55ff49b1bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d68e00640 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b10460 0x55ff49b1b6e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1d500105f0 tx=0x7f1d5000f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643e2872c8abf7aa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d4b400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:54:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d4b400640 1 -- 172.21.15.83:0/2571707576 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50029040 con 0x55ff49b10460 2024-04-23T14:54:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d4b400640 1 -- 172.21.15.83:0/2571707576 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1d5000fae0 con 0x55ff49b10460 2024-04-23T14:54:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d6f011740 1 -- 172.21.15.83:0/2571707576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d5c00f9a0 con 0x55ff49b10460 2024-04-23T14:54:11.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d4b400640 1 -- 172.21.15.83:0/2571707576 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1d5000fce0 con 0x55ff49b10460 2024-04-23T14:54:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d6f011740 10 client.16137 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.573+0000 7f1d6f011740 1 -- 172.21.15.83:0/2571707576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff49b91520 con 0x55ff49b10460 2024-04-23T14:54:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.574+0000 7f1d4b400640 1 -- 172.21.15.83:0/2571707576 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d5c00f9a0 con 0x55ff49b10460 2024-04-23T14:54:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.574+0000 7f1d4b400640 1 -- 172.21.15.83:0/2571707576 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f1d50003860 con 0x55ff49b10460 2024-04-23T14:54:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.574+0000 7f1d6f011740 10 client.16137 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:11.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.574+0000 7f1d6f011740 10 client.16137 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:54:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.574+0000 7f1d6f011740 20 client.16137 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.574+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ff49b91c40 0x55ff49b94030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.574+0000 7f1d6f011740 4 client.16137 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:54:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.574+0000 7f1d6f011740 1 -- 172.21.15.83:0/2571707576 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff49a7ca60 con 0x55ff49b91c40 2024-04-23T14:54:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.574+0000 7f1d6a200640 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ff49b91c40 0x55ff49b94030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d6a200640 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ff49b91c40 0x55ff49b94030 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=770c8c3dd565c4b server_cookie=3ea3c613e9b44155 in_seq=0 out_seq=0 2024-04-23T14:54:11.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d4b400640 10 client.16137 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:54:11.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d4b400640 1 -- 172.21.15.83:0/2571707576 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff49a7ca60 con 0x55ff49b91c40 2024-04-23T14:54:11.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d4b400640 10 client.16137 handle_command_reply: tid=0 2024-04-23T14:54:11.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d6f011740 1 client.16137 shutdown 2024-04-23T14:54:11.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d6f011740 1 -- 172.21.15.83:0/2571707576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b10460 msgr2=0x55ff49b1b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b10460 0x55ff49b1b6e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1d500105f0 tx=0x7f1d5000f800 comp rx=0 tx=0).stop 2024-04-23T14:54:11.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d6f011740 1 -- 172.21.15.83:0/2571707576 shutdown_connections 2024-04-23T14:54:11.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ff49b91c40 0x55ff49b94030 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ff49b1ab30 0x55ff49b1bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ff49b10460 0x55ff49b1b6e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d6f011740 1 --2- 172.21.15.83:0/2571707576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ff49b0fb40 0x55ff49b232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d4b400640 0 client.16137 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:54:11.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.575+0000 7f1d6f011740 1 -- 172.21.15.83:0/2571707576 >> 172.21.15.83:0/2571707576 conn(0x55ff49b0be60 msgr2=0x55ff49b0a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6f011740 1 -- 172.21.15.83:0/2571707576 shutdown_connections 2024-04-23T14:54:11.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6f011740 1 -- 172.21.15.83:0/2571707576 wait complete. 2024-04-23T14:54:11.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6f011740 20 client.16137 trim_cache size 0 max 16384 2024-04-23T14:54:11.586 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:54:11.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1817707731 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1d4c0a3d20 msgr2=0x7f1d4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6ac00640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1d4c0a3d20 0x7f1d4c0a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1d580ee360 tx=0x7f1d580f46a0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1817707731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d64076530 msgr2=0x7f1d6419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6ac00640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d64076530 0x7f1d6419edd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009230 tx=0x7f1d58004040 comp rx=0 tx=0).stop 2024-04-23T14:54:11.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1817707731 shutdown_connections 2024-04-23T14:54:11.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6ac00640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1d64112d40 0x7f1d641a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6ac00640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1d4c0a3d20 0x7f1d4c0a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6ac00640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d64076e50 0x7f1d6419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6ac00640 1 --2- 172.21.15.83:0/1817707731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d64076530 0x7f1d6419edd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.576+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1817707731 >> 172.21.15.83:0/1817707731 conn(0x7f1d640ffb70 msgr2=0x7f1d64101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.577+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1817707731 shutdown_connections 2024-04-23T14:54:11.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.577+0000 7f1d6ac00640 1 -- 172.21.15.83:0/1817707731 wait complete. 2024-04-23T14:54:11.591 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:11 smithi083 ceph-mon[27867]: pgmap v175: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T14:54:11.591 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:11 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2004641004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:54:11.594 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:54:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.795+0000 7f221f000640 1 -- 172.21.15.83:0/1819783687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2218107310 msgr2=0x7f221810eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.795+0000 7f221f000640 1 --2- 172.21.15.83:0/1819783687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2218107310 0x7f221810eee0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2210009ca0 tx=0x7f221002a8c0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.796+0000 7f221f000640 1 -- 172.21.15.83:0/1819783687 shutdown_connections 2024-04-23T14:54:11.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.796+0000 7f221f000640 1 --2- 172.21.15.83:0/1819783687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2218107310 0x7f221810eee0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.796+0000 7f221f000640 1 --2- 172.21.15.83:0/1819783687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2218102bf0 0x7f2218106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.796+0000 7f221f000640 1 --2- 172.21.15.83:0/1819783687 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2218102240 0x7f2218102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.796+0000 7f221f000640 1 -- 172.21.15.83:0/1819783687 >> 172.21.15.83:0/1819783687 conn(0x7f2218079450 msgr2=0x7f2218079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.796+0000 7f221f000640 1 -- 172.21.15.83:0/1819783687 shutdown_connections 2024-04-23T14:54:11.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.796+0000 7f221f000640 1 -- 172.21.15.83:0/1819783687 wait complete. 2024-04-23T14:54:11.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.797+0000 7f221f000640 1 Processor -- start 2024-04-23T14:54:11.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.797+0000 7f221f000640 1 -- start start 2024-04-23T14:54:11.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.797+0000 7f221f000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2218102240 0x7f22181a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.797+0000 7f221f000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2218107310 0x7f22181a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.797+0000 7f221f000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f221819a8b0 0x7f221819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.797+0000 7f221f000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2218112400 con 0x7f2218107310 2024-04-23T14:54:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.797+0000 7f221f000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2218112280 con 0x7f2218102240 2024-04-23T14:54:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.797+0000 7f221f000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2218112580 con 0x7f221819a8b0 2024-04-23T14:54:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.798+0000 7f220f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2218107310 0x7f22181a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.798+0000 7f2217e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2218102240 0x7f22181a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.798+0000 7f220f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2218107310 0x7f22181a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47558/0 (socket says 172.21.15.83:47558) 2024-04-23T14:54:11.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.798+0000 7f2217e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2218102240 0x7f22181a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:54992/0 (socket says 172.21.15.83:54992) 2024-04-23T14:54:11.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.798+0000 7f220f400640 1 -- 172.21.15.83:0/3355284275 learned_addr learned my addr 172.21.15.83:0/3355284275 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.798+0000 7f221d000640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f221819a8b0 0x7f221819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.798+0000 7f2217e00640 1 -- 172.21.15.83:0/3355284275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f221819a8b0 msgr2=0x7f221819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.798+0000 7f2217e00640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f221819a8b0 0x7f221819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.798+0000 7f2217e00640 1 -- 172.21.15.83:0/3355284275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2218107310 msgr2=0x7f22181a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.798+0000 7f2217e00640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2218107310 0x7f22181a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.798+0000 7f2217e00640 1 -- 172.21.15.83:0/3355284275 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221819b3d0 con 0x7f2218102240 2024-04-23T14:54:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.799+0000 7f2217e00640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2218102240 0x7f22181a0700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f220800e9f0 tx=0x7f220800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.799+0000 7f2215600640 1 -- 172.21.15.83:0/3355284275 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220800cd80 con 0x7f2218102240 2024-04-23T14:54:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.799+0000 7f2215600640 1 -- 172.21.15.83:0/3355284275 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22080035f0 con 0x7f2218102240 2024-04-23T14:54:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.799+0000 7f2215600640 1 -- 172.21.15.83:0/3355284275 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208010690 con 0x7f2218102240 2024-04-23T14:54:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.799+0000 7f221f000640 1 -- 172.21.15.83:0/3355284275 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221819b6c0 con 0x7f2218102240 2024-04-23T14:54:11.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.799+0000 7f221f000640 1 -- 172.21.15.83:0/3355284275 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22181a7960 con 0x7f2218102240 2024-04-23T14:54:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.802+0000 7f2221e2f740 1 Processor -- start 2024-04-23T14:54:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.802+0000 7f2221e2f740 1 -- start start 2024-04-23T14:54:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.802+0000 7f2215600640 1 -- 172.21.15.83:0/3355284275 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2208003b30 con 0x7f2218102240 2024-04-23T14:54:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f2221e2f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2eca950 0x5579a2ecad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f2221e2f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ecb270 0x5579a2ed5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f2221e2f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2ed5940 0x5579a2ed7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f2221e2f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5579a2e301d0 con 0x5579a2ecb270 2024-04-23T14:54:11.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f2221e2f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5579a2dd6450 con 0x5579a2ed5940 2024-04-23T14:54:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f2221e2f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5579a2e2c190 con 0x5579a2eca950 2024-04-23T14:54:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f221d000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2eca950 0x5579a2ecad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f2217e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ecb270 0x5579a2ed5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f221d000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2eca950 0x5579a2ecad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50826/0 (socket says 172.21.15.83:50826) 2024-04-23T14:54:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f221d000640 1 -- 172.21.15.83:0/2552190328 learned_addr learned my addr 172.21.15.83:0/2552190328 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f220f400640 1 --2- 172.21.15.83:0/2552190328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2ed5940 0x5579a2ed7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f221d000640 1 -- 172.21.15.83:0/2552190328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2ed5940 msgr2=0x5579a2ed7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f221d000640 1 --2- 172.21.15.83:0/2552190328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2ed5940 0x5579a2ed7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f221d000640 1 -- 172.21.15.83:0/2552190328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ecb270 msgr2=0x5579a2ed5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f221d000640 1 --2- 172.21.15.83:0/2552190328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ecb270 0x5579a2ed5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.803+0000 7f221d000640 1 -- 172.21.15.83:0/2552190328 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a2e93310 con 0x5579a2eca950 2024-04-23T14:54:11.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.804+0000 7f221d000640 1 --2- 172.21.15.83:0/2552190328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2eca950 0x5579a2ecad30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2210009ca0 tx=0x7f22100093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cfdc6459dd33cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.804+0000 7f220d600640 1 -- 172.21.15.83:0/2552190328 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210031db0 con 0x5579a2eca950 2024-04-23T14:54:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.804+0000 7f2215600640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f21fc0a3da0 0x7f21fc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.804+0000 7f220d600640 1 -- 172.21.15.83:0/2552190328 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2210036030 con 0x5579a2eca950 2024-04-23T14:54:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.804+0000 7f220d600640 1 -- 172.21.15.83:0/2552190328 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22100042a0 con 0x5579a2eca950 2024-04-23T14:54:11.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.804+0000 7f2217e00640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f21fc0a3da0 0x7f21fc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.804+0000 7f2215600640 1 -- 172.21.15.83:0/3355284275 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f22080d0aa0 con 0x7f2218102240 2024-04-23T14:54:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.805+0000 7f2221e2f740 1 -- 172.21.15.83:0/2552190328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2eca950 msgr2=0x5579a2ecad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.805+0000 7f2221e2f740 1 --2- 172.21.15.83:0/2552190328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2eca950 0x5579a2ecad30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2210009ca0 tx=0x7f22100093e0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.805+0000 7f2217e00640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f21fc0a3da0 0x7f21fc0a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5579a2ecbf10 tx=0x7f22080d44a0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.805+0000 7f2221e2f740 1 -- 172.21.15.83:0/2552190328 shutdown_connections 2024-04-23T14:54:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.805+0000 7f2221e2f740 1 --2- 172.21.15.83:0/2552190328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2ed5940 0x5579a2ed7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.805+0000 7f2221e2f740 1 --2- 172.21.15.83:0/2552190328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ecb270 0x5579a2ed5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.805+0000 7f2221e2f740 1 --2- 172.21.15.83:0/2552190328 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2eca950 0x5579a2ecad30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.805+0000 7f2221e2f740 1 -- 172.21.15.83:0/2552190328 >> 172.21.15.83:0/2552190328 conn(0x5579a2ec6c70 msgr2=0x5579a2ec7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.805+0000 7f2221e2f740 1 -- 172.21.15.83:0/2552190328 shutdown_connections 2024-04-23T14:54:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.805+0000 7f2221e2f740 1 -- 172.21.15.83:0/2552190328 wait complete. 2024-04-23T14:54:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.806+0000 7f2221e2f740 1 -- start start 2024-04-23T14:54:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.806+0000 7f2221e2f740 1 Processor -- start 2024-04-23T14:54:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.806+0000 7f2221e2f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2eca950 0x5579a2ed5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.806+0000 7f2221e2f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ecb270 0x5579a2ed2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.806+0000 7f2221e2f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ed5940 0x5579a2ed2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.806+0000 7f2221e2f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5579a2ed85f0 con 0x5579a2ecb270 2024-04-23T14:54:11.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.806+0000 7f2221e2f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5579a2de9810 con 0x5579a2eca950 2024-04-23T14:54:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.806+0000 7f2221e2f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5579a2ed8770 con 0x5579a2ed5940 2024-04-23T14:54:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.807+0000 7f2221e2f740 10 client.0 _ll_register_callbacks cb 0x5579a2db7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.807+0000 7f220f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ecb270 0x5579a2ed2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.807+0000 7f220f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ecb270 0x5579a2ed2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47584/0 (socket says 172.21.15.83:47584) 2024-04-23T14:54:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.807+0000 7f221d000640 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ed5940 0x5579a2ed2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.807+0000 7f220f400640 1 -- 172.21.15.83:0/3774004307 learned_addr learned my addr 172.21.15.83:0/3774004307 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.807+0000 7f221d000640 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2eca950 0x5579a2ed5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.807+0000 7f220f400640 1 -- 172.21.15.83:0/3774004307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ed5940 msgr2=0x5579a2ed2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.807+0000 7f220f400640 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ed5940 0x5579a2ed2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.807+0000 7f220f400640 1 -- 172.21.15.83:0/3774004307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2eca950 msgr2=0x5579a2ed5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.807+0000 7f220f400640 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2eca950 0x5579a2ed5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.807+0000 7f220f400640 1 -- 172.21.15.83:0/3774004307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a2dfa3f0 con 0x5579a2ecb270 2024-04-23T14:54:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.807+0000 7f221d000640 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ed5940 0x5579a2ed2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.808+0000 7f220f400640 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ecb270 0x5579a2ed2620 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2200012700 tx=0x7f2200012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53de249feaf5d2e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.808+0000 7f2206000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:54:11.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.808+0000 7f2206000640 1 -- 172.21.15.83:0/3774004307 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220000f920 con 0x5579a2ecb270 2024-04-23T14:54:11.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.808+0000 7f2221e2f740 1 -- 172.21.15.83:0/3774004307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2210036030 con 0x5579a2ecb270 2024-04-23T14:54:11.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.808+0000 7f2206000640 1 -- 172.21.15.83:0/3774004307 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f220000fac0 con 0x5579a2ecb270 2024-04-23T14:54:11.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.808+0000 7f2206000640 1 -- 172.21.15.83:0/3774004307 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f220000fcc0 con 0x5579a2ecb270 2024-04-23T14:54:11.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.808+0000 7f2221e2f740 10 client.16149 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:11.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.808+0000 7f2221e2f740 1 -- 172.21.15.83:0/3774004307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5579a2b423f0 con 0x5579a2ecb270 2024-04-23T14:54:11.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.809+0000 7f2206000640 1 -- 172.21.15.83:0/3774004307 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2210036030 con 0x5579a2ecb270 2024-04-23T14:54:11.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.809+0000 7f2206000640 1 -- 172.21.15.83:0/3774004307 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f2200004b50 con 0x5579a2ecb270 2024-04-23T14:54:11.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.809+0000 7f2221e2f740 10 client.16149 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:11.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.809+0000 7f2221e2f740 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:11.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.809+0000 7f2221e2f740 20 client.16149 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:11.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.809+0000 7f2221e2f740 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5579a2ee5960 0x5579a2ee7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.809+0000 7f2221e2f740 4 client.16149 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:54:11.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.809+0000 7f2221e2f740 1 -- 172.21.15.83:0/3774004307 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579a2e3dbf0 con 0x5579a2ee5960 2024-04-23T14:54:11.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.809+0000 7f221d000640 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5579a2ee5960 0x5579a2ee7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.810+0000 7f221d000640 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5579a2ee5960 0x5579a2ee7d70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae3ab27466613bf server_cookie=771b0251c000641f in_seq=0 out_seq=0 2024-04-23T14:54:11.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.810+0000 7f2206000640 10 client.16149 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:11.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2206000640 1 -- 172.21.15.83:0/3774004307 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579a2e3dbf0 con 0x5579a2ee5960 2024-04-23T14:54:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2206000640 10 client.16149 handle_command_reply: tid=0 2024-04-23T14:54:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2221e2f740 1 client.16149 shutdown 2024-04-23T14:54:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2221e2f740 1 -- 172.21.15.83:0/3774004307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ecb270 msgr2=0x5579a2ed2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2221e2f740 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ecb270 0x5579a2ed2620 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2200012700 tx=0x7f2200012bc0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2221e2f740 1 -- 172.21.15.83:0/3774004307 shutdown_connections 2024-04-23T14:54:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2221e2f740 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5579a2ee5960 0x5579a2ee7d70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2221e2f740 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ed5940 0x5579a2ed2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2221e2f740 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ecb270 0x5579a2ed2620 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2206000640 0 client.16149 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2221e2f740 1 --2- 172.21.15.83:0/3774004307 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2eca950 0x5579a2ed5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2221e2f740 1 -- 172.21.15.83:0/3774004307 >> 172.21.15.83:0/3774004307 conn(0x5579a2ec6c70 msgr2=0x5579a2ecc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2221e2f740 1 -- 172.21.15.83:0/3774004307 shutdown_connections 2024-04-23T14:54:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2221e2f740 1 -- 172.21.15.83:0/3774004307 wait complete. 2024-04-23T14:54:11.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.811+0000 7f2221e2f740 20 client.16149 trim_cache size 0 max 16384 2024-04-23T14:54:11.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:11 smithi173 ceph-mon[32443]: pgmap v175: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T14:54:11.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:11 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2004641004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:54:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.826+0000 7f2221e2f740 1 Processor -- start 2024-04-23T14:54:11.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.826+0000 7f2221e2f740 1 -- start start 2024-04-23T14:54:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.826+0000 7f2221e2f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2eca950 0x5579a2ee14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.826+0000 7f2221e2f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 0x5579a2ed4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.826+0000 7f2221e2f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ed5940 0x5579a2eed640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.826+0000 7f2221e2f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22181b2800 con 0x5579a2ed5940 2024-04-23T14:54:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.826+0000 7f2221e2f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f22181b00d0 con 0x5579a2eca950 2024-04-23T14:54:11.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.827+0000 7f2221e2f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f22181adf20 con 0x5579a2ecb270 2024-04-23T14:54:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.827+0000 7f221d000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2eca950 0x5579a2ee14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.827+0000 7f221d000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2eca950 0x5579a2ee14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:55018/0 (socket says 172.21.15.83:55018) 2024-04-23T14:54:11.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.827+0000 7f221d000640 1 -- 172.21.15.83:0/2767922240 learned_addr learned my addr 172.21.15.83:0/2767922240 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.827+0000 7f221d000640 1 --2- 172.21.15.83:0/2767922240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ed5940 0x5579a2eed640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.827+0000 7f220f400640 1 --2- 172.21.15.83:0/2767922240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 0x5579a2ed4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.827+0000 7f221d000640 1 -- 172.21.15.83:0/2767922240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 msgr2=0x5579a2ed4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.827+0000 7f221d000640 1 --2- 172.21.15.83:0/2767922240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 0x5579a2ed4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.827+0000 7f221d000640 1 -- 172.21.15.83:0/2767922240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ed5940 msgr2=0x5579a2eed640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.827+0000 7f221d000640 1 --2- 172.21.15.83:0/2767922240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ed5940 0x5579a2eed640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.827+0000 7f221d000640 1 -- 172.21.15.83:0/2767922240 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a2c0c7d0 con 0x5579a2eca950 2024-04-23T14:54:11.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.827+0000 7f220f400640 1 --2- 172.21.15.83:0/2767922240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 0x5579a2ed4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.828+0000 7f221d000640 1 --2- 172.21.15.83:0/2767922240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2eca950 0x5579a2ee14e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22100096c0 tx=0x7f2210005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeba779e6d207654 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.828+0000 7f220cc00640 1 -- 172.21.15.83:0/2767922240 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210031810 con 0x5579a2eca950 2024-04-23T14:54:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.828+0000 7f220cc00640 1 -- 172.21.15.83:0/2767922240 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2210036040 con 0x5579a2eca950 2024-04-23T14:54:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.828+0000 7f220cc00640 1 -- 172.21.15.83:0/2767922240 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210004650 con 0x5579a2eca950 2024-04-23T14:54:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.828+0000 7f2221e2f740 1 -- 172.21.15.83:0/2767922240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2eca950 msgr2=0x5579a2ee14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.829+0000 7f2221e2f740 1 --2- 172.21.15.83:0/2767922240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2eca950 0x5579a2ee14e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f22100096c0 tx=0x7f2210005df0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.829+0000 7f2221e2f740 1 -- 172.21.15.83:0/2767922240 shutdown_connections 2024-04-23T14:54:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.829+0000 7f2221e2f740 1 --2- 172.21.15.83:0/2767922240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2ed5940 0x5579a2eed640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.829+0000 7f2221e2f740 1 --2- 172.21.15.83:0/2767922240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 0x5579a2ed4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.829+0000 7f2221e2f740 1 --2- 172.21.15.83:0/2767922240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2eca950 0x5579a2ee14e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.829+0000 7f2221e2f740 1 -- 172.21.15.83:0/2767922240 >> 172.21.15.83:0/2767922240 conn(0x5579a2ec6c70 msgr2=0x5579a2ecd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.829+0000 7f2221e2f740 1 -- 172.21.15.83:0/2767922240 shutdown_connections 2024-04-23T14:54:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.829+0000 7f2221e2f740 1 -- 172.21.15.83:0/2767922240 wait complete. 2024-04-23T14:54:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.829+0000 7f2221e2f740 1 -- start start 2024-04-23T14:54:11.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.829+0000 7f2221e2f740 1 Processor -- start 2024-04-23T14:54:11.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f2221e2f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2eca950 0x5579a2eddec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f2221e2f740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 0x5579a2ecf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f2221e2f740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2ed5940 0x5579a2ecffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f2221e2f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22181ae590 con 0x5579a2eca950 2024-04-23T14:54:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f2221e2f740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f22181afa00 con 0x5579a2ed5940 2024-04-23T14:54:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f2221e2f740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f22181aec60 con 0x5579a2ecb270 2024-04-23T14:54:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f2221e2f740 10 client.0 _ll_register_callbacks cb 0x7f22000098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f221d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2eca950 0x5579a2eddec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f220f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 0x5579a2ecf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f220f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 0x5579a2ecf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50856/0 (socket says 172.21.15.83:50856) 2024-04-23T14:54:11.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f221d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2eca950 0x5579a2eddec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47596/0 (socket says 172.21.15.83:47596) 2024-04-23T14:54:11.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f220f400640 1 -- 172.21.15.83:0/3504517772 learned_addr learned my addr 172.21.15.83:0/3504517772 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:11.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f221d000640 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2ed5940 0x5579a2ecffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f220f400640 1 -- 172.21.15.83:0/3504517772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2ed5940 msgr2=0x5579a2ecffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f220f400640 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2ed5940 0x5579a2ecffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f220f400640 1 -- 172.21.15.83:0/3504517772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2eca950 msgr2=0x5579a2eddec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:11.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f220f400640 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2eca950 0x5579a2eddec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f220f400640 1 -- 172.21.15.83:0/3504517772 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a2ed65e0 con 0x5579a2ecb270 2024-04-23T14:54:11.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.830+0000 7f221d000640 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2eca950 0x5579a2eddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:11.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.831+0000 7f220f400640 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 0x5579a2ecf9a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2200012700 tx=0x7f2200009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96e7a851aa1b040a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:11.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.831+0000 7f2206000640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:54:11.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.831+0000 7f2206000640 1 -- 172.21.15.83:0/3504517772 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200029040 con 0x5579a2ecb270 2024-04-23T14:54:11.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.831+0000 7f2206000640 1 -- 172.21.15.83:0/3504517772 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f220000f920 con 0x5579a2ecb270 2024-04-23T14:54:11.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.831+0000 7f2221e2f740 1 -- 172.21.15.83:0/3504517772 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2210036040 con 0x5579a2ecb270 2024-04-23T14:54:11.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.831+0000 7f2206000640 1 -- 172.21.15.83:0/3504517772 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2200005d30 con 0x5579a2ecb270 2024-04-23T14:54:11.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.831+0000 7f2221e2f740 10 client.26332 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:11.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.831+0000 7f2221e2f740 1 -- 172.21.15.83:0/3504517772 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5579a2f4c0c0 con 0x5579a2ecb270 2024-04-23T14:54:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.832+0000 7f2206000640 1 -- 172.21.15.83:0/3504517772 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2210036040 con 0x5579a2ecb270 2024-04-23T14:54:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.832+0000 7f2206000640 1 -- 172.21.15.83:0/3504517772 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f2200004680 con 0x5579a2ecb270 2024-04-23T14:54:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.832+0000 7f2221e2f740 10 client.26332 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.832+0000 7f2221e2f740 10 client.26332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.832+0000 7f2221e2f740 20 client.26332 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.832+0000 7f2221e2f740 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5579a2f4c820 0x5579a2f4ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.832+0000 7f2221e2f740 4 client.26332 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:54:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.832+0000 7f2221e2f740 1 -- 172.21.15.83:0/3504517772 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579a2e37a60 con 0x5579a2f4c820 2024-04-23T14:54:11.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.832+0000 7f221d000640 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5579a2f4c820 0x5579a2f4ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2206000640 10 client.26332 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f221d000640 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5579a2f4c820 0x5579a2f4ec40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf56f6d0ef73b095 server_cookie=fdc6ad360616d8f in_seq=0 out_seq=0 2024-04-23T14:54:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2206000640 1 -- 172.21.15.83:0/3504517772 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579a2e37a60 con 0x5579a2f4c820 2024-04-23T14:54:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2206000640 10 client.26332 handle_command_reply: tid=0 2024-04-23T14:54:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2221e2f740 1 client.26332 shutdown 2024-04-23T14:54:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2221e2f740 1 -- 172.21.15.83:0/3504517772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 msgr2=0x5579a2ecf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2221e2f740 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 0x5579a2ecf9a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2200012700 tx=0x7f2200009790 comp rx=0 tx=0).stop 2024-04-23T14:54:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2221e2f740 1 -- 172.21.15.83:0/3504517772 shutdown_connections 2024-04-23T14:54:11.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2221e2f740 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5579a2f4c820 0x5579a2f4ec40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2221e2f740 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5579a2ed5940 0x5579a2ecffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2221e2f740 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5579a2ecb270 0x5579a2ecf9a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2221e2f740 1 --2- 172.21.15.83:0/3504517772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5579a2eca950 0x5579a2eddec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2206000640 0 client.26332 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.833+0000 7f2221e2f740 1 -- 172.21.15.83:0/3504517772 >> 172.21.15.83:0/3504517772 conn(0x5579a2ec6c70 msgr2=0x5579a2ec5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f2221e2f740 1 -- 172.21.15.83:0/3504517772 shutdown_connections 2024-04-23T14:54:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f2221e2f740 1 -- 172.21.15.83:0/3504517772 wait complete. 2024-04-23T14:54:11.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f2221e2f740 20 client.26332 trim_cache size 0 max 16384 2024-04-23T14:54:11.842 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:54:11.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f221f000640 1 -- 172.21.15.83:0/3355284275 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f21fc0a3da0 msgr2=0x7f21fc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f221f000640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f21fc0a3da0 0x7f21fc0a6260 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5579a2ecbf10 tx=0x7f22080d44a0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f221f000640 1 -- 172.21.15.83:0/3355284275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2218102240 msgr2=0x7f22181a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:11.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f221f000640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2218102240 0x7f22181a0700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f220800e9f0 tx=0x7f220800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f221f000640 1 -- 172.21.15.83:0/3355284275 shutdown_connections 2024-04-23T14:54:11.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f221f000640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f221819a8b0 0x7f221819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f221f000640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f21fc0a3da0 0x7f21fc0a6260 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f221f000640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2218107310 0x7f22181a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f221f000640 1 --2- 172.21.15.83:0/3355284275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2218102240 0x7f22181a0700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:11.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.834+0000 7f221f000640 1 -- 172.21.15.83:0/3355284275 >> 172.21.15.83:0/3355284275 conn(0x7f2218079450 msgr2=0x7f22180ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:11.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.835+0000 7f221f000640 1 -- 172.21.15.83:0/3355284275 shutdown_connections 2024-04-23T14:54:11.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:11.835+0000 7f221f000640 1 -- 172.21.15.83:0/3355284275 wait complete. 2024-04-23T14:54:11.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:11 smithi203 ceph-mon[32800]: pgmap v175: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T14:54:11.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:11 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2004641004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:54:12.853 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:54:12.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:54:12.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:54:13.071 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.071+0000 7fd62be00640 1 -- 172.21.15.83:0/882122271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd624109580 msgr2=0x7fd624111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.071+0000 7fd62be00640 1 --2- 172.21.15.83:0/882122271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd624109580 0x7fd624111150 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009be0 tx=0x7fd61c02a6c0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.071+0000 7fd62be00640 1 -- 172.21.15.83:0/882122271 shutdown_connections 2024-04-23T14:54:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.071+0000 7fd62be00640 1 --2- 172.21.15.83:0/882122271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd624109580 0x7fd624111150 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.071+0000 7fd62be00640 1 --2- 172.21.15.83:0/882122271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd624104e60 0x7fd624108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.071+0000 7fd62be00640 1 --2- 172.21.15.83:0/882122271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6241044b0 0x7fd624104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.071+0000 7fd62be00640 1 -- 172.21.15.83:0/882122271 >> 172.21.15.83:0/882122271 conn(0x7fd6240ffd90 msgr2=0x7fd6241021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.071+0000 7fd62be00640 1 -- 172.21.15.83:0/882122271 shutdown_connections 2024-04-23T14:54:13.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.071+0000 7fd62be00640 1 -- 172.21.15.83:0/882122271 wait complete. 2024-04-23T14:54:13.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.072+0000 7fd62be00640 1 Processor -- start 2024-04-23T14:54:13.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.072+0000 7fd62be00640 1 -- start start 2024-04-23T14:54:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd62be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6241044b0 0x7fd6241a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd62be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd624109580 0x7fd6241a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd62be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd62419a960 0x7fd62419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd62be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd624114640 con 0x7fd6241044b0 2024-04-23T14:54:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd62be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241144c0 con 0x7fd62419a960 2024-04-23T14:54:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd62be00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241147c0 con 0x7fd624109580 2024-04-23T14:54:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd629400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6241044b0 0x7fd6241a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd629400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6241044b0 0x7fd6241a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47608/0 (socket says 172.21.15.83:47608) 2024-04-23T14:54:13.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd629400640 1 -- 172.21.15.83:0/2086280864 learned_addr learned my addr 172.21.15.83:0/2086280864 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd629e00640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd62419a960 0x7fd62419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd628a00640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd624109580 0x7fd6241a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd629400640 1 -- 172.21.15.83:0/2086280864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd624109580 msgr2=0x7fd6241a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd629400640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd624109580 0x7fd6241a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd629400640 1 -- 172.21.15.83:0/2086280864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd62419a960 msgr2=0x7fd62419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd629400640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd62419a960 0x7fd62419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.073+0000 7fd629400640 1 -- 172.21.15.83:0/2086280864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62419b480 con 0x7fd6241044b0 2024-04-23T14:54:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.074+0000 7fd629400640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6241044b0 0x7fd6241a07b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e9c0 tx=0x7fd61400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.074+0000 7fd622000640 1 -- 172.21.15.83:0/2086280864 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61400cde0 con 0x7fd6241044b0 2024-04-23T14:54:13.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.074+0000 7fd622000640 1 -- 172.21.15.83:0/2086280864 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6140035f0 con 0x7fd6241044b0 2024-04-23T14:54:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.074+0000 7fd62be00640 1 -- 172.21.15.83:0/2086280864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62419b770 con 0x7fd6241044b0 2024-04-23T14:54:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.074+0000 7fd622000640 1 -- 172.21.15.83:0/2086280864 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614010650 con 0x7fd6241044b0 2024-04-23T14:54:13.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.074+0000 7fd62be00640 1 -- 172.21.15.83:0/2086280864 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6241a79c0 con 0x7fd6241044b0 2024-04-23T14:54:13.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.077+0000 7fd62ec0a740 1 Processor -- start 2024-04-23T14:54:13.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.077+0000 7fd62ec0a740 1 -- start start 2024-04-23T14:54:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.078+0000 7fd622000640 1 -- 172.21.15.83:0/2086280864 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd6140037a0 con 0x7fd6241044b0 2024-04-23T14:54:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.078+0000 7fd62ec0a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a090 0x55be5768a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.078+0000 7fd62ec0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be5768a9b0 0x55be57694b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.078+0000 7fd62ec0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be57695080 0x55be57697470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.078+0000 7fd62ec0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55be575efe70 con 0x55be5768a9b0 2024-04-23T14:54:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.078+0000 7fd62ec0a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55be57596380 con 0x55be5768a090 2024-04-23T14:54:13.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.078+0000 7fd62ec0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55be575ebe30 con 0x55be57695080 2024-04-23T14:54:13.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.078+0000 7fd629e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a090 0x55be5768a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.078+0000 7fd629400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be5768a9b0 0x55be57694b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.078+0000 7fd629e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a090 0x55be5768a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:55052/0 (socket says 172.21.15.83:55052) 2024-04-23T14:54:13.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.078+0000 7fd629e00640 1 -- 172.21.15.83:0/350383301 learned_addr learned my addr 172.21.15.83:0/350383301 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:13.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.078+0000 7fd628a00640 1 --2- 172.21.15.83:0/350383301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be57695080 0x55be57697470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.079+0000 7fd629e00640 1 -- 172.21.15.83:0/350383301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be57695080 msgr2=0x55be57697470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.079+0000 7fd629e00640 1 --2- 172.21.15.83:0/350383301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be57695080 0x55be57697470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.079+0000 7fd629e00640 1 -- 172.21.15.83:0/350383301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be5768a9b0 msgr2=0x55be57694b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.079+0000 7fd629e00640 1 --2- 172.21.15.83:0/350383301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be5768a9b0 0x55be57694b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.079+0000 7fd629e00640 1 -- 172.21.15.83:0/350383301 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be573ca7e0 con 0x55be5768a090 2024-04-23T14:54:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.079+0000 7fd629e00640 1 --2- 172.21.15.83:0/350383301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a090 0x55be5768a470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009be0 tx=0x7fd61c030a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce3ee42e4cf524fd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.079+0000 7fd622000640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd6080a3d20 0x7fd6080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.079+0000 7fd61a600640 1 -- 172.21.15.83:0/350383301 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c041030 con 0x55be5768a090 2024-04-23T14:54:13.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.079+0000 7fd61a600640 1 -- 172.21.15.83:0/350383301 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c030c60 con 0x55be5768a090 2024-04-23T14:54:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.079+0000 7fd61a600640 1 -- 172.21.15.83:0/350383301 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c036040 con 0x55be5768a090 2024-04-23T14:54:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.079+0000 7fd622000640 1 -- 172.21.15.83:0/2086280864 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd6140d0b50 con 0x7fd6241044b0 2024-04-23T14:54:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.080+0000 7fd629400640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd6080a3d20 0x7fd6080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.080+0000 7fd62ec0a740 1 -- 172.21.15.83:0/350383301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a090 msgr2=0x55be5768a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.080+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/350383301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a090 0x55be5768a470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009be0 tx=0x7fd61c030a30 comp rx=0 tx=0).stop 2024-04-23T14:54:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.080+0000 7fd62ec0a740 1 -- 172.21.15.83:0/350383301 shutdown_connections 2024-04-23T14:54:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.080+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/350383301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be57695080 0x55be57697470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.080+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/350383301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be5768a9b0 0x55be57694b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.080+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/350383301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a090 0x55be5768a470 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.080+0000 7fd62ec0a740 1 -- 172.21.15.83:0/350383301 >> 172.21.15.83:0/350383301 conn(0x55be576863b0 msgr2=0x55be57686790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.080+0000 7fd629400640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd6080a3d20 0x7fd6080a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55be5768b650 tx=0x7fd6140d4600 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.080+0000 7fd62ec0a740 1 -- 172.21.15.83:0/350383301 shutdown_connections 2024-04-23T14:54:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.080+0000 7fd62ec0a740 1 -- 172.21.15.83:0/350383301 wait complete. 2024-04-23T14:54:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.081+0000 7fd62ec0a740 1 -- start start 2024-04-23T14:54:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.081+0000 7fd62ec0a740 1 Processor -- start 2024-04-23T14:54:13.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.081+0000 7fd62ec0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be5768a090 0x55be57691fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.081+0000 7fd62ec0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a9b0 0x55be576924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.081+0000 7fd62ec0a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be57695080 0x55be57692a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.081+0000 7fd62ec0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55be57697e10 con 0x55be5768a090 2024-04-23T14:54:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.081+0000 7fd62ec0a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55be575a9660 con 0x55be57695080 2024-04-23T14:54:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.081+0000 7fd62ec0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55be57697f90 con 0x55be5768a9b0 2024-04-23T14:54:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.082+0000 7fd62ec0a740 10 client.0 _ll_register_callbacks cb 0x55be57597aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.082+0000 7fd628a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a9b0 0x55be576924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.082+0000 7fd629e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be5768a090 0x55be57691fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.082+0000 7fd628a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a9b0 0x55be576924e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50890/0 (socket says 172.21.15.83:50890) 2024-04-23T14:54:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.082+0000 7fd628a00640 1 -- 172.21.15.83:0/1667218937 learned_addr learned my addr 172.21.15.83:0/1667218937 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.082+0000 7fd629e00640 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be57695080 0x55be57692a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.082+0000 7fd628a00640 1 -- 172.21.15.83:0/1667218937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be57695080 msgr2=0x55be57692a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.082+0000 7fd628a00640 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be57695080 0x55be57692a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.082+0000 7fd628a00640 1 -- 172.21.15.83:0/1667218937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be5768a090 msgr2=0x55be57691fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.082+0000 7fd628a00640 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be5768a090 0x55be57691fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.082+0000 7fd628a00640 1 -- 172.21.15.83:0/1667218937 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5742af70 con 0x55be5768a9b0 2024-04-23T14:54:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.082+0000 7fd629e00640 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be5768a090 0x55be57691fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:13.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.083+0000 7fd628a00640 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a9b0 0x55be576924e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd610012700 tx=0x7fd610012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b10f220decd2d8b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.083+0000 7fd60ea00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:54:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.083+0000 7fd60ea00640 1 -- 172.21.15.83:0/1667218937 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61000f8c0 con 0x55be5768a9b0 2024-04-23T14:54:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.083+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1667218937 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd61c030c60 con 0x55be5768a9b0 2024-04-23T14:54:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.083+0000 7fd60ea00640 1 -- 172.21.15.83:0/1667218937 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd61000fa60 con 0x55be5768a9b0 2024-04-23T14:54:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.083+0000 7fd60ea00640 1 -- 172.21.15.83:0/1667218937 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd61000fc60 con 0x55be5768a9b0 2024-04-23T14:54:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.083+0000 7fd62ec0a740 10 client.26344 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.083+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1667218937 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be572c2170 con 0x55be5768a9b0 2024-04-23T14:54:13.090 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.084+0000 7fd60ea00640 1 -- 172.21.15.83:0/1667218937 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c030c60 con 0x55be5768a9b0 2024-04-23T14:54:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.084+0000 7fd60ea00640 1 -- 172.21.15.83:0/1667218937 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fd610010430 con 0x55be5768a9b0 2024-04-23T14:54:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.084+0000 7fd62ec0a740 10 client.26344 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.084+0000 7fd62ec0a740 10 client.26344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.084+0000 7fd62ec0a740 20 client.26344 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.084+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55be576a53d0 0x55be576a7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.084+0000 7fd62ec0a740 4 client.26344 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:54:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.084+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1667218937 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be575fd890 con 0x55be576a53d0 2024-04-23T14:54:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.084+0000 7fd629e00640 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55be576a53d0 0x55be576a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.091 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.085+0000 7fd629e00640 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55be576a53d0 0x55be576a7830 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=938108862aab6b94 server_cookie=f325105740174b56 in_seq=0 out_seq=0 2024-04-23T14:54:13.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.085+0000 7fd60ea00640 10 client.26344 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd60ea00640 1 -- 172.21.15.83:0/1667218937 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be575fd890 con 0x55be576a53d0 2024-04-23T14:54:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd60ea00640 10 client.26344 handle_command_reply: tid=0 2024-04-23T14:54:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd62ec0a740 1 client.26344 shutdown 2024-04-23T14:54:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1667218937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a9b0 msgr2=0x55be576924e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a9b0 0x55be576924e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd610012700 tx=0x7fd610012bc0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1667218937 shutdown_connections 2024-04-23T14:54:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55be576a53d0 0x55be576a7830 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be57695080 0x55be57692a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a9b0 0x55be576924e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd60ea00640 0 client.26344 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/1667218937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be5768a090 0x55be57691fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1667218937 >> 172.21.15.83:0/1667218937 conn(0x55be576863b0 msgr2=0x55be5768c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1667218937 shutdown_connections 2024-04-23T14:54:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1667218937 wait complete. 2024-04-23T14:54:13.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.086+0000 7fd62ec0a740 20 client.26344 trim_cache size 0 max 16384 2024-04-23T14:54:13.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd62ec0a740 1 Processor -- start 2024-04-23T14:54:13.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd62ec0a740 1 -- start start 2024-04-23T14:54:13.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd62ec0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a090 0x55be576a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd62ec0a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a9b0 0x55be576ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd62ec0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be57695080 0x55be576ad640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd62ec0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241b2810 con 0x55be57695080 2024-04-23T14:54:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd62ec0a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241b00e0 con 0x55be5768a9b0 2024-04-23T14:54:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd62ec0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241adf30 con 0x55be5768a090 2024-04-23T14:54:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd629e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be57695080 0x55be576ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd629e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a090 0x55be576a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd628a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a9b0 0x55be576ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd629e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a090 0x55be576a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50904/0 (socket says 172.21.15.83:50904) 2024-04-23T14:54:13.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.106+0000 7fd629e00640 1 -- 172.21.15.83:0/1610370633 learned_addr learned my addr 172.21.15.83:0/1610370633 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:13.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.107+0000 7fd629e00640 1 -- 172.21.15.83:0/1610370633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a9b0 msgr2=0x55be576ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.107+0000 7fd629e00640 1 --2- 172.21.15.83:0/1610370633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a9b0 0x55be576ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.107+0000 7fd629e00640 1 -- 172.21.15.83:0/1610370633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be57695080 msgr2=0x55be576ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.107+0000 7fd629e00640 1 --2- 172.21.15.83:0/1610370633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be57695080 0x55be576ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.107+0000 7fd629e00640 1 -- 172.21.15.83:0/1610370633 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be57681d10 con 0x55be5768a090 2024-04-23T14:54:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.107+0000 7fd629e00640 1 --2- 172.21.15.83:0/1610370633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a090 0x55be576a0d00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd61c000f80 tx=0x7fd61c03f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb0de9a0b095b89a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.107+0000 7fd619c00640 1 -- 172.21.15.83:0/1610370633 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c041040 con 0x55be5768a090 2024-04-23T14:54:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.107+0000 7fd619c00640 1 -- 172.21.15.83:0/1610370633 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c051040 con 0x55be5768a090 2024-04-23T14:54:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.107+0000 7fd619c00640 1 -- 172.21.15.83:0/1610370633 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c036040 con 0x55be5768a090 2024-04-23T14:54:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.108+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1610370633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a090 msgr2=0x55be576a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.108+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/1610370633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a090 0x55be576a0d00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd61c000f80 tx=0x7fd61c03f040 comp rx=0 tx=0).stop 2024-04-23T14:54:13.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.108+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1610370633 shutdown_connections 2024-04-23T14:54:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.108+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/1610370633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be57695080 0x55be576ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.108+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/1610370633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a9b0 0x55be576ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.108+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/1610370633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a090 0x55be576a0d00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.108+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1610370633 >> 172.21.15.83:0/1610370633 conn(0x55be576863b0 msgr2=0x55be57685510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:13.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.108+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1610370633 shutdown_connections 2024-04-23T14:54:13.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.108+0000 7fd62ec0a740 1 -- 172.21.15.83:0/1610370633 wait complete. 2024-04-23T14:54:13.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.108+0000 7fd62ec0a740 1 -- start start 2024-04-23T14:54:13.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.109+0000 7fd62ec0a740 1 Processor -- start 2024-04-23T14:54:13.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.109+0000 7fd62ec0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a090 0x55be5769d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.109+0000 7fd62ec0a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a9b0 0x55be5769a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.109+0000 7fd62ec0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be57695080 0x55be5769a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.109+0000 7fd62ec0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241ae5a0 con 0x55be57695080 2024-04-23T14:54:13.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.109+0000 7fd62ec0a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241afa10 con 0x55be5768a9b0 2024-04-23T14:54:13.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.109+0000 7fd62ec0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd6241aec70 con 0x55be5768a090 2024-04-23T14:54:13.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.109+0000 7fd62ec0a740 10 client.0 _ll_register_callbacks cb 0x7fd610009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.109+0000 7fd629e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be57695080 0x55be5769a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.110+0000 7fd629e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be57695080 0x55be5769a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47638/0 (socket says 172.21.15.83:47638) 2024-04-23T14:54:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.110+0000 7fd629e00640 1 -- 172.21.15.83:0/336065749 learned_addr learned my addr 172.21.15.83:0/336065749 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.110+0000 7fd629e00640 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a090 0x55be5769d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.110+0000 7fd628a00640 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a9b0 0x55be5769a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.110+0000 7fd629e00640 1 -- 172.21.15.83:0/336065749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a090 msgr2=0x55be5769d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.110+0000 7fd629e00640 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a090 0x55be5769d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.110+0000 7fd629e00640 1 -- 172.21.15.83:0/336065749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a9b0 msgr2=0x55be5769a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.110+0000 7fd629e00640 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a9b0 0x55be5769a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.110+0000 7fd629e00640 1 -- 172.21.15.83:0/336065749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be5769ac90 con 0x55be57695080 2024-04-23T14:54:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.110+0000 7fd629e00640 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be57695080 0x55be5769a720 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009b30 tx=0x7fd61c0515c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e70acc5b4fc89f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.110+0000 7fd60ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:54:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.110+0000 7fd60ea00640 1 -- 172.21.15.83:0/336065749 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c041070 con 0x55be57695080 2024-04-23T14:54:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd60ea00640 1 -- 172.21.15.83:0/336065749 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd61c0511f0 con 0x55be57695080 2024-04-23T14:54:13.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd62ec0a740 1 -- 172.21.15.83:0/336065749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd61c051040 con 0x55be57695080 2024-04-23T14:54:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd60ea00640 1 -- 172.21.15.83:0/336065749 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd61c036040 con 0x55be57695080 2024-04-23T14:54:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd62ec0a740 10 client.16173 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd62ec0a740 1 -- 172.21.15.83:0/336065749 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be5770c740 con 0x55be57695080 2024-04-23T14:54:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd60ea00640 1 -- 172.21.15.83:0/336065749 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c051040 con 0x55be57695080 2024-04-23T14:54:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd60ea00640 1 -- 172.21.15.83:0/336065749 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fd61c062030 con 0x55be57695080 2024-04-23T14:54:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd62ec0a740 10 client.16173 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd62ec0a740 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:13.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd62ec0a740 20 client.16173 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55be5770c910 0x55be5770ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd62ec0a740 4 client.16173 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:54:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.111+0000 7fd62ec0a740 1 -- 172.21.15.83:0/336065749 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55be575f7700 con 0x55be5770c910 2024-04-23T14:54:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.112+0000 7fd628a00640 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55be5770c910 0x55be5770ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.112+0000 7fd628a00640 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55be5770c910 0x55be5770ed00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e7318acb2e7107d server_cookie=cb9d0e6939c946b2 in_seq=0 out_seq=0 2024-04-23T14:54:13.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.112+0000 7fd60ea00640 10 client.16173 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.113+0000 7fd60ea00640 1 -- 172.21.15.83:0/336065749 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55be575f7700 con 0x55be5770c910 2024-04-23T14:54:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.113+0000 7fd60ea00640 10 client.16173 handle_command_reply: tid=0 2024-04-23T14:54:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.113+0000 7fd62ec0a740 1 client.16173 shutdown 2024-04-23T14:54:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.114+0000 7fd62ec0a740 1 -- 172.21.15.83:0/336065749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be57695080 msgr2=0x55be5769a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.114+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be57695080 0x55be5769a720 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009b30 tx=0x7fd61c0515c0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.114+0000 7fd62ec0a740 1 -- 172.21.15.83:0/336065749 shutdown_connections 2024-04-23T14:54:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.114+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55be5770c910 0x55be5770ed00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.114+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55be57695080 0x55be5769a720 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.114+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55be5768a9b0 0x55be5769a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.114+0000 7fd62ec0a740 1 --2- 172.21.15.83:0/336065749 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55be5768a090 0x55be5769d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.114+0000 7fd62ec0a740 1 -- 172.21.15.83:0/336065749 >> 172.21.15.83:0/336065749 conn(0x55be576863b0 msgr2=0x55be57691f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:13.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.114+0000 7fd60ea00640 0 client.16173 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:13.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.114+0000 7fd62ec0a740 1 -- 172.21.15.83:0/336065749 shutdown_connections 2024-04-23T14:54:13.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.114+0000 7fd62ec0a740 1 -- 172.21.15.83:0/336065749 wait complete. 2024-04-23T14:54:13.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.114+0000 7fd62ec0a740 20 client.16173 trim_cache size 0 max 16384 2024-04-23T14:54:13.123 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:54:13.123 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:54:13.123 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "b37d02c7-2ff9-4252-9942-1b8b60d06ec1", 2024-04-23T14:54:13.123 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:54:13.123 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:54:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.115+0000 7fd62be00640 1 -- 172.21.15.83:0/2086280864 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd6080a3d20 msgr2=0x7fd6080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.115+0000 7fd62be00640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd6080a3d20 0x7fd6080a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55be5768b650 tx=0x7fd6140d4600 comp rx=0 tx=0).stop 2024-04-23T14:54:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.115+0000 7fd62be00640 1 -- 172.21.15.83:0/2086280864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6241044b0 msgr2=0x7fd6241a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.115+0000 7fd62be00640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6241044b0 0x7fd6241a07b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e9c0 tx=0x7fd61400ee80 comp rx=0 tx=0).stop 2024-04-23T14:54:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.115+0000 7fd62be00640 1 -- 172.21.15.83:0/2086280864 shutdown_connections 2024-04-23T14:54:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.115+0000 7fd62be00640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd62419a960 0x7fd62419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.115+0000 7fd62be00640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd624109580 0x7fd6241a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.115+0000 7fd62be00640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd6080a3d20 0x7fd6080a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.115+0000 7fd62be00640 1 --2- 172.21.15.83:0/2086280864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6241044b0 0x7fd6241a07b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.115+0000 7fd62be00640 1 -- 172.21.15.83:0/2086280864 >> 172.21.15.83:0/2086280864 conn(0x7fd6240ffd90 msgr2=0x7fd6241020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:13.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.115+0000 7fd62be00640 1 -- 172.21.15.83:0/2086280864 shutdown_connections 2024-04-23T14:54:13.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.115+0000 7fd62be00640 1 -- 172.21.15.83:0/2086280864 wait complete. 2024-04-23T14:54:13.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:54:13.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:54:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.340+0000 7eff69400640 1 -- 172.21.15.83:0/3772098506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64112ee0 msgr2=0x7eff641152d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.340+0000 7eff69400640 1 --2- 172.21.15.83:0/3772098506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64112ee0 0x7eff641152d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7eff54009be0 tx=0x7eff5402a6d0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.340+0000 7eff69400640 1 -- 172.21.15.83:0/3772098506 shutdown_connections 2024-04-23T14:54:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.340+0000 7eff69400640 1 --2- 172.21.15.83:0/3772098506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64112ee0 0x7eff641152d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.340+0000 7eff69400640 1 --2- 172.21.15.83:0/3772098506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff64076e50 0x7eff641129a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.340+0000 7eff69400640 1 --2- 172.21.15.83:0/3772098506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff64076530 0x7eff64076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.340+0000 7eff69400640 1 -- 172.21.15.83:0/3772098506 >> 172.21.15.83:0/3772098506 conn(0x7eff640ffd90 msgr2=0x7eff641021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:13.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.341+0000 7eff69400640 1 -- 172.21.15.83:0/3772098506 shutdown_connections 2024-04-23T14:54:13.342 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.341+0000 7eff69400640 1 -- 172.21.15.83:0/3772098506 wait complete. 2024-04-23T14:54:13.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff69400640 1 Processor -- start 2024-04-23T14:54:13.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff69400640 1 -- start start 2024-04-23T14:54:13.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff69400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff64076530 0x7eff6419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.344 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff69400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64076e50 0x7eff6419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff69400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff64112ee0 0x7eff641a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff69400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff64118a40 con 0x7eff64076e50 2024-04-23T14:54:13.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff69400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff641188c0 con 0x7eff64076530 2024-04-23T14:54:13.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff69400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7eff64118bc0 con 0x7eff64112ee0 2024-04-23T14:54:13.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff61e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64076e50 0x7eff6419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff61e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64076e50 0x7eff6419f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47652/0 (socket says 172.21.15.83:47652) 2024-04-23T14:54:13.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff61e00640 1 -- 172.21.15.83:0/3529289828 learned_addr learned my addr 172.21.15.83:0/3529289828 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:13.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff63200640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff64112ee0 0x7eff641a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.342+0000 7eff62800640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff64076530 0x7eff6419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.343+0000 7eff61e00640 1 -- 172.21.15.83:0/3529289828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff64112ee0 msgr2=0x7eff641a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.343+0000 7eff61e00640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff64112ee0 0x7eff641a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.343+0000 7eff61e00640 1 -- 172.21.15.83:0/3529289828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff64076530 msgr2=0x7eff6419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.343+0000 7eff61e00640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff64076530 0x7eff6419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.343+0000 7eff61e00640 1 -- 172.21.15.83:0/3529289828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff641a3f40 con 0x7eff64076e50 2024-04-23T14:54:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.343+0000 7eff61e00640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64076e50 0x7eff6419f4e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00e9f0 tx=0x7eff4c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.343+0000 7eff5b400640 1 -- 172.21.15.83:0/3529289828 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00ccc0 con 0x7eff64076e50 2024-04-23T14:54:13.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.343+0000 7eff5b400640 1 -- 172.21.15.83:0/3529289828 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff4c010910 con 0x7eff64076e50 2024-04-23T14:54:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.343+0000 7eff5b400640 1 -- 172.21.15.83:0/3529289828 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c0188a0 con 0x7eff64076e50 2024-04-23T14:54:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.343+0000 7eff69400640 1 -- 172.21.15.83:0/3529289828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff641a4230 con 0x7eff64076e50 2024-04-23T14:54:13.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.344+0000 7eff69400640 1 -- 172.21.15.83:0/3529289828 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff641abac0 con 0x7eff64076e50 2024-04-23T14:54:13.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.347+0000 7eff6c238740 1 Processor -- start 2024-04-23T14:54:13.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.347+0000 7eff6c238740 1 -- start start 2024-04-23T14:54:13.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.347+0000 7eff5b400640 1 -- 172.21.15.83:0/3529289828 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7eff4c010ac0 con 0x7eff64076e50 2024-04-23T14:54:13.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.347+0000 7eff6c238740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bc1860 0x561551bc1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.347+0000 7eff6c238740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc2180 0x561551bcc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.347+0000 7eff6c238740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bcc850 0x561551bcec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.347+0000 7eff6c238740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561551b271f0 con 0x561551bc2180 2024-04-23T14:54:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.347+0000 7eff6c238740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561551acd350 con 0x561551bc1860 2024-04-23T14:54:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.347+0000 7eff6c238740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561551b231b0 con 0x561551bcc850 2024-04-23T14:54:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.348+0000 7eff62800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc2180 0x561551bcc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.348+0000 7eff62800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc2180 0x561551bcc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47656/0 (socket says 172.21.15.83:47656) 2024-04-23T14:54:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.348+0000 7eff62800640 1 -- 172.21.15.83:0/3479453023 learned_addr learned my addr 172.21.15.83:0/3479453023 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.348+0000 7eff61e00640 1 --2- 172.21.15.83:0/3479453023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bcc850 0x561551bcec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.348+0000 7eff63200640 1 --2- 172.21.15.83:0/3479453023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bc1860 0x561551bc1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.348+0000 7eff62800640 1 -- 172.21.15.83:0/3479453023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bcc850 msgr2=0x561551bcec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.348+0000 7eff62800640 1 --2- 172.21.15.83:0/3479453023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bcc850 0x561551bcec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.348+0000 7eff62800640 1 -- 172.21.15.83:0/3479453023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bc1860 msgr2=0x561551bc1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.348+0000 7eff62800640 1 --2- 172.21.15.83:0/3479453023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bc1860 0x561551bc1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.348+0000 7eff62800640 1 -- 172.21.15.83:0/3479453023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561551b21f50 con 0x561551bc2180 2024-04-23T14:54:13.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.348+0000 7eff63200640 1 --2- 172.21.15.83:0/3479453023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bc1860 0x561551bc1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:13.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.349+0000 7eff62800640 1 --2- 172.21.15.83:0/3479453023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc2180 0x561551bcc310 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009cd0 tx=0x7eff5c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ee973b245e52f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:13.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.349+0000 7eff53400640 1 -- 172.21.15.83:0/3479453023 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c01e070 con 0x561551bc2180 2024-04-23T14:54:13.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.349+0000 7eff53400640 1 -- 172.21.15.83:0/3479453023 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5c019b10 con 0x561551bc2180 2024-04-23T14:54:13.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.349+0000 7eff53400640 1 -- 172.21.15.83:0/3479453023 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00eaf0 con 0x561551bc2180 2024-04-23T14:54:13.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.349+0000 7eff5b400640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7eff480a3da0 0x7eff480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.349+0000 7eff5b400640 1 -- 172.21.15.83:0/3529289828 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7eff4c0d0b80 con 0x7eff64076e50 2024-04-23T14:54:13.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.349+0000 7eff63200640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7eff480a3da0 0x7eff480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.350+0000 7eff6c238740 1 -- 172.21.15.83:0/3479453023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc2180 msgr2=0x561551bcc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.350+0000 7eff6c238740 1 --2- 172.21.15.83:0/3479453023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc2180 0x561551bcc310 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009cd0 tx=0x7eff5c011810 comp rx=0 tx=0).stop 2024-04-23T14:54:13.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.350+0000 7eff6c238740 1 -- 172.21.15.83:0/3479453023 shutdown_connections 2024-04-23T14:54:13.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.350+0000 7eff6c238740 1 --2- 172.21.15.83:0/3479453023 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bcc850 0x561551bcec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.350+0000 7eff6c238740 1 --2- 172.21.15.83:0/3479453023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc2180 0x561551bcc310 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.350+0000 7eff6c238740 1 --2- 172.21.15.83:0/3479453023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bc1860 0x561551bc1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.350+0000 7eff6c238740 1 -- 172.21.15.83:0/3479453023 >> 172.21.15.83:0/3479453023 conn(0x561551bbdb80 msgr2=0x561551bbdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:13.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.350+0000 7eff6c238740 1 -- 172.21.15.83:0/3479453023 shutdown_connections 2024-04-23T14:54:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.350+0000 7eff6c238740 1 -- 172.21.15.83:0/3479453023 wait complete. 2024-04-23T14:54:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.350+0000 7eff63200640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7eff480a3da0 0x7eff480a6260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff5402abf0 tx=0x7eff54005fd0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.350+0000 7eff6c238740 1 -- start start 2024-04-23T14:54:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.351+0000 7eff6c238740 1 Processor -- start 2024-04-23T14:54:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.351+0000 7eff6c238740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 0x561551bc97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.351+0000 7eff6c238740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bc9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.351+0000 7eff6c238740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bcc850 0x561551bca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.351+0000 7eff6c238740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561551bcf640 con 0x561551bc1860 2024-04-23T14:54:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.351+0000 7eff6c238740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561551ae0690 con 0x561551bcc850 2024-04-23T14:54:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.351+0000 7eff6c238740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561551bcf7c0 con 0x561551bc2180 2024-04-23T14:54:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.351+0000 7eff6c238740 10 client.0 _ll_register_callbacks cb 0x561551b1c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.351+0000 7eff62800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 0x561551bc97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.351+0000 7eff62800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 0x561551bc97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47670/0 (socket says 172.21.15.83:47670) 2024-04-23T14:54:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.351+0000 7eff62800640 1 -- 172.21.15.83:0/3405646629 learned_addr learned my addr 172.21.15.83:0/3405646629 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.352+0000 7eff61e00640 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bc9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.352+0000 7eff63200640 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bcc850 0x561551bca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.352+0000 7eff62800640 1 -- 172.21.15.83:0/3405646629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 msgr2=0x561551bc9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.352+0000 7eff62800640 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bc9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.352+0000 7eff62800640 1 -- 172.21.15.83:0/3405646629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bcc850 msgr2=0x561551bca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.352+0000 7eff62800640 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bcc850 0x561551bca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.352+0000 7eff62800640 1 -- 172.21.15.83:0/3405646629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561551adf920 con 0x561551bc1860 2024-04-23T14:54:13.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.352+0000 7eff63200640 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bcc850 0x561551bca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:13.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.352+0000 7eff62800640 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 0x561551bc97f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7eff5c01c040 tx=0x7eff5c0198b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4152423679aa0f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:13.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.352+0000 7eff47e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:54:13.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.352+0000 7eff47e00640 1 -- 172.21.15.83:0/3405646629 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c028070 con 0x561551bc1860 2024-04-23T14:54:13.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff47e00640 1 -- 172.21.15.83:0/3405646629 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7eff5c005040 con 0x561551bc1860 2024-04-23T14:54:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff6c238740 1 -- 172.21.15.83:0/3405646629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5c019b10 con 0x561551bc1860 2024-04-23T14:54:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff47e00640 1 -- 172.21.15.83:0/3405646629 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7eff5c034050 con 0x561551bc1860 2024-04-23T14:54:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff6c238740 10 client.16185 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff6c238740 1 -- 172.21.15.83:0/3405646629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5615519037d0 con 0x561551bc1860 2024-04-23T14:54:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff47e00640 1 -- 172.21.15.83:0/3405646629 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5c019b10 con 0x561551bc1860 2024-04-23T14:54:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff47e00640 1 -- 172.21.15.83:0/3405646629 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7eff5c00eca0 con 0x561551bc1860 2024-04-23T14:54:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff6c238740 10 client.16185 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:13.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff6c238740 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:13.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff6c238740 20 client.16185 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:13.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff6c238740 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561551bdcb10 0x561551bdef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff6c238740 4 client.16185 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:54:13.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.353+0000 7eff6c238740 1 -- 172.21.15.83:0/3405646629 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561551b34c10 con 0x561551bdcb10 2024-04-23T14:54:13.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.354+0000 7eff61e00640 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561551bdcb10 0x561551bdef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.354+0000 7eff61e00640 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561551bdcb10 0x561551bdef00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c953bd3e12693 server_cookie=484f1b8de1f8f1d1 in_seq=0 out_seq=0 2024-04-23T14:54:13.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.354+0000 7eff47e00640 10 client.16185 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:13.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.355+0000 7eff47e00640 1 -- 172.21.15.83:0/3405646629 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561551b34c10 con 0x561551bdcb10 2024-04-23T14:54:13.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.355+0000 7eff47e00640 10 client.16185 handle_command_reply: tid=0 2024-04-23T14:54:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.355+0000 7eff6c238740 1 client.16185 shutdown 2024-04-23T14:54:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.355+0000 7eff6c238740 1 -- 172.21.15.83:0/3405646629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 msgr2=0x561551bc97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.355+0000 7eff6c238740 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 0x561551bc97f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7eff5c01c040 tx=0x7eff5c0198b0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.355+0000 7eff6c238740 1 -- 172.21.15.83:0/3405646629 shutdown_connections 2024-04-23T14:54:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.355+0000 7eff6c238740 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561551bdcb10 0x561551bdef00 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.355+0000 7eff6c238740 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bcc850 0x561551bca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.355+0000 7eff6c238740 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bc9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.355+0000 7eff6c238740 1 --2- 172.21.15.83:0/3405646629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 0x561551bc97f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.355+0000 7eff47e00640 0 client.16185 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:13.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.355+0000 7eff6c238740 1 -- 172.21.15.83:0/3405646629 >> 172.21.15.83:0/3405646629 conn(0x561551bbdb80 msgr2=0x561551bc3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:13.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.356+0000 7eff6c238740 1 -- 172.21.15.83:0/3405646629 shutdown_connections 2024-04-23T14:54:13.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.356+0000 7eff6c238740 1 -- 172.21.15.83:0/3405646629 wait complete. 2024-04-23T14:54:13.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.356+0000 7eff6c238740 20 client.16185 trim_cache size 0 max 16384 2024-04-23T14:54:13.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff6c238740 1 Processor -- start 2024-04-23T14:54:13.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff6c238740 1 -- start start 2024-04-23T14:54:13.376 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff6c238740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 0x561551bd8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff6c238740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bc9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff6c238740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bcc850 0x561551be4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff6c238740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff641b6c20 con 0x561551bc1860 2024-04-23T14:54:13.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff6c238740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff641b44f0 con 0x561551bcc850 2024-04-23T14:54:13.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff6c238740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7eff641b2340 con 0x561551bc2180 2024-04-23T14:54:13.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff62800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 0x561551bd8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff62800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 0x561551bd8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47676/0 (socket says 172.21.15.83:47676) 2024-04-23T14:54:13.377 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff62800640 1 -- 172.21.15.83:0/2311369401 learned_addr learned my addr 172.21.15.83:0/2311369401 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:13.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff61e00640 1 --2- 172.21.15.83:0/2311369401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bc9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.375+0000 7eff63200640 1 --2- 172.21.15.83:0/2311369401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bcc850 0x561551be4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.376+0000 7eff62800640 1 -- 172.21.15.83:0/2311369401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 msgr2=0x561551bc9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.376+0000 7eff62800640 1 --2- 172.21.15.83:0/2311369401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bc9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.376+0000 7eff62800640 1 -- 172.21.15.83:0/2311369401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bcc850 msgr2=0x561551be4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.376+0000 7eff62800640 1 --2- 172.21.15.83:0/2311369401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bcc850 0x561551be4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.376+0000 7eff62800640 1 -- 172.21.15.83:0/2311369401 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615518393f0 con 0x561551bc1860 2024-04-23T14:54:13.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.376+0000 7eff63200640 1 --2- 172.21.15.83:0/2311369401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bcc850 0x561551be4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:13.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.376+0000 7eff62800640 1 --2- 172.21.15.83:0/2311369401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 0x561551bd8530 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7eff5c004a50 tx=0x7eff5c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d044e9b6f7879a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:13.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.376+0000 7eff52a00640 1 -- 172.21.15.83:0/2311369401 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c031020 con 0x561551bc1860 2024-04-23T14:54:13.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.376+0000 7eff52a00640 1 -- 172.21.15.83:0/2311369401 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5c004cd0 con 0x561551bc1860 2024-04-23T14:54:13.379 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.376+0000 7eff52a00640 1 -- 172.21.15.83:0/2311369401 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c028070 con 0x561551bc1860 2024-04-23T14:54:13.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.377+0000 7eff6c238740 1 -- 172.21.15.83:0/2311369401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 msgr2=0x561551bd8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.377+0000 7eff6c238740 1 --2- 172.21.15.83:0/2311369401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 0x561551bd8530 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7eff5c004a50 tx=0x7eff5c019740 comp rx=0 tx=0).stop 2024-04-23T14:54:13.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.377+0000 7eff6c238740 1 -- 172.21.15.83:0/2311369401 shutdown_connections 2024-04-23T14:54:13.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.377+0000 7eff6c238740 1 --2- 172.21.15.83:0/2311369401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bcc850 0x561551be4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.377+0000 7eff6c238740 1 --2- 172.21.15.83:0/2311369401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bc9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.377+0000 7eff6c238740 1 --2- 172.21.15.83:0/2311369401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bc1860 0x561551bd8530 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.377+0000 7eff6c238740 1 -- 172.21.15.83:0/2311369401 >> 172.21.15.83:0/2311369401 conn(0x561551bbdb80 msgr2=0x561551bc97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.377+0000 7eff6c238740 1 -- 172.21.15.83:0/2311369401 shutdown_connections 2024-04-23T14:54:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.377+0000 7eff6c238740 1 -- 172.21.15.83:0/2311369401 wait complete. 2024-04-23T14:54:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.377+0000 7eff6c238740 1 -- start start 2024-04-23T14:54:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff6c238740 1 Processor -- start 2024-04-23T14:54:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff6c238740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bc1860 0x561551bd4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff6c238740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bcff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff6c238740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bcc850 0x561551bd0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff6c238740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff641b29b0 con 0x561551bcc850 2024-04-23T14:54:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff6c238740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7eff641b3e20 con 0x561551bc1860 2024-04-23T14:54:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff6c238740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7eff641b3080 con 0x561551bc2180 2024-04-23T14:54:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff6c238740 10 client.0 _ll_register_callbacks cb 0x7eff5c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff63200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bcc850 0x561551bd0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff62800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bc1860 0x561551bd4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff63200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bcc850 0x561551bd0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:47690/0 (socket says 172.21.15.83:47690) 2024-04-23T14:54:13.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff63200640 1 -- 172.21.15.83:0/4009478950 learned_addr learned my addr 172.21.15.83:0/4009478950 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.378+0000 7eff61e00640 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bcff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.379+0000 7eff63200640 1 -- 172.21.15.83:0/4009478950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 msgr2=0x561551bcff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.379+0000 7eff63200640 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bcff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.379+0000 7eff63200640 1 -- 172.21.15.83:0/4009478950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bc1860 msgr2=0x561551bd4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.379+0000 7eff63200640 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bc1860 0x561551bd4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.379+0000 7eff63200640 1 -- 172.21.15.83:0/4009478950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615517f90f0 con 0x561551bcc850 2024-04-23T14:54:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.379+0000 7eff62800640 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bc1860 0x561551bd4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.379+0000 7eff61e00640 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bcff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:13.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.379+0000 7eff63200640 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bcc850 0x561551bd0480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7eff54009be0 tx=0x7eff5402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612afb2294f0eb63 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.379+0000 7eff47e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:54:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.379+0000 7eff47e00640 1 -- 172.21.15.83:0/4009478950 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54036040 con 0x561551bcc850 2024-04-23T14:54:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.379+0000 7eff47e00640 1 -- 172.21.15.83:0/4009478950 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7eff54003b80 con 0x561551bcc850 2024-04-23T14:54:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.379+0000 7eff6c238740 1 -- 172.21.15.83:0/4009478950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5c004cd0 con 0x561551bcc850 2024-04-23T14:54:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.380+0000 7eff47e00640 1 -- 172.21.15.83:0/4009478950 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7eff54003d80 con 0x561551bcc850 2024-04-23T14:54:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.380+0000 7eff6c238740 10 client.16191 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.380+0000 7eff6c238740 1 -- 172.21.15.83:0/4009478950 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561551aec810 con 0x561551bcc850 2024-04-23T14:54:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.380+0000 7eff47e00640 1 -- 172.21.15.83:0/4009478950 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5c004cd0 con 0x561551bcc850 2024-04-23T14:54:13.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.380+0000 7eff47e00640 1 -- 172.21.15.83:0/4009478950 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7eff5405d7e0 con 0x561551bcc850 2024-04-23T14:54:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.380+0000 7eff6c238740 10 client.16191 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.380+0000 7eff6c238740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.380+0000 7eff6c238740 20 client.16191 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.380+0000 7eff6c238740 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561551c430b0 0x561551c454d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.380+0000 7eff6c238740 4 client.16191 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:54:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.380+0000 7eff6c238740 1 -- 172.21.15.83:0/4009478950 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561551b2ea80 con 0x561551c430b0 2024-04-23T14:54:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.381+0000 7eff62800640 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561551c430b0 0x561551c454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:13.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.381+0000 7eff62800640 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561551c430b0 0x561551c454d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce2ed3278fbd491 server_cookie=5c263477afe5f7cf in_seq=0 out_seq=0 2024-04-23T14:54:13.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.381+0000 7eff47e00640 10 client.16191 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:13.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.381+0000 7eff47e00640 1 -- 172.21.15.83:0/4009478950 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561551b2ea80 con 0x561551c430b0 2024-04-23T14:54:13.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.381+0000 7eff47e00640 10 client.16191 handle_command_reply: tid=0 2024-04-23T14:54:13.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.381+0000 7eff6c238740 1 client.16191 shutdown 2024-04-23T14:54:13.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.381+0000 7eff6c238740 1 -- 172.21.15.83:0/4009478950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bcc850 msgr2=0x561551bd0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.381+0000 7eff6c238740 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bcc850 0x561551bd0480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7eff54009be0 tx=0x7eff5402a6d0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff6c238740 1 -- 172.21.15.83:0/4009478950 shutdown_connections 2024-04-23T14:54:13.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff6c238740 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561551c430b0 0x561551c454d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff6c238740 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561551bcc850 0x561551bd0480 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff6c238740 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561551bc2180 0x561551bcff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff6c238740 1 --2- 172.21.15.83:0/4009478950 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561551bc1860 0x561551bd4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff47e00640 0 client.16191 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:13.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff6c238740 1 -- 172.21.15.83:0/4009478950 >> 172.21.15.83:0/4009478950 conn(0x561551bbdb80 msgr2=0x561551bbcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:13.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff6c238740 1 -- 172.21.15.83:0/4009478950 shutdown_connections 2024-04-23T14:54:13.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff6c238740 1 -- 172.21.15.83:0/4009478950 wait complete. 2024-04-23T14:54:13.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff6c238740 20 client.16191 trim_cache size 0 max 16384 2024-04-23T14:54:13.391 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:54:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:54:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:54:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "b37d02c7-2ff9-4252-9942-1b8b60d06ec1": { 2024-04-23T14:54:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:54:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "b37d02c7-2ff9-4252-9942-1b8b60d06ec1", 2024-04-23T14:54:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:54:13.392 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:54:13.392 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:54:13.392 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:54:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff69400640 1 -- 172.21.15.83:0/3529289828 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7eff480a3da0 msgr2=0x7eff480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff69400640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7eff480a3da0 0x7eff480a6260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff5402abf0 tx=0x7eff54005fd0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff69400640 1 -- 172.21.15.83:0/3529289828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64076e50 msgr2=0x7eff6419f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.382+0000 7eff69400640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64076e50 0x7eff6419f4e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00e9f0 tx=0x7eff4c00edf0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.383+0000 7eff69400640 1 -- 172.21.15.83:0/3529289828 shutdown_connections 2024-04-23T14:54:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.383+0000 7eff69400640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff64112ee0 0x7eff641a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.383+0000 7eff69400640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7eff480a3da0 0x7eff480a6260 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.383+0000 7eff69400640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff64076e50 0x7eff6419f4e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.383+0000 7eff69400640 1 --2- 172.21.15.83:0/3529289828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff64076530 0x7eff6419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:13.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.383+0000 7eff69400640 1 -- 172.21.15.83:0/3529289828 >> 172.21.15.83:0/3529289828 conn(0x7eff640ffd90 msgr2=0x7eff64102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:13.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.383+0000 7eff69400640 1 -- 172.21.15.83:0/3529289828 shutdown_connections 2024-04-23T14:54:13.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:13.383+0000 7eff69400640 1 -- 172.21.15.83:0/3529289828 wait complete. 2024-04-23T14:54:13.400 INFO:tasks.cephfs.filesystem:scrub status for tag:b37d02c7-2ff9-4252-9942-1b8b60d06ec1 - {'path': '/', 'tag': 'b37d02c7-2ff9-4252-9942-1b8b60d06ec1', 'options': 'recursive,force'} 2024-04-23T14:54:13.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:13 smithi173 ceph-mon[32443]: pgmap v176: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 22 op/s 2024-04-23T14:54:13.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:13 smithi203 ceph-mon[32800]: pgmap v176: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 22 op/s 2024-04-23T14:54:14.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:13 smithi083 ceph-mon[27867]: pgmap v176: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 29 KiB/s wr, 22 op/s 2024-04-23T14:54:14.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:54:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:54:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:54:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:54:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:54:14.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:54:14.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:54:14.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:14 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:54:14.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:54:14.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:54:14.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:54:14.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:54:14.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:54:14.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:54:14.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:14.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:14 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:15.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:54:15.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:15.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:54:15.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:54:15.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:15.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:54:15.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:15.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:54:15.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:15.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:54:15.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:15.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:54:15.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:15.010 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:14 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:15.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:15 smithi173 ceph-mon[32443]: pgmap v177: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 KiB/s wr, 21 op/s 2024-04-23T14:54:15.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:15 smithi203 ceph-mon[32800]: pgmap v177: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 KiB/s wr, 21 op/s 2024-04-23T14:54:16.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:15 smithi083 ceph-mon[27867]: pgmap v177: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 KiB/s wr, 21 op/s 2024-04-23T14:54:17.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:17 smithi173 ceph-mon[32443]: pgmap v178: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-04-23T14:54:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:17 smithi203 ceph-mon[32800]: pgmap v178: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-04-23T14:54:18.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:17 smithi083 ceph-mon[27867]: pgmap v178: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-04-23T14:54:19.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:19 smithi173 ceph-mon[32443]: pgmap v179: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-04-23T14:54:19.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:19 smithi203 ceph-mon[32800]: pgmap v179: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-04-23T14:54:20.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:19 smithi083 ceph-mon[27867]: pgmap v179: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-04-23T14:54:21.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:21 smithi173 ceph-mon[32443]: pgmap v180: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-04-23T14:54:21.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:21 smithi203 ceph-mon[32800]: pgmap v180: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-04-23T14:54:22.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:21 smithi083 ceph-mon[27867]: pgmap v180: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-04-23T14:54:23.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:23 smithi203 ceph-mon[32800]: pgmap v181: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-04-23T14:54:24.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:23 smithi083 ceph-mon[27867]: pgmap v181: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-04-23T14:54:24.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:23 smithi173 ceph-mon[32443]: pgmap v181: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.4 KiB/s wr, 6 op/s 2024-04-23T14:54:24.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:24 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:54:24.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:24 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:54:24.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:24 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:54:25.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:24 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:54:25.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:24 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:54:25.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:24 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:54:25.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:24 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:54:25.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:24 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:54:25.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:24 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:54:25.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:25 smithi203 ceph-mon[32800]: pgmap v182: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-23T14:54:26.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:25 smithi083 ceph-mon[27867]: pgmap v182: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-23T14:54:26.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:25 smithi173 ceph-mon[32443]: pgmap v182: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-23T14:54:27.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:27 smithi203 ceph-mon[32800]: pgmap v183: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-23T14:54:28.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:27 smithi083 ceph-mon[27867]: pgmap v183: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-23T14:54:28.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:27 smithi173 ceph-mon[32443]: pgmap v183: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-23T14:54:29.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:29 smithi203 ceph-mon[32800]: pgmap v184: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:54:30.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:29 smithi083 ceph-mon[27867]: pgmap v184: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:54:30.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:29 smithi173 ceph-mon[32443]: pgmap v184: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:54:31.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:31 smithi203 ceph-mon[32800]: pgmap v185: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:54:32.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:31 smithi083 ceph-mon[27867]: pgmap v185: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:54:32.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:31 smithi173 ceph-mon[32443]: pgmap v185: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:54:33.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:33 smithi203 ceph-mon[32800]: pgmap v186: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:54:34.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:33 smithi083 ceph-mon[27867]: pgmap v186: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:54:34.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:33 smithi173 ceph-mon[32443]: pgmap v186: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:54:35.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:35 smithi203 ceph-mon[32800]: pgmap v187: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:54:36.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:35 smithi083 ceph-mon[27867]: pgmap v187: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:54:36.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:35 smithi173 ceph-mon[32443]: pgmap v187: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:54:37.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:37 smithi203 ceph-mon[32800]: pgmap v188: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:54:38.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:37 smithi083 ceph-mon[27867]: pgmap v188: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:54:38.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:37 smithi173 ceph-mon[32443]: pgmap v188: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:54:39.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:39 smithi203 ceph-mon[32800]: pgmap v189: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:54:40.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:39 smithi083 ceph-mon[27867]: pgmap v189: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:54:40.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:39 smithi173 ceph-mon[32443]: pgmap v189: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:54:41.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:41 smithi203 ceph-mon[32800]: pgmap v190: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:54:42.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:41 smithi083 ceph-mon[27867]: pgmap v190: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:54:42.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:41 smithi173 ceph-mon[32443]: pgmap v190: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:54:43.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:54:43.401 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:54:43.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.617+0000 7f81a3e00640 1 -- 172.21.15.83:0/14806084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81a4108f40 msgr2=0x7f81a41153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.617+0000 7f81a3e00640 1 --2- 172.21.15.83:0/14806084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81a4108f40 0x7f81a41153b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f819c009cd0 tx=0x7f819c02a8e0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.617+0000 7f81a3e00640 1 -- 172.21.15.83:0/14806084 shutdown_connections 2024-04-23T14:54:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.617+0000 7f81a3e00640 1 --2- 172.21.15.83:0/14806084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81a4108f40 0x7f81a41153b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.617+0000 7f81a3e00640 1 --2- 172.21.15.83:0/14806084 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81a4108580 0x7f81a4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.617+0000 7f81a3e00640 1 --2- 172.21.15.83:0/14806084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a410d500 0x7f81a410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.617+0000 7f81a3e00640 1 -- 172.21.15.83:0/14806084 >> 172.21.15.83:0/14806084 conn(0x7f81a4077e20 msgr2=0x7f81a4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.617+0000 7f81a3e00640 1 -- 172.21.15.83:0/14806084 shutdown_connections 2024-04-23T14:54:43.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.617+0000 7f81a3e00640 1 -- 172.21.15.83:0/14806084 wait complete. 2024-04-23T14:54:43.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.618+0000 7f81a3e00640 1 Processor -- start 2024-04-23T14:54:43.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.618+0000 7f81a3e00640 1 -- start start 2024-04-23T14:54:43.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a4108580 0x7f81a41a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81a4108f40 0x7f81a41a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81a410d500 0x7f81a419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4118920 con 0x7f81a4108f40 2024-04-23T14:54:43.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a3e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41187a0 con 0x7f81a4108580 2024-04-23T14:54:43.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a3e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4118aa0 con 0x7f81a410d500 2024-04-23T14:54:43.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81a4108f40 0x7f81a41a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a4108580 0x7f81a41a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81a4108f40 0x7f81a41a4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58088/0 (socket says 172.21.15.83:58088) 2024-04-23T14:54:43.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a4108580 0x7f81a41a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:59226/0 (socket says 172.21.15.83:59226) 2024-04-23T14:54:43.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a2000640 1 -- 172.21.15.83:0/773214562 learned_addr learned my addr 172.21.15.83:0/773214562 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:43.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a3400640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81a410d500 0x7f81a419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a2000640 1 -- 172.21.15.83:0/773214562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81a410d500 msgr2=0x7f81a419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:43.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a2000640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81a410d500 0x7f81a419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a2000640 1 -- 172.21.15.83:0/773214562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a4108580 msgr2=0x7f81a41a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:43.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a2000640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a4108580 0x7f81a41a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.619+0000 7f81a2000640 1 -- 172.21.15.83:0/773214562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a419f280 con 0x7f81a4108f40 2024-04-23T14:54:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.620+0000 7f81a2000640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81a4108f40 0x7f81a41a4e90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f818c00e9f0 tx=0x7f818c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.620+0000 7f81a8a00640 1 -- 172.21.15.83:0/773214562 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c00cd80 con 0x7f81a4108f40 2024-04-23T14:54:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.620+0000 7f81a8a00640 1 -- 172.21.15.83:0/773214562 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f818c005500 con 0x7f81a4108f40 2024-04-23T14:54:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.620+0000 7f81a3e00640 1 -- 172.21.15.83:0/773214562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a419f570 con 0x7f81a4108f40 2024-04-23T14:54:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.620+0000 7f81a8a00640 1 -- 172.21.15.83:0/773214562 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c010690 con 0x7f81a4108f40 2024-04-23T14:54:43.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.620+0000 7f81a3e00640 1 -- 172.21.15.83:0/773214562 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a41ab810 con 0x7f81a4108f40 2024-04-23T14:54:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.623+0000 7f81acc12740 1 Processor -- start 2024-04-23T14:54:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81acc12740 1 -- start start 2024-04-23T14:54:43.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81a8a00640 1 -- 172.21.15.83:0/773214562 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f818c0046c0 con 0x7f81a4108f40 2024-04-23T14:54:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81acc12740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 0x55aaf9d50e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81acc12740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d51350 0x55aaf9d5b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81acc12740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d5de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81acc12740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9cb61f0 con 0x55aaf9d51350 2024-04-23T14:54:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81acc12740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9c5c350 con 0x55aaf9d5ba20 2024-04-23T14:54:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81acc12740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9cb21b0 con 0x55aaf9d50a30 2024-04-23T14:54:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81a3400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 0x55aaf9d50e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81a2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d51350 0x55aaf9d5b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81a3400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 0x55aaf9d50e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:34440/0 (socket says 172.21.15.83:34440) 2024-04-23T14:54:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81a3400640 1 -- 172.21.15.83:0/1691719265 learned_addr learned my addr 172.21.15.83:0/1691719265 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.624+0000 7f81a2000640 1 --2- 172.21.15.83:0/1691719265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d5de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.625+0000 7f81a3400640 1 -- 172.21.15.83:0/1691719265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d5ba20 msgr2=0x55aaf9d5de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.625+0000 7f81a3400640 1 --2- 172.21.15.83:0/1691719265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d5de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.625+0000 7f81a3400640 1 -- 172.21.15.83:0/1691719265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d51350 msgr2=0x55aaf9d5b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.625+0000 7f81a3400640 1 --2- 172.21.15.83:0/1691719265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d51350 0x55aaf9d5b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.625+0000 7f81a3400640 1 -- 172.21.15.83:0/1691719265 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf9cb0f50 con 0x55aaf9d50a30 2024-04-23T14:54:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.625+0000 7f81a3400640 1 --2- 172.21.15.83:0/1691719265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 0x55aaf9d50e10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f819c009cd0 tx=0x7f819c005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a95883e7bf185250 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:43.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.625+0000 7f8193e00640 1 -- 172.21.15.83:0/1691719265 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c049070 con 0x55aaf9d50a30 2024-04-23T14:54:43.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.625+0000 7f8193e00640 1 -- 172.21.15.83:0/1691719265 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819c036040 con 0x55aaf9d50a30 2024-04-23T14:54:43.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.625+0000 7f8193e00640 1 -- 172.21.15.83:0/1691719265 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c004240 con 0x55aaf9d50a30 2024-04-23T14:54:43.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.626+0000 7f81a8a00640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f81880a3d20 0x7f81880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.626+0000 7f81a8a00640 1 -- 172.21.15.83:0/773214562 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f818c0d0c40 con 0x7f81a4108f40 2024-04-23T14:54:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.626+0000 7f81a2a00640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f81880a3d20 0x7f81880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.626+0000 7f81acc12740 1 -- 172.21.15.83:0/1691719265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 msgr2=0x55aaf9d50e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.626+0000 7f81acc12740 1 --2- 172.21.15.83:0/1691719265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 0x55aaf9d50e10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f819c009cd0 tx=0x7f819c005f80 comp rx=0 tx=0).stop 2024-04-23T14:54:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.626+0000 7f81acc12740 1 -- 172.21.15.83:0/1691719265 shutdown_connections 2024-04-23T14:54:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.626+0000 7f81acc12740 1 --2- 172.21.15.83:0/1691719265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d5de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.626+0000 7f81acc12740 1 --2- 172.21.15.83:0/1691719265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d51350 0x55aaf9d5b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.626+0000 7f81acc12740 1 --2- 172.21.15.83:0/1691719265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 0x55aaf9d50e10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.626+0000 7f81acc12740 1 -- 172.21.15.83:0/1691719265 >> 172.21.15.83:0/1691719265 conn(0x55aaf9d4cd50 msgr2=0x55aaf9d4d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.626+0000 7f81acc12740 1 -- 172.21.15.83:0/1691719265 shutdown_connections 2024-04-23T14:54:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.626+0000 7f81acc12740 1 -- 172.21.15.83:0/1691719265 wait complete. 2024-04-23T14:54:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.627+0000 7f81a2a00640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f81880a3d20 0x7f81880a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55aaf9d51ff0 tx=0x7f8194005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.627+0000 7f81acc12740 1 -- start start 2024-04-23T14:54:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.627+0000 7f81acc12740 1 Processor -- start 2024-04-23T14:54:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81acc12740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 0x55aaf9d589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81acc12740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d51350 0x55aaf9d58ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81acc12740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d59420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81acc12740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9d5e7b0 con 0x55aaf9d5ba20 2024-04-23T14:54:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81acc12740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9c6f690 con 0x55aaf9d51350 2024-04-23T14:54:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81acc12740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55aaf9d5e930 con 0x55aaf9d50a30 2024-04-23T14:54:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81acc12740 10 client.0 _ll_register_callbacks cb 0x55aaf9cab650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81a2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 0x55aaf9d589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81a3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d59420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81a2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 0x55aaf9d589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:34446/0 (socket says 172.21.15.83:34446) 2024-04-23T14:54:43.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81a2a00640 1 -- 172.21.15.83:0/1584449033 learned_addr learned my addr 172.21.15.83:0/1584449033 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81a2000640 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d51350 0x55aaf9d58ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81a2a00640 1 -- 172.21.15.83:0/1584449033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d51350 msgr2=0x55aaf9d58ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81a2a00640 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d51350 0x55aaf9d58ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81a2a00640 1 -- 172.21.15.83:0/1584449033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d5ba20 msgr2=0x55aaf9d59420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81a2a00640 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d59420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81a2a00640 1 -- 172.21.15.83:0/1584449033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf9c6e920 con 0x55aaf9d50a30 2024-04-23T14:54:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.628+0000 7f81a3400640 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d59420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.629+0000 7f81a2a00640 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 0x55aaf9d589a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8194004620 tx=0x7f8194004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55bce78c731932a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:43.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.629+0000 7f8190c00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:54:43.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.629+0000 7f8190c00640 1 -- 172.21.15.83:0/1584449033 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819401f630 con 0x55aaf9d50a30 2024-04-23T14:54:43.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.629+0000 7f8190c00640 1 -- 172.21.15.83:0/1584449033 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f819401f7d0 con 0x55aaf9d50a30 2024-04-23T14:54:43.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.629+0000 7f81acc12740 1 -- 172.21.15.83:0/1584449033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f819c036040 con 0x55aaf9d50a30 2024-04-23T14:54:43.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.629+0000 7f8190c00640 1 -- 172.21.15.83:0/1584449033 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f819401f9d0 con 0x55aaf9d50a30 2024-04-23T14:54:43.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.629+0000 7f81acc12740 10 client.26380 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:43.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.629+0000 7f81acc12740 1 -- 172.21.15.83:0/1584449033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aaf9a927d0 con 0x55aaf9d50a30 2024-04-23T14:54:43.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.630+0000 7f8190c00640 1 -- 172.21.15.83:0/1584449033 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f819c036040 con 0x55aaf9d50a30 2024-04-23T14:54:43.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.630+0000 7f8190c00640 1 -- 172.21.15.83:0/1584449033 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f8194027070 con 0x55aaf9d50a30 2024-04-23T14:54:43.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.630+0000 7f81acc12740 10 client.26380 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:43.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.630+0000 7f81acc12740 10 client.26380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:43.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.630+0000 7f81acc12740 20 client.26380 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:43.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.630+0000 7f81acc12740 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55aaf9d6be30 0x55aaf9d6e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.630+0000 7f81acc12740 4 client.26380 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:54:43.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.630+0000 7f81acc12740 1 -- 172.21.15.83:0/1584449033 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaf9cc3c10 con 0x55aaf9d6be30 2024-04-23T14:54:43.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.631+0000 7f81a2000640 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55aaf9d6be30 0x55aaf9d6e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.631+0000 7f81a2000640 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55aaf9d6be30 0x55aaf9d6e2a0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e4316dfb4351e4 server_cookie=d07b55cde0929de3 in_seq=0 out_seq=0 2024-04-23T14:54:43.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.631+0000 7f8190c00640 10 client.26380 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.632+0000 7f8190c00640 1 -- 172.21.15.83:0/1584449033 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aaf9cc3c10 con 0x55aaf9d6be30 2024-04-23T14:54:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.632+0000 7f8190c00640 10 client.26380 handle_command_reply: tid=0 2024-04-23T14:54:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.632+0000 7f81acc12740 1 client.26380 shutdown 2024-04-23T14:54:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.632+0000 7f81acc12740 1 -- 172.21.15.83:0/1584449033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 msgr2=0x55aaf9d589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.632+0000 7f81acc12740 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 0x55aaf9d589a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8194004620 tx=0x7f8194004730 comp rx=0 tx=0).stop 2024-04-23T14:54:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.632+0000 7f81acc12740 1 -- 172.21.15.83:0/1584449033 shutdown_connections 2024-04-23T14:54:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.632+0000 7f81acc12740 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55aaf9d6be30 0x55aaf9d6e2a0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.632+0000 7f81acc12740 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d59420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.632+0000 7f81acc12740 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d51350 0x55aaf9d58ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.632+0000 7f8190c00640 0 client.26380 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.632+0000 7f81acc12740 1 --2- 172.21.15.83:0/1584449033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d50a30 0x55aaf9d589a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.632+0000 7f81acc12740 1 -- 172.21.15.83:0/1584449033 >> 172.21.15.83:0/1584449033 conn(0x55aaf9d4cd50 msgr2=0x55aaf9d4bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.633+0000 7f81acc12740 1 -- 172.21.15.83:0/1584449033 shutdown_connections 2024-04-23T14:54:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.633+0000 7f81acc12740 1 -- 172.21.15.83:0/1584449033 wait complete. 2024-04-23T14:54:43.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.633+0000 7f81acc12740 20 client.26380 trim_cache size 0 max 16384 2024-04-23T14:54:43.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.652+0000 7f81acc12740 1 Processor -- start 2024-04-23T14:54:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.652+0000 7f81acc12740 1 -- start start 2024-04-23T14:54:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.653+0000 7f81acc12740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d50a30 0x55aaf9d676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.653+0000 7f81acc12740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d51350 0x55aaf9d57b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.653+0000 7f81acc12740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d59070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.653+0000 7f81acc12740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41b6910 con 0x55aaf9d5ba20 2024-04-23T14:54:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.653+0000 7f81acc12740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41b41e0 con 0x55aaf9d50a30 2024-04-23T14:54:43.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.653+0000 7f81acc12740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41b2030 con 0x55aaf9d51350 2024-04-23T14:54:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.653+0000 7f81a2000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d51350 0x55aaf9d57b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.653+0000 7f81a3400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d59070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.653+0000 7f81a2000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d51350 0x55aaf9d57b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:34454/0 (socket says 172.21.15.83:34454) 2024-04-23T14:54:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.653+0000 7f81a2000640 1 -- 172.21.15.83:0/1224845629 learned_addr learned my addr 172.21.15.83:0/1224845629 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.653+0000 7f81a2a00640 1 --2- 172.21.15.83:0/1224845629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d50a30 0x55aaf9d676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.654+0000 7f81a2000640 1 -- 172.21.15.83:0/1224845629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d50a30 msgr2=0x55aaf9d676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.654+0000 7f81a2000640 1 --2- 172.21.15.83:0/1224845629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d50a30 0x55aaf9d676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.654+0000 7f81a2000640 1 -- 172.21.15.83:0/1224845629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d5ba20 msgr2=0x55aaf9d59070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.654+0000 7f81a2000640 1 --2- 172.21.15.83:0/1224845629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d59070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.654+0000 7f81a2000640 1 -- 172.21.15.83:0/1224845629 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf99c83f0 con 0x55aaf9d51350 2024-04-23T14:54:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.654+0000 7f81a2000640 1 --2- 172.21.15.83:0/1224845629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d51350 0x55aaf9d57b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f818c098ff0 tx=0x7f818c0bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=642b7ec436729fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.654+0000 7f8193400640 1 -- 172.21.15.83:0/1224845629 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c0b6070 con 0x55aaf9d51350 2024-04-23T14:54:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.654+0000 7f8193400640 1 -- 172.21.15.83:0/1224845629 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f818c09d290 con 0x55aaf9d51350 2024-04-23T14:54:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.654+0000 7f8193400640 1 -- 172.21.15.83:0/1224845629 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c099200 con 0x55aaf9d51350 2024-04-23T14:54:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.655+0000 7f81acc12740 1 -- 172.21.15.83:0/1224845629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d51350 msgr2=0x55aaf9d57b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.655+0000 7f81acc12740 1 --2- 172.21.15.83:0/1224845629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d51350 0x55aaf9d57b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f818c098ff0 tx=0x7f818c0bd040 comp rx=0 tx=0).stop 2024-04-23T14:54:43.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.655+0000 7f81acc12740 1 -- 172.21.15.83:0/1224845629 shutdown_connections 2024-04-23T14:54:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.655+0000 7f81acc12740 1 --2- 172.21.15.83:0/1224845629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d59070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.655+0000 7f81acc12740 1 --2- 172.21.15.83:0/1224845629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d51350 0x55aaf9d57b00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.655+0000 7f81acc12740 1 --2- 172.21.15.83:0/1224845629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d50a30 0x55aaf9d676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.655+0000 7f81acc12740 1 -- 172.21.15.83:0/1224845629 >> 172.21.15.83:0/1224845629 conn(0x55aaf9d4cd50 msgr2=0x55aaf9d58960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.655+0000 7f81acc12740 1 -- 172.21.15.83:0/1224845629 shutdown_connections 2024-04-23T14:54:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.655+0000 7f81acc12740 1 -- 172.21.15.83:0/1224845629 wait complete. 2024-04-23T14:54:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.656+0000 7f81acc12740 1 -- start start 2024-04-23T14:54:43.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.656+0000 7f81acc12740 1 Processor -- start 2024-04-23T14:54:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.656+0000 7f81acc12740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d50a30 0x55aaf9d63c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.656+0000 7f81acc12740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d51350 0x55aaf9d64160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.656+0000 7f81acc12740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d5f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.656+0000 7f81acc12740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41b26a0 con 0x55aaf9d50a30 2024-04-23T14:54:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.656+0000 7f81acc12740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41b3b10 con 0x55aaf9d51350 2024-04-23T14:54:43.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.656+0000 7f81acc12740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41b2d70 con 0x55aaf9d5ba20 2024-04-23T14:54:43.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81acc12740 10 client.0 _ll_register_callbacks cb 0x7f8194009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:43.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81a2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d50a30 0x55aaf9d63c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81a3400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d5f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81a3400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d5f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:34468/0 (socket says 172.21.15.83:34468) 2024-04-23T14:54:43.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81a2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d50a30 0x55aaf9d63c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58126/0 (socket says 172.21.15.83:58126) 2024-04-23T14:54:43.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81a3400640 1 -- 172.21.15.83:0/1301570956 learned_addr learned my addr 172.21.15.83:0/1301570956 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:43.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81a2000640 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d51350 0x55aaf9d64160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81a3400640 1 -- 172.21.15.83:0/1301570956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d51350 msgr2=0x55aaf9d64160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:43.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81a3400640 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d51350 0x55aaf9d64160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81a3400640 1 -- 172.21.15.83:0/1301570956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d50a30 msgr2=0x55aaf9d63c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81a3400640 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d50a30 0x55aaf9d63c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81a3400640 1 -- 172.21.15.83:0/1301570956 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf99880f0 con 0x55aaf9d5ba20 2024-04-23T14:54:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.657+0000 7f81a2a00640 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d50a30 0x55aaf9d63c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.658+0000 7f81a3400640 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d5f380 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f819c004ab0 tx=0x7f819c003c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5433b61b4d24136b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.658+0000 7f8190c00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:54:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.658+0000 7f8190c00640 1 -- 172.21.15.83:0/1301570956 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c054070 con 0x55aaf9d5ba20 2024-04-23T14:54:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.658+0000 7f8190c00640 1 -- 172.21.15.83:0/1301570956 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f819c0315b0 con 0x55aaf9d5ba20 2024-04-23T14:54:43.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.658+0000 7f81acc12740 1 -- 172.21.15.83:0/1301570956 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f818c09d290 con 0x55aaf9d5ba20 2024-04-23T14:54:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.658+0000 7f8190c00640 1 -- 172.21.15.83:0/1301570956 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f819c060050 con 0x55aaf9d5ba20 2024-04-23T14:54:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.658+0000 7f81acc12740 10 client.26386 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.658+0000 7f81acc12740 1 -- 172.21.15.83:0/1301570956 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aaf9c7b810 con 0x55aaf9d5ba20 2024-04-23T14:54:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.658+0000 7f8190c00640 1 -- 172.21.15.83:0/1301570956 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f818c09d290 con 0x55aaf9d5ba20 2024-04-23T14:54:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.659+0000 7f8190c00640 1 -- 172.21.15.83:0/1301570956 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f819c030070 con 0x55aaf9d5ba20 2024-04-23T14:54:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.659+0000 7f81acc12740 10 client.26386 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.659+0000 7f81acc12740 10 client.26386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:43.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.659+0000 7f81acc12740 20 client.26386 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.659+0000 7f81acc12740 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55aaf9dd2060 0x55aaf9dd4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.659+0000 7f81acc12740 4 client.26386 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:54:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.659+0000 7f81acc12740 1 -- 172.21.15.83:0/1301570956 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aaf9cbda80 con 0x55aaf9dd2060 2024-04-23T14:54:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.659+0000 7f81a2a00640 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55aaf9dd2060 0x55aaf9dd4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81a2a00640 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55aaf9dd2060 0x55aaf9dd4450 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c47513a132e6698b server_cookie=7170e9d6924e181c in_seq=0 out_seq=0 2024-04-23T14:54:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f8190c00640 10 client.26386 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f8190c00640 1 -- 172.21.15.83:0/1301570956 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aaf9cbda80 con 0x55aaf9dd2060 2024-04-23T14:54:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f8190c00640 10 client.26386 handle_command_reply: tid=0 2024-04-23T14:54:43.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81acc12740 1 client.26386 shutdown 2024-04-23T14:54:43.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81acc12740 1 -- 172.21.15.83:0/1301570956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d5ba20 msgr2=0x55aaf9d5f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:43.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81acc12740 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d5f380 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f819c004ab0 tx=0x7f819c003c40 comp rx=0 tx=0).stop 2024-04-23T14:54:43.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81acc12740 1 -- 172.21.15.83:0/1301570956 shutdown_connections 2024-04-23T14:54:43.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81acc12740 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55aaf9dd2060 0x55aaf9dd4450 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81acc12740 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aaf9d5ba20 0x55aaf9d5f380 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81acc12740 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55aaf9d51350 0x55aaf9d64160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81acc12740 1 --2- 172.21.15.83:0/1301570956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aaf9d50a30 0x55aaf9d63c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f8190c00640 0 client.26386 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:43.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81acc12740 1 -- 172.21.15.83:0/1301570956 >> 172.21.15.83:0/1301570956 conn(0x55aaf9d4cd50 msgr2=0x55aaf9d58940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:43.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81acc12740 1 -- 172.21.15.83:0/1301570956 shutdown_connections 2024-04-23T14:54:43.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81acc12740 1 -- 172.21.15.83:0/1301570956 wait complete. 2024-04-23T14:54:43.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.660+0000 7f81acc12740 20 client.26386 trim_cache size 0 max 16384 2024-04-23T14:54:43.670 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:54:43.670 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:54:43.670 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:54:43.671 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:54:43.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.661+0000 7f81a3e00640 1 -- 172.21.15.83:0/773214562 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f81880a3d20 msgr2=0x7f81880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:43.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.661+0000 7f81a3e00640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f81880a3d20 0x7f81880a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55aaf9d51ff0 tx=0x7f8194005f90 comp rx=0 tx=0).stop 2024-04-23T14:54:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.661+0000 7f81a3e00640 1 -- 172.21.15.83:0/773214562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81a4108f40 msgr2=0x7f81a41a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.661+0000 7f81a3e00640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81a4108f40 0x7f81a41a4e90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f818c00e9f0 tx=0x7f818c00edf0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.661+0000 7f81a3e00640 1 -- 172.21.15.83:0/773214562 shutdown_connections 2024-04-23T14:54:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.661+0000 7f81a3e00640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81a410d500 0x7f81a419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.661+0000 7f81a3e00640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f81880a3d20 0x7f81880a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.661+0000 7f81a3e00640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81a4108f40 0x7f81a41a4e90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.661+0000 7f81a3e00640 1 --2- 172.21.15.83:0/773214562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81a4108580 0x7f81a41a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.661+0000 7f81a3e00640 1 -- 172.21.15.83:0/773214562 >> 172.21.15.83:0/773214562 conn(0x7f81a4077e20 msgr2=0x7f81a4078260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:43.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.661+0000 7f81a3e00640 1 -- 172.21.15.83:0/773214562 shutdown_connections 2024-04-23T14:54:43.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.661+0000 7f81a3e00640 1 -- 172.21.15.83:0/773214562 wait complete. 2024-04-23T14:54:43.680 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:54:43.680 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:54:43.747 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:43 smithi083 ceph-mon[27867]: pgmap v191: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:54:43.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.879+0000 7f710a400640 1 -- 172.21.15.83:0/765389007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71041084a0 msgr2=0x7f7104108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:43.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.879+0000 7f710a400640 1 --2- 172.21.15.83:0/765389007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71041084a0 0x7f7104108900 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009860 tx=0x7f70f002a3d0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.879+0000 7f710a400640 1 -- 172.21.15.83:0/765389007 shutdown_connections 2024-04-23T14:54:43.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.879+0000 7f710a400640 1 --2- 172.21.15.83:0/765389007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7104108e40 0x7f71041152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.879+0000 7f710a400640 1 --2- 172.21.15.83:0/765389007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71041084a0 0x7f7104108900 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.879+0000 7f710a400640 1 --2- 172.21.15.83:0/765389007 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f710410d3c0 0x7f710410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.879+0000 7f710a400640 1 -- 172.21.15.83:0/765389007 >> 172.21.15.83:0/765389007 conn(0x7f7104077b80 msgr2=0x7f7104079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:43.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.879+0000 7f710a400640 1 -- 172.21.15.83:0/765389007 shutdown_connections 2024-04-23T14:54:43.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.880+0000 7f710a400640 1 -- 172.21.15.83:0/765389007 wait complete. 2024-04-23T14:54:43.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.880+0000 7f710a400640 1 Processor -- start 2024-04-23T14:54:43.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.881+0000 7f710a400640 1 -- start start 2024-04-23T14:54:43.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.881+0000 7f710a400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71041084a0 0x7f71041a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.881+0000 7f710a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7104108e40 0x7f71041a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.881+0000 7f710a400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f710410d3c0 0x7f710419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.881+0000 7f710a400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71041187e0 con 0x7f7104108e40 2024-04-23T14:54:43.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.881+0000 7f710a400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7104118660 con 0x7f71041084a0 2024-04-23T14:54:43.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.881+0000 7f710a400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7104118960 con 0x7f710410d3c0 2024-04-23T14:54:43.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.881+0000 7f7103e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7104108e40 0x7f71041a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.881+0000 7f7103e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7104108e40 0x7f71041a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58150/0 (socket says 172.21.15.83:58150) 2024-04-23T14:54:43.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.881+0000 7f7103e00640 1 -- 172.21.15.83:0/1466126439 learned_addr learned my addr 172.21.15.83:0/1466126439 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:43.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.881+0000 7f7109a00640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f710410d3c0 0x7f710419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.881+0000 7f7109000640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71041084a0 0x7f71041a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.882+0000 7f7103e00640 1 -- 172.21.15.83:0/1466126439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f710410d3c0 msgr2=0x7f710419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:43.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.882+0000 7f7103e00640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f710410d3c0 0x7f710419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.882+0000 7f7103e00640 1 -- 172.21.15.83:0/1466126439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71041084a0 msgr2=0x7f71041a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:43.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.882+0000 7f7103e00640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71041084a0 0x7f71041a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:43.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.882+0000 7f7103e00640 1 -- 172.21.15.83:0/1466126439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710419f2a0 con 0x7f7104108e40 2024-04-23T14:54:43.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.882+0000 7f7109000640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71041084a0 0x7f71041a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:43.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.882+0000 7f7103e00640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7104108e40 0x7f71041a4e60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f70f002a8f0 tx=0x7f70f0030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:43.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.882+0000 7f7101600640 1 -- 172.21.15.83:0/1466126439 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0040020 con 0x7f7104108e40 2024-04-23T14:54:43.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.882+0000 7f7101600640 1 -- 172.21.15.83:0/1466126439 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70f0005510 con 0x7f7104108e40 2024-04-23T14:54:43.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.883+0000 7f710a400640 1 -- 172.21.15.83:0/1466126439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710419f530 con 0x7f7104108e40 2024-04-23T14:54:43.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.883+0000 7f7101600640 1 -- 172.21.15.83:0/1466126439 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f00046c0 con 0x7f7104108e40 2024-04-23T14:54:43.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.883+0000 7f710a400640 1 -- 172.21.15.83:0/1466126439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71041ab7e0 con 0x7f7104108e40 2024-04-23T14:54:43.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.884+0000 7f710a400640 1 -- 172.21.15.83:0/1466126439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7104069190 con 0x7f7104108e40 2024-04-23T14:54:43.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.886+0000 7f7101600640 1 -- 172.21.15.83:0/1466126439 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f70f0043020 con 0x7f7104108e40 2024-04-23T14:54:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.887+0000 7f7101600640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f70ec0a3d20 0x7f70ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.888+0000 7f7101600640 1 -- 172.21.15.83:0/1466126439 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f70f00f1790 con 0x7f7104108e40 2024-04-23T14:54:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.888+0000 7f7109000640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f70ec0a3d20 0x7f70ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:43.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.889+0000 7f7109000640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f70ec0a3d20 0x7f70ec0a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f70f80096c0 tx=0x7f70f8009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:43.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:43.893+0000 7f7101600640 1 -- 172.21.15.83:0/1466126439 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f70f004a020 con 0x7f7104108e40 2024-04-23T14:54:43.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:43 smithi203 ceph-mon[32800]: pgmap v191: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:54:44.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:43 smithi173 ceph-mon[32443]: pgmap v191: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:54:44.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.136+0000 7f710a400640 1 -- 172.21.15.83:0/1466126439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7104110900 con 0x7f7104108e40 2024-04-23T14:54:44.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.138+0000 7f7101600640 1 -- 172.21.15.83:0/1466126439 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7f70f00bdde0 con 0x7f7104108e40 2024-04-23T14:54:44.139 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:54:44.139 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:54:44.140 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:54:44.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.140+0000 7f710a400640 1 -- 172.21.15.83:0/1466126439 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f70ec0a3d20 msgr2=0x7f70ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.140+0000 7f710a400640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f70ec0a3d20 0x7f70ec0a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f70f80096c0 tx=0x7f70f8009260 comp rx=0 tx=0).stop 2024-04-23T14:54:44.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.140+0000 7f710a400640 1 -- 172.21.15.83:0/1466126439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7104108e40 msgr2=0x7f71041a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.140+0000 7f710a400640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7104108e40 0x7f71041a4e60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f70f002a8f0 tx=0x7f70f0030650 comp rx=0 tx=0).stop 2024-04-23T14:54:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.140+0000 7f710a400640 1 -- 172.21.15.83:0/1466126439 shutdown_connections 2024-04-23T14:54:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.140+0000 7f710a400640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f710410d3c0 0x7f710419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.140+0000 7f710a400640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f70ec0a3d20 0x7f70ec0a61e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.140+0000 7f710a400640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7104108e40 0x7f71041a4e60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.140+0000 7f710a400640 1 --2- 172.21.15.83:0/1466126439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71041084a0 0x7f71041a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.140+0000 7f710a400640 1 -- 172.21.15.83:0/1466126439 >> 172.21.15.83:0/1466126439 conn(0x7f7104077b80 msgr2=0x7f7104078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.141+0000 7f710a400640 1 -- 172.21.15.83:0/1466126439 shutdown_connections 2024-04-23T14:54:44.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.141+0000 7f710a400640 1 -- 172.21.15.83:0/1466126439 wait complete. 2024-04-23T14:54:44.162 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:54:44.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.362+0000 7f5a9c400640 1 -- 172.21.15.83:0/1677260284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a94104450 msgr2=0x7f5a94104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.362+0000 7f5a9c400640 1 --2- 172.21.15.83:0/1677260284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a94104450 0x7f5a94104830 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009900 tx=0x7f5a8802a410 comp rx=0 tx=0).stop 2024-04-23T14:54:44.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.363+0000 7f5a9c400640 1 -- 172.21.15.83:0/1677260284 shutdown_connections 2024-04-23T14:54:44.364 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.363+0000 7f5a9c400640 1 --2- 172.21.15.83:0/1677260284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a94109520 0x7f5a941110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.363+0000 7f5a9c400640 1 --2- 172.21.15.83:0/1677260284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a94104e00 0x7f5a94108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.363+0000 7f5a9c400640 1 --2- 172.21.15.83:0/1677260284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a94104450 0x7f5a94104830 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.363+0000 7f5a9c400640 1 -- 172.21.15.83:0/1677260284 >> 172.21.15.83:0/1677260284 conn(0x7f5a940ffc70 msgr2=0x7f5a94102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.363+0000 7f5a9c400640 1 -- 172.21.15.83:0/1677260284 shutdown_connections 2024-04-23T14:54:44.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.363+0000 7f5a9c400640 1 -- 172.21.15.83:0/1677260284 wait complete. 2024-04-23T14:54:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.364+0000 7f5a9c400640 1 Processor -- start 2024-04-23T14:54:44.368 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.364+0000 7f5a9c400640 1 -- start start 2024-04-23T14:54:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.364+0000 7f5a9c400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a94104450 0x7f5a941a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.364+0000 7f5a9c400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a94109520 0x7f5a941a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.364+0000 7f5a9c400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a9419a970 0x7f5a9419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.364+0000 7f5a9c400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94114630 con 0x7f5a94109520 2024-04-23T14:54:44.369 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.364+0000 7f5a9c400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941144b0 con 0x7f5a9419a970 2024-04-23T14:54:44.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.364+0000 7f5a9c400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941147b0 con 0x7f5a94104450 2024-04-23T14:54:44.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.365+0000 7f5a99000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a94109520 0x7f5a941a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.365+0000 7f5a99000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a94109520 0x7f5a941a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58176/0 (socket says 172.21.15.83:58176) 2024-04-23T14:54:44.370 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.365+0000 7f5a99000640 1 -- 172.21.15.83:0/1288829488 learned_addr learned my addr 172.21.15.83:0/1288829488 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.365+0000 7f5a99a00640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a94104450 0x7f5a941a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.365+0000 7f5a9a400640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a9419a970 0x7f5a9419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.365+0000 7f5a99000640 1 -- 172.21.15.83:0/1288829488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a94104450 msgr2=0x7f5a941a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.371 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.365+0000 7f5a99000640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a94104450 0x7f5a941a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.365+0000 7f5a99000640 1 -- 172.21.15.83:0/1288829488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a9419a970 msgr2=0x7f5a9419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.365+0000 7f5a99000640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a9419a970 0x7f5a9419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.365+0000 7f5a99000640 1 -- 172.21.15.83:0/1288829488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9419b490 con 0x7f5a94109520 2024-04-23T14:54:44.372 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.365+0000 7f5a99000640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a94109520 0x7f5a941a0cb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000e9a0 tx=0x7f5a8000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.366+0000 7f5a92000640 1 -- 172.21.15.83:0/1288829488 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000cd80 con 0x7f5a94109520 2024-04-23T14:54:44.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.366+0000 7f5a92000640 1 -- 172.21.15.83:0/1288829488 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a800035f0 con 0x7f5a94109520 2024-04-23T14:54:44.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.366+0000 7f5a92000640 1 -- 172.21.15.83:0/1288829488 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80010670 con 0x7f5a94109520 2024-04-23T14:54:44.373 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.366+0000 7f5a9c400640 1 -- 172.21.15.83:0/1288829488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9419b780 con 0x7f5a94109520 2024-04-23T14:54:44.374 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.366+0000 7f5a9c400640 1 -- 172.21.15.83:0/1288829488 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a941a7930 con 0x7f5a94109520 2024-04-23T14:54:44.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.369+0000 7f5a9f225740 1 Processor -- start 2024-04-23T14:54:44.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.369+0000 7f5a9f225740 1 -- start start 2024-04-23T14:54:44.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9f225740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 0x558a56e2fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a92000640 1 -- 172.21.15.83:0/1288829488 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5a800037a0 con 0x7f5a94109520 2024-04-23T14:54:44.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9f225740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e30270 0x558a56e3a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9f225740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e3a940 0x558a56e3cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9f225740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558a56d951d0 con 0x558a56e2f950 2024-04-23T14:54:44.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9f225740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558a56d3b450 con 0x558a56e30270 2024-04-23T14:54:44.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9f225740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558a56d91190 con 0x558a56e3a940 2024-04-23T14:54:44.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 0x558a56e2fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 0x558a56e2fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58188/0 (socket says 172.21.15.83:58188) 2024-04-23T14:54:44.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9a400640 1 -- 172.21.15.83:0/670131343 learned_addr learned my addr 172.21.15.83:0/670131343 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a99a00640 1 --2- 172.21.15.83:0/670131343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e30270 0x558a56e3a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a99000640 1 --2- 172.21.15.83:0/670131343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e3a940 0x558a56e3cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9a400640 1 -- 172.21.15.83:0/670131343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e3a940 msgr2=0x558a56e3cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9a400640 1 --2- 172.21.15.83:0/670131343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e3a940 0x558a56e3cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.384 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9a400640 1 -- 172.21.15.83:0/670131343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e30270 msgr2=0x558a56e3a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9a400640 1 --2- 172.21.15.83:0/670131343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e30270 0x558a56e3a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a9a400640 1 -- 172.21.15.83:0/670131343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a56df8310 con 0x558a56e2f950 2024-04-23T14:54:44.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.370+0000 7f5a99a00640 1 --2- 172.21.15.83:0/670131343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e30270 0x558a56e3a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:44.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.371+0000 7f5a9a400640 1 --2- 172.21.15.83:0/670131343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 0x558a56e2fd30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009f50 tx=0x7f5a8c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5515425579184b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.371+0000 7f5a86a00640 1 -- 172.21.15.83:0/670131343 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c004d20 con 0x558a56e2f950 2024-04-23T14:54:44.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.371+0000 7f5a86a00640 1 -- 172.21.15.83:0/670131343 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a8c004050 con 0x558a56e2f950 2024-04-23T14:54:44.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.371+0000 7f5a86a00640 1 -- 172.21.15.83:0/670131343 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c005280 con 0x558a56e2f950 2024-04-23T14:54:44.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.371+0000 7f5a92000640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5a7c0a3da0 0x7f5a7c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.372+0000 7f5a92000640 1 -- 172.21.15.83:0/1288829488 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f5a800d0a30 con 0x7f5a94109520 2024-04-23T14:54:44.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.372+0000 7f5a99a00640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5a7c0a3da0 0x7f5a7c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.372+0000 7f5a9f225740 1 -- 172.21.15.83:0/670131343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 msgr2=0x558a56e2fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.372+0000 7f5a9f225740 1 --2- 172.21.15.83:0/670131343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 0x558a56e2fd30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009f50 tx=0x7f5a8c0118d0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.372+0000 7f5a9f225740 1 -- 172.21.15.83:0/670131343 shutdown_connections 2024-04-23T14:54:44.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.372+0000 7f5a9f225740 1 --2- 172.21.15.83:0/670131343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e3a940 0x558a56e3cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.372+0000 7f5a9f225740 1 --2- 172.21.15.83:0/670131343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e30270 0x558a56e3a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.372+0000 7f5a9f225740 1 --2- 172.21.15.83:0/670131343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 0x558a56e2fd30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.372+0000 7f5a9f225740 1 -- 172.21.15.83:0/670131343 >> 172.21.15.83:0/670131343 conn(0x558a56e2bc70 msgr2=0x558a56e2c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.372+0000 7f5a9f225740 1 -- 172.21.15.83:0/670131343 shutdown_connections 2024-04-23T14:54:44.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.372+0000 7f5a9f225740 1 -- 172.21.15.83:0/670131343 wait complete. 2024-04-23T14:54:44.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.372+0000 7f5a99a00640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5a7c0a3da0 0x7f5a7c0a6260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5a88006010 tx=0x7f5a88000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.373+0000 7f5a9f225740 1 -- start start 2024-04-23T14:54:44.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.373+0000 7f5a9f225740 1 Processor -- start 2024-04-23T14:54:44.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.373+0000 7f5a9f225740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 0x558a56e3a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.373+0000 7f5a9f225740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e30270 0x558a56e37660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.373+0000 7f5a9f225740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e3a940 0x558a56e37da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.373+0000 7f5a9f225740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558a56e3d7d0 con 0x558a56e2f950 2024-04-23T14:54:44.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.373+0000 7f5a9f225740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558a56d4e810 con 0x558a56e30270 2024-04-23T14:54:44.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.373+0000 7f5a9f225740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558a56e3d950 con 0x558a56e3a940 2024-04-23T14:54:44.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.373+0000 7f5a9f225740 10 client.0 _ll_register_callbacks cb 0x558a56d1c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:44.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.374+0000 7f5a99a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 0x558a56e3a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.374+0000 7f5a99000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e30270 0x558a56e37660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.374+0000 7f5a99a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 0x558a56e3a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58202/0 (socket says 172.21.15.83:58202) 2024-04-23T14:54:44.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.374+0000 7f5a99a00640 1 -- 172.21.15.83:0/1256063205 learned_addr learned my addr 172.21.15.83:0/1256063205 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.374+0000 7f5a9a400640 1 --2- 172.21.15.83:0/1256063205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e3a940 0x558a56e37da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.374+0000 7f5a99000640 1 -- 172.21.15.83:0/1256063205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e3a940 msgr2=0x558a56e37da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.374+0000 7f5a99000640 1 --2- 172.21.15.83:0/1256063205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e3a940 0x558a56e37da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.374+0000 7f5a99000640 1 -- 172.21.15.83:0/1256063205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 msgr2=0x558a56e3a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.374+0000 7f5a99000640 1 --2- 172.21.15.83:0/1256063205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 0x558a56e3a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.374+0000 7f5a99000640 1 -- 172.21.15.83:0/1256063205 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a56d5f3f0 con 0x558a56e30270 2024-04-23T14:54:44.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.374+0000 7f5a99000640 1 --2- 172.21.15.83:0/1256063205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e30270 0x558a56e37660 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5a800ce130 tx=0x7f5a800bd100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d408dc6d535f2a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.375+0000 7f5a7b400640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:54:44.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.375+0000 7f5a7b400640 1 -- 172.21.15.83:0/1256063205 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8009c4c0 con 0x558a56e30270 2024-04-23T14:54:44.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.375+0000 7f5a9f225740 1 -- 172.21.15.83:0/1256063205 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a8c004050 con 0x558a56e30270 2024-04-23T14:54:44.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.375+0000 7f5a7b400640 1 -- 172.21.15.83:0/1256063205 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f5a800870b0 con 0x558a56e30270 2024-04-23T14:54:44.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.375+0000 7f5a7b400640 1 -- 172.21.15.83:0/1256063205 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a8009c2e0 con 0x558a56e30270 2024-04-23T14:54:44.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.375+0000 7f5a9f225740 10 client.26435 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:44.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.375+0000 7f5a9f225740 1 -- 172.21.15.83:0/1256063205 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a56aa73f0 con 0x558a56e30270 2024-04-23T14:54:44.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.375+0000 7f5a7b400640 1 -- 172.21.15.83:0/1256063205 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a8c004050 con 0x558a56e30270 2024-04-23T14:54:44.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.375+0000 7f5a7b400640 1 -- 172.21.15.83:0/1256063205 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f5a800bbb40 con 0x558a56e30270 2024-04-23T14:54:44.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.376+0000 7f5a9f225740 10 client.26435 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:44.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.376+0000 7f5a9f225740 10 client.26435 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:54:44.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.376+0000 7f5a9f225740 20 client.26435 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:44.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.376+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1256063205 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558a56e4ab10 0x558a56e4cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.376+0000 7f5a9f225740 4 client.26435 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:54:44.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.376+0000 7f5a9f225740 1 -- 172.21.15.83:0/1256063205 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a56da2bf0 con 0x558a56e4ab10 2024-04-23T14:54:44.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.376+0000 7f5a99a00640 1 --2- 172.21.15.83:0/1256063205 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558a56e4ab10 0x558a56e4cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.377+0000 7f5a7b400640 10 client.26435 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:54:44.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.377+0000 7f5a99a00640 1 --2- 172.21.15.83:0/1256063205 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558a56e4ab10 0x558a56e4cf20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c836ec1dfada5c1 server_cookie=803529cd43c44b4f in_seq=0 out_seq=0 2024-04-23T14:54:44.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.377+0000 7f5a7b400640 1 -- 172.21.15.83:0/1256063205 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a56da2bf0 con 0x558a56e4ab10 2024-04-23T14:54:44.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.377+0000 7f5a7b400640 10 client.26435 handle_command_reply: tid=0 2024-04-23T14:54:44.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.377+0000 7f5a9f225740 1 client.26435 shutdown 2024-04-23T14:54:44.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.378+0000 7f5a9f225740 1 -- 172.21.15.83:0/1256063205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e30270 msgr2=0x558a56e37660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.378+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1256063205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e30270 0x558a56e37660 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5a800ce130 tx=0x7f5a800bd100 comp rx=0 tx=0).stop 2024-04-23T14:54:44.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.378+0000 7f5a9f225740 1 -- 172.21.15.83:0/1256063205 shutdown_connections 2024-04-23T14:54:44.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.378+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1256063205 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558a56e4ab10 0x558a56e4cf20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.378+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1256063205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e3a940 0x558a56e37da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.378+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1256063205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e30270 0x558a56e37660 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.378+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1256063205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e2f950 0x558a56e3a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.378+0000 7f5a7b400640 0 client.26435 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:54:44.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.378+0000 7f5a9f225740 1 -- 172.21.15.83:0/1256063205 >> 172.21.15.83:0/1256063205 conn(0x558a56e2bc70 msgr2=0x558a56e31970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.378+0000 7f5a9f225740 1 -- 172.21.15.83:0/1256063205 shutdown_connections 2024-04-23T14:54:44.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.378+0000 7f5a9f225740 1 -- 172.21.15.83:0/1256063205 wait complete. 2024-04-23T14:54:44.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.378+0000 7f5a9f225740 20 client.26435 trim_cache size 0 max 16384 2024-04-23T14:54:44.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.392+0000 7f5a9f225740 1 Processor -- start 2024-04-23T14:54:44.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.392+0000 7f5a9f225740 1 -- start start 2024-04-23T14:54:44.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a9f225740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e2f950 0x558a56e2b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a9f225740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e30270 0x558a56e39ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a9f225740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e3a940 0x558a56e3a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a9f225740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941b2850 con 0x558a56e30270 2024-04-23T14:54:44.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a9f225740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941b0120 con 0x558a56e3a940 2024-04-23T14:54:44.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a9f225740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941adf70 con 0x558a56e2f950 2024-04-23T14:54:44.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a99000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e30270 0x558a56e39ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a99000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e30270 0x558a56e39ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58206/0 (socket says 172.21.15.83:58206) 2024-04-23T14:54:44.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a99000640 1 -- 172.21.15.83:0/1613654905 learned_addr learned my addr 172.21.15.83:0/1613654905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a99a00640 1 --2- 172.21.15.83:0/1613654905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e2f950 0x558a56e2b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a9a400640 1 --2- 172.21.15.83:0/1613654905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e3a940 0x558a56e3a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a99000640 1 -- 172.21.15.83:0/1613654905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e2f950 msgr2=0x558a56e2b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a99000640 1 --2- 172.21.15.83:0/1613654905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e2f950 0x558a56e2b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a99000640 1 -- 172.21.15.83:0/1613654905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e3a940 msgr2=0x558a56e3a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a99000640 1 --2- 172.21.15.83:0/1613654905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e3a940 0x558a56e3a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a99000640 1 -- 172.21.15.83:0/1613654905 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a56b717d0 con 0x558a56e30270 2024-04-23T14:54:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.393+0000 7f5a9a400640 1 --2- 172.21.15.83:0/1613654905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e3a940 0x558a56e3a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.394+0000 7f5a99000640 1 --2- 172.21.15.83:0/1613654905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e30270 0x558a56e39ea0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a80099300 tx=0x7f5a80003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c2057f6ae72fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.394+0000 7f5a86000640 1 -- 172.21.15.83:0/1613654905 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8009c4c0 con 0x558a56e30270 2024-04-23T14:54:44.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.394+0000 7f5a86000640 1 -- 172.21.15.83:0/1613654905 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a800a0d30 con 0x558a56e30270 2024-04-23T14:54:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.394+0000 7f5a86000640 1 -- 172.21.15.83:0/1613654905 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a80087040 con 0x558a56e30270 2024-04-23T14:54:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.395+0000 7f5a9f225740 1 -- 172.21.15.83:0/1613654905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e30270 msgr2=0x558a56e39ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.395+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1613654905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e30270 0x558a56e39ea0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a80099300 tx=0x7f5a80003d20 comp rx=0 tx=0).stop 2024-04-23T14:54:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.395+0000 7f5a9f225740 1 -- 172.21.15.83:0/1613654905 shutdown_connections 2024-04-23T14:54:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.395+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1613654905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e3a940 0x558a56e3a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.395+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1613654905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e30270 0x558a56e39ea0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.395+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1613654905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e2f950 0x558a56e2b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.395+0000 7f5a9f225740 1 -- 172.21.15.83:0/1613654905 >> 172.21.15.83:0/1613654905 conn(0x558a56e2bc70 msgr2=0x558a56e328c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.395+0000 7f5a9f225740 1 -- 172.21.15.83:0/1613654905 shutdown_connections 2024-04-23T14:54:44.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.395+0000 7f5a9f225740 1 -- 172.21.15.83:0/1613654905 wait complete. 2024-04-23T14:54:44.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.395+0000 7f5a9f225740 1 -- start start 2024-04-23T14:54:44.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.395+0000 7f5a9f225740 1 Processor -- start 2024-04-23T14:54:44.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a9f225740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e2f950 0x558a56e43000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a9f225740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e30270 0x558a56e3b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a9f225740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e3a940 0x558a56e3b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a9f225740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941ae5e0 con 0x558a56e3a940 2024-04-23T14:54:44.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a9f225740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941afa50 con 0x558a56e2f950 2024-04-23T14:54:44.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a9f225740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941aecb0 con 0x558a56e30270 2024-04-23T14:54:44.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a9f225740 10 client.0 _ll_register_callbacks cb 0x7f5a8000ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:44.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a99000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e30270 0x558a56e3b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a9a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e3a940 0x558a56e3b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a99000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e30270 0x558a56e3b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:34534/0 (socket says 172.21.15.83:34534) 2024-04-23T14:54:44.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a99000640 1 -- 172.21.15.83:0/1172040732 learned_addr learned my addr 172.21.15.83:0/1172040732 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a99a00640 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e2f950 0x558a56e43000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a99000640 1 -- 172.21.15.83:0/1172040732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e2f950 msgr2=0x558a56e43000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a99000640 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e2f950 0x558a56e43000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a99000640 1 -- 172.21.15.83:0/1172040732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e3a940 msgr2=0x558a56e3b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a99000640 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e3a940 0x558a56e3b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a99000640 1 -- 172.21.15.83:0/1172040732 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a56e3c110 con 0x558a56e30270 2024-04-23T14:54:44.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.396+0000 7f5a9a400640 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e3a940 0x558a56e3b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:44.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.397+0000 7f5a99000640 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e30270 0x558a56e3b3a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5a800a0ef0 tx=0x7f5a800ce630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753466333173a5db server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.397+0000 7f5a7b400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:54:44.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.397+0000 7f5a7b400640 1 -- 172.21.15.83:0/1172040732 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8009c980 con 0x558a56e30270 2024-04-23T14:54:44.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.397+0000 7f5a7b400640 1 -- 172.21.15.83:0/1172040732 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f5a8009bdd0 con 0x558a56e30270 2024-04-23T14:54:44.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.397+0000 7f5a9f225740 1 -- 172.21.15.83:0/1172040732 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558a56d9ca60 con 0x558a56e30270 2024-04-23T14:54:44.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.397+0000 7f5a7b400640 1 -- 172.21.15.83:0/1172040732 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a800a55f0 con 0x558a56e30270 2024-04-23T14:54:44.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.397+0000 7f5a9f225740 10 client.26410 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:44.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.397+0000 7f5a9f225740 1 -- 172.21.15.83:0/1172040732 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a56eb10c0 con 0x558a56e30270 2024-04-23T14:54:44.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.397+0000 7f5a7b400640 1 -- 172.21.15.83:0/1172040732 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558a56d9ca60 con 0x558a56e30270 2024-04-23T14:54:44.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.398+0000 7f5a7b400640 1 -- 172.21.15.83:0/1172040732 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f5a800041e0 con 0x558a56e30270 2024-04-23T14:54:44.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.398+0000 7f5a9f225740 10 client.26410 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:44.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.398+0000 7f5a9f225740 10 client.26410 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:54:44.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.398+0000 7f5a9f225740 20 client.26410 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:44.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.398+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558a56eb15f0 0x558a56eb3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.398+0000 7f5a9f225740 4 client.26410 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:54:44.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.398+0000 7f5a9f225740 1 -- 172.21.15.83:0/1172040732 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a56d184d0 con 0x558a56eb15f0 2024-04-23T14:54:44.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.398+0000 7f5a99a00640 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558a56eb15f0 0x558a56eb3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.398+0000 7f5a99a00640 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558a56eb15f0 0x558a56eb3a10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45a804019c9f3afb server_cookie=1a13e03b357da01a in_seq=0 out_seq=0 2024-04-23T14:54:44.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.398+0000 7f5a7b400640 10 client.26410 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:54:44.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a7b400640 1 -- 172.21.15.83:0/1172040732 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a56d184d0 con 0x558a56eb15f0 2024-04-23T14:54:44.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a7b400640 10 client.26410 handle_command_reply: tid=0 2024-04-23T14:54:44.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a9f225740 1 client.26410 shutdown 2024-04-23T14:54:44.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a9f225740 1 -- 172.21.15.83:0/1172040732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e30270 msgr2=0x558a56e3b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e30270 0x558a56e3b3a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5a800a0ef0 tx=0x7f5a800ce630 comp rx=0 tx=0).stop 2024-04-23T14:54:44.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a9f225740 1 -- 172.21.15.83:0/1172040732 shutdown_connections 2024-04-23T14:54:44.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x558a56eb15f0 0x558a56eb3a10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558a56e3a940 0x558a56e3b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558a56e30270 0x558a56e3b3a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a9f225740 1 --2- 172.21.15.83:0/1172040732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558a56e2f950 0x558a56e43000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a9f225740 1 -- 172.21.15.83:0/1172040732 >> 172.21.15.83:0/1172040732 conn(0x558a56e2bc70 msgr2=0x558a56e310e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a7b400640 0 client.26410 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:54:44.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a9f225740 1 -- 172.21.15.83:0/1172040732 shutdown_connections 2024-04-23T14:54:44.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a9f225740 1 -- 172.21.15.83:0/1172040732 wait complete. 2024-04-23T14:54:44.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.399+0000 7f5a9f225740 20 client.26410 trim_cache size 0 max 16384 2024-04-23T14:54:44.410 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:54:44.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.400+0000 7f5a9c400640 1 -- 172.21.15.83:0/1288829488 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5a7c0a3da0 msgr2=0x7f5a7c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.400+0000 7f5a9c400640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5a7c0a3da0 0x7f5a7c0a6260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5a88006010 tx=0x7f5a88000bc0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.400+0000 7f5a9c400640 1 -- 172.21.15.83:0/1288829488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a94109520 msgr2=0x7f5a941a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.400+0000 7f5a9c400640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a94109520 0x7f5a941a0cb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000e9a0 tx=0x7f5a8000ee60 comp rx=0 tx=0).stop 2024-04-23T14:54:44.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.400+0000 7f5a9c400640 1 -- 172.21.15.83:0/1288829488 shutdown_connections 2024-04-23T14:54:44.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.400+0000 7f5a9c400640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a9419a970 0x7f5a9419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.400+0000 7f5a9c400640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5a7c0a3da0 0x7f5a7c0a6260 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.400+0000 7f5a9c400640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a94109520 0x7f5a941a0cb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.400+0000 7f5a9c400640 1 --2- 172.21.15.83:0/1288829488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a94104450 0x7f5a941a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.400+0000 7f5a9c400640 1 -- 172.21.15.83:0/1288829488 >> 172.21.15.83:0/1288829488 conn(0x7f5a940ffc70 msgr2=0x7f5a94101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.400+0000 7f5a9c400640 1 -- 172.21.15.83:0/1288829488 shutdown_connections 2024-04-23T14:54:44.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.400+0000 7f5a9c400640 1 -- 172.21.15.83:0/1288829488 wait complete. 2024-04-23T14:54:44.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:54:44.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.607+0000 7feb23200640 1 -- 172.21.15.83:0/3264472016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb1c102bf0 msgr2=0x7feb1c106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.607+0000 7feb23200640 1 --2- 172.21.15.83:0/3264472016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb1c102bf0 0x7feb1c106c30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feb08009ca0 tx=0x7feb0802a8b0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.607+0000 7feb23200640 1 -- 172.21.15.83:0/3264472016 shutdown_connections 2024-04-23T14:54:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.607+0000 7feb23200640 1 --2- 172.21.15.83:0/3264472016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb1c107310 0x7feb1c10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.607+0000 7feb23200640 1 --2- 172.21.15.83:0/3264472016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb1c102bf0 0x7feb1c106c30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.607+0000 7feb23200640 1 --2- 172.21.15.83:0/3264472016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb1c102240 0x7feb1c102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.607+0000 7feb23200640 1 -- 172.21.15.83:0/3264472016 >> 172.21.15.83:0/3264472016 conn(0x7feb1c079450 msgr2=0x7feb1c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.607+0000 7feb23200640 1 -- 172.21.15.83:0/3264472016 shutdown_connections 2024-04-23T14:54:44.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.607+0000 7feb23200640 1 -- 172.21.15.83:0/3264472016 wait complete. 2024-04-23T14:54:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.608+0000 7feb23200640 1 Processor -- start 2024-04-23T14:54:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb23200640 1 -- start start 2024-04-23T14:54:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb23200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb1c102240 0x7feb1c1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb23200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb1c107310 0x7feb1c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.611 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb23200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb1c19a950 0x7feb1c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb23200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c112380 con 0x7feb1c19a950 2024-04-23T14:54:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb23200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c112200 con 0x7feb1c102240 2024-04-23T14:54:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb23200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c112500 con 0x7feb1c107310 2024-04-23T14:54:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb1be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb1c102240 0x7feb1c1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb1b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb1c107310 0x7feb1c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb21200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb1c19a950 0x7feb1c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb1be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb1c102240 0x7feb1c1a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:59342/0 (socket says 172.21.15.83:59342) 2024-04-23T14:54:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb1b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb1c107310 0x7feb1c1a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:34554/0 (socket says 172.21.15.83:34554) 2024-04-23T14:54:44.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.609+0000 7feb1be00640 1 -- 172.21.15.83:0/2388581290 learned_addr learned my addr 172.21.15.83:0/2388581290 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.610+0000 7feb21200640 1 -- 172.21.15.83:0/2388581290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb1c107310 msgr2=0x7feb1c1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.610+0000 7feb21200640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb1c107310 0x7feb1c1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.610+0000 7feb21200640 1 -- 172.21.15.83:0/2388581290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb1c102240 msgr2=0x7feb1c1a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.610+0000 7feb21200640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb1c102240 0x7feb1c1a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.610+0000 7feb21200640 1 -- 172.21.15.83:0/2388581290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c19b470 con 0x7feb1c19a950 2024-04-23T14:54:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.610+0000 7feb1be00640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb1c102240 0x7feb1c1a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:44.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.610+0000 7feb21200640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb1c19a950 0x7feb1c19ae00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feb1400d680 tx=0x7feb1400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.611+0000 7feb18c00640 1 -- 172.21.15.83:0/2388581290 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14017070 con 0x7feb1c19a950 2024-04-23T14:54:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.611+0000 7feb23200640 1 -- 172.21.15.83:0/2388581290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c19b760 con 0x7feb1c19a950 2024-04-23T14:54:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.611+0000 7feb18c00640 1 -- 172.21.15.83:0/2388581290 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb14009820 con 0x7feb1c19a950 2024-04-23T14:54:44.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.611+0000 7feb18c00640 1 -- 172.21.15.83:0/2388581290 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14004e00 con 0x7feb1c19a950 2024-04-23T14:54:44.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.611+0000 7feb23200640 1 -- 172.21.15.83:0/2388581290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c1a7960 con 0x7feb1c19a950 2024-04-23T14:54:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.614+0000 7feb25f93740 1 Processor -- start 2024-04-23T14:54:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.614+0000 7feb18c00640 1 -- 172.21.15.83:0/2388581290 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feb140099d0 con 0x7feb1c19a950 2024-04-23T14:54:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.614+0000 7feb25f93740 1 -- start start 2024-04-23T14:54:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.615+0000 7feb25f93740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af48950 0x55fa3af48d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.615+0000 7feb25f93740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af49270 0x55fa3af53400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.615+0000 7feb25f93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af53940 0x55fa3af55d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.615+0000 7feb25f93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fa3aeae1d0 con 0x55fa3af53940 2024-04-23T14:54:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.615+0000 7feb25f93740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fa3ae54450 con 0x55fa3af49270 2024-04-23T14:54:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.615+0000 7feb25f93740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fa3aeaa190 con 0x55fa3af48950 2024-04-23T14:54:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.615+0000 7feb1b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af48950 0x55fa3af48d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.615+0000 7feb1b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af53940 0x55fa3af55d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.615+0000 7feb1be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af49270 0x55fa3af53400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.615+0000 7feb1b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af53940 0x55fa3af55d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58232/0 (socket says 172.21.15.83:58232) 2024-04-23T14:54:44.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.615+0000 7feb1b400640 1 -- 172.21.15.83:0/1753369470 learned_addr learned my addr 172.21.15.83:0/1753369470 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb1be00640 1 -- 172.21.15.83:0/1753369470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af48950 msgr2=0x55fa3af48d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb1be00640 1 --2- 172.21.15.83:0/1753369470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af48950 0x55fa3af48d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb1be00640 1 -- 172.21.15.83:0/1753369470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af53940 msgr2=0x55fa3af55d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb1be00640 1 --2- 172.21.15.83:0/1753369470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af53940 0x55fa3af55d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb1be00640 1 -- 172.21.15.83:0/1753369470 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa3af11310 con 0x55fa3af49270 2024-04-23T14:54:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb1b400640 1 --2- 172.21.15.83:0/1753369470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af53940 0x55fa3af55d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:44.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb18c00640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feb000a3d20 0x7feb000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb1be00640 1 --2- 172.21.15.83:0/1753369470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af49270 0x55fa3af53400 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb1c19ba90 tx=0x7feb0c0109e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef43b366ce86fd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb18c00640 1 -- 172.21.15.83:0/2388581290 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7feb140cff90 con 0x7feb1c19a950 2024-04-23T14:54:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb1b400640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feb000a3d20 0x7feb000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb11600640 1 -- 172.21.15.83:0/1753369470 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c011820 con 0x55fa3af49270 2024-04-23T14:54:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb11600640 1 -- 172.21.15.83:0/1753369470 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0c01b430 con 0x55fa3af49270 2024-04-23T14:54:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.616+0000 7feb11600640 1 -- 172.21.15.83:0/1753369470 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c00e610 con 0x55fa3af49270 2024-04-23T14:54:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.617+0000 7feb25f93740 1 -- 172.21.15.83:0/1753369470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af49270 msgr2=0x55fa3af53400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.617+0000 7feb1b400640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feb000a3d20 0x7feb000a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb080308d0 tx=0x7feb08004a40 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.617+0000 7feb25f93740 1 --2- 172.21.15.83:0/1753369470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af49270 0x55fa3af53400 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb1c19ba90 tx=0x7feb0c0109e0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.617+0000 7feb25f93740 1 -- 172.21.15.83:0/1753369470 shutdown_connections 2024-04-23T14:54:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.617+0000 7feb25f93740 1 --2- 172.21.15.83:0/1753369470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af53940 0x55fa3af55d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.617+0000 7feb25f93740 1 --2- 172.21.15.83:0/1753369470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af49270 0x55fa3af53400 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.617+0000 7feb25f93740 1 --2- 172.21.15.83:0/1753369470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af48950 0x55fa3af48d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.617+0000 7feb25f93740 1 -- 172.21.15.83:0/1753369470 >> 172.21.15.83:0/1753369470 conn(0x55fa3af44c70 msgr2=0x55fa3af45050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.617+0000 7feb25f93740 1 -- 172.21.15.83:0/1753369470 shutdown_connections 2024-04-23T14:54:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.617+0000 7feb25f93740 1 -- 172.21.15.83:0/1753369470 wait complete. 2024-04-23T14:54:44.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.618+0000 7feb25f93740 1 -- start start 2024-04-23T14:54:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.618+0000 7feb25f93740 1 Processor -- start 2024-04-23T14:54:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.618+0000 7feb25f93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af48950 0x55fa3af533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.618+0000 7feb25f93740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af49270 0x55fa3af506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.618+0000 7feb25f93740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af53940 0x55fa3af50db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.618+0000 7feb25f93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fa3af567d0 con 0x55fa3af48950 2024-04-23T14:54:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.618+0000 7feb25f93740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fa3ae67810 con 0x55fa3af53940 2024-04-23T14:54:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.618+0000 7feb25f93740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fa3af56950 con 0x55fa3af49270 2024-04-23T14:54:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.619+0000 7feb25f93740 10 client.0 _ll_register_callbacks cb 0x55fa3ae35140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:44.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.619+0000 7feb1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af48950 0x55fa3af533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.619+0000 7feb1b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af49270 0x55fa3af506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.619+0000 7feb1be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af48950 0x55fa3af533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58242/0 (socket says 172.21.15.83:58242) 2024-04-23T14:54:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.619+0000 7feb1be00640 1 -- 172.21.15.83:0/2232242002 learned_addr learned my addr 172.21.15.83:0/2232242002 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.619+0000 7feb21200640 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af53940 0x55fa3af50db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.619+0000 7feb1be00640 1 -- 172.21.15.83:0/2232242002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af49270 msgr2=0x55fa3af506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.619+0000 7feb1be00640 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af49270 0x55fa3af506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.619+0000 7feb1be00640 1 -- 172.21.15.83:0/2232242002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af53940 msgr2=0x55fa3af50db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.619+0000 7feb1be00640 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af53940 0x55fa3af50db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.619+0000 7feb1be00640 1 -- 172.21.15.83:0/2232242002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa3ae783f0 con 0x55fa3af48950 2024-04-23T14:54:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.619+0000 7feb21200640 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af53940 0x55fa3af50db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.620+0000 7feb1be00640 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af48950 0x55fa3af533d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7feb0c01bdf0 tx=0x7feb0c01bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87679963a77297b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.620+0000 7feb06000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:54:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.620+0000 7feb06000640 1 -- 172.21.15.83:0/2232242002 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c028020 con 0x55fa3af48950 2024-04-23T14:54:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.620+0000 7feb25f93740 1 -- 172.21.15.83:0/2232242002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0c01b430 con 0x55fa3af48950 2024-04-23T14:54:44.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.620+0000 7feb06000640 1 -- 172.21.15.83:0/2232242002 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7feb0c005440 con 0x55fa3af48950 2024-04-23T14:54:44.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.620+0000 7feb06000640 1 -- 172.21.15.83:0/2232242002 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feb0c00bd20 con 0x55fa3af48950 2024-04-23T14:54:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.620+0000 7feb25f93740 10 client.16245 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.620+0000 7feb25f93740 1 -- 172.21.15.83:0/2232242002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa3abc03f0 con 0x55fa3af48950 2024-04-23T14:54:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.621+0000 7feb06000640 1 -- 172.21.15.83:0/2232242002 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0c01b430 con 0x55fa3af48950 2024-04-23T14:54:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.621+0000 7feb06000640 1 -- 172.21.15.83:0/2232242002 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7feb0c023070 con 0x55fa3af48950 2024-04-23T14:54:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.621+0000 7feb25f93740 10 client.16245 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.621+0000 7feb25f93740 10 client.16245 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:54:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.621+0000 7feb25f93740 20 client.16245 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:44.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.621+0000 7feb25f93740 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa3af63a90 0x55fa3af65e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.621+0000 7feb25f93740 4 client.16245 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:54:44.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.621+0000 7feb25f93740 1 -- 172.21.15.83:0/2232242002 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa3aebbbf0 con 0x55fa3af63a90 2024-04-23T14:54:44.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.621+0000 7feb1b400640 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa3af63a90 0x55fa3af65e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.622+0000 7feb1b400640 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa3af63a90 0x55fa3af65e80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35c5e126cd6ac4bb server_cookie=a40bb181673a91b in_seq=0 out_seq=0 2024-04-23T14:54:44.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.622+0000 7feb06000640 10 client.16245 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:54:44.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb06000640 1 -- 172.21.15.83:0/2232242002 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa3aebbbf0 con 0x55fa3af63a90 2024-04-23T14:54:44.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb06000640 10 client.16245 handle_command_reply: tid=0 2024-04-23T14:54:44.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb25f93740 1 client.16245 shutdown 2024-04-23T14:54:44.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb25f93740 1 -- 172.21.15.83:0/2232242002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af48950 msgr2=0x55fa3af533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb25f93740 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af48950 0x55fa3af533d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7feb0c01bdf0 tx=0x7feb0c01bb70 comp rx=0 tx=0).stop 2024-04-23T14:54:44.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb25f93740 1 -- 172.21.15.83:0/2232242002 shutdown_connections 2024-04-23T14:54:44.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb25f93740 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af53940 0x55fa3af50db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb25f93740 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa3af63a90 0x55fa3af65e80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb25f93740 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af49270 0x55fa3af506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb25f93740 1 --2- 172.21.15.83:0/2232242002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af48950 0x55fa3af533d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb06000640 0 client.16245 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:54:44.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb25f93740 1 -- 172.21.15.83:0/2232242002 >> 172.21.15.83:0/2232242002 conn(0x55fa3af44c70 msgr2=0x55fa3af4aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb25f93740 1 -- 172.21.15.83:0/2232242002 shutdown_connections 2024-04-23T14:54:44.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.623+0000 7feb25f93740 1 -- 172.21.15.83:0/2232242002 wait complete. 2024-04-23T14:54:44.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.624+0000 7feb25f93740 20 client.16245 trim_cache size 0 max 16384 2024-04-23T14:54:44.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.639+0000 7feb25f93740 1 Processor -- start 2024-04-23T14:54:44.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.639+0000 7feb25f93740 1 -- start start 2024-04-23T14:54:44.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.639+0000 7feb25f93740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af48950 0x55fa3af5f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.639+0000 7feb25f93740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af49270 0x55fa3af4f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb25f93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af53940 0x55fa3af52f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb25f93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1b2870 con 0x55fa3af53940 2024-04-23T14:54:44.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb25f93740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1b0140 con 0x55fa3af48950 2024-04-23T14:54:44.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb25f93740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1adf90 con 0x55fa3af49270 2024-04-23T14:54:44.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb1be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af48950 0x55fa3af5f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.642 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb21200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af53940 0x55fa3af52f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb1be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af48950 0x55fa3af5f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:59380/0 (socket says 172.21.15.83:59380) 2024-04-23T14:54:44.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb1be00640 1 -- 172.21.15.83:0/3923103590 learned_addr learned my addr 172.21.15.83:0/3923103590 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb1b400640 1 --2- 172.21.15.83:0/3923103590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af49270 0x55fa3af4f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb1be00640 1 -- 172.21.15.83:0/3923103590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af49270 msgr2=0x55fa3af4f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb1be00640 1 --2- 172.21.15.83:0/3923103590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af49270 0x55fa3af4f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb1be00640 1 -- 172.21.15.83:0/3923103590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af53940 msgr2=0x55fa3af52f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb1be00640 1 --2- 172.21.15.83:0/3923103590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af53940 0x55fa3af52f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.640+0000 7feb1be00640 1 -- 172.21.15.83:0/3923103590 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa3ac8a7d0 con 0x55fa3af48950 2024-04-23T14:54:44.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.641+0000 7feb1be00640 1 --2- 172.21.15.83:0/3923103590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af48950 0x55fa3af5f690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00bfe0 tx=0x7feb0c01be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ba1784e5d3eb940 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.644 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.644+0000 7feb10c00640 1 -- 172.21.15.83:0/3923103590 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c03b070 con 0x55fa3af48950 2024-04-23T14:54:44.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.644+0000 7feb10c00640 1 -- 172.21.15.83:0/3923103590 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0c01bbe0 con 0x55fa3af48950 2024-04-23T14:54:44.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.644+0000 7feb10c00640 1 -- 172.21.15.83:0/3923103590 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c003850 con 0x55fa3af48950 2024-04-23T14:54:44.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.644+0000 7feb25f93740 1 -- 172.21.15.83:0/3923103590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af48950 msgr2=0x55fa3af5f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.644+0000 7feb25f93740 1 --2- 172.21.15.83:0/3923103590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af48950 0x55fa3af5f690 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feb0c00bfe0 tx=0x7feb0c01be20 comp rx=0 tx=0).stop 2024-04-23T14:54:44.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.645+0000 7feb25f93740 1 -- 172.21.15.83:0/3923103590 shutdown_connections 2024-04-23T14:54:44.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.645+0000 7feb25f93740 1 --2- 172.21.15.83:0/3923103590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af53940 0x55fa3af52f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.645+0000 7feb25f93740 1 --2- 172.21.15.83:0/3923103590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af49270 0x55fa3af4f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.645+0000 7feb25f93740 1 --2- 172.21.15.83:0/3923103590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af48950 0x55fa3af5f690 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.645+0000 7feb25f93740 1 -- 172.21.15.83:0/3923103590 >> 172.21.15.83:0/3923103590 conn(0x55fa3af44c70 msgr2=0x55fa3af4ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.645+0000 7feb25f93740 1 -- 172.21.15.83:0/3923103590 shutdown_connections 2024-04-23T14:54:44.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.645+0000 7feb25f93740 1 -- 172.21.15.83:0/3923103590 wait complete. 2024-04-23T14:54:44.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.645+0000 7feb25f93740 1 -- start start 2024-04-23T14:54:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.645+0000 7feb25f93740 1 Processor -- start 2024-04-23T14:54:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb25f93740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af48950 0x55fa3af54090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb25f93740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af49270 0x55fa3af545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb25f93740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af53940 0x55fa3af54b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb25f93740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1ae600 con 0x55fa3af49270 2024-04-23T14:54:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb25f93740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1afa70 con 0x55fa3af53940 2024-04-23T14:54:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb25f93740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c1aecd0 con 0x55fa3af48950 2024-04-23T14:54:44.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb25f93740 10 client.0 _ll_register_callbacks cb 0x7feb08009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb1be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af48950 0x55fa3af54090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb1b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af49270 0x55fa3af545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb1be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af48950 0x55fa3af54090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:34606/0 (socket says 172.21.15.83:34606) 2024-04-23T14:54:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb1be00640 1 -- 172.21.15.83:0/2226417383 learned_addr learned my addr 172.21.15.83:0/2226417383 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb21200640 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af53940 0x55fa3af54b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb1b400640 1 -- 172.21.15.83:0/2226417383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af48950 msgr2=0x55fa3af54090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb1b400640 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af48950 0x55fa3af54090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.652 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb1b400640 1 -- 172.21.15.83:0/2226417383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af53940 msgr2=0x55fa3af54b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb1b400640 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af53940 0x55fa3af54b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb1b400640 1 -- 172.21.15.83:0/2226417383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa3af5aa20 con 0x55fa3af49270 2024-04-23T14:54:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.646+0000 7feb1be00640 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af48950 0x55fa3af54090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.647+0000 7feb1b400640 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af49270 0x55fa3af545d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55fa3af45e00 tx=0x7feb0802a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770a21ad182b055b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.647+0000 7feb06000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:54:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.647+0000 7feb06000640 1 -- 172.21.15.83:0/2226417383 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08045040 con 0x55fa3af49270 2024-04-23T14:54:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.647+0000 7feb25f93740 1 -- 172.21.15.83:0/2226417383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0c01bbe0 con 0x55fa3af49270 2024-04-23T14:54:44.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.647+0000 7feb06000640 1 -- 172.21.15.83:0/2226417383 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7feb08003cf0 con 0x55fa3af49270 2024-04-23T14:54:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.647+0000 7feb06000640 1 -- 172.21.15.83:0/2226417383 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feb08059420 con 0x55fa3af49270 2024-04-23T14:54:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.647+0000 7feb25f93740 10 client.16251 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.647+0000 7feb25f93740 1 -- 172.21.15.83:0/2226417383 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa3af5bd60 con 0x55fa3af49270 2024-04-23T14:54:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.648+0000 7feb06000640 1 -- 172.21.15.83:0/2226417383 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb0c01bbe0 con 0x55fa3af49270 2024-04-23T14:54:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.648+0000 7feb06000640 1 -- 172.21.15.83:0/2226417383 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7feb0804d410 con 0x55fa3af49270 2024-04-23T14:54:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.648+0000 7feb25f93740 10 client.16251 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.648+0000 7feb25f93740 10 client.16251 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:54:44.654 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.648+0000 7feb25f93740 20 client.16251 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.648+0000 7feb25f93740 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa3afc9cf0 0x55fa3afcc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.648+0000 7feb25f93740 4 client.16251 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:54:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.648+0000 7feb25f93740 1 -- 172.21.15.83:0/2226417383 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa3aeb5a60 con 0x55fa3afc9cf0 2024-04-23T14:54:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.648+0000 7feb1be00640 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa3afc9cf0 0x55fa3afcc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb1be00640 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa3afc9cf0 0x55fa3afcc0e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f92ea7ace751a98 server_cookie=24d5d0798f7d18b4 in_seq=0 out_seq=0 2024-04-23T14:54:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb06000640 10 client.16251 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:54:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb06000640 1 -- 172.21.15.83:0/2226417383 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa3aeb5a60 con 0x55fa3afc9cf0 2024-04-23T14:54:44.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb06000640 10 client.16251 handle_command_reply: tid=0 2024-04-23T14:54:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb25f93740 1 client.16251 shutdown 2024-04-23T14:54:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb25f93740 1 -- 172.21.15.83:0/2226417383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af49270 msgr2=0x55fa3af545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb25f93740 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af49270 0x55fa3af545d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55fa3af45e00 tx=0x7feb0802a8b0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb25f93740 1 -- 172.21.15.83:0/2226417383 shutdown_connections 2024-04-23T14:54:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb25f93740 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fa3af53940 0x55fa3af54b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb25f93740 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa3af49270 0x55fa3af545d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb25f93740 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fa3afc9cf0 0x55fa3afcc0e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb25f93740 1 --2- 172.21.15.83:0/2226417383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fa3af48950 0x55fa3af54090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb25f93740 1 -- 172.21.15.83:0/2226417383 >> 172.21.15.83:0/2226417383 conn(0x55fa3af44c70 msgr2=0x55fa3af4f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb06000640 0 client.16251 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:54:44.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb25f93740 1 -- 172.21.15.83:0/2226417383 shutdown_connections 2024-04-23T14:54:44.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.649+0000 7feb25f93740 1 -- 172.21.15.83:0/2226417383 wait complete. 2024-04-23T14:54:44.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.650+0000 7feb25f93740 20 client.16251 trim_cache size 0 max 16384 2024-04-23T14:54:44.657 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:54:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.650+0000 7feb23200640 1 -- 172.21.15.83:0/2388581290 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feb000a3d20 msgr2=0x7feb000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.650+0000 7feb23200640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feb000a3d20 0x7feb000a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb080308d0 tx=0x7feb08004a40 comp rx=0 tx=0).stop 2024-04-23T14:54:44.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.650+0000 7feb23200640 1 -- 172.21.15.83:0/2388581290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb1c19a950 msgr2=0x7feb1c19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.650+0000 7feb23200640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb1c19a950 0x7feb1c19ae00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feb1400d680 tx=0x7feb1400db40 comp rx=0 tx=0).stop 2024-04-23T14:54:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.650+0000 7feb23200640 1 -- 172.21.15.83:0/2388581290 shutdown_connections 2024-04-23T14:54:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.650+0000 7feb23200640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7feb000a3d20 0x7feb000a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.650+0000 7feb23200640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb1c19a950 0x7feb1c19ae00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.650+0000 7feb23200640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb1c107310 0x7feb1c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.650+0000 7feb23200640 1 --2- 172.21.15.83:0/2388581290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb1c102240 0x7feb1c1a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.650+0000 7feb23200640 1 -- 172.21.15.83:0/2388581290 >> 172.21.15.83:0/2388581290 conn(0x7feb1c079450 msgr2=0x7feb1c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.651+0000 7feb23200640 1 -- 172.21.15.83:0/2388581290 shutdown_connections 2024-04-23T14:54:44.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.651+0000 7feb23200640 1 -- 172.21.15.83:0/2388581290 wait complete. 2024-04-23T14:54:44.666 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:44 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1466126439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:54:44.669 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:54:44.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.863+0000 7fb4b2000640 1 -- 172.21.15.83:0/1134296674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac109580 msgr2=0x7fb4ac111150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.863+0000 7fb4b2000640 1 --2- 172.21.15.83:0/1134296674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac109580 0x7fb4ac111150 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009ca0 tx=0x7fb49c02a8c0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.863+0000 7fb4b2000640 1 -- 172.21.15.83:0/1134296674 shutdown_connections 2024-04-23T14:54:44.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.863+0000 7fb4b2000640 1 --2- 172.21.15.83:0/1134296674 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac109580 0x7fb4ac111150 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.863+0000 7fb4b2000640 1 --2- 172.21.15.83:0/1134296674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4ac104e60 0x7fb4ac108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.863+0000 7fb4b2000640 1 --2- 172.21.15.83:0/1134296674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4ac1044b0 0x7fb4ac104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.863+0000 7fb4b2000640 1 -- 172.21.15.83:0/1134296674 >> 172.21.15.83:0/1134296674 conn(0x7fb4ac0ffd90 msgr2=0x7fb4ac1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.863+0000 7fb4b2000640 1 -- 172.21.15.83:0/1134296674 shutdown_connections 2024-04-23T14:54:44.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.863+0000 7fb4b2000640 1 -- 172.21.15.83:0/1134296674 wait complete. 2024-04-23T14:54:44.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.864+0000 7fb4b2000640 1 Processor -- start 2024-04-23T14:54:44.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.864+0000 7fb4b2000640 1 -- start start 2024-04-23T14:54:44.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.865+0000 7fb4b2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac1044b0 0x7fb4ac1a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.865+0000 7fb4b2000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4ac109580 0x7fb4ac1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.865+0000 7fb4b2000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4ac19a950 0x7fb4ac19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.865+0000 7fb4b2000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac114690 con 0x7fb4ac1044b0 2024-04-23T14:54:44.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.865+0000 7fb4b2000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac114510 con 0x7fb4ac19a950 2024-04-23T14:54:44.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.865+0000 7fb4b2000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac114810 con 0x7fb4ac109580 2024-04-23T14:54:44.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.865+0000 7fb4ab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac1044b0 0x7fb4ac1a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.865+0000 7fb4ab400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac1044b0 0x7fb4ac1a07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38958/0 (socket says 172.21.15.83:38958) 2024-04-23T14:54:44.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.865+0000 7fb4ab400640 1 -- 172.21.15.83:0/47409300 learned_addr learned my addr 172.21.15.83:0/47409300 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.865+0000 7fb4aaa00640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4ac109580 0x7fb4ac1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.865+0000 7fb4abe00640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4ac19a950 0x7fb4ac19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.866+0000 7fb4ab400640 1 -- 172.21.15.83:0/47409300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4ac109580 msgr2=0x7fb4ac1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.866+0000 7fb4ab400640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4ac109580 0x7fb4ac1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.866+0000 7fb4ab400640 1 -- 172.21.15.83:0/47409300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4ac19a950 msgr2=0x7fb4ac19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.866+0000 7fb4ab400640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4ac19a950 0x7fb4ac19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.866+0000 7fb4ab400640 1 -- 172.21.15.83:0/47409300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ac19b470 con 0x7fb4ac1044b0 2024-04-23T14:54:44.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.866+0000 7fb4aaa00640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4ac109580 0x7fb4ac1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:44.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.866+0000 7fb4ab400640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac1044b0 0x7fb4ac1a07a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400e9f0 tx=0x7fb4a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.866+0000 7fb4a3e00640 1 -- 172.21.15.83:0/47409300 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a400cd80 con 0x7fb4ac1044b0 2024-04-23T14:54:44.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.866+0000 7fb4a3e00640 1 -- 172.21.15.83:0/47409300 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a4005500 con 0x7fb4ac1044b0 2024-04-23T14:54:44.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.866+0000 7fb4a3e00640 1 -- 172.21.15.83:0/47409300 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4010690 con 0x7fb4ac1044b0 2024-04-23T14:54:44.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.866+0000 7fb4b2000640 1 -- 172.21.15.83:0/47409300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ac19b760 con 0x7fb4ac1044b0 2024-04-23T14:54:44.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.866+0000 7fb4b2000640 1 -- 172.21.15.83:0/47409300 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ac1a79b0 con 0x7fb4ac1044b0 2024-04-23T14:54:44.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.870+0000 7fb4b4fbe740 1 Processor -- start 2024-04-23T14:54:44.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.871+0000 7fb4b4fbe740 1 -- start start 2024-04-23T14:54:44.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.871+0000 7fb4a3e00640 1 -- 172.21.15.83:0/47409300 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb4a40046c0 con 0x7fb4ac1044b0 2024-04-23T14:54:44.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.871+0000 7fb4b4fbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 0x556ea764bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.871+0000 7fb4b4fbe740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7656400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.871+0000 7fb4b4fbe740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea7656940 0x556ea7658d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.871+0000 7fb4b4fbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ea75b11d0 con 0x556ea764b950 2024-04-23T14:54:44.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.871+0000 7fb4b4fbe740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556ea7557450 con 0x556ea764c270 2024-04-23T14:54:44.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.871+0000 7fb4b4fbe740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556ea75ad190 con 0x556ea7656940 2024-04-23T14:54:44.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.871+0000 7fb4abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 0x556ea764bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.871+0000 7fb4abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 0x556ea764bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38972/0 (socket says 172.21.15.83:38972) 2024-04-23T14:54:44.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.871+0000 7fb4abe00640 1 -- 172.21.15.83:0/3851439094 learned_addr learned my addr 172.21.15.83:0/3851439094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.871+0000 7fb4aaa00640 1 --2- 172.21.15.83:0/3851439094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea7656940 0x556ea7658d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.872+0000 7fb4ab400640 1 --2- 172.21.15.83:0/3851439094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7656400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.872+0000 7fb4abe00640 1 -- 172.21.15.83:0/3851439094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea7656940 msgr2=0x556ea7658d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.872+0000 7fb4abe00640 1 --2- 172.21.15.83:0/3851439094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea7656940 0x556ea7658d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.872+0000 7fb4abe00640 1 -- 172.21.15.83:0/3851439094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 msgr2=0x556ea7656400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.872+0000 7fb4abe00640 1 --2- 172.21.15.83:0/3851439094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7656400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.872+0000 7fb4abe00640 1 -- 172.21.15.83:0/3851439094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ea7614310 con 0x556ea764b950 2024-04-23T14:54:44.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.872+0000 7fb4ab400640 1 --2- 172.21.15.83:0/3851439094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7656400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:44.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.872+0000 7fb4abe00640 1 --2- 172.21.15.83:0/3851439094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 0x556ea764bd30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009870 tx=0x7fb49c0098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53151d5fd24e07f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.872+0000 7fb49be00640 1 -- 172.21.15.83:0/3851439094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c031db0 con 0x556ea764b950 2024-04-23T14:54:44.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.872+0000 7fb49be00640 1 -- 172.21.15.83:0/3851439094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49c036040 con 0x556ea764b950 2024-04-23T14:54:44.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.873+0000 7fb49be00640 1 -- 172.21.15.83:0/3851439094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0043c0 con 0x556ea764b950 2024-04-23T14:54:44.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.873+0000 7fb4a3e00640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb4900a3da0 0x7fb4900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.873+0000 7fb4a3e00640 1 -- 172.21.15.83:0/47409300 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb4a40d0cd0 con 0x7fb4ac1044b0 2024-04-23T14:54:44.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.873+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/3851439094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 msgr2=0x556ea764bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.873+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/3851439094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 0x556ea764bd30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009870 tx=0x7fb49c0098a0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.873+0000 7fb4ab400640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb4900a3da0 0x7fb4900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.873+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/3851439094 shutdown_connections 2024-04-23T14:54:44.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.873+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/3851439094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea7656940 0x556ea7658d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.873+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/3851439094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7656400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.873+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/3851439094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 0x556ea764bd30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.873+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/3851439094 >> 172.21.15.83:0/3851439094 conn(0x556ea7647c70 msgr2=0x556ea7648050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.874+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/3851439094 shutdown_connections 2024-04-23T14:54:44.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.874+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/3851439094 wait complete. 2024-04-23T14:54:44.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.874+0000 7fb4ab400640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb4900a3da0 0x7fb4900a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40ce4e0 tx=0x7fb4a4004a40 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.874+0000 7fb4b4fbe740 1 -- start start 2024-04-23T14:54:44.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.874+0000 7fb4b4fbe740 1 Processor -- start 2024-04-23T14:54:44.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.875+0000 7fb4b4fbe740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764b950 0x556ea7656390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.875+0000 7fb4b4fbe740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7653680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.875+0000 7fb4b4fbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea7656940 0x556ea7653d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.875+0000 7fb4b4fbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556ea7659770 con 0x556ea7656940 2024-04-23T14:54:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.875+0000 7fb4b4fbe740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x556ea756a810 con 0x556ea764c270 2024-04-23T14:54:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.875+0000 7fb4b4fbe740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x556ea76598f0 con 0x556ea764b950 2024-04-23T14:54:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.875+0000 7fb4b4fbe740 10 client.0 _ll_register_callbacks cb 0x556ea7538140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.875+0000 7fb4aaa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7653680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.875+0000 7fb4aaa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7653680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37420/0 (socket says 172.21.15.83:37420) 2024-04-23T14:54:44.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.875+0000 7fb4aaa00640 1 -- 172.21.15.83:0/1671603601 learned_addr learned my addr 172.21.15.83:0/1671603601 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.875+0000 7fb4abe00640 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea7656940 0x556ea7653d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.875+0000 7fb4abe00640 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764b950 0x556ea7656390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.876+0000 7fb4aaa00640 1 -- 172.21.15.83:0/1671603601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764b950 msgr2=0x556ea7656390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.876+0000 7fb4aaa00640 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764b950 0x556ea7656390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.876+0000 7fb4aaa00640 1 -- 172.21.15.83:0/1671603601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea7656940 msgr2=0x556ea7653d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.876+0000 7fb4aaa00640 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea7656940 0x556ea7653d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.876+0000 7fb4aaa00640 1 -- 172.21.15.83:0/1671603601 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ea757b3f0 con 0x556ea764c270 2024-04-23T14:54:44.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.876+0000 7fb4abe00640 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764b950 0x556ea7656390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.876+0000 7fb4aaa00640 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7653680 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb494012700 tx=0x7fb494012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8d441c94b35702 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.876+0000 7fb498c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:54:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.876+0000 7fb498c00640 1 -- 172.21.15.83:0/1671603601 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494004320 con 0x556ea764c270 2024-04-23T14:54:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.876+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/1671603601 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49c036040 con 0x556ea764c270 2024-04-23T14:54:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.876+0000 7fb498c00640 1 -- 172.21.15.83:0/1671603601 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb4940119a0 con 0x556ea764c270 2024-04-23T14:54:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.876+0000 7fb498c00640 1 -- 172.21.15.83:0/1671603601 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4940044c0 con 0x556ea764c270 2024-04-23T14:54:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.877+0000 7fb4b4fbe740 10 client.26471 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.877+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/1671603601 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556ea72c33f0 con 0x556ea764c270 2024-04-23T14:54:44.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.877+0000 7fb498c00640 1 -- 172.21.15.83:0/1671603601 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49c036040 con 0x556ea764c270 2024-04-23T14:54:44.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.877+0000 7fb498c00640 1 -- 172.21.15.83:0/1671603601 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fb494025470 con 0x556ea764c270 2024-04-23T14:54:44.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.877+0000 7fb4b4fbe740 10 client.26471 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:44.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.877+0000 7fb4b4fbe740 10 client.26471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:44.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.877+0000 7fb4b4fbe740 20 client.26471 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:44.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.877+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ea7666ac0 0x556ea7668eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.877+0000 7fb4b4fbe740 4 client.26471 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:54:44.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.877+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/1671603601 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ea75bebf0 con 0x556ea7666ac0 2024-04-23T14:54:44.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.878+0000 7fb4abe00640 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ea7666ac0 0x556ea7668eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.878+0000 7fb4abe00640 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ea7666ac0 0x556ea7668eb0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=758338de9e0a0c99 server_cookie=b579eb4458a4d6a4 in_seq=0 out_seq=0 2024-04-23T14:54:44.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.878+0000 7fb498c00640 10 client.26471 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:44.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb498c00640 1 -- 172.21.15.83:0/1671603601 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ea75bebf0 con 0x556ea7666ac0 2024-04-23T14:54:44.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb498c00640 10 client.26471 handle_command_reply: tid=0 2024-04-23T14:54:44.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb4b4fbe740 1 client.26471 shutdown 2024-04-23T14:54:44.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/1671603601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 msgr2=0x556ea7653680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7653680 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb494012700 tx=0x7fb494012bc0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/1671603601 shutdown_connections 2024-04-23T14:54:44.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ea7666ac0 0x556ea7668eb0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea7656940 0x556ea7653d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7653680 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/1671603601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764b950 0x556ea7656390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb498c00640 0 client.26471 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/1671603601 >> 172.21.15.83:0/1671603601 conn(0x556ea7647c70 msgr2=0x556ea764d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/1671603601 shutdown_connections 2024-04-23T14:54:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/1671603601 wait complete. 2024-04-23T14:54:44.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.879+0000 7fb4b4fbe740 20 client.26471 trim_cache size 0 max 16384 2024-04-23T14:54:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.894+0000 7fb4b4fbe740 1 Processor -- start 2024-04-23T14:54:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.894+0000 7fb4b4fbe740 1 -- start start 2024-04-23T14:54:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.894+0000 7fb4b4fbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 0x556ea7662630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.894+0000 7fb4b4fbe740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764c270 0x556ea7647610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.894+0000 7fb4b4fbe740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea7656940 0x556ea7652660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.894+0000 7fb4b4fbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1b28b0 con 0x556ea764b950 2024-04-23T14:54:44.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.894+0000 7fb4b4fbe740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1b0180 con 0x556ea7656940 2024-04-23T14:54:44.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.894+0000 7fb4b4fbe740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1adfd0 con 0x556ea764c270 2024-04-23T14:54:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.895+0000 7fb4abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 0x556ea7662630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.895+0000 7fb4aaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764c270 0x556ea7647610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.895+0000 7fb4abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 0x556ea7662630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38990/0 (socket says 172.21.15.83:38990) 2024-04-23T14:54:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.895+0000 7fb4abe00640 1 -- 172.21.15.83:0/2013283659 learned_addr learned my addr 172.21.15.83:0/2013283659 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.895+0000 7fb4abe00640 1 --2- 172.21.15.83:0/2013283659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea7656940 0x556ea7652660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.895+0000 7fb4abe00640 1 -- 172.21.15.83:0/2013283659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764c270 msgr2=0x556ea7647610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.895+0000 7fb4abe00640 1 --2- 172.21.15.83:0/2013283659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764c270 0x556ea7647610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.895+0000 7fb4abe00640 1 -- 172.21.15.83:0/2013283659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea7656940 msgr2=0x556ea7652660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.895+0000 7fb4abe00640 1 --2- 172.21.15.83:0/2013283659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea7656940 0x556ea7652660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.895+0000 7fb4abe00640 1 -- 172.21.15.83:0/2013283659 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ea738d7d0 con 0x556ea764b950 2024-04-23T14:54:44.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.896+0000 7fb4abe00640 1 --2- 172.21.15.83:0/2013283659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 0x556ea7662630 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb49c02ade0 tx=0x7fb49c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f50936508df4ab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.896+0000 7fb49b400640 1 -- 172.21.15.83:0/2013283659 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c031bb0 con 0x556ea764b950 2024-04-23T14:54:44.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.896+0000 7fb49b400640 1 -- 172.21.15.83:0/2013283659 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49c036070 con 0x556ea764b950 2024-04-23T14:54:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.896+0000 7fb49b400640 1 -- 172.21.15.83:0/2013283659 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c058790 con 0x556ea764b950 2024-04-23T14:54:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.896+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2013283659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 msgr2=0x556ea7662630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.896+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/2013283659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 0x556ea7662630 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb49c02ade0 tx=0x7fb49c009800 comp rx=0 tx=0).stop 2024-04-23T14:54:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.897+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2013283659 shutdown_connections 2024-04-23T14:54:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.897+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/2013283659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea7656940 0x556ea7652660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.897+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/2013283659 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764c270 0x556ea7647610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.897+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/2013283659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea764b950 0x556ea7662630 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.897+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2013283659 >> 172.21.15.83:0/2013283659 conn(0x556ea7647c70 msgr2=0x556ea7651f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.897+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2013283659 shutdown_connections 2024-04-23T14:54:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.897+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2013283659 wait complete. 2024-04-23T14:54:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.897+0000 7fb4b4fbe740 1 -- start start 2024-04-23T14:54:44.901 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.897+0000 7fb4b4fbe740 1 Processor -- start 2024-04-23T14:54:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4b4fbe740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764b950 0x556ea7657670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4b4fbe740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7657bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4b4fbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea7656940 0x556ea76580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4b4fbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1ae640 con 0x556ea7656940 2024-04-23T14:54:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4b4fbe740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1afab0 con 0x556ea764c270 2024-04-23T14:54:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4b4fbe740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1aed10 con 0x556ea764b950 2024-04-23T14:54:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4b4fbe740 10 client.0 _ll_register_callbacks cb 0x7fb4940097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4aaa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7657bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4abe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764b950 0x556ea7657670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4aaa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7657bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37432/0 (socket says 172.21.15.83:37432) 2024-04-23T14:54:44.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4aaa00640 1 -- 172.21.15.83:0/2732967554 learned_addr learned my addr 172.21.15.83:0/2732967554 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4abe00640 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea7656940 0x556ea76580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4aaa00640 1 -- 172.21.15.83:0/2732967554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764b950 msgr2=0x556ea7657670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4aaa00640 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764b950 0x556ea7657670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4aaa00640 1 -- 172.21.15.83:0/2732967554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea7656940 msgr2=0x556ea76580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4aaa00640 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea7656940 0x556ea76580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4aaa00640 1 -- 172.21.15.83:0/2732967554 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ea765ccf0 con 0x556ea764c270 2024-04-23T14:54:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.898+0000 7fb4abe00640 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea7656940 0x556ea76580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.899+0000 7fb4aaa00640 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7657bb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb494005eb0 tx=0x7fb494025ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d90ed941578bef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:44.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.899+0000 7fb498c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:54:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.899+0000 7fb498c00640 1 -- 172.21.15.83:0/2732967554 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49402b040 con 0x556ea764c270 2024-04-23T14:54:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.899+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2732967554 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49c036070 con 0x556ea764c270 2024-04-23T14:54:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.899+0000 7fb498c00640 1 -- 172.21.15.83:0/2732967554 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb494009d30 con 0x556ea764c270 2024-04-23T14:54:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.899+0000 7fb498c00640 1 -- 172.21.15.83:0/2732967554 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb494009600 con 0x556ea764c270 2024-04-23T14:54:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.899+0000 7fb4b4fbe740 10 client.26477 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.899+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2732967554 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556ea76cd300 con 0x556ea764c270 2024-04-23T14:54:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.900+0000 7fb498c00640 1 -- 172.21.15.83:0/2732967554 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49c036070 con 0x556ea764c270 2024-04-23T14:54:44.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.900+0000 7fb498c00640 1 -- 172.21.15.83:0/2732967554 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fb49402e070 con 0x556ea764c270 2024-04-23T14:54:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.900+0000 7fb4b4fbe740 10 client.26477 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.900+0000 7fb4b4fbe740 10 client.26477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.900+0000 7fb4b4fbe740 20 client.26477 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.900+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ea76cdaa0 0x556ea76cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.900+0000 7fb4b4fbe740 4 client.26477 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:54:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.900+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2732967554 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ea75b8a60 con 0x556ea76cdaa0 2024-04-23T14:54:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.900+0000 7fb4abe00640 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ea76cdaa0 0x556ea76cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.900+0000 7fb4abe00640 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ea76cdaa0 0x556ea76cfec0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c67cbbf8b441db6b server_cookie=286080488d8adb2c in_seq=0 out_seq=0 2024-04-23T14:54:44.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.900+0000 7fb498c00640 10 client.26477 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:44.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb498c00640 1 -- 172.21.15.83:0/2732967554 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ea75b8a60 con 0x556ea76cdaa0 2024-04-23T14:54:44.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb498c00640 10 client.26477 handle_command_reply: tid=0 2024-04-23T14:54:44.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb4b4fbe740 1 client.26477 shutdown 2024-04-23T14:54:44.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2732967554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 msgr2=0x556ea7657bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7657bb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb494005eb0 tx=0x7fb494025ce0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2732967554 shutdown_connections 2024-04-23T14:54:44.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x556ea76cdaa0 0x556ea76cfec0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556ea7656940 0x556ea76580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556ea764c270 0x556ea7657bb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb498c00640 0 client.26477 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:44.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb4b4fbe740 1 --2- 172.21.15.83:0/2732967554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556ea764b950 0x556ea7657670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2732967554 >> 172.21.15.83:0/2732967554 conn(0x556ea7647c70 msgr2=0x556ea7646560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2732967554 shutdown_connections 2024-04-23T14:54:44.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb4b4fbe740 1 -- 172.21.15.83:0/2732967554 wait complete. 2024-04-23T14:54:44.910 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.901+0000 7fb4b4fbe740 20 client.26477 trim_cache size 0 max 16384 2024-04-23T14:54:44.911 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:54:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.902+0000 7fb4b2000640 1 -- 172.21.15.83:0/47409300 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb4900a3da0 msgr2=0x7fb4900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.902+0000 7fb4b2000640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb4900a3da0 0x7fb4900a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40ce4e0 tx=0x7fb4a4004a40 comp rx=0 tx=0).stop 2024-04-23T14:54:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.902+0000 7fb4b2000640 1 -- 172.21.15.83:0/47409300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac1044b0 msgr2=0x7fb4ac1a07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.902+0000 7fb4b2000640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac1044b0 0x7fb4ac1a07a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400e9f0 tx=0x7fb4a400edf0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.902+0000 7fb4b2000640 1 -- 172.21.15.83:0/47409300 shutdown_connections 2024-04-23T14:54:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.902+0000 7fb4b2000640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4ac19a950 0x7fb4ac19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.902+0000 7fb4b2000640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4ac109580 0x7fb4ac1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.902+0000 7fb4b2000640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb4900a3da0 0x7fb4900a6260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.902+0000 7fb4b2000640 1 --2- 172.21.15.83:0/47409300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4ac1044b0 0x7fb4ac1a07a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:44.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.902+0000 7fb4b2000640 1 -- 172.21.15.83:0/47409300 >> 172.21.15.83:0/47409300 conn(0x7fb4ac0ffd90 msgr2=0x7fb4ac102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:44.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.902+0000 7fb4b2000640 1 -- 172.21.15.83:0/47409300 shutdown_connections 2024-04-23T14:54:44.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:44.902+0000 7fb4b2000640 1 -- 172.21.15.83:0/47409300 wait complete. 2024-04-23T14:54:44.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:44 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1466126439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:54:45.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:44 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1466126439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:54:45.921 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:54:45.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:54:45.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:54:45.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:45 smithi203 ceph-mon[32800]: pgmap v192: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:54:45.996 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:45 smithi083 ceph-mon[27867]: pgmap v192: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:54:46.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:45 smithi173 ceph-mon[32443]: pgmap v192: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:54:46.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.135+0000 7f99dac00640 1 -- 172.21.15.83:0/166463378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99d4108610 msgr2=0x7f99d4108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.135+0000 7f99dac00640 1 --2- 172.21.15.83:0/166463378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99d4108610 0x7f99d4108a90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009860 tx=0x7f99bc02a340 comp rx=0 tx=0).stop 2024-04-23T14:54:46.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.136+0000 7f99dac00640 1 -- 172.21.15.83:0/166463378 shutdown_connections 2024-04-23T14:54:46.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.136+0000 7f99dac00640 1 --2- 172.21.15.83:0/166463378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99d4108fd0 0x7f99d4115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.137 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.136+0000 7f99dac00640 1 --2- 172.21.15.83:0/166463378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99d4108610 0x7f99d4108a90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.136+0000 7f99dac00640 1 --2- 172.21.15.83:0/166463378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d410e660 0x7f99d410ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.136+0000 7f99dac00640 1 -- 172.21.15.83:0/166463378 >> 172.21.15.83:0/166463378 conn(0x7f99d4076540 msgr2=0x7f99d4078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:46.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.136+0000 7f99dac00640 1 -- 172.21.15.83:0/166463378 shutdown_connections 2024-04-23T14:54:46.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.136+0000 7f99dac00640 1 -- 172.21.15.83:0/166463378 wait complete. 2024-04-23T14:54:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.137+0000 7f99dac00640 1 Processor -- start 2024-04-23T14:54:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.137+0000 7f99dac00640 1 -- start start 2024-04-23T14:54:46.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.137+0000 7f99dac00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99d4108610 0x7f99d419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99dac00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99d4108fd0 0x7f99d419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99dac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d410e660 0x7f99d41a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99dac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99d41189a0 con 0x7f99d410e660 2024-04-23T14:54:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99dac00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f99d4118820 con 0x7f99d4108610 2024-04-23T14:54:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99dac00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99d4118b20 con 0x7f99d4108fd0 2024-04-23T14:54:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99d8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d410e660 0x7f99d41a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99d4108610 0x7f99d419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99d4108610 0x7f99d419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37446/0 (socket says 172.21.15.83:37446) 2024-04-23T14:54:46.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99cbe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99d4108fd0 0x7f99d419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99d3e00640 1 -- 172.21.15.83:0/3364925866 learned_addr learned my addr 172.21.15.83:0/3364925866 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:46.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99d8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d410e660 0x7f99d41a37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39020/0 (socket says 172.21.15.83:39020) 2024-04-23T14:54:46.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99cbe00640 1 -- 172.21.15.83:0/3364925866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99d4108610 msgr2=0x7f99d419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99cbe00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99d4108610 0x7f99d419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99cbe00640 1 -- 172.21.15.83:0/3364925866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d410e660 msgr2=0x7f99d41a37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99cbe00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d410e660 0x7f99d41a37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99cbe00640 1 -- 172.21.15.83:0/3364925866 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d41a3ed0 con 0x7f99d4108fd0 2024-04-23T14:54:46.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99d3e00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99d4108610 0x7f99d419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:46.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.138+0000 7f99d8c00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d410e660 0x7f99d41a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:46.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.139+0000 7f99cbe00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99d4108fd0 0x7f99d419f420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f99bc02a860 tx=0x7f99bc033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:46.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.139+0000 7f99d1600640 1 -- 172.21.15.83:0/3364925866 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc037040 con 0x7f99d4108fd0 2024-04-23T14:54:46.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.139+0000 7f99d1600640 1 -- 172.21.15.83:0/3364925866 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99bc033d50 con 0x7f99d4108fd0 2024-04-23T14:54:46.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.139+0000 7f99dac00640 1 -- 172.21.15.83:0/3364925866 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d41a4160 con 0x7f99d4108fd0 2024-04-23T14:54:46.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.139+0000 7f99d1600640 1 -- 172.21.15.83:0/3364925866 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0039e0 con 0x7f99d4108fd0 2024-04-23T14:54:46.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.139+0000 7f99dac00640 1 -- 172.21.15.83:0/3364925866 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d41aba50 con 0x7f99d4108fd0 2024-04-23T14:54:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.142+0000 7f99ddb6b740 1 Processor -- start 2024-04-23T14:54:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.142+0000 7f99d1600640 1 -- 172.21.15.83:0/3364925866 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f99bc042020 con 0x7f99d4108fd0 2024-04-23T14:54:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.143+0000 7f99ddb6b740 1 -- start start 2024-04-23T14:54:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.143+0000 7f99ddb6b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 0x55ee6def5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.143+0000 7f99ddb6b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6deffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.143+0000 7f99ddb6b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6df00080 0x55ee6df02470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.143+0000 7f99ddb6b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee6de5ae70 con 0x55ee6def59b0 2024-04-23T14:54:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.143+0000 7f99ddb6b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ee6de01380 con 0x55ee6df00080 2024-04-23T14:54:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.143+0000 7f99ddb6b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ee6de56e30 con 0x55ee6def5090 2024-04-23T14:54:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.143+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6deffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.143+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6deffb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39028/0 (socket says 172.21.15.83:39028) 2024-04-23T14:54:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.143+0000 7f99d3e00640 1 -- 172.21.15.83:0/2203752237 learned_addr learned my addr 172.21.15.83:0/2203752237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.143+0000 7f99d8c00640 1 --2- 172.21.15.83:0/2203752237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 0x55ee6def5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.143+0000 7f99cbe00640 1 --2- 172.21.15.83:0/2203752237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6df00080 0x55ee6df02470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.144+0000 7f99d3e00640 1 -- 172.21.15.83:0/2203752237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 msgr2=0x55ee6def5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.144+0000 7f99d3e00640 1 --2- 172.21.15.83:0/2203752237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 0x55ee6def5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.144+0000 7f99d3e00640 1 -- 172.21.15.83:0/2203752237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6df00080 msgr2=0x55ee6df02470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.144+0000 7f99d3e00640 1 --2- 172.21.15.83:0/2203752237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6df00080 0x55ee6df02470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.144+0000 7f99d3e00640 1 -- 172.21.15.83:0/2203752237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6dc357e0 con 0x55ee6def59b0 2024-04-23T14:54:46.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.144+0000 7f99cbe00640 1 --2- 172.21.15.83:0/2203752237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6df00080 0x55ee6df02470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.144+0000 7f99d3e00640 1 --2- 172.21.15.83:0/2203752237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6deffb40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f99c4011540 tx=0x7f99c4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a21b516ed6ad2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.144+0000 7f99c9200640 1 -- 172.21.15.83:0/2203752237 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c400f640 con 0x55ee6def59b0 2024-04-23T14:54:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.144+0000 7f99c9200640 1 -- 172.21.15.83:0/2203752237 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c400fcc0 con 0x55ee6def59b0 2024-04-23T14:54:46.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.144+0000 7f99c9200640 1 -- 172.21.15.83:0/2203752237 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4019de0 con 0x55ee6def59b0 2024-04-23T14:54:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.144+0000 7f99d1600640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f99b80a3d20 0x7f99b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.145+0000 7f99d1600640 1 -- 172.21.15.83:0/3364925866 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f99bc0efcd0 con 0x7f99d4108fd0 2024-04-23T14:54:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.145+0000 7f99d8c00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f99b80a3d20 0x7f99b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.145+0000 7f99ddb6b740 1 -- 172.21.15.83:0/2203752237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 msgr2=0x55ee6deffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.145+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/2203752237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6deffb40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f99c4011540 tx=0x7f99c4011a00 comp rx=0 tx=0).stop 2024-04-23T14:54:46.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.145+0000 7f99ddb6b740 1 -- 172.21.15.83:0/2203752237 shutdown_connections 2024-04-23T14:54:46.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.145+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/2203752237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6df00080 0x55ee6df02470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.145+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/2203752237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6deffb40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.145+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/2203752237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 0x55ee6def5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.145+0000 7f99ddb6b740 1 -- 172.21.15.83:0/2203752237 >> 172.21.15.83:0/2203752237 conn(0x55ee6def13b0 msgr2=0x55ee6def1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.145+0000 7f99ddb6b740 1 -- 172.21.15.83:0/2203752237 shutdown_connections 2024-04-23T14:54:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.145+0000 7f99d8c00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f99b80a3d20 0x7f99b80a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009fd0 tx=0x7f99cc009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:46.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.145+0000 7f99ddb6b740 1 -- 172.21.15.83:0/2203752237 wait complete. 2024-04-23T14:54:46.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.146+0000 7f99ddb6b740 1 -- start start 2024-04-23T14:54:46.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.146+0000 7f99ddb6b740 1 Processor -- start 2024-04-23T14:54:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.146+0000 7f99ddb6b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 0x55ee6defcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.146+0000 7f99ddb6b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6defd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.146+0000 7f99ddb6b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6df00080 0x55ee6defda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99ddb6b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee6df02e70 con 0x55ee6def59b0 2024-04-23T14:54:46.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99ddb6b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ee6de14660 con 0x55ee6df00080 2024-04-23T14:54:46.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99ddb6b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ee6df02ff0 con 0x55ee6def5090 2024-04-23T14:54:46.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99ddb6b740 10 client.0 _ll_register_callbacks cb 0x55ee6de02aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:46.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6defd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6defd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39032/0 (socket says 172.21.15.83:39032) 2024-04-23T14:54:46.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99cbe00640 1 -- 172.21.15.83:0/1738590445 learned_addr learned my addr 172.21.15.83:0/1738590445 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99d3e00640 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 0x55ee6defcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99d8c00640 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6df00080 0x55ee6defda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99cbe00640 1 -- 172.21.15.83:0/1738590445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 msgr2=0x55ee6defcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99cbe00640 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 0x55ee6defcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99cbe00640 1 -- 172.21.15.83:0/1738590445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6df00080 msgr2=0x55ee6defda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99cbe00640 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6df00080 0x55ee6defda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99cbe00640 1 -- 172.21.15.83:0/1738590445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6dc95f70 con 0x55ee6def59b0 2024-04-23T14:54:46.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.147+0000 7f99d8c00640 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6df00080 0x55ee6defda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:46.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.148+0000 7f99cbe00640 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6defd500 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0ed540 tx=0x7f99bc0f3540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eff78cb50aac76d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:46.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.148+0000 7f99c1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:54:46.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.148+0000 7f99c1600640 1 -- 172.21.15.83:0/1738590445 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0bb690 con 0x55ee6def59b0 2024-04-23T14:54:46.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.148+0000 7f99c1600640 1 -- 172.21.15.83:0/1738590445 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f99bc0c4510 con 0x55ee6def59b0 2024-04-23T14:54:46.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.148+0000 7f99ddb6b740 1 -- 172.21.15.83:0/1738590445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c400fcc0 con 0x55ee6def59b0 2024-04-23T14:54:46.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.148+0000 7f99c1600640 1 -- 172.21.15.83:0/1738590445 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99bc0ed850 con 0x55ee6def59b0 2024-04-23T14:54:46.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.148+0000 7f99ddb6b740 10 client.16281 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:46.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.148+0000 7f99ddb6b740 1 -- 172.21.15.83:0/1738590445 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee6db2d170 con 0x55ee6def59b0 2024-04-23T14:54:46.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.148+0000 7f99c1600640 1 -- 172.21.15.83:0/1738590445 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c400fcc0 con 0x55ee6def59b0 2024-04-23T14:54:46.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.148+0000 7f99c1600640 1 -- 172.21.15.83:0/1738590445 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f99bc03a070 con 0x55ee6def59b0 2024-04-23T14:54:46.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.149+0000 7f99ddb6b740 10 client.16281 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:46.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.149+0000 7f99ddb6b740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:46.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.149+0000 7f99ddb6b740 20 client.16281 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:46.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.149+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ee6df102a0 0x55ee6df12710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.149+0000 7f99ddb6b740 4 client.16281 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:54:46.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.149+0000 7f99ddb6b740 1 -- 172.21.15.83:0/1738590445 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee6de68890 con 0x55ee6df102a0 2024-04-23T14:54:46.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.149+0000 7f99d3e00640 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ee6df102a0 0x55ee6df12710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.149+0000 7f99d3e00640 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ee6df102a0 0x55ee6df12710 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be63417fd3383f7b server_cookie=664a22cb013da0d9 in_seq=0 out_seq=0 2024-04-23T14:54:46.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.149+0000 7f99c1600640 10 client.16281 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:46.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.150+0000 7f99c1600640 1 -- 172.21.15.83:0/1738590445 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee6de68890 con 0x55ee6df102a0 2024-04-23T14:54:46.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.150+0000 7f99c1600640 10 client.16281 handle_command_reply: tid=0 2024-04-23T14:54:46.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.150+0000 7f99ddb6b740 1 client.16281 shutdown 2024-04-23T14:54:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.150+0000 7f99ddb6b740 1 -- 172.21.15.83:0/1738590445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 msgr2=0x55ee6defd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.150+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6defd500 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0ed540 tx=0x7f99bc0f3540 comp rx=0 tx=0).stop 2024-04-23T14:54:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.151+0000 7f99ddb6b740 1 -- 172.21.15.83:0/1738590445 shutdown_connections 2024-04-23T14:54:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.151+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ee6df102a0 0x55ee6df12710 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.151+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6df00080 0x55ee6defda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.151+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6defd500 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.151+0000 7f99c1600640 0 client.16281 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:46.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.151+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/1738590445 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 0x55ee6defcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.151+0000 7f99ddb6b740 1 -- 172.21.15.83:0/1738590445 >> 172.21.15.83:0/1738590445 conn(0x55ee6def13b0 msgr2=0x55ee6def0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:46.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.151+0000 7f99ddb6b740 1 -- 172.21.15.83:0/1738590445 shutdown_connections 2024-04-23T14:54:46.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.151+0000 7f99ddb6b740 1 -- 172.21.15.83:0/1738590445 wait complete. 2024-04-23T14:54:46.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.151+0000 7f99ddb6b740 20 client.16281 trim_cache size 0 max 16384 2024-04-23T14:54:46.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.170+0000 7f99ddb6b740 1 Processor -- start 2024-04-23T14:54:46.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.170+0000 7f99ddb6b740 1 -- start start 2024-04-23T14:54:46.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.171+0000 7f99ddb6b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 0x55ee6df0bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.171+0000 7f99ddb6b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def59b0 0x55ee6def0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.171+0000 7f99ddb6b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6df00080 0x55ee6df18020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.171+0000 7f99ddb6b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99d41b6c00 con 0x55ee6df00080 2024-04-23T14:54:46.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.171+0000 7f99ddb6b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f99d41b44d0 con 0x55ee6def59b0 2024-04-23T14:54:46.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.171+0000 7f99ddb6b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99d41b2320 con 0x55ee6def5090 2024-04-23T14:54:46.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.171+0000 7f99cbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def59b0 0x55ee6def0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.171+0000 7f99d8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6df00080 0x55ee6df18020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.171+0000 7f99cbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def59b0 0x55ee6def0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37474/0 (socket says 172.21.15.83:37474) 2024-04-23T14:54:46.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.171+0000 7f99cbe00640 1 -- 172.21.15.83:0/2201925286 learned_addr learned my addr 172.21.15.83:0/2201925286 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:46.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.171+0000 7f99d8c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6df00080 0x55ee6df18020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39038/0 (socket says 172.21.15.83:39038) 2024-04-23T14:54:46.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.171+0000 7f99d3e00640 1 --2- 172.21.15.83:0/2201925286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 0x55ee6df0bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.172+0000 7f99cbe00640 1 -- 172.21.15.83:0/2201925286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 msgr2=0x55ee6df0bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.172+0000 7f99cbe00640 1 --2- 172.21.15.83:0/2201925286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 0x55ee6df0bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.172+0000 7f99cbe00640 1 -- 172.21.15.83:0/2201925286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6df00080 msgr2=0x55ee6df18020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.172+0000 7f99cbe00640 1 --2- 172.21.15.83:0/2201925286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6df00080 0x55ee6df18020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.172+0000 7f99cbe00640 1 -- 172.21.15.83:0/2201925286 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6defbf70 con 0x55ee6def59b0 2024-04-23T14:54:46.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.172+0000 7f99cbe00640 1 --2- 172.21.15.83:0/2201925286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def59b0 0x55ee6def0390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0b9fe0 tx=0x7f99bc004510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7997a10bbd3f7670 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:46.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.172+0000 7f99c3e00640 1 -- 172.21.15.83:0/2201925286 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0bbb00 con 0x55ee6def59b0 2024-04-23T14:54:46.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.172+0000 7f99c3e00640 1 -- 172.21.15.83:0/2201925286 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99bc0bbd30 con 0x55ee6def59b0 2024-04-23T14:54:46.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.172+0000 7f99c3e00640 1 -- 172.21.15.83:0/2201925286 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0ed820 con 0x55ee6def59b0 2024-04-23T14:54:46.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.173+0000 7f99ddb6b740 1 -- 172.21.15.83:0/2201925286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def59b0 msgr2=0x55ee6def0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.173+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/2201925286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def59b0 0x55ee6def0390 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0b9fe0 tx=0x7f99bc004510 comp rx=0 tx=0).stop 2024-04-23T14:54:46.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.173+0000 7f99ddb6b740 1 -- 172.21.15.83:0/2201925286 shutdown_connections 2024-04-23T14:54:46.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.173+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/2201925286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6df00080 0x55ee6df18020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.173+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/2201925286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def59b0 0x55ee6def0390 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.173+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/2201925286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6def5090 0x55ee6df0bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.173+0000 7f99ddb6b740 1 -- 172.21.15.83:0/2201925286 >> 172.21.15.83:0/2201925286 conn(0x55ee6def13b0 msgr2=0x55ee6deefc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:46.178 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.173+0000 7f99ddb6b740 1 -- 172.21.15.83:0/2201925286 shutdown_connections 2024-04-23T14:54:46.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.173+0000 7f99ddb6b740 1 -- 172.21.15.83:0/2201925286 wait complete. 2024-04-23T14:54:46.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.173+0000 7f99ddb6b740 1 -- start start 2024-04-23T14:54:46.179 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.174+0000 7f99ddb6b740 1 Processor -- start 2024-04-23T14:54:46.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.174+0000 7f99ddb6b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def5090 0x55ee6df082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.174+0000 7f99ddb6b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6df087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.174+0000 7f99ddb6b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6df00080 0x55ee6df05270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.174+0000 7f99ddb6b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99d41b2990 con 0x55ee6def59b0 2024-04-23T14:54:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.174+0000 7f99ddb6b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f99d41b3e00 con 0x55ee6def5090 2024-04-23T14:54:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.174+0000 7f99ddb6b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99d41b3060 con 0x55ee6df00080 2024-04-23T14:54:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.174+0000 7f99ddb6b740 10 client.0 _ll_register_callbacks cb 0x7f99bc005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:46.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.174+0000 7f99cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6df087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.174+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def5090 0x55ee6df082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.174+0000 7f99cbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6df087e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39042/0 (socket says 172.21.15.83:39042) 2024-04-23T14:54:46.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.174+0000 7f99cbe00640 1 -- 172.21.15.83:0/4102992403 learned_addr learned my addr 172.21.15.83:0/4102992403 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:46.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99d8c00640 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6df00080 0x55ee6df05270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99d3e00640 1 -- 172.21.15.83:0/4102992403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6df00080 msgr2=0x55ee6df05270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99d3e00640 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6df00080 0x55ee6df05270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99d3e00640 1 -- 172.21.15.83:0/4102992403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 msgr2=0x55ee6df087e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99d3e00640 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6df087e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99d3e00640 1 -- 172.21.15.83:0/4102992403 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6df05a70 con 0x55ee6def5090 2024-04-23T14:54:46.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99cbe00640 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6df087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:46.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99d8c00640 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6df00080 0x55ee6df05270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:54:46.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99d3e00640 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def5090 0x55ee6df082a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f99c4011f00 tx=0x7f99c4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a61d7f819cc8d784 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:46.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99c1600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:54:46.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99c1600640 1 -- 172.21.15.83:0/4102992403 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c4011020 con 0x55ee6def5090 2024-04-23T14:54:46.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99ddb6b740 1 -- 172.21.15.83:0/4102992403 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99bc0bbd30 con 0x55ee6def5090 2024-04-23T14:54:46.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.175+0000 7f99c1600640 1 -- 172.21.15.83:0/4102992403 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f99c40111c0 con 0x55ee6def5090 2024-04-23T14:54:46.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.176+0000 7f99c1600640 1 -- 172.21.15.83:0/4102992403 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99c4037050 con 0x55ee6def5090 2024-04-23T14:54:46.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.176+0000 7f99ddb6b740 10 client.26495 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:46.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.176+0000 7f99ddb6b740 1 -- 172.21.15.83:0/4102992403 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ee6df77340 con 0x55ee6def5090 2024-04-23T14:54:46.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.176+0000 7f99c1600640 1 -- 172.21.15.83:0/4102992403 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99bc0bbd30 con 0x55ee6def5090 2024-04-23T14:54:46.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.176+0000 7f99c1600640 1 -- 172.21.15.83:0/4102992403 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f99c40291e0 con 0x55ee6def5090 2024-04-23T14:54:46.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.176+0000 7f99ddb6b740 10 client.26495 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:46.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.176+0000 7f99ddb6b740 10 client.26495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:46.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.176+0000 7f99ddb6b740 20 client.26495 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:46.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.176+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ee6df77920 0x55ee6df79d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.176+0000 7f99ddb6b740 4 client.26495 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:54:46.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.176+0000 7f99ddb6b740 1 -- 172.21.15.83:0/4102992403 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ee6de62700 con 0x55ee6df77920 2024-04-23T14:54:46.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.177+0000 7f99cbe00640 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ee6df77920 0x55ee6df79d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.177+0000 7f99cbe00640 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ee6df77920 0x55ee6df79d10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5492da7c98423ad6 server_cookie=426a7468ac586a6e in_seq=0 out_seq=0 2024-04-23T14:54:46.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.177+0000 7f99c1600640 10 client.26495 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:46.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.178+0000 7f99c1600640 1 -- 172.21.15.83:0/4102992403 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ee6de62700 con 0x55ee6df77920 2024-04-23T14:54:46.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.178+0000 7f99c1600640 10 client.26495 handle_command_reply: tid=0 2024-04-23T14:54:46.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.178+0000 7f99ddb6b740 1 client.26495 shutdown 2024-04-23T14:54:46.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.178+0000 7f99ddb6b740 1 -- 172.21.15.83:0/4102992403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def5090 msgr2=0x55ee6df082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.178+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def5090 0x55ee6df082a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f99c4011f00 tx=0x7f99c4009260 comp rx=0 tx=0).stop 2024-04-23T14:54:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.178+0000 7f99ddb6b740 1 -- 172.21.15.83:0/4102992403 shutdown_connections 2024-04-23T14:54:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.178+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55ee6df77920 0x55ee6df79d10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.178+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ee6df00080 0x55ee6df05270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.178+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee6def59b0 0x55ee6df087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.178+0000 7f99ddb6b740 1 --2- 172.21.15.83:0/4102992403 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee6def5090 0x55ee6df082a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.178+0000 7f99c1600640 0 client.26495 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:46.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.178+0000 7f99ddb6b740 1 -- 172.21.15.83:0/4102992403 >> 172.21.15.83:0/4102992403 conn(0x55ee6def13b0 msgr2=0x55ee6def0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:46.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.179+0000 7f99ddb6b740 1 -- 172.21.15.83:0/4102992403 shutdown_connections 2024-04-23T14:54:46.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.179+0000 7f99ddb6b740 1 -- 172.21.15.83:0/4102992403 wait complete. 2024-04-23T14:54:46.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.179+0000 7f99ddb6b740 20 client.26495 trim_cache size 0 max 16384 2024-04-23T14:54:46.200 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:54:46.200 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:54:46.200 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "f7ac3036-09ec-46f7-99d6-0d6ebeb6bbb6", 2024-04-23T14:54:46.200 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:54:46.200 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:54:46.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.179+0000 7f99dac00640 1 -- 172.21.15.83:0/3364925866 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f99b80a3d20 msgr2=0x7f99b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.179+0000 7f99dac00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f99b80a3d20 0x7f99b80a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009fd0 tx=0x7f99cc009500 comp rx=0 tx=0).stop 2024-04-23T14:54:46.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.179+0000 7f99dac00640 1 -- 172.21.15.83:0/3364925866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99d4108fd0 msgr2=0x7f99d419f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.179+0000 7f99dac00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99d4108fd0 0x7f99d419f420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f99bc02a860 tx=0x7f99bc033b60 comp rx=0 tx=0).stop 2024-04-23T14:54:46.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.180+0000 7f99dac00640 1 -- 172.21.15.83:0/3364925866 shutdown_connections 2024-04-23T14:54:46.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.180+0000 7f99dac00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f99b80a3d20 0x7f99b80a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.180+0000 7f99dac00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99d410e660 0x7f99d41a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.180+0000 7f99dac00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99d4108fd0 0x7f99d419f420 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.180+0000 7f99dac00640 1 --2- 172.21.15.83:0/3364925866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99d4108610 0x7f99d419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.180+0000 7f99dac00640 1 -- 172.21.15.83:0/3364925866 >> 172.21.15.83:0/3364925866 conn(0x7f99d4076540 msgr2=0x7f99d4077d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:46.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.180+0000 7f99dac00640 1 -- 172.21.15.83:0/3364925866 shutdown_connections 2024-04-23T14:54:46.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.180+0000 7f99dac00640 1 -- 172.21.15.83:0/3364925866 wait complete. 2024-04-23T14:54:46.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:54:46.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:54:46.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.395+0000 7f2d4d000640 1 -- 172.21.15.83:0/1321336003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d481015f0 msgr2=0x7f2d4810e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.395+0000 7f2d4d000640 1 --2- 172.21.15.83:0/1321336003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d481015f0 0x7f2d4810e2b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009860 tx=0x7f2d3002a340 comp rx=0 tx=0).stop 2024-04-23T14:54:46.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.395+0000 7f2d4d000640 1 -- 172.21.15.83:0/1321336003 shutdown_connections 2024-04-23T14:54:46.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.395+0000 7f2d4d000640 1 --2- 172.21.15.83:0/1321336003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d4810e7f0 0x7f2d48110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.395+0000 7f2d4d000640 1 --2- 172.21.15.83:0/1321336003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d481015f0 0x7f2d4810e2b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.395+0000 7f2d4d000640 1 --2- 172.21.15.83:0/1321336003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d48100c40 0x7f2d48101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.395+0000 7f2d4d000640 1 -- 172.21.15.83:0/1321336003 >> 172.21.15.83:0/1321336003 conn(0x7f2d480fc860 msgr2=0x7f2d480fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:46.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.395+0000 7f2d4d000640 1 -- 172.21.15.83:0/1321336003 shutdown_connections 2024-04-23T14:54:46.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.395+0000 7f2d4d000640 1 -- 172.21.15.83:0/1321336003 wait complete. 2024-04-23T14:54:46.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.396+0000 7f2d4d000640 1 Processor -- start 2024-04-23T14:54:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.396+0000 7f2d4d000640 1 -- start start 2024-04-23T14:54:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.397+0000 7f2d4d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d48100c40 0x7f2d481a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.397+0000 7f2d4d000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d481015f0 0x7f2d481a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.397+0000 7f2d4d000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d4810e7f0 0x7f2d4819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.397+0000 7f2d4d000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d48114470 con 0x7f2d48100c40 2024-04-23T14:54:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.397+0000 7f2d4d000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481142f0 con 0x7f2d481015f0 2024-04-23T14:54:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.397+0000 7f2d4d000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481145f0 con 0x7f2d4810e7f0 2024-04-23T14:54:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.397+0000 7f2d47400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d48100c40 0x7f2d481a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.397+0000 7f2d3ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d481015f0 0x7f2d481a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.397+0000 7f2d3ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d481015f0 0x7f2d481a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:37496/0 (socket says 172.21.15.83:37496) 2024-04-23T14:54:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.397+0000 7f2d3ea00640 1 -- 172.21.15.83:0/3738941814 learned_addr learned my addr 172.21.15.83:0/3738941814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.397+0000 7f2d47e00640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d4810e7f0 0x7f2d4819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d47400640 1 -- 172.21.15.83:0/3738941814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d4810e7f0 msgr2=0x7f2d4819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d47400640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d4810e7f0 0x7f2d4819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d47400640 1 -- 172.21.15.83:0/3738941814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d481015f0 msgr2=0x7f2d481a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d47400640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d481015f0 0x7f2d481a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d47400640 1 -- 172.21.15.83:0/3738941814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4819afa0 con 0x7f2d48100c40 2024-04-23T14:54:46.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d3ea00640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d481015f0 0x7f2d481a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:46.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d47e00640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d4810e7f0 0x7f2d4819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:46.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d47400640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d48100c40 0x7f2d481a0660 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800e790 tx=0x7f2d3800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:46.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d44c00640 1 -- 172.21.15.83:0/3738941814 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800eea0 con 0x7f2d48100c40 2024-04-23T14:54:46.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d44c00640 1 -- 172.21.15.83:0/3738941814 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3800ccc0 con 0x7f2d48100c40 2024-04-23T14:54:46.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d4d000640 1 -- 172.21.15.83:0/3738941814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4819b290 con 0x7f2d48100c40 2024-04-23T14:54:46.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d44c00640 1 -- 172.21.15.83:0/3738941814 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38018780 con 0x7f2d48100c40 2024-04-23T14:54:46.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.398+0000 7f2d4d000640 1 -- 172.21.15.83:0/3738941814 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d481a74b0 con 0x7f2d48100c40 2024-04-23T14:54:46.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.402+0000 7f2d51404740 1 Processor -- start 2024-04-23T14:54:46.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.402+0000 7f2d51404740 1 -- start start 2024-04-23T14:54:46.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.402+0000 7f2d44c00640 1 -- 172.21.15.83:0/3738941814 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2d38010430 con 0x7f2d48100c40 2024-04-23T14:54:46.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.402+0000 7f2d51404740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422140a30 0x561422140e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.402+0000 7f2d51404740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 0x56142214b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.402+0000 7f2d51404740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142214ba20 0x56142214de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.402+0000 7f2d51404740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5614220a61f0 con 0x56142214ba20 2024-04-23T14:54:46.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.402+0000 7f2d51404740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56142204c350 con 0x561422140a30 2024-04-23T14:54:46.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.402+0000 7f2d51404740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5614220a21b0 con 0x561422141350 2024-04-23T14:54:46.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.403+0000 7f2d47400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 0x56142214b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.403+0000 7f2d3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142214ba20 0x56142214de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.403+0000 7f2d47e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422140a30 0x561422140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.403+0000 7f2d47400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 0x56142214b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42582/0 (socket says 172.21.15.83:42582) 2024-04-23T14:54:46.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.403+0000 7f2d47400640 1 -- 172.21.15.83:0/1439525399 learned_addr learned my addr 172.21.15.83:0/1439525399 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:46.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.403+0000 7f2d47e00640 1 -- 172.21.15.83:0/1439525399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 msgr2=0x56142214b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.403+0000 7f2d47e00640 1 --2- 172.21.15.83:0/1439525399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 0x56142214b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.403+0000 7f2d47e00640 1 -- 172.21.15.83:0/1439525399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142214ba20 msgr2=0x56142214de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.403+0000 7f2d47e00640 1 --2- 172.21.15.83:0/1439525399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142214ba20 0x56142214de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.403+0000 7f2d47e00640 1 -- 172.21.15.83:0/1439525399 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614220a0f50 con 0x561422140a30 2024-04-23T14:54:46.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.403+0000 7f2d47400640 1 --2- 172.21.15.83:0/1439525399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 0x56142214b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:46.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.403+0000 7f2d47e00640 1 --2- 172.21.15.83:0/1439525399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422140a30 0x561422140e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2d48102e20 tx=0x7f2d40011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0497b9fef8412cd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:46.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.404+0000 7f2d44c00640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2d2c0a3d20 0x7f2d2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.404+0000 7f2d44c00640 1 -- 172.21.15.83:0/3738941814 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f2d380d0990 con 0x7f2d48100c40 2024-04-23T14:54:46.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.404+0000 7f2d3cc00640 1 -- 172.21.15.83:0/1439525399 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40004d20 con 0x561422140a30 2024-04-23T14:54:46.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.404+0000 7f2d47400640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2d2c0a3d20 0x7f2d2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.404+0000 7f2d3cc00640 1 -- 172.21.15.83:0/1439525399 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d4000f970 con 0x561422140a30 2024-04-23T14:54:46.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.404+0000 7f2d3cc00640 1 -- 172.21.15.83:0/1439525399 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d400044a0 con 0x561422140a30 2024-04-23T14:54:46.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.405+0000 7f2d51404740 1 -- 172.21.15.83:0/1439525399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422140a30 msgr2=0x561422140e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.405+0000 7f2d47400640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2d2c0a3d20 0x7f2d2c0a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d380993c0 tx=0x7f2d380d4430 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:46.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.405+0000 7f2d51404740 1 --2- 172.21.15.83:0/1439525399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422140a30 0x561422140e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2d48102e20 tx=0x7f2d40011bf0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.405+0000 7f2d51404740 1 -- 172.21.15.83:0/1439525399 shutdown_connections 2024-04-23T14:54:46.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.405+0000 7f2d51404740 1 --2- 172.21.15.83:0/1439525399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56142214ba20 0x56142214de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.405+0000 7f2d51404740 1 --2- 172.21.15.83:0/1439525399 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 0x56142214b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.405+0000 7f2d51404740 1 --2- 172.21.15.83:0/1439525399 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422140a30 0x561422140e10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.405+0000 7f2d51404740 1 -- 172.21.15.83:0/1439525399 >> 172.21.15.83:0/1439525399 conn(0x56142213cd50 msgr2=0x56142213d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:46.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.405+0000 7f2d51404740 1 -- 172.21.15.83:0/1439525399 shutdown_connections 2024-04-23T14:54:46.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.405+0000 7f2d51404740 1 -- 172.21.15.83:0/1439525399 wait complete. 2024-04-23T14:54:46.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.406+0000 7f2d51404740 1 -- start start 2024-04-23T14:54:46.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.406+0000 7f2d51404740 1 Processor -- start 2024-04-23T14:54:46.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.406+0000 7f2d51404740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 0x561422148920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.406+0000 7f2d51404740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 0x561422148e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.406+0000 7f2d51404740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56142214ba20 0x5614221493a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.406+0000 7f2d51404740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56142214e810 con 0x561422140a30 2024-04-23T14:54:46.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.406+0000 7f2d51404740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56142205f690 con 0x56142214ba20 2024-04-23T14:54:46.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.406+0000 7f2d51404740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56142214e990 con 0x561422141350 2024-04-23T14:54:46.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.406+0000 7f2d51404740 10 client.0 _ll_register_callbacks cb 0x56142209b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:46.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.406+0000 7f2d47e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 0x561422148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.407+0000 7f2d47e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 0x561422148920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39062/0 (socket says 172.21.15.83:39062) 2024-04-23T14:54:46.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.407+0000 7f2d47e00640 1 -- 172.21.15.83:0/2879462580 learned_addr learned my addr 172.21.15.83:0/2879462580 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:46.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.407+0000 7f2d3ea00640 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 0x561422148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.407+0000 7f2d47e00640 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56142214ba20 0x5614221493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.407+0000 7f2d47e00640 1 -- 172.21.15.83:0/2879462580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 msgr2=0x561422148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.407+0000 7f2d47e00640 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 0x561422148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.407+0000 7f2d47e00640 1 -- 172.21.15.83:0/2879462580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56142214ba20 msgr2=0x5614221493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.407+0000 7f2d47e00640 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56142214ba20 0x5614221493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.407+0000 7f2d47e00640 1 -- 172.21.15.83:0/2879462580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142205e920 con 0x561422140a30 2024-04-23T14:54:46.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.407+0000 7f2d3ea00640 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 0x561422148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:46.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.407+0000 7f2d47e00640 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 0x561422148920 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2d400040c0 tx=0x7f2d40004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afad30bc68034d61 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:46.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.407+0000 7f2d35600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:54:46.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.408+0000 7f2d35600640 1 -- 172.21.15.83:0/2879462580 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4002e060 con 0x561422140a30 2024-04-23T14:54:46.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.408+0000 7f2d35600640 1 -- 172.21.15.83:0/2879462580 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f2d40004220 con 0x561422140a30 2024-04-23T14:54:46.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.408+0000 7f2d51404740 1 -- 172.21.15.83:0/2879462580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d4000f970 con 0x561422140a30 2024-04-23T14:54:46.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.408+0000 7f2d35600640 1 -- 172.21.15.83:0/2879462580 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d40004420 con 0x561422140a30 2024-04-23T14:54:46.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.408+0000 7f2d51404740 10 client.16299 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:46.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.408+0000 7f2d51404740 1 -- 172.21.15.83:0/2879462580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561421e827d0 con 0x561422140a30 2024-04-23T14:54:46.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.408+0000 7f2d35600640 1 -- 172.21.15.83:0/2879462580 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d4000f970 con 0x561422140a30 2024-04-23T14:54:46.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.409+0000 7f2d35600640 1 -- 172.21.15.83:0/2879462580 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f2d40018070 con 0x561422140a30 2024-04-23T14:54:46.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.409+0000 7f2d51404740 10 client.16299 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:46.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.409+0000 7f2d51404740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:46.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.409+0000 7f2d51404740 20 client.16299 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:46.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.409+0000 7f2d51404740 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56142215bca0 0x56142215e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.409+0000 7f2d51404740 4 client.16299 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:54:46.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.409+0000 7f2d51404740 1 -- 172.21.15.83:0/2879462580 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614220b3c10 con 0x56142215bca0 2024-04-23T14:54:46.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.409+0000 7f2d3ea00640 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56142215bca0 0x56142215e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.410+0000 7f2d3ea00640 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56142215bca0 0x56142215e090 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d10eb351f40632a2 server_cookie=acc4e173b04a044b in_seq=0 out_seq=0 2024-04-23T14:54:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.410+0000 7f2d35600640 10 client.16299 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d35600640 1 -- 172.21.15.83:0/2879462580 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614220b3c10 con 0x56142215bca0 2024-04-23T14:54:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d35600640 10 client.16299 handle_command_reply: tid=0 2024-04-23T14:54:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d51404740 1 client.16299 shutdown 2024-04-23T14:54:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d51404740 1 -- 172.21.15.83:0/2879462580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 msgr2=0x561422148920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d51404740 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 0x561422148920 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2d400040c0 tx=0x7f2d40004140 comp rx=0 tx=0).stop 2024-04-23T14:54:46.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d51404740 1 -- 172.21.15.83:0/2879462580 shutdown_connections 2024-04-23T14:54:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d51404740 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56142215bca0 0x56142215e090 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d51404740 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56142214ba20 0x5614221493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d51404740 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561422141350 0x561422148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d35600640 0 client.16299 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d51404740 1 --2- 172.21.15.83:0/2879462580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 0x561422148920 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d51404740 1 -- 172.21.15.83:0/2879462580 >> 172.21.15.83:0/2879462580 conn(0x56142213cd50 msgr2=0x561422142b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d51404740 1 -- 172.21.15.83:0/2879462580 shutdown_connections 2024-04-23T14:54:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d51404740 1 -- 172.21.15.83:0/2879462580 wait complete. 2024-04-23T14:54:46.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.411+0000 7f2d51404740 20 client.16299 trim_cache size 0 max 16384 2024-04-23T14:54:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.430+0000 7f2d51404740 1 Processor -- start 2024-04-23T14:54:46.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.430+0000 7f2d51404740 1 -- start start 2024-04-23T14:54:46.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.430+0000 7f2d51404740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422140a30 0x5614221576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.430+0000 7f2d51404740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422141350 0x561422148200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.430+0000 7f2d51404740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56142214ba20 0x5614221637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.430+0000 7f2d51404740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481b2740 con 0x561422141350 2024-04-23T14:54:46.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.430+0000 7f2d51404740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481b0010 con 0x561422140a30 2024-04-23T14:54:46.432 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.431+0000 7f2d51404740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481ade60 con 0x56142214ba20 2024-04-23T14:54:46.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.431+0000 7f2d3ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422141350 0x561422148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.431+0000 7f2d47e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56142214ba20 0x5614221637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.431+0000 7f2d47e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56142214ba20 0x5614221637b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42602/0 (socket says 172.21.15.83:42602) 2024-04-23T14:54:46.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.431+0000 7f2d47e00640 1 -- 172.21.15.83:0/4291168389 learned_addr learned my addr 172.21.15.83:0/4291168389 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.431+0000 7f2d47e00640 1 --2- 172.21.15.83:0/4291168389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422140a30 0x5614221576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.431+0000 7f2d3ea00640 1 -- 172.21.15.83:0/4291168389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56142214ba20 msgr2=0x5614221637b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.431+0000 7f2d3ea00640 1 --2- 172.21.15.83:0/4291168389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56142214ba20 0x5614221637b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.431+0000 7f2d3ea00640 1 -- 172.21.15.83:0/4291168389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422140a30 msgr2=0x5614221576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.431+0000 7f2d3ea00640 1 --2- 172.21.15.83:0/4291168389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422140a30 0x5614221576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.431+0000 7f2d3ea00640 1 -- 172.21.15.83:0/4291168389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561421db83f0 con 0x561422141350 2024-04-23T14:54:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.432+0000 7f2d3ea00640 1 --2- 172.21.15.83:0/4291168389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422141350 0x561422148200 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2d30000c00 tx=0x7f2d300056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e049b35c8c24286 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:46.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.432+0000 7f2d37e00640 1 -- 172.21.15.83:0/4291168389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3004e070 con 0x561422141350 2024-04-23T14:54:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.432+0000 7f2d37e00640 1 -- 172.21.15.83:0/4291168389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d300035f0 con 0x561422141350 2024-04-23T14:54:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.432+0000 7f2d37e00640 1 -- 172.21.15.83:0/4291168389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30037040 con 0x561422141350 2024-04-23T14:54:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.433+0000 7f2d51404740 1 -- 172.21.15.83:0/4291168389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422141350 msgr2=0x561422148200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.433+0000 7f2d51404740 1 --2- 172.21.15.83:0/4291168389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422141350 0x561422148200 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2d30000c00 tx=0x7f2d300056d0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.433+0000 7f2d51404740 1 -- 172.21.15.83:0/4291168389 shutdown_connections 2024-04-23T14:54:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.433+0000 7f2d51404740 1 --2- 172.21.15.83:0/4291168389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56142214ba20 0x5614221637b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.433+0000 7f2d51404740 1 --2- 172.21.15.83:0/4291168389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422141350 0x561422148200 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.433+0000 7f2d51404740 1 --2- 172.21.15.83:0/4291168389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422140a30 0x5614221576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.433+0000 7f2d51404740 1 -- 172.21.15.83:0/4291168389 >> 172.21.15.83:0/4291168389 conn(0x56142213cd50 msgr2=0x561422142a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:46.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.433+0000 7f2d51404740 1 -- 172.21.15.83:0/4291168389 shutdown_connections 2024-04-23T14:54:46.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.433+0000 7f2d51404740 1 -- 172.21.15.83:0/4291168389 wait complete. 2024-04-23T14:54:46.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.433+0000 7f2d51404740 1 -- start start 2024-04-23T14:54:46.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.434+0000 7f2d51404740 1 Processor -- start 2024-04-23T14:54:46.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.434+0000 7f2d51404740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 0x561422153c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.434+0000 7f2d51404740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422141350 0x56142214f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.434+0000 7f2d51404740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56142214ba20 0x56142214f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.434+0000 7f2d51404740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481ae4d0 con 0x561422140a30 2024-04-23T14:54:46.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.434+0000 7f2d51404740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481af940 con 0x561422141350 2024-04-23T14:54:46.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.434+0000 7f2d51404740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481aeba0 con 0x56142214ba20 2024-04-23T14:54:46.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.434+0000 7f2d51404740 10 client.0 _ll_register_callbacks cb 0x7f2d400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:54:46.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.434+0000 7f2d47e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 0x561422153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.434+0000 7f2d47e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 0x561422153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39078/0 (socket says 172.21.15.83:39078) 2024-04-23T14:54:46.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.434+0000 7f2d47e00640 1 -- 172.21.15.83:0/302935626 learned_addr learned my addr 172.21.15.83:0/302935626 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:54:46.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.434+0000 7f2d3ea00640 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422141350 0x56142214f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.435+0000 7f2d47e00640 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56142214ba20 0x56142214f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.435+0000 7f2d3ea00640 1 -- 172.21.15.83:0/302935626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56142214ba20 msgr2=0x56142214f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.435+0000 7f2d3ea00640 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56142214ba20 0x56142214f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.435+0000 7f2d3ea00640 1 -- 172.21.15.83:0/302935626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 msgr2=0x561422153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:54:46.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.435+0000 7f2d3ea00640 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 0x561422153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.435+0000 7f2d3ea00640 1 -- 172.21.15.83:0/302935626 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561421d780f0 con 0x561422141350 2024-04-23T14:54:46.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.435+0000 7f2d47e00640 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 0x561422153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:54:46.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.435+0000 7f2d3ea00640 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422141350 0x56142214f160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x561422141f20 tx=0x7f2d300309d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74034bd138c1de4e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:54:46.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.435+0000 7f2d35600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:54:46.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.435+0000 7f2d35600640 1 -- 172.21.15.83:0/302935626 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30057060 con 0x561422141350 2024-04-23T14:54:46.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.435+0000 7f2d35600640 1 -- 172.21.15.83:0/302935626 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f2d300037a0 con 0x561422141350 2024-04-23T14:54:46.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.435+0000 7f2d51404740 1 -- 172.21.15.83:0/302935626 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d300035f0 con 0x561422141350 2024-04-23T14:54:46.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.436+0000 7f2d35600640 1 -- 172.21.15.83:0/302935626 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d3004e050 con 0x561422141350 2024-04-23T14:54:46.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.436+0000 7f2d51404740 10 client.26513 fetch_fsmap learned FSMap version 29 2024-04-23T14:54:46.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.436+0000 7f2d51404740 1 -- 172.21.15.83:0/302935626 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56142206b810 con 0x561422141350 2024-04-23T14:54:46.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.436+0000 7f2d35600640 1 -- 172.21.15.83:0/302935626 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d300035f0 con 0x561422141350 2024-04-23T14:54:46.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.436+0000 7f2d35600640 1 -- 172.21.15.83:0/302935626 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f2d4000fcc0 con 0x561422141350 2024-04-23T14:54:46.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.436+0000 7f2d51404740 10 client.26513 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:54:46.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.436+0000 7f2d51404740 10 client.26513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:54:46.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.436+0000 7f2d51404740 20 client.26513 populate_metadata read hostname 'smithi083' 2024-04-23T14:54:46.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.436+0000 7f2d51404740 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5614221c20e0 0x5614221c4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:54:46.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.436+0000 7f2d51404740 4 client.26513 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:54:46.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.436+0000 7f2d51404740 1 -- 172.21.15.83:0/302935626 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5614220ada80 con 0x5614221c20e0 2024-04-23T14:54:46.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.437+0000 7f2d47e00640 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5614221c20e0 0x5614221c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:54:46.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.437+0000 7f2d47e00640 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5614221c20e0 0x5614221c4500 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3736afa571b13246 server_cookie=39577c2c57d5687b in_seq=0 out_seq=0 2024-04-23T14:54:46.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.437+0000 7f2d35600640 10 client.26513 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:46.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.437+0000 7f2d35600640 1 -- 172.21.15.83:0/302935626 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5614220ada80 con 0x5614221c20e0 2024-04-23T14:54:46.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.437+0000 7f2d35600640 10 client.26513 handle_command_reply: tid=0 2024-04-23T14:54:46.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.437+0000 7f2d51404740 1 client.26513 shutdown 2024-04-23T14:54:46.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.437+0000 7f2d51404740 1 -- 172.21.15.83:0/302935626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422141350 msgr2=0x56142214f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.437+0000 7f2d51404740 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422141350 0x56142214f160 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x561422141f20 tx=0x7f2d300309d0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.438+0000 7f2d51404740 1 -- 172.21.15.83:0/302935626 shutdown_connections 2024-04-23T14:54:46.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.438+0000 7f2d51404740 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5614221c20e0 0x5614221c4500 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.438+0000 7f2d51404740 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56142214ba20 0x56142214f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.438+0000 7f2d35600640 0 client.26513 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:54:46.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.438+0000 7f2d51404740 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561422141350 0x56142214f160 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.438+0000 7f2d51404740 1 --2- 172.21.15.83:0/302935626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561422140a30 0x561422153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.438+0000 7f2d51404740 1 -- 172.21.15.83:0/302935626 >> 172.21.15.83:0/302935626 conn(0x56142213cd50 msgr2=0x5614221421d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.438+0000 7f2d51404740 1 -- 172.21.15.83:0/302935626 shutdown_connections 2024-04-23T14:54:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.438+0000 7f2d51404740 1 -- 172.21.15.83:0/302935626 wait complete. 2024-04-23T14:54:46.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.438+0000 7f2d51404740 20 client.26513 trim_cache size 0 max 16384 2024-04-23T14:54:46.446 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:54:46.446 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:54:46.446 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:54:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: "f7ac3036-09ec-46f7-99d6-0d6ebeb6bbb6": { 2024-04-23T14:54:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:54:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "f7ac3036-09ec-46f7-99d6-0d6ebeb6bbb6", 2024-04-23T14:54:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:54:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:54:46.447 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:54:46.447 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:54:46.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.439+0000 7f2d4d000640 1 -- 172.21.15.83:0/3738941814 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2d2c0a3d20 msgr2=0x7f2d2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.439+0000 7f2d4d000640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2d2c0a3d20 0x7f2d2c0a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d380993c0 tx=0x7f2d380d4430 comp rx=0 tx=0).stop 2024-04-23T14:54:46.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.439+0000 7f2d4d000640 1 -- 172.21.15.83:0/3738941814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d48100c40 msgr2=0x7f2d481a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:54:46.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.439+0000 7f2d4d000640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d48100c40 0x7f2d481a0660 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800e790 tx=0x7f2d3800ec50 comp rx=0 tx=0).stop 2024-04-23T14:54:46.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.439+0000 7f2d4d000640 1 -- 172.21.15.83:0/3738941814 shutdown_connections 2024-04-23T14:54:46.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.439+0000 7f2d4d000640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d4810e7f0 0x7f2d4819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.439+0000 7f2d4d000640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d481015f0 0x7f2d481a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.439+0000 7f2d4d000640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2d2c0a3d20 0x7f2d2c0a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.439+0000 7f2d4d000640 1 --2- 172.21.15.83:0/3738941814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d48100c40 0x7f2d481a0660 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:54:46.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.439+0000 7f2d4d000640 1 -- 172.21.15.83:0/3738941814 >> 172.21.15.83:0/3738941814 conn(0x7f2d480fc860 msgr2=0x7f2d480fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:54:46.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.439+0000 7f2d4d000640 1 -- 172.21.15.83:0/3738941814 shutdown_connections 2024-04-23T14:54:46.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:54:46.439+0000 7f2d4d000640 1 -- 172.21.15.83:0/3738941814 wait complete. 2024-04-23T14:54:46.458 INFO:tasks.cephfs.filesystem:scrub status for tag:f7ac3036-09ec-46f7-99d6-0d6ebeb6bbb6 - {'path': '/', 'tag': 'f7ac3036-09ec-46f7-99d6-0d6ebeb6bbb6', 'options': 'recursive,force'} 2024-04-23T14:54:47.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: pgmap v193: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:54:47.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:54:47.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:47.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:54:47.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:54:47.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:47.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:54:47.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:47.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:54:47.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:47.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:54:47.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:47.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:54:47.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:47.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:47 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: pgmap v193: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:54:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:54:48.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:54:48.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:54:48.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:54:48.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:54:48.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:54:48.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:54:48.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:47 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: pgmap v193: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:54:48.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:54:48.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:54:48.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:54:48.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:54:48.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:54:48.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:54:48.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:54:48.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:48.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:47 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:54:49.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:49 smithi203 ceph-mon[32800]: pgmap v194: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:54:50.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:49 smithi083 ceph-mon[27867]: pgmap v194: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:54:50.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:49 smithi173 ceph-mon[32443]: pgmap v194: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:54:50.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:50 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:54:51.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:50 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:54:51.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:50 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:54:51.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:51 smithi203 ceph-mon[32800]: pgmap v195: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:54:51.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:51 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:54:51.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:51 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:54:51.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:51 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:54:51.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:51 smithi173 ceph-mon[32443]: pgmap v195: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:54:51.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:51 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:54:51.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:51 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:54:51.977 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:51 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:54:52.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:51 smithi083 ceph-mon[27867]: pgmap v195: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:54:52.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:51 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:54:52.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:51 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:54:52.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:51 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:54:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:53 smithi203 ceph-mon[32800]: pgmap v196: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:54:54.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:53 smithi083 ceph-mon[27867]: pgmap v196: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:54:54.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:53 smithi173 ceph-mon[32443]: pgmap v196: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:54:55.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:54 smithi083 ceph-mon[27867]: pgmap v197: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:54:55.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:54 smithi173 ceph-mon[32443]: pgmap v197: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:54:55.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:54 smithi203 ceph-mon[32800]: pgmap v197: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:54:57.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:56 smithi203 ceph-mon[32800]: pgmap v198: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T14:54:57.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:56 smithi083 ceph-mon[27867]: pgmap v198: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T14:54:57.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:56 smithi173 ceph-mon[32443]: pgmap v198: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T14:54:59.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:58 smithi203 ceph-mon[32800]: pgmap v199: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:54:59.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:58 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:54:59.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:58 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:54:59.226 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:54:58 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:54:59.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:58 smithi083 ceph-mon[27867]: pgmap v199: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:54:59.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:58 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:54:59.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:58 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:54:59.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:54:58 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:54:59.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:58 smithi173 ceph-mon[32443]: pgmap v199: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:54:59.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:58 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:54:59.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:58 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:54:59.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:54:58 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:55:01.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:00 smithi203 ceph-mon[32800]: pgmap v200: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:55:01.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:00 smithi083 ceph-mon[27867]: pgmap v200: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:55:01.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:00 smithi173 ceph-mon[32443]: pgmap v200: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:55:03.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:02 smithi203 ceph-mon[32800]: pgmap v201: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-23T14:55:03.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:02 smithi083 ceph-mon[27867]: pgmap v201: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-23T14:55:03.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:02 smithi173 ceph-mon[32443]: pgmap v201: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-23T14:55:05.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:04 smithi203 ceph-mon[32800]: pgmap v202: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:55:05.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:04 smithi083 ceph-mon[27867]: pgmap v202: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:55:05.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:04 smithi173 ceph-mon[32443]: pgmap v202: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:55:07.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:06 smithi203 ceph-mon[32800]: pgmap v203: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-23T14:55:07.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:06 smithi083 ceph-mon[27867]: pgmap v203: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-23T14:55:07.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:06 smithi173 ceph-mon[32443]: pgmap v203: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-23T14:55:09.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:08 smithi203 ceph-mon[32800]: pgmap v204: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-23T14:55:09.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:08 smithi083 ceph-mon[27867]: pgmap v204: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-23T14:55:09.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:08 smithi173 ceph-mon[32443]: pgmap v204: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-23T14:55:10.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:09 smithi203 ceph-mon[32800]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T14:55:10.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:09 smithi083 ceph-mon[27867]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T14:55:10.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:09 smithi173 ceph-mon[32443]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T14:55:11.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:10 smithi203 ceph-mon[32800]: pgmap v206: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-23T14:55:11.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:10 smithi083 ceph-mon[27867]: pgmap v206: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-23T14:55:11.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:10 smithi173 ceph-mon[32443]: pgmap v206: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-23T14:55:13.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:12 smithi203 ceph-mon[32800]: pgmap v207: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T14:55:13.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:12 smithi083 ceph-mon[27867]: pgmap v207: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T14:55:13.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:12 smithi173 ceph-mon[32443]: pgmap v207: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T14:55:15.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:14 smithi203 ceph-mon[32800]: pgmap v208: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-23T14:55:15.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:14 smithi083 ceph-mon[27867]: pgmap v208: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-23T14:55:15.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:14 smithi173 ceph-mon[32443]: pgmap v208: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-23T14:55:16.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:55:16.460 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:55:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.669+0000 7f750ae00640 1 -- 172.21.15.83:0/2978419719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750410d500 msgr2=0x7f750410d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.669+0000 7f750ae00640 1 --2- 172.21.15.83:0/2978419719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750410d500 0x7f750410d8e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009930 tx=0x7f74f802a420 comp rx=0 tx=0).stop 2024-04-23T14:55:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.669+0000 7f750ae00640 1 -- 172.21.15.83:0/2978419719 shutdown_connections 2024-04-23T14:55:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.669+0000 7f750ae00640 1 --2- 172.21.15.83:0/2978419719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7504108f40 0x7f75041153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.669+0000 7f750ae00640 1 --2- 172.21.15.83:0/2978419719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7504108580 0x7f7504108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.669+0000 7f750ae00640 1 --2- 172.21.15.83:0/2978419719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750410d500 0x7f750410d8e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.669+0000 7f750ae00640 1 -- 172.21.15.83:0/2978419719 >> 172.21.15.83:0/2978419719 conn(0x7f7504077e20 msgr2=0x7f7504106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:16.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.670+0000 7f750ae00640 1 -- 172.21.15.83:0/2978419719 shutdown_connections 2024-04-23T14:55:16.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.670+0000 7f750ae00640 1 -- 172.21.15.83:0/2978419719 wait complete. 2024-04-23T14:55:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.670+0000 7f750ae00640 1 Processor -- start 2024-04-23T14:55:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.671+0000 7f750ae00640 1 -- start start 2024-04-23T14:55:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.671+0000 7f750ae00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504108580 0x7f75041a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.671+0000 7f750ae00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7504108f40 0x7f75041a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.671+0000 7f750ae00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f750410d500 0x7f750419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.671+0000 7f750ae00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75041188c0 con 0x7f7504108580 2024-04-23T14:55:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.671+0000 7f750ae00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7504118740 con 0x7f750410d500 2024-04-23T14:55:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.671+0000 7f750ae00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7504118a40 con 0x7f7504108f40 2024-04-23T14:55:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.671+0000 7f7509a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504108580 0x7f75041a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.671+0000 7f7509a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504108580 0x7f75041a4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48520/0 (socket says 172.21.15.83:48520) 2024-04-23T14:55:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.671+0000 7f7509a00640 1 -- 172.21.15.83:0/1827470162 learned_addr learned my addr 172.21.15.83:0/1827470162 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.671+0000 7f7509000640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7504108f40 0x7f75041a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.671+0000 7f750a400640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f750410d500 0x7f750419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.672+0000 7f7509a00640 1 -- 172.21.15.83:0/1827470162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7504108f40 msgr2=0x7f75041a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.672+0000 7f7509a00640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7504108f40 0x7f75041a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.672+0000 7f7509a00640 1 -- 172.21.15.83:0/1827470162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f750410d500 msgr2=0x7f750419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.672+0000 7f7509a00640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f750410d500 0x7f750419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.672+0000 7f7509a00640 1 -- 172.21.15.83:0/1827470162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750419f420 con 0x7f7504108580 2024-04-23T14:55:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.672+0000 7f7509a00640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504108580 0x7f75041a4a50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f74f8005940 tx=0x7f74f80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.672+0000 7f7502000640 1 -- 172.21.15.83:0/1827470162 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f803f030 con 0x7f7504108580 2024-04-23T14:55:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.672+0000 7f7502000640 1 -- 172.21.15.83:0/1827470162 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74f80044c0 con 0x7f7504108580 2024-04-23T14:55:16.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.672+0000 7f750ae00640 1 -- 172.21.15.83:0/1827470162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750419f6b0 con 0x7f7504108580 2024-04-23T14:55:16.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.672+0000 7f7502000640 1 -- 172.21.15.83:0/1827470162 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8035070 con 0x7f7504108580 2024-04-23T14:55:16.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.672+0000 7f750ae00640 1 -- 172.21.15.83:0/1827470162 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75041ab9d0 con 0x7f7504108580 2024-04-23T14:55:16.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.676+0000 7f750f27e740 1 Processor -- start 2024-04-23T14:55:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.676+0000 7f7502000640 1 -- 172.21.15.83:0/1827470162 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f74f8042030 con 0x7f7504108580 2024-04-23T14:55:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.676+0000 7f750f27e740 1 -- start start 2024-04-23T14:55:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.676+0000 7f750f27e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a0514e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.676+0000 7f750f27e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a0515350 0x5558a051f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.676+0000 7f750f27e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a051fa20 0x5558a0521e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.676+0000 7f750f27e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5558a047a1f0 con 0x5558a0514a30 2024-04-23T14:55:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.676+0000 7f750f27e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5558a0420350 con 0x5558a0515350 2024-04-23T14:55:16.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.676+0000 7f750f27e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5558a04761b0 con 0x5558a051fa20 2024-04-23T14:55:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.676+0000 7f750a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a0514e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.677+0000 7f750a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a0514e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48532/0 (socket says 172.21.15.83:48532) 2024-04-23T14:55:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.677+0000 7f750a400640 1 -- 172.21.15.83:0/3898578929 learned_addr learned my addr 172.21.15.83:0/3898578929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.677+0000 7f7509a00640 1 --2- 172.21.15.83:0/3898578929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a0515350 0x5558a051f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.677+0000 7f7509000640 1 --2- 172.21.15.83:0/3898578929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a051fa20 0x5558a0521e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.677+0000 7f750a400640 1 -- 172.21.15.83:0/3898578929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a051fa20 msgr2=0x5558a0521e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.677+0000 7f750a400640 1 --2- 172.21.15.83:0/3898578929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a051fa20 0x5558a0521e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.677+0000 7f750a400640 1 -- 172.21.15.83:0/3898578929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a0515350 msgr2=0x5558a051f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:16.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.677+0000 7f750a400640 1 --2- 172.21.15.83:0/3898578929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a0515350 0x5558a051f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.677+0000 7f750a400640 1 -- 172.21.15.83:0/3898578929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a0474f50 con 0x5558a0514a30 2024-04-23T14:55:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.677+0000 7f7502000640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f74ec0a3da0 0x7f74ec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.677+0000 7f750a400640 1 --2- 172.21.15.83:0/3898578929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a0514e10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009fc0 tx=0x7f74fc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e69e6129b58abc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.678+0000 7f7502000640 1 -- 172.21.15.83:0/1827470162 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f74f80f0b70 con 0x7f7504108580 2024-04-23T14:55:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.678+0000 7f7509a00640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f74ec0a3da0 0x7f74ec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.678+0000 7f74f6a00640 1 -- 172.21.15.83:0/3898578929 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc004d20 con 0x5558a0514a30 2024-04-23T14:55:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.678+0000 7f74f6a00640 1 -- 172.21.15.83:0/3898578929 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74fc004050 con 0x5558a0514a30 2024-04-23T14:55:16.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.678+0000 7f74f6a00640 1 -- 172.21.15.83:0/3898578929 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc005260 con 0x5558a0514a30 2024-04-23T14:55:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.679+0000 7f750f27e740 1 -- 172.21.15.83:0/3898578929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 msgr2=0x5558a0514e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.679+0000 7f750f27e740 1 --2- 172.21.15.83:0/3898578929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a0514e10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009fc0 tx=0x7f74fc0118d0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.679+0000 7f7509a00640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f74ec0a3da0 0x7f74ec0a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x5558a0515ff0 tx=0x7f74f80f4620 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.679+0000 7f750f27e740 1 -- 172.21.15.83:0/3898578929 shutdown_connections 2024-04-23T14:55:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.679+0000 7f750f27e740 1 --2- 172.21.15.83:0/3898578929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a051fa20 0x5558a0521e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.679+0000 7f750f27e740 1 --2- 172.21.15.83:0/3898578929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a0515350 0x5558a051f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.679+0000 7f750f27e740 1 --2- 172.21.15.83:0/3898578929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a0514e10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.679+0000 7f750f27e740 1 -- 172.21.15.83:0/3898578929 >> 172.21.15.83:0/3898578929 conn(0x5558a0510d50 msgr2=0x5558a0511130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:16.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.679+0000 7f750f27e740 1 -- 172.21.15.83:0/3898578929 shutdown_connections 2024-04-23T14:55:16.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.679+0000 7f750f27e740 1 -- 172.21.15.83:0/3898578929 wait complete. 2024-04-23T14:55:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.679+0000 7f750f27e740 1 -- start start 2024-04-23T14:55:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.680+0000 7f750f27e740 1 Processor -- start 2024-04-23T14:55:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.680+0000 7f750f27e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a051c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.688 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.680+0000 7f750f27e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a0515350 0x5558a051ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.680+0000 7f750f27e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a051fa20 0x5558a051d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.680+0000 7f750f27e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5558a0522810 con 0x5558a0514a30 2024-04-23T14:55:16.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.680+0000 7f750f27e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5558a0433690 con 0x5558a051fa20 2024-04-23T14:55:16.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.680+0000 7f750f27e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5558a0522990 con 0x5558a0515350 2024-04-23T14:55:16.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.680+0000 7f750f27e740 10 client.0 _ll_register_callbacks cb 0x5558a046f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:16.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.680+0000 7f750a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a051c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.680+0000 7f750a400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a051c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48546/0 (socket says 172.21.15.83:48546) 2024-04-23T14:55:16.689 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.680+0000 7f750a400640 1 -- 172.21.15.83:0/2563578688 learned_addr learned my addr 172.21.15.83:0/2563578688 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.680+0000 7f7509000640 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a0515350 0x5558a051ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.681+0000 7f750a400640 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a051fa20 0x5558a051d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.681+0000 7f750a400640 1 -- 172.21.15.83:0/2563578688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a0515350 msgr2=0x5558a051ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.681+0000 7f750a400640 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a0515350 0x5558a051ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.681+0000 7f750a400640 1 -- 172.21.15.83:0/2563578688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a051fa20 msgr2=0x5558a051d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.681+0000 7f750a400640 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a051fa20 0x5558a051d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.690 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.681+0000 7f750a400640 1 -- 172.21.15.83:0/2563578688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a0432920 con 0x5558a0514a30 2024-04-23T14:55:16.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.681+0000 7f750a400640 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a051c990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009dd0 tx=0x7f74fc009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414700d9b3f6d4ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:16.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.681+0000 7f74eb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:55:16.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.681+0000 7f74eb400640 1 -- 172.21.15.83:0/2563578688 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc004610 con 0x5558a0514a30 2024-04-23T14:55:16.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.681+0000 7f74eb400640 1 -- 172.21.15.83:0/2563578688 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f74fc00f4e0 con 0x5558a0514a30 2024-04-23T14:55:16.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.681+0000 7f750f27e740 1 -- 172.21.15.83:0/2563578688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74fc004050 con 0x5558a0514a30 2024-04-23T14:55:16.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.682+0000 7f74eb400640 1 -- 172.21.15.83:0/2563578688 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74fc035050 con 0x5558a0514a30 2024-04-23T14:55:16.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.682+0000 7f750f27e740 10 client.16317 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:16.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.682+0000 7f750f27e740 1 -- 172.21.15.83:0/2563578688 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558a02567d0 con 0x5558a0514a30 2024-04-23T14:55:16.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.682+0000 7f74eb400640 1 -- 172.21.15.83:0/2563578688 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74fc004050 con 0x5558a0514a30 2024-04-23T14:55:16.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.682+0000 7f74eb400640 1 -- 172.21.15.83:0/2563578688 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f74fc0053d0 con 0x5558a0514a30 2024-04-23T14:55:16.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.682+0000 7f750f27e740 10 client.16317 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:16.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.682+0000 7f750f27e740 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:16.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.682+0000 7f750f27e740 20 client.16317 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:16.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.682+0000 7f750f27e740 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5558a052fd40 0x5558a0532130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.682+0000 7f750f27e740 4 client.16317 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:55:16.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.682+0000 7f750f27e740 1 -- 172.21.15.83:0/2563578688 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558a0487c10 con 0x5558a052fd40 2024-04-23T14:55:16.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.683+0000 7f7509000640 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5558a052fd40 0x5558a0532130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.683+0000 7f7509000640 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5558a052fd40 0x5558a0532130 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b9182f06e5f3f server_cookie=35135bf13fa5e20d in_seq=0 out_seq=0 2024-04-23T14:55:16.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.683+0000 7f74eb400640 10 client.16317 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:16.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.684+0000 7f74eb400640 1 -- 172.21.15.83:0/2563578688 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558a0487c10 con 0x5558a052fd40 2024-04-23T14:55:16.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.684+0000 7f74eb400640 10 client.16317 handle_command_reply: tid=0 2024-04-23T14:55:16.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.684+0000 7f750f27e740 1 client.16317 shutdown 2024-04-23T14:55:16.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.684+0000 7f750f27e740 1 -- 172.21.15.83:0/2563578688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 msgr2=0x5558a051c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:16.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.684+0000 7f750f27e740 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a051c990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009dd0 tx=0x7f74fc009d20 comp rx=0 tx=0).stop 2024-04-23T14:55:16.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.684+0000 7f750f27e740 1 -- 172.21.15.83:0/2563578688 shutdown_connections 2024-04-23T14:55:16.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.684+0000 7f750f27e740 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5558a052fd40 0x5558a0532130 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.684+0000 7f750f27e740 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a051fa20 0x5558a051d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.684+0000 7f750f27e740 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a0515350 0x5558a051ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.684+0000 7f750f27e740 1 --2- 172.21.15.83:0/2563578688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a051c990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.684+0000 7f74eb400640 0 client.16317 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:16.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.684+0000 7f750f27e740 1 -- 172.21.15.83:0/2563578688 >> 172.21.15.83:0/2563578688 conn(0x5558a0510d50 msgr2=0x5558a050ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:16.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.685+0000 7f750f27e740 1 -- 172.21.15.83:0/2563578688 shutdown_connections 2024-04-23T14:55:16.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.685+0000 7f750f27e740 1 -- 172.21.15.83:0/2563578688 wait complete. 2024-04-23T14:55:16.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.685+0000 7f750f27e740 20 client.16317 trim_cache size 0 max 16384 2024-04-23T14:55:16.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f750f27e740 1 Processor -- start 2024-04-23T14:55:16.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f750f27e740 1 -- start start 2024-04-23T14:55:16.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f750f27e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a0514a30 0x5558a051f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f750f27e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0515350 0x5558a052b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f750f27e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a051fa20 0x5558a05378c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f750f27e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b6ab0 con 0x5558a0515350 2024-04-23T14:55:16.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f750f27e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b4380 con 0x5558a0514a30 2024-04-23T14:55:16.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f750f27e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b21f0 con 0x5558a051fa20 2024-04-23T14:55:16.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f750a400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a0514a30 0x5558a051f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f750a400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a0514a30 0x5558a051f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38168/0 (socket says 172.21.15.83:38168) 2024-04-23T14:55:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f750a400640 1 -- 172.21.15.83:0/2224217458 learned_addr learned my addr 172.21.15.83:0/2224217458 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f750a400640 1 --2- 172.21.15.83:0/2224217458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a051fa20 0x5558a05378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.704+0000 7f7509000640 1 --2- 172.21.15.83:0/2224217458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0515350 0x5558a052b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.705+0000 7f750a400640 1 -- 172.21.15.83:0/2224217458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a051fa20 msgr2=0x5558a05378c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.705+0000 7f750a400640 1 --2- 172.21.15.83:0/2224217458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a051fa20 0x5558a05378c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.705+0000 7f750a400640 1 -- 172.21.15.83:0/2224217458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0515350 msgr2=0x5558a052b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:16.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.705+0000 7f750a400640 1 --2- 172.21.15.83:0/2224217458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0515350 0x5558a052b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.705+0000 7f750a400640 1 -- 172.21.15.83:0/2224217458 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a018c3f0 con 0x5558a0514a30 2024-04-23T14:55:16.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.705+0000 7f750a400640 1 --2- 172.21.15.83:0/2224217458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a0514a30 0x5558a051f1d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009fd0 tx=0x7f74fc014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86fbf653f41b257 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:16.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.705+0000 7f74f6000640 1 -- 172.21.15.83:0/2224217458 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0059c0 con 0x5558a0514a30 2024-04-23T14:55:16.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.705+0000 7f74f6000640 1 -- 172.21.15.83:0/2224217458 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74fc02c070 con 0x5558a0514a30 2024-04-23T14:55:16.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.705+0000 7f74f6000640 1 -- 172.21.15.83:0/2224217458 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0196b0 con 0x5558a0514a30 2024-04-23T14:55:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.706+0000 7f750f27e740 1 -- 172.21.15.83:0/2224217458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a0514a30 msgr2=0x5558a051f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.706+0000 7f750f27e740 1 --2- 172.21.15.83:0/2224217458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a0514a30 0x5558a051f1d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009fd0 tx=0x7f74fc014040 comp rx=0 tx=0).stop 2024-04-23T14:55:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.706+0000 7f750f27e740 1 -- 172.21.15.83:0/2224217458 shutdown_connections 2024-04-23T14:55:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.706+0000 7f750f27e740 1 --2- 172.21.15.83:0/2224217458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a051fa20 0x5558a05378c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.706+0000 7f750f27e740 1 --2- 172.21.15.83:0/2224217458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0515350 0x5558a052b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.706+0000 7f750f27e740 1 --2- 172.21.15.83:0/2224217458 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a0514a30 0x5558a051f1d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.706+0000 7f750f27e740 1 -- 172.21.15.83:0/2224217458 >> 172.21.15.83:0/2224217458 conn(0x5558a0510d50 msgr2=0x5558a051d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.706+0000 7f750f27e740 1 -- 172.21.15.83:0/2224217458 shutdown_connections 2024-04-23T14:55:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.706+0000 7f750f27e740 1 -- 172.21.15.83:0/2224217458 wait complete. 2024-04-23T14:55:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.707+0000 7f750f27e740 1 -- start start 2024-04-23T14:55:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.707+0000 7f750f27e740 1 Processor -- start 2024-04-23T14:55:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.707+0000 7f750f27e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a0527fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.707+0000 7f750f27e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a0515350 0x5558a0523350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.707+0000 7f750f27e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a051fa20 0x5558a0523890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.707+0000 7f750f27e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b2840 con 0x5558a0514a30 2024-04-23T14:55:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.707+0000 7f750f27e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b3cb0 con 0x5558a051fa20 2024-04-23T14:55:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.707+0000 7f750f27e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f75041b2f10 con 0x5558a0515350 2024-04-23T14:55:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.707+0000 7f750f27e740 10 client.0 _ll_register_callbacks cb 0x7f74f80f44f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f7509000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a0515350 0x5558a0523350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f7509000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a0515350 0x5558a0523350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:39578/0 (socket says 172.21.15.83:39578) 2024-04-23T14:55:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f7509000640 1 -- 172.21.15.83:0/1491793616 learned_addr learned my addr 172.21.15.83:0/1491793616 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f750a400640 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a0527fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f750a400640 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a051fa20 0x5558a0523890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f7509000640 1 -- 172.21.15.83:0/1491793616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a051fa20 msgr2=0x5558a0523890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f7509000640 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a051fa20 0x5558a0523890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f7509000640 1 -- 172.21.15.83:0/1491793616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 msgr2=0x5558a0527fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f7509000640 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a0527fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f7509000640 1 -- 172.21.15.83:0/1491793616 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a014c0f0 con 0x5558a0515350 2024-04-23T14:55:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f750a400640 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a051fa20 0x5558a0523890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:55:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f7509000640 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a0515350 0x5558a0523350 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5558a0511ee0 tx=0x7f74f0010a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=946200b40ab2b3c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.708+0000 7f74eb400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:55:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.709+0000 7f74eb400640 1 -- 172.21.15.83:0/1491793616 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0020630 con 0x5558a0515350 2024-04-23T14:55:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.709+0000 7f750f27e740 1 -- 172.21.15.83:0/1491793616 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74fc02c070 con 0x5558a0515350 2024-04-23T14:55:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.709+0000 7f74eb400640 1 -- 172.21.15.83:0/1491793616 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f74f00207d0 con 0x5558a0515350 2024-04-23T14:55:16.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.709+0000 7f74eb400640 1 -- 172.21.15.83:0/1491793616 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74f0005020 con 0x5558a0515350 2024-04-23T14:55:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.709+0000 7f750f27e740 10 client.26500 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.709+0000 7f750f27e740 1 -- 172.21.15.83:0/1491793616 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558a043f810 con 0x5558a0515350 2024-04-23T14:55:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.709+0000 7f74eb400640 1 -- 172.21.15.83:0/1491793616 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74fc02c070 con 0x5558a0515350 2024-04-23T14:55:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.709+0000 7f74eb400640 1 -- 172.21.15.83:0/1491793616 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f74f0028420 con 0x5558a0515350 2024-04-23T14:55:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.710+0000 7f750f27e740 10 client.26500 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.710+0000 7f750f27e740 10 client.26500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.710+0000 7f750f27e740 20 client.26500 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:16.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.710+0000 7f750f27e740 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5558a05965b0 0x5558a05989d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.710+0000 7f750f27e740 4 client.26500 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:55:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.710+0000 7f750f27e740 1 -- 172.21.15.83:0/1491793616 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5558a0481a80 con 0x5558a05965b0 2024-04-23T14:55:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.710+0000 7f750a400640 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5558a05965b0 0x5558a05989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.710+0000 7f74eb400640 10 client.26500 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.710+0000 7f750a400640 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5558a05965b0 0x5558a05989d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d231453a78ae5a81 server_cookie=91a40cd98c01644e in_seq=0 out_seq=0 2024-04-23T14:55:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f74eb400640 1 -- 172.21.15.83:0/1491793616 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5558a0481a80 con 0x5558a05965b0 2024-04-23T14:55:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f74eb400640 10 client.26500 handle_command_reply: tid=0 2024-04-23T14:55:16.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f750f27e740 1 client.26500 shutdown 2024-04-23T14:55:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f750f27e740 1 -- 172.21.15.83:0/1491793616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a0515350 msgr2=0x5558a0523350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f750f27e740 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a0515350 0x5558a0523350 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5558a0511ee0 tx=0x7f74f0010a20 comp rx=0 tx=0).stop 2024-04-23T14:55:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f750f27e740 1 -- 172.21.15.83:0/1491793616 shutdown_connections 2024-04-23T14:55:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f750f27e740 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5558a05965b0 0x5558a05989d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f750f27e740 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5558a051fa20 0x5558a0523890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f750f27e740 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558a0515350 0x5558a0523350 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f750f27e740 1 --2- 172.21.15.83:0/1491793616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5558a0514a30 0x5558a0527fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f750f27e740 1 -- 172.21.15.83:0/1491793616 >> 172.21.15.83:0/1491793616 conn(0x5558a0510d50 msgr2=0x5558a0516020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f74eb400640 0 client.26500 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f750f27e740 1 -- 172.21.15.83:0/1491793616 shutdown_connections 2024-04-23T14:55:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f750f27e740 1 -- 172.21.15.83:0/1491793616 wait complete. 2024-04-23T14:55:16.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.711+0000 7f750f27e740 20 client.26500 trim_cache size 0 max 16384 2024-04-23T14:55:16.723 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:55:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:55:16.723 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:55:16.723 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:55:16.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.712+0000 7f750ae00640 1 -- 172.21.15.83:0/1827470162 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f74ec0a3da0 msgr2=0x7f74ec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:16.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.712+0000 7f750ae00640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f74ec0a3da0 0x7f74ec0a6260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x5558a0515ff0 tx=0x7f74f80f4620 comp rx=0 tx=0).stop 2024-04-23T14:55:16.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.712+0000 7f750ae00640 1 -- 172.21.15.83:0/1827470162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504108580 msgr2=0x7f75041a4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:16.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.712+0000 7f750ae00640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504108580 0x7f75041a4a50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f74f8005940 tx=0x7f74f80040c0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.712+0000 7f750ae00640 1 -- 172.21.15.83:0/1827470162 shutdown_connections 2024-04-23T14:55:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.712+0000 7f750ae00640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f750410d500 0x7f750419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.712+0000 7f750ae00640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7504108f40 0x7f75041a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.712+0000 7f750ae00640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f74ec0a3da0 0x7f74ec0a6260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.712+0000 7f750ae00640 1 --2- 172.21.15.83:0/1827470162 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7504108580 0x7f75041a4a50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.712+0000 7f750ae00640 1 -- 172.21.15.83:0/1827470162 >> 172.21.15.83:0/1827470162 conn(0x7f7504077e20 msgr2=0x7f75040783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.712+0000 7f750ae00640 1 -- 172.21.15.83:0/1827470162 shutdown_connections 2024-04-23T14:55:16.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.712+0000 7f750ae00640 1 -- 172.21.15.83:0/1827470162 wait complete. 2024-04-23T14:55:16.731 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:55:16.732 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:55:16.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.929+0000 7f98eac00640 1 -- 172.21.15.83:0/1401455862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e4105d30 msgr2=0x7f98e4106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:16.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.929+0000 7f98eac00640 1 --2- 172.21.15.83:0/1401455862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e4105d30 0x7f98e4106110 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009930 tx=0x7f98d802a4d0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.929+0000 7f98eac00640 1 -- 172.21.15.83:0/1401455862 shutdown_connections 2024-04-23T14:55:16.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.929+0000 7f98eac00640 1 --2- 172.21.15.83:0/1401455862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e41019b0 0x7f98e410dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.929+0000 7f98eac00640 1 --2- 172.21.15.83:0/1401455862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e4100ff0 0x7f98e4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.929+0000 7f98eac00640 1 --2- 172.21.15.83:0/1401455862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e4105d30 0x7f98e4106110 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.929+0000 7f98eac00640 1 -- 172.21.15.83:0/1401455862 >> 172.21.15.83:0/1401455862 conn(0x7f98e4076840 msgr2=0x7f98e4076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:16.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.929+0000 7f98eac00640 1 -- 172.21.15.83:0/1401455862 shutdown_connections 2024-04-23T14:55:16.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.929+0000 7f98eac00640 1 -- 172.21.15.83:0/1401455862 wait complete. 2024-04-23T14:55:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.930+0000 7f98eac00640 1 Processor -- start 2024-04-23T14:55:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.931+0000 7f98eac00640 1 -- start start 2024-04-23T14:55:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.931+0000 7f98eac00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e4100ff0 0x7f98e41a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.931+0000 7f98eac00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e41019b0 0x7f98e41a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.931+0000 7f98eac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e4105d30 0x7f98e419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.931+0000 7f98eac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4111110 con 0x7f98e4105d30 2024-04-23T14:55:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.931+0000 7f98eac00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4110f90 con 0x7f98e41019b0 2024-04-23T14:55:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.931+0000 7f98eac00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4111290 con 0x7f98e4100ff0 2024-04-23T14:55:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.931+0000 7f98e9800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e4100ff0 0x7f98e41a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.931+0000 7f98ea200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e4105d30 0x7f98e419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.931+0000 7f98e9800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e4100ff0 0x7f98e41a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:39600/0 (socket says 172.21.15.83:39600) 2024-04-23T14:55:16.935 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.931+0000 7f98e9800640 1 -- 172.21.15.83:0/3371879000 learned_addr learned my addr 172.21.15.83:0/3371879000 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.931+0000 7f98e8e00640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e41019b0 0x7f98e41a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.932+0000 7f98ea200640 1 -- 172.21.15.83:0/3371879000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e4100ff0 msgr2=0x7f98e41a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.932+0000 7f98ea200640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e4100ff0 0x7f98e41a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.932+0000 7f98ea200640 1 -- 172.21.15.83:0/3371879000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e41019b0 msgr2=0x7f98e41a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.932+0000 7f98ea200640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e41019b0 0x7f98e41a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.932+0000 7f98ea200640 1 -- 172.21.15.83:0/3371879000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e419af40 con 0x7f98e4105d30 2024-04-23T14:55:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.932+0000 7f98e8e00640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e41019b0 0x7f98e41a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:16.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.932+0000 7f98ea200640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e4105d30 0x7f98e419a680 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00ec50 tx=0x7f98dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:16.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.932+0000 7f98e2000640 1 -- 172.21.15.83:0/3371879000 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc003b70 con 0x7f98e4105d30 2024-04-23T14:55:16.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.932+0000 7f98e2000640 1 -- 172.21.15.83:0/3371879000 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98dc003d10 con 0x7f98e4105d30 2024-04-23T14:55:16.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.932+0000 7f98eac00640 1 -- 172.21.15.83:0/3371879000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e419b230 con 0x7f98e4105d30 2024-04-23T14:55:16.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.932+0000 7f98e2000640 1 -- 172.21.15.83:0/3371879000 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc010890 con 0x7f98e4105d30 2024-04-23T14:55:16.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.933+0000 7f98eac00640 1 -- 172.21.15.83:0/3371879000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e41a7540 con 0x7f98e4105d30 2024-04-23T14:55:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.934+0000 7f98eac00640 1 -- 172.21.15.83:0/3371879000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e4102460 con 0x7f98e4105d30 2024-04-23T14:55:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.936+0000 7f98e2000640 1 -- 172.21.15.83:0/3371879000 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f98dc003fc0 con 0x7f98e4105d30 2024-04-23T14:55:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.937+0000 7f98e2000640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f98cc0a3d20 0x7f98cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.937+0000 7f98e2000640 1 -- 172.21.15.83:0/3371879000 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8586+0+0 (secure 0 0 0) 0x7f98dc0d0e30 con 0x7f98e4105d30 2024-04-23T14:55:16.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.937+0000 7f98e9800640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f98cc0a3d20 0x7f98cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:16.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.938+0000 7f98e9800640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f98cc0a3d20 0x7f98cc0a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005c40 tx=0x7f98d8005b50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:16.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:16.942+0000 7f98e2000640 1 -- 172.21.15.83:0/3371879000 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98dc09d430 con 0x7f98e4105d30 2024-04-23T14:55:17.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.186+0000 7f98eac00640 1 -- 172.21.15.83:0/3371879000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98e419b460 con 0x7f98e4105d30 2024-04-23T14:55:17.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.188+0000 7f98e2000640 1 -- 172.21.15.83:0/3371879000 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7f98dc09cd30 con 0x7f98e4105d30 2024-04-23T14:55:17.189 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:55:17.189 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:55:17.189 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:55:17.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.190+0000 7f98eac00640 1 -- 172.21.15.83:0/3371879000 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f98cc0a3d20 msgr2=0x7f98cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.190+0000 7f98eac00640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f98cc0a3d20 0x7f98cc0a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005c40 tx=0x7f98d8005b50 comp rx=0 tx=0).stop 2024-04-23T14:55:17.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.190+0000 7f98eac00640 1 -- 172.21.15.83:0/3371879000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e4105d30 msgr2=0x7f98e419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.190+0000 7f98eac00640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e4105d30 0x7f98e419a680 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00ec50 tx=0x7f98dc00c6a0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.190+0000 7f98eac00640 1 -- 172.21.15.83:0/3371879000 shutdown_connections 2024-04-23T14:55:17.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.190+0000 7f98eac00640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f98cc0a3d20 0x7f98cc0a61e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.190+0000 7f98eac00640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98e4105d30 0x7f98e419a680 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.190+0000 7f98eac00640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e41019b0 0x7f98e41a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.190+0000 7f98eac00640 1 --2- 172.21.15.83:0/3371879000 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f98e4100ff0 0x7f98e41a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.190+0000 7f98eac00640 1 -- 172.21.15.83:0/3371879000 >> 172.21.15.83:0/3371879000 conn(0x7f98e4076840 msgr2=0x7f98e410c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.191+0000 7f98eac00640 1 -- 172.21.15.83:0/3371879000 shutdown_connections 2024-04-23T14:55:17.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.191+0000 7f98eac00640 1 -- 172.21.15.83:0/3371879000 wait complete. 2024-04-23T14:55:17.207 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:16 smithi083 ceph-mon[27867]: pgmap v209: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:55:17.210 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:55:17.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:16 smithi203 ceph-mon[32800]: pgmap v209: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:55:17.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:16 smithi173 ceph-mon[32443]: pgmap v209: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:55:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.402+0000 7fd8e6000640 1 -- 172.21.15.83:0/446640367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8e0108580 msgr2=0x7fd8e0108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.402+0000 7fd8e6000640 1 --2- 172.21.15.83:0/446640367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8e0108580 0x7fd8e0108a00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009930 tx=0x7fd8c802a420 comp rx=0 tx=0).stop 2024-04-23T14:55:17.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.402+0000 7fd8e6000640 1 -- 172.21.15.83:0/446640367 shutdown_connections 2024-04-23T14:55:17.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.402+0000 7fd8e6000640 1 --2- 172.21.15.83:0/446640367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0108f40 0x7fd8e01153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.402+0000 7fd8e6000640 1 --2- 172.21.15.83:0/446640367 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8e0108580 0x7fd8e0108a00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.402+0000 7fd8e6000640 1 --2- 172.21.15.83:0/446640367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8e010d500 0x7fd8e010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.402+0000 7fd8e6000640 1 -- 172.21.15.83:0/446640367 >> 172.21.15.83:0/446640367 conn(0x7fd8e0077e20 msgr2=0x7fd8e0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.402+0000 7fd8e6000640 1 -- 172.21.15.83:0/446640367 shutdown_connections 2024-04-23T14:55:17.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.402+0000 7fd8e6000640 1 -- 172.21.15.83:0/446640367 wait complete. 2024-04-23T14:55:17.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.403+0000 7fd8e6000640 1 Processor -- start 2024-04-23T14:55:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.403+0000 7fd8e6000640 1 -- start start 2024-04-23T14:55:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.403+0000 7fd8e6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0108580 0x7fd8e019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.403+0000 7fd8e6000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8e0108f40 0x7fd8e019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.403+0000 7fd8e6000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8e010d500 0x7fd8e01a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.403+0000 7fd8e6000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e01188d0 con 0x7fd8e0108580 2024-04-23T14:55:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.403+0000 7fd8e6000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e0118750 con 0x7fd8e010d500 2024-04-23T14:55:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.403+0000 7fd8e6000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e0118a50 con 0x7fd8e0108f40 2024-04-23T14:55:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.404+0000 7fd8e4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0108580 0x7fd8e019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.404+0000 7fd8e4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0108580 0x7fd8e019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48582/0 (socket says 172.21.15.83:48582) 2024-04-23T14:55:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.404+0000 7fd8e4c00640 1 -- 172.21.15.83:0/3461518128 learned_addr learned my addr 172.21.15.83:0/3461518128 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.404+0000 7fd8d7e00640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8e0108f40 0x7fd8e019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.404+0000 7fd8e5600640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8e010d500 0x7fd8e01a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.404+0000 7fd8e4c00640 1 -- 172.21.15.83:0/3461518128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8e0108f40 msgr2=0x7fd8e019f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.404+0000 7fd8e4c00640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8e0108f40 0x7fd8e019f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.404+0000 7fd8e4c00640 1 -- 172.21.15.83:0/3461518128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8e010d500 msgr2=0x7fd8e01a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.404+0000 7fd8e4c00640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8e010d500 0x7fd8e01a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.404+0000 7fd8e4c00640 1 -- 172.21.15.83:0/3461518128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e01a3d20 con 0x7fd8e0108580 2024-04-23T14:55:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.404+0000 7fd8e5600640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8e010d500 0x7fd8e01a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:55:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.404+0000 7fd8d7e00640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8e0108f40 0x7fd8e019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.405+0000 7fd8e4c00640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0108580 0x7fd8e019ed30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000e9c0 tx=0x7fd8d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.405+0000 7fd8de000640 1 -- 172.21.15.83:0/3461518128 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d000cd80 con 0x7fd8e0108580 2024-04-23T14:55:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.405+0000 7fd8de000640 1 -- 172.21.15.83:0/3461518128 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8d00035f0 con 0x7fd8e0108580 2024-04-23T14:55:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.405+0000 7fd8e6000640 1 -- 172.21.15.83:0/3461518128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e01a4010 con 0x7fd8e0108580 2024-04-23T14:55:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.405+0000 7fd8de000640 1 -- 172.21.15.83:0/3461518128 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0010690 con 0x7fd8e0108580 2024-04-23T14:55:17.409 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.405+0000 7fd8e6000640 1 -- 172.21.15.83:0/3461518128 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e01ab8a0 con 0x7fd8e0108580 2024-04-23T14:55:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.408+0000 7fd8ea3b3740 1 Processor -- start 2024-04-23T14:55:17.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.408+0000 7fd8de000640 1 -- 172.21.15.83:0/3461518128 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd8d00037a0 con 0x7fd8e0108580 2024-04-23T14:55:17.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.409+0000 7fd8ea3b3740 1 -- start start 2024-04-23T14:55:17.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.409+0000 7fd8ea3b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc70f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.409+0000 7fd8ea3b3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc71460 0x55942bc7b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.409+0000 7fd8ea3b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.409+0000 7fd8ea3b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55942bbd61d0 con 0x55942bc70b40 2024-04-23T14:55:17.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.409+0000 7fd8ea3b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55942bb7c450 con 0x55942bc7bb30 2024-04-23T14:55:17.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.409+0000 7fd8ea3b3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55942bbd2190 con 0x55942bc71460 2024-04-23T14:55:17.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.409+0000 7fd8e5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc70f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.409+0000 7fd8e4c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc71460 0x55942bc7b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.409+0000 7fd8e5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc70f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48594/0 (socket says 172.21.15.83:48594) 2024-04-23T14:55:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.409+0000 7fd8e5600640 1 -- 172.21.15.83:0/2093686416 learned_addr learned my addr 172.21.15.83:0/2093686416 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.409+0000 7fd8d7e00640 1 --2- 172.21.15.83:0/2093686416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.410+0000 7fd8e5600640 1 -- 172.21.15.83:0/2093686416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc71460 msgr2=0x55942bc7b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.410+0000 7fd8e5600640 1 --2- 172.21.15.83:0/2093686416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc71460 0x55942bc7b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.410+0000 7fd8e5600640 1 -- 172.21.15.83:0/2093686416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 msgr2=0x55942bc7df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.410+0000 7fd8e5600640 1 --2- 172.21.15.83:0/2093686416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.410+0000 7fd8e5600640 1 -- 172.21.15.83:0/2093686416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55942bc39310 con 0x55942bc70b40 2024-04-23T14:55:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.410+0000 7fd8de000640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd8c40a3d20 0x7fd8c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.410+0000 7fd8e5600640 1 --2- 172.21.15.83:0/2093686416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc70f20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80113e0 tx=0x7fd8d80118a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb59e97baabeb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.410+0000 7fd8de000640 1 -- 172.21.15.83:0/3461518128 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fd8d00d0c90 con 0x7fd8e0108580 2024-04-23T14:55:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.410+0000 7fd8e4c00640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd8c40a3d20 0x7fd8c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.411+0000 7fd8d5c00640 1 -- 172.21.15.83:0/2093686416 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8012770 con 0x55942bc70b40 2024-04-23T14:55:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.411+0000 7fd8d5c00640 1 -- 172.21.15.83:0/2093686416 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8d8012df0 con 0x55942bc70b40 2024-04-23T14:55:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.411+0000 7fd8d5c00640 1 -- 172.21.15.83:0/2093686416 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d801a3f0 con 0x55942bc70b40 2024-04-23T14:55:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.411+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/2093686416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 msgr2=0x55942bc70f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.411+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/2093686416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc70f20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80113e0 tx=0x7fd8d80118a0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.411+0000 7fd8e4c00640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd8c40a3d20 0x7fd8c40a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55942bc72100 tx=0x7fd8d00d4740 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.411+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/2093686416 shutdown_connections 2024-04-23T14:55:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.411+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/2093686416 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.411+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/2093686416 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc71460 0x55942bc7b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.411+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/2093686416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc70f20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.411+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/2093686416 >> 172.21.15.83:0/2093686416 conn(0x55942bc6ce60 msgr2=0x55942bc6d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.412+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/2093686416 shutdown_connections 2024-04-23T14:55:17.416 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.412+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/2093686416 wait complete. 2024-04-23T14:55:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.412+0000 7fd8ea3b3740 1 -- start start 2024-04-23T14:55:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.412+0000 7fd8ea3b3740 1 Processor -- start 2024-04-23T14:55:17.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8ea3b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc7b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8ea3b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc71460 0x55942bc78880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8ea3b3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc7bb30 0x55942bc78fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8ea3b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55942bc7e9c0 con 0x55942bc70b40 2024-04-23T14:55:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8ea3b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55942bb8f810 con 0x55942bc71460 2024-04-23T14:55:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8ea3b3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55942bc7eb40 con 0x55942bc7bb30 2024-04-23T14:55:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8ea3b3740 10 client.0 _ll_register_callbacks cb 0x55942bb5d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8d7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc71460 0x55942bc78880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.421 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8e5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc7b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8d7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc71460 0x55942bc78880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38224/0 (socket says 172.21.15.83:38224) 2024-04-23T14:55:17.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8d7e00640 1 -- 172.21.15.83:0/770022912 learned_addr learned my addr 172.21.15.83:0/770022912 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8e5600640 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc7bb30 0x55942bc78fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8d7e00640 1 -- 172.21.15.83:0/770022912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc7bb30 msgr2=0x55942bc78fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8d7e00640 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc7bb30 0x55942bc78fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8d7e00640 1 -- 172.21.15.83:0/770022912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 msgr2=0x55942bc7b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8d7e00640 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc7b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.422 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8d7e00640 1 -- 172.21.15.83:0/770022912 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55942bba03f0 con 0x55942bc71460 2024-04-23T14:55:17.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.413+0000 7fd8e5600640 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc7b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:17.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.414+0000 7fd8d7e00640 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc71460 0x55942bc78880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd8c802a940 tx=0x7fd8c8031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd43318e97cc5642 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.414+0000 7fd8ce000640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:55:17.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.414+0000 7fd8ce000640 1 -- 172.21.15.83:0/770022912 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8035040 con 0x55942bc71460 2024-04-23T14:55:17.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.414+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/770022912 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8d8012df0 con 0x55942bc71460 2024-04-23T14:55:17.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.414+0000 7fd8ce000640 1 -- 172.21.15.83:0/770022912 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fd8c8004450 con 0x55942bc71460 2024-04-23T14:55:17.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.414+0000 7fd8ce000640 1 -- 172.21.15.83:0/770022912 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8c80035f0 con 0x55942bc71460 2024-04-23T14:55:17.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.414+0000 7fd8ea3b3740 10 client.26546 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:17.423 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.414+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/770022912 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55942b8e83f0 con 0x55942bc71460 2024-04-23T14:55:17.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.415+0000 7fd8ce000640 1 -- 172.21.15.83:0/770022912 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8d8012df0 con 0x55942bc71460 2024-04-23T14:55:17.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.415+0000 7fd8ce000640 1 -- 172.21.15.83:0/770022912 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fd8c803d9b0 con 0x55942bc71460 2024-04-23T14:55:17.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.415+0000 7fd8ea3b3740 10 client.26546 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:17.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.415+0000 7fd8ea3b3740 10 client.26546 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:55:17.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.415+0000 7fd8ea3b3740 20 client.26546 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:17.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.415+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55942bc8bcf0 0x55942bc8e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.415+0000 7fd8ea3b3740 4 client.26546 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:55:17.424 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.415+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/770022912 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55942bbe3bf0 con 0x55942bc8bcf0 2024-04-23T14:55:17.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.416+0000 7fd8e5600640 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55942bc8bcf0 0x55942bc8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.416+0000 7fd8e5600640 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55942bc8bcf0 0x55942bc8e100 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5312a7598f08882e server_cookie=1419b6a9f7e82169 in_seq=0 out_seq=0 2024-04-23T14:55:17.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.416+0000 7fd8ce000640 10 client.26546 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:55:17.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ce000640 1 -- 172.21.15.83:0/770022912 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55942bbe3bf0 con 0x55942bc8bcf0 2024-04-23T14:55:17.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ce000640 10 client.26546 handle_command_reply: tid=0 2024-04-23T14:55:17.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ea3b3740 1 client.26546 shutdown 2024-04-23T14:55:17.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/770022912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc71460 msgr2=0x55942bc78880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc71460 0x55942bc78880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd8c802a940 tx=0x7fd8c8031850 comp rx=0 tx=0).stop 2024-04-23T14:55:17.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/770022912 shutdown_connections 2024-04-23T14:55:17.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55942bc8bcf0 0x55942bc8e100 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc7bb30 0x55942bc78fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ce000640 0 client.26546 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:55:17.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc71460 0x55942bc78880 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/770022912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc7b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/770022912 >> 172.21.15.83:0/770022912 conn(0x55942bc6ce60 msgr2=0x55942bc72b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.426 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/770022912 shutdown_connections 2024-04-23T14:55:17.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/770022912 wait complete. 2024-04-23T14:55:17.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.417+0000 7fd8ea3b3740 20 client.26546 trim_cache size 0 max 16384 2024-04-23T14:55:17.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.432+0000 7fd8ea3b3740 1 Processor -- start 2024-04-23T14:55:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.432+0000 7fd8ea3b3740 1 -- start start 2024-04-23T14:55:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.432+0000 7fd8ea3b3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc70b40 0x55942bc878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.432+0000 7fd8ea3b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc71460 0x55942bc77860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.432+0000 7fd8ea3b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.432+0000 7fd8ea3b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e01b6a20 con 0x55942bc71460 2024-04-23T14:55:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.432+0000 7fd8ea3b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e01b42f0 con 0x55942bc7bb30 2024-04-23T14:55:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.432+0000 7fd8ea3b3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e01b2160 con 0x55942bc70b40 2024-04-23T14:55:17.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.432+0000 7fd8e5600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc70b40 0x55942bc878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.432+0000 7fd8e5600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc70b40 0x55942bc878b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:39658/0 (socket says 172.21.15.83:39658) 2024-04-23T14:55:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.432+0000 7fd8e5600640 1 -- 172.21.15.83:0/17409813 learned_addr learned my addr 172.21.15.83:0/17409813 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.432+0000 7fd8d7e00640 1 --2- 172.21.15.83:0/17409813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc71460 0x55942bc77860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.433+0000 7fd8e5600640 1 --2- 172.21.15.83:0/17409813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.433+0000 7fd8e5600640 1 -- 172.21.15.83:0/17409813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 msgr2=0x55942bc7b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.433+0000 7fd8e5600640 1 --2- 172.21.15.83:0/17409813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.433+0000 7fd8e5600640 1 -- 172.21.15.83:0/17409813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc71460 msgr2=0x55942bc77860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.433+0000 7fd8e5600640 1 --2- 172.21.15.83:0/17409813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc71460 0x55942bc77860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.433+0000 7fd8e5600640 1 -- 172.21.15.83:0/17409813 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55942b9b27d0 con 0x55942bc70b40 2024-04-23T14:55:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.433+0000 7fd8e5600640 1 --2- 172.21.15.83:0/17409813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc70b40 0x55942bc878b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8006010 tx=0x7fd8d800fa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c31f516d71b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.433+0000 7fd8d5200640 1 -- 172.21.15.83:0/17409813 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d800fbd0 con 0x55942bc70b40 2024-04-23T14:55:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.433+0000 7fd8d5200640 1 -- 172.21.15.83:0/17409813 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8d800fd70 con 0x55942bc70b40 2024-04-23T14:55:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.433+0000 7fd8d5200640 1 -- 172.21.15.83:0/17409813 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8012dd0 con 0x55942bc70b40 2024-04-23T14:55:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.434+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/17409813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc70b40 msgr2=0x55942bc878b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.434+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/17409813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc70b40 0x55942bc878b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8006010 tx=0x7fd8d800fa50 comp rx=0 tx=0).stop 2024-04-23T14:55:17.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.434+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/17409813 shutdown_connections 2024-04-23T14:55:17.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.434+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/17409813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.434+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/17409813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc71460 0x55942bc77860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.434+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/17409813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc70b40 0x55942bc878b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.434+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/17409813 >> 172.21.15.83:0/17409813 conn(0x55942bc6ce60 msgr2=0x55942bc77150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.434+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/17409813 shutdown_connections 2024-04-23T14:55:17.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.434+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/17409813 wait complete. 2024-04-23T14:55:17.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.435+0000 7fd8ea3b3740 1 -- start start 2024-04-23T14:55:17.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.435+0000 7fd8ea3b3740 1 Processor -- start 2024-04-23T14:55:17.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.435+0000 7fd8ea3b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc84290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.435+0000 7fd8ea3b3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc71460 0x55942bc7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.435+0000 7fd8ea3b3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.435+0000 7fd8ea3b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e01b27b0 con 0x55942bc70b40 2024-04-23T14:55:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.435+0000 7fd8ea3b3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e01b3c20 con 0x55942bc7bb30 2024-04-23T14:55:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.435+0000 7fd8ea3b3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e01b2e80 con 0x55942bc71460 2024-04-23T14:55:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.435+0000 7fd8ea3b3740 10 client.0 _ll_register_callbacks cb 0x7fd8c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8e5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc84290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8d7e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc71460 0x55942bc7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8e5600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc84290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48612/0 (socket says 172.21.15.83:48612) 2024-04-23T14:55:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8e5600640 1 -- 172.21.15.83:0/3924716088 learned_addr learned my addr 172.21.15.83:0/3924716088 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8e5600640 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8d7e00640 1 -- 172.21.15.83:0/3924716088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 msgr2=0x55942bc7cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8d7e00640 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8d7e00640 1 -- 172.21.15.83:0/3924716088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 msgr2=0x55942bc84290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8d7e00640 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc84290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8d7e00640 1 -- 172.21.15.83:0/3924716088 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55942bc7d400 con 0x55942bc71460 2024-04-23T14:55:17.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8e5600640 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:55:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8d7e00640 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc71460 0x55942bc7c630 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd8c803ddb0 tx=0x7fd8c8031b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7db00dbe6bee3dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.436+0000 7fd8ce000640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:55:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ce000640 1 -- 172.21.15.83:0/3924716088 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c805d060 con 0x55942bc71460 2024-04-23T14:55:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/3924716088 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8d800fd70 con 0x55942bc71460 2024-04-23T14:55:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ce000640 1 -- 172.21.15.83:0/3924716088 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fd8c8005240 con 0x55942bc71460 2024-04-23T14:55:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ce000640 1 -- 172.21.15.83:0/3924716088 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8c8035070 con 0x55942bc71460 2024-04-23T14:55:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ea3b3740 10 client.26524 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/3924716088 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55942bcf22a0 con 0x55942bc71460 2024-04-23T14:55:17.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ce000640 1 -- 172.21.15.83:0/3924716088 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8d800fd70 con 0x55942bc71460 2024-04-23T14:55:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ce000640 1 -- 172.21.15.83:0/3924716088 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fd8c8057070 con 0x55942bc71460 2024-04-23T14:55:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ea3b3740 10 client.26524 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ea3b3740 10 client.26524 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:55:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ea3b3740 20 client.26524 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55942bcf2a20 0x55942bcf4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ea3b3740 4 client.26524 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:55:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.437+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/3924716088 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55942bbdda60 con 0x55942bcf2a20 2024-04-23T14:55:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.438+0000 7fd8e5600640 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55942bcf2a20 0x55942bcf4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.438+0000 7fd8e5600640 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55942bcf2a20 0x55942bcf4e40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d71c40ed236778a5 server_cookie=a398056914b90059 in_seq=0 out_seq=0 2024-04-23T14:55:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.438+0000 7fd8ce000640 10 client.26524 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:55:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.438+0000 7fd8ce000640 1 -- 172.21.15.83:0/3924716088 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55942bbdda60 con 0x55942bcf2a20 2024-04-23T14:55:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.438+0000 7fd8ce000640 10 client.26524 handle_command_reply: tid=0 2024-04-23T14:55:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.438+0000 7fd8ea3b3740 1 client.26524 shutdown 2024-04-23T14:55:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.439+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/3924716088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc71460 msgr2=0x55942bc7c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.439+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc71460 0x55942bc7c630 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd8c803ddb0 tx=0x7fd8c8031b00 comp rx=0 tx=0).stop 2024-04-23T14:55:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.439+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/3924716088 shutdown_connections 2024-04-23T14:55:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.439+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55942bcf2a20 0x55942bcf4e40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.439+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55942bc7bb30 0x55942bc7cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.439+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55942bc71460 0x55942bc7c630 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.439+0000 7fd8ea3b3740 1 --2- 172.21.15.83:0/3924716088 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55942bc70b40 0x55942bc84290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.439+0000 7fd8ce000640 0 client.26524 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:55:17.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.439+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/3924716088 >> 172.21.15.83:0/3924716088 conn(0x55942bc6ce60 msgr2=0x55942bc72290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.439+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/3924716088 shutdown_connections 2024-04-23T14:55:17.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.439+0000 7fd8ea3b3740 1 -- 172.21.15.83:0/3924716088 wait complete. 2024-04-23T14:55:17.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.439+0000 7fd8ea3b3740 20 client.26524 trim_cache size 0 max 16384 2024-04-23T14:55:17.448 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:55:17.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.440+0000 7fd8e6000640 1 -- 172.21.15.83:0/3461518128 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd8c40a3d20 msgr2=0x7fd8c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.440+0000 7fd8e6000640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd8c40a3d20 0x7fd8c40a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x55942bc72100 tx=0x7fd8d00d4740 comp rx=0 tx=0).stop 2024-04-23T14:55:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.440+0000 7fd8e6000640 1 -- 172.21.15.83:0/3461518128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0108580 msgr2=0x7fd8e019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.440+0000 7fd8e6000640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0108580 0x7fd8e019ed30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd8d000e9c0 tx=0x7fd8d000ee80 comp rx=0 tx=0).stop 2024-04-23T14:55:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.440+0000 7fd8e6000640 1 -- 172.21.15.83:0/3461518128 shutdown_connections 2024-04-23T14:55:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.440+0000 7fd8e6000640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8e010d500 0x7fd8e01a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.440+0000 7fd8e6000640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd8e0108f40 0x7fd8e019f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.440+0000 7fd8e6000640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd8c40a3d20 0x7fd8c40a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.440+0000 7fd8e6000640 1 --2- 172.21.15.83:0/3461518128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0108580 0x7fd8e019ed30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.440+0000 7fd8e6000640 1 -- 172.21.15.83:0/3461518128 >> 172.21.15.83:0/3461518128 conn(0x7fd8e0077e20 msgr2=0x7fd8e0078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.440+0000 7fd8e6000640 1 -- 172.21.15.83:0/3461518128 shutdown_connections 2024-04-23T14:55:17.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.440+0000 7fd8e6000640 1 -- 172.21.15.83:0/3461518128 wait complete. 2024-04-23T14:55:17.459 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:55:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.671+0000 7fb26d800640 1 -- 172.21.15.83:0/605680894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb268108fd0 msgr2=0x7fb268115490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.671+0000 7fb26d800640 1 --2- 172.21.15.83:0/605680894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb268108fd0 0x7fb268115490 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb260009cd0 tx=0x7fb26002a8e0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.672+0000 7fb26d800640 1 -- 172.21.15.83:0/605680894 shutdown_connections 2024-04-23T14:55:17.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.672+0000 7fb26d800640 1 --2- 172.21.15.83:0/605680894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb268108fd0 0x7fb268115490 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.672+0000 7fb26d800640 1 --2- 172.21.15.83:0/605680894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb268108610 0x7fb268108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.672+0000 7fb26d800640 1 --2- 172.21.15.83:0/605680894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb26810e660 0x7fb26810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.672+0000 7fb26d800640 1 -- 172.21.15.83:0/605680894 >> 172.21.15.83:0/605680894 conn(0x7fb268076540 msgr2=0x7fb268078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.672+0000 7fb26d800640 1 -- 172.21.15.83:0/605680894 shutdown_connections 2024-04-23T14:55:17.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.672+0000 7fb26d800640 1 -- 172.21.15.83:0/605680894 wait complete. 2024-04-23T14:55:17.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.673+0000 7fb26d800640 1 Processor -- start 2024-04-23T14:55:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.673+0000 7fb26d800640 1 -- start start 2024-04-23T14:55:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.673+0000 7fb26d800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb268108610 0x7fb26819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.673+0000 7fb26d800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb268108fd0 0x7fb26819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.673+0000 7fb26d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26810e660 0x7fb2681a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.673+0000 7fb26d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681189f0 con 0x7fb26810e660 2024-04-23T14:55:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.673+0000 7fb26d800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb268118870 con 0x7fb268108610 2024-04-23T14:55:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.673+0000 7fb26d800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb268118b70 con 0x7fb268108fd0 2024-04-23T14:55:17.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.673+0000 7fb267400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26810e660 0x7fb2681a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.673+0000 7fb267400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26810e660 0x7fb2681a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48636/0 (socket says 172.21.15.83:48636) 2024-04-23T14:55:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.673+0000 7fb267400640 1 -- 172.21.15.83:0/3702618157 learned_addr learned my addr 172.21.15.83:0/3702618157 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.674+0000 7fb25fe00640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb268108fd0 0x7fb26819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.674+0000 7fb266a00640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb268108610 0x7fb26819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.674+0000 7fb267400640 1 -- 172.21.15.83:0/3702618157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb268108fd0 msgr2=0x7fb26819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.674+0000 7fb267400640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb268108fd0 0x7fb26819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.674+0000 7fb267400640 1 -- 172.21.15.83:0/3702618157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb268108610 msgr2=0x7fb26819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.674+0000 7fb267400640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb268108610 0x7fb26819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.674+0000 7fb267400640 1 -- 172.21.15.83:0/3702618157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2681a3e00 con 0x7fb26810e660 2024-04-23T14:55:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.674+0000 7fb267400640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26810e660 0x7fb2681a36d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb260033040 tx=0x7fb260031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.675+0000 7fb25f400640 1 -- 172.21.15.83:0/3702618157 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260004040 con 0x7fb26810e660 2024-04-23T14:55:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.675+0000 7fb26d800640 1 -- 172.21.15.83:0/3702618157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2681a4090 con 0x7fb26810e660 2024-04-23T14:55:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.675+0000 7fb25f400640 1 -- 172.21.15.83:0/3702618157 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2600046c0 con 0x7fb26810e660 2024-04-23T14:55:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.675+0000 7fb25f400640 1 -- 172.21.15.83:0/3702618157 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260030ed0 con 0x7fb26810e660 2024-04-23T14:55:17.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.675+0000 7fb26d800640 1 -- 172.21.15.83:0/3702618157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2681ab980 con 0x7fb26810e660 2024-04-23T14:55:17.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.679+0000 7fb27079e740 1 Processor -- start 2024-04-23T14:55:17.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.679+0000 7fb25f400640 1 -- 172.21.15.83:0/3702618157 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb2600041e0 con 0x7fb26810e660 2024-04-23T14:55:17.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.679+0000 7fb27079e740 1 -- start start 2024-04-23T14:55:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.679+0000 7fb27079e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2ca6950 0x5619d2ca6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.679+0000 7fb27079e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cb1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb27079e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2cb1940 0x5619d2cb3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb27079e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5619d2c0c1d0 con 0x5619d2ca7270 2024-04-23T14:55:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb27079e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5619d2bb2450 con 0x5619d2ca6950 2024-04-23T14:55:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb27079e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5619d2c08190 con 0x5619d2cb1940 2024-04-23T14:55:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb25fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2ca6950 0x5619d2ca6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb266a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cb1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb25fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2ca6950 0x5619d2ca6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38262/0 (socket says 172.21.15.83:38262) 2024-04-23T14:55:17.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb25fe00640 1 -- 172.21.15.83:0/1960722474 learned_addr learned my addr 172.21.15.83:0/1960722474 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb25fe00640 1 --2- 172.21.15.83:0/1960722474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2cb1940 0x5619d2cb3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb25fe00640 1 -- 172.21.15.83:0/1960722474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2cb1940 msgr2=0x5619d2cb3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb25fe00640 1 --2- 172.21.15.83:0/1960722474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2cb1940 0x5619d2cb3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb25fe00640 1 -- 172.21.15.83:0/1960722474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 msgr2=0x5619d2cb1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb25fe00640 1 --2- 172.21.15.83:0/1960722474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cb1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb25fe00640 1 -- 172.21.15.83:0/1960722474 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d2c6f310 con 0x5619d2ca6950 2024-04-23T14:55:17.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.680+0000 7fb25f400640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb24c0a3d20 0x7fb24c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.681+0000 7fb25fe00640 1 --2- 172.21.15.83:0/1960722474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2ca6950 0x5619d2ca6d30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb2500058c0 tx=0x7fb250005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd5f0c5660f0492 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.681+0000 7fb25f400640 1 -- 172.21.15.83:0/3702618157 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fb26002e030 con 0x7fb26810e660 2024-04-23T14:55:17.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.681+0000 7fb266a00640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb24c0a3d20 0x7fb24c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.681+0000 7fb257e00640 1 -- 172.21.15.83:0/1960722474 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250004030 con 0x5619d2ca6950 2024-04-23T14:55:17.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.681+0000 7fb257e00640 1 -- 172.21.15.83:0/1960722474 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb250005500 con 0x5619d2ca6950 2024-04-23T14:55:17.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.681+0000 7fb257e00640 1 -- 172.21.15.83:0/1960722474 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250004d30 con 0x5619d2ca6950 2024-04-23T14:55:17.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.682+0000 7fb27079e740 1 -- 172.21.15.83:0/1960722474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2ca6950 msgr2=0x5619d2ca6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.682+0000 7fb266a00640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb24c0a3d20 0x7fb24c0a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5619d2ca7f10 tx=0x7fb258005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.682+0000 7fb27079e740 1 --2- 172.21.15.83:0/1960722474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2ca6950 0x5619d2ca6d30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb2500058c0 tx=0x7fb250005850 comp rx=0 tx=0).stop 2024-04-23T14:55:17.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.682+0000 7fb27079e740 1 -- 172.21.15.83:0/1960722474 shutdown_connections 2024-04-23T14:55:17.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.682+0000 7fb27079e740 1 --2- 172.21.15.83:0/1960722474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2cb1940 0x5619d2cb3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.682+0000 7fb27079e740 1 --2- 172.21.15.83:0/1960722474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cb1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.682+0000 7fb27079e740 1 --2- 172.21.15.83:0/1960722474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2ca6950 0x5619d2ca6d30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.682+0000 7fb27079e740 1 -- 172.21.15.83:0/1960722474 >> 172.21.15.83:0/1960722474 conn(0x5619d2ca2c70 msgr2=0x5619d2ca3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.682+0000 7fb27079e740 1 -- 172.21.15.83:0/1960722474 shutdown_connections 2024-04-23T14:55:17.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.682+0000 7fb27079e740 1 -- 172.21.15.83:0/1960722474 wait complete. 2024-04-23T14:55:17.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.682+0000 7fb27079e740 1 -- start start 2024-04-23T14:55:17.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb27079e740 1 Processor -- start 2024-04-23T14:55:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb27079e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca6950 0x5619d2cb1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb27079e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2ca7270 0x5619d2cae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb27079e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2cb1940 0x5619d2caec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb27079e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5619d2cb47d0 con 0x5619d2ca6950 2024-04-23T14:55:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb27079e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5619d2bc5810 con 0x5619d2ca7270 2024-04-23T14:55:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb27079e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5619d2cb4950 con 0x5619d2cb1940 2024-04-23T14:55:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb27079e740 10 client.0 _ll_register_callbacks cb 0x5619d2b93140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb266a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca6950 0x5619d2cb1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb25fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2ca7270 0x5619d2cae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb266a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca6950 0x5619d2cb1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48654/0 (socket says 172.21.15.83:48654) 2024-04-23T14:55:17.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb266a00640 1 -- 172.21.15.83:0/4179448439 learned_addr learned my addr 172.21.15.83:0/4179448439 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.683+0000 7fb267400640 1 --2- 172.21.15.83:0/4179448439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2cb1940 0x5619d2caec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.684+0000 7fb266a00640 1 -- 172.21.15.83:0/4179448439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2cb1940 msgr2=0x5619d2caec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.684+0000 7fb266a00640 1 --2- 172.21.15.83:0/4179448439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2cb1940 0x5619d2caec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.684+0000 7fb266a00640 1 -- 172.21.15.83:0/4179448439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2ca7270 msgr2=0x5619d2cae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.684+0000 7fb266a00640 1 --2- 172.21.15.83:0/4179448439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2ca7270 0x5619d2cae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.684+0000 7fb266a00640 1 -- 172.21.15.83:0/4179448439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d2bd63f0 con 0x5619d2ca6950 2024-04-23T14:55:17.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.684+0000 7fb266a00640 1 --2- 172.21.15.83:0/4179448439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca6950 0x5619d2cb1310 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb258013010 tx=0x7fb258004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c756672cd5cd516 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.684+0000 7fb254c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:55:17.693 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.684+0000 7fb254c00640 1 -- 172.21.15.83:0/4179448439 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25801f630 con 0x5619d2ca6950 2024-04-23T14:55:17.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.684+0000 7fb254c00640 1 -- 172.21.15.83:0/4179448439 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fb25801f7d0 con 0x5619d2ca6950 2024-04-23T14:55:17.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.684+0000 7fb27079e740 1 -- 172.21.15.83:0/4179448439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb250005500 con 0x5619d2ca6950 2024-04-23T14:55:17.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.685+0000 7fb254c00640 1 -- 172.21.15.83:0/4179448439 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb25801f9d0 con 0x5619d2ca6950 2024-04-23T14:55:17.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.685+0000 7fb27079e740 10 client.16359 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:17.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.685+0000 7fb27079e740 1 -- 172.21.15.83:0/4179448439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5619d291e3f0 con 0x5619d2ca6950 2024-04-23T14:55:17.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.685+0000 7fb254c00640 1 -- 172.21.15.83:0/4179448439 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb250005500 con 0x5619d2ca6950 2024-04-23T14:55:17.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.685+0000 7fb254c00640 1 -- 172.21.15.83:0/4179448439 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fb258027070 con 0x5619d2ca6950 2024-04-23T14:55:17.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.685+0000 7fb27079e740 10 client.16359 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.685+0000 7fb27079e740 10 client.16359 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:55:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.685+0000 7fb27079e740 20 client.16359 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.685+0000 7fb27079e740 1 --2- 172.21.15.83:0/4179448439 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5619d2cc1bc0 0x5619d2cc4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.685+0000 7fb27079e740 4 client.16359 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:55:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.685+0000 7fb27079e740 1 -- 172.21.15.83:0/4179448439 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619d2c19bf0 con 0x5619d2cc1bc0 2024-04-23T14:55:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.686+0000 7fb25fe00640 1 --2- 172.21.15.83:0/4179448439 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5619d2cc1bc0 0x5619d2cc4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.686+0000 7fb25fe00640 1 --2- 172.21.15.83:0/4179448439 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5619d2cc1bc0 0x5619d2cc4020 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c76edde761af2d9 server_cookie=74b6bd9cbfdbe659 in_seq=0 out_seq=0 2024-04-23T14:55:17.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.686+0000 7fb254c00640 10 client.16359 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:55:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb254c00640 1 -- 172.21.15.83:0/4179448439 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619d2c19bf0 con 0x5619d2cc1bc0 2024-04-23T14:55:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb254c00640 10 client.16359 handle_command_reply: tid=0 2024-04-23T14:55:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb27079e740 1 client.16359 shutdown 2024-04-23T14:55:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb27079e740 1 -- 172.21.15.83:0/4179448439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca6950 msgr2=0x5619d2cb1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb27079e740 1 --2- 172.21.15.83:0/4179448439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca6950 0x5619d2cb1310 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb258013010 tx=0x7fb258004730 comp rx=0 tx=0).stop 2024-04-23T14:55:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb27079e740 1 -- 172.21.15.83:0/4179448439 shutdown_connections 2024-04-23T14:55:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb27079e740 1 --2- 172.21.15.83:0/4179448439 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5619d2cc1bc0 0x5619d2cc4020 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb27079e740 1 --2- 172.21.15.83:0/4179448439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2cb1940 0x5619d2caec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb27079e740 1 --2- 172.21.15.83:0/4179448439 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2ca7270 0x5619d2cae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb254c00640 0 client.16359 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:55:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb27079e740 1 --2- 172.21.15.83:0/4179448439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca6950 0x5619d2cb1310 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb27079e740 1 -- 172.21.15.83:0/4179448439 >> 172.21.15.83:0/4179448439 conn(0x5619d2ca2c70 msgr2=0x5619d2ca8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.687+0000 7fb27079e740 1 -- 172.21.15.83:0/4179448439 shutdown_connections 2024-04-23T14:55:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.688+0000 7fb27079e740 1 -- 172.21.15.83:0/4179448439 wait complete. 2024-04-23T14:55:17.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.688+0000 7fb27079e740 20 client.16359 trim_cache size 0 max 16384 2024-04-23T14:55:17.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.702+0000 7fb27079e740 1 Processor -- start 2024-04-23T14:55:17.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.702+0000 7fb27079e740 1 -- start start 2024-04-23T14:55:17.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.702+0000 7fb27079e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2ca6950 0x5619d2cbd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.702+0000 7fb27079e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.702+0000 7fb27079e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2cb1940 0x5619d2cb0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.702+0000 7fb27079e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b6b80 con 0x5619d2ca7270 2024-04-23T14:55:17.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.702+0000 7fb27079e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b4450 con 0x5619d2cb1940 2024-04-23T14:55:17.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.702+0000 7fb27079e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b22c0 con 0x5619d2ca6950 2024-04-23T14:55:17.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.702+0000 7fb25fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb25fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cad5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48660/0 (socket says 172.21.15.83:48660) 2024-04-23T14:55:17.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb25fe00640 1 -- 172.21.15.83:0/2798774854 learned_addr learned my addr 172.21.15.83:0/2798774854 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb266a00640 1 --2- 172.21.15.83:0/2798774854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2ca6950 0x5619d2cbd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb267400640 1 --2- 172.21.15.83:0/2798774854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2cb1940 0x5619d2cb0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb25fe00640 1 -- 172.21.15.83:0/2798774854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2ca6950 msgr2=0x5619d2cbd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb25fe00640 1 --2- 172.21.15.83:0/2798774854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2ca6950 0x5619d2cbd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb25fe00640 1 -- 172.21.15.83:0/2798774854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2cb1940 msgr2=0x5619d2cb0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb25fe00640 1 --2- 172.21.15.83:0/2798774854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2cb1940 0x5619d2cb0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb25fe00640 1 -- 172.21.15.83:0/2798774854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d29e87d0 con 0x5619d2ca7270 2024-04-23T14:55:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb25fe00640 1 --2- 172.21.15.83:0/2798774854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cad5e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb2500049c0 tx=0x7fb250009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57c20aafa9cc05f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb257400640 1 -- 172.21.15.83:0/2798774854 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25000f9d0 con 0x5619d2ca7270 2024-04-23T14:55:17.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb257400640 1 -- 172.21.15.83:0/2798774854 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25002b910 con 0x5619d2ca7270 2024-04-23T14:55:17.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.703+0000 7fb257400640 1 -- 172.21.15.83:0/2798774854 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25001f840 con 0x5619d2ca7270 2024-04-23T14:55:17.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.704+0000 7fb27079e740 1 -- 172.21.15.83:0/2798774854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 msgr2=0x5619d2cad5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.704+0000 7fb27079e740 1 --2- 172.21.15.83:0/2798774854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cad5e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb2500049c0 tx=0x7fb250009fd0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.704+0000 7fb27079e740 1 -- 172.21.15.83:0/2798774854 shutdown_connections 2024-04-23T14:55:17.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.704+0000 7fb27079e740 1 --2- 172.21.15.83:0/2798774854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2cb1940 0x5619d2cb0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.704+0000 7fb27079e740 1 --2- 172.21.15.83:0/2798774854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cad5e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.704+0000 7fb27079e740 1 --2- 172.21.15.83:0/2798774854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2ca6950 0x5619d2cbd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.704+0000 7fb27079e740 1 -- 172.21.15.83:0/2798774854 >> 172.21.15.83:0/2798774854 conn(0x5619d2ca2c70 msgr2=0x5619d2caced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.704+0000 7fb27079e740 1 -- 172.21.15.83:0/2798774854 shutdown_connections 2024-04-23T14:55:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.704+0000 7fb27079e740 1 -- 172.21.15.83:0/2798774854 wait complete. 2024-04-23T14:55:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.705+0000 7fb27079e740 1 -- start start 2024-04-23T14:55:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.705+0000 7fb27079e740 1 Processor -- start 2024-04-23T14:55:17.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.705+0000 7fb27079e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2ca6950 0x5619d2cb1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.705+0000 7fb27079e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cb22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.705+0000 7fb27079e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2cb1940 0x5619d2cb2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.705+0000 7fb27079e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b2910 con 0x5619d2ca7270 2024-04-23T14:55:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.705+0000 7fb27079e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b3d80 con 0x5619d2cb1940 2024-04-23T14:55:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.705+0000 7fb27079e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b2fe0 con 0x5619d2ca6950 2024-04-23T14:55:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.705+0000 7fb27079e740 10 client.0 _ll_register_callbacks cb 0x7fb258009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.705+0000 7fb25fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cb22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.706+0000 7fb25fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cb22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48662/0 (socket says 172.21.15.83:48662) 2024-04-23T14:55:17.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.706+0000 7fb266a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2ca6950 0x5619d2cb1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.706+0000 7fb25fe00640 1 -- 172.21.15.83:0/1667638900 learned_addr learned my addr 172.21.15.83:0/1667638900 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.706+0000 7fb267400640 1 --2- 172.21.15.83:0/1667638900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2cb1940 0x5619d2cb2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.706+0000 7fb25fe00640 1 -- 172.21.15.83:0/1667638900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2ca6950 msgr2=0x5619d2cb1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.706+0000 7fb25fe00640 1 --2- 172.21.15.83:0/1667638900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2ca6950 0x5619d2cb1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.706+0000 7fb25fe00640 1 -- 172.21.15.83:0/1667638900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2cb1940 msgr2=0x5619d2cb2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.706+0000 7fb25fe00640 1 --2- 172.21.15.83:0/1667638900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2cb1940 0x5619d2cb2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.706+0000 7fb25fe00640 1 -- 172.21.15.83:0/1667638900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d2cb8e40 con 0x5619d2ca7270 2024-04-23T14:55:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.706+0000 7fb25fe00640 1 --2- 172.21.15.83:0/1667638900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cb22c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb2500117f0 tx=0x7fb250004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd7ccd00e314ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.706+0000 7fb254c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:55:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.706+0000 7fb254c00640 1 -- 172.21.15.83:0/1667638900 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb250039020 con 0x5619d2ca7270 2024-04-23T14:55:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb254c00640 1 -- 172.21.15.83:0/1667638900 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fb250011040 con 0x5619d2ca7270 2024-04-23T14:55:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb27079e740 1 -- 172.21.15.83:0/1667638900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb25002b910 con 0x5619d2ca7270 2024-04-23T14:55:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb254c00640 1 -- 172.21.15.83:0/1667638900 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb25002f040 con 0x5619d2ca7270 2024-04-23T14:55:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb27079e740 10 client.16365 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb27079e740 1 -- 172.21.15.83:0/1667638900 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5619d2cba140 con 0x5619d2ca7270 2024-04-23T14:55:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb254c00640 1 -- 172.21.15.83:0/1667638900 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb25002b910 con 0x5619d2ca7270 2024-04-23T14:55:17.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb254c00640 1 -- 172.21.15.83:0/1667638900 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fb25001f440 con 0x5619d2ca7270 2024-04-23T14:55:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb27079e740 10 client.16365 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb27079e740 10 client.16365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:55:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb27079e740 20 client.16365 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb27079e740 1 --2- 172.21.15.83:0/1667638900 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5619d2d28160 0x5619d2d2a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb27079e740 4 client.16365 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:55:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.707+0000 7fb27079e740 1 -- 172.21.15.83:0/1667638900 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619d2c13a60 con 0x5619d2d28160 2024-04-23T14:55:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.708+0000 7fb266a00640 1 --2- 172.21.15.83:0/1667638900 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5619d2d28160 0x5619d2d2a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.708+0000 7fb266a00640 1 --2- 172.21.15.83:0/1667638900 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5619d2d28160 0x5619d2d2a550 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ee05fe49deefd0e server_cookie=6b82ef47144c360c in_seq=0 out_seq=0 2024-04-23T14:55:17.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.708+0000 7fb254c00640 10 client.16365 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:55:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.709+0000 7fb254c00640 1 -- 172.21.15.83:0/1667638900 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619d2c13a60 con 0x5619d2d28160 2024-04-23T14:55:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.709+0000 7fb254c00640 10 client.16365 handle_command_reply: tid=0 2024-04-23T14:55:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.709+0000 7fb27079e740 1 client.16365 shutdown 2024-04-23T14:55:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.709+0000 7fb27079e740 1 -- 172.21.15.83:0/1667638900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 msgr2=0x5619d2cb22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.709+0000 7fb27079e740 1 --2- 172.21.15.83:0/1667638900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cb22c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb2500117f0 tx=0x7fb250004860 comp rx=0 tx=0).stop 2024-04-23T14:55:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.709+0000 7fb27079e740 1 -- 172.21.15.83:0/1667638900 shutdown_connections 2024-04-23T14:55:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.709+0000 7fb27079e740 1 --2- 172.21.15.83:0/1667638900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5619d2cb1940 0x5619d2cb2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.709+0000 7fb27079e740 1 --2- 172.21.15.83:0/1667638900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619d2ca7270 0x5619d2cb22c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.709+0000 7fb27079e740 1 --2- 172.21.15.83:0/1667638900 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5619d2d28160 0x5619d2d2a550 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.709+0000 7fb27079e740 1 --2- 172.21.15.83:0/1667638900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5619d2ca6950 0x5619d2cb1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.709+0000 7fb254c00640 0 client.16365 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:55:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.709+0000 7fb27079e740 1 -- 172.21.15.83:0/1667638900 >> 172.21.15.83:0/1667638900 conn(0x5619d2ca2c70 msgr2=0x5619d2ca1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.710+0000 7fb27079e740 1 -- 172.21.15.83:0/1667638900 shutdown_connections 2024-04-23T14:55:17.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.710+0000 7fb27079e740 1 -- 172.21.15.83:0/1667638900 wait complete. 2024-04-23T14:55:17.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.710+0000 7fb27079e740 20 client.16365 trim_cache size 0 max 16384 2024-04-23T14:55:17.718 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:55:17.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.711+0000 7fb26d800640 1 -- 172.21.15.83:0/3702618157 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb24c0a3d20 msgr2=0x7fb24c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.711+0000 7fb26d800640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb24c0a3d20 0x7fb24c0a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5619d2ca7f10 tx=0x7fb258005f90 comp rx=0 tx=0).stop 2024-04-23T14:55:17.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.711+0000 7fb26d800640 1 -- 172.21.15.83:0/3702618157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26810e660 msgr2=0x7fb2681a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.711+0000 7fb26d800640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26810e660 0x7fb2681a36d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb260033040 tx=0x7fb260031e60 comp rx=0 tx=0).stop 2024-04-23T14:55:17.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.711+0000 7fb26d800640 1 -- 172.21.15.83:0/3702618157 shutdown_connections 2024-04-23T14:55:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.711+0000 7fb26d800640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb24c0a3d20 0x7fb24c0a61e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.711+0000 7fb26d800640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb26810e660 0x7fb2681a36d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.711+0000 7fb26d800640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb268108fd0 0x7fb26819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.711+0000 7fb26d800640 1 --2- 172.21.15.83:0/3702618157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb268108610 0x7fb26819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.711+0000 7fb26d800640 1 -- 172.21.15.83:0/3702618157 >> 172.21.15.83:0/3702618157 conn(0x7fb268076540 msgr2=0x7fb268077a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.711+0000 7fb26d800640 1 -- 172.21.15.83:0/3702618157 shutdown_connections 2024-04-23T14:55:17.720 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.711+0000 7fb26d800640 1 -- 172.21.15.83:0/3702618157 wait complete. 2024-04-23T14:55:17.728 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:55:17.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.913+0000 7fa295400640 1 -- 172.21.15.83:0/1189507240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa290104cd0 msgr2=0x7fa290108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.913+0000 7fa295400640 1 --2- 172.21.15.83:0/1189507240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa290104cd0 0x7fa290108d10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009900 tx=0x7fa27c02a410 comp rx=0 tx=0).stop 2024-04-23T14:55:17.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.914+0000 7fa295400640 1 -- 172.21.15.83:0/1189507240 shutdown_connections 2024-04-23T14:55:17.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.914+0000 7fa295400640 1 --2- 172.21.15.83:0/1189507240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2901093f0 0x7fa290110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.914+0000 7fa295400640 1 --2- 172.21.15.83:0/1189507240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa290104cd0 0x7fa290108d10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.914+0000 7fa295400640 1 --2- 172.21.15.83:0/1189507240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa290104320 0x7fa290104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.914+0000 7fa295400640 1 -- 172.21.15.83:0/1189507240 >> 172.21.15.83:0/1189507240 conn(0x7fa2900ffb40 msgr2=0x7fa290101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.914+0000 7fa295400640 1 -- 172.21.15.83:0/1189507240 shutdown_connections 2024-04-23T14:55:17.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.914+0000 7fa295400640 1 -- 172.21.15.83:0/1189507240 wait complete. 2024-04-23T14:55:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.915+0000 7fa295400640 1 Processor -- start 2024-04-23T14:55:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.915+0000 7fa295400640 1 -- start start 2024-04-23T14:55:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.915+0000 7fa295400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa290104320 0x7fa2901a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.915+0000 7fa295400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2901093f0 0x7fa2901a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.915+0000 7fa295400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa29019a880 0x7fa29019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.915+0000 7fa295400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa290114480 con 0x7fa290104320 2024-04-23T14:55:17.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.915+0000 7fa295400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa290114300 con 0x7fa2901093f0 2024-04-23T14:55:17.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.915+0000 7fa295400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa290114600 con 0x7fa29019a880 2024-04-23T14:55:17.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.915+0000 7fa28fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa290104320 0x7fa2901a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.915+0000 7fa28fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa290104320 0x7fa2901a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48684/0 (socket says 172.21.15.83:48684) 2024-04-23T14:55:17.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.915+0000 7fa28fe00640 1 -- 172.21.15.83:0/3817090968 learned_addr learned my addr 172.21.15.83:0/3817090968 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa294a00640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa29019a880 0x7fa29019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa28f400640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2901093f0 0x7fa2901a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa28fe00640 1 -- 172.21.15.83:0/3817090968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa29019a880 msgr2=0x7fa29019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa28fe00640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa29019a880 0x7fa29019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa28fe00640 1 -- 172.21.15.83:0/3817090968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2901093f0 msgr2=0x7fa2901a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa28fe00640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2901093f0 0x7fa2901a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa28fe00640 1 -- 172.21.15.83:0/3817090968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29019b3a0 con 0x7fa290104320 2024-04-23T14:55:17.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa28f400640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2901093f0 0x7fa2901a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:55:17.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa28fe00640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa290104320 0x7fa2901a06d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa28000e9f0 tx=0x7fa28000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa28cc00640 1 -- 172.21.15.83:0/3817090968 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28000cd80 con 0x7fa290104320 2024-04-23T14:55:17.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa28cc00640 1 -- 172.21.15.83:0/3817090968 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa280010910 con 0x7fa290104320 2024-04-23T14:55:17.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa295400640 1 -- 172.21.15.83:0/3817090968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29019b690 con 0x7fa290104320 2024-04-23T14:55:17.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.916+0000 7fa28cc00640 1 -- 172.21.15.83:0/3817090968 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280018990 con 0x7fa290104320 2024-04-23T14:55:17.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.917+0000 7fa295400640 1 -- 172.21.15.83:0/3817090968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2901a7930 con 0x7fa290104320 2024-04-23T14:55:17.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.921+0000 7fa2997ea740 1 Processor -- start 2024-04-23T14:55:17.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.921+0000 7fa2997ea740 1 -- start start 2024-04-23T14:55:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.921+0000 7fa28cc00640 1 -- 172.21.15.83:0/3817090968 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa280010ac0 con 0x7fa290104320 2024-04-23T14:55:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.921+0000 7fa2997ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc00bb40 0x562adc00bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.921+0000 7fa2997ea740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc0165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.921+0000 7fa2997ea740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc016b30 0x562adc018f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.921+0000 7fa2997ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562adbf711d0 con 0x562adc00bb40 2024-04-23T14:55:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.921+0000 7fa2997ea740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562adbf17450 con 0x562adc00c460 2024-04-23T14:55:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.921+0000 7fa2997ea740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562adbf6d190 con 0x562adc016b30 2024-04-23T14:55:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.922+0000 7fa294a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc00bb40 0x562adc00bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.922+0000 7fa294a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc00bb40 0x562adc00bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48698/0 (socket says 172.21.15.83:48698) 2024-04-23T14:55:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.922+0000 7fa28fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc0165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.922+0000 7fa294a00640 1 -- 172.21.15.83:0/3066012405 learned_addr learned my addr 172.21.15.83:0/3066012405 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.924 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.922+0000 7fa28f400640 1 --2- 172.21.15.83:0/3066012405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc016b30 0x562adc018f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.922+0000 7fa294a00640 1 -- 172.21.15.83:0/3066012405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc016b30 msgr2=0x562adc018f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.922+0000 7fa294a00640 1 --2- 172.21.15.83:0/3066012405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc016b30 0x562adc018f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.922+0000 7fa294a00640 1 -- 172.21.15.83:0/3066012405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 msgr2=0x562adc0165f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.922+0000 7fa294a00640 1 --2- 172.21.15.83:0/3066012405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc0165f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.922+0000 7fa294a00640 1 -- 172.21.15.83:0/3066012405 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562adbfd4310 con 0x562adc00bb40 2024-04-23T14:55:17.925 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.923+0000 7fa294a00640 1 --2- 172.21.15.83:0/3066012405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc00bb40 0x562adc00bf20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa288009f50 tx=0x7fa2880118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3fdc5a363969f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.928 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.923+0000 7fa285200640 1 -- 172.21.15.83:0/3066012405 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288004d20 con 0x562adc00bb40 2024-04-23T14:55:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.923+0000 7fa285200640 1 -- 172.21.15.83:0/3066012405 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa288004050 con 0x562adc00bb40 2024-04-23T14:55:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.923+0000 7fa285200640 1 -- 172.21.15.83:0/3066012405 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288005220 con 0x562adc00bb40 2024-04-23T14:55:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.923+0000 7fa28cc00640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa2740a3da0 0x7fa2740a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.924+0000 7fa2997ea740 1 -- 172.21.15.83:0/3066012405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc00bb40 msgr2=0x562adc00bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.924+0000 7fa2997ea740 1 --2- 172.21.15.83:0/3066012405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc00bb40 0x562adc00bf20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa288009f50 tx=0x7fa2880118d0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.924+0000 7fa28cc00640 1 -- 172.21.15.83:0/3817090968 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fa2800d0b40 con 0x7fa290104320 2024-04-23T14:55:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.924+0000 7fa2997ea740 1 -- 172.21.15.83:0/3066012405 shutdown_connections 2024-04-23T14:55:17.929 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.924+0000 7fa2997ea740 1 --2- 172.21.15.83:0/3066012405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc016b30 0x562adc018f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.924+0000 7fa2997ea740 1 --2- 172.21.15.83:0/3066012405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc0165f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.924+0000 7fa2997ea740 1 --2- 172.21.15.83:0/3066012405 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc00bb40 0x562adc00bf20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.924+0000 7fa2997ea740 1 -- 172.21.15.83:0/3066012405 >> 172.21.15.83:0/3066012405 conn(0x562adc007e60 msgr2=0x562adc008240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.924+0000 7fa28fe00640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa2740a3da0 0x7fa2740a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.924+0000 7fa2997ea740 1 -- 172.21.15.83:0/3066012405 shutdown_connections 2024-04-23T14:55:17.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.924+0000 7fa2997ea740 1 -- 172.21.15.83:0/3066012405 wait complete. 2024-04-23T14:55:17.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.925+0000 7fa28fe00640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa2740a3da0 0x7fa2740a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2800ce330 tx=0x7fa2800d4690 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.930 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.925+0000 7fa2997ea740 1 -- start start 2024-04-23T14:55:17.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.925+0000 7fa2997ea740 1 Processor -- start 2024-04-23T14:55:17.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.925+0000 7fa2997ea740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00bb40 0x562adc016590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.925+0000 7fa2997ea740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc013880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.925+0000 7fa2997ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc013fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.925+0000 7fa2997ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562adc0199c0 con 0x562adc016b30 2024-04-23T14:55:17.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.925+0000 7fa2997ea740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562adbf2a810 con 0x562adc00c460 2024-04-23T14:55:17.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.925+0000 7fa2997ea740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562adc019b40 con 0x562adc00bb40 2024-04-23T14:55:17.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.925+0000 7fa2997ea740 10 client.0 _ll_register_callbacks cb 0x562adbef8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:17.931 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa28f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc013880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa294a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00bb40 0x562adc016590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa28f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc013880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38322/0 (socket says 172.21.15.83:38322) 2024-04-23T14:55:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa28f400640 1 -- 172.21.15.83:0/239599939 learned_addr learned my addr 172.21.15.83:0/239599939 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa294a00640 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc013fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa28f400640 1 -- 172.21.15.83:0/239599939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00bb40 msgr2=0x562adc016590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa28f400640 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00bb40 0x562adc016590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa28f400640 1 -- 172.21.15.83:0/239599939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 msgr2=0x562adc013fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa28f400640 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc013fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.932 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa28f400640 1 -- 172.21.15.83:0/239599939 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562adbf3b3f0 con 0x562adc00c460 2024-04-23T14:55:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa294a00640 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc013fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:55:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa28f400640 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc013880 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009900 tx=0x7fa27c031860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a6e21b09f13bd57 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.926+0000 7fa279600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:55:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.927+0000 7fa279600640 1 -- 172.21.15.83:0/239599939 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c035040 con 0x562adc00c460 2024-04-23T14:55:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.927+0000 7fa279600640 1 -- 172.21.15.83:0/239599939 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fa27c004240 con 0x562adc00c460 2024-04-23T14:55:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.927+0000 7fa2997ea740 1 -- 172.21.15.83:0/239599939 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa288004050 con 0x562adc00c460 2024-04-23T14:55:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.928+0000 7fa279600640 1 -- 172.21.15.83:0/239599939 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa27c004460 con 0x562adc00c460 2024-04-23T14:55:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.928+0000 7fa2997ea740 10 client.26579 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:17.933 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.928+0000 7fa2997ea740 1 -- 172.21.15.83:0/239599939 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562adbc833f0 con 0x562adc00c460 2024-04-23T14:55:17.934 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.928+0000 7fa279600640 1 -- 172.21.15.83:0/239599939 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa288004050 con 0x562adc00c460 2024-04-23T14:55:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.928+0000 7fa279600640 1 -- 172.21.15.83:0/239599939 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fa27c004a10 con 0x562adc00c460 2024-04-23T14:55:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.928+0000 7fa2997ea740 10 client.26579 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.928+0000 7fa2997ea740 10 client.26579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.928+0000 7fa2997ea740 20 client.26579 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.928+0000 7fa2997ea740 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562adc026d20 0x562adc029180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.928+0000 7fa2997ea740 4 client.26579 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:55:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.928+0000 7fa2997ea740 1 -- 172.21.15.83:0/239599939 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562adbf7ebf0 con 0x562adc026d20 2024-04-23T14:55:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.929+0000 7fa294a00640 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562adc026d20 0x562adc029180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.936 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.929+0000 7fa294a00640 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562adc026d20 0x562adc029180 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3283bc04034b1c45 server_cookie=ecba6a29d239f31a in_seq=0 out_seq=0 2024-04-23T14:55:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.929+0000 7fa279600640 10 client.26579 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa279600640 1 -- 172.21.15.83:0/239599939 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562adbf7ebf0 con 0x562adc026d20 2024-04-23T14:55:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa279600640 10 client.26579 handle_command_reply: tid=0 2024-04-23T14:55:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa2997ea740 1 client.26579 shutdown 2024-04-23T14:55:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa2997ea740 1 -- 172.21.15.83:0/239599939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 msgr2=0x562adc013880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa2997ea740 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc013880 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009900 tx=0x7fa27c031860 comp rx=0 tx=0).stop 2024-04-23T14:55:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa2997ea740 1 -- 172.21.15.83:0/239599939 shutdown_connections 2024-04-23T14:55:17.937 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa2997ea740 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562adc026d20 0x562adc029180 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa2997ea740 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc013fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa2997ea740 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc013880 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa2997ea740 1 --2- 172.21.15.83:0/239599939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00bb40 0x562adc016590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa279600640 0 client.26579 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:17.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa2997ea740 1 -- 172.21.15.83:0/239599939 >> 172.21.15.83:0/239599939 conn(0x562adc007e60 msgr2=0x562adc00db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.930+0000 7fa2997ea740 1 -- 172.21.15.83:0/239599939 shutdown_connections 2024-04-23T14:55:17.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.931+0000 7fa2997ea740 1 -- 172.21.15.83:0/239599939 wait complete. 2024-04-23T14:55:17.938 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.931+0000 7fa2997ea740 20 client.26579 trim_cache size 0 max 16384 2024-04-23T14:55:17.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.945+0000 7fa2997ea740 1 Processor -- start 2024-04-23T14:55:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.945+0000 7fa2997ea740 1 -- start start 2024-04-23T14:55:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa2997ea740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00bb40 0x562adc00e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa2997ea740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc0129e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa2997ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc0160c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa2997ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901b2740 con 0x562adc016b30 2024-04-23T14:55:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa2997ea740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901b0010 con 0x562adc00c460 2024-04-23T14:55:17.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa2997ea740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901ade80 con 0x562adc00bb40 2024-04-23T14:55:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa28f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc0129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa294a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00bb40 0x562adc00e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa28f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc0129e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38334/0 (socket says 172.21.15.83:38334) 2024-04-23T14:55:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa28f400640 1 -- 172.21.15.83:0/1277311823 learned_addr learned my addr 172.21.15.83:0/1277311823 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa294a00640 1 --2- 172.21.15.83:0/1277311823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc0160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa28f400640 1 -- 172.21.15.83:0/1277311823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00bb40 msgr2=0x562adc00e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa28f400640 1 --2- 172.21.15.83:0/1277311823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00bb40 0x562adc00e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa28f400640 1 -- 172.21.15.83:0/1277311823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 msgr2=0x562adc0160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa28f400640 1 --2- 172.21.15.83:0/1277311823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc0160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa28f400640 1 -- 172.21.15.83:0/1277311823 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562adc00c290 con 0x562adc00c460 2024-04-23T14:55:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.946+0000 7fa294a00640 1 --2- 172.21.15.83:0/1277311823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc0160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:55:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.947+0000 7fa28f400640 1 --2- 172.21.15.83:0/1277311823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc0129e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa27c006010 tx=0x7fa27c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b85aa3d1ac16ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.947+0000 7fa27be00640 1 -- 172.21.15.83:0/1277311823 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c052070 con 0x562adc00c460 2024-04-23T14:55:17.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.947+0000 7fa27be00640 1 -- 172.21.15.83:0/1277311823 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa27c031db0 con 0x562adc00c460 2024-04-23T14:55:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.947+0000 7fa27be00640 1 -- 172.21.15.83:0/1277311823 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c035040 con 0x562adc00c460 2024-04-23T14:55:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.948+0000 7fa2997ea740 1 -- 172.21.15.83:0/1277311823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 msgr2=0x562adc0129e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.948+0000 7fa2997ea740 1 --2- 172.21.15.83:0/1277311823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc0129e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa27c006010 tx=0x7fa27c005e80 comp rx=0 tx=0).stop 2024-04-23T14:55:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.948+0000 7fa2997ea740 1 -- 172.21.15.83:0/1277311823 shutdown_connections 2024-04-23T14:55:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.948+0000 7fa2997ea740 1 --2- 172.21.15.83:0/1277311823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc0160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.948+0000 7fa2997ea740 1 --2- 172.21.15.83:0/1277311823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00c460 0x562adc0129e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.948+0000 7fa2997ea740 1 --2- 172.21.15.83:0/1277311823 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00bb40 0x562adc00e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.948+0000 7fa2997ea740 1 -- 172.21.15.83:0/1277311823 >> 172.21.15.83:0/1277311823 conn(0x562adc007e60 msgr2=0x562adc0066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.948+0000 7fa2997ea740 1 -- 172.21.15.83:0/1277311823 shutdown_connections 2024-04-23T14:55:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.948+0000 7fa2997ea740 1 -- 172.21.15.83:0/1277311823 wait complete. 2024-04-23T14:55:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.948+0000 7fa2997ea740 1 -- start start 2024-04-23T14:55:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.948+0000 7fa2997ea740 1 Processor -- start 2024-04-23T14:55:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa2997ea740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00bb40 0x562adc0179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa2997ea740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00c460 0x562adc017f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa2997ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc018460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa2997ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901ae4d0 con 0x562adc016b30 2024-04-23T14:55:17.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa2997ea740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901af940 con 0x562adc00bb40 2024-04-23T14:55:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa2997ea740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa2901aeba0 con 0x562adc00c460 2024-04-23T14:55:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa2997ea740 10 client.0 _ll_register_callbacks cb 0x7fa2880093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa294a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00bb40 0x562adc0179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa28f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00c460 0x562adc017f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa294a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc018460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa28f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00c460 0x562adc017f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:39762/0 (socket says 172.21.15.83:39762) 2024-04-23T14:55:17.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa28f400640 1 -- 172.21.15.83:0/536402137 learned_addr learned my addr 172.21.15.83:0/536402137 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa28f400640 1 -- 172.21.15.83:0/536402137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00bb40 msgr2=0x562adc0179e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa28f400640 1 --2- 172.21.15.83:0/536402137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00bb40 0x562adc0179e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa28f400640 1 -- 172.21.15.83:0/536402137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 msgr2=0x562adc018460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa28f400640 1 --2- 172.21.15.83:0/536402137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc018460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.949+0000 7fa28f400640 1 -- 172.21.15.83:0/536402137 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562adbd4d7d0 con 0x562adc00c460 2024-04-23T14:55:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.950+0000 7fa28f400640 1 --2- 172.21.15.83:0/536402137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00c460 0x562adc017f20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa27c031fa0 tx=0x7fa27c031a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e5a70f04e0b676 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.950+0000 7fa279600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:55:17.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.950+0000 7fa279600640 1 -- 172.21.15.83:0/536402137 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27c052070 con 0x562adc00c460 2024-04-23T14:55:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.950+0000 7fa279600640 1 -- 172.21.15.83:0/536402137 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fa27c0053e0 con 0x562adc00c460 2024-04-23T14:55:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.950+0000 7fa2997ea740 1 -- 172.21.15.83:0/536402137 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa27c031db0 con 0x562adc00c460 2024-04-23T14:55:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.950+0000 7fa279600640 1 -- 172.21.15.83:0/536402137 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa27c066050 con 0x562adc00c460 2024-04-23T14:55:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.951+0000 7fa2997ea740 10 client.26560 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.951+0000 7fa2997ea740 1 -- 172.21.15.83:0/536402137 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562adc095d70 con 0x562adc00c460 2024-04-23T14:55:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.951+0000 7fa279600640 1 -- 172.21.15.83:0/536402137 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa27c031db0 con 0x562adc00c460 2024-04-23T14:55:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.951+0000 7fa279600640 1 -- 172.21.15.83:0/536402137 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fa27c05f060 con 0x562adc00c460 2024-04-23T14:55:17.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.951+0000 7fa2997ea740 10 client.26560 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:17.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.951+0000 7fa2997ea740 10 client.26560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:17.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.951+0000 7fa2997ea740 20 client.26560 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:17.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.951+0000 7fa2997ea740 1 --2- 172.21.15.83:0/536402137 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562adc0963c0 0x562adc0987e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:17.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.951+0000 7fa2997ea740 4 client.26560 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:55:17.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.951+0000 7fa2997ea740 1 -- 172.21.15.83:0/536402137 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562adbf78a60 con 0x562adc0963c0 2024-04-23T14:55:17.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.952+0000 7fa294a00640 1 --2- 172.21.15.83:0/536402137 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562adc0963c0 0x562adc0987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:17.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.952+0000 7fa294a00640 1 --2- 172.21.15.83:0/536402137 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562adc0963c0 0x562adc0987e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b12992f57f9cde server_cookie=d42fee115cb12fb3 in_seq=0 out_seq=0 2024-04-23T14:55:17.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.952+0000 7fa279600640 10 client.26560 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:17.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.952+0000 7fa279600640 1 -- 172.21.15.83:0/536402137 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562adbf78a60 con 0x562adc0963c0 2024-04-23T14:55:17.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.952+0000 7fa279600640 10 client.26560 handle_command_reply: tid=0 2024-04-23T14:55:17.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.952+0000 7fa2997ea740 1 client.26560 shutdown 2024-04-23T14:55:17.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.953+0000 7fa2997ea740 1 -- 172.21.15.83:0/536402137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00c460 msgr2=0x562adc017f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.953+0000 7fa2997ea740 1 --2- 172.21.15.83:0/536402137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00c460 0x562adc017f20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa27c031fa0 tx=0x7fa27c031a70 comp rx=0 tx=0).stop 2024-04-23T14:55:17.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.953+0000 7fa2997ea740 1 -- 172.21.15.83:0/536402137 shutdown_connections 2024-04-23T14:55:17.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.953+0000 7fa2997ea740 1 --2- 172.21.15.83:0/536402137 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562adc0963c0 0x562adc0987e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.953+0000 7fa2997ea740 1 --2- 172.21.15.83:0/536402137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562adc016b30 0x562adc018460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.953+0000 7fa279600640 0 client.26560 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:17.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.953+0000 7fa2997ea740 1 --2- 172.21.15.83:0/536402137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562adc00c460 0x562adc017f20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.953+0000 7fa2997ea740 1 --2- 172.21.15.83:0/536402137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562adc00bb40 0x562adc0179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.953+0000 7fa2997ea740 1 -- 172.21.15.83:0/536402137 >> 172.21.15.83:0/536402137 conn(0x562adc007e60 msgr2=0x562adc0066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.953+0000 7fa2997ea740 1 -- 172.21.15.83:0/536402137 shutdown_connections 2024-04-23T14:55:17.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.953+0000 7fa2997ea740 1 -- 172.21.15.83:0/536402137 wait complete. 2024-04-23T14:55:17.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.953+0000 7fa2997ea740 20 client.26560 trim_cache size 0 max 16384 2024-04-23T14:55:17.962 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:55:17.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.954+0000 7fa295400640 1 -- 172.21.15.83:0/3817090968 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa2740a3da0 msgr2=0x7fa2740a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.954+0000 7fa295400640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa2740a3da0 0x7fa2740a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2800ce330 tx=0x7fa2800d4690 comp rx=0 tx=0).stop 2024-04-23T14:55:17.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.954+0000 7fa295400640 1 -- 172.21.15.83:0/3817090968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa290104320 msgr2=0x7fa2901a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:17.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.954+0000 7fa295400640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa290104320 0x7fa2901a06d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa28000e9f0 tx=0x7fa28000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.954+0000 7fa295400640 1 -- 172.21.15.83:0/3817090968 shutdown_connections 2024-04-23T14:55:17.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.954+0000 7fa295400640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa29019a880 0x7fa29019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.954+0000 7fa295400640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2901093f0 0x7fa2901a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.954+0000 7fa295400640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa2740a3da0 0x7fa2740a6260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.954+0000 7fa295400640 1 --2- 172.21.15.83:0/3817090968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa290104320 0x7fa2901a06d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:17.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.954+0000 7fa295400640 1 -- 172.21.15.83:0/3817090968 >> 172.21.15.83:0/3817090968 conn(0x7fa2900ffb40 msgr2=0x7fa290101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:17.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.954+0000 7fa295400640 1 -- 172.21.15.83:0/3817090968 shutdown_connections 2024-04-23T14:55:17.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:17.954+0000 7fa295400640 1 -- 172.21.15.83:0/3817090968 wait complete. 2024-04-23T14:55:18.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:17 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3371879000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:55:18.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:17 smithi203 ceph-mon[32800]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T14:55:18.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:17 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3371879000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:55:18.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:17 smithi083 ceph-mon[27867]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T14:55:18.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:17 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3371879000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:55:18.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:17 smithi173 ceph-mon[32443]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T14:55:18.972 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:55:18.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:55:18.972 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:55:19.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.187+0000 7f8125c00640 1 -- 172.21.15.83:0/3076774743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120104310 msgr2=0x7f81201046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.187+0000 7f8125c00640 1 --2- 172.21.15.83:0/3076774743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120104310 0x7f81201046f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8110009900 tx=0x7f811002a410 comp rx=0 tx=0).stop 2024-04-23T14:55:19.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.187+0000 7f8125c00640 1 -- 172.21.15.83:0/3076774743 shutdown_connections 2024-04-23T14:55:19.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.187+0000 7f8125c00640 1 --2- 172.21.15.83:0/3076774743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81201093e0 0x7f8120110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.187+0000 7f8125c00640 1 --2- 172.21.15.83:0/3076774743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8120104cc0 0x7f8120108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.187+0000 7f8125c00640 1 --2- 172.21.15.83:0/3076774743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8120104310 0x7f81201046f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.187+0000 7f8125c00640 1 -- 172.21.15.83:0/3076774743 >> 172.21.15.83:0/3076774743 conn(0x7f81200ffb10 msgr2=0x7f8120101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:19.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.187+0000 7f8125c00640 1 -- 172.21.15.83:0/3076774743 shutdown_connections 2024-04-23T14:55:19.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.187+0000 7f8125c00640 1 -- 172.21.15.83:0/3076774743 wait complete. 2024-04-23T14:55:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.188+0000 7f8125c00640 1 Processor -- start 2024-04-23T14:55:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.189+0000 7f8125c00640 1 -- start start 2024-04-23T14:55:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.189+0000 7f8125c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8120104310 0x7f81201a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.189+0000 7f8125c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81201093e0 0x7f81201a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.189+0000 7f8125c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f812019a810 0x7f812019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.189+0000 7f8125c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8120114490 con 0x7f81201093e0 2024-04-23T14:55:19.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.189+0000 7f8125c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8120114310 con 0x7f8120104310 2024-04-23T14:55:19.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.189+0000 7f8125c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8120114610 con 0x7f812019a810 2024-04-23T14:55:19.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.189+0000 7f811f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81201093e0 0x7f81201a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.189+0000 7f811f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81201093e0 0x7f81201a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48740/0 (socket says 172.21.15.83:48740) 2024-04-23T14:55:19.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.189+0000 7f811f400640 1 -- 172.21.15.83:0/1062671361 learned_addr learned my addr 172.21.15.83:0/1062671361 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:19.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.189+0000 7f811fe00640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8120104310 0x7f81201a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.190+0000 7f8125200640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f812019a810 0x7f812019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.190+0000 7f811f400640 1 -- 172.21.15.83:0/1062671361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f812019a810 msgr2=0x7f812019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.190+0000 7f811f400640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f812019a810 0x7f812019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.190+0000 7f811f400640 1 -- 172.21.15.83:0/1062671361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8120104310 msgr2=0x7f81201a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.190+0000 7f811f400640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8120104310 0x7f81201a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.190+0000 7f811f400640 1 -- 172.21.15.83:0/1062671361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812019b3a0 con 0x7f81201093e0 2024-04-23T14:55:19.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.190+0000 7f811f400640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81201093e0 0x7f81201a0b90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f810c00e9f0 tx=0x7f810c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.190+0000 7f811cc00640 1 -- 172.21.15.83:0/1062671361 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c00ccc0 con 0x7f81201093e0 2024-04-23T14:55:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.190+0000 7f811cc00640 1 -- 172.21.15.83:0/1062671361 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f810c005500 con 0x7f81201093e0 2024-04-23T14:55:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.191+0000 7f811cc00640 1 -- 172.21.15.83:0/1062671361 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c010690 con 0x7f81201093e0 2024-04-23T14:55:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.191+0000 7f8125c00640 1 -- 172.21.15.83:0/1062671361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812019b690 con 0x7f81201093e0 2024-04-23T14:55:19.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.191+0000 7f8125c00640 1 -- 172.21.15.83:0/1062671361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81201a77d0 con 0x7f81201093e0 2024-04-23T14:55:19.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.194+0000 7f812a083740 1 Processor -- start 2024-04-23T14:55:19.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.194+0000 7f812a083740 1 -- start start 2024-04-23T14:55:19.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.194+0000 7f811cc00640 1 -- 172.21.15.83:0/1062671361 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f810c0046c0 con 0x7f81201093e0 2024-04-23T14:55:19.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.194+0000 7f812a083740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51270 0x55e561d51650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.194+0000 7f812a083740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51b90 0x55e561d5bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.194+0000 7f812a083740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 0x55e561d5e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.194+0000 7f812a083740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e561cb6e70 con 0x55e561d51b90 2024-04-23T14:55:19.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.194+0000 7f812a083740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e561c5d380 con 0x55e561d51270 2024-04-23T14:55:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.194+0000 7f812a083740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e561cb2e30 con 0x55e561d5c260 2024-04-23T14:55:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.195+0000 7f8125200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51270 0x55e561d51650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.195+0000 7f811fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51b90 0x55e561d5bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.195+0000 7f8125200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51270 0x55e561d51650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38388/0 (socket says 172.21.15.83:38388) 2024-04-23T14:55:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.195+0000 7f8125200640 1 -- 172.21.15.83:0/848978296 learned_addr learned my addr 172.21.15.83:0/848978296 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:19.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.195+0000 7f811f400640 1 --2- 172.21.15.83:0/848978296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 0x55e561d5e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.195+0000 7f8125200640 1 -- 172.21.15.83:0/848978296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 msgr2=0x55e561d5e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.195+0000 7f8125200640 1 --2- 172.21.15.83:0/848978296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 0x55e561d5e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.195+0000 7f8125200640 1 -- 172.21.15.83:0/848978296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51b90 msgr2=0x55e561d5bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.195+0000 7f8125200640 1 --2- 172.21.15.83:0/848978296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51b90 0x55e561d5bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.195+0000 7f8125200640 1 -- 172.21.15.83:0/848978296 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e561a917e0 con 0x55e561d51270 2024-04-23T14:55:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.196+0000 7f8125200640 1 --2- 172.21.15.83:0/848978296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51270 0x55e561d51650 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8118009f60 tx=0x7f8118009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58bcf33ad716e1ad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.196+0000 7f8115600640 1 -- 172.21.15.83:0/848978296 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811801e070 con 0x55e561d51270 2024-04-23T14:55:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.196+0000 7f811cc00640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f81040a3da0 0x7f81040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.196+0000 7f8115600640 1 -- 172.21.15.83:0/848978296 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8118003c30 con 0x55e561d51270 2024-04-23T14:55:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.196+0000 7f8115600640 1 -- 172.21.15.83:0/848978296 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811800f980 con 0x55e561d51270 2024-04-23T14:55:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.196+0000 7f811cc00640 1 -- 172.21.15.83:0/1062671361 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f810c0d0b40 con 0x7f81201093e0 2024-04-23T14:55:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.196+0000 7f811fe00640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f81040a3da0 0x7f81040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.197+0000 7f812a083740 1 -- 172.21.15.83:0/848978296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51270 msgr2=0x55e561d51650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.197+0000 7f812a083740 1 --2- 172.21.15.83:0/848978296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51270 0x55e561d51650 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8118009f60 tx=0x7f8118009f90 comp rx=0 tx=0).stop 2024-04-23T14:55:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.197+0000 7f811fe00640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f81040a3da0 0x7f81040a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8110009230 tx=0x7f81100058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.197+0000 7f812a083740 1 -- 172.21.15.83:0/848978296 shutdown_connections 2024-04-23T14:55:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.197+0000 7f812a083740 1 --2- 172.21.15.83:0/848978296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 0x55e561d5e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.197+0000 7f812a083740 1 --2- 172.21.15.83:0/848978296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51b90 0x55e561d5bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.197+0000 7f812a083740 1 --2- 172.21.15.83:0/848978296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51270 0x55e561d51650 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.197+0000 7f812a083740 1 -- 172.21.15.83:0/848978296 >> 172.21.15.83:0/848978296 conn(0x55e561d4d590 msgr2=0x55e561d4d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:19.202 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.197+0000 7f812a083740 1 -- 172.21.15.83:0/848978296 shutdown_connections 2024-04-23T14:55:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.197+0000 7f812a083740 1 -- 172.21.15.83:0/848978296 wait complete. 2024-04-23T14:55:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.198+0000 7f812a083740 1 -- start start 2024-04-23T14:55:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.198+0000 7f812a083740 1 Processor -- start 2024-04-23T14:55:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.198+0000 7f812a083740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51270 0x55e561d59240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.198+0000 7f812a083740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51b90 0x55e561d59780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.198+0000 7f812a083740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 0x55e561d59cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.198+0000 7f812a083740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e561d5f050 con 0x55e561d51b90 2024-04-23T14:55:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.198+0000 7f812a083740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e561c70660 con 0x55e561d51270 2024-04-23T14:55:19.203 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.198+0000 7f812a083740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e561d5f1d0 con 0x55e561d5c260 2024-04-23T14:55:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.198+0000 7f812a083740 10 client.0 _ll_register_callbacks cb 0x55e561c5eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.199+0000 7f811f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51b90 0x55e561d59780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.199+0000 7f811fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51270 0x55e561d59240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.199+0000 7f811f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51b90 0x55e561d59780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48770/0 (socket says 172.21.15.83:48770) 2024-04-23T14:55:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.199+0000 7f811f400640 1 -- 172.21.15.83:0/755996614 learned_addr learned my addr 172.21.15.83:0/755996614 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.199+0000 7f8125200640 1 --2- 172.21.15.83:0/755996614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 0x55e561d59cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.199+0000 7f811f400640 1 -- 172.21.15.83:0/755996614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 msgr2=0x55e561d59cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.199+0000 7f811f400640 1 --2- 172.21.15.83:0/755996614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 0x55e561d59cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.204 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.199+0000 7f811f400640 1 -- 172.21.15.83:0/755996614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51270 msgr2=0x55e561d59240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.199+0000 7f811f400640 1 --2- 172.21.15.83:0/755996614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51270 0x55e561d59240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.199+0000 7f811f400640 1 -- 172.21.15.83:0/755996614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e561af1f70 con 0x55e561d51b90 2024-04-23T14:55:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.199+0000 7f811f400640 1 --2- 172.21.15.83:0/755996614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51b90 0x55e561d59780 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f810c00e9c0 tx=0x7f810c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d19006a947f49b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:19.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.199+0000 7f810a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:55:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.200+0000 7f810a000640 1 -- 172.21.15.83:0/755996614 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c09c3d0 con 0x55e561d51b90 2024-04-23T14:55:19.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.200+0000 7f810a000640 1 -- 172.21.15.83:0/755996614 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f810c09ced0 con 0x55e561d51b90 2024-04-23T14:55:19.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.200+0000 7f812a083740 1 -- 172.21.15.83:0/755996614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8118003c30 con 0x55e561d51b90 2024-04-23T14:55:19.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.200+0000 7f810a000640 1 -- 172.21.15.83:0/755996614 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f810c0ce6b0 con 0x55e561d51b90 2024-04-23T14:55:19.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.200+0000 7f812a083740 10 client.16395 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:19.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.200+0000 7f812a083740 1 -- 172.21.15.83:0/755996614 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e561989170 con 0x55e561d51b90 2024-04-23T14:55:19.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.200+0000 7f810a000640 1 -- 172.21.15.83:0/755996614 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8118003c30 con 0x55e561d51b90 2024-04-23T14:55:19.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.200+0000 7f810a000640 1 -- 172.21.15.83:0/755996614 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f810c0a5250 con 0x55e561d51b90 2024-04-23T14:55:19.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.200+0000 7f812a083740 10 client.16395 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:19.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.200+0000 7f812a083740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:19.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.201+0000 7f812a083740 20 client.16395 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.201+0000 7f812a083740 1 --2- 172.21.15.83:0/755996614 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e561d6c630 0x55e561d6ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.201+0000 7f812a083740 4 client.16395 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:55:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.201+0000 7f812a083740 1 -- 172.21.15.83:0/755996614 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e561cc4890 con 0x55e561d6c630 2024-04-23T14:55:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.201+0000 7f811fe00640 1 --2- 172.21.15.83:0/755996614 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e561d6c630 0x55e561d6ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.201+0000 7f811fe00640 1 --2- 172.21.15.83:0/755996614 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e561d6c630 0x55e561d6ea40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7132b8be2d345d47 server_cookie=d301dcac7279784b in_seq=0 out_seq=0 2024-04-23T14:55:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.201+0000 7f810a000640 10 client.16395 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.202+0000 7f810a000640 1 -- 172.21.15.83:0/755996614 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e561cc4890 con 0x55e561d6c630 2024-04-23T14:55:19.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.202+0000 7f810a000640 10 client.16395 handle_command_reply: tid=0 2024-04-23T14:55:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.202+0000 7f812a083740 1 client.16395 shutdown 2024-04-23T14:55:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.202+0000 7f812a083740 1 -- 172.21.15.83:0/755996614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51b90 msgr2=0x55e561d59780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.202+0000 7f812a083740 1 --2- 172.21.15.83:0/755996614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51b90 0x55e561d59780 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f810c00e9c0 tx=0x7f810c009780 comp rx=0 tx=0).stop 2024-04-23T14:55:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.203+0000 7f812a083740 1 -- 172.21.15.83:0/755996614 shutdown_connections 2024-04-23T14:55:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.203+0000 7f812a083740 1 --2- 172.21.15.83:0/755996614 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e561d6c630 0x55e561d6ea40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.203+0000 7f812a083740 1 --2- 172.21.15.83:0/755996614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 0x55e561d59cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.203+0000 7f812a083740 1 --2- 172.21.15.83:0/755996614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51b90 0x55e561d59780 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.203+0000 7f812a083740 1 --2- 172.21.15.83:0/755996614 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51270 0x55e561d59240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.203+0000 7f810a000640 0 client.16395 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:19.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.203+0000 7f812a083740 1 -- 172.21.15.83:0/755996614 >> 172.21.15.83:0/755996614 conn(0x55e561d4d590 msgr2=0x55e561d53390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:19.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.203+0000 7f812a083740 1 -- 172.21.15.83:0/755996614 shutdown_connections 2024-04-23T14:55:19.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.203+0000 7f812a083740 1 -- 172.21.15.83:0/755996614 wait complete. 2024-04-23T14:55:19.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.203+0000 7f812a083740 20 client.16395 trim_cache size 0 max 16384 2024-04-23T14:55:19.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.222+0000 7f812a083740 1 Processor -- start 2024-04-23T14:55:19.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.222+0000 7f812a083740 1 -- start start 2024-04-23T14:55:19.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.222+0000 7f812a083740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 0x55e561d67f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.222+0000 7f812a083740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d51b90 0x55e561d53a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.222+0000 7f812a083740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d5c260 0x55e561d742c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.222+0000 7f812a083740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81201b2660 con 0x55e561d51270 2024-04-23T14:55:19.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.222+0000 7f812a083740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81201aff30 con 0x55e561d5c260 2024-04-23T14:55:19.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.222+0000 7f812a083740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81201adda0 con 0x55e561d51b90 2024-04-23T14:55:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.223+0000 7f811f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d51b90 0x55e561d53a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.223+0000 7f811f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d51b90 0x55e561d53a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:39784/0 (socket says 172.21.15.83:39784) 2024-04-23T14:55:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.223+0000 7f811f400640 1 -- 172.21.15.83:0/3411521577 learned_addr learned my addr 172.21.15.83:0/3411521577 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.223+0000 7f8125200640 1 --2- 172.21.15.83:0/3411521577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d5c260 0x55e561d742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.223+0000 7f811fe00640 1 --2- 172.21.15.83:0/3411521577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 0x55e561d67f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.223+0000 7f811f400640 1 -- 172.21.15.83:0/3411521577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d5c260 msgr2=0x55e561d742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.223+0000 7f811f400640 1 --2- 172.21.15.83:0/3411521577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d5c260 0x55e561d742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.223+0000 7f811f400640 1 -- 172.21.15.83:0/3411521577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 msgr2=0x55e561d67f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.223+0000 7f811f400640 1 --2- 172.21.15.83:0/3411521577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 0x55e561d67f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.223+0000 7f811f400640 1 -- 172.21.15.83:0/3411521577 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e561d48e80 con 0x55e561d51b90 2024-04-23T14:55:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.223+0000 7f811fe00640 1 --2- 172.21.15.83:0/3411521577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 0x55e561d67f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:55:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.223+0000 7f811f400640 1 --2- 172.21.15.83:0/3411521577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d51b90 0x55e561d53a10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f810c0bd010 tx=0x7f810c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebb7a8099e17332b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.224+0000 7f8114c00640 1 -- 172.21.15.83:0/3411521577 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c0056b0 con 0x55e561d51b90 2024-04-23T14:55:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.224+0000 7f8114c00640 1 -- 172.21.15.83:0/3411521577 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f810c09d170 con 0x55e561d51b90 2024-04-23T14:55:19.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.224+0000 7f8114c00640 1 -- 172.21.15.83:0/3411521577 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c0a5250 con 0x55e561d51b90 2024-04-23T14:55:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.224+0000 7f812a083740 1 -- 172.21.15.83:0/3411521577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d51b90 msgr2=0x55e561d53a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.224+0000 7f812a083740 1 --2- 172.21.15.83:0/3411521577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d51b90 0x55e561d53a10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f810c0bd010 tx=0x7f810c009310 comp rx=0 tx=0).stop 2024-04-23T14:55:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.224+0000 7f812a083740 1 -- 172.21.15.83:0/3411521577 shutdown_connections 2024-04-23T14:55:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.224+0000 7f812a083740 1 --2- 172.21.15.83:0/3411521577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d5c260 0x55e561d742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.224+0000 7f812a083740 1 --2- 172.21.15.83:0/3411521577 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d51b90 0x55e561d53a10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.224+0000 7f812a083740 1 --2- 172.21.15.83:0/3411521577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 0x55e561d67f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.224+0000 7f812a083740 1 -- 172.21.15.83:0/3411521577 >> 172.21.15.83:0/3411521577 conn(0x55e561d4d590 msgr2=0x55e561d4c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.224+0000 7f812a083740 1 -- 172.21.15.83:0/3411521577 shutdown_connections 2024-04-23T14:55:19.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.224+0000 7f812a083740 1 -- 172.21.15.83:0/3411521577 wait complete. 2024-04-23T14:55:19.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.225+0000 7f812a083740 1 -- start start 2024-04-23T14:55:19.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.225+0000 7f812a083740 1 Processor -- start 2024-04-23T14:55:19.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.225+0000 7f812a083740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 0x55e561d55e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.225+0000 7f812a083740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51b90 0x55e561d56390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.225+0000 7f812a083740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 0x55e561d568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.225+0000 7f812a083740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81201ae3f0 con 0x55e561d51270 2024-04-23T14:55:19.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.225+0000 7f812a083740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f81201af860 con 0x55e561d51b90 2024-04-23T14:55:19.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.225+0000 7f812a083740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f81201aeac0 con 0x55e561d5c260 2024-04-23T14:55:19.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.226+0000 7f812a083740 10 client.0 _ll_register_callbacks cb 0x7f810c00ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:19.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.226+0000 7f811fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 0x55e561d55e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.226+0000 7f811fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 0x55e561d55e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48788/0 (socket says 172.21.15.83:48788) 2024-04-23T14:55:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.226+0000 7f811fe00640 1 -- 172.21.15.83:0/2212580384 learned_addr learned my addr 172.21.15.83:0/2212580384 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.226+0000 7f811f400640 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51b90 0x55e561d56390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.226+0000 7f8125200640 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 0x55e561d568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.226+0000 7f811fe00640 1 -- 172.21.15.83:0/2212580384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 msgr2=0x55e561d568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.226+0000 7f811fe00640 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 0x55e561d568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.226+0000 7f811fe00640 1 -- 172.21.15.83:0/2212580384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51b90 msgr2=0x55e561d56390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.226+0000 7f811fe00640 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51b90 0x55e561d56390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.226+0000 7f811fe00640 1 -- 172.21.15.83:0/2212580384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e561dd1b30 con 0x55e561d51270 2024-04-23T14:55:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.227+0000 7f811fe00640 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 0x55e561d55e50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8110031a10 tx=0x7f8110009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14dd3534ba2bd760 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.227+0000 7f810a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:55:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.227+0000 7f810a000640 1 -- 172.21.15.83:0/2212580384 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8110035040 con 0x55e561d51270 2024-04-23T14:55:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.227+0000 7f810a000640 1 -- 172.21.15.83:0/2212580384 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f81100545a0 con 0x55e561d51270 2024-04-23T14:55:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.227+0000 7f812a083740 1 -- 172.21.15.83:0/2212580384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f810c09d170 con 0x55e561d51270 2024-04-23T14:55:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.227+0000 7f810a000640 1 -- 172.21.15.83:0/2212580384 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8110064050 con 0x55e561d51270 2024-04-23T14:55:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.227+0000 7f812a083740 10 client.16398 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.227+0000 7f812a083740 1 -- 172.21.15.83:0/2212580384 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e561dd2d40 con 0x55e561d51270 2024-04-23T14:55:19.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.227+0000 7f810a000640 1 -- 172.21.15.83:0/2212580384 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f810c09d170 con 0x55e561d51270 2024-04-23T14:55:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.228+0000 7f810a000640 1 -- 172.21.15.83:0/2212580384 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f811003d4e0 con 0x55e561d51270 2024-04-23T14:55:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.228+0000 7f812a083740 10 client.16398 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.228+0000 7f812a083740 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.228+0000 7f812a083740 20 client.16398 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.228+0000 7f812a083740 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e561dd3470 0x55e561dd5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.228+0000 7f812a083740 4 client.16398 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:55:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.228+0000 7f812a083740 1 -- 172.21.15.83:0/2212580384 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e561cbe700 con 0x55e561dd3470 2024-04-23T14:55:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.228+0000 7f811f400640 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e561dd3470 0x55e561dd5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.228+0000 7f811f400640 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e561dd3470 0x55e561dd5890 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=948eddf80ce4759d server_cookie=f2e0394ef22994b in_seq=0 out_seq=0 2024-04-23T14:55:19.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.228+0000 7f810a000640 10 client.16398 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:19.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f810a000640 1 -- 172.21.15.83:0/2212580384 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e561cbe700 con 0x55e561dd3470 2024-04-23T14:55:19.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f810a000640 10 client.16398 handle_command_reply: tid=0 2024-04-23T14:55:19.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f812a083740 1 client.16398 shutdown 2024-04-23T14:55:19.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f812a083740 1 -- 172.21.15.83:0/2212580384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 msgr2=0x55e561d55e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f812a083740 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 0x55e561d55e50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8110031a10 tx=0x7f8110009260 comp rx=0 tx=0).stop 2024-04-23T14:55:19.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f812a083740 1 -- 172.21.15.83:0/2212580384 shutdown_connections 2024-04-23T14:55:19.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f812a083740 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e561dd3470 0x55e561dd5890 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f812a083740 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e561d5c260 0x55e561d568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f812a083740 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e561d51b90 0x55e561d56390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f812a083740 1 --2- 172.21.15.83:0/2212580384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e561d51270 0x55e561d55e50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f812a083740 1 -- 172.21.15.83:0/2212580384 >> 172.21.15.83:0/2212580384 conn(0x55e561d4d590 msgr2=0x55e561d59200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:19.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f810a000640 0 client.16398 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:19.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f812a083740 1 -- 172.21.15.83:0/2212580384 shutdown_connections 2024-04-23T14:55:19.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f812a083740 1 -- 172.21.15.83:0/2212580384 wait complete. 2024-04-23T14:55:19.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.230+0000 7f812a083740 20 client.16398 trim_cache size 0 max 16384 2024-04-23T14:55:19.239 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:55:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:55:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "519dac35-1b49-4cdc-9778-19c7e8f24f3b", 2024-04-23T14:55:19.239 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:55:19.239 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:55:19.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.231+0000 7f8125c00640 1 -- 172.21.15.83:0/1062671361 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f81040a3da0 msgr2=0x7f81040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.231+0000 7f8125c00640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f81040a3da0 0x7f81040a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8110009230 tx=0x7f81100058d0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.231+0000 7f8125c00640 1 -- 172.21.15.83:0/1062671361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81201093e0 msgr2=0x7f81201a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.231+0000 7f8125c00640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81201093e0 0x7f81201a0b90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f810c00e9f0 tx=0x7f810c00edf0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.231+0000 7f8125c00640 1 -- 172.21.15.83:0/1062671361 shutdown_connections 2024-04-23T14:55:19.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.231+0000 7f8125c00640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f812019a810 0x7f812019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.231+0000 7f8125c00640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f81040a3da0 0x7f81040a6260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.231+0000 7f8125c00640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81201093e0 0x7f81201a0b90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.231+0000 7f8125c00640 1 --2- 172.21.15.83:0/1062671361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8120104310 0x7f81201a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.231+0000 7f8125c00640 1 -- 172.21.15.83:0/1062671361 >> 172.21.15.83:0/1062671361 conn(0x7f81200ffb10 msgr2=0x7f8120101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:19.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.231+0000 7f8125c00640 1 -- 172.21.15.83:0/1062671361 shutdown_connections 2024-04-23T14:55:19.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.231+0000 7f8125c00640 1 -- 172.21.15.83:0/1062671361 wait complete. 2024-04-23T14:55:19.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:55:19.252 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:55:19.319 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:19 smithi083 ceph-mon[27867]: pgmap v211: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 6 op/s 2024-04-23T14:55:19.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.446+0000 7f03ab000640 1 -- 172.21.15.83:0/1670704861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03a410b180 msgr2=0x7f03a4117690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.446+0000 7f03ab000640 1 --2- 172.21.15.83:0/1670704861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03a410b180 0x7f03a4117690 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f039c009cd0 tx=0x7f039c02a8f0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.446+0000 7f03ab000640 1 -- 172.21.15.83:0/1670704861 shutdown_connections 2024-04-23T14:55:19.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.446+0000 7f03ab000640 1 --2- 172.21.15.83:0/1670704861 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03a410b180 0x7f03a4117690 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.446+0000 7f03ab000640 1 --2- 172.21.15.83:0/1670704861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a410a7c0 0x7f03a410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.446+0000 7f03ab000640 1 --2- 172.21.15.83:0/1670704861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03a41107d0 0x7f03a4110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.446+0000 7f03ab000640 1 -- 172.21.15.83:0/1670704861 >> 172.21.15.83:0/1670704861 conn(0x7f03a4073600 msgr2=0x7f03a4073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:19.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.446+0000 7f03ab000640 1 -- 172.21.15.83:0/1670704861 shutdown_connections 2024-04-23T14:55:19.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.446+0000 7f03ab000640 1 -- 172.21.15.83:0/1670704861 wait complete. 2024-04-23T14:55:19.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.447+0000 7f03ab000640 1 Processor -- start 2024-04-23T14:55:19.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.447+0000 7f03ab000640 1 -- start start 2024-04-23T14:55:19.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03ab000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a410a7c0 0x7f03a406b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03ab000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03a410b180 0x7f03a406bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03ab000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03a41107d0 0x7f03a406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03ab000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03a411abe0 con 0x7f03a410b180 2024-04-23T14:55:19.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03ab000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f03a411aa60 con 0x7f03a410a7c0 2024-04-23T14:55:19.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03ab000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03a411ad60 con 0x7f03a41107d0 2024-04-23T14:55:19.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f039b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03a410b180 0x7f03a406bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03a3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a410a7c0 0x7f03a406b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03a3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a410a7c0 0x7f03a406b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:38446/0 (socket says 172.21.15.83:38446) 2024-04-23T14:55:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03a9000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03a41107d0 0x7f03a406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03a3e00640 1 -- 172.21.15.83:0/595932211 learned_addr learned my addr 172.21.15.83:0/595932211 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03a3e00640 1 -- 172.21.15.83:0/595932211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03a41107d0 msgr2=0x7f03a406c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03a3e00640 1 --2- 172.21.15.83:0/595932211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03a41107d0 0x7f03a406c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03a3e00640 1 -- 172.21.15.83:0/595932211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03a410b180 msgr2=0x7f03a406bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03a3e00640 1 --2- 172.21.15.83:0/595932211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03a410b180 0x7f03a406bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.448+0000 7f03a3e00640 1 -- 172.21.15.83:0/595932211 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a41b3b00 con 0x7f03a410a7c0 2024-04-23T14:55:19.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.449+0000 7f03a3e00640 1 --2- 172.21.15.83:0/595932211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a410a7c0 0x7f03a406b840 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f039400ea60 tx=0x7f039400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:19.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.449+0000 7f03a1600640 1 -- 172.21.15.83:0/595932211 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039400cde0 con 0x7f03a410a7c0 2024-04-23T14:55:19.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.449+0000 7f03a1600640 1 -- 172.21.15.83:0/595932211 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03940035f0 con 0x7f03a410a7c0 2024-04-23T14:55:19.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.449+0000 7f03ab000640 1 -- 172.21.15.83:0/595932211 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a41b3d30 con 0x7f03a410a7c0 2024-04-23T14:55:19.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.449+0000 7f03a1600640 1 -- 172.21.15.83:0/595932211 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394010670 con 0x7f03a410a7c0 2024-04-23T14:55:19.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.449+0000 7f03ab000640 1 -- 172.21.15.83:0/595932211 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a41b4220 con 0x7f03a410a7c0 2024-04-23T14:55:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.452+0000 7f03a1600640 1 -- 172.21.15.83:0/595932211 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f03940037a0 con 0x7f03a410a7c0 2024-04-23T14:55:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.452+0000 7f03add98740 1 Processor -- start 2024-04-23T14:55:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.453+0000 7f03add98740 1 -- start start 2024-04-23T14:55:19.455 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.453+0000 7f03add98740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bc5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.453+0000 7f03add98740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc6180 0x563093bd0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.453+0000 7f03add98740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093bd2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.453+0000 7f03add98740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563093b2b1f0 con 0x563093bc5860 2024-04-23T14:55:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.453+0000 7f03add98740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563093ad1350 con 0x563093bd0850 2024-04-23T14:55:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.453+0000 7f03add98740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563093b271b0 con 0x563093bc6180 2024-04-23T14:55:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.453+0000 7f03a9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bc5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.453+0000 7f03a9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bc5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48822/0 (socket says 172.21.15.83:48822) 2024-04-23T14:55:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.453+0000 7f03a9000640 1 -- 172.21.15.83:0/2080196815 learned_addr learned my addr 172.21.15.83:0/2080196815 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:19.456 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.453+0000 7f03a3e00640 1 --2- 172.21.15.83:0/2080196815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc6180 0x563093bd0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.453+0000 7f039b400640 1 --2- 172.21.15.83:0/2080196815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093bd2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.454+0000 7f03a9000640 1 -- 172.21.15.83:0/2080196815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc6180 msgr2=0x563093bd0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.454+0000 7f03a9000640 1 --2- 172.21.15.83:0/2080196815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc6180 0x563093bd0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.454+0000 7f03a9000640 1 -- 172.21.15.83:0/2080196815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 msgr2=0x563093bd2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.454+0000 7f03a9000640 1 --2- 172.21.15.83:0/2080196815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093bd2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.454+0000 7f03a9000640 1 -- 172.21.15.83:0/2080196815 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563093b25f50 con 0x563093bc5860 2024-04-23T14:55:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.454+0000 7f03a9000640 1 --2- 172.21.15.83:0/2080196815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bc5c40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f039c004aa0 tx=0x7f039c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cf28535987aeb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:19.457 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.454+0000 7f0399600640 1 -- 172.21.15.83:0/2080196815 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c030660 con 0x563093bc5860 2024-04-23T14:55:19.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.454+0000 7f0399600640 1 -- 172.21.15.83:0/2080196815 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039c036030 con 0x563093bc5860 2024-04-23T14:55:19.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.454+0000 7f03a1600640 1 --2- 172.21.15.83:0/595932211 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f03880a3d20 0x7f03880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.454+0000 7f0399600640 1 -- 172.21.15.83:0/2080196815 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c0043a0 con 0x563093bc5860 2024-04-23T14:55:19.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.455+0000 7f03a1600640 1 -- 172.21.15.83:0/595932211 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f03940d0ad0 con 0x7f03a410a7c0 2024-04-23T14:55:19.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.455+0000 7f03a3e00640 1 --2- 172.21.15.83:0/595932211 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f03880a3d20 0x7f03880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.455+0000 7f03add98740 1 -- 172.21.15.83:0/2080196815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 msgr2=0x563093bc5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.455+0000 7f03add98740 1 --2- 172.21.15.83:0/2080196815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bc5c40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f039c004aa0 tx=0x7f039c009360 comp rx=0 tx=0).stop 2024-04-23T14:55:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.455+0000 7f03add98740 1 -- 172.21.15.83:0/2080196815 shutdown_connections 2024-04-23T14:55:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.455+0000 7f03add98740 1 --2- 172.21.15.83:0/2080196815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093bd2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.455+0000 7f03add98740 1 --2- 172.21.15.83:0/2080196815 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc6180 0x563093bd0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.455+0000 7f03add98740 1 --2- 172.21.15.83:0/2080196815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bc5c40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.455+0000 7f03add98740 1 -- 172.21.15.83:0/2080196815 >> 172.21.15.83:0/2080196815 conn(0x563093bc1b80 msgr2=0x563093bc1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.455+0000 7f03add98740 1 -- 172.21.15.83:0/2080196815 shutdown_connections 2024-04-23T14:55:19.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.455+0000 7f03add98740 1 -- 172.21.15.83:0/2080196815 wait complete. 2024-04-23T14:55:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.455+0000 7f03a3e00640 1 --2- 172.21.15.83:0/595932211 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f03880a3d20 0x7f03880a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x563093bc6e20 tx=0x7f0394003f30 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.456+0000 7f03add98740 1 -- start start 2024-04-23T14:55:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.456+0000 7f03add98740 1 Processor -- start 2024-04-23T14:55:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.456+0000 7f03add98740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc5860 0x563093bcd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f03add98740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc6180 0x563093bcdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f03add98740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093bce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f03add98740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563093bd35e0 con 0x563093bc6180 2024-04-23T14:55:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f03add98740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563093ae4690 con 0x563093bd0850 2024-04-23T14:55:19.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f03add98740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563093bd3760 con 0x563093bc5860 2024-04-23T14:55:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f03add98740 10 client.0 _ll_register_callbacks cb 0x563093b20650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f039b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc6180 0x563093bcdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f039b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc6180 0x563093bcdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48832/0 (socket says 172.21.15.83:48832) 2024-04-23T14:55:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f039b400640 1 -- 172.21.15.83:0/152694056 learned_addr learned my addr 172.21.15.83:0/152694056 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f03a9000640 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc5860 0x563093bcd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f03a9000640 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093bce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f039b400640 1 -- 172.21.15.83:0/152694056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc5860 msgr2=0x563093bcd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f039b400640 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc5860 0x563093bcd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f039b400640 1 -- 172.21.15.83:0/152694056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 msgr2=0x563093bce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f039b400640 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093bce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f039b400640 1 -- 172.21.15.83:0/152694056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563093ae3920 con 0x563093bc6180 2024-04-23T14:55:19.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.457+0000 7f03a9000640 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093bce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:55:19.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.458+0000 7f039b400640 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc6180 0x563093bcdc70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f038c012700 tx=0x7f038c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3d074ea264ad41 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:19.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.458+0000 7f0392000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:55:19.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.458+0000 7f0392000640 1 -- 172.21.15.83:0/152694056 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c00f920 con 0x563093bc6180 2024-04-23T14:55:19.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.458+0000 7f0392000640 1 -- 172.21.15.83:0/152694056 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f038c00fac0 con 0x563093bc6180 2024-04-23T14:55:19.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.458+0000 7f03add98740 1 -- 172.21.15.83:0/152694056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f039c036030 con 0x563093bc6180 2024-04-23T14:55:19.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.458+0000 7f0392000640 1 -- 172.21.15.83:0/152694056 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f038c00fcc0 con 0x563093bc6180 2024-04-23T14:55:19.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.458+0000 7f03add98740 10 client.16410 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:19.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.458+0000 7f03add98740 1 -- 172.21.15.83:0/152694056 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5630939077d0 con 0x563093bc6180 2024-04-23T14:55:19.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.458+0000 7f0392000640 1 -- 172.21.15.83:0/152694056 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039c036030 con 0x563093bc6180 2024-04-23T14:55:19.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.459+0000 7f0392000640 1 -- 172.21.15.83:0/152694056 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f038c0104c0 con 0x563093bc6180 2024-04-23T14:55:19.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.459+0000 7f03add98740 10 client.16410 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:19.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.459+0000 7f03add98740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:19.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.459+0000 7f03add98740 20 client.16410 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:19.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.459+0000 7f03add98740 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563093be0bc0 0x563093be3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.459+0000 7f03add98740 4 client.16410 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:55:19.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.459+0000 7f03add98740 1 -- 172.21.15.83:0/152694056 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563093b38c10 con 0x563093be0bc0 2024-04-23T14:55:19.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.459+0000 7f03a9000640 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563093be0bc0 0x563093be3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.460+0000 7f03a9000640 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563093be0bc0 0x563093be3000 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a024cfcd20fca9b7 server_cookie=73c046aed9c1ba90 in_seq=0 out_seq=0 2024-04-23T14:55:19.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.460+0000 7f0392000640 10 client.16410 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:19.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.460+0000 7f0392000640 1 -- 172.21.15.83:0/152694056 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563093b38c10 con 0x563093be0bc0 2024-04-23T14:55:19.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.460+0000 7f0392000640 10 client.16410 handle_command_reply: tid=0 2024-04-23T14:55:19.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f03add98740 1 client.16410 shutdown 2024-04-23T14:55:19.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f03add98740 1 -- 172.21.15.83:0/152694056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc6180 msgr2=0x563093bcdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f03add98740 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc6180 0x563093bcdc70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f038c012700 tx=0x7f038c012bc0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f03add98740 1 -- 172.21.15.83:0/152694056 shutdown_connections 2024-04-23T14:55:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f03add98740 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563093be0bc0 0x563093be3000 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f03add98740 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093bce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f03add98740 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc6180 0x563093bcdc70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f0392000640 0 client.16410 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f03add98740 1 --2- 172.21.15.83:0/152694056 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc5860 0x563093bcd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f03add98740 1 -- 172.21.15.83:0/152694056 >> 172.21.15.83:0/152694056 conn(0x563093bc1b80 msgr2=0x563093bc7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:19.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f03add98740 1 -- 172.21.15.83:0/152694056 shutdown_connections 2024-04-23T14:55:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f03add98740 1 -- 172.21.15.83:0/152694056 wait complete. 2024-04-23T14:55:19.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.461+0000 7f03add98740 20 client.16410 trim_cache size 0 max 16384 2024-04-23T14:55:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.480+0000 7f03add98740 1 Processor -- start 2024-04-23T14:55:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.480+0000 7f03add98740 1 -- start start 2024-04-23T14:55:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03add98740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bdc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03add98740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bc6180 0x563093bc7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03add98740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bd0850 0x563093be88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03add98740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03a41bf490 con 0x563093bc5860 2024-04-23T14:55:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03add98740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f03a41bcd60 con 0x563093bc6180 2024-04-23T14:55:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03add98740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03a41babd0 con 0x563093bd0850 2024-04-23T14:55:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03a9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bdc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03a9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bdc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48844/0 (socket says 172.21.15.83:48844) 2024-04-23T14:55:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03a9000640 1 -- 172.21.15.83:0/1771874221 learned_addr learned my addr 172.21.15.83:0/1771874221 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f039b400640 1 --2- 172.21.15.83:0/1771874221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bc6180 0x563093bc7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03a9000640 1 --2- 172.21.15.83:0/1771874221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bd0850 0x563093be88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03a9000640 1 -- 172.21.15.83:0/1771874221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bd0850 msgr2=0x563093be88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03a9000640 1 --2- 172.21.15.83:0/1771874221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bd0850 0x563093be88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03a9000640 1 -- 172.21.15.83:0/1771874221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bc6180 msgr2=0x563093bc7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03a9000640 1 --2- 172.21.15.83:0/1771874221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bc6180 0x563093bc7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f03a9000640 1 -- 172.21.15.83:0/1771874221 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563093bc5fb0 con 0x563093bc5860 2024-04-23T14:55:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.481+0000 7f039b400640 1 --2- 172.21.15.83:0/1771874221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bc6180 0x563093bc7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.482+0000 7f03a9000640 1 --2- 172.21.15.83:0/1771874221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bdc4d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f039c005cc0 tx=0x7f039c047040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9fd8dfa83169de server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.482+0000 7f0398c00640 1 -- 172.21.15.83:0/1771874221 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c030520 con 0x563093bc5860 2024-04-23T14:55:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.482+0000 7f0398c00640 1 -- 172.21.15.83:0/1771874221 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039c036070 con 0x563093bc5860 2024-04-23T14:55:19.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.482+0000 7f0398c00640 1 -- 172.21.15.83:0/1771874221 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c058620 con 0x563093bc5860 2024-04-23T14:55:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.483+0000 7f03add98740 1 -- 172.21.15.83:0/1771874221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 msgr2=0x563093bdc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.483+0000 7f03add98740 1 --2- 172.21.15.83:0/1771874221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bdc4d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f039c005cc0 tx=0x7f039c047040 comp rx=0 tx=0).stop 2024-04-23T14:55:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.483+0000 7f03add98740 1 -- 172.21.15.83:0/1771874221 shutdown_connections 2024-04-23T14:55:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.483+0000 7f03add98740 1 --2- 172.21.15.83:0/1771874221 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bd0850 0x563093be88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.483+0000 7f03add98740 1 --2- 172.21.15.83:0/1771874221 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bc6180 0x563093bc7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.490 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.483+0000 7f03add98740 1 --2- 172.21.15.83:0/1771874221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bdc4d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.483+0000 7f03add98740 1 -- 172.21.15.83:0/1771874221 >> 172.21.15.83:0/1771874221 conn(0x563093bc1b80 msgr2=0x563093bcdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.483+0000 7f03add98740 1 -- 172.21.15.83:0/1771874221 shutdown_connections 2024-04-23T14:55:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.483+0000 7f03add98740 1 -- 172.21.15.83:0/1771874221 wait complete. 2024-04-23T14:55:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.483+0000 7f03add98740 1 -- start start 2024-04-23T14:55:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.484+0000 7f03add98740 1 Processor -- start 2024-04-23T14:55:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.484+0000 7f03add98740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bcab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.484+0000 7f03add98740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc6180 0x563093bcb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.491 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.484+0000 7f03add98740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093c46140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.484+0000 7f03add98740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03a41bb220 con 0x563093bc5860 2024-04-23T14:55:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.484+0000 7f03add98740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f03a41bc690 con 0x563093bd0850 2024-04-23T14:55:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.484+0000 7f03add98740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f03a41bb8f0 con 0x563093bc6180 2024-04-23T14:55:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.484+0000 7f03add98740 10 client.0 _ll_register_callbacks cb 0x7f038c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.484+0000 7f03a9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bcab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.484+0000 7f03a9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bcab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48854/0 (socket says 172.21.15.83:48854) 2024-04-23T14:55:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.484+0000 7f039b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc6180 0x563093bcb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.492 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.484+0000 7f03a9000640 1 -- 172.21.15.83:0/148829205 learned_addr learned my addr 172.21.15.83:0/148829205 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f03a9000640 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093c46140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f039b400640 1 -- 172.21.15.83:0/148829205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 msgr2=0x563093c46140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f039b400640 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093c46140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f039b400640 1 -- 172.21.15.83:0/148829205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 msgr2=0x563093bcab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f039b400640 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bcab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f039b400640 1 -- 172.21.15.83:0/148829205 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56309383d3f0 con 0x563093bc6180 2024-04-23T14:55:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f03a9000640 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093c46140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:55:19.493 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f039b400640 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc6180 0x563093bcb060 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f038c010960 tx=0x7f038c010e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff5a7aa09cebad9b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f0392000640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:55:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f0392000640 1 -- 172.21.15.83:0/148829205 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c029040 con 0x563093bc6180 2024-04-23T14:55:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f0392000640 1 -- 172.21.15.83:0/148829205 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f038c010430 con 0x563093bc6180 2024-04-23T14:55:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f03add98740 1 -- 172.21.15.83:0/148829205 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f039c036070 con 0x563093bc6180 2024-04-23T14:55:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f0392000640 1 -- 172.21.15.83:0/148829205 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f038c035050 con 0x563093bc6180 2024-04-23T14:55:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.485+0000 7f03add98740 10 client.26596 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.486+0000 7f03add98740 1 -- 172.21.15.83:0/148829205 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5630937fd0f0 con 0x563093bc6180 2024-04-23T14:55:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.486+0000 7f0392000640 1 -- 172.21.15.83:0/148829205 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039c036070 con 0x563093bc6180 2024-04-23T14:55:19.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.486+0000 7f0392000640 1 -- 172.21.15.83:0/148829205 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f038c003780 con 0x563093bc6180 2024-04-23T14:55:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.486+0000 7f03add98740 10 client.26596 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.486+0000 7f03add98740 10 client.26596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.486+0000 7f03add98740 20 client.26596 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.486+0000 7f03add98740 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563093c47d10 0x563093c4a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.486+0000 7f03add98740 4 client.26596 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:55:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.486+0000 7f03add98740 1 -- 172.21.15.83:0/148829205 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563093b32a80 con 0x563093c47d10 2024-04-23T14:55:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.486+0000 7f03a9000640 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563093c47d10 0x563093c4a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:19.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f03a9000640 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563093c47d10 0x563093c4a130 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b1168d11bac9303 server_cookie=3dd3fcad5994955f in_seq=0 out_seq=0 2024-04-23T14:55:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f0392000640 10 client.26596 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f0392000640 1 -- 172.21.15.83:0/148829205 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563093b32a80 con 0x563093c47d10 2024-04-23T14:55:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f0392000640 10 client.26596 handle_command_reply: tid=0 2024-04-23T14:55:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f03add98740 1 client.26596 shutdown 2024-04-23T14:55:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f03add98740 1 -- 172.21.15.83:0/148829205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc6180 msgr2=0x563093bcb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f03add98740 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc6180 0x563093bcb060 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f038c010960 tx=0x7f038c010e00 comp rx=0 tx=0).stop 2024-04-23T14:55:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f03add98740 1 -- 172.21.15.83:0/148829205 shutdown_connections 2024-04-23T14:55:19.496 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f03add98740 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563093c47d10 0x563093c4a130 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f03add98740 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563093bd0850 0x563093c46140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f03add98740 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563093bc6180 0x563093bcb060 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f0392000640 0 client.26596 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f03add98740 1 --2- 172.21.15.83:0/148829205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563093bc5860 0x563093bcab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.487+0000 7f03add98740 1 -- 172.21.15.83:0/148829205 >> 172.21.15.83:0/148829205 conn(0x563093bc1b80 msgr2=0x563093bc0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.488+0000 7f03add98740 1 -- 172.21.15.83:0/148829205 shutdown_connections 2024-04-23T14:55:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.488+0000 7f03add98740 1 -- 172.21.15.83:0/148829205 wait complete. 2024-04-23T14:55:19.497 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.488+0000 7f03add98740 20 client.26596 trim_cache size 0 max 16384 2024-04-23T14:55:19.498 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:55:19.498 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:55:19.498 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:55:19.498 INFO:teuthology.orchestra.run.smithi083.stdout: "519dac35-1b49-4cdc-9778-19c7e8f24f3b": { 2024-04-23T14:55:19.498 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:55:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "519dac35-1b49-4cdc-9778-19c7e8f24f3b", 2024-04-23T14:55:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:55:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:55:19.499 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:55:19.499 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:55:19.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.488+0000 7f03ab000640 1 -- 172.21.15.83:0/595932211 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f03880a3d20 msgr2=0x7f03880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.488+0000 7f03ab000640 1 --2- 172.21.15.83:0/595932211 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f03880a3d20 0x7f03880a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x563093bc6e20 tx=0x7f0394003f30 comp rx=0 tx=0).stop 2024-04-23T14:55:19.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.488+0000 7f03ab000640 1 -- 172.21.15.83:0/595932211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a410a7c0 msgr2=0x7f03a406b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:19.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.488+0000 7f03ab000640 1 --2- 172.21.15.83:0/595932211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a410a7c0 0x7f03a406b840 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f039400ea60 tx=0x7f039400ef20 comp rx=0 tx=0).stop 2024-04-23T14:55:19.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.488+0000 7f03ab000640 1 -- 172.21.15.83:0/595932211 shutdown_connections 2024-04-23T14:55:19.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.488+0000 7f03ab000640 1 --2- 172.21.15.83:0/595932211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03a41107d0 0x7f03a406c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.489+0000 7f03ab000640 1 --2- 172.21.15.83:0/595932211 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f03880a3d20 0x7f03880a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.489+0000 7f03ab000640 1 --2- 172.21.15.83:0/595932211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03a410b180 0x7f03a406bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.489+0000 7f03ab000640 1 --2- 172.21.15.83:0/595932211 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03a410a7c0 0x7f03a406b840 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:19.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.489+0000 7f03ab000640 1 -- 172.21.15.83:0/595932211 >> 172.21.15.83:0/595932211 conn(0x7f03a4073600 msgr2=0x7f03a406ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:19.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.489+0000 7f03ab000640 1 -- 172.21.15.83:0/595932211 shutdown_connections 2024-04-23T14:55:19.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:19.489+0000 7f03ab000640 1 -- 172.21.15.83:0/595932211 wait complete. 2024-04-23T14:55:19.507 INFO:tasks.cephfs.filesystem:scrub status for tag:519dac35-1b49-4cdc-9778-19c7e8f24f3b - {'path': '/', 'tag': '519dac35-1b49-4cdc-9778-19c7e8f24f3b', 'options': 'recursive,force'} 2024-04-23T14:55:19.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:19 smithi173 ceph-mon[32443]: pgmap v211: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 6 op/s 2024-04-23T14:55:19.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:19 smithi203 ceph-mon[32800]: pgmap v211: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 229 B/s wr, 6 op/s 2024-04-23T14:55:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:55:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:55:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:55:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:55:20.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:55:20.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:55:20.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:55:20.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:20 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:55:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:55:20.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:55:20.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:55:20.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:55:20.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:55:20.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:55:20.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:20 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:55:20.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:55:20.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:55:20.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:55:20.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:55:20.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:55:20.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:55:20.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:20.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:20 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:21.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:21 smithi173 ceph-mon[32443]: pgmap v212: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:55:21.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:21 smithi203 ceph-mon[32800]: pgmap v212: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:55:21.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:21 smithi083 ceph-mon[27867]: pgmap v212: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:55:23.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:23 smithi173 ceph-mon[32443]: pgmap v213: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:55:23.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:23 smithi203 ceph-mon[32800]: pgmap v213: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:55:23.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:23 smithi083 ceph-mon[27867]: pgmap v213: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:55:25.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:25 smithi173 ceph-mon[32443]: pgmap v214: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:55:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:25 smithi203 ceph-mon[32800]: pgmap v214: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:55:25.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:25 smithi083 ceph-mon[27867]: pgmap v214: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:55:27.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:27 smithi173 ceph-mon[32443]: pgmap v215: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T14:55:27.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:27 smithi203 ceph-mon[32800]: pgmap v215: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T14:55:27.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:27 smithi083 ceph-mon[27867]: pgmap v215: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T14:55:29.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:29 smithi173 ceph-mon[32443]: pgmap v216: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-23T14:55:29.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:29 smithi203 ceph-mon[32800]: pgmap v216: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-23T14:55:29.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:29 smithi083 ceph-mon[27867]: pgmap v216: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-23T14:55:31.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:31 smithi173 ceph-mon[32443]: pgmap v217: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:55:31.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:31 smithi203 ceph-mon[32800]: pgmap v217: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:55:31.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:31 smithi083 ceph-mon[27867]: pgmap v217: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:55:33.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:33 smithi173 ceph-mon[32443]: pgmap v218: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:55:33.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:33 smithi203 ceph-mon[32800]: pgmap v218: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:55:33.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:33 smithi083 ceph-mon[27867]: pgmap v218: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:55:34.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:34 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:55:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:34 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:55:34.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:34 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:55:34.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:34 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:55:34.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:34 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:55:34.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:34 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:55:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:34 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:55:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:34 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:55:34.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:34 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:55:35.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:35 smithi173 ceph-mon[32443]: pgmap v219: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T14:55:35.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:35 smithi203 ceph-mon[32800]: pgmap v219: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T14:55:35.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:35 smithi083 ceph-mon[27867]: pgmap v219: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T14:55:37.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:37 smithi173 ceph-mon[32443]: pgmap v220: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:55:37.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:37 smithi203 ceph-mon[32800]: pgmap v220: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:55:37.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:37 smithi083 ceph-mon[27867]: pgmap v220: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:55:39.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:39 smithi173 ceph-mon[32443]: pgmap v221: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:55:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:39 smithi203 ceph-mon[32800]: pgmap v221: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:55:39.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:39 smithi083 ceph-mon[27867]: pgmap v221: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:55:41.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:41 smithi203 ceph-mon[32800]: pgmap v222: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:55:41.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:41 smithi083 ceph-mon[27867]: pgmap v222: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:55:41.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:41 smithi173 ceph-mon[32443]: pgmap v222: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:55:43.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:43 smithi203 ceph-mon[32800]: pgmap v223: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:55:43.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:43 smithi083 ceph-mon[27867]: pgmap v223: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:55:43.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:43 smithi173 ceph-mon[32443]: pgmap v223: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:55:45.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:45 smithi203 ceph-mon[32800]: pgmap v224: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:55:45.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:45 smithi083 ceph-mon[27867]: pgmap v224: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:55:45.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:45 smithi173 ceph-mon[32443]: pgmap v224: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:55:47.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:47 smithi203 ceph-mon[32800]: pgmap v225: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:55:47.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:47 smithi083 ceph-mon[27867]: pgmap v225: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:55:47.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:47 smithi173 ceph-mon[32443]: pgmap v225: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:55:49.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:55:49.509 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:55:49.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:49 smithi203 ceph-mon[32800]: pgmap v226: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:55:49.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.722+0000 7fbf3f200640 1 -- 172.21.15.83:0/901212472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf40108f40 msgr2=0x7fbf401153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.722+0000 7fbf3f200640 1 --2- 172.21.15.83:0/901212472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf40108f40 0x7fbf401153b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009be0 tx=0x7fbf3802a6d0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.722+0000 7fbf3f200640 1 -- 172.21.15.83:0/901212472 shutdown_connections 2024-04-23T14:55:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.722+0000 7fbf3f200640 1 --2- 172.21.15.83:0/901212472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf40108f40 0x7fbf401153b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.722+0000 7fbf3f200640 1 --2- 172.21.15.83:0/901212472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf40108580 0x7fbf40108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.722+0000 7fbf3f200640 1 --2- 172.21.15.83:0/901212472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf4010d500 0x7fbf4010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.722+0000 7fbf3f200640 1 -- 172.21.15.83:0/901212472 >> 172.21.15.83:0/901212472 conn(0x7fbf40077e20 msgr2=0x7fbf40106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.722+0000 7fbf3f200640 1 -- 172.21.15.83:0/901212472 shutdown_connections 2024-04-23T14:55:49.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.722+0000 7fbf3f200640 1 -- 172.21.15.83:0/901212472 wait complete. 2024-04-23T14:55:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.723+0000 7fbf3f200640 1 Processor -- start 2024-04-23T14:55:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.723+0000 7fbf3f200640 1 -- start start 2024-04-23T14:55:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.724+0000 7fbf3f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40108580 0x7fbf401a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.724+0000 7fbf3f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf40108f40 0x7fbf401a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.724+0000 7fbf3f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf4010d500 0x7fbf4019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.724+0000 7fbf3f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401188b0 con 0x7fbf40108580 2024-04-23T14:55:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.724+0000 7fbf3f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbf40118730 con 0x7fbf4010d500 2024-04-23T14:55:49.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.724+0000 7fbf3f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbf40118a30 con 0x7fbf40108f40 2024-04-23T14:55:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.724+0000 7fbf3de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40108580 0x7fbf401a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.724+0000 7fbf3de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40108580 0x7fbf401a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50974/0 (socket says 172.21.15.83:50974) 2024-04-23T14:55:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.724+0000 7fbf3de00640 1 -- 172.21.15.83:0/3159580798 learned_addr learned my addr 172.21.15.83:0/3159580798 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.724+0000 7fbf37e00640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf40108f40 0x7fbf401a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.724+0000 7fbf3e800640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf4010d500 0x7fbf4019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.725+0000 7fbf3de00640 1 -- 172.21.15.83:0/3159580798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf40108f40 msgr2=0x7fbf401a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.725+0000 7fbf3de00640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf40108f40 0x7fbf401a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.725+0000 7fbf3de00640 1 -- 172.21.15.83:0/3159580798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf4010d500 msgr2=0x7fbf4019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.725+0000 7fbf3de00640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf4010d500 0x7fbf4019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.725+0000 7fbf3de00640 1 -- 172.21.15.83:0/3159580798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4019f230 con 0x7fbf40108580 2024-04-23T14:55:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.725+0000 7fbf3de00640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40108580 0x7fbf401a4900 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000e9f0 tx=0x7fbf3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.725+0000 7fbf36a00640 1 -- 172.21.15.83:0/3159580798 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3000ccc0 con 0x7fbf40108580 2024-04-23T14:55:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.725+0000 7fbf36a00640 1 -- 172.21.15.83:0/3159580798 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf300035f0 con 0x7fbf40108580 2024-04-23T14:55:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.725+0000 7fbf36a00640 1 -- 172.21.15.83:0/3159580798 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3000ce60 con 0x7fbf40108580 2024-04-23T14:55:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.725+0000 7fbf3f200640 1 -- 172.21.15.83:0/3159580798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4019f520 con 0x7fbf40108580 2024-04-23T14:55:49.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.725+0000 7fbf3f200640 1 -- 172.21.15.83:0/3159580798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf401ab7c0 con 0x7fbf40108580 2024-04-23T14:55:49.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.728+0000 7fbf4801a740 1 Processor -- start 2024-04-23T14:55:49.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.728+0000 7fbf36a00640 1 -- 172.21.15.83:0/3159580798 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbf300037a0 con 0x7fbf40108580 2024-04-23T14:55:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.729+0000 7fbf4801a740 1 -- start start 2024-04-23T14:55:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.729+0000 7fbf4801a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3ba30 0x562909c3be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.729+0000 7fbf4801a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3c350 0x562909c464e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.729+0000 7fbf4801a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c48e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.729+0000 7fbf4801a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562909ba11f0 con 0x562909c46a20 2024-04-23T14:55:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.729+0000 7fbf4801a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562909b47350 con 0x562909c3ba30 2024-04-23T14:55:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.729+0000 7fbf4801a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562909b9d1b0 con 0x562909c3c350 2024-04-23T14:55:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.729+0000 7fbf3e800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3ba30 0x562909c3be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.729+0000 7fbf3de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3c350 0x562909c464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.729+0000 7fbf37e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c48e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.729+0000 7fbf3e800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3ba30 0x562909c3be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:42648/0 (socket says 172.21.15.83:42648) 2024-04-23T14:55:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.729+0000 7fbf3e800640 1 -- 172.21.15.83:0/4290179206 learned_addr learned my addr 172.21.15.83:0/4290179206 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.730+0000 7fbf3e800640 1 -- 172.21.15.83:0/4290179206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3c350 msgr2=0x562909c464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.730+0000 7fbf3e800640 1 --2- 172.21.15.83:0/4290179206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3c350 0x562909c464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.730+0000 7fbf3e800640 1 -- 172.21.15.83:0/4290179206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 msgr2=0x562909c48e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.730+0000 7fbf3e800640 1 --2- 172.21.15.83:0/4290179206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c48e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.730+0000 7fbf3e800640 1 -- 172.21.15.83:0/4290179206 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562909b9bf50 con 0x562909c3ba30 2024-04-23T14:55:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.730+0000 7fbf3e800640 1 --2- 172.21.15.83:0/4290179206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3ba30 0x562909c3be10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009be0 tx=0x7fbf38032e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2024954c82f9f1e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.730+0000 7fbf36a00640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fbf200a3d20 0x7fbf200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.730+0000 7fbf36a00640 1 -- 172.21.15.83:0/3159580798 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fbf300d0a90 con 0x7fbf40108580 2024-04-23T14:55:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.730+0000 7fbf3de00640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fbf200a3d20 0x7fbf200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.730+0000 7fbf2f400640 1 -- 172.21.15.83:0/4290179206 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38041070 con 0x562909c3ba30 2024-04-23T14:55:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.730+0000 7fbf2f400640 1 -- 172.21.15.83:0/4290179206 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf380305a0 con 0x562909c3ba30 2024-04-23T14:55:49.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.730+0000 7fbf2f400640 1 -- 172.21.15.83:0/4290179206 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38036040 con 0x562909c3ba30 2024-04-23T14:55:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.731+0000 7fbf4801a740 1 -- 172.21.15.83:0/4290179206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3ba30 msgr2=0x562909c3be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.731+0000 7fbf4801a740 1 --2- 172.21.15.83:0/4290179206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3ba30 0x562909c3be10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009be0 tx=0x7fbf38032e10 comp rx=0 tx=0).stop 2024-04-23T14:55:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.731+0000 7fbf3de00640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fbf200a3d20 0x7fbf200a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x562909c3cff0 tx=0x7fbf300d4540 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.731+0000 7fbf4801a740 1 -- 172.21.15.83:0/4290179206 shutdown_connections 2024-04-23T14:55:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.731+0000 7fbf4801a740 1 --2- 172.21.15.83:0/4290179206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c48e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.731+0000 7fbf4801a740 1 --2- 172.21.15.83:0/4290179206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3c350 0x562909c464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.731+0000 7fbf4801a740 1 --2- 172.21.15.83:0/4290179206 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3ba30 0x562909c3be10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.731+0000 7fbf4801a740 1 -- 172.21.15.83:0/4290179206 >> 172.21.15.83:0/4290179206 conn(0x562909c37d50 msgr2=0x562909c38130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.731+0000 7fbf4801a740 1 -- 172.21.15.83:0/4290179206 shutdown_connections 2024-04-23T14:55:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.731+0000 7fbf4801a740 1 -- 172.21.15.83:0/4290179206 wait complete. 2024-04-23T14:55:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.732+0000 7fbf4801a740 1 -- start start 2024-04-23T14:55:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.732+0000 7fbf4801a740 1 Processor -- start 2024-04-23T14:55:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.732+0000 7fbf4801a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3ba30 0x562909c439a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.732+0000 7fbf4801a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3c350 0x562909c43ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.732+0000 7fbf4801a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c44420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.732+0000 7fbf4801a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562909c497b0 con 0x562909c46a20 2024-04-23T14:55:49.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf4801a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562909b5a690 con 0x562909c3c350 2024-04-23T14:55:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf4801a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562909c49930 con 0x562909c3ba30 2024-04-23T14:55:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf4801a740 10 client.0 _ll_register_callbacks cb 0x562909b96650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf3e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c44420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf3e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c44420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:50990/0 (socket says 172.21.15.83:50990) 2024-04-23T14:55:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf3e800640 1 -- 172.21.15.83:0/1834773487 learned_addr learned my addr 172.21.15.83:0/1834773487 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf37e00640 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3c350 0x562909c43ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf3e800640 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3ba30 0x562909c439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf37e00640 1 -- 172.21.15.83:0/1834773487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3ba30 msgr2=0x562909c439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:49.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf37e00640 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3ba30 0x562909c439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf37e00640 1 -- 172.21.15.83:0/1834773487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 msgr2=0x562909c44420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:49.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf37e00640 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c44420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf37e00640 1 -- 172.21.15.83:0/1834773487 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562909b59920 con 0x562909c3c350 2024-04-23T14:55:49.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.733+0000 7fbf3e800640 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c44420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:49.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.734+0000 7fbf37e00640 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3c350 0x562909c43ee0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf40107eb0 tx=0x7fbf28012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52281d2f4689191a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:49.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.734+0000 7fbf27e00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:55:49.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.734+0000 7fbf27e00640 1 -- 172.21.15.83:0/1834773487 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2800f8f0 con 0x562909c3c350 2024-04-23T14:55:49.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.734+0000 7fbf27e00640 1 -- 172.21.15.83:0/1834773487 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fbf2800fa90 con 0x562909c3c350 2024-04-23T14:55:49.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.734+0000 7fbf4801a740 1 -- 172.21.15.83:0/1834773487 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf380305a0 con 0x562909c3c350 2024-04-23T14:55:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.734+0000 7fbf27e00640 1 -- 172.21.15.83:0/1834773487 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbf2800fc90 con 0x562909c3c350 2024-04-23T14:55:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.734+0000 7fbf4801a740 10 client.26630 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.734+0000 7fbf4801a740 1 -- 172.21.15.83:0/1834773487 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56290997d7d0 con 0x562909c3c350 2024-04-23T14:55:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.734+0000 7fbf27e00640 1 -- 172.21.15.83:0/1834773487 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf380305a0 con 0x562909c3c350 2024-04-23T14:55:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.735+0000 7fbf27e00640 1 -- 172.21.15.83:0/1834773487 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fbf2801c420 con 0x562909c3c350 2024-04-23T14:55:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.735+0000 7fbf4801a740 10 client.26630 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.735+0000 7fbf4801a740 10 client.26630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:49.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.735+0000 7fbf4801a740 20 client.26630 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.735+0000 7fbf4801a740 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562909c56f00 0x562909c59320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.735+0000 7fbf4801a740 4 client.26630 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:55:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.735+0000 7fbf4801a740 1 -- 172.21.15.83:0/1834773487 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562909baec10 con 0x562909c56f00 2024-04-23T14:55:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.735+0000 7fbf3e800640 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562909c56f00 0x562909c59320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.736+0000 7fbf27e00640 10 client.26630 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.736+0000 7fbf3e800640 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562909c56f00 0x562909c59320 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4db57c6a5523983 server_cookie=95eba3f9eb08b997 in_seq=0 out_seq=0 2024-04-23T14:55:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.736+0000 7fbf27e00640 1 -- 172.21.15.83:0/1834773487 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562909baec10 con 0x562909c56f00 2024-04-23T14:55:49.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.736+0000 7fbf27e00640 10 client.26630 handle_command_reply: tid=0 2024-04-23T14:55:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.736+0000 7fbf4801a740 1 client.26630 shutdown 2024-04-23T14:55:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.737+0000 7fbf4801a740 1 -- 172.21.15.83:0/1834773487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3c350 msgr2=0x562909c43ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.737+0000 7fbf4801a740 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3c350 0x562909c43ee0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf40107eb0 tx=0x7fbf28012b60 comp rx=0 tx=0).stop 2024-04-23T14:55:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.737+0000 7fbf4801a740 1 -- 172.21.15.83:0/1834773487 shutdown_connections 2024-04-23T14:55:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.737+0000 7fbf4801a740 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562909c56f00 0x562909c59320 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.737+0000 7fbf4801a740 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c44420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.737+0000 7fbf4801a740 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3c350 0x562909c43ee0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.737+0000 7fbf27e00640 0 client.26630 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.737+0000 7fbf4801a740 1 --2- 172.21.15.83:0/1834773487 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3ba30 0x562909c439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.737+0000 7fbf4801a740 1 -- 172.21.15.83:0/1834773487 >> 172.21.15.83:0/1834773487 conn(0x562909c37d50 msgr2=0x562909c36fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.737+0000 7fbf4801a740 1 -- 172.21.15.83:0/1834773487 shutdown_connections 2024-04-23T14:55:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.737+0000 7fbf4801a740 1 -- 172.21.15.83:0/1834773487 wait complete. 2024-04-23T14:55:49.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.737+0000 7fbf4801a740 20 client.26630 trim_cache size 0 max 16384 2024-04-23T14:55:49.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:49 smithi083 ceph-mon[27867]: pgmap v226: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:55:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.756+0000 7fbf4801a740 1 Processor -- start 2024-04-23T14:55:49.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.757+0000 7fbf4801a740 1 -- start start 2024-04-23T14:55:49.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.757+0000 7fbf4801a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3ba30 0x562909c52700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.757+0000 7fbf4801a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3c350 0x562909c441f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.757+0000 7fbf4801a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c5ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.757+0000 7fbf4801a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b6960 con 0x562909c46a20 2024-04-23T14:55:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.757+0000 7fbf4801a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b4230 con 0x562909c3c350 2024-04-23T14:55:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.757+0000 7fbf4801a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b20a0 con 0x562909c3ba30 2024-04-23T14:55:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.757+0000 7fbf3e800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3ba30 0x562909c52700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.757+0000 7fbf3e800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3ba30 0x562909c52700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42366/0 (socket says 172.21.15.83:42366) 2024-04-23T14:55:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.757+0000 7fbf3e800640 1 -- 172.21.15.83:0/522883042 learned_addr learned my addr 172.21.15.83:0/522883042 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.757+0000 7fbf37e00640 1 --2- 172.21.15.83:0/522883042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3c350 0x562909c441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.757+0000 7fbf3e800640 1 --2- 172.21.15.83:0/522883042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c5ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.758+0000 7fbf3e800640 1 -- 172.21.15.83:0/522883042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3c350 msgr2=0x562909c441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.758+0000 7fbf3e800640 1 --2- 172.21.15.83:0/522883042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3c350 0x562909c441f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.758+0000 7fbf3e800640 1 -- 172.21.15.83:0/522883042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 msgr2=0x562909c5ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.758+0000 7fbf3e800640 1 --2- 172.21.15.83:0/522883042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c5ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.758+0000 7fbf3e800640 1 -- 172.21.15.83:0/522883042 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629098b33f0 con 0x562909c3ba30 2024-04-23T14:55:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.758+0000 7fbf3e800640 1 --2- 172.21.15.83:0/522883042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3ba30 0x562909c52700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbf38003d10 tx=0x7fbf38003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7d64739aa66c310 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.758+0000 7fbf2ea00640 1 -- 172.21.15.83:0/522883042 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38036040 con 0x562909c3ba30 2024-04-23T14:55:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.758+0000 7fbf2ea00640 1 -- 172.21.15.83:0/522883042 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf380305a0 con 0x562909c3ba30 2024-04-23T14:55:49.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.758+0000 7fbf2ea00640 1 -- 172.21.15.83:0/522883042 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38041040 con 0x562909c3ba30 2024-04-23T14:55:49.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.759+0000 7fbf4801a740 1 -- 172.21.15.83:0/522883042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3ba30 msgr2=0x562909c52700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:49.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.759+0000 7fbf4801a740 1 --2- 172.21.15.83:0/522883042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3ba30 0x562909c52700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbf38003d10 tx=0x7fbf38003d40 comp rx=0 tx=0).stop 2024-04-23T14:55:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.759+0000 7fbf4801a740 1 -- 172.21.15.83:0/522883042 shutdown_connections 2024-04-23T14:55:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.759+0000 7fbf4801a740 1 --2- 172.21.15.83:0/522883042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c46a20 0x562909c5ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.759+0000 7fbf4801a740 1 --2- 172.21.15.83:0/522883042 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c3c350 0x562909c441f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.759+0000 7fbf4801a740 1 --2- 172.21.15.83:0/522883042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3ba30 0x562909c52700 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.759+0000 7fbf4801a740 1 -- 172.21.15.83:0/522883042 >> 172.21.15.83:0/522883042 conn(0x562909c37d50 msgr2=0x562909c428e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.759+0000 7fbf4801a740 1 -- 172.21.15.83:0/522883042 shutdown_connections 2024-04-23T14:55:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.759+0000 7fbf4801a740 1 -- 172.21.15.83:0/522883042 wait complete. 2024-04-23T14:55:49.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.760+0000 7fbf4801a740 1 -- start start 2024-04-23T14:55:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.760+0000 7fbf4801a740 1 Processor -- start 2024-04-23T14:55:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.760+0000 7fbf4801a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c3ba30 0x562909c40790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.760+0000 7fbf4801a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3c350 0x562909c40cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.760+0000 7fbf4801a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c46a20 0x562909c41210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.760+0000 7fbf4801a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b26f0 con 0x562909c3ba30 2024-04-23T14:55:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.760+0000 7fbf4801a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b3b60 con 0x562909c46a20 2024-04-23T14:55:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.760+0000 7fbf4801a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b2dc0 con 0x562909c3c350 2024-04-23T14:55:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.760+0000 7fbf4801a740 10 client.0 _ll_register_callbacks cb 0x7fbf28009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:49.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.761+0000 7fbf3e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c3ba30 0x562909c40790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.761+0000 7fbf3e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c3ba30 0x562909c40790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51004/0 (socket says 172.21.15.83:51004) 2024-04-23T14:55:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.761+0000 7fbf3e800640 1 -- 172.21.15.83:0/3479527366 learned_addr learned my addr 172.21.15.83:0/3479527366 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.761+0000 7fbf37e00640 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3c350 0x562909c40cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.761+0000 7fbf3e800640 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c46a20 0x562909c41210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.761+0000 7fbf3e800640 1 -- 172.21.15.83:0/3479527366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3c350 msgr2=0x562909c40cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.761+0000 7fbf3e800640 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3c350 0x562909c40cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.761+0000 7fbf3e800640 1 -- 172.21.15.83:0/3479527366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c46a20 msgr2=0x562909c41210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.761+0000 7fbf3e800640 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c46a20 0x562909c41210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.761+0000 7fbf3e800640 1 -- 172.21.15.83:0/3479527366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629098730f0 con 0x562909c3ba30 2024-04-23T14:55:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.761+0000 7fbf3e800640 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c3ba30 0x562909c40790 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbf3802a710 tx=0x7fbf38003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a50991bd9a5f2bd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.761+0000 7fbf27e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:55:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf27e00640 1 -- 172.21.15.83:0/3479527366 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38041070 con 0x562909c3ba30 2024-04-23T14:55:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf27e00640 1 -- 172.21.15.83:0/3479527366 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fbf38004dc0 con 0x562909c3ba30 2024-04-23T14:55:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf4801a740 1 -- 172.21.15.83:0/3479527366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf380305a0 con 0x562909c3ba30 2024-04-23T14:55:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf27e00640 1 -- 172.21.15.83:0/3479527366 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbf38030a20 con 0x562909c3ba30 2024-04-23T14:55:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf4801a740 10 client.16434 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf4801a740 1 -- 172.21.15.83:0/3479527366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562909b66810 con 0x562909c3ba30 2024-04-23T14:55:49.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf27e00640 1 -- 172.21.15.83:0/3479527366 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf380305a0 con 0x562909c3ba30 2024-04-23T14:55:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf27e00640 1 -- 172.21.15.83:0/3479527366 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fbf3803d460 con 0x562909c3ba30 2024-04-23T14:55:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf4801a740 10 client.16434 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf4801a740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf4801a740 20 client.16434 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf4801a740 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562909cbd7b0 0x562909cbfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf4801a740 4 client.16434 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:55:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.762+0000 7fbf4801a740 1 -- 172.21.15.83:0/3479527366 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562909ba8a80 con 0x562909cbd7b0 2024-04-23T14:55:49.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.763+0000 7fbf37e00640 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562909cbd7b0 0x562909cbfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.763+0000 7fbf37e00640 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562909cbd7b0 0x562909cbfba0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=746995642df91a8f server_cookie=226ab0368b0379db in_seq=0 out_seq=0 2024-04-23T14:55:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.763+0000 7fbf27e00640 10 client.16434 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.763+0000 7fbf27e00640 1 -- 172.21.15.83:0/3479527366 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562909ba8a80 con 0x562909cbd7b0 2024-04-23T14:55:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.763+0000 7fbf27e00640 10 client.16434 handle_command_reply: tid=0 2024-04-23T14:55:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.763+0000 7fbf4801a740 1 client.16434 shutdown 2024-04-23T14:55:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.764+0000 7fbf4801a740 1 -- 172.21.15.83:0/3479527366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c3ba30 msgr2=0x562909c40790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.764+0000 7fbf4801a740 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c3ba30 0x562909c40790 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbf3802a710 tx=0x7fbf38003bf0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.764+0000 7fbf4801a740 1 -- 172.21.15.83:0/3479527366 shutdown_connections 2024-04-23T14:55:49.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.764+0000 7fbf4801a740 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562909cbd7b0 0x562909cbfba0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.764+0000 7fbf4801a740 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562909c46a20 0x562909c41210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.764+0000 7fbf4801a740 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562909c3c350 0x562909c40cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.764+0000 7fbf4801a740 1 --2- 172.21.15.83:0/3479527366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562909c3ba30 0x562909c40790 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.764+0000 7fbf4801a740 1 -- 172.21.15.83:0/3479527366 >> 172.21.15.83:0/3479527366 conn(0x562909c37d50 msgr2=0x562909c441f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.764+0000 7fbf27e00640 0 client.16434 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.764+0000 7fbf4801a740 1 -- 172.21.15.83:0/3479527366 shutdown_connections 2024-04-23T14:55:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.764+0000 7fbf4801a740 1 -- 172.21.15.83:0/3479527366 wait complete. 2024-04-23T14:55:49.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.764+0000 7fbf4801a740 20 client.16434 trim_cache size 0 max 16384 2024-04-23T14:55:49.772 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:55:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:55:49.772 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:55:49.772 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:55:49.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.765+0000 7fbf3f200640 1 -- 172.21.15.83:0/3159580798 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fbf200a3d20 msgr2=0x7fbf200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:49.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.765+0000 7fbf3f200640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fbf200a3d20 0x7fbf200a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x562909c3cff0 tx=0x7fbf300d4540 comp rx=0 tx=0).stop 2024-04-23T14:55:49.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.765+0000 7fbf3f200640 1 -- 172.21.15.83:0/3159580798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40108580 msgr2=0x7fbf401a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:49.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.765+0000 7fbf3f200640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40108580 0x7fbf401a4900 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000e9f0 tx=0x7fbf3000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.765+0000 7fbf3f200640 1 -- 172.21.15.83:0/3159580798 shutdown_connections 2024-04-23T14:55:49.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.765+0000 7fbf3f200640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf4010d500 0x7fbf4019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.765+0000 7fbf3f200640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf40108f40 0x7fbf401a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.765+0000 7fbf3f200640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fbf200a3d20 0x7fbf200a61e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.765+0000 7fbf3f200640 1 --2- 172.21.15.83:0/3159580798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40108580 0x7fbf401a4900 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.765+0000 7fbf3f200640 1 -- 172.21.15.83:0/3159580798 >> 172.21.15.83:0/3159580798 conn(0x7fbf40077e20 msgr2=0x7fbf400782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:49.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.765+0000 7fbf3f200640 1 -- 172.21.15.83:0/3159580798 shutdown_connections 2024-04-23T14:55:49.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.765+0000 7fbf3f200640 1 -- 172.21.15.83:0/3159580798 wait complete. 2024-04-23T14:55:49.784 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:55:49.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:55:49.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:49 smithi173 ceph-mon[32443]: pgmap v226: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:55:49.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.988+0000 7f6ede200640 1 -- 172.21.15.83:0/3155924781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed8104c20 msgr2=0x7f6ed8108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:49.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.988+0000 7f6ede200640 1 --2- 172.21.15.83:0/3155924781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed8104c20 0x7f6ed8108c60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009900 tx=0x7f6ec002a3f0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.988+0000 7f6ede200640 1 -- 172.21.15.83:0/3155924781 shutdown_connections 2024-04-23T14:55:49.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.988+0000 7f6ede200640 1 --2- 172.21.15.83:0/3155924781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8109340 0x7f6ed8110f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.988+0000 7f6ede200640 1 --2- 172.21.15.83:0/3155924781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed8104c20 0x7f6ed8108c60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.988+0000 7f6ede200640 1 --2- 172.21.15.83:0/3155924781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8104270 0x7f6ed8104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.988+0000 7f6ede200640 1 -- 172.21.15.83:0/3155924781 >> 172.21.15.83:0/3155924781 conn(0x7f6ed80ffad0 msgr2=0x7f6ed8101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:49.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.988+0000 7f6ede200640 1 -- 172.21.15.83:0/3155924781 shutdown_connections 2024-04-23T14:55:49.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.988+0000 7f6ede200640 1 -- 172.21.15.83:0/3155924781 wait complete. 2024-04-23T14:55:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.989+0000 7f6ede200640 1 Processor -- start 2024-04-23T14:55:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.990+0000 7f6ede200640 1 -- start start 2024-04-23T14:55:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.990+0000 7f6ede200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed8104270 0x7f6ed81a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.990+0000 7f6ede200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8109340 0x7f6ed81a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.990+0000 7f6ede200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed819a840 0x7f6ed819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.990+0000 7f6ede200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8114400 con 0x7f6ed8109340 2024-04-23T14:55:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.990+0000 7f6ede200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8114280 con 0x7f6ed819a840 2024-04-23T14:55:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.990+0000 7f6ede200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8114580 con 0x7f6ed8104270 2024-04-23T14:55:49.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.990+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8109340 0x7f6ed81a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.991+0000 7f6edce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed8104270 0x7f6ed81a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.991+0000 7f6ecfe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8109340 0x7f6ed81a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51022/0 (socket says 172.21.15.83:51022) 2024-04-23T14:55:49.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.991+0000 7f6ecfe00640 1 -- 172.21.15.83:0/3659266419 learned_addr learned my addr 172.21.15.83:0/3659266419 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:49.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.991+0000 7f6edd800640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed819a840 0x7f6ed819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.991+0000 7f6ecfe00640 1 -- 172.21.15.83:0/3659266419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed8104270 msgr2=0x7f6ed81a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:49.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.991+0000 7f6ecfe00640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed8104270 0x7f6ed81a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.991+0000 7f6ecfe00640 1 -- 172.21.15.83:0/3659266419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed819a840 msgr2=0x7f6ed819acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:49.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.991+0000 7f6ecfe00640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed819a840 0x7f6ed819acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:49.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.991+0000 7f6ecfe00640 1 -- 172.21.15.83:0/3659266419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed819b420 con 0x7f6ed8109340 2024-04-23T14:55:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.991+0000 7f6edce00640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed8104270 0x7f6ed81a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.991+0000 7f6ecfe00640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8109340 0x7f6ed81a0c30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6ec002a910 tx=0x7f6ec0003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.992+0000 7f6ed6000640 1 -- 172.21.15.83:0/3659266419 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec003f070 con 0x7f6ed8109340 2024-04-23T14:55:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.992+0000 7f6ed6000640 1 -- 172.21.15.83:0/3659266419 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ec0031c90 con 0x7f6ed8109340 2024-04-23T14:55:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.992+0000 7f6ede200640 1 -- 172.21.15.83:0/3659266419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed81a7500 con 0x7f6ed8109340 2024-04-23T14:55:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.992+0000 7f6ed6000640 1 -- 172.21.15.83:0/3659266419 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0035030 con 0x7f6ed8109340 2024-04-23T14:55:49.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.992+0000 7f6ede200640 1 -- 172.21.15.83:0/3659266419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed81a78b0 con 0x7f6ed8109340 2024-04-23T14:55:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.993+0000 7f6ede200640 1 -- 172.21.15.83:0/3659266419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed8105740 con 0x7f6ed8109340 2024-04-23T14:55:49.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.995+0000 7f6ed6000640 1 -- 172.21.15.83:0/3659266419 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ec0042070 con 0x7f6ed8109340 2024-04-23T14:55:49.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.996+0000 7f6ed6000640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f6ebc0a3d20 0x7f6ebc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:49.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.997+0000 7f6ed6000640 1 -- 172.21.15.83:0/3659266419 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f6ec00f0ec0 con 0x7f6ed8109340 2024-04-23T14:55:49.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.997+0000 7f6edce00640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f6ebc0a3d20 0x7f6ebc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:49.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:49.997+0000 7f6edce00640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f6ebc0a3d20 0x7f6ebc0a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009fc0 tx=0x7f6ec8005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.001+0000 7f6ed6000640 1 -- 172.21.15.83:0/3659266419 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ec00bd4f0 con 0x7f6ed8109340 2024-04-23T14:55:50.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.243+0000 7f6ede200640 1 -- 172.21.15.83:0/3659266419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ed810c570 con 0x7f6ed8109340 2024-04-23T14:55:50.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.245+0000 7f6ed6000640 1 -- 172.21.15.83:0/3659266419 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7f6ec00bcdf0 con 0x7f6ed8109340 2024-04-23T14:55:50.245 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:55:50.245 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:55:50.245 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:55:50.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.247+0000 7f6ede200640 1 -- 172.21.15.83:0/3659266419 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f6ebc0a3d20 msgr2=0x7f6ebc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.247+0000 7f6ede200640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f6ebc0a3d20 0x7f6ebc0a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009fc0 tx=0x7f6ec8005f90 comp rx=0 tx=0).stop 2024-04-23T14:55:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.247+0000 7f6ede200640 1 -- 172.21.15.83:0/3659266419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8109340 msgr2=0x7f6ed81a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.247+0000 7f6ede200640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8109340 0x7f6ed81a0c30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6ec002a910 tx=0x7f6ec0003b20 comp rx=0 tx=0).stop 2024-04-23T14:55:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.247+0000 7f6ede200640 1 -- 172.21.15.83:0/3659266419 shutdown_connections 2024-04-23T14:55:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.247+0000 7f6ede200640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed819a840 0x7f6ed819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.247+0000 7f6ede200640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f6ebc0a3d20 0x7f6ebc0a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.247+0000 7f6ede200640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8109340 0x7f6ed81a0c30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.247+0000 7f6ede200640 1 --2- 172.21.15.83:0/3659266419 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed8104270 0x7f6ed81a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.247+0000 7f6ede200640 1 -- 172.21.15.83:0/3659266419 >> 172.21.15.83:0/3659266419 conn(0x7f6ed80ffad0 msgr2=0x7f6ed8101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.247+0000 7f6ede200640 1 -- 172.21.15.83:0/3659266419 shutdown_connections 2024-04-23T14:55:50.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.248+0000 7f6ede200640 1 -- 172.21.15.83:0/3659266419 wait complete. 2024-04-23T14:55:50.268 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:55:50.467 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.467+0000 7f90e4400640 1 -- 172.21.15.83:0/3497597664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90dc109520 msgr2=0x7f90dc1110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.467+0000 7f90e4400640 1 --2- 172.21.15.83:0/3497597664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90dc109520 0x7f90dc1110f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009be0 tx=0x7f90d402a6d0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.467+0000 7f90e4400640 1 -- 172.21.15.83:0/3497597664 shutdown_connections 2024-04-23T14:55:50.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.467+0000 7f90e4400640 1 --2- 172.21.15.83:0/3497597664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90dc109520 0x7f90dc1110f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.467+0000 7f90e4400640 1 --2- 172.21.15.83:0/3497597664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90dc104e00 0x7f90dc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.467+0000 7f90e4400640 1 --2- 172.21.15.83:0/3497597664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90dc104450 0x7f90dc104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.467+0000 7f90e4400640 1 -- 172.21.15.83:0/3497597664 >> 172.21.15.83:0/3497597664 conn(0x7f90dc0ffc70 msgr2=0x7f90dc102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.467+0000 7f90e4400640 1 -- 172.21.15.83:0/3497597664 shutdown_connections 2024-04-23T14:55:50.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.467+0000 7f90e4400640 1 -- 172.21.15.83:0/3497597664 wait complete. 2024-04-23T14:55:50.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.468+0000 7f90e4400640 1 Processor -- start 2024-04-23T14:55:50.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.468+0000 7f90e4400640 1 -- start start 2024-04-23T14:55:50.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e4400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90dc104450 0x7f90dc1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e4400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90dc109520 0x7f90dc1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e4400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90dc19a8a0 0x7f90dc19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e4400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1145c0 con 0x7f90dc19a8a0 2024-04-23T14:55:50.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e4400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc114440 con 0x7f90dc104450 2024-04-23T14:55:50.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e4400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc114740 con 0x7f90dc109520 2024-04-23T14:55:50.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e1a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90dc104450 0x7f90dc1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e1a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90dc104450 0x7f90dc1a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:42714/0 (socket says 172.21.15.83:42714) 2024-04-23T14:55:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e1a00640 1 -- 172.21.15.83:0/1033445877 learned_addr learned my addr 172.21.15.83:0/1033445877 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e1000640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90dc109520 0x7f90dc1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e2400640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90dc19a8a0 0x7f90dc19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e1a00640 1 -- 172.21.15.83:0/1033445877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90dc109520 msgr2=0x7f90dc1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e1a00640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90dc109520 0x7f90dc1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e1a00640 1 -- 172.21.15.83:0/1033445877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90dc19a8a0 msgr2=0x7f90dc19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e1a00640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90dc19a8a0 0x7f90dc19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.469+0000 7f90e1a00640 1 -- 172.21.15.83:0/1033445877 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc19b3c0 con 0x7f90dc104450 2024-04-23T14:55:50.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.470+0000 7f90e1a00640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90dc104450 0x7f90dc1a06f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f90d000e9c0 tx=0x7f90d000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.470+0000 7f90da000640 1 -- 172.21.15.83:0/1033445877 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d000cde0 con 0x7f90dc104450 2024-04-23T14:55:50.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.470+0000 7f90da000640 1 -- 172.21.15.83:0/1033445877 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90d00035f0 con 0x7f90dc104450 2024-04-23T14:55:50.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.470+0000 7f90da000640 1 -- 172.21.15.83:0/1033445877 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0010650 con 0x7f90dc104450 2024-04-23T14:55:50.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.470+0000 7f90e4400640 1 -- 172.21.15.83:0/1033445877 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90dc19b6b0 con 0x7f90dc104450 2024-04-23T14:55:50.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.470+0000 7f90e4400640 1 -- 172.21.15.83:0/1033445877 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90dc1a78b0 con 0x7f90dc104450 2024-04-23T14:55:50.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.473+0000 7f90e71d3740 1 Processor -- start 2024-04-23T14:55:50.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e71d3740 1 -- start start 2024-04-23T14:55:50.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90da000640 1 -- 172.21.15.83:0/1033445877 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f90d00037a0 con 0x7f90dc104450 2024-04-23T14:55:50.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e71d3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f5936d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e71d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5937270 0x55a4f5941400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e71d3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5941940 0x55a4f5943d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e71d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a4f589c1d0 con 0x55a4f5937270 2024-04-23T14:55:50.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e71d3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a4f5842450 con 0x55a4f5941940 2024-04-23T14:55:50.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e71d3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a4f5898190 con 0x55a4f5936950 2024-04-23T14:55:50.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5937270 0x55a4f5941400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e1000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5941940 0x55a4f5943d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e1a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5937270 0x55a4f5941400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51048/0 (socket says 172.21.15.83:51048) 2024-04-23T14:55:50.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e1000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5941940 0x55a4f5943d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:42718/0 (socket says 172.21.15.83:42718) 2024-04-23T14:55:50.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e1a00640 1 -- 172.21.15.83:0/859010444 learned_addr learned my addr 172.21.15.83:0/859010444 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.474+0000 7f90e2400640 1 --2- 172.21.15.83:0/859010444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f5936d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.475+0000 7f90e2400640 1 -- 172.21.15.83:0/859010444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5941940 msgr2=0x55a4f5943d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.475+0000 7f90e2400640 1 --2- 172.21.15.83:0/859010444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5941940 0x55a4f5943d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.475+0000 7f90e2400640 1 -- 172.21.15.83:0/859010444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5937270 msgr2=0x55a4f5941400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.475+0000 7f90e2400640 1 --2- 172.21.15.83:0/859010444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5937270 0x55a4f5941400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.475+0000 7f90e2400640 1 -- 172.21.15.83:0/859010444 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f58ff310 con 0x55a4f5936950 2024-04-23T14:55:50.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.475+0000 7f90e2400640 1 --2- 172.21.15.83:0/859010444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f5936d30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009b30 tx=0x7f90d4030a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0a886ce2db51388 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.475+0000 7f90da000640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f90c40a3d20 0x7f90c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.475+0000 7f90caa00640 1 -- 172.21.15.83:0/859010444 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4041030 con 0x55a4f5936950 2024-04-23T14:55:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.475+0000 7f90caa00640 1 -- 172.21.15.83:0/859010444 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90d4030c60 con 0x55a4f5936950 2024-04-23T14:55:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.475+0000 7f90da000640 1 -- 172.21.15.83:0/1033445877 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f90d00d0b10 con 0x7f90dc104450 2024-04-23T14:55:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.475+0000 7f90caa00640 1 -- 172.21.15.83:0/859010444 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4036040 con 0x55a4f5936950 2024-04-23T14:55:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.476+0000 7f90e1a00640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f90c40a3d20 0x7f90c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.476+0000 7f90e71d3740 1 -- 172.21.15.83:0/859010444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 msgr2=0x55a4f5936d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.476+0000 7f90e1a00640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f90c40a3d20 0x7f90c40a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f90d0098f00 tx=0x7f90d00d4650 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.476+0000 7f90e71d3740 1 --2- 172.21.15.83:0/859010444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f5936d30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009b30 tx=0x7f90d4030a30 comp rx=0 tx=0).stop 2024-04-23T14:55:50.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.476+0000 7f90e71d3740 1 -- 172.21.15.83:0/859010444 shutdown_connections 2024-04-23T14:55:50.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.476+0000 7f90e71d3740 1 --2- 172.21.15.83:0/859010444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5941940 0x55a4f5943d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.476+0000 7f90e71d3740 1 --2- 172.21.15.83:0/859010444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5937270 0x55a4f5941400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.476+0000 7f90e71d3740 1 --2- 172.21.15.83:0/859010444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f5936d30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.476+0000 7f90e71d3740 1 -- 172.21.15.83:0/859010444 >> 172.21.15.83:0/859010444 conn(0x55a4f5932c70 msgr2=0x55a4f5933050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.477+0000 7f90e71d3740 1 -- 172.21.15.83:0/859010444 shutdown_connections 2024-04-23T14:55:50.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.477+0000 7f90e71d3740 1 -- 172.21.15.83:0/859010444 wait complete. 2024-04-23T14:55:50.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.477+0000 7f90e71d3740 1 -- start start 2024-04-23T14:55:50.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.477+0000 7f90e71d3740 1 Processor -- start 2024-04-23T14:55:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e71d3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f593e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e71d3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5937270 0x55a4f593ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e71d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5941940 0x55a4f593f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e71d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a4f5944770 con 0x55a4f5941940 2024-04-23T14:55:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e71d3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55a4f5855810 con 0x55a4f5937270 2024-04-23T14:55:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e71d3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55a4f59448f0 con 0x55a4f5936950 2024-04-23T14:55:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e71d3740 10 client.0 _ll_register_callbacks cb 0x55a4f5823140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e2400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f593e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e2400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f593e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42436/0 (socket says 172.21.15.83:42436) 2024-04-23T14:55:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e2400640 1 -- 172.21.15.83:0/1220041506 learned_addr learned my addr 172.21.15.83:0/1220041506 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e1000640 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5937270 0x55a4f593ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e2400640 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5941940 0x55a4f593f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e2400640 1 -- 172.21.15.83:0/1220041506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5937270 msgr2=0x55a4f593ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e2400640 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5937270 0x55a4f593ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e2400640 1 -- 172.21.15.83:0/1220041506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5941940 msgr2=0x55a4f593f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e2400640 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5941940 0x55a4f593f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.478+0000 7f90e2400640 1 -- 172.21.15.83:0/1220041506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f58663f0 con 0x55a4f5936950 2024-04-23T14:55:50.484 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.479+0000 7f90e2400640 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f593e710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f90d4003fd0 tx=0x7f90d4009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39c59cc9525b2d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.479+0000 7f90bf400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:55:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.479+0000 7f90bf400640 1 -- 172.21.15.83:0/1220041506 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4036050 con 0x55a4f5936950 2024-04-23T14:55:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.479+0000 7f90e71d3740 1 -- 172.21.15.83:0/1220041506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d4030c60 con 0x55a4f5936950 2024-04-23T14:55:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.479+0000 7f90bf400640 1 -- 172.21.15.83:0/1220041506 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f90d4003970 con 0x55a4f5936950 2024-04-23T14:55:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.479+0000 7f90bf400640 1 -- 172.21.15.83:0/1220041506 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f90d4059050 con 0x55a4f5936950 2024-04-23T14:55:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.479+0000 7f90e71d3740 10 client.26632 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.479+0000 7f90e71d3740 1 -- 172.21.15.83:0/1220041506 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a4f55ae3f0 con 0x55a4f5936950 2024-04-23T14:55:50.485 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.480+0000 7f90bf400640 1 -- 172.21.15.83:0/1220041506 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90d4030c60 con 0x55a4f5936950 2024-04-23T14:55:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.480+0000 7f90bf400640 1 -- 172.21.15.83:0/1220041506 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f90d4004690 con 0x55a4f5936950 2024-04-23T14:55:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.480+0000 7f90e71d3740 10 client.26632 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.480+0000 7f90e71d3740 10 client.26632 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:55:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.480+0000 7f90e71d3740 20 client.26632 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.480+0000 7f90e71d3740 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a4f59519e0 0x55a4f5953dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.480+0000 7f90e71d3740 4 client.26632 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:55:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.480+0000 7f90e71d3740 1 -- 172.21.15.83:0/1220041506 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4f58a9bf0 con 0x55a4f59519e0 2024-04-23T14:55:50.486 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.480+0000 7f90e1000640 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a4f59519e0 0x55a4f5953dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.487 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.481+0000 7f90e1000640 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a4f59519e0 0x55a4f5953dd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6b8203e09bbd489 server_cookie=8c0999b4647026dd in_seq=0 out_seq=0 2024-04-23T14:55:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.481+0000 7f90bf400640 10 client.26632 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:55:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90bf400640 1 -- 172.21.15.83:0/1220041506 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4f58a9bf0 con 0x55a4f59519e0 2024-04-23T14:55:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90bf400640 10 client.26632 handle_command_reply: tid=0 2024-04-23T14:55:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90e71d3740 1 client.26632 shutdown 2024-04-23T14:55:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90e71d3740 1 -- 172.21.15.83:0/1220041506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 msgr2=0x55a4f593e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90e71d3740 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f593e710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f90d4003fd0 tx=0x7f90d4009300 comp rx=0 tx=0).stop 2024-04-23T14:55:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90e71d3740 1 -- 172.21.15.83:0/1220041506 shutdown_connections 2024-04-23T14:55:50.488 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90e71d3740 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a4f59519e0 0x55a4f5953dd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90e71d3740 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5941940 0x55a4f593f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90e71d3740 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5937270 0x55a4f593ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90bf400640 0 client.26632 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:55:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90e71d3740 1 --2- 172.21.15.83:0/1220041506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f593e710 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90e71d3740 1 -- 172.21.15.83:0/1220041506 >> 172.21.15.83:0/1220041506 conn(0x55a4f5932c70 msgr2=0x55a4f5938a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90e71d3740 1 -- 172.21.15.83:0/1220041506 shutdown_connections 2024-04-23T14:55:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90e71d3740 1 -- 172.21.15.83:0/1220041506 wait complete. 2024-04-23T14:55:50.489 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.482+0000 7f90e71d3740 20 client.26632 trim_cache size 0 max 16384 2024-04-23T14:55:50.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e71d3740 1 Processor -- start 2024-04-23T14:55:50.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e71d3740 1 -- start start 2024-04-23T14:55:50.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e71d3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f594d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.498 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e71d3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5937270 0x55a4f593e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e71d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5941940 0x55a4f5932570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e71d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1b28d0 con 0x55a4f5941940 2024-04-23T14:55:50.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e71d3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1b01a0 con 0x55a4f5937270 2024-04-23T14:55:50.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e71d3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1ae010 con 0x55a4f5936950 2024-04-23T14:55:50.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e2400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5941940 0x55a4f5932570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e2400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5941940 0x55a4f5932570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51060/0 (socket says 172.21.15.83:51060) 2024-04-23T14:55:50.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e2400640 1 -- 172.21.15.83:0/2586264928 learned_addr learned my addr 172.21.15.83:0/2586264928 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e1000640 1 --2- 172.21.15.83:0/2586264928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5937270 0x55a4f593e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.497+0000 7f90e2400640 1 --2- 172.21.15.83:0/2586264928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f594d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.498+0000 7f90e2400640 1 -- 172.21.15.83:0/2586264928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 msgr2=0x55a4f594d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.498+0000 7f90e2400640 1 --2- 172.21.15.83:0/2586264928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f594d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.498+0000 7f90e2400640 1 -- 172.21.15.83:0/2586264928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5937270 msgr2=0x55a4f593e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.498+0000 7f90e2400640 1 --2- 172.21.15.83:0/2586264928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5937270 0x55a4f593e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.498+0000 7f90e2400640 1 -- 172.21.15.83:0/2586264928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f59370a0 con 0x55a4f5941940 2024-04-23T14:55:50.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.498+0000 7f90e2400640 1 --2- 172.21.15.83:0/2586264928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5941940 0x55a4f5932570 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f90d403d980 tx=0x7f90d40021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347c2d1636f84524 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.498+0000 7f90ca000640 1 -- 172.21.15.83:0/2586264928 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4036070 con 0x55a4f5941940 2024-04-23T14:55:50.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.498+0000 7f90ca000640 1 -- 172.21.15.83:0/2586264928 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90d4030c60 con 0x55a4f5941940 2024-04-23T14:55:50.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.498+0000 7f90ca000640 1 -- 172.21.15.83:0/2586264928 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4041040 con 0x55a4f5941940 2024-04-23T14:55:50.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.499+0000 7f90e71d3740 1 -- 172.21.15.83:0/2586264928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5941940 msgr2=0x55a4f5932570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.499+0000 7f90e71d3740 1 --2- 172.21.15.83:0/2586264928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5941940 0x55a4f5932570 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f90d403d980 tx=0x7f90d40021f0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.499+0000 7f90e71d3740 1 -- 172.21.15.83:0/2586264928 shutdown_connections 2024-04-23T14:55:50.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.499+0000 7f90e71d3740 1 --2- 172.21.15.83:0/2586264928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5941940 0x55a4f5932570 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.499+0000 7f90e71d3740 1 --2- 172.21.15.83:0/2586264928 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5937270 0x55a4f593e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.499+0000 7f90e71d3740 1 --2- 172.21.15.83:0/2586264928 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5936950 0x55a4f594d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.499+0000 7f90e71d3740 1 -- 172.21.15.83:0/2586264928 >> 172.21.15.83:0/2586264928 conn(0x55a4f5932c70 msgr2=0x55a4f5938930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.499+0000 7f90e71d3740 1 -- 172.21.15.83:0/2586264928 shutdown_connections 2024-04-23T14:55:50.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.499+0000 7f90e71d3740 1 -- 172.21.15.83:0/2586264928 wait complete. 2024-04-23T14:55:50.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.499+0000 7f90e71d3740 1 -- start start 2024-04-23T14:55:50.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.500+0000 7f90e71d3740 1 Processor -- start 2024-04-23T14:55:50.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.500+0000 7f90e71d3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5936950 0x55a4f593ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.500+0000 7f90e71d3740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5937270 0x55a4f593bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.500+0000 7f90e71d3740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5941940 0x55a4f5948150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.500+0000 7f90e71d3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1ae660 con 0x55a4f5936950 2024-04-23T14:55:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.500+0000 7f90e71d3740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1afad0 con 0x55a4f5941940 2024-04-23T14:55:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.500+0000 7f90e71d3740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc1aed30 con 0x55a4f5937270 2024-04-23T14:55:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.500+0000 7f90e71d3740 10 client.0 _ll_register_callbacks cb 0x55a4f5941890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.500+0000 7f90e2400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5936950 0x55a4f593ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.500+0000 7f90e2400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5936950 0x55a4f593ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51072/0 (socket says 172.21.15.83:51072) 2024-04-23T14:55:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.500+0000 7f90e2400640 1 -- 172.21.15.83:0/1341489873 learned_addr learned my addr 172.21.15.83:0/1341489873 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.500+0000 7f90e1000640 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5937270 0x55a4f593bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.501+0000 7f90e2400640 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5941940 0x55a4f5948150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.501+0000 7f90e1000640 1 -- 172.21.15.83:0/1341489873 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5941940 msgr2=0x55a4f5948150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.501+0000 7f90e1000640 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5941940 0x55a4f5948150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.501+0000 7f90e1000640 1 -- 172.21.15.83:0/1341489873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5936950 msgr2=0x55a4f593ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.501+0000 7f90e1000640 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5936950 0x55a4f593ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.501+0000 7f90e1000640 1 -- 172.21.15.83:0/1341489873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f56787d0 con 0x55a4f5937270 2024-04-23T14:55:50.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.501+0000 7f90e2400640 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5941940 0x55a4f5948150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:55:50.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.501+0000 7f90e1000640 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5937270 0x55a4f593bf80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55a4f593df70 tx=0x7f90cc004100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e3914ff32409a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.501+0000 7f90bf400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:55:50.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.501+0000 7f90bf400640 1 -- 172.21.15.83:0/1341489873 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc024070 con 0x55a4f5937270 2024-04-23T14:55:50.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.501+0000 7f90e71d3740 1 -- 172.21.15.83:0/1341489873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d4030c60 con 0x55a4f5937270 2024-04-23T14:55:50.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.502+0000 7f90bf400640 1 -- 172.21.15.83:0/1341489873 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f90cc0035f0 con 0x55a4f5937270 2024-04-23T14:55:50.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.502+0000 7f90bf400640 1 -- 172.21.15.83:0/1341489873 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f90cc005020 con 0x55a4f5937270 2024-04-23T14:55:50.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.502+0000 7f90e71d3740 10 client.26638 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:50.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.502+0000 7f90e71d3740 1 -- 172.21.15.83:0/1341489873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a4f59b8290 con 0x55a4f5937270 2024-04-23T14:55:50.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.502+0000 7f90bf400640 1 -- 172.21.15.83:0/1341489873 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90d4030c60 con 0x55a4f5937270 2024-04-23T14:55:50.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.502+0000 7f90bf400640 1 -- 172.21.15.83:0/1341489873 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f90cc027020 con 0x55a4f5937270 2024-04-23T14:55:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.502+0000 7f90e71d3740 10 client.26638 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.502+0000 7f90e71d3740 10 client.26638 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:55:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.502+0000 7f90e71d3740 20 client.26638 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.502+0000 7f90e71d3740 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a4f59b89e0 0x55a4f59bae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.502+0000 7f90e71d3740 4 client.26638 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:55:50.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.502+0000 7f90e71d3740 1 -- 172.21.15.83:0/1341489873 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a4f58a3a60 con 0x55a4f59b89e0 2024-04-23T14:55:50.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.503+0000 7f90e2400640 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a4f59b89e0 0x55a4f59bae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.503+0000 7f90e2400640 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a4f59b89e0 0x55a4f59bae00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=baaaa0d545a28ae1 server_cookie=73c3131349283d20 in_seq=0 out_seq=0 2024-04-23T14:55:50.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.503+0000 7f90bf400640 10 client.26638 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:55:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.503+0000 7f90bf400640 1 -- 172.21.15.83:0/1341489873 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a4f58a3a60 con 0x55a4f59b89e0 2024-04-23T14:55:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.503+0000 7f90bf400640 10 client.26638 handle_command_reply: tid=0 2024-04-23T14:55:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.503+0000 7f90e71d3740 1 client.26638 shutdown 2024-04-23T14:55:50.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.504+0000 7f90e71d3740 1 -- 172.21.15.83:0/1341489873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5937270 msgr2=0x55a4f593bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.504+0000 7f90e71d3740 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5937270 0x55a4f593bf80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55a4f593df70 tx=0x7f90cc004100 comp rx=0 tx=0).stop 2024-04-23T14:55:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.504+0000 7f90e71d3740 1 -- 172.21.15.83:0/1341489873 shutdown_connections 2024-04-23T14:55:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.504+0000 7f90e71d3740 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55a4f59b89e0 0x55a4f59bae00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.504+0000 7f90e71d3740 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55a4f5941940 0x55a4f5948150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.504+0000 7f90e71d3740 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a4f5937270 0x55a4f593bf80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.504+0000 7f90e71d3740 1 --2- 172.21.15.83:0/1341489873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a4f5936950 0x55a4f593ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.504+0000 7f90bf400640 0 client.26638 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:55:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.504+0000 7f90e71d3740 1 -- 172.21.15.83:0/1341489873 >> 172.21.15.83:0/1341489873 conn(0x55a4f5932c70 msgr2=0x55a4f593cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.504+0000 7f90e71d3740 1 -- 172.21.15.83:0/1341489873 shutdown_connections 2024-04-23T14:55:50.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.504+0000 7f90e71d3740 1 -- 172.21.15.83:0/1341489873 wait complete. 2024-04-23T14:55:50.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.504+0000 7f90e71d3740 20 client.26638 trim_cache size 0 max 16384 2024-04-23T14:55:50.517 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:55:50.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.505+0000 7f90e4400640 1 -- 172.21.15.83:0/1033445877 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f90c40a3d20 msgr2=0x7f90c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.505+0000 7f90e4400640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f90c40a3d20 0x7f90c40a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f90d0098f00 tx=0x7f90d00d4650 comp rx=0 tx=0).stop 2024-04-23T14:55:50.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.505+0000 7f90e4400640 1 -- 172.21.15.83:0/1033445877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90dc104450 msgr2=0x7f90dc1a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.505+0000 7f90e4400640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90dc104450 0x7f90dc1a06f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f90d000e9c0 tx=0x7f90d000ee80 comp rx=0 tx=0).stop 2024-04-23T14:55:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.505+0000 7f90e4400640 1 -- 172.21.15.83:0/1033445877 shutdown_connections 2024-04-23T14:55:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.505+0000 7f90e4400640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f90c40a3d20 0x7f90c40a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.505+0000 7f90e4400640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90dc19a8a0 0x7f90dc19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.505+0000 7f90e4400640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90dc109520 0x7f90dc1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.505+0000 7f90e4400640 1 --2- 172.21.15.83:0/1033445877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90dc104450 0x7f90dc1a06f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.505+0000 7f90e4400640 1 -- 172.21.15.83:0/1033445877 >> 172.21.15.83:0/1033445877 conn(0x7f90dc0ffc70 msgr2=0x7f90dc101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.505+0000 7f90e4400640 1 -- 172.21.15.83:0/1033445877 shutdown_connections 2024-04-23T14:55:50.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.505+0000 7f90e4400640 1 -- 172.21.15.83:0/1033445877 wait complete. 2024-04-23T14:55:50.522 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:50 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3659266419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:55:50.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:55:50.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.717+0000 7f7770c00640 1 -- 172.21.15.83:0/1099313070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c10d500 msgr2=0x7f776c10d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.717+0000 7f7770c00640 1 --2- 172.21.15.83:0/1099313070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c10d500 0x7f776c10d8e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f775c009890 tx=0x7f775c02a360 comp rx=0 tx=0).stop 2024-04-23T14:55:50.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.718+0000 7f7770c00640 1 -- 172.21.15.83:0/1099313070 shutdown_connections 2024-04-23T14:55:50.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.718+0000 7f7770c00640 1 --2- 172.21.15.83:0/1099313070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f776c108f40 0x7f776c1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.718+0000 7f7770c00640 1 --2- 172.21.15.83:0/1099313070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f776c108580 0x7f776c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.718+0000 7f7770c00640 1 --2- 172.21.15.83:0/1099313070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c10d500 0x7f776c10d8e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.718+0000 7f7770c00640 1 -- 172.21.15.83:0/1099313070 >> 172.21.15.83:0/1099313070 conn(0x7f776c077e20 msgr2=0x7f776c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.718+0000 7f7770c00640 1 -- 172.21.15.83:0/1099313070 shutdown_connections 2024-04-23T14:55:50.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.718+0000 7f7770c00640 1 -- 172.21.15.83:0/1099313070 wait complete. 2024-04-23T14:55:50.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.719+0000 7f7770c00640 1 Processor -- start 2024-04-23T14:55:50.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.719+0000 7f7770c00640 1 -- start start 2024-04-23T14:55:50.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.719+0000 7f7770c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c108580 0x7f776c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.719+0000 7f7770c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f776c108f40 0x7f776c19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.719+0000 7f7770c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f776c10d500 0x7f776c1a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.719+0000 7f7770c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1188b0 con 0x7f776c108580 2024-04-23T14:55:50.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.719+0000 7f7770c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f776c118730 con 0x7f776c108f40 2024-04-23T14:55:50.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.719+0000 7f7770c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f776c118a30 con 0x7f776c10d500 2024-04-23T14:55:50.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.719+0000 7f776b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c108580 0x7f776c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.719+0000 7f776b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c108580 0x7f776c19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51084/0 (socket says 172.21.15.83:51084) 2024-04-23T14:55:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.719+0000 7f776b400640 1 -- 172.21.15.83:0/621602308 learned_addr learned my addr 172.21.15.83:0/621602308 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.720+0000 7f776be00640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f776c10d500 0x7f776c1a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.720+0000 7f7763e00640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f776c108f40 0x7f776c19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.720+0000 7f776b400640 1 -- 172.21.15.83:0/621602308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f776c10d500 msgr2=0x7f776c1a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.720+0000 7f776b400640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f776c10d500 0x7f776c1a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.720+0000 7f776b400640 1 -- 172.21.15.83:0/621602308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f776c108f40 msgr2=0x7f776c19f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.720+0000 7f776b400640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f776c108f40 0x7f776c19f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.720+0000 7f776b400640 1 -- 172.21.15.83:0/621602308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c1a3d20 con 0x7f776c108580 2024-04-23T14:55:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.720+0000 7f776b400640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c108580 0x7f776c19ed30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f775c009780 tx=0x7f775c033ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.721+0000 7f7768c00640 1 -- 172.21.15.83:0/621602308 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c037040 con 0x7f776c108580 2024-04-23T14:55:50.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.721+0000 7f7768c00640 1 -- 172.21.15.83:0/621602308 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f775c040070 con 0x7f776c108580 2024-04-23T14:55:50.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.721+0000 7f7770c00640 1 -- 172.21.15.83:0/621602308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c1a3fb0 con 0x7f776c108580 2024-04-23T14:55:50.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.721+0000 7f7768c00640 1 -- 172.21.15.83:0/621602308 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c003890 con 0x7f776c108580 2024-04-23T14:55:50.725 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.721+0000 7f7770c00640 1 -- 172.21.15.83:0/621602308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c1ab8a0 con 0x7f776c108580 2024-04-23T14:55:50.726 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:50 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3659266419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:55:50.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.724+0000 7f7774ff5740 1 Processor -- start 2024-04-23T14:55:50.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.724+0000 7f7768c00640 1 -- 172.21.15.83:0/621602308 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f775c030580 con 0x7f776c108580 2024-04-23T14:55:50.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.724+0000 7f7774ff5740 1 -- start start 2024-04-23T14:55:50.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.725+0000 7f7774ff5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5c0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.725+0000 7f7774ff5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5c1460 0x55fede5cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.725+0000 7f7774ff5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5cbb30 0x55fede5cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.725+0000 7f7774ff5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fede5261d0 con 0x55fede5c0b40 2024-04-23T14:55:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.725+0000 7f7774ff5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fede4cc450 con 0x55fede5c1460 2024-04-23T14:55:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.725+0000 7f7774ff5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fede522190 con 0x55fede5cbb30 2024-04-23T14:55:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.725+0000 7f7763e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5cbb30 0x55fede5cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.725+0000 7f776be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5c0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.725+0000 7f7763e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5cbb30 0x55fede5cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42460/0 (socket says 172.21.15.83:42460) 2024-04-23T14:55:50.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.725+0000 7f7763e00640 1 -- 172.21.15.83:0/2477936232 learned_addr learned my addr 172.21.15.83:0/2477936232 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.725+0000 7f776b400640 1 --2- 172.21.15.83:0/2477936232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5c1460 0x55fede5cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f7763e00640 1 -- 172.21.15.83:0/2477936232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5c1460 msgr2=0x55fede5cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f7763e00640 1 --2- 172.21.15.83:0/2477936232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5c1460 0x55fede5cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f7763e00640 1 -- 172.21.15.83:0/2477936232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 msgr2=0x55fede5c0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f7763e00640 1 --2- 172.21.15.83:0/2477936232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5c0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f7763e00640 1 -- 172.21.15.83:0/2477936232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fede589310 con 0x55fede5cbb30 2024-04-23T14:55:50.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f776be00640 1 --2- 172.21.15.83:0/2477936232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5c0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:50.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f7763e00640 1 --2- 172.21.15.83:0/2477936232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5cbb30 0x55fede5cdf20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7754009c90 tx=0x7f7754011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7304fe08cd71c7d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f7768c00640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f77500a3da0 0x7f77500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f7768c00640 1 -- 172.21.15.83:0/621602308 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f775c0f0680 con 0x7f776c108580 2024-04-23T14:55:50.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f775be00640 1 -- 172.21.15.83:0/2477936232 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775401e070 con 0x55fede5cbb30 2024-04-23T14:55:50.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f775be00640 1 -- 172.21.15.83:0/2477936232 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7754019b10 con 0x55fede5cbb30 2024-04-23T14:55:50.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f775be00640 1 -- 172.21.15.83:0/2477936232 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775400eaf0 con 0x55fede5cbb30 2024-04-23T14:55:50.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.726+0000 7f776be00640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f77500a3da0 0x7f77500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.727+0000 7f7774ff5740 1 -- 172.21.15.83:0/2477936232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5cbb30 msgr2=0x55fede5cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.727+0000 7f776be00640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f77500a3da0 0x7f77500a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7764005fd0 tx=0x7f7764010040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.727+0000 7f7774ff5740 1 --2- 172.21.15.83:0/2477936232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5cbb30 0x55fede5cdf20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7754009c90 tx=0x7f7754011810 comp rx=0 tx=0).stop 2024-04-23T14:55:50.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.727+0000 7f7774ff5740 1 -- 172.21.15.83:0/2477936232 shutdown_connections 2024-04-23T14:55:50.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.727+0000 7f7774ff5740 1 --2- 172.21.15.83:0/2477936232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5cbb30 0x55fede5cdf20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.727+0000 7f7774ff5740 1 --2- 172.21.15.83:0/2477936232 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5c1460 0x55fede5cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.727+0000 7f7774ff5740 1 --2- 172.21.15.83:0/2477936232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5c0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.727+0000 7f7774ff5740 1 -- 172.21.15.83:0/2477936232 >> 172.21.15.83:0/2477936232 conn(0x55fede5bce60 msgr2=0x55fede5bd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.740 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.727+0000 7f7774ff5740 1 -- 172.21.15.83:0/2477936232 shutdown_connections 2024-04-23T14:55:50.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.727+0000 7f7774ff5740 1 -- 172.21.15.83:0/2477936232 wait complete. 2024-04-23T14:55:50.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.728+0000 7f7774ff5740 1 -- start start 2024-04-23T14:55:50.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.728+0000 7f7774ff5740 1 Processor -- start 2024-04-23T14:55:50.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.728+0000 7f7774ff5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c0b40 0x55fede5cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f7774ff5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c1460 0x55fede5c8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f7774ff5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.741 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f7774ff5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fede5ce9c0 con 0x55fede5c1460 2024-04-23T14:55:50.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f7774ff5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55fede4df810 con 0x55fede5cbb30 2024-04-23T14:55:50.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f7774ff5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55fede5ceb40 con 0x55fede5c0b40 2024-04-23T14:55:50.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f7774ff5740 10 client.0 _ll_register_callbacks cb 0x55fede4ad140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:50.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f7763e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c1460 0x55fede5c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f776be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f7763e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c1460 0x55fede5c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51096/0 (socket says 172.21.15.83:51096) 2024-04-23T14:55:50.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f7763e00640 1 -- 172.21.15.83:0/446372342 learned_addr learned my addr 172.21.15.83:0/446372342 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f776b400640 1 --2- 172.21.15.83:0/446372342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c0b40 0x55fede5cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f776be00640 1 -- 172.21.15.83:0/446372342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c0b40 msgr2=0x55fede5cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f776be00640 1 --2- 172.21.15.83:0/446372342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c0b40 0x55fede5cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f776be00640 1 -- 172.21.15.83:0/446372342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c1460 msgr2=0x55fede5c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f776be00640 1 --2- 172.21.15.83:0/446372342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c1460 0x55fede5c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.729+0000 7f776be00640 1 -- 172.21.15.83:0/446372342 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fede4f03f0 con 0x55fede5cbb30 2024-04-23T14:55:50.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.730+0000 7f776be00640 1 --2- 172.21.15.83:0/446372342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5c8cf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f776400d700 tx=0x7f7764004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8951a3b8789c868 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.730+0000 7f7758c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:55:50.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.730+0000 7f7758c00640 1 -- 172.21.15.83:0/446372342 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776402a060 con 0x55fede5cbb30 2024-04-23T14:55:50.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.730+0000 7f7774ff5740 1 -- 172.21.15.83:0/446372342 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7754019b10 con 0x55fede5cbb30 2024-04-23T14:55:50.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.730+0000 7f7758c00640 1 -- 172.21.15.83:0/446372342 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f7764004750 con 0x55fede5cbb30 2024-04-23T14:55:50.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.730+0000 7f7758c00640 1 -- 172.21.15.83:0/446372342 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f776401a450 con 0x55fede5cbb30 2024-04-23T14:55:50.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.730+0000 7f7774ff5740 10 client.26669 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:50.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.730+0000 7f7774ff5740 1 -- 172.21.15.83:0/446372342 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fede2383f0 con 0x55fede5cbb30 2024-04-23T14:55:50.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.731+0000 7f7758c00640 1 -- 172.21.15.83:0/446372342 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7754019b10 con 0x55fede5cbb30 2024-04-23T14:55:50.744 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.731+0000 7f7758c00640 1 -- 172.21.15.83:0/446372342 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f776400b040 con 0x55fede5cbb30 2024-04-23T14:55:50.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.731+0000 7f7774ff5740 10 client.26669 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:50.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.731+0000 7f7774ff5740 10 client.26669 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:55:50.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.731+0000 7f7774ff5740 20 client.26669 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:50.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.731+0000 7f7774ff5740 1 --2- 172.21.15.83:0/446372342 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fede5dbd50 0x55fede5de160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.731+0000 7f7774ff5740 4 client.26669 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:55:50.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.731+0000 7f7774ff5740 1 -- 172.21.15.83:0/446372342 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fede533bf0 con 0x55fede5dbd50 2024-04-23T14:55:50.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.731+0000 7f776b400640 1 --2- 172.21.15.83:0/446372342 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fede5dbd50 0x55fede5de160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.732+0000 7f776b400640 1 --2- 172.21.15.83:0/446372342 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fede5dbd50 0x55fede5de160 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9d9d36310257f08 server_cookie=49893122f5ce6f25 in_seq=0 out_seq=0 2024-04-23T14:55:50.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.732+0000 7f7758c00640 10 client.26669 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:55:50.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.732+0000 7f7758c00640 1 -- 172.21.15.83:0/446372342 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fede533bf0 con 0x55fede5dbd50 2024-04-23T14:55:50.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.732+0000 7f7758c00640 10 client.26669 handle_command_reply: tid=0 2024-04-23T14:55:50.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7774ff5740 1 client.26669 shutdown 2024-04-23T14:55:50.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7774ff5740 1 -- 172.21.15.83:0/446372342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 msgr2=0x55fede5c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7774ff5740 1 --2- 172.21.15.83:0/446372342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5c8cf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f776400d700 tx=0x7f7764004ab0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7774ff5740 1 -- 172.21.15.83:0/446372342 shutdown_connections 2024-04-23T14:55:50.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7774ff5740 1 --2- 172.21.15.83:0/446372342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5c8cf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7774ff5740 1 --2- 172.21.15.83:0/446372342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c1460 0x55fede5c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7774ff5740 1 --2- 172.21.15.83:0/446372342 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fede5dbd50 0x55fede5de160 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7774ff5740 1 --2- 172.21.15.83:0/446372342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c0b40 0x55fede5cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7758c00640 0 client.26669 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:55:50.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7774ff5740 1 -- 172.21.15.83:0/446372342 >> 172.21.15.83:0/446372342 conn(0x55fede5bce60 msgr2=0x55fede5c2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7774ff5740 1 -- 172.21.15.83:0/446372342 shutdown_connections 2024-04-23T14:55:50.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7774ff5740 1 -- 172.21.15.83:0/446372342 wait complete. 2024-04-23T14:55:50.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.733+0000 7f7774ff5740 20 client.26669 trim_cache size 0 max 16384 2024-04-23T14:55:50.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f7774ff5740 1 Processor -- start 2024-04-23T14:55:50.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f7774ff5740 1 -- start start 2024-04-23T14:55:50.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f7774ff5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5d7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f7774ff5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c1460 0x55fede5cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f7774ff5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f7774ff5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b69b0 con 0x55fede5c0b40 2024-04-23T14:55:50.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f7774ff5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b4280 con 0x55fede5cbb30 2024-04-23T14:55:50.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f7774ff5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b20f0 con 0x55fede5c1460 2024-04-23T14:55:50.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f776b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f776b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5d7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51106/0 (socket says 172.21.15.83:51106) 2024-04-23T14:55:50.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f776b400640 1 -- 172.21.15.83:0/3259525744 learned_addr learned my addr 172.21.15.83:0/3259525744 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f7763e00640 1 --2- 172.21.15.83:0/3259525744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c1460 0x55fede5cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.748+0000 7f776be00640 1 --2- 172.21.15.83:0/3259525744 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.749+0000 7f776b400640 1 -- 172.21.15.83:0/3259525744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c1460 msgr2=0x55fede5cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.749+0000 7f776b400640 1 --2- 172.21.15.83:0/3259525744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c1460 0x55fede5cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.749+0000 7f776b400640 1 -- 172.21.15.83:0/3259525744 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 msgr2=0x55fede5e3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.749+0000 7f776b400640 1 --2- 172.21.15.83:0/3259525744 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5e3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.749+0000 7f776b400640 1 -- 172.21.15.83:0/3259525744 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fede3027d0 con 0x55fede5c0b40 2024-04-23T14:55:50.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.749+0000 7f7763e00640 1 --2- 172.21.15.83:0/3259525744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c1460 0x55fede5cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:50.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.749+0000 7f776b400640 1 --2- 172.21.15.83:0/3259525744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5d7910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f775c0b9090 tx=0x7f775c0f2ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb90594a756eb2cd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.749+0000 7f775b400640 1 -- 172.21.15.83:0/3259525744 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c0bc020 con 0x55fede5c0b40 2024-04-23T14:55:50.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.749+0000 7f775b400640 1 -- 172.21.15.83:0/3259525744 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f775c0a70b0 con 0x55fede5c0b40 2024-04-23T14:55:50.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.749+0000 7f775b400640 1 -- 172.21.15.83:0/3259525744 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c0edfe0 con 0x55fede5c0b40 2024-04-23T14:55:50.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.750+0000 7f7774ff5740 1 -- 172.21.15.83:0/3259525744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 msgr2=0x55fede5d7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.750+0000 7f7774ff5740 1 --2- 172.21.15.83:0/3259525744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5d7910 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f775c0b9090 tx=0x7f775c0f2ad0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.750+0000 7f7774ff5740 1 -- 172.21.15.83:0/3259525744 shutdown_connections 2024-04-23T14:55:50.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.750+0000 7f7774ff5740 1 --2- 172.21.15.83:0/3259525744 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5e3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.750+0000 7f7774ff5740 1 --2- 172.21.15.83:0/3259525744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c1460 0x55fede5cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.750+0000 7f7774ff5740 1 --2- 172.21.15.83:0/3259525744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5d7910 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.750+0000 7f7774ff5740 1 -- 172.21.15.83:0/3259525744 >> 172.21.15.83:0/3259525744 conn(0x55fede5bce60 msgr2=0x55fede5c7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.750+0000 7f7774ff5740 1 -- 172.21.15.83:0/3259525744 shutdown_connections 2024-04-23T14:55:50.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.750+0000 7f7774ff5740 1 -- 172.21.15.83:0/3259525744 wait complete. 2024-04-23T14:55:50.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.750+0000 7f7774ff5740 1 -- start start 2024-04-23T14:55:50.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.751+0000 7f7774ff5740 1 Processor -- start 2024-04-23T14:55:50.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.751+0000 7f7774ff5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5d4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.751+0000 7f7774ff5740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c1460 0x55fede5cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.751+0000 7f7774ff5740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.751+0000 7f7774ff5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b2740 con 0x55fede5c0b40 2024-04-23T14:55:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.751+0000 7f7774ff5740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b3bb0 con 0x55fede5cbb30 2024-04-23T14:55:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.751+0000 7f7774ff5740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f776c1b2e10 con 0x55fede5c1460 2024-04-23T14:55:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.751+0000 7f7774ff5740 10 client.0 _ll_register_callbacks cb 0x7f776400e450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.751+0000 7f776be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.751+0000 7f776b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.751+0000 7f776be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:42794/0 (socket says 172.21.15.83:42794) 2024-04-23T14:55:50.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.751+0000 7f776be00640 1 -- 172.21.15.83:0/726719299 learned_addr learned my addr 172.21.15.83:0/726719299 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.752+0000 7f7763e00640 1 --2- 172.21.15.83:0/726719299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c1460 0x55fede5cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.752+0000 7f776be00640 1 -- 172.21.15.83:0/726719299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c1460 msgr2=0x55fede5cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.752+0000 7f776be00640 1 --2- 172.21.15.83:0/726719299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c1460 0x55fede5cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.752+0000 7f776be00640 1 -- 172.21.15.83:0/726719299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 msgr2=0x55fede5d4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.752+0000 7f776be00640 1 --2- 172.21.15.83:0/726719299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5d4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.752+0000 7f776be00640 1 -- 172.21.15.83:0/726719299 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fede5cff40 con 0x55fede5cbb30 2024-04-23T14:55:50.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.752+0000 7f776be00640 1 --2- 172.21.15.83:0/726719299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5cf900 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7764025980 tx=0x7f7764025e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe651840c4de044 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.752+0000 7f7758c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:55:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.752+0000 7f7758c00640 1 -- 172.21.15.83:0/726719299 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776403b020 con 0x55fede5cbb30 2024-04-23T14:55:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7774ff5740 1 -- 172.21.15.83:0/726719299 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f775c0a70b0 con 0x55fede5cbb30 2024-04-23T14:55:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7758c00640 1 -- 172.21.15.83:0/726719299 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f7764025640 con 0x55fede5cbb30 2024-04-23T14:55:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7758c00640 1 -- 172.21.15.83:0/726719299 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7764046050 con 0x55fede5cbb30 2024-04-23T14:55:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7774ff5740 10 client.26675 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7774ff5740 1 -- 172.21.15.83:0/726719299 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fede6420f0 con 0x55fede5cbb30 2024-04-23T14:55:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7758c00640 1 -- 172.21.15.83:0/726719299 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f775c0a70b0 con 0x55fede5cbb30 2024-04-23T14:55:50.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7758c00640 1 -- 172.21.15.83:0/726719299 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f776400b070 con 0x55fede5cbb30 2024-04-23T14:55:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7774ff5740 10 client.26675 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7774ff5740 10 client.26675 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:55:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7774ff5740 20 client.26675 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7774ff5740 1 --2- 172.21.15.83:0/726719299 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fede6427d0 0x55fede644bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7774ff5740 4 client.26675 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:55:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.753+0000 7f7774ff5740 1 -- 172.21.15.83:0/726719299 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fede52da60 con 0x55fede6427d0 2024-04-23T14:55:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.754+0000 7f776b400640 1 --2- 172.21.15.83:0/726719299 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fede6427d0 0x55fede644bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.754+0000 7f776b400640 1 --2- 172.21.15.83:0/726719299 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fede6427d0 0x55fede644bf0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df3e28efd6ee1adb server_cookie=d0ed9d6dfc44c32 in_seq=0 out_seq=0 2024-04-23T14:55:50.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.754+0000 7f7758c00640 10 client.26675 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:55:50.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.754+0000 7f7758c00640 1 -- 172.21.15.83:0/726719299 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fede52da60 con 0x55fede6427d0 2024-04-23T14:55:50.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.754+0000 7f7758c00640 10 client.26675 handle_command_reply: tid=0 2024-04-23T14:55:50.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.754+0000 7f7774ff5740 1 client.26675 shutdown 2024-04-23T14:55:50.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.755+0000 7f7774ff5740 1 -- 172.21.15.83:0/726719299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 msgr2=0x55fede5cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.755+0000 7f7774ff5740 1 --2- 172.21.15.83:0/726719299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5cf900 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7764025980 tx=0x7f7764025e20 comp rx=0 tx=0).stop 2024-04-23T14:55:50.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.755+0000 7f7774ff5740 1 -- 172.21.15.83:0/726719299 shutdown_connections 2024-04-23T14:55:50.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.755+0000 7f7774ff5740 1 --2- 172.21.15.83:0/726719299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fede5cbb30 0x55fede5cf900 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.755+0000 7f7774ff5740 1 --2- 172.21.15.83:0/726719299 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55fede6427d0 0x55fede644bf0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.755+0000 7f7774ff5740 1 --2- 172.21.15.83:0/726719299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fede5c1460 0x55fede5cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.755+0000 7f7774ff5740 1 --2- 172.21.15.83:0/726719299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fede5c0b40 0x55fede5d4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.755+0000 7f7758c00640 0 client.26675 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:55:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.755+0000 7f7774ff5740 1 -- 172.21.15.83:0/726719299 >> 172.21.15.83:0/726719299 conn(0x55fede5bce60 msgr2=0x55fede5bb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.755+0000 7f7774ff5740 1 -- 172.21.15.83:0/726719299 shutdown_connections 2024-04-23T14:55:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.755+0000 7f7774ff5740 1 -- 172.21.15.83:0/726719299 wait complete. 2024-04-23T14:55:50.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.755+0000 7f7774ff5740 20 client.26675 trim_cache size 0 max 16384 2024-04-23T14:55:50.770 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:55:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.756+0000 7f7770c00640 1 -- 172.21.15.83:0/621602308 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f77500a3da0 msgr2=0x7f77500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.756+0000 7f7770c00640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f77500a3da0 0x7f77500a6260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7764005fd0 tx=0x7f7764010040 comp rx=0 tx=0).stop 2024-04-23T14:55:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.756+0000 7f7770c00640 1 -- 172.21.15.83:0/621602308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c108580 msgr2=0x7f776c19ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.756+0000 7f7770c00640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c108580 0x7f776c19ed30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f775c009780 tx=0x7f775c033ae0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.756+0000 7f7770c00640 1 -- 172.21.15.83:0/621602308 shutdown_connections 2024-04-23T14:55:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.756+0000 7f7770c00640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f776c10d500 0x7f776c1a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.756+0000 7f7770c00640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f776c108f40 0x7f776c19f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.756+0000 7f7770c00640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f77500a3da0 0x7f77500a6260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.756+0000 7f7770c00640 1 --2- 172.21.15.83:0/621602308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c108580 0x7f776c19ed30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.756+0000 7f7770c00640 1 -- 172.21.15.83:0/621602308 >> 172.21.15.83:0/621602308 conn(0x7f776c077e20 msgr2=0x7f776c078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.756+0000 7f7770c00640 1 -- 172.21.15.83:0/621602308 shutdown_connections 2024-04-23T14:55:50.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.756+0000 7f7770c00640 1 -- 172.21.15.83:0/621602308 wait complete. 2024-04-23T14:55:50.773 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:55:50.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:50 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3659266419' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:55:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.974+0000 7fa149600640 1 -- 172.21.15.83:0/3690353291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa144104450 msgr2=0x7fa144104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.974+0000 7fa149600640 1 --2- 172.21.15.83:0/3690353291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa144104450 0x7fa144104830 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa134009900 tx=0x7fa13402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.974+0000 7fa149600640 1 -- 172.21.15.83:0/3690353291 shutdown_connections 2024-04-23T14:55:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.974+0000 7fa149600640 1 --2- 172.21.15.83:0/3690353291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa144109520 0x7fa1441110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.974+0000 7fa149600640 1 --2- 172.21.15.83:0/3690353291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144104e00 0x7fa144108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.974+0000 7fa149600640 1 --2- 172.21.15.83:0/3690353291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa144104450 0x7fa144104830 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.974+0000 7fa149600640 1 -- 172.21.15.83:0/3690353291 >> 172.21.15.83:0/3690353291 conn(0x7fa1440ffc70 msgr2=0x7fa144102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.974+0000 7fa149600640 1 -- 172.21.15.83:0/3690353291 shutdown_connections 2024-04-23T14:55:50.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.974+0000 7fa149600640 1 -- 172.21.15.83:0/3690353291 wait complete. 2024-04-23T14:55:50.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.975+0000 7fa149600640 1 Processor -- start 2024-04-23T14:55:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa149600640 1 -- start start 2024-04-23T14:55:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa149600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144104450 0x7fa1441a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa149600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa144109520 0x7fa1441a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa149600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa14419a920 0x7fa14419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa149600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa144114620 con 0x7fa144109520 2024-04-23T14:55:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa149600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1441144a0 con 0x7fa144104450 2024-04-23T14:55:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa149600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1441147a0 con 0x7fa14419a920 2024-04-23T14:55:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa142a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144104450 0x7fa1441a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa143400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa14419a920 0x7fa14419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa142a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144104450 0x7fa1441a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:42818/0 (socket says 172.21.15.83:42818) 2024-04-23T14:55:50.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa142a00640 1 -- 172.21.15.83:0/3038074402 learned_addr learned my addr 172.21.15.83:0/3038074402 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa143400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa14419a920 0x7fa14419add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42490/0 (socket says 172.21.15.83:42490) 2024-04-23T14:55:50.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.976+0000 7fa13be00640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa144109520 0x7fa1441a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.977+0000 7fa143400640 1 -- 172.21.15.83:0/3038074402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144104450 msgr2=0x7fa1441a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.977+0000 7fa143400640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144104450 0x7fa1441a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.977+0000 7fa143400640 1 -- 172.21.15.83:0/3038074402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa144109520 msgr2=0x7fa1441a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.977+0000 7fa143400640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa144109520 0x7fa1441a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.977+0000 7fa143400640 1 -- 172.21.15.83:0/3038074402 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14419b440 con 0x7fa14419a920 2024-04-23T14:55:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.977+0000 7fa143400640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa14419a920 0x7fa14419add0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009fd0 tx=0x7fa13c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.977+0000 7fa13b400640 1 -- 172.21.15.83:0/3038074402 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c00ce20 con 0x7fa14419a920 2024-04-23T14:55:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.977+0000 7fa13b400640 1 -- 172.21.15.83:0/3038074402 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa13c005020 con 0x7fa14419a920 2024-04-23T14:55:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.977+0000 7fa13b400640 1 -- 172.21.15.83:0/3038074402 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c010720 con 0x7fa14419a920 2024-04-23T14:55:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.977+0000 7fa149600640 1 -- 172.21.15.83:0/3038074402 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14419b730 con 0x7fa14419a920 2024-04-23T14:55:50.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.978+0000 7fa149600640 1 -- 172.21.15.83:0/3038074402 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1441a7930 con 0x7fa14419a920 2024-04-23T14:55:50.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.981+0000 7fa14c5eb740 1 Processor -- start 2024-04-23T14:55:50.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.981+0000 7fa13b400640 1 -- 172.21.15.83:0/3038074402 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa13c005540 con 0x7fa14419a920 2024-04-23T14:55:50.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.981+0000 7fa14c5eb740 1 -- start start 2024-04-23T14:55:50.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa14c5eb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c23d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa14c5eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c24270 0x55f502c2e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa14c5eb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c2e940 0x55f502c30d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa14c5eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f502b891d0 con 0x55f502c24270 2024-04-23T14:55:50.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa14c5eb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f502b2f450 con 0x55f502c23950 2024-04-23T14:55:50.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa14c5eb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f502b85190 con 0x55f502c2e940 2024-04-23T14:55:50.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa13be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c23d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa13be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c23d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:42826/0 (socket says 172.21.15.83:42826) 2024-04-23T14:55:50.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa13be00640 1 -- 172.21.15.83:0/1125407787 learned_addr learned my addr 172.21.15.83:0/1125407787 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa13be00640 1 --2- 172.21.15.83:0/1125407787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c2e940 0x55f502c30d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa142a00640 1 --2- 172.21.15.83:0/1125407787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c24270 0x55f502c2e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa13be00640 1 -- 172.21.15.83:0/1125407787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c2e940 msgr2=0x55f502c30d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa13be00640 1 --2- 172.21.15.83:0/1125407787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c2e940 0x55f502c30d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa13be00640 1 -- 172.21.15.83:0/1125407787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c24270 msgr2=0x55f502c2e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa13be00640 1 --2- 172.21.15.83:0/1125407787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c24270 0x55f502c2e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.982+0000 7fa13be00640 1 -- 172.21.15.83:0/1125407787 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f502bec310 con 0x55f502c23950 2024-04-23T14:55:50.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.983+0000 7fa13be00640 1 --2- 172.21.15.83:0/1125407787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c23d30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0058c0 tx=0x7fa12c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46c492040b8b9058 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.983+0000 7fa13b400640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa1280a3da0 0x7fa1280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.983+0000 7fa133400640 1 -- 172.21.15.83:0/1125407787 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c005020 con 0x55f502c23950 2024-04-23T14:55:50.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.983+0000 7fa133400640 1 -- 172.21.15.83:0/1125407787 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa12c0047a0 con 0x55f502c23950 2024-04-23T14:55:50.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.983+0000 7fa133400640 1 -- 172.21.15.83:0/1125407787 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c00fb20 con 0x55f502c23950 2024-04-23T14:55:50.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.983+0000 7fa13b400640 1 -- 172.21.15.83:0/3038074402 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fa13c0d0e40 con 0x7fa14419a920 2024-04-23T14:55:50.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.983+0000 7fa142a00640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa1280a3da0 0x7fa1280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.983+0000 7fa14c5eb740 1 -- 172.21.15.83:0/1125407787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 msgr2=0x55f502c23d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.983+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/1125407787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c23d30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa12c0058c0 tx=0x7fa12c005850 comp rx=0 tx=0).stop 2024-04-23T14:55:50.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.984+0000 7fa14c5eb740 1 -- 172.21.15.83:0/1125407787 shutdown_connections 2024-04-23T14:55:50.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.984+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/1125407787 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c2e940 0x55f502c30d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.984+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/1125407787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c24270 0x55f502c2e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.984+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/1125407787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c23d30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.984+0000 7fa14c5eb740 1 -- 172.21.15.83:0/1125407787 >> 172.21.15.83:0/1125407787 conn(0x55f502c1fc70 msgr2=0x55f502c20050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.984+0000 7fa14c5eb740 1 -- 172.21.15.83:0/1125407787 shutdown_connections 2024-04-23T14:55:50.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.984+0000 7fa142a00640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa1280a3da0 0x7fa1280a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55f502c24f10 tx=0x7fa1340046c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.984+0000 7fa14c5eb740 1 -- 172.21.15.83:0/1125407787 wait complete. 2024-04-23T14:55:50.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.984+0000 7fa14c5eb740 1 -- start start 2024-04-23T14:55:50.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa14c5eb740 1 Processor -- start 2024-04-23T14:55:50.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa14c5eb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c2e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa14c5eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c24270 0x55f502c2b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa14c5eb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c2e940 0x55f502c2bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa14c5eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f502c317d0 con 0x55f502c24270 2024-04-23T14:55:50.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa14c5eb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f502b42810 con 0x55f502c23950 2024-04-23T14:55:50.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa14c5eb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f502c31950 con 0x55f502c2e940 2024-04-23T14:55:50.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa14c5eb740 10 client.0 _ll_register_callbacks cb 0x55f502b10140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:50.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa143400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c2e940 0x55f502c2bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa142a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c2e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa13be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c24270 0x55f502c2b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa143400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c2e940 0x55f502c2bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42510/0 (socket says 172.21.15.83:42510) 2024-04-23T14:55:50.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.985+0000 7fa143400640 1 -- 172.21.15.83:0/4119861093 learned_addr learned my addr 172.21.15.83:0/4119861093 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:50.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa143400640 1 -- 172.21.15.83:0/4119861093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 msgr2=0x55f502c2e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa143400640 1 --2- 172.21.15.83:0/4119861093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c2e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa143400640 1 -- 172.21.15.83:0/4119861093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c24270 msgr2=0x55f502c2b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:50.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa143400640 1 --2- 172.21.15.83:0/4119861093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c24270 0x55f502c2b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa143400640 1 -- 172.21.15.83:0/4119861093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f502b533f0 con 0x55f502c2e940 2024-04-23T14:55:50.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa143400640 1 --2- 172.21.15.83:0/4119861093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c2e940 0x55f502c2bde0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa13c005fd0 tx=0x7fa13c09cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f397cda618212ffc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:50.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa127e00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:55:50.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa127e00640 1 -- 172.21.15.83:0/4119861093 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0b6260 con 0x55f502c2e940 2024-04-23T14:55:50.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa127e00640 1 -- 172.21.15.83:0/4119861093 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fa13c0993e0 con 0x55f502c2e940 2024-04-23T14:55:50.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa14c5eb740 1 -- 172.21.15.83:0/4119861093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa12c0047a0 con 0x55f502c2e940 2024-04-23T14:55:50.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa127e00640 1 -- 172.21.15.83:0/4119861093 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa13c09cfd0 con 0x55f502c2e940 2024-04-23T14:55:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa14c5eb740 10 client.26668 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.986+0000 7fa14c5eb740 1 -- 172.21.15.83:0/4119861093 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f50289b3f0 con 0x55f502c2e940 2024-04-23T14:55:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.987+0000 7fa127e00640 1 -- 172.21.15.83:0/4119861093 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa12c0047a0 con 0x55f502c2e940 2024-04-23T14:55:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.987+0000 7fa127e00640 1 -- 172.21.15.83:0/4119861093 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fa13c07bb90 con 0x55f502c2e940 2024-04-23T14:55:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.987+0000 7fa14c5eb740 10 client.26668 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.987+0000 7fa14c5eb740 10 client.26668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.987+0000 7fa14c5eb740 20 client.26668 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:50.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.987+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/4119861093 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f502c3ebe0 0x55f502c40ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:50.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.987+0000 7fa14c5eb740 4 client.26668 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:55:50.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.987+0000 7fa14c5eb740 1 -- 172.21.15.83:0/4119861093 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f502b96bf0 con 0x55f502c3ebe0 2024-04-23T14:55:50.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.987+0000 7fa142a00640 1 --2- 172.21.15.83:0/4119861093 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f502c3ebe0 0x55f502c40ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:50.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.988+0000 7fa142a00640 1 --2- 172.21.15.83:0/4119861093 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f502c3ebe0 0x55f502c40ff0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6d675afb83a2fad server_cookie=bc8290314d89d15d in_seq=0 out_seq=0 2024-04-23T14:55:50.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.988+0000 7fa127e00640 10 client.26668 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:50.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa127e00640 1 -- 172.21.15.83:0/4119861093 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f502b96bf0 con 0x55f502c3ebe0 2024-04-23T14:55:50.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa127e00640 10 client.26668 handle_command_reply: tid=0 2024-04-23T14:55:50.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa14c5eb740 1 client.26668 shutdown 2024-04-23T14:55:50.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa14c5eb740 1 -- 172.21.15.83:0/4119861093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c2e940 msgr2=0x55f502c2bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:50.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/4119861093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c2e940 0x55f502c2bde0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa13c005fd0 tx=0x7fa13c09cd70 comp rx=0 tx=0).stop 2024-04-23T14:55:50.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa14c5eb740 1 -- 172.21.15.83:0/4119861093 shutdown_connections 2024-04-23T14:55:50.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/4119861093 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f502c3ebe0 0x55f502c40ff0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/4119861093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c2e940 0x55f502c2bde0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/4119861093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c24270 0x55f502c2b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/4119861093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c2e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:50.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa127e00640 0 client.26668 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:50.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa14c5eb740 1 -- 172.21.15.83:0/4119861093 >> 172.21.15.83:0/4119861093 conn(0x55f502c1fc70 msgr2=0x55f502c25970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:50.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa14c5eb740 1 -- 172.21.15.83:0/4119861093 shutdown_connections 2024-04-23T14:55:50.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa14c5eb740 1 -- 172.21.15.83:0/4119861093 wait complete. 2024-04-23T14:55:50.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:50.989+0000 7fa14c5eb740 20 client.26668 trim_cache size 0 max 16384 2024-04-23T14:55:51.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.004+0000 7fa14c5eb740 1 Processor -- start 2024-04-23T14:55:51.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.004+0000 7fa14c5eb740 1 -- start start 2024-04-23T14:55:51.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.004+0000 7fa14c5eb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c3a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:51.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.004+0000 7fa14c5eb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c24270 0x55f502c26040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:51.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa14c5eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c2e940 0x55f502c2dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:51.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa14c5eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1441b2830 con 0x55f502c2e940 2024-04-23T14:55:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa14c5eb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1441b0100 con 0x55f502c23950 2024-04-23T14:55:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa14c5eb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1441adf70 con 0x55f502c24270 2024-04-23T14:55:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa142a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c3a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa143400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c2e940 0x55f502c2dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa142a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c3a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:42850/0 (socket says 172.21.15.83:42850) 2024-04-23T14:55:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa142a00640 1 -- 172.21.15.83:0/2713105528 learned_addr learned my addr 172.21.15.83:0/2713105528 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa13be00640 1 --2- 172.21.15.83:0/2713105528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c24270 0x55f502c26040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa142a00640 1 -- 172.21.15.83:0/2713105528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c24270 msgr2=0x55f502c26040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:51.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa142a00640 1 --2- 172.21.15.83:0/2713105528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c24270 0x55f502c26040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa142a00640 1 -- 172.21.15.83:0/2713105528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c2e940 msgr2=0x55f502c2dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:51.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa142a00640 1 --2- 172.21.15.83:0/2713105528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c2e940 0x55f502c2dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.005+0000 7fa142a00640 1 -- 172.21.15.83:0/2713105528 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f502c240a0 con 0x55f502c23950 2024-04-23T14:55:51.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.006+0000 7fa142a00640 1 --2- 172.21.15.83:0/2713105528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c3a6c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa134004770 tx=0x7fa13403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38c39a6080c5787 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:51.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.006+0000 7fa132a00640 1 -- 172.21.15.83:0/2713105528 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134035040 con 0x55f502c23950 2024-04-23T14:55:51.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.006+0000 7fa132a00640 1 -- 172.21.15.83:0/2713105528 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa134059070 con 0x55f502c23950 2024-04-23T14:55:51.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.006+0000 7fa132a00640 1 -- 172.21.15.83:0/2713105528 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134003e20 con 0x55f502c23950 2024-04-23T14:55:51.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.006+0000 7fa14c5eb740 1 -- 172.21.15.83:0/2713105528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 msgr2=0x55f502c3a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:51.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.006+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/2713105528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c3a6c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa134004770 tx=0x7fa13403f000 comp rx=0 tx=0).stop 2024-04-23T14:55:51.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.007+0000 7fa14c5eb740 1 -- 172.21.15.83:0/2713105528 shutdown_connections 2024-04-23T14:55:51.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.007+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/2713105528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c2e940 0x55f502c2dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.007+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/2713105528 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c24270 0x55f502c26040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.007+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/2713105528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c23950 0x55f502c3a6c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.007+0000 7fa14c5eb740 1 -- 172.21.15.83:0/2713105528 >> 172.21.15.83:0/2713105528 conn(0x55f502c1fc70 msgr2=0x55f502c25930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:51.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.007+0000 7fa14c5eb740 1 -- 172.21.15.83:0/2713105528 shutdown_connections 2024-04-23T14:55:51.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.007+0000 7fa14c5eb740 1 -- 172.21.15.83:0/2713105528 wait complete. 2024-04-23T14:55:51.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.007+0000 7fa14c5eb740 1 -- start start 2024-04-23T14:55:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.007+0000 7fa14c5eb740 1 Processor -- start 2024-04-23T14:55:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa14c5eb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c23950 0x55f502c36bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa14c5eb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c24270 0x55f502c37110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa14c5eb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c2e940 0x55f502c2fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa14c5eb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1441ae5c0 con 0x55f502c23950 2024-04-23T14:55:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa14c5eb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fa1441afa30 con 0x55f502c2e940 2024-04-23T14:55:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa14c5eb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fa1441aec90 con 0x55f502c24270 2024-04-23T14:55:51.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa14c5eb740 10 client.0 _ll_register_callbacks cb 0x7fa13c005830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa142a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c23950 0x55f502c36bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa143400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c2e940 0x55f502c2fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa143400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c2e940 0x55f502c2fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:42854/0 (socket says 172.21.15.83:42854) 2024-04-23T14:55:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa142a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c23950 0x55f502c36bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51166/0 (socket says 172.21.15.83:51166) 2024-04-23T14:55:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa143400640 1 -- 172.21.15.83:0/643141101 learned_addr learned my addr 172.21.15.83:0/643141101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa13be00640 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c24270 0x55f502c37110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa143400640 1 -- 172.21.15.83:0/643141101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c24270 msgr2=0x55f502c37110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa143400640 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c24270 0x55f502c37110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa143400640 1 -- 172.21.15.83:0/643141101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c23950 msgr2=0x55f502c36bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa143400640 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c23950 0x55f502c36bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.008+0000 7fa143400640 1 -- 172.21.15.83:0/643141101 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5029657d0 con 0x55f502c2e940 2024-04-23T14:55:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.009+0000 7fa142a00640 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c23950 0x55f502c36bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.009+0000 7fa143400640 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c2e940 0x55f502c2fb60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0991d0 tx=0x7fa13c09cec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21e80f0c0cc8238c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.009+0000 7fa127e00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:55:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.009+0000 7fa127e00640 1 -- 172.21.15.83:0/643141101 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c09cb40 con 0x55f502c2e940 2024-04-23T14:55:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.009+0000 7fa127e00640 1 -- 172.21.15.83:0/643141101 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7fa13c0b6060 con 0x55f502c2e940 2024-04-23T14:55:51.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.009+0000 7fa14c5eb740 1 -- 172.21.15.83:0/643141101 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa134059070 con 0x55f502c2e940 2024-04-23T14:55:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.009+0000 7fa127e00640 1 -- 172.21.15.83:0/643141101 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa13c07bd90 con 0x55f502c2e940 2024-04-23T14:55:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.009+0000 7fa14c5eb740 10 client.26693 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.009+0000 7fa14c5eb740 1 -- 172.21.15.83:0/643141101 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f502ca5700 con 0x55f502c2e940 2024-04-23T14:55:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.009+0000 7fa127e00640 1 -- 172.21.15.83:0/643141101 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa134059070 con 0x55f502c2e940 2024-04-23T14:55:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.009+0000 7fa127e00640 1 -- 172.21.15.83:0/643141101 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fa13c09c210 con 0x55f502c2e940 2024-04-23T14:55:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.010+0000 7fa14c5eb740 10 client.26693 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.010+0000 7fa14c5eb740 10 client.26693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.010+0000 7fa14c5eb740 20 client.26693 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:51.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.010+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f502ca5e50 0x55f502ca8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.010+0000 7fa14c5eb740 4 client.26693 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:55:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.010+0000 7fa14c5eb740 1 -- 172.21.15.83:0/643141101 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f502b90a60 con 0x55f502ca5e50 2024-04-23T14:55:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.010+0000 7fa142a00640 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f502ca5e50 0x55f502ca8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.010+0000 7fa142a00640 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f502ca5e50 0x55f502ca8270 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc80f5157ab3a743 server_cookie=383f1cba8df27a67 in_seq=0 out_seq=0 2024-04-23T14:55:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.010+0000 7fa127e00640 10 client.26693 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa127e00640 1 -- 172.21.15.83:0/643141101 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f502b90a60 con 0x55f502ca5e50 2024-04-23T14:55:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa127e00640 10 client.26693 handle_command_reply: tid=0 2024-04-23T14:55:51.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa14c5eb740 1 client.26693 shutdown 2024-04-23T14:55:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa14c5eb740 1 -- 172.21.15.83:0/643141101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c2e940 msgr2=0x55f502c2fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c2e940 0x55f502c2fb60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0991d0 tx=0x7fa13c09cec0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa14c5eb740 1 -- 172.21.15.83:0/643141101 shutdown_connections 2024-04-23T14:55:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f502ca5e50 0x55f502ca8270 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f502c2e940 0x55f502c2fb60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f502c24270 0x55f502c37110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa14c5eb740 1 --2- 172.21.15.83:0/643141101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f502c23950 0x55f502c36bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa127e00640 0 client.26693 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:51.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa14c5eb740 1 -- 172.21.15.83:0/643141101 >> 172.21.15.83:0/643141101 conn(0x55f502c1fc70 msgr2=0x55f502c396b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:51.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa14c5eb740 1 -- 172.21.15.83:0/643141101 shutdown_connections 2024-04-23T14:55:51.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa14c5eb740 1 -- 172.21.15.83:0/643141101 wait complete. 2024-04-23T14:55:51.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.011+0000 7fa14c5eb740 20 client.26693 trim_cache size 0 max 16384 2024-04-23T14:55:51.020 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:55:51.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.012+0000 7fa149600640 1 -- 172.21.15.83:0/3038074402 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa1280a3da0 msgr2=0x7fa1280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:51.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.012+0000 7fa149600640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa1280a3da0 0x7fa1280a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55f502c24f10 tx=0x7fa1340046c0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.012+0000 7fa149600640 1 -- 172.21.15.83:0/3038074402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa14419a920 msgr2=0x7fa14419add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:51.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.012+0000 7fa149600640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa14419a920 0x7fa14419add0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009fd0 tx=0x7fa13c00ef10 comp rx=0 tx=0).stop 2024-04-23T14:55:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.012+0000 7fa149600640 1 -- 172.21.15.83:0/3038074402 shutdown_connections 2024-04-23T14:55:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.012+0000 7fa149600640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa14419a920 0x7fa14419add0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.012+0000 7fa149600640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fa1280a3da0 0x7fa1280a6260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.012+0000 7fa149600640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa144109520 0x7fa1441a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.012+0000 7fa149600640 1 --2- 172.21.15.83:0/3038074402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa144104450 0x7fa1441a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.012+0000 7fa149600640 1 -- 172.21.15.83:0/3038074402 >> 172.21.15.83:0/3038074402 conn(0x7fa1440ffc70 msgr2=0x7fa144101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.012+0000 7fa149600640 1 -- 172.21.15.83:0/3038074402 shutdown_connections 2024-04-23T14:55:51.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:51.012+0000 7fa149600640 1 -- 172.21.15.83:0/3038074402 wait complete. 2024-04-23T14:55:51.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:51 smithi203 ceph-mon[32800]: pgmap v227: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:55:51.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:51 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:55:51.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:51 smithi173 ceph-mon[32443]: pgmap v227: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:55:51.727 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:51 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:55:51.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:51 smithi083 ceph-mon[27867]: pgmap v227: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:55:51.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:51 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:55:52.031 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:55:52.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:55:52.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:55:52.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.246+0000 7f73a2200640 1 -- 172.21.15.83:0/1666578529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c108f40 msgr2=0x7f739c1153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.246+0000 7f73a2200640 1 --2- 172.21.15.83:0/1666578529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c108f40 0x7f739c1153b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7394009cd0 tx=0x7f739402a8e0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.246+0000 7f73a2200640 1 -- 172.21.15.83:0/1666578529 shutdown_connections 2024-04-23T14:55:52.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.246+0000 7f73a2200640 1 --2- 172.21.15.83:0/1666578529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c108f40 0x7f739c1153b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.246+0000 7f73a2200640 1 --2- 172.21.15.83:0/1666578529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f739c108580 0x7f739c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.246+0000 7f73a2200640 1 --2- 172.21.15.83:0/1666578529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f739c10d500 0x7f739c10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.246+0000 7f73a2200640 1 -- 172.21.15.83:0/1666578529 >> 172.21.15.83:0/1666578529 conn(0x7f739c077e20 msgr2=0x7f739c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:52.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.246+0000 7f73a2200640 1 -- 172.21.15.83:0/1666578529 shutdown_connections 2024-04-23T14:55:52.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.246+0000 7f73a2200640 1 -- 172.21.15.83:0/1666578529 wait complete. 2024-04-23T14:55:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.247+0000 7f73a2200640 1 Processor -- start 2024-04-23T14:55:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.247+0000 7f73a2200640 1 -- start start 2024-04-23T14:55:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.247+0000 7f73a2200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f739c108580 0x7f739c1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.247+0000 7f73a2200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f739c108f40 0x7f739c1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.247+0000 7f73a2200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c10d500 0x7f739c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.247+0000 7f73a2200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f739c118920 con 0x7f739c10d500 2024-04-23T14:55:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.247+0000 7f73a2200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1187a0 con 0x7f739c108580 2024-04-23T14:55:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.247+0000 7f73a2200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f739c118aa0 con 0x7f739c108f40 2024-04-23T14:55:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.248+0000 7f73a1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c10d500 0x7f739c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.248+0000 7f73a0e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f739c108580 0x7f739c1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.248+0000 7f73a1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c10d500 0x7f739c19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51184/0 (socket says 172.21.15.83:51184) 2024-04-23T14:55:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.248+0000 7f73a1800640 1 -- 172.21.15.83:0/3940068981 learned_addr learned my addr 172.21.15.83:0/3940068981 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.248+0000 7f739be00640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f739c108f40 0x7f739c1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.248+0000 7f73a0e00640 1 -- 172.21.15.83:0/3940068981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f739c108f40 msgr2=0x7f739c1a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.248+0000 7f73a0e00640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f739c108f40 0x7f739c1a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.248+0000 7f73a0e00640 1 -- 172.21.15.83:0/3940068981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c10d500 msgr2=0x7f739c19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.248+0000 7f73a0e00640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c10d500 0x7f739c19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.248+0000 7f73a0e00640 1 -- 172.21.15.83:0/3940068981 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739c19f2c0 con 0x7f739c108580 2024-04-23T14:55:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.248+0000 7f73a1800640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c10d500 0x7f739c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.248+0000 7f73a0e00640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f739c108580 0x7f739c1a4940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f739000e9c0 tx=0x7f739000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.249+0000 7f7399600640 1 -- 172.21.15.83:0/3940068981 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739000ccc0 con 0x7f739c108580 2024-04-23T14:55:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.249+0000 7f7399600640 1 -- 172.21.15.83:0/3940068981 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73900035f0 con 0x7f739c108580 2024-04-23T14:55:52.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.249+0000 7f7399600640 1 -- 172.21.15.83:0/3940068981 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739000ce60 con 0x7f739c108580 2024-04-23T14:55:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.249+0000 7f73a2200640 1 -- 172.21.15.83:0/3940068981 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739c19f5b0 con 0x7f739c108580 2024-04-23T14:55:52.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.249+0000 7f73a2200640 1 -- 172.21.15.83:0/3940068981 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f739c1ab800 con 0x7f739c108580 2024-04-23T14:55:52.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.252+0000 7f73a6775740 1 Processor -- start 2024-04-23T14:55:52.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.252+0000 7f73a6775740 1 -- start start 2024-04-23T14:55:52.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.252+0000 7f7399600640 1 -- 172.21.15.83:0/3940068981 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f73900037a0 con 0x7f739c108580 2024-04-23T14:55:52.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.252+0000 7f73a6775740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd77650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.252+0000 7f73a6775740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 0x561bedd81d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.252+0000 7f73a6775740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd82260 0x561bedd84650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.252+0000 7f73a6775740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561bedcdce70 con 0x561bedd77b90 2024-04-23T14:55:52.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.252+0000 7f73a6775740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561bedc83380 con 0x561bedd77270 2024-04-23T14:55:52.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.252+0000 7f73a6775740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561bedcd8e30 con 0x561bedd82260 2024-04-23T14:55:52.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.253+0000 7f73a0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 0x561bedd81d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.253+0000 7f739be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd82260 0x561bedd84650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.253+0000 7f73a1800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd77650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.253+0000 7f739be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd82260 0x561bedd84650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42548/0 (socket says 172.21.15.83:42548) 2024-04-23T14:55:52.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.253+0000 7f73a0e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 0x561bedd81d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51198/0 (socket says 172.21.15.83:51198) 2024-04-23T14:55:52.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.253+0000 7f739be00640 1 -- 172.21.15.83:0/555684893 learned_addr learned my addr 172.21.15.83:0/555684893 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:52.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.253+0000 7f739be00640 1 -- 172.21.15.83:0/555684893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 msgr2=0x561bedd77650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.253+0000 7f739be00640 1 --2- 172.21.15.83:0/555684893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd77650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.253+0000 7f739be00640 1 -- 172.21.15.83:0/555684893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 msgr2=0x561bedd81d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.253+0000 7f739be00640 1 --2- 172.21.15.83:0/555684893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 0x561bedd81d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.253+0000 7f739be00640 1 -- 172.21.15.83:0/555684893 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bedab77e0 con 0x561bedd82260 2024-04-23T14:55:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.253+0000 7f73a1800640 1 --2- 172.21.15.83:0/555684893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd77650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.254+0000 7f739be00640 1 --2- 172.21.15.83:0/555684893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd82260 0x561bedd84650 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7388009c60 tx=0x7f7388011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=854a434f07eb361a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.254+0000 7f7399600640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f73840a3d20 0x7f73840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.254+0000 7f738e000640 1 -- 172.21.15.83:0/555684893 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388004230 con 0x561bedd82260 2024-04-23T14:55:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.254+0000 7f738e000640 1 -- 172.21.15.83:0/555684893 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73880043d0 con 0x561bedd82260 2024-04-23T14:55:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.254+0000 7f738e000640 1 -- 172.21.15.83:0/555684893 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388004e00 con 0x561bedd82260 2024-04-23T14:55:52.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.254+0000 7f7399600640 1 -- 172.21.15.83:0/3940068981 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f73900d0d20 con 0x7f739c108580 2024-04-23T14:55:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.254+0000 7f73a1800640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f73840a3d20 0x7f73840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.255+0000 7f73a6775740 1 -- 172.21.15.83:0/555684893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd82260 msgr2=0x561bedd84650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.255+0000 7f73a6775740 1 --2- 172.21.15.83:0/555684893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd82260 0x561bedd84650 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7388009c60 tx=0x7f7388011810 comp rx=0 tx=0).stop 2024-04-23T14:55:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.255+0000 7f73a1800640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f73840a3d20 0x7f73840a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7394006010 tx=0x7f739403d040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.255+0000 7f73a6775740 1 -- 172.21.15.83:0/555684893 shutdown_connections 2024-04-23T14:55:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.255+0000 7f73a6775740 1 --2- 172.21.15.83:0/555684893 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd82260 0x561bedd84650 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.255+0000 7f73a6775740 1 --2- 172.21.15.83:0/555684893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 0x561bedd81d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.255+0000 7f73a6775740 1 --2- 172.21.15.83:0/555684893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd77650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.255+0000 7f73a6775740 1 -- 172.21.15.83:0/555684893 >> 172.21.15.83:0/555684893 conn(0x561bedd73590 msgr2=0x561bedd73970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.255+0000 7f73a6775740 1 -- 172.21.15.83:0/555684893 shutdown_connections 2024-04-23T14:55:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.255+0000 7f73a6775740 1 -- 172.21.15.83:0/555684893 wait complete. 2024-04-23T14:55:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.255+0000 7f73a6775740 1 -- start start 2024-04-23T14:55:52.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f73a6775740 1 Processor -- start 2024-04-23T14:55:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f73a6775740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd7f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f73a6775740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd77b90 0x561bedd7f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f73a6775740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd82260 0x561bedd7fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f73a6775740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561bedd85050 con 0x561bedd82260 2024-04-23T14:55:52.266 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f73a6775740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561bedc96660 con 0x561bedd77270 2024-04-23T14:55:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f73a6775740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561bedd851d0 con 0x561bedd77b90 2024-04-23T14:55:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f73a6775740 10 client.0 _ll_register_callbacks cb 0x561bedc84aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f73a1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd82260 0x561bedd7fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f739be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd77b90 0x561bedd7f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f739be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd77b90 0x561bedd7f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42556/0 (socket says 172.21.15.83:42556) 2024-04-23T14:55:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f739be00640 1 -- 172.21.15.83:0/2368603438 learned_addr learned my addr 172.21.15.83:0/2368603438 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.256+0000 7f73a0e00640 1 --2- 172.21.15.83:0/2368603438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd7f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.257+0000 7f739be00640 1 -- 172.21.15.83:0/2368603438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 msgr2=0x561bedd7f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.267 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.257+0000 7f739be00640 1 --2- 172.21.15.83:0/2368603438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd7f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.257+0000 7f739be00640 1 -- 172.21.15.83:0/2368603438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd82260 msgr2=0x561bedd7fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.257+0000 7f739be00640 1 --2- 172.21.15.83:0/2368603438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd82260 0x561bedd7fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.257+0000 7f739be00640 1 -- 172.21.15.83:0/2368603438 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bedb17f70 con 0x561bedd77b90 2024-04-23T14:55:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.257+0000 7f739be00640 1 --2- 172.21.15.83:0/2368603438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd77b90 0x561bedd7f640 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7388004a20 tx=0x7f7388019ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96773b7781e7bced server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.257+0000 7f7382a00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:55:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.257+0000 7f7382a00640 1 -- 172.21.15.83:0/2368603438 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388024620 con 0x561bedd77b90 2024-04-23T14:55:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.257+0000 7f7382a00640 1 -- 172.21.15.83:0/2368603438 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f73880247c0 con 0x561bedd77b90 2024-04-23T14:55:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.258+0000 7f73a6775740 1 -- 172.21.15.83:0/2368603438 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73880043d0 con 0x561bedd77b90 2024-04-23T14:55:52.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.258+0000 7f7382a00640 1 -- 172.21.15.83:0/2368603438 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f73880249c0 con 0x561bedd77b90 2024-04-23T14:55:52.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.258+0000 7f73a6775740 10 client.26686 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:52.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.258+0000 7f73a6775740 1 -- 172.21.15.83:0/2368603438 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561bed9af170 con 0x561bedd77b90 2024-04-23T14:55:52.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.259+0000 7f7382a00640 1 -- 172.21.15.83:0/2368603438 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73880043d0 con 0x561bedd77b90 2024-04-23T14:55:52.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.259+0000 7f7382a00640 1 -- 172.21.15.83:0/2368603438 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f7388018070 con 0x561bedd77b90 2024-04-23T14:55:52.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.259+0000 7f73a6775740 10 client.26686 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:52.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.259+0000 7f73a6775740 10 client.26686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:52.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.259+0000 7f73a6775740 20 client.26686 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:52.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.259+0000 7f73a6775740 1 --2- 172.21.15.83:0/2368603438 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561bedd92390 0x561bedd94780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.259+0000 7f73a6775740 4 client.26686 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:55:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.259+0000 7f73a6775740 1 -- 172.21.15.83:0/2368603438 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bedcea890 con 0x561bedd92390 2024-04-23T14:55:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.260+0000 7f73a0e00640 1 --2- 172.21.15.83:0/2368603438 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561bedd92390 0x561bedd94780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.260+0000 7f73a0e00640 1 --2- 172.21.15.83:0/2368603438 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561bedd92390 0x561bedd94780 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d10f15cd26a246a server_cookie=1955e0aac4cace12 in_seq=0 out_seq=0 2024-04-23T14:55:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.260+0000 7f7382a00640 10 client.26686 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f7382a00640 1 -- 172.21.15.83:0/2368603438 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561bedcea890 con 0x561bedd92390 2024-04-23T14:55:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f7382a00640 10 client.26686 handle_command_reply: tid=0 2024-04-23T14:55:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f73a6775740 1 client.26686 shutdown 2024-04-23T14:55:52.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f73a6775740 1 -- 172.21.15.83:0/2368603438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd77b90 msgr2=0x561bedd7f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f73a6775740 1 --2- 172.21.15.83:0/2368603438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd77b90 0x561bedd7f640 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7388004a20 tx=0x7f7388019ae0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f73a6775740 1 -- 172.21.15.83:0/2368603438 shutdown_connections 2024-04-23T14:55:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f73a6775740 1 --2- 172.21.15.83:0/2368603438 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561bedd92390 0x561bedd94780 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f73a6775740 1 --2- 172.21.15.83:0/2368603438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd82260 0x561bedd7fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f73a6775740 1 --2- 172.21.15.83:0/2368603438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd77b90 0x561bedd7f640 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f73a6775740 1 --2- 172.21.15.83:0/2368603438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd7f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f7382a00640 0 client.26686 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:52.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f73a6775740 1 -- 172.21.15.83:0/2368603438 >> 172.21.15.83:0/2368603438 conn(0x561bedd73590 msgr2=0x561bedd726f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:52.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f73a6775740 1 -- 172.21.15.83:0/2368603438 shutdown_connections 2024-04-23T14:55:52.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f73a6775740 1 -- 172.21.15.83:0/2368603438 wait complete. 2024-04-23T14:55:52.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.261+0000 7f73a6775740 20 client.26686 trim_cache size 0 max 16384 2024-04-23T14:55:52.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f73a6775740 1 Processor -- start 2024-04-23T14:55:52.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f73a6775740 1 -- start start 2024-04-23T14:55:52.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f73a6775740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd8df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.281 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f73a6775740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 0x561bedd724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f73a6775740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd82260 0x561bedd79a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f73a6775740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1b6a50 con 0x561bedd77b90 2024-04-23T14:55:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f73a6775740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1b4320 con 0x561bedd77270 2024-04-23T14:55:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f73a6775740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1b2190 con 0x561bedd82260 2024-04-23T14:55:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f739be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 0x561bedd724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f739be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 0x561bedd724d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51206/0 (socket says 172.21.15.83:51206) 2024-04-23T14:55:52.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f739be00640 1 -- 172.21.15.83:0/422448001 learned_addr learned my addr 172.21.15.83:0/422448001 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:52.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f73a1800640 1 --2- 172.21.15.83:0/422448001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd82260 0x561bedd79a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.281+0000 7f73a0e00640 1 --2- 172.21.15.83:0/422448001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd8df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.286 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.282+0000 7f739be00640 1 -- 172.21.15.83:0/422448001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd82260 msgr2=0x561bedd79a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.282+0000 7f739be00640 1 --2- 172.21.15.83:0/422448001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd82260 0x561bedd79a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.282+0000 7f739be00640 1 -- 172.21.15.83:0/422448001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 msgr2=0x561bedd8df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.282+0000 7f739be00640 1 --2- 172.21.15.83:0/422448001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd8df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.282+0000 7f739be00640 1 -- 172.21.15.83:0/422448001 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bedd6eef0 con 0x561bedd77b90 2024-04-23T14:55:52.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.282+0000 7f739be00640 1 --2- 172.21.15.83:0/422448001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 0x561bedd724d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f738801c040 tx=0x7f7388011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa159df95a1aaca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:52.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.282+0000 7f738d600640 1 -- 172.21.15.83:0/422448001 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388005d30 con 0x561bedd77b90 2024-04-23T14:55:52.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.282+0000 7f738d600640 1 -- 172.21.15.83:0/422448001 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7388029070 con 0x561bedd77b90 2024-04-23T14:55:52.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.282+0000 7f738d600640 1 -- 172.21.15.83:0/422448001 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7388024dc0 con 0x561bedd77b90 2024-04-23T14:55:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.283+0000 7f73a6775740 1 -- 172.21.15.83:0/422448001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 msgr2=0x561bedd724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.283+0000 7f73a6775740 1 --2- 172.21.15.83:0/422448001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 0x561bedd724d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f738801c040 tx=0x7f7388011810 comp rx=0 tx=0).stop 2024-04-23T14:55:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.283+0000 7f73a6775740 1 -- 172.21.15.83:0/422448001 shutdown_connections 2024-04-23T14:55:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.283+0000 7f73a6775740 1 --2- 172.21.15.83:0/422448001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd82260 0x561bedd79a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.283+0000 7f73a6775740 1 --2- 172.21.15.83:0/422448001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd77b90 0x561bedd724d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.283+0000 7f73a6775740 1 --2- 172.21.15.83:0/422448001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77270 0x561bedd8df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.283+0000 7f73a6775740 1 -- 172.21.15.83:0/422448001 >> 172.21.15.83:0/422448001 conn(0x561bedd73590 msgr2=0x561bedd7e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.283+0000 7f73a6775740 1 -- 172.21.15.83:0/422448001 shutdown_connections 2024-04-23T14:55:52.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.283+0000 7f73a6775740 1 -- 172.21.15.83:0/422448001 wait complete. 2024-04-23T14:55:52.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.284+0000 7f73a6775740 1 -- start start 2024-04-23T14:55:52.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.284+0000 7f73a6775740 1 Processor -- start 2024-04-23T14:55:52.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.284+0000 7f73a6775740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd77270 0x561bedd7bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.284+0000 7f73a6775740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77b90 0x561bedd7c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.284+0000 7f73a6775740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd82260 0x561bedd7c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.284+0000 7f73a6775740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1b27e0 con 0x561bedd82260 2024-04-23T14:55:52.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.284+0000 7f73a6775740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1b3c50 con 0x561bedd77b90 2024-04-23T14:55:52.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.284+0000 7f73a6775740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f739c1b2eb0 con 0x561bedd77270 2024-04-23T14:55:52.289 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.284+0000 7f73a6775740 10 client.0 _ll_register_callbacks cb 0x7f73900d46a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.285+0000 7f73a1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd82260 0x561bedd7c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.285+0000 7f73a1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd82260 0x561bedd7c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51220/0 (socket says 172.21.15.83:51220) 2024-04-23T14:55:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.285+0000 7f73a1800640 1 -- 172.21.15.83:0/275363471 learned_addr learned my addr 172.21.15.83:0/275363471 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.285+0000 7f73a0e00640 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd77270 0x561bedd7bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.285+0000 7f739be00640 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77b90 0x561bedd7c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.285+0000 7f73a1800640 1 -- 172.21.15.83:0/275363471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd77270 msgr2=0x561bedd7bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.285+0000 7f73a1800640 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd77270 0x561bedd7bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.285+0000 7f73a1800640 1 -- 172.21.15.83:0/275363471 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77b90 msgr2=0x561bedd7c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.290 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.285+0000 7f73a1800640 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77b90 0x561bedd7c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.285+0000 7f73a1800640 1 -- 172.21.15.83:0/275363471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561beddf78c0 con 0x561bedd82260 2024-04-23T14:55:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.285+0000 7f73a1800640 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd82260 0x561bedd7c810 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7394009860 tx=0x7f739402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3571676afab4600c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.285+0000 7f7382a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:55:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.286+0000 7f7382a00640 1 -- 172.21.15.83:0/275363471 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394057020 con 0x561bedd82260 2024-04-23T14:55:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.286+0000 7f7382a00640 1 -- 172.21.15.83:0/275363471 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f73940309b0 con 0x561bedd82260 2024-04-23T14:55:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.286+0000 7f73a6775740 1 -- 172.21.15.83:0/275363471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7388029070 con 0x561bedd82260 2024-04-23T14:55:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.286+0000 7f7382a00640 1 -- 172.21.15.83:0/275363471 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7394062050 con 0x561bedd82260 2024-04-23T14:55:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.286+0000 7f73a6775740 10 client.16512 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:52.291 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.286+0000 7f73a6775740 1 -- 172.21.15.83:0/275363471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561beddf8b60 con 0x561bedd82260 2024-04-23T14:55:52.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.286+0000 7f7382a00640 1 -- 172.21.15.83:0/275363471 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7388029070 con 0x561bedd82260 2024-04-23T14:55:52.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.286+0000 7f7382a00640 1 -- 172.21.15.83:0/275363471 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f7394030070 con 0x561bedd82260 2024-04-23T14:55:52.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.286+0000 7f73a6775740 10 client.16512 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:52.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.286+0000 7f73a6775740 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:52.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.286+0000 7f73a6775740 20 client.16512 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:52.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.286+0000 7f73a6775740 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561beddf93e0 0x561beddfb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.287+0000 7f73a6775740 4 client.16512 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:55:52.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.287+0000 7f73a6775740 1 -- 172.21.15.83:0/275363471 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561bedce4700 con 0x561beddf93e0 2024-04-23T14:55:52.292 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.287+0000 7f73a0e00640 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561beddf93e0 0x561beddfb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.287+0000 7f73a0e00640 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561beddf93e0 0x561beddfb800 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f5b33023b77680a server_cookie=6a7beedab30e9021 in_seq=0 out_seq=0 2024-04-23T14:55:52.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.287+0000 7f7382a00640 10 client.16512 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:52.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.288+0000 7f7382a00640 1 -- 172.21.15.83:0/275363471 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561bedce4700 con 0x561beddf93e0 2024-04-23T14:55:52.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.288+0000 7f7382a00640 10 client.16512 handle_command_reply: tid=0 2024-04-23T14:55:52.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.288+0000 7f73a6775740 1 client.16512 shutdown 2024-04-23T14:55:52.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.288+0000 7f73a6775740 1 -- 172.21.15.83:0/275363471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd82260 msgr2=0x561bedd7c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.288+0000 7f73a6775740 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd82260 0x561bedd7c810 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7394009860 tx=0x7f739402a8e0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.294 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.288+0000 7f73a6775740 1 -- 172.21.15.83:0/275363471 shutdown_connections 2024-04-23T14:55:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.289+0000 7f73a6775740 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561beddf93e0 0x561beddfb800 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.289+0000 7f73a6775740 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561bedd82260 0x561bedd7c810 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.289+0000 7f73a6775740 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561bedd77b90 0x561bedd7c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.289+0000 7f73a6775740 1 --2- 172.21.15.83:0/275363471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561bedd77270 0x561bedd7bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.289+0000 7f73a6775740 1 -- 172.21.15.83:0/275363471 >> 172.21.15.83:0/275363471 conn(0x561bedd73590 msgr2=0x561bedd7f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.289+0000 7f7382a00640 0 client.16512 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.289+0000 7f73a6775740 1 -- 172.21.15.83:0/275363471 shutdown_connections 2024-04-23T14:55:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.289+0000 7f73a6775740 1 -- 172.21.15.83:0/275363471 wait complete. 2024-04-23T14:55:52.295 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.289+0000 7f73a6775740 20 client.16512 trim_cache size 0 max 16384 2024-04-23T14:55:52.296 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:55:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:55:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "5d50aec0-b1ed-4a58-a906-4730c1eb9f34", 2024-04-23T14:55:52.296 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:55:52.296 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:55:52.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.289+0000 7f73a2200640 1 -- 172.21.15.83:0/3940068981 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f73840a3d20 msgr2=0x7f73840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.289+0000 7f73a2200640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f73840a3d20 0x7f73840a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7394006010 tx=0x7f739403d040 comp rx=0 tx=0).stop 2024-04-23T14:55:52.297 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.290+0000 7f73a2200640 1 -- 172.21.15.83:0/3940068981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f739c108580 msgr2=0x7f739c1a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.290+0000 7f73a2200640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f739c108580 0x7f739c1a4940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f739000e9c0 tx=0x7f739000ee80 comp rx=0 tx=0).stop 2024-04-23T14:55:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.290+0000 7f73a2200640 1 -- 172.21.15.83:0/3940068981 shutdown_connections 2024-04-23T14:55:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.290+0000 7f73a2200640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f73840a3d20 0x7f73840a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.290+0000 7f73a2200640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f739c10d500 0x7f739c19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.290+0000 7f73a2200640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f739c108f40 0x7f739c1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.290+0000 7f73a2200640 1 --2- 172.21.15.83:0/3940068981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f739c108580 0x7f739c1a4940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.290+0000 7f73a2200640 1 -- 172.21.15.83:0/3940068981 >> 172.21.15.83:0/3940068981 conn(0x7f739c077e20 msgr2=0x7f739c078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:52.298 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.290+0000 7f73a2200640 1 -- 172.21.15.83:0/3940068981 shutdown_connections 2024-04-23T14:55:52.299 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.290+0000 7f73a2200640 1 -- 172.21.15.83:0/3940068981 wait complete. 2024-04-23T14:55:52.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:55:52.308 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:55:52.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.503+0000 7f69d6800640 1 -- 172.21.15.83:0/363669976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69d0108580 msgr2=0x7f69d0108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.503+0000 7f69d6800640 1 --2- 172.21.15.83:0/363669976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69d0108580 0x7f69d0108a00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009930 tx=0x7f69bc02a420 comp rx=0 tx=0).stop 2024-04-23T14:55:52.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.503+0000 7f69d6800640 1 -- 172.21.15.83:0/363669976 shutdown_connections 2024-04-23T14:55:52.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.503+0000 7f69d6800640 1 --2- 172.21.15.83:0/363669976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69d0108f40 0x7f69d01153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.503+0000 7f69d6800640 1 --2- 172.21.15.83:0/363669976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69d0108580 0x7f69d0108a00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.503+0000 7f69d6800640 1 --2- 172.21.15.83:0/363669976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69d010d500 0x7f69d010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.503+0000 7f69d6800640 1 -- 172.21.15.83:0/363669976 >> 172.21.15.83:0/363669976 conn(0x7f69d0077e20 msgr2=0x7f69d0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:52.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.504+0000 7f69d6800640 1 -- 172.21.15.83:0/363669976 shutdown_connections 2024-04-23T14:55:52.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.504+0000 7f69d6800640 1 -- 172.21.15.83:0/363669976 wait complete. 2024-04-23T14:55:52.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.504+0000 7f69d6800640 1 Processor -- start 2024-04-23T14:55:52.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.505+0000 7f69d6800640 1 -- start start 2024-04-23T14:55:52.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.505+0000 7f69d6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69d0108580 0x7f69d019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.505+0000 7f69d6800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69d0108f40 0x7f69d019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.505+0000 7f69d6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69d010d500 0x7f69d01a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.505+0000 7f69d6800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0118900 con 0x7f69d010d500 2024-04-23T14:55:52.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.505+0000 7f69d6800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0118780 con 0x7f69d0108580 2024-04-23T14:55:52.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.505+0000 7f69d6800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f69d0118a80 con 0x7f69d0108f40 2024-04-23T14:55:52.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.505+0000 7f69d4a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69d0108f40 0x7f69d019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.505+0000 7f69d4a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69d0108f40 0x7f69d019f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42592/0 (socket says 172.21.15.83:42592) 2024-04-23T14:55:52.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.505+0000 7f69d4a00640 1 -- 172.21.15.83:0/1883191456 learned_addr learned my addr 172.21.15.83:0/1883191456 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:52.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.505+0000 7f69d5e00640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69d010d500 0x7f69d01a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.505+0000 7f69d5400640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69d0108580 0x7f69d019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.506+0000 7f69d4a00640 1 -- 172.21.15.83:0/1883191456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69d0108580 msgr2=0x7f69d019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.506+0000 7f69d4a00640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69d0108580 0x7f69d019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.506+0000 7f69d4a00640 1 -- 172.21.15.83:0/1883191456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69d010d500 msgr2=0x7f69d01a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.506+0000 7f69d4a00640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69d010d500 0x7f69d01a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.506+0000 7f69d4a00640 1 -- 172.21.15.83:0/1883191456 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d01a3e10 con 0x7f69d0108f40 2024-04-23T14:55:52.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.506+0000 7f69d5400640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69d0108580 0x7f69d019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:55:52.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.506+0000 7f69d4a00640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69d0108f40 0x7f69d019f360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f69bc02a940 tx=0x7f69bc003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:52.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.506+0000 7f69ce000640 1 -- 172.21.15.83:0/1883191456 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc03f070 con 0x7f69d0108f40 2024-04-23T14:55:52.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.506+0000 7f69ce000640 1 -- 172.21.15.83:0/1883191456 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69bc031c90 con 0x7f69d0108f40 2024-04-23T14:55:52.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.506+0000 7f69ce000640 1 -- 172.21.15.83:0/1883191456 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69bc035030 con 0x7f69d0108f40 2024-04-23T14:55:52.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.506+0000 7f69d6800640 1 -- 172.21.15.83:0/1883191456 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69d01a40a0 con 0x7f69d0108f40 2024-04-23T14:55:52.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.506+0000 7f69d6800640 1 -- 172.21.15.83:0/1883191456 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d01ab990 con 0x7f69d0108f40 2024-04-23T14:55:52.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.509+0000 7f69ce000640 1 -- 172.21.15.83:0/1883191456 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f69bc042070 con 0x7f69d0108f40 2024-04-23T14:55:52.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69dac18740 1 Processor -- start 2024-04-23T14:55:52.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69dac18740 1 -- start start 2024-04-23T14:55:52.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69dac18740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 0x55801f62ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69dac18740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62b350 0x55801f6354e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69dac18740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f635a20 0x55801f637e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69dac18740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55801f5901f0 con 0x55801f62aa30 2024-04-23T14:55:52.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69dac18740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55801f536350 con 0x55801f635a20 2024-04-23T14:55:52.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69dac18740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55801f58c1b0 con 0x55801f62b350 2024-04-23T14:55:52.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69d5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 0x55801f62ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69d5400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62b350 0x55801f6354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69d5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 0x55801f62ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51256/0 (socket says 172.21.15.83:51256) 2024-04-23T14:55:52.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69d5400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62b350 0x55801f6354e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:42604/0 (socket says 172.21.15.83:42604) 2024-04-23T14:55:52.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.510+0000 7f69d5e00640 1 -- 172.21.15.83:0/2591756935 learned_addr learned my addr 172.21.15.83:0/2591756935 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:52.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69d4a00640 1 --2- 172.21.15.83:0/2591756935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f635a20 0x55801f637e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69d5400640 1 -- 172.21.15.83:0/2591756935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f635a20 msgr2=0x55801f637e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69d5400640 1 --2- 172.21.15.83:0/2591756935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f635a20 0x55801f637e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69d5400640 1 -- 172.21.15.83:0/2591756935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 msgr2=0x55801f62ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69d5400640 1 --2- 172.21.15.83:0/2591756935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 0x55801f62ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69d5400640 1 -- 172.21.15.83:0/2591756935 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55801f58af50 con 0x55801f62b350 2024-04-23T14:55:52.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69d4a00640 1 --2- 172.21.15.83:0/2591756935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f635a20 0x55801f637e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:55:52.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69d5400640 1 --2- 172.21.15.83:0/2591756935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62b350 0x55801f6354e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009b80 tx=0x7f69c40117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=558582168a2a7c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:52.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69ce000640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f69b80a3da0 0x7f69b80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69ce000640 1 -- 172.21.15.83:0/1883191456 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f69bc0f0b40 con 0x7f69d0108f40 2024-04-23T14:55:52.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69c2600640 1 -- 172.21.15.83:0/2591756935 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4004050 con 0x55801f62b350 2024-04-23T14:55:52.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69c2600640 1 -- 172.21.15.83:0/2591756935 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c4016070 con 0x55801f62b350 2024-04-23T14:55:52.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.511+0000 7f69c2600640 1 -- 172.21.15.83:0/2591756935 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4004b20 con 0x55801f62b350 2024-04-23T14:55:52.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.512+0000 7f69d5e00640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f69b80a3da0 0x7f69b80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.512+0000 7f69d5e00640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f69b80a3da0 0x7f69b80a6260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f69c80116b0 tx=0x7f69c8009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:52.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.512+0000 7f69dac18740 1 -- 172.21.15.83:0/2591756935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62b350 msgr2=0x55801f6354e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.512+0000 7f69dac18740 1 --2- 172.21.15.83:0/2591756935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62b350 0x55801f6354e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009b80 tx=0x7f69c40117b0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.512+0000 7f69dac18740 1 -- 172.21.15.83:0/2591756935 shutdown_connections 2024-04-23T14:55:52.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.512+0000 7f69dac18740 1 --2- 172.21.15.83:0/2591756935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f635a20 0x55801f637e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.512+0000 7f69dac18740 1 --2- 172.21.15.83:0/2591756935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62b350 0x55801f6354e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.512+0000 7f69dac18740 1 --2- 172.21.15.83:0/2591756935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 0x55801f62ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.512+0000 7f69dac18740 1 -- 172.21.15.83:0/2591756935 >> 172.21.15.83:0/2591756935 conn(0x55801f626d50 msgr2=0x55801f627130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.512+0000 7f69dac18740 1 -- 172.21.15.83:0/2591756935 shutdown_connections 2024-04-23T14:55:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.512+0000 7f69dac18740 1 -- 172.21.15.83:0/2591756935 wait complete. 2024-04-23T14:55:52.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.513+0000 7f69dac18740 1 -- start start 2024-04-23T14:55:52.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.513+0000 7f69dac18740 1 Processor -- start 2024-04-23T14:55:52.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.513+0000 7f69dac18740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62aa30 0x55801f632a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69dac18740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62b350 0x55801f632f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69dac18740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 0x55801f633480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69dac18740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55801f638810 con 0x55801f635a20 2024-04-23T14:55:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69dac18740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55801f549690 con 0x55801f62aa30 2024-04-23T14:55:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69dac18740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55801f638990 con 0x55801f62b350 2024-04-23T14:55:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69dac18740 10 client.0 _ll_register_callbacks cb 0x55801f585650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69d5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 0x55801f633480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69d5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 0x55801f633480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51270/0 (socket says 172.21.15.83:51270) 2024-04-23T14:55:52.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69d5e00640 1 -- 172.21.15.83:0/1262305841 learned_addr learned my addr 172.21.15.83:0/1262305841 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:52.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69d5400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62aa30 0x55801f632a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69d4a00640 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62b350 0x55801f632f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69d5e00640 1 -- 172.21.15.83:0/1262305841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62b350 msgr2=0x55801f632f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69d5e00640 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62b350 0x55801f632f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69d5e00640 1 -- 172.21.15.83:0/1262305841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62aa30 msgr2=0x55801f632a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69d5e00640 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62aa30 0x55801f632a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69d5e00640 1 -- 172.21.15.83:0/1262305841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55801f548920 con 0x55801f635a20 2024-04-23T14:55:52.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.514+0000 7f69d5400640 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62aa30 0x55801f632a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:52.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.515+0000 7f69d5e00640 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 0x55801f633480 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f69c800f690 tx=0x7f69c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d98b968353b870 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:52.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.515+0000 7f69b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:55:52.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.515+0000 7f69b6a00640 1 -- 172.21.15.83:0/1262305841 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c802c060 con 0x55801f635a20 2024-04-23T14:55:52.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.515+0000 7f69b6a00640 1 -- 172.21.15.83:0/1262305841 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f69c8017030 con 0x55801f635a20 2024-04-23T14:55:52.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.515+0000 7f69dac18740 1 -- 172.21.15.83:0/1262305841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69c4016070 con 0x55801f635a20 2024-04-23T14:55:52.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.515+0000 7f69b6a00640 1 -- 172.21.15.83:0/1262305841 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69c801f8c0 con 0x55801f635a20 2024-04-23T14:55:52.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.515+0000 7f69dac18740 10 client.16524 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:52.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.515+0000 7f69dac18740 1 -- 172.21.15.83:0/1262305841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55801f36c7d0 con 0x55801f635a20 2024-04-23T14:55:52.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.515+0000 7f69b6a00640 1 -- 172.21.15.83:0/1262305841 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c4016070 con 0x55801f635a20 2024-04-23T14:55:52.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.516+0000 7f69b6a00640 1 -- 172.21.15.83:0/1262305841 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f69c8010c30 con 0x55801f635a20 2024-04-23T14:55:52.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.516+0000 7f69dac18740 10 client.16524 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:52.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.516+0000 7f69dac18740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:52.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.516+0000 7f69dac18740 20 client.16524 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:52.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.516+0000 7f69dac18740 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55801f645e80 0x55801f6482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.516+0000 7f69dac18740 4 client.16524 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:55:52.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.516+0000 7f69dac18740 1 -- 172.21.15.83:0/1262305841 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55801f59dc10 con 0x55801f645e80 2024-04-23T14:55:52.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.516+0000 7f69d5400640 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55801f645e80 0x55801f6482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.517+0000 7f69d5400640 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55801f645e80 0x55801f6482b0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61032bbcaa4dc5be server_cookie=b7fab5707953cded in_seq=0 out_seq=0 2024-04-23T14:55:52.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.517+0000 7f69b6a00640 10 client.16524 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:52.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69b6a00640 1 -- 172.21.15.83:0/1262305841 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55801f59dc10 con 0x55801f645e80 2024-04-23T14:55:52.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69b6a00640 10 client.16524 handle_command_reply: tid=0 2024-04-23T14:55:52.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69dac18740 1 client.16524 shutdown 2024-04-23T14:55:52.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69dac18740 1 -- 172.21.15.83:0/1262305841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 msgr2=0x55801f633480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69dac18740 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 0x55801f633480 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f69c800f690 tx=0x7f69c8004ab0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69dac18740 1 -- 172.21.15.83:0/1262305841 shutdown_connections 2024-04-23T14:55:52.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69dac18740 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55801f645e80 0x55801f6482b0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69dac18740 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 0x55801f633480 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69dac18740 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62b350 0x55801f632f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69dac18740 1 --2- 172.21.15.83:0/1262305841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62aa30 0x55801f632a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69b6a00640 0 client.16524 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:52.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69dac18740 1 -- 172.21.15.83:0/1262305841 >> 172.21.15.83:0/1262305841 conn(0x55801f626d50 msgr2=0x55801f625fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:52.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69dac18740 1 -- 172.21.15.83:0/1262305841 shutdown_connections 2024-04-23T14:55:52.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69dac18740 1 -- 172.21.15.83:0/1262305841 wait complete. 2024-04-23T14:55:52.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.518+0000 7f69dac18740 20 client.16524 trim_cache size 0 max 16384 2024-04-23T14:55:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.537+0000 7f69dac18740 1 Processor -- start 2024-04-23T14:55:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.537+0000 7f69dac18740 1 -- start start 2024-04-23T14:55:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.537+0000 7f69dac18740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62aa30 0x55801f626750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.537+0000 7f69dac18740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62b350 0x55801f641700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.537+0000 7f69dac18740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 0x55801f62c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.537+0000 7f69dac18740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01b6a90 con 0x55801f635a20 2024-04-23T14:55:52.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69dac18740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01b4360 con 0x55801f62b350 2024-04-23T14:55:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69dac18740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01b21d0 con 0x55801f62aa30 2024-04-23T14:55:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69d5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 0x55801f62c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69d5e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 0x55801f62c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51274/0 (socket says 172.21.15.83:51274) 2024-04-23T14:55:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69d5e00640 1 -- 172.21.15.83:0/1475979204 learned_addr learned my addr 172.21.15.83:0/1475979204 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69d4a00640 1 --2- 172.21.15.83:0/1475979204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62b350 0x55801f641700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69d5400640 1 --2- 172.21.15.83:0/1475979204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62aa30 0x55801f626750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69d5e00640 1 -- 172.21.15.83:0/1475979204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62aa30 msgr2=0x55801f626750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69d5e00640 1 --2- 172.21.15.83:0/1475979204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62aa30 0x55801f626750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69d5e00640 1 -- 172.21.15.83:0/1475979204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62b350 msgr2=0x55801f641700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69d5e00640 1 --2- 172.21.15.83:0/1475979204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62b350 0x55801f641700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69d5e00640 1 -- 172.21.15.83:0/1475979204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55801f2a23f0 con 0x55801f635a20 2024-04-23T14:55:52.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69d4a00640 1 --2- 172.21.15.83:0/1475979204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62b350 0x55801f641700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:55:52.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.538+0000 7f69d5400640 1 --2- 172.21.15.83:0/1475979204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62aa30 0x55801f626750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:55:52.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.539+0000 7f69d5e00640 1 --2- 172.21.15.83:0/1475979204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 0x55801f62c890 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f69c800ff50 tx=0x7f69c800ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ceab5def71fe90 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:52.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.539+0000 7f69c1c00640 1 -- 172.21.15.83:0/1475979204 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c801f2f0 con 0x55801f635a20 2024-04-23T14:55:52.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.539+0000 7f69c1c00640 1 -- 172.21.15.83:0/1475979204 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c80276a0 con 0x55801f635a20 2024-04-23T14:55:52.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.539+0000 7f69c1c00640 1 -- 172.21.15.83:0/1475979204 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c8030660 con 0x55801f635a20 2024-04-23T14:55:52.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.539+0000 7f69dac18740 1 -- 172.21.15.83:0/1475979204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 msgr2=0x55801f62c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.539+0000 7f69dac18740 1 --2- 172.21.15.83:0/1475979204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 0x55801f62c890 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f69c800ff50 tx=0x7f69c800ff80 comp rx=0 tx=0).stop 2024-04-23T14:55:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.540+0000 7f69dac18740 1 -- 172.21.15.83:0/1475979204 shutdown_connections 2024-04-23T14:55:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.540+0000 7f69dac18740 1 --2- 172.21.15.83:0/1475979204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f635a20 0x55801f62c890 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.540+0000 7f69dac18740 1 --2- 172.21.15.83:0/1475979204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62b350 0x55801f641700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.540+0000 7f69dac18740 1 --2- 172.21.15.83:0/1475979204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f62aa30 0x55801f626750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.540+0000 7f69dac18740 1 -- 172.21.15.83:0/1475979204 >> 172.21.15.83:0/1475979204 conn(0x55801f626d50 msgr2=0x55801f62c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.540+0000 7f69dac18740 1 -- 172.21.15.83:0/1475979204 shutdown_connections 2024-04-23T14:55:52.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.540+0000 7f69dac18740 1 -- 172.21.15.83:0/1475979204 wait complete. 2024-04-23T14:55:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.540+0000 7f69dac18740 1 -- start start 2024-04-23T14:55:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.540+0000 7f69dac18740 1 Processor -- start 2024-04-23T14:55:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69dac18740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 0x55801f63dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69dac18740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62b350 0x55801f63e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69dac18740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f635a20 0x55801f62f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69dac18740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01b2820 con 0x55801f62aa30 2024-04-23T14:55:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69dac18740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01b3c90 con 0x55801f62b350 2024-04-23T14:55:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69dac18740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f69d01b2ef0 con 0x55801f635a20 2024-04-23T14:55:52.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69dac18740 10 client.0 _ll_register_callbacks cb 0x7f69c8009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:55:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69d5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 0x55801f63dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69d5400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 0x55801f63dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:51284/0 (socket says 172.21.15.83:51284) 2024-04-23T14:55:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69d5400640 1 -- 172.21.15.83:0/2509805996 learned_addr learned my addr 172.21.15.83:0/2509805996 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:55:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69d4a00640 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62b350 0x55801f63e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69d5e00640 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f635a20 0x55801f62f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69d5400640 1 -- 172.21.15.83:0/2509805996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f635a20 msgr2=0x55801f62f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69d5400640 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f635a20 0x55801f62f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69d5400640 1 -- 172.21.15.83:0/2509805996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62b350 msgr2=0x55801f63e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:55:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69d5400640 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62b350 0x55801f63e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.541+0000 7f69d5400640 1 -- 172.21.15.83:0/2509805996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55801f2620f0 con 0x55801f62aa30 2024-04-23T14:55:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.542+0000 7f69d5400640 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 0x55801f63dc10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009b80 tx=0x7f69c4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d72b6bd76ad9433 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:55:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.542+0000 7f69b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:55:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.542+0000 7f69b6a00640 1 -- 172.21.15.83:0/2509805996 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4005040 con 0x55801f62aa30 2024-04-23T14:55:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.542+0000 7f69b6a00640 1 -- 172.21.15.83:0/2509805996 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8542+0+0 (secure 0 0 0) 0x7f69c40051e0 con 0x55801f62aa30 2024-04-23T14:55:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.542+0000 7f69dac18740 1 -- 172.21.15.83:0/2509805996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69c80276a0 con 0x55801f62aa30 2024-04-23T14:55:52.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.542+0000 7f69b6a00640 1 -- 172.21.15.83:0/2509805996 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69c4038050 con 0x55801f62aa30 2024-04-23T14:55:52.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.542+0000 7f69dac18740 10 client.16530 fetch_fsmap learned FSMap version 29 2024-04-23T14:55:52.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.542+0000 7f69dac18740 1 -- 172.21.15.83:0/2509805996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55801f555810 con 0x55801f62aa30 2024-04-23T14:55:52.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.542+0000 7f69b6a00640 1 -- 172.21.15.83:0/2509805996 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c80276a0 con 0x55801f62aa30 2024-04-23T14:55:52.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.543+0000 7f69b6a00640 1 -- 172.21.15.83:0/2509805996 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f69c4022040 con 0x55801f62aa30 2024-04-23T14:55:52.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.543+0000 7f69dac18740 10 client.16530 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:55:52.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.543+0000 7f69dac18740 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:55:52.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.543+0000 7f69dac18740 20 client.16530 populate_metadata read hostname 'smithi083' 2024-04-23T14:55:52.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.543+0000 7f69dac18740 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55801f6ac130 0x55801f6ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:55:52.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.543+0000 7f69dac18740 4 client.16530 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:55:52.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.543+0000 7f69dac18740 1 -- 172.21.15.83:0/2509805996 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55801f597a80 con 0x55801f6ac130 2024-04-23T14:55:52.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.543+0000 7f69d4a00640 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55801f6ac130 0x55801f6ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:55:52.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.543+0000 7f69d4a00640 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55801f6ac130 0x55801f6ae520 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36159e68b2504021 server_cookie=56b99eb68de0a3cf in_seq=0 out_seq=0 2024-04-23T14:55:52.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.543+0000 7f69b6a00640 10 client.16530 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:52.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69b6a00640 1 -- 172.21.15.83:0/2509805996 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55801f597a80 con 0x55801f6ac130 2024-04-23T14:55:52.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69b6a00640 10 client.16530 handle_command_reply: tid=0 2024-04-23T14:55:52.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69dac18740 1 client.16530 shutdown 2024-04-23T14:55:52.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69dac18740 1 -- 172.21.15.83:0/2509805996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 msgr2=0x55801f63dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69dac18740 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 0x55801f63dc10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009b80 tx=0x7f69c4009620 comp rx=0 tx=0).stop 2024-04-23T14:55:52.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69dac18740 1 -- 172.21.15.83:0/2509805996 shutdown_connections 2024-04-23T14:55:52.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69dac18740 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55801f6ac130 0x55801f6ae520 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69dac18740 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55801f635a20 0x55801f62f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69dac18740 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55801f62b350 0x55801f63e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69b6a00640 0 client.16530 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:55:52.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69dac18740 1 --2- 172.21.15.83:0/2509805996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55801f62aa30 0x55801f63dc10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69dac18740 1 -- 172.21.15.83:0/2509805996 >> 172.21.15.83:0/2509805996 conn(0x55801f626d50 msgr2=0x55801f62ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:52.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69dac18740 1 -- 172.21.15.83:0/2509805996 shutdown_connections 2024-04-23T14:55:52.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69dac18740 1 -- 172.21.15.83:0/2509805996 wait complete. 2024-04-23T14:55:52.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.544+0000 7f69dac18740 20 client.16530 trim_cache size 0 max 16384 2024-04-23T14:55:52.553 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:55:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:55:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:55:52.553 INFO:teuthology.orchestra.run.smithi083.stdout: "5d50aec0-b1ed-4a58-a906-4730c1eb9f34": { 2024-04-23T14:55:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:55:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "5d50aec0-b1ed-4a58-a906-4730c1eb9f34", 2024-04-23T14:55:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:55:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:55:52.554 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:55:52.554 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:55:52.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.545+0000 7f69d6800640 1 -- 172.21.15.83:0/1883191456 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f69b80a3da0 msgr2=0x7f69b80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.545+0000 7f69d6800640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f69b80a3da0 0x7f69b80a6260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f69c80116b0 tx=0x7f69c8009500 comp rx=0 tx=0).stop 2024-04-23T14:55:52.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.545+0000 7f69d6800640 1 -- 172.21.15.83:0/1883191456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69d0108f40 msgr2=0x7f69d019f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:55:52.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.545+0000 7f69d6800640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69d0108f40 0x7f69d019f360 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f69bc02a940 tx=0x7f69bc003b20 comp rx=0 tx=0).stop 2024-04-23T14:55:52.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.545+0000 7f69d6800640 1 -- 172.21.15.83:0/1883191456 shutdown_connections 2024-04-23T14:55:52.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.545+0000 7f69d6800640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f69b80a3da0 0x7f69b80a6260 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.545+0000 7f69d6800640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f69d010d500 0x7f69d01a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.545+0000 7f69d6800640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69d0108f40 0x7f69d019f360 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.545+0000 7f69d6800640 1 --2- 172.21.15.83:0/1883191456 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69d0108580 0x7f69d019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:55:52.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.545+0000 7f69d6800640 1 -- 172.21.15.83:0/1883191456 >> 172.21.15.83:0/1883191456 conn(0x7f69d0077e20 msgr2=0x7f69d0078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:55:52.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.545+0000 7f69d6800640 1 -- 172.21.15.83:0/1883191456 shutdown_connections 2024-04-23T14:55:52.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:55:52.545+0000 7f69d6800640 1 -- 172.21.15.83:0/1883191456 wait complete. 2024-04-23T14:55:52.562 INFO:tasks.cephfs.filesystem:scrub status for tag:5d50aec0-b1ed-4a58-a906-4730c1eb9f34 - {'path': '/', 'tag': '5d50aec0-b1ed-4a58-a906-4730c1eb9f34', 'options': 'recursive,force'} 2024-04-23T14:55:52.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:52 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:55:52.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:52 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:55:52.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:52 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:55:52.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:52 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:55:52.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:52 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:55:52.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:52 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:55:52.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:52 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:55:52.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:52 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:55:52.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:52 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:55:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: pgmap v228: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:55:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:55:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:55:53.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:55:53.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:53.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:55:53.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:53.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:55:53.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:53.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:55:53.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:53.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:55:53.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:53.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:53 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: pgmap v228: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:55:54.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:55:54.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:55:54.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:55:54.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:55:54.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:55:54.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:55:54.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:55:54.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:53 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: pgmap v228: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:55:54.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:55:54.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:55:54.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:55:54.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:55:54.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:55:54.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:55:54.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:55:54.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:54.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:53 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:55:55.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:55 smithi203 ceph-mon[32800]: pgmap v229: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T14:55:56.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:55 smithi083 ceph-mon[27867]: pgmap v229: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T14:55:56.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:55 smithi173 ceph-mon[32443]: pgmap v229: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T14:55:57.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:57 smithi203 ceph-mon[32800]: pgmap v230: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T14:55:58.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:57 smithi083 ceph-mon[27867]: pgmap v230: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T14:55:58.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:57 smithi173 ceph-mon[32443]: pgmap v230: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T14:55:59.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:55:59 smithi203 ceph-mon[32800]: pgmap v231: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:56:00.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:55:59 smithi083 ceph-mon[27867]: pgmap v231: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:56:00.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:55:59 smithi173 ceph-mon[32443]: pgmap v231: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:56:01.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:01 smithi203 ceph-mon[32800]: pgmap v232: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:56:02.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:01 smithi083 ceph-mon[27867]: pgmap v232: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:56:02.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:01 smithi173 ceph-mon[32443]: pgmap v232: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:56:03.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:03 smithi203 ceph-mon[32800]: pgmap v233: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:56:04.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:03 smithi083 ceph-mon[27867]: pgmap v233: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:56:04.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:03 smithi173 ceph-mon[32443]: pgmap v233: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:56:04.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:04 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:56:04.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:04 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:56:04.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:04 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:56:05.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:04 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:56:05.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:04 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:56:05.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:04 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:56:05.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:04 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:56:05.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:04 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:56:05.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:04 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:56:05.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:05 smithi203 ceph-mon[32800]: pgmap v234: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:56:06.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:05 smithi083 ceph-mon[27867]: pgmap v234: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:56:06.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:05 smithi173 ceph-mon[32443]: pgmap v234: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:56:07.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:07 smithi203 ceph-mon[32800]: pgmap v235: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-23T14:56:08.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:07 smithi083 ceph-mon[27867]: pgmap v235: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-23T14:56:08.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:07 smithi173 ceph-mon[32443]: pgmap v235: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-23T14:56:09.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:09 smithi203 ceph-mon[32800]: pgmap v236: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-23T14:56:10.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:09 smithi083 ceph-mon[27867]: pgmap v236: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-23T14:56:10.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:09 smithi173 ceph-mon[32443]: pgmap v236: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-23T14:56:10.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:10 smithi203 ceph-mon[32800]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T14:56:11.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:10 smithi083 ceph-mon[27867]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T14:56:11.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:10 smithi173 ceph-mon[32443]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T14:56:11.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:11 smithi203 ceph-mon[32800]: pgmap v238: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-23T14:56:12.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:11 smithi083 ceph-mon[27867]: pgmap v238: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-23T14:56:12.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:11 smithi173 ceph-mon[32443]: pgmap v238: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-23T14:56:13.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:13 smithi203 ceph-mon[32800]: pgmap v239: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-23T14:56:14.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:13 smithi083 ceph-mon[27867]: pgmap v239: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-23T14:56:14.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:13 smithi173 ceph-mon[32443]: pgmap v239: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-23T14:56:15.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:15 smithi203 ceph-mon[32800]: pgmap v240: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-23T14:56:16.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:15 smithi083 ceph-mon[27867]: pgmap v240: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-23T14:56:16.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:15 smithi173 ceph-mon[32443]: pgmap v240: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-23T14:56:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:17 smithi203 ceph-mon[32800]: pgmap v241: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:56:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:17 smithi203 ceph-mon[32800]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T14:56:18.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:17 smithi083 ceph-mon[27867]: pgmap v241: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:56:18.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:17 smithi083 ceph-mon[27867]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T14:56:18.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:17 smithi173 ceph-mon[32443]: pgmap v241: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:56:18.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:17 smithi173 ceph-mon[32443]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T14:56:19.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:19 smithi203 ceph-mon[32800]: pgmap v243: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 250 B/s wr, 6 op/s 2024-04-23T14:56:20.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:19 smithi083 ceph-mon[27867]: pgmap v243: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 250 B/s wr, 6 op/s 2024-04-23T14:56:20.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:19 smithi173 ceph-mon[32443]: pgmap v243: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 250 B/s wr, 6 op/s 2024-04-23T14:56:21.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:20 smithi083 ceph-mon[27867]: pgmap v244: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:56:21.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:20 smithi173 ceph-mon[32443]: pgmap v244: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:56:21.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:20 smithi203 ceph-mon[32800]: pgmap v244: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:56:22.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:56:22.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:56:22.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.780+0000 7f2df4c00640 1 -- 172.21.15.83:0/1436489582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2df0108f40 msgr2=0x7f2df01153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:22.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.780+0000 7f2df4c00640 1 --2- 172.21.15.83:0/1436489582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2df0108f40 0x7f2df01153b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009cd0 tx=0x7f2de802a8f0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.780+0000 7f2df4c00640 1 -- 172.21.15.83:0/1436489582 shutdown_connections 2024-04-23T14:56:22.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.780+0000 7f2df4c00640 1 --2- 172.21.15.83:0/1436489582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2df0108f40 0x7f2df01153b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.780+0000 7f2df4c00640 1 --2- 172.21.15.83:0/1436489582 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df0108580 0x7f2df0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.780+0000 7f2df4c00640 1 --2- 172.21.15.83:0/1436489582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2df010d500 0x7f2df010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.780+0000 7f2df4c00640 1 -- 172.21.15.83:0/1436489582 >> 172.21.15.83:0/1436489582 conn(0x7f2df0077e20 msgr2=0x7f2df0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:22.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.780+0000 7f2df4c00640 1 -- 172.21.15.83:0/1436489582 shutdown_connections 2024-04-23T14:56:22.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.780+0000 7f2df4c00640 1 -- 172.21.15.83:0/1436489582 wait complete. 2024-04-23T14:56:22.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.781+0000 7f2df4c00640 1 Processor -- start 2024-04-23T14:56:22.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.781+0000 7f2df4c00640 1 -- start start 2024-04-23T14:56:22.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2df4c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df0108580 0x7f2df01a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2df4c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2df0108f40 0x7f2df01a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2df4c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2df010d500 0x7f2df019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2df4c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0118900 con 0x7f2df010d500 2024-04-23T14:56:22.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2df4c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0118780 con 0x7f2df0108580 2024-04-23T14:56:22.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2df4c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0118a80 con 0x7f2df0108f40 2024-04-23T14:56:22.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2defe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2df010d500 0x7f2df019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2de6a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2df0108f40 0x7f2df01a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2defe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2df010d500 0x7f2df019eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33032/0 (socket says 172.21.15.83:33032) 2024-04-23T14:56:22.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2defe00640 1 -- 172.21.15.83:0/3990978301 learned_addr learned my addr 172.21.15.83:0/3990978301 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:22.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2def400640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df0108580 0x7f2df01a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2de6a00640 1 -- 172.21.15.83:0/3990978301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df0108580 msgr2=0x7f2df01a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:22.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2de6a00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df0108580 0x7f2df01a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2de6a00640 1 -- 172.21.15.83:0/3990978301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2df010d500 msgr2=0x7f2df019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:22.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2de6a00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2df010d500 0x7f2df019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2de6a00640 1 -- 172.21.15.83:0/3990978301 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df019f2a0 con 0x7f2df0108f40 2024-04-23T14:56:22.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.782+0000 7f2defe00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2df010d500 0x7f2df019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:22.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.783+0000 7f2de6a00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2df0108f40 0x7f2df01a4e60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800e9f0 tx=0x7f2dd800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:22.793 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.783+0000 7f2decc00640 1 -- 172.21.15.83:0/3990978301 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd800cd80 con 0x7f2df0108f40 2024-04-23T14:56:22.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.783+0000 7f2df4c00640 1 -- 172.21.15.83:0/3990978301 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df019f590 con 0x7f2df0108f40 2024-04-23T14:56:22.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.783+0000 7f2decc00640 1 -- 172.21.15.83:0/3990978301 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd80035f0 con 0x7f2df0108f40 2024-04-23T14:56:22.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.783+0000 7f2decc00640 1 -- 172.21.15.83:0/3990978301 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8010670 con 0x7f2df0108f40 2024-04-23T14:56:22.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.783+0000 7f2df4c00640 1 -- 172.21.15.83:0/3990978301 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01ab7e0 con 0x7f2df0108f40 2024-04-23T14:56:22.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.787+0000 7f2df8ff2740 1 Processor -- start 2024-04-23T14:56:22.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.787+0000 7f2df8ff2740 1 -- start start 2024-04-23T14:56:22.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2df8ff2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec4a30 0x55c03eec4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.808 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2df8ff2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec5350 0x55c03eecf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2df8ff2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eecfa20 0x55c03eed1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2df8ff2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c03ee2a1f0 con 0x55c03eec4a30 2024-04-23T14:56:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2df8ff2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c03edd0350 con 0x55c03eec5350 2024-04-23T14:56:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2df8ff2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c03ee261b0 con 0x55c03eecfa20 2024-04-23T14:56:22.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2decc00640 1 -- 172.21.15.83:0/3990978301 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2dd80037a0 con 0x7f2df0108f40 2024-04-23T14:56:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2defe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec4a30 0x55c03eec4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2def400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec5350 0x55c03eecf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2defe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec4a30 0x55c03eec4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33042/0 (socket says 172.21.15.83:33042) 2024-04-23T14:56:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2def400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec5350 0x55c03eecf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:34152/0 (socket says 172.21.15.83:34152) 2024-04-23T14:56:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2defe00640 1 -- 172.21.15.83:0/2927164145 learned_addr learned my addr 172.21.15.83:0/2927164145 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:22.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.788+0000 7f2de6a00640 1 --2- 172.21.15.83:0/2927164145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eecfa20 0x55c03eed1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.789+0000 7f2def400640 1 -- 172.21.15.83:0/2927164145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eecfa20 msgr2=0x55c03eed1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:22.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.789+0000 7f2def400640 1 --2- 172.21.15.83:0/2927164145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eecfa20 0x55c03eed1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.789+0000 7f2def400640 1 -- 172.21.15.83:0/2927164145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec4a30 msgr2=0x55c03eec4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:22.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.789+0000 7f2def400640 1 --2- 172.21.15.83:0/2927164145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec4a30 0x55c03eec4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.789+0000 7f2def400640 1 -- 172.21.15.83:0/2927164145 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c03ee24f50 con 0x55c03eec5350 2024-04-23T14:56:22.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.789+0000 7f2def400640 1 --2- 172.21.15.83:0/2927164145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec5350 0x55c03eecf4e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009bf0 tx=0x7f2de0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0f8498fb56d7b4a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:22.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.789+0000 7f2ddfe00640 1 -- 172.21.15.83:0/2927164145 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0009da0 con 0x55c03eec5350 2024-04-23T14:56:22.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.789+0000 7f2ddfe00640 1 -- 172.21.15.83:0/2927164145 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de0005020 con 0x55c03eec5350 2024-04-23T14:56:22.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.789+0000 7f2ddfe00640 1 -- 172.21.15.83:0/2927164145 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0004bc0 con 0x55c03eec5350 2024-04-23T14:56:22.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.790+0000 7f2decc00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2dd00a3d20 0x7f2dd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.790+0000 7f2decc00640 1 -- 172.21.15.83:0/3990978301 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f2dd80d0b10 con 0x7f2df0108f40 2024-04-23T14:56:22.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.790+0000 7f2df8ff2740 1 -- 172.21.15.83:0/2927164145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec5350 msgr2=0x55c03eecf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:22.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.790+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/2927164145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec5350 0x55c03eecf4e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009bf0 tx=0x7f2de0011810 comp rx=0 tx=0).stop 2024-04-23T14:56:22.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.790+0000 7f2defe00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2dd00a3d20 0x7f2dd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.790+0000 7f2df8ff2740 1 -- 172.21.15.83:0/2927164145 shutdown_connections 2024-04-23T14:56:22.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.790+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/2927164145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eecfa20 0x55c03eed1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.790+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/2927164145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec5350 0x55c03eecf4e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.790+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/2927164145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec4a30 0x55c03eec4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.790+0000 7f2df8ff2740 1 -- 172.21.15.83:0/2927164145 >> 172.21.15.83:0/2927164145 conn(0x55c03eec0d50 msgr2=0x55c03eec1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:22.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.790+0000 7f2df8ff2740 1 -- 172.21.15.83:0/2927164145 shutdown_connections 2024-04-23T14:56:22.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.790+0000 7f2df8ff2740 1 -- 172.21.15.83:0/2927164145 wait complete. 2024-04-23T14:56:22.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.791+0000 7f2df8ff2740 1 -- start start 2024-04-23T14:56:22.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.791+0000 7f2defe00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2dd00a3d20 0x7f2dd00a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2de8004af0 tx=0x7f2de8005d80 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:22.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.791+0000 7f2df8ff2740 1 Processor -- start 2024-04-23T14:56:22.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.791+0000 7f2df8ff2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eec4a30 0x55c03eecc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2df8ff2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 0x55c03eecceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2df8ff2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eecfa20 0x55c03eecd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2df8ff2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c03eed2810 con 0x55c03eec5350 2024-04-23T14:56:22.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2df8ff2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c03ede3690 con 0x55c03eecfa20 2024-04-23T14:56:22.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2df8ff2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c03eed2990 con 0x55c03eec4a30 2024-04-23T14:56:22.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2df8ff2740 10 client.0 _ll_register_callbacks cb 0x55c03ee1f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:22.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2de6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 0x55c03eecceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2def400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eec4a30 0x55c03eecc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2de6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 0x55c03eecceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33058/0 (socket says 172.21.15.83:33058) 2024-04-23T14:56:22.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2de6a00640 1 -- 172.21.15.83:0/3888639361 learned_addr learned my addr 172.21.15.83:0/3888639361 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:22.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2defe00640 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eecfa20 0x55c03eecd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2de6a00640 1 -- 172.21.15.83:0/3888639361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eec4a30 msgr2=0x55c03eecc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:22.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2de6a00640 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eec4a30 0x55c03eecc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2de6a00640 1 -- 172.21.15.83:0/3888639361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eecfa20 msgr2=0x55c03eecd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:22.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2de6a00640 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eecfa20 0x55c03eecd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2de6a00640 1 -- 172.21.15.83:0/3888639361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c03ede2920 con 0x55c03eec5350 2024-04-23T14:56:22.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.792+0000 7f2def400640 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eec4a30 0x55c03eecc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:22.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.793+0000 7f2de6a00640 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 0x55c03eecceb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800e9c0 tx=0x7f2dd809c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c596073ae704f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:22.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.793+0000 7f2ddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:22.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.793+0000 7f2ddcc00640 1 -- 172.21.15.83:0/3888639361 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80a0df0 con 0x55c03eec5350 2024-04-23T14:56:22.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.793+0000 7f2ddcc00640 1 -- 172.21.15.83:0/3888639361 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f2dd80990b0 con 0x55c03eec5350 2024-04-23T14:56:22.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.793+0000 7f2df8ff2740 1 -- 172.21.15.83:0/3888639361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de0005020 con 0x55c03eec5350 2024-04-23T14:56:22.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.793+0000 7f2ddcc00640 1 -- 172.21.15.83:0/3888639361 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2dd80a56b0 con 0x55c03eec5350 2024-04-23T14:56:22.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.793+0000 7f2df8ff2740 10 client.16542 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:22.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.793+0000 7f2df8ff2740 1 -- 172.21.15.83:0/3888639361 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c03ec067d0 con 0x55c03eec5350 2024-04-23T14:56:22.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.793+0000 7f2ddcc00640 1 -- 172.21.15.83:0/3888639361 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de0005020 con 0x55c03eec5350 2024-04-23T14:56:22.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.793+0000 7f2ddcc00640 1 -- 172.21.15.83:0/3888639361 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f2dd809be90 con 0x55c03eec5350 2024-04-23T14:56:22.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.794+0000 7f2df8ff2740 10 client.16542 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:22.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.794+0000 7f2df8ff2740 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:22.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.794+0000 7f2df8ff2740 20 client.16542 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:22.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.794+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c03eedfcd0 0x55c03eee2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.794+0000 7f2df8ff2740 4 client.16542 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:56:22.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.794+0000 7f2df8ff2740 1 -- 172.21.15.83:0/3888639361 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c03ee37c10 con 0x55c03eedfcd0 2024-04-23T14:56:22.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.794+0000 7f2def400640 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c03eedfcd0 0x55c03eee2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.795+0000 7f2ddcc00640 10 client.16542 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:22.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.795+0000 7f2def400640 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c03eedfcd0 0x55c03eee2120 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae396025b8d5384 server_cookie=8c3d48e2e6059603 in_seq=0 out_seq=0 2024-04-23T14:56:22.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.795+0000 7f2ddcc00640 1 -- 172.21.15.83:0/3888639361 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c03ee37c10 con 0x55c03eedfcd0 2024-04-23T14:56:22.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.795+0000 7f2ddcc00640 10 client.16542 handle_command_reply: tid=0 2024-04-23T14:56:22.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.795+0000 7f2df8ff2740 1 client.16542 shutdown 2024-04-23T14:56:22.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.796+0000 7f2df8ff2740 1 -- 172.21.15.83:0/3888639361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 msgr2=0x55c03eecceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:22.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.796+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 0x55c03eecceb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800e9c0 tx=0x7f2dd809c640 comp rx=0 tx=0).stop 2024-04-23T14:56:22.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.796+0000 7f2df8ff2740 1 -- 172.21.15.83:0/3888639361 shutdown_connections 2024-04-23T14:56:22.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.796+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c03eedfcd0 0x55c03eee2120 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.796+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eecfa20 0x55c03eecd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.796+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 0x55c03eecceb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.796+0000 7f2ddcc00640 0 client.16542 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:22.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.796+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/3888639361 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eec4a30 0x55c03eecc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.796+0000 7f2df8ff2740 1 -- 172.21.15.83:0/3888639361 >> 172.21.15.83:0/3888639361 conn(0x55c03eec0d50 msgr2=0x55c03eebffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:22.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.796+0000 7f2df8ff2740 1 -- 172.21.15.83:0/3888639361 shutdown_connections 2024-04-23T14:56:22.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.796+0000 7f2df8ff2740 1 -- 172.21.15.83:0/3888639361 wait complete. 2024-04-23T14:56:22.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.796+0000 7f2df8ff2740 20 client.16542 trim_cache size 0 max 16384 2024-04-23T14:56:22.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2df8ff2740 1 Processor -- start 2024-04-23T14:56:22.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2df8ff2740 1 -- start start 2024-04-23T14:56:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2df8ff2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec4a30 0x55c03eedb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2df8ff2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 0x55c03eec71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2df8ff2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eecfa20 0x55c03eee7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2df8ff2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b6a60 con 0x55c03eec5350 2024-04-23T14:56:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2df8ff2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b4330 con 0x55c03eec4a30 2024-04-23T14:56:22.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2df8ff2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b21a0 con 0x55c03eecfa20 2024-04-23T14:56:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2de6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 0x55c03eec71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2de6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 0x55c03eec71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33064/0 (socket says 172.21.15.83:33064) 2024-04-23T14:56:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2de6a00640 1 -- 172.21.15.83:0/4154550417 learned_addr learned my addr 172.21.15.83:0/4154550417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2def400640 1 --2- 172.21.15.83:0/4154550417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec4a30 0x55c03eedb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.816+0000 7f2defe00640 1 --2- 172.21.15.83:0/4154550417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eecfa20 0x55c03eee7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.817+0000 7f2de6a00640 1 -- 172.21.15.83:0/4154550417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eecfa20 msgr2=0x55c03eee7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.817+0000 7f2de6a00640 1 --2- 172.21.15.83:0/4154550417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eecfa20 0x55c03eee7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.817+0000 7f2de6a00640 1 -- 172.21.15.83:0/4154550417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec4a30 msgr2=0x55c03eedb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:22.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.817+0000 7f2de6a00640 1 --2- 172.21.15.83:0/4154550417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec4a30 0x55c03eedb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.817+0000 7f2de6a00640 1 -- 172.21.15.83:0/4154550417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c03eb3c3f0 con 0x55c03eec5350 2024-04-23T14:56:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.817+0000 7f2de6a00640 1 --2- 172.21.15.83:0/4154550417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 0x55c03eec71d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2dd809c740 tx=0x7f2dd809c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab2c9bc74ea857f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.817+0000 7f2ddf400640 1 -- 172.21.15.83:0/4154550417 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80d4070 con 0x55c03eec5350 2024-04-23T14:56:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.817+0000 7f2ddf400640 1 -- 172.21.15.83:0/4154550417 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd80d8040 con 0x55c03eec5350 2024-04-23T14:56:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.817+0000 7f2ddf400640 1 -- 172.21.15.83:0/4154550417 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8003c50 con 0x55c03eec5350 2024-04-23T14:56:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.818+0000 7f2df8ff2740 1 -- 172.21.15.83:0/4154550417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 msgr2=0x55c03eec71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.818+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/4154550417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 0x55c03eec71d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2dd809c740 tx=0x7f2dd809c810 comp rx=0 tx=0).stop 2024-04-23T14:56:22.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.818+0000 7f2df8ff2740 1 -- 172.21.15.83:0/4154550417 shutdown_connections 2024-04-23T14:56:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.818+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/4154550417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eecfa20 0x55c03eee7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.818+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/4154550417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eec5350 0x55c03eec71d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.818+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/4154550417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec4a30 0x55c03eedb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.818+0000 7f2df8ff2740 1 -- 172.21.15.83:0/4154550417 >> 172.21.15.83:0/4154550417 conn(0x55c03eec0d50 msgr2=0x55c03eece1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.818+0000 7f2df8ff2740 1 -- 172.21.15.83:0/4154550417 shutdown_connections 2024-04-23T14:56:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.818+0000 7f2df8ff2740 1 -- 172.21.15.83:0/4154550417 wait complete. 2024-04-23T14:56:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.819+0000 7f2df8ff2740 1 -- start start 2024-04-23T14:56:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.819+0000 7f2df8ff2740 1 Processor -- start 2024-04-23T14:56:22.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.819+0000 7f2df8ff2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eec4a30 0x55c03eec94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.819+0000 7f2df8ff2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec5350 0x55c03eec9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.819+0000 7f2df8ff2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eecfa20 0x55c03eec9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.819+0000 7f2df8ff2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b27f0 con 0x55c03eecfa20 2024-04-23T14:56:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.819+0000 7f2df8ff2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b3c60 con 0x55c03eec5350 2024-04-23T14:56:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.819+0000 7f2df8ff2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b2ec0 con 0x55c03eec4a30 2024-04-23T14:56:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.819+0000 7f2df8ff2740 10 client.0 _ll_register_callbacks cb 0x7f2dd8003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.820+0000 7f2defe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eecfa20 0x55c03eec9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.820+0000 7f2defe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eecfa20 0x55c03eec9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33078/0 (socket says 172.21.15.83:33078) 2024-04-23T14:56:22.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.820+0000 7f2defe00640 1 -- 172.21.15.83:0/1342810735 learned_addr learned my addr 172.21.15.83:0/1342810735 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.820+0000 7f2de6a00640 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec5350 0x55c03eec9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.820+0000 7f2def400640 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eec4a30 0x55c03eec94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.820+0000 7f2defe00640 1 -- 172.21.15.83:0/1342810735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eec4a30 msgr2=0x55c03eec94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.820+0000 7f2defe00640 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eec4a30 0x55c03eec94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.820+0000 7f2defe00640 1 -- 172.21.15.83:0/1342810735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec5350 msgr2=0x55c03eec9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.820+0000 7f2defe00640 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec5350 0x55c03eec9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.820+0000 7f2defe00640 1 -- 172.21.15.83:0/1342810735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c03eafc0f0 con 0x55c03eecfa20 2024-04-23T14:56:22.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.820+0000 7f2defe00640 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eecfa20 0x55c03eec9f70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2de8049450 tx=0x7f2de802a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944262e453cde72c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.820+0000 7f2ddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.821+0000 7f2ddcc00640 1 -- 172.21.15.83:0/1342810735 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8058060 con 0x55c03eecfa20 2024-04-23T14:56:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.821+0000 7f2ddcc00640 1 -- 172.21.15.83:0/1342810735 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f2de8049980 con 0x55c03eecfa20 2024-04-23T14:56:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.821+0000 7f2df8ff2740 1 -- 172.21.15.83:0/1342810735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dd80d8040 con 0x55c03eecfa20 2024-04-23T14:56:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.821+0000 7f2ddcc00640 1 -- 172.21.15.83:0/1342810735 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2de8049b80 con 0x55c03eecfa20 2024-04-23T14:56:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.821+0000 7f2df8ff2740 10 client.16548 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.821+0000 7f2df8ff2740 1 -- 172.21.15.83:0/1342810735 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c03edef810 con 0x55c03eecfa20 2024-04-23T14:56:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.821+0000 7f2ddcc00640 1 -- 172.21.15.83:0/1342810735 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd80d8040 con 0x55c03eecfa20 2024-04-23T14:56:22.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.821+0000 7f2ddcc00640 1 -- 172.21.15.83:0/1342810735 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f2de8047400 con 0x55c03eecfa20 2024-04-23T14:56:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.821+0000 7f2df8ff2740 10 client.16548 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.822+0000 7f2df8ff2740 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.822+0000 7f2df8ff2740 20 client.16548 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.822+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c03ef46820 0x55c03ef48c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.822+0000 7f2df8ff2740 4 client.16548 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:56:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.822+0000 7f2df8ff2740 1 -- 172.21.15.83:0/1342810735 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c03ee31a80 con 0x55c03ef46820 2024-04-23T14:56:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.822+0000 7f2def400640 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c03ef46820 0x55c03ef48c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.822+0000 7f2def400640 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c03ef46820 0x55c03ef48c40 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7872d7bd1599d76d server_cookie=43aa5d0c3cdd5579 in_seq=0 out_seq=0 2024-04-23T14:56:22.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.822+0000 7f2ddcc00640 10 client.16548 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2ddcc00640 1 -- 172.21.15.83:0/1342810735 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c03ee31a80 con 0x55c03ef46820 2024-04-23T14:56:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2ddcc00640 10 client.16548 handle_command_reply: tid=0 2024-04-23T14:56:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2df8ff2740 1 client.16548 shutdown 2024-04-23T14:56:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2df8ff2740 1 -- 172.21.15.83:0/1342810735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eecfa20 msgr2=0x55c03eec9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eecfa20 0x55c03eec9f70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2de8049450 tx=0x7f2de802a8f0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2df8ff2740 1 -- 172.21.15.83:0/1342810735 shutdown_connections 2024-04-23T14:56:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55c03ef46820 0x55c03ef48c40 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c03eecfa20 0x55c03eec9f70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c03eec5350 0x55c03eec9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2ddcc00640 0 client.16548 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2df8ff2740 1 --2- 172.21.15.83:0/1342810735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c03eec4a30 0x55c03eec94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2df8ff2740 1 -- 172.21.15.83:0/1342810735 >> 172.21.15.83:0/1342810735 conn(0x55c03eec0d50 msgr2=0x55c03eecea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2df8ff2740 1 -- 172.21.15.83:0/1342810735 shutdown_connections 2024-04-23T14:56:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2df8ff2740 1 -- 172.21.15.83:0/1342810735 wait complete. 2024-04-23T14:56:22.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.823+0000 7f2df8ff2740 20 client.16548 trim_cache size 0 max 16384 2024-04-23T14:56:22.851 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:56:22.851 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:56:22.851 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:56:22.851 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:56:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.824+0000 7f2df4c00640 1 -- 172.21.15.83:0/3990978301 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2dd00a3d20 msgr2=0x7f2dd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:22.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.824+0000 7f2df4c00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2dd00a3d20 0x7f2dd00a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2de8004af0 tx=0x7f2de8005d80 comp rx=0 tx=0).stop 2024-04-23T14:56:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.824+0000 7f2df4c00640 1 -- 172.21.15.83:0/3990978301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2df0108f40 msgr2=0x7f2df01a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.824+0000 7f2df4c00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2df0108f40 0x7f2df01a4e60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800e9f0 tx=0x7f2dd800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.824+0000 7f2df4c00640 1 -- 172.21.15.83:0/3990978301 shutdown_connections 2024-04-23T14:56:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.824+0000 7f2df4c00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f2dd00a3d20 0x7f2dd00a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.824+0000 7f2df4c00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2df010d500 0x7f2df019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.824+0000 7f2df4c00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2df0108f40 0x7f2df01a4e60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.824+0000 7f2df4c00640 1 --2- 172.21.15.83:0/3990978301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2df0108580 0x7f2df01a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:22.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.824+0000 7f2df4c00640 1 -- 172.21.15.83:0/3990978301 >> 172.21.15.83:0/3990978301 conn(0x7f2df0077e20 msgr2=0x7f2df0078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.824+0000 7f2df4c00640 1 -- 172.21.15.83:0/3990978301 shutdown_connections 2024-04-23T14:56:22.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:22.824+0000 7f2df4c00640 1 -- 172.21.15.83:0/3990978301 wait complete. 2024-04-23T14:56:22.854 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:56:22.854 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:56:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.055+0000 7fd109c00640 1 -- 172.21.15.83:0/4060473924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd104104440 msgr2=0x7fd104104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.059 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.055+0000 7fd109c00640 1 --2- 172.21.15.83:0/4060473924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd104104440 0x7fd104104820 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009900 tx=0x7fd0f402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.055+0000 7fd109c00640 1 -- 172.21.15.83:0/4060473924 shutdown_connections 2024-04-23T14:56:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.055+0000 7fd109c00640 1 --2- 172.21.15.83:0/4060473924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd104109510 0x7fd1041110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.055+0000 7fd109c00640 1 --2- 172.21.15.83:0/4060473924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd104104df0 0x7fd104108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.060 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.055+0000 7fd109c00640 1 --2- 172.21.15.83:0/4060473924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd104104440 0x7fd104104820 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.055+0000 7fd109c00640 1 -- 172.21.15.83:0/4060473924 >> 172.21.15.83:0/4060473924 conn(0x7fd1040ffcc0 msgr2=0x7fd1041020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.055+0000 7fd109c00640 1 -- 172.21.15.83:0/4060473924 shutdown_connections 2024-04-23T14:56:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.055+0000 7fd109c00640 1 -- 172.21.15.83:0/4060473924 wait complete. 2024-04-23T14:56:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.056+0000 7fd109c00640 1 Processor -- start 2024-04-23T14:56:23.061 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.056+0000 7fd109c00640 1 -- start start 2024-04-23T14:56:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.056+0000 7fd109c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd104104440 0x7fd1041a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.056+0000 7fd109c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd104109510 0x7fd1041a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.056+0000 7fd109c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd10419a930 0x7fd10419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.056+0000 7fd109c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd104114610 con 0x7fd104109510 2024-04-23T14:56:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.056+0000 7fd109c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fd104114490 con 0x7fd104104440 2024-04-23T14:56:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.056+0000 7fd109c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fd104114790 con 0x7fd10419a930 2024-04-23T14:56:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.057+0000 7fd102000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd104109510 0x7fd1041a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.057+0000 7fd103400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd10419a930 0x7fd10419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.057+0000 7fd102000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd104109510 0x7fd1041a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33098/0 (socket says 172.21.15.83:33098) 2024-04-23T14:56:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.057+0000 7fd102000640 1 -- 172.21.15.83:0/2785055058 learned_addr learned my addr 172.21.15.83:0/2785055058 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.057+0000 7fd102a00640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd104104440 0x7fd1041a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.057+0000 7fd102000640 1 -- 172.21.15.83:0/2785055058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd10419a930 msgr2=0x7fd10419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.057+0000 7fd102000640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd10419a930 0x7fd10419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.057+0000 7fd102000640 1 -- 172.21.15.83:0/2785055058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd104104440 msgr2=0x7fd1041a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.057+0000 7fd102000640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd104104440 0x7fd1041a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.057+0000 7fd102000640 1 -- 172.21.15.83:0/2785055058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10419b450 con 0x7fd104109510 2024-04-23T14:56:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.057+0000 7fd102000640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd104109510 0x7fd1041a0cc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000e9f0 tx=0x7fd0f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.058+0000 7fd0fb400640 1 -- 172.21.15.83:0/2785055058 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f000ccc0 con 0x7fd104109510 2024-04-23T14:56:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.058+0000 7fd0fb400640 1 -- 172.21.15.83:0/2785055058 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f0005500 con 0x7fd104109510 2024-04-23T14:56:23.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.058+0000 7fd0fb400640 1 -- 172.21.15.83:0/2785055058 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0010690 con 0x7fd104109510 2024-04-23T14:56:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.058+0000 7fd109c00640 1 -- 172.21.15.83:0/2785055058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10419b740 con 0x7fd104109510 2024-04-23T14:56:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.058+0000 7fd109c00640 1 -- 172.21.15.83:0/2785055058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1041a7940 con 0x7fd104109510 2024-04-23T14:56:23.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.059+0000 7fd109c00640 1 -- 172.21.15.83:0/2785055058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd104069270 con 0x7fd104109510 2024-04-23T14:56:23.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.062+0000 7fd0fb400640 1 -- 172.21.15.83:0/2785055058 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd0f00046c0 con 0x7fd104109510 2024-04-23T14:56:23.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.063+0000 7fd0fb400640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0e80a3d20 0x7fd0e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.063+0000 7fd0fb400640 1 -- 172.21.15.83:0/2785055058 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fd0f00d0bc0 con 0x7fd104109510 2024-04-23T14:56:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.063+0000 7fd102a00640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0e80a3d20 0x7fd0e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.064+0000 7fd102a00640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0e80a3d20 0x7fd0e80a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4006010 tx=0x7fd0f4000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.068+0000 7fd0fb400640 1 -- 172.21.15.83:0/2785055058 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd0f009d1f0 con 0x7fd104109510 2024-04-23T14:56:23.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.311+0000 7fd109c00640 1 -- 172.21.15.83:0/2785055058 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd104104df0 con 0x7fd104109510 2024-04-23T14:56:23.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.313+0000 7fd0fb400640 1 -- 172.21.15.83:0/2785055058 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7fd0f009caf0 con 0x7fd104109510 2024-04-23T14:56:23.313 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:56:23.313 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:56:23.313 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:56:23.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.316+0000 7fd109c00640 1 -- 172.21.15.83:0/2785055058 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0e80a3d20 msgr2=0x7fd0e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.316+0000 7fd109c00640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0e80a3d20 0x7fd0e80a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4006010 tx=0x7fd0f4000bc0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.316+0000 7fd109c00640 1 -- 172.21.15.83:0/2785055058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd104109510 msgr2=0x7fd1041a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.316+0000 7fd109c00640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd104109510 0x7fd1041a0cc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000e9f0 tx=0x7fd0f000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.316+0000 7fd109c00640 1 -- 172.21.15.83:0/2785055058 shutdown_connections 2024-04-23T14:56:23.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.316+0000 7fd109c00640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd10419a930 0x7fd10419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.316+0000 7fd109c00640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fd0e80a3d20 0x7fd0e80a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.316+0000 7fd109c00640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd104109510 0x7fd1041a0cc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.316+0000 7fd109c00640 1 --2- 172.21.15.83:0/2785055058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd104104440 0x7fd1041a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.316+0000 7fd109c00640 1 -- 172.21.15.83:0/2785055058 >> 172.21.15.83:0/2785055058 conn(0x7fd1040ffcc0 msgr2=0x7fd104102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.316+0000 7fd109c00640 1 -- 172.21.15.83:0/2785055058 shutdown_connections 2024-04-23T14:56:23.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.316+0000 7fd109c00640 1 -- 172.21.15.83:0/2785055058 wait complete. 2024-04-23T14:56:23.336 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:56:23.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.535+0000 7fab0ea00640 1 -- 172.21.15.83:0/1344703100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab08108610 msgr2=0x7fab08108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.535+0000 7fab0ea00640 1 --2- 172.21.15.83:0/1344703100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab08108610 0x7fab08108a90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009860 tx=0x7faaf002a340 comp rx=0 tx=0).stop 2024-04-23T14:56:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.535+0000 7fab0ea00640 1 -- 172.21.15.83:0/1344703100 shutdown_connections 2024-04-23T14:56:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.535+0000 7fab0ea00640 1 --2- 172.21.15.83:0/1344703100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab08108fd0 0x7fab08115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.535+0000 7fab0ea00640 1 --2- 172.21.15.83:0/1344703100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab08108610 0x7fab08108a90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.535+0000 7fab0ea00640 1 --2- 172.21.15.83:0/1344703100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab0810e660 0x7fab0810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.535+0000 7fab0ea00640 1 -- 172.21.15.83:0/1344703100 >> 172.21.15.83:0/1344703100 conn(0x7fab08076540 msgr2=0x7fab08078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.535+0000 7fab0ea00640 1 -- 172.21.15.83:0/1344703100 shutdown_connections 2024-04-23T14:56:23.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.535+0000 7fab0ea00640 1 -- 172.21.15.83:0/1344703100 wait complete. 2024-04-23T14:56:23.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.536+0000 7fab0ea00640 1 Processor -- start 2024-04-23T14:56:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.536+0000 7fab0ea00640 1 -- start start 2024-04-23T14:56:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab0ea00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab08108610 0x7fab0819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab0ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab08108fd0 0x7fab0819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab0ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab0810e660 0x7fab081a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab0ea00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab081189e0 con 0x7fab08108fd0 2024-04-23T14:56:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab0ea00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fab08118860 con 0x7fab0810e660 2024-04-23T14:56:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab0ea00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fab08118b60 con 0x7fab08108610 2024-04-23T14:56:23.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7faaff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab08108fd0 0x7fab0819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab07e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab08108610 0x7fab0819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab0ca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab0810e660 0x7fab081a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab07e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab08108610 0x7fab0819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:60064/0 (socket says 172.21.15.83:60064) 2024-04-23T14:56:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab07e00640 1 -- 172.21.15.83:0/1189030515 learned_addr learned my addr 172.21.15.83:0/1189030515 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:23.541 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab07e00640 1 -- 172.21.15.83:0/1189030515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab0810e660 msgr2=0x7fab081a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab07e00640 1 --2- 172.21.15.83:0/1189030515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab0810e660 0x7fab081a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab07e00640 1 -- 172.21.15.83:0/1189030515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab08108fd0 msgr2=0x7fab0819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab07e00640 1 --2- 172.21.15.83:0/1189030515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab08108fd0 0x7fab0819f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.537+0000 7fab07e00640 1 -- 172.21.15.83:0/1189030515 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab081a3e10 con 0x7fab08108610 2024-04-23T14:56:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.538+0000 7fab07e00640 1 --2- 172.21.15.83:0/1189030515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab08108610 0x7fab0819ee20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faaf800e9a0 tx=0x7faaf800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.538+0000 7fab05600640 1 -- 172.21.15.83:0/1189030515 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf800cd80 con 0x7fab08108610 2024-04-23T14:56:23.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.538+0000 7fab05600640 1 -- 172.21.15.83:0/1189030515 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaf8010910 con 0x7fab08108610 2024-04-23T14:56:23.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.538+0000 7fab0ea00640 1 -- 172.21.15.83:0/1189030515 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab081a4100 con 0x7fab08108610 2024-04-23T14:56:23.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.538+0000 7fab05600640 1 -- 172.21.15.83:0/1189030515 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8018a30 con 0x7fab08108610 2024-04-23T14:56:23.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.538+0000 7fab0ea00640 1 -- 172.21.15.83:0/1189030515 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab081ab990 con 0x7fab08108610 2024-04-23T14:56:23.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.541+0000 7fab117bf740 1 Processor -- start 2024-04-23T14:56:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.541+0000 7fab117bf740 1 -- start start 2024-04-23T14:56:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.541+0000 7fab05600640 1 -- 172.21.15.83:0/1189030515 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faaf8010ac0 con 0x7fab08108610 2024-04-23T14:56:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab117bf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225f4950 0x5645225f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab117bf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x5645225ff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab117bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225ff940 0x564522601d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab117bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56452255a1d0 con 0x5645225ff940 2024-04-23T14:56:23.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab117bf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564522500450 con 0x5645225f5270 2024-04-23T14:56:23.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab117bf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564522556190 con 0x5645225f4950 2024-04-23T14:56:23.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab0ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225f4950 0x5645225f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab0ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225f4950 0x5645225f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:60076/0 (socket says 172.21.15.83:60076) 2024-04-23T14:56:23.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab0ca00640 1 -- 172.21.15.83:0/3172975580 learned_addr learned my addr 172.21.15.83:0/3172975580 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:23.552 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab07e00640 1 --2- 172.21.15.83:0/3172975580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x5645225ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7faaff400640 1 --2- 172.21.15.83:0/3172975580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225ff940 0x564522601d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab0ca00640 1 -- 172.21.15.83:0/3172975580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 msgr2=0x5645225ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab0ca00640 1 --2- 172.21.15.83:0/3172975580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x5645225ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab0ca00640 1 -- 172.21.15.83:0/3172975580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225ff940 msgr2=0x564522601d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab0ca00640 1 --2- 172.21.15.83:0/3172975580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225ff940 0x564522601d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.542+0000 7fab0ca00640 1 -- 172.21.15.83:0/3172975580 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645225bd310 con 0x5645225f4950 2024-04-23T14:56:23.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.543+0000 7fab0ca00640 1 --2- 172.21.15.83:0/3172975580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225f4950 0x5645225f4d30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fab00009fd0 tx=0x7fab000118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b034b43ee77b1925 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.543+0000 7faafd200640 1 -- 172.21.15.83:0/3172975580 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00004d20 con 0x5645225f4950 2024-04-23T14:56:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.543+0000 7faafd200640 1 -- 172.21.15.83:0/3172975580 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab00004050 con 0x5645225f4950 2024-04-23T14:56:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.543+0000 7faafd200640 1 -- 172.21.15.83:0/3172975580 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00005280 con 0x5645225f4950 2024-04-23T14:56:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.543+0000 7fab05600640 1 --2- 172.21.15.83:0/1189030515 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faaec0a3d20 0x7faaec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.544+0000 7fab117bf740 1 -- 172.21.15.83:0/3172975580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225f4950 msgr2=0x5645225f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.544+0000 7fab117bf740 1 --2- 172.21.15.83:0/3172975580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225f4950 0x5645225f4d30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fab00009fd0 tx=0x7fab000118d0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.544+0000 7fab117bf740 1 -- 172.21.15.83:0/3172975580 shutdown_connections 2024-04-23T14:56:23.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.544+0000 7fab117bf740 1 --2- 172.21.15.83:0/3172975580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225ff940 0x564522601d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.544+0000 7fab05600640 1 -- 172.21.15.83:0/1189030515 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7faaf80d0ba0 con 0x7fab08108610 2024-04-23T14:56:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.544+0000 7fab117bf740 1 --2- 172.21.15.83:0/3172975580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x5645225ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.544+0000 7fab117bf740 1 --2- 172.21.15.83:0/3172975580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225f4950 0x5645225f4d30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.544+0000 7fab117bf740 1 -- 172.21.15.83:0/3172975580 >> 172.21.15.83:0/3172975580 conn(0x5645225f0c70 msgr2=0x5645225f1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.544+0000 7fab07e00640 1 --2- 172.21.15.83:0/1189030515 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faaec0a3d20 0x7faaec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.544+0000 7fab117bf740 1 -- 172.21.15.83:0/3172975580 shutdown_connections 2024-04-23T14:56:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.544+0000 7fab117bf740 1 -- 172.21.15.83:0/3172975580 wait complete. 2024-04-23T14:56:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.545+0000 7fab117bf740 1 -- start start 2024-04-23T14:56:23.555 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.545+0000 7fab07e00640 1 --2- 172.21.15.83:0/1189030515 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faaec0a3d20 0x7faaec0a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5645225f5f10 tx=0x7faaf80d4650 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.545+0000 7fab117bf740 1 Processor -- start 2024-04-23T14:56:23.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.545+0000 7fab117bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x5645225ff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.545+0000 7fab117bf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x5645225fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.545+0000 7fab117bf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225ff940 0x5645225fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.545+0000 7fab117bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5645226027d0 con 0x5645225f4950 2024-04-23T14:56:23.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.545+0000 7fab117bf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x564522513810 con 0x5645225f5270 2024-04-23T14:56:23.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.545+0000 7fab117bf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x564522602950 con 0x5645225ff940 2024-04-23T14:56:23.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.546+0000 7fab117bf740 10 client.0 _ll_register_callbacks cb 0x5645224e1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:23.556 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.546+0000 7fab0ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x5645225ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.546+0000 7fab0ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x5645225ff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33122/0 (socket says 172.21.15.83:33122) 2024-04-23T14:56:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.546+0000 7fab0ca00640 1 -- 172.21.15.83:0/2239137379 learned_addr learned my addr 172.21.15.83:0/2239137379 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.546+0000 7faaff400640 1 --2- 172.21.15.83:0/2239137379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x5645225fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.546+0000 7fab0ca00640 1 -- 172.21.15.83:0/2239137379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225ff940 msgr2=0x5645225fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.546+0000 7fab0ca00640 1 --2- 172.21.15.83:0/2239137379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225ff940 0x5645225fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.546+0000 7fab0ca00640 1 -- 172.21.15.83:0/2239137379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 msgr2=0x5645225fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.546+0000 7fab0ca00640 1 --2- 172.21.15.83:0/2239137379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x5645225fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.557 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.546+0000 7fab0ca00640 1 -- 172.21.15.83:0/2239137379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645225243f0 con 0x5645225f4950 2024-04-23T14:56:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.546+0000 7fab0ca00640 1 --2- 172.21.15.83:0/2239137379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x5645225ff390 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fab00011dd0 tx=0x7fab00004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed4d96894c95db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.546+0000 7faaf5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.547+0000 7faaf5600640 1 -- 172.21.15.83:0/2239137379 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00029020 con 0x5645225f4950 2024-04-23T14:56:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.547+0000 7faaf5600640 1 -- 172.21.15.83:0/2239137379 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fab0000f4e0 con 0x5645225f4950 2024-04-23T14:56:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.547+0000 7fab117bf740 1 -- 172.21.15.83:0/2239137379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab00004050 con 0x5645225f4950 2024-04-23T14:56:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.547+0000 7faaf5600640 1 -- 172.21.15.83:0/2239137379 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab00034050 con 0x5645225f4950 2024-04-23T14:56:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.547+0000 7fab117bf740 10 client.16566 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.547+0000 7fab117bf740 1 -- 172.21.15.83:0/2239137379 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56452226c3f0 con 0x5645225f4950 2024-04-23T14:56:23.558 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.547+0000 7faaf5600640 1 -- 172.21.15.83:0/2239137379 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab00004050 con 0x5645225f4950 2024-04-23T14:56:23.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.547+0000 7faaf5600640 1 -- 172.21.15.83:0/2239137379 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fab00025600 con 0x5645225f4950 2024-04-23T14:56:23.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.547+0000 7fab117bf740 10 client.16566 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:23.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.547+0000 7fab117bf740 10 client.16566 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:56:23.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.547+0000 7fab117bf740 20 client.16566 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:23.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.548+0000 7fab117bf740 1 --2- 172.21.15.83:0/2239137379 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x56452260fb10 0x564522611f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.548+0000 7fab117bf740 4 client.16566 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:56:23.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.548+0000 7fab117bf740 1 -- 172.21.15.83:0/2239137379 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564522567bf0 con 0x56452260fb10 2024-04-23T14:56:23.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.548+0000 7faaff400640 1 --2- 172.21.15.83:0/2239137379 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x56452260fb10 0x564522611f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.559 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.549+0000 7faaff400640 1 --2- 172.21.15.83:0/2239137379 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x56452260fb10 0x564522611f00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4faf6f624eb6f24f server_cookie=6d60612b6f971183 in_seq=0 out_seq=0 2024-04-23T14:56:23.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.549+0000 7faaf5600640 10 client.16566 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:56:23.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.549+0000 7faaf5600640 1 -- 172.21.15.83:0/2239137379 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564522567bf0 con 0x56452260fb10 2024-04-23T14:56:23.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.549+0000 7faaf5600640 10 client.16566 handle_command_reply: tid=0 2024-04-23T14:56:23.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.549+0000 7fab117bf740 1 client.16566 shutdown 2024-04-23T14:56:23.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.550+0000 7fab117bf740 1 -- 172.21.15.83:0/2239137379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 msgr2=0x5645225ff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.550+0000 7fab117bf740 1 --2- 172.21.15.83:0/2239137379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x5645225ff390 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fab00011dd0 tx=0x7fab00004460 comp rx=0 tx=0).stop 2024-04-23T14:56:23.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.550+0000 7fab117bf740 1 -- 172.21.15.83:0/2239137379 shutdown_connections 2024-04-23T14:56:23.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.550+0000 7fab117bf740 1 --2- 172.21.15.83:0/2239137379 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x56452260fb10 0x564522611f00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.560 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.550+0000 7fab117bf740 1 --2- 172.21.15.83:0/2239137379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225ff940 0x5645225fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.550+0000 7faaf5600640 0 client.16566 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:56:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.550+0000 7fab117bf740 1 --2- 172.21.15.83:0/2239137379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x5645225fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.550+0000 7fab117bf740 1 --2- 172.21.15.83:0/2239137379 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x5645225ff390 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.550+0000 7fab117bf740 1 -- 172.21.15.83:0/2239137379 >> 172.21.15.83:0/2239137379 conn(0x5645225f0c70 msgr2=0x5645225f6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.550+0000 7fab117bf740 1 -- 172.21.15.83:0/2239137379 shutdown_connections 2024-04-23T14:56:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.550+0000 7fab117bf740 1 -- 172.21.15.83:0/2239137379 wait complete. 2024-04-23T14:56:23.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.550+0000 7fab117bf740 20 client.16566 trim_cache size 0 max 16384 2024-04-23T14:56:23.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.564+0000 7fab117bf740 1 Processor -- start 2024-04-23T14:56:23.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.564+0000 7fab117bf740 1 -- start start 2024-04-23T14:56:23.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab117bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x56452260b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab117bf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225f5270 0x5645225feec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab117bf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225ff940 0x5645225ff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab117bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab081b6c20 con 0x5645225f4950 2024-04-23T14:56:23.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab117bf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fab081b44f0 con 0x5645225ff940 2024-04-23T14:56:23.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab117bf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fab081b2360 con 0x5645225f5270 2024-04-23T14:56:23.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab0ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x56452260b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab0ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x56452260b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33134/0 (socket says 172.21.15.83:33134) 2024-04-23T14:56:23.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab0ca00640 1 -- 172.21.15.83:0/378213112 learned_addr learned my addr 172.21.15.83:0/378213112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:23.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7faaff400640 1 --2- 172.21.15.83:0/378213112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225f5270 0x5645225feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab0ca00640 1 --2- 172.21.15.83:0/378213112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225ff940 0x5645225ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab0ca00640 1 -- 172.21.15.83:0/378213112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225f5270 msgr2=0x5645225feec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab0ca00640 1 --2- 172.21.15.83:0/378213112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225f5270 0x5645225feec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab0ca00640 1 -- 172.21.15.83:0/378213112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225ff940 msgr2=0x5645225ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab0ca00640 1 --2- 172.21.15.83:0/378213112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225ff940 0x5645225ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab0ca00640 1 -- 172.21.15.83:0/378213112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645223367d0 con 0x5645225f4950 2024-04-23T14:56:23.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.565+0000 7fab0ca00640 1 --2- 172.21.15.83:0/378213112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x56452260b690 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fab00002230 tx=0x7fab00009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8317f4ed27446585 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.566+0000 7faaf7e00640 1 -- 172.21.15.83:0/378213112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00029040 con 0x5645225f4950 2024-04-23T14:56:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.566+0000 7faaf7e00640 1 -- 172.21.15.83:0/378213112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab00025dd0 con 0x5645225f4950 2024-04-23T14:56:23.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.566+0000 7faaf7e00640 1 -- 172.21.15.83:0/378213112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0000f740 con 0x5645225f4950 2024-04-23T14:56:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.566+0000 7fab117bf740 1 -- 172.21.15.83:0/378213112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 msgr2=0x56452260b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.566+0000 7fab117bf740 1 --2- 172.21.15.83:0/378213112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x56452260b690 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fab00002230 tx=0x7fab00009480 comp rx=0 tx=0).stop 2024-04-23T14:56:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.566+0000 7fab117bf740 1 -- 172.21.15.83:0/378213112 shutdown_connections 2024-04-23T14:56:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.566+0000 7fab117bf740 1 --2- 172.21.15.83:0/378213112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225ff940 0x5645225ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.566+0000 7fab117bf740 1 --2- 172.21.15.83:0/378213112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225f5270 0x5645225feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.566+0000 7fab117bf740 1 --2- 172.21.15.83:0/378213112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x56452260b690 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.572 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.566+0000 7fab117bf740 1 -- 172.21.15.83:0/378213112 >> 172.21.15.83:0/378213112 conn(0x5645225f0c70 msgr2=0x5645225fb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.566+0000 7fab117bf740 1 -- 172.21.15.83:0/378213112 shutdown_connections 2024-04-23T14:56:23.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.566+0000 7fab117bf740 1 -- 172.21.15.83:0/378213112 wait complete. 2024-04-23T14:56:23.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.567+0000 7fab117bf740 1 -- start start 2024-04-23T14:56:23.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.567+0000 7fab117bf740 1 Processor -- start 2024-04-23T14:56:23.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.567+0000 7fab117bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x564522600800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.567+0000 7fab117bf740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x564522600d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.567+0000 7fab117bf740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225ff940 0x564522601280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.567+0000 7fab117bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab081b29b0 con 0x5645225f4950 2024-04-23T14:56:23.573 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.567+0000 7fab117bf740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fab081b3e20 con 0x5645225f5270 2024-04-23T14:56:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.567+0000 7fab117bf740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fab081b3080 con 0x5645225ff940 2024-04-23T14:56:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7fab117bf740 10 client.0 _ll_register_callbacks cb 0x7faaf8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7fab0ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x564522600800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7fab0ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x564522600800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33150/0 (socket says 172.21.15.83:33150) 2024-04-23T14:56:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7fab0ca00640 1 -- 172.21.15.83:0/3951231506 learned_addr learned my addr 172.21.15.83:0/3951231506 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7faaff400640 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x564522600d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7fab0ca00640 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225ff940 0x564522601280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7fab0ca00640 1 -- 172.21.15.83:0/3951231506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225ff940 msgr2=0x564522601280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7fab0ca00640 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225ff940 0x564522601280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7fab0ca00640 1 -- 172.21.15.83:0/3951231506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 msgr2=0x564522600d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7fab0ca00640 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x564522600d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7fab0ca00640 1 -- 172.21.15.83:0/3951231506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645226060e0 con 0x5645225f4950 2024-04-23T14:56:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7faaff400640 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x564522600d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7fab0ca00640 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x564522600800 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x5645225f05b0 tx=0x7fab00003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6882a6c6111d3aae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.568+0000 7faaf5600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:23.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7faaf5600640 1 -- 172.21.15.83:0/3951231506 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00029070 con 0x5645225f4950 2024-04-23T14:56:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7faaf5600640 1 -- 172.21.15.83:0/3951231506 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fab00004610 con 0x5645225f4950 2024-04-23T14:56:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7fab117bf740 1 -- 172.21.15.83:0/3951231506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab00025dd0 con 0x5645225f4950 2024-04-23T14:56:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7faaf5600640 1 -- 172.21.15.83:0/3951231506 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab0003f040 con 0x5645225f4950 2024-04-23T14:56:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7fab117bf740 10 client.16572 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7fab117bf740 1 -- 172.21.15.83:0/3951231506 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5645226763e0 con 0x5645225f4950 2024-04-23T14:56:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7faaf5600640 1 -- 172.21.15.83:0/3951231506 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab00025dd0 con 0x5645225f4950 2024-04-23T14:56:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7faaf5600640 1 -- 172.21.15.83:0/3951231506 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fab0004c070 con 0x5645225f4950 2024-04-23T14:56:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7fab117bf740 10 client.16572 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:23.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7fab117bf740 10 client.16572 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:56:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7fab117bf740 20 client.16572 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7fab117bf740 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x5645226769f0 0x564522678e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7fab117bf740 4 client.16572 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:56:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.569+0000 7fab117bf740 1 -- 172.21.15.83:0/3951231506 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564522561a60 con 0x5645226769f0 2024-04-23T14:56:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.570+0000 7faaff400640 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x5645226769f0 0x564522678e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.570+0000 7faaff400640 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x5645226769f0 0x564522678e10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13df483e0082f1f5 server_cookie=3c7ceaaf68346693 in_seq=0 out_seq=0 2024-04-23T14:56:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.570+0000 7faaf5600640 10 client.16572 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:56:23.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.570+0000 7faaf5600640 1 -- 172.21.15.83:0/3951231506 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564522561a60 con 0x5645226769f0 2024-04-23T14:56:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.570+0000 7faaf5600640 10 client.16572 handle_command_reply: tid=0 2024-04-23T14:56:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.570+0000 7fab117bf740 1 client.16572 shutdown 2024-04-23T14:56:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7fab117bf740 1 -- 172.21.15.83:0/3951231506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 msgr2=0x564522600800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7fab117bf740 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x564522600800 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x5645225f05b0 tx=0x7fab00003ee0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7fab117bf740 1 -- 172.21.15.83:0/3951231506 shutdown_connections 2024-04-23T14:56:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7fab117bf740 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x5645226769f0 0x564522678e10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7fab117bf740 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5645225ff940 0x564522601280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7faaf5600640 0 client.16572 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:56:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7fab117bf740 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5645225f5270 0x564522600d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7fab117bf740 1 --2- 172.21.15.83:0/3951231506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5645225f4950 0x564522600800 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7fab117bf740 1 -- 172.21.15.83:0/3951231506 >> 172.21.15.83:0/3951231506 conn(0x5645225f0c70 msgr2=0x5645225ef560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7fab117bf740 1 -- 172.21.15.83:0/3951231506 shutdown_connections 2024-04-23T14:56:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7fab117bf740 1 -- 172.21.15.83:0/3951231506 wait complete. 2024-04-23T14:56:23.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7fab117bf740 20 client.16572 trim_cache size 0 max 16384 2024-04-23T14:56:23.579 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:56:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.571+0000 7fab0ea00640 1 -- 172.21.15.83:0/1189030515 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faaec0a3d20 msgr2=0x7faaec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.572+0000 7fab0ea00640 1 --2- 172.21.15.83:0/1189030515 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faaec0a3d20 0x7faaec0a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x5645225f5f10 tx=0x7faaf80d4650 comp rx=0 tx=0).stop 2024-04-23T14:56:23.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.572+0000 7fab0ea00640 1 -- 172.21.15.83:0/1189030515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab08108610 msgr2=0x7fab0819ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.572+0000 7fab0ea00640 1 --2- 172.21.15.83:0/1189030515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab08108610 0x7fab0819ee20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faaf800e9a0 tx=0x7faaf800ee60 comp rx=0 tx=0).stop 2024-04-23T14:56:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.572+0000 7fab0ea00640 1 -- 172.21.15.83:0/1189030515 shutdown_connections 2024-04-23T14:56:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.572+0000 7fab0ea00640 1 --2- 172.21.15.83:0/1189030515 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab0810e660 0x7fab081a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.572+0000 7fab0ea00640 1 --2- 172.21.15.83:0/1189030515 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7faaec0a3d20 0x7faaec0a61e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.572+0000 7fab0ea00640 1 --2- 172.21.15.83:0/1189030515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab08108fd0 0x7fab0819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.572+0000 7fab0ea00640 1 --2- 172.21.15.83:0/1189030515 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab08108610 0x7fab0819ee20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.572+0000 7fab0ea00640 1 -- 172.21.15.83:0/1189030515 >> 172.21.15.83:0/1189030515 conn(0x7fab08076540 msgr2=0x7fab08077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.572+0000 7fab0ea00640 1 -- 172.21.15.83:0/1189030515 shutdown_connections 2024-04-23T14:56:23.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.572+0000 7fab0ea00640 1 -- 172.21.15.83:0/1189030515 wait complete. 2024-04-23T14:56:23.582 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:23 smithi173 ceph-mon[32443]: pgmap v245: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:56:23.586 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:23 smithi083 ceph-mon[27867]: pgmap v245: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:56:23.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:56:23.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:23 smithi203 ceph-mon[32800]: pgmap v245: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:56:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.782+0000 7f9f47e00640 1 -- 172.21.15.83:0/3054674880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f4810d500 msgr2=0x7f9f4810d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.782+0000 7f9f47e00640 1 --2- 172.21.15.83:0/3054674880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f4810d500 0x7f9f4810d8e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009900 tx=0x7f9f3802a3f0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.782+0000 7f9f47e00640 1 -- 172.21.15.83:0/3054674880 shutdown_connections 2024-04-23T14:56:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.782+0000 7f9f47e00640 1 --2- 172.21.15.83:0/3054674880 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f48108f40 0x7f9f481153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.782+0000 7f9f47e00640 1 --2- 172.21.15.83:0/3054674880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f48108580 0x7f9f48108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.782+0000 7f9f47e00640 1 --2- 172.21.15.83:0/3054674880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f4810d500 0x7f9f4810d8e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.782+0000 7f9f47e00640 1 -- 172.21.15.83:0/3054674880 >> 172.21.15.83:0/3054674880 conn(0x7f9f48077e20 msgr2=0x7f9f48106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.782+0000 7f9f47e00640 1 -- 172.21.15.83:0/3054674880 shutdown_connections 2024-04-23T14:56:23.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.782+0000 7f9f47e00640 1 -- 172.21.15.83:0/3054674880 wait complete. 2024-04-23T14:56:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.783+0000 7f9f47e00640 1 Processor -- start 2024-04-23T14:56:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.783+0000 7f9f47e00640 1 -- start start 2024-04-23T14:56:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.783+0000 7f9f47e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f48108580 0x7f9f4819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.783+0000 7f9f47e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f48108f40 0x7f9f4819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.783+0000 7f9f47e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f4810d500 0x7f9f481a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.783+0000 7f9f47e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48118910 con 0x7f9f48108f40 2024-04-23T14:56:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.783+0000 7f9f47e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48118790 con 0x7f9f4810d500 2024-04-23T14:56:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.783+0000 7f9f47e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48118a90 con 0x7f9f48108580 2024-04-23T14:56:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.783+0000 7f9f3e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f48108f40 0x7f9f4819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.783+0000 7f9f3e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f48108f40 0x7f9f4819f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33178/0 (socket says 172.21.15.83:33178) 2024-04-23T14:56:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.783+0000 7f9f3e000640 1 -- 172.21.15.83:0/1270915002 learned_addr learned my addr 172.21.15.83:0/1270915002 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.783+0000 7f9f46a00640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f48108580 0x7f9f4819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.784+0000 7f9f47400640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f4810d500 0x7f9f481a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.784+0000 7f9f3e000640 1 -- 172.21.15.83:0/1270915002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f48108580 msgr2=0x7f9f4819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.784+0000 7f9f3e000640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f48108580 0x7f9f4819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.784+0000 7f9f3e000640 1 -- 172.21.15.83:0/1270915002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f4810d500 msgr2=0x7f9f481a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.784+0000 7f9f3e000640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f4810d500 0x7f9f481a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.784+0000 7f9f3e000640 1 -- 172.21.15.83:0/1270915002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f481a3d40 con 0x7f9f48108f40 2024-04-23T14:56:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.784+0000 7f9f3e000640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f48108f40 0x7f9f4819f290 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000eb90 tx=0x7f9f3000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.784+0000 7f9f3fe00640 1 -- 172.21.15.83:0/1270915002 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30004d20 con 0x7f9f48108f40 2024-04-23T14:56:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.784+0000 7f9f47e00640 1 -- 172.21.15.83:0/1270915002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f481a4030 con 0x7f9f48108f40 2024-04-23T14:56:23.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.784+0000 7f9f3fe00640 1 -- 172.21.15.83:0/1270915002 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3000ee90 con 0x7f9f48108f40 2024-04-23T14:56:23.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.784+0000 7f9f3fe00640 1 -- 172.21.15.83:0/1270915002 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30010830 con 0x7f9f48108f40 2024-04-23T14:56:23.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.785+0000 7f9f47e00640 1 -- 172.21.15.83:0/1270915002 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f481ab8c0 con 0x7f9f48108f40 2024-04-23T14:56:23.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.788+0000 7f9f503a1740 1 Processor -- start 2024-04-23T14:56:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.788+0000 7f9f3fe00640 1 -- 172.21.15.83:0/1270915002 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9f30005020 con 0x7f9f48108f40 2024-04-23T14:56:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.788+0000 7f9f503a1740 1 -- start start 2024-04-23T14:56:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.788+0000 7f9f503a1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b7b40 0x5575ff2b7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.788+0000 7f9f503a1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b8460 0x5575ff2c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.788+0000 7f9f503a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2c2b30 0x5575ff2c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.788+0000 7f9f503a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5575ff21d1d0 con 0x5575ff2c2b30 2024-04-23T14:56:23.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.788+0000 7f9f503a1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5575ff1c3450 con 0x5575ff2b8460 2024-04-23T14:56:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.788+0000 7f9f503a1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5575ff219190 con 0x5575ff2b7b40 2024-04-23T14:56:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.789+0000 7f9f47400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b7b40 0x5575ff2b7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.789+0000 7f9f3e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2c2b30 0x5575ff2c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.789+0000 7f9f46a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b8460 0x5575ff2c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.789+0000 7f9f47400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b7b40 0x5575ff2b7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:60122/0 (socket says 172.21.15.83:60122) 2024-04-23T14:56:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.789+0000 7f9f47400640 1 -- 172.21.15.83:0/2741322681 learned_addr learned my addr 172.21.15.83:0/2741322681 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.789+0000 7f9f47400640 1 -- 172.21.15.83:0/2741322681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b8460 msgr2=0x5575ff2c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.789+0000 7f9f47400640 1 --2- 172.21.15.83:0/2741322681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b8460 0x5575ff2c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.789+0000 7f9f47400640 1 -- 172.21.15.83:0/2741322681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2c2b30 msgr2=0x5575ff2c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.789+0000 7f9f47400640 1 --2- 172.21.15.83:0/2741322681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2c2b30 0x5575ff2c4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.789+0000 7f9f47400640 1 -- 172.21.15.83:0/2741322681 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575ff280310 con 0x5575ff2b7b40 2024-04-23T14:56:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.789+0000 7f9f3fe00640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9f2c0a3d20 0x7f9f2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.789+0000 7f9f47400640 1 --2- 172.21.15.83:0/2741322681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b7b40 0x5575ff2b7f20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009c90 tx=0x7f9f40011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d3fb34f4d2d4c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.790+0000 7f9f3fe00640 1 -- 172.21.15.83:0/1270915002 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f9f300d0e80 con 0x7f9f48108f40 2024-04-23T14:56:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.790+0000 7f9f46a00640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9f2c0a3d20 0x7f9f2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.790+0000 7f9f37400640 1 -- 172.21.15.83:0/2741322681 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4001e070 con 0x5575ff2b7b40 2024-04-23T14:56:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.790+0000 7f9f37400640 1 -- 172.21.15.83:0/2741322681 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f40009e40 con 0x5575ff2b7b40 2024-04-23T14:56:23.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.790+0000 7f9f37400640 1 -- 172.21.15.83:0/2741322681 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000ead0 con 0x5575ff2b7b40 2024-04-23T14:56:23.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.790+0000 7f9f503a1740 1 -- 172.21.15.83:0/2741322681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b7b40 msgr2=0x5575ff2b7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.790+0000 7f9f46a00640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9f2c0a3d20 0x7f9f2c0a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9f38006010 tx=0x7f9f38000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.790+0000 7f9f503a1740 1 --2- 172.21.15.83:0/2741322681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b7b40 0x5575ff2b7f20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009c90 tx=0x7f9f40011810 comp rx=0 tx=0).stop 2024-04-23T14:56:23.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.791+0000 7f9f503a1740 1 -- 172.21.15.83:0/2741322681 shutdown_connections 2024-04-23T14:56:23.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.791+0000 7f9f503a1740 1 --2- 172.21.15.83:0/2741322681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2c2b30 0x5575ff2c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.791+0000 7f9f503a1740 1 --2- 172.21.15.83:0/2741322681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b8460 0x5575ff2c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.791+0000 7f9f503a1740 1 --2- 172.21.15.83:0/2741322681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b7b40 0x5575ff2b7f20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.791+0000 7f9f503a1740 1 -- 172.21.15.83:0/2741322681 >> 172.21.15.83:0/2741322681 conn(0x5575ff2b3e60 msgr2=0x5575ff2b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.791+0000 7f9f503a1740 1 -- 172.21.15.83:0/2741322681 shutdown_connections 2024-04-23T14:56:23.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.791+0000 7f9f503a1740 1 -- 172.21.15.83:0/2741322681 wait complete. 2024-04-23T14:56:23.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.791+0000 7f9f503a1740 1 -- start start 2024-04-23T14:56:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.791+0000 7f9f503a1740 1 Processor -- start 2024-04-23T14:56:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f503a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 0x5575ff2c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f503a1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b8460 0x5575ff2bf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f503a1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2c2b30 0x5575ff2bffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f503a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5575ff2c59c0 con 0x5575ff2b7b40 2024-04-23T14:56:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f503a1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5575ff1d6810 con 0x5575ff2c2b30 2024-04-23T14:56:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f503a1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5575ff2c5b40 con 0x5575ff2b8460 2024-04-23T14:56:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f503a1740 10 client.0 _ll_register_callbacks cb 0x5575ff1a4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:23.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f3e000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b8460 0x5575ff2bf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f46a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 0x5575ff2c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f3e000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b8460 0x5575ff2bf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:60134/0 (socket says 172.21.15.83:60134) 2024-04-23T14:56:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f46a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 0x5575ff2c25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33200/0 (socket says 172.21.15.83:33200) 2024-04-23T14:56:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f3e000640 1 -- 172.21.15.83:0/2900694909 learned_addr learned my addr 172.21.15.83:0/2900694909 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f47400640 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2c2b30 0x5575ff2bffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f3e000640 1 -- 172.21.15.83:0/2900694909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2c2b30 msgr2=0x5575ff2bffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f3e000640 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2c2b30 0x5575ff2bffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f3e000640 1 -- 172.21.15.83:0/2900694909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 msgr2=0x5575ff2c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f3e000640 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 0x5575ff2c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.792+0000 7f9f3e000640 1 -- 172.21.15.83:0/2900694909 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575ff1e73f0 con 0x5575ff2b8460 2024-04-23T14:56:23.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.793+0000 7f9f47400640 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2c2b30 0x5575ff2bffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:23.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.793+0000 7f9f46a00640 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 0x5575ff2c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:23.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.793+0000 7f9f3e000640 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b8460 0x5575ff2bf8c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000cba0 tx=0x7f9f300d4520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56bdddc03b5cfab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.793+0000 7f9f2be00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:56:23.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.793+0000 7f9f2be00640 1 -- 172.21.15.83:0/2900694909 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3009cb80 con 0x5575ff2b8460 2024-04-23T14:56:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.793+0000 7f9f2be00640 1 -- 172.21.15.83:0/2900694909 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f9f300ce7e0 con 0x5575ff2b8460 2024-04-23T14:56:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.793+0000 7f9f503a1740 1 -- 172.21.15.83:0/2900694909 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575ff22abf0 con 0x5575ff2b8460 2024-04-23T14:56:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.793+0000 7f9f2be00640 1 -- 172.21.15.83:0/2900694909 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f3009c710 con 0x5575ff2b8460 2024-04-23T14:56:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.793+0000 7f9f503a1740 10 client.26761 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.793+0000 7f9f503a1740 1 -- 172.21.15.83:0/2900694909 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5575fef2f3f0 con 0x5575ff2b8460 2024-04-23T14:56:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.794+0000 7f9f2be00640 1 -- 172.21.15.83:0/2900694909 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5575ff22abf0 con 0x5575ff2b8460 2024-04-23T14:56:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.794+0000 7f9f2be00640 1 -- 172.21.15.83:0/2900694909 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f9f30087040 con 0x5575ff2b8460 2024-04-23T14:56:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.794+0000 7f9f503a1740 10 client.26761 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:23.802 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.794+0000 7f9f503a1740 10 client.26761 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:56:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.794+0000 7f9f503a1740 20 client.26761 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.794+0000 7f9f503a1740 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5575ff2d2dd0 0x5575ff2d51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.794+0000 7f9f503a1740 4 client.26761 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:56:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.794+0000 7f9f503a1740 1 -- 172.21.15.83:0/2900694909 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575ff224a60 con 0x5575ff2d2dd0 2024-04-23T14:56:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.794+0000 7f9f46a00640 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5575ff2d2dd0 0x5575ff2d51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.795+0000 7f9f46a00640 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5575ff2d2dd0 0x5575ff2d51c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28e546cf25831215 server_cookie=10301a304b4eb5c7 in_seq=0 out_seq=0 2024-04-23T14:56:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.795+0000 7f9f2be00640 10 client.26761 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:56:23.803 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f2be00640 1 -- 172.21.15.83:0/2900694909 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575ff224a60 con 0x5575ff2d2dd0 2024-04-23T14:56:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f2be00640 10 client.26761 handle_command_reply: tid=0 2024-04-23T14:56:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f503a1740 1 client.26761 shutdown 2024-04-23T14:56:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f503a1740 1 -- 172.21.15.83:0/2900694909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b8460 msgr2=0x5575ff2bf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f503a1740 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b8460 0x5575ff2bf8c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000cba0 tx=0x7f9f300d4520 comp rx=0 tx=0).stop 2024-04-23T14:56:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f503a1740 1 -- 172.21.15.83:0/2900694909 shutdown_connections 2024-04-23T14:56:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f503a1740 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2c2b30 0x5575ff2bffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f503a1740 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5575ff2d2dd0 0x5575ff2d51c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f503a1740 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b8460 0x5575ff2bf8c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f503a1740 1 --2- 172.21.15.83:0/2900694909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 0x5575ff2c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f503a1740 1 -- 172.21.15.83:0/2900694909 >> 172.21.15.83:0/2900694909 conn(0x5575ff2b3e60 msgr2=0x5575ff2b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f2be00640 0 client.26761 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:56:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f503a1740 1 -- 172.21.15.83:0/2900694909 shutdown_connections 2024-04-23T14:56:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.796+0000 7f9f503a1740 1 -- 172.21.15.83:0/2900694909 wait complete. 2024-04-23T14:56:23.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.797+0000 7f9f503a1740 20 client.26761 trim_cache size 0 max 16384 2024-04-23T14:56:23.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f503a1740 1 Processor -- start 2024-04-23T14:56:23.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f503a1740 1 -- start start 2024-04-23T14:56:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f503a1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b7b40 0x5575ff2ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f503a1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b8460 0x5575ff2b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f503a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2c2b30 0x5575ff2c2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f503a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481b6a50 con 0x5575ff2c2b30 2024-04-23T14:56:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f503a1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481b4320 con 0x5575ff2b7b40 2024-04-23T14:56:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f503a1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481b2190 con 0x5575ff2b8460 2024-04-23T14:56:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f47400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2c2b30 0x5575ff2c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f47400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2c2b30 0x5575ff2c2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33216/0 (socket says 172.21.15.83:33216) 2024-04-23T14:56:23.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f47400640 1 -- 172.21.15.83:0/1385496070 learned_addr learned my addr 172.21.15.83:0/1385496070 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f3e000640 1 --2- 172.21.15.83:0/1385496070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b8460 0x5575ff2b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.811+0000 7f9f46a00640 1 --2- 172.21.15.83:0/1385496070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b7b40 0x5575ff2ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.812+0000 7f9f47400640 1 -- 172.21.15.83:0/1385496070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b8460 msgr2=0x5575ff2b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.812+0000 7f9f47400640 1 --2- 172.21.15.83:0/1385496070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b8460 0x5575ff2b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.812+0000 7f9f47400640 1 -- 172.21.15.83:0/1385496070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b7b40 msgr2=0x5575ff2ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.812+0000 7f9f47400640 1 --2- 172.21.15.83:0/1385496070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b7b40 0x5575ff2ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.812+0000 7f9f47400640 1 -- 172.21.15.83:0/1385496070 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575ff2c0320 con 0x5575ff2c2b30 2024-04-23T14:56:23.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.812+0000 7f9f47400640 1 --2- 172.21.15.83:0/1385496070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2c2b30 0x5575ff2c2100 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009c30 tx=0x7f9f4000e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1182be1e7714e3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.812+0000 7f9f36a00640 1 -- 172.21.15.83:0/1385496070 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40029020 con 0x5575ff2c2b30 2024-04-23T14:56:23.819 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.812+0000 7f9f36a00640 1 -- 172.21.15.83:0/1385496070 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4001f040 con 0x5575ff2c2b30 2024-04-23T14:56:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.812+0000 7f9f36a00640 1 -- 172.21.15.83:0/1385496070 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4001e040 con 0x5575ff2c2b30 2024-04-23T14:56:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.813+0000 7f9f503a1740 1 -- 172.21.15.83:0/1385496070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2c2b30 msgr2=0x5575ff2c2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.813+0000 7f9f503a1740 1 --2- 172.21.15.83:0/1385496070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2c2b30 0x5575ff2c2100 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009c30 tx=0x7f9f4000e5f0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.813+0000 7f9f503a1740 1 -- 172.21.15.83:0/1385496070 shutdown_connections 2024-04-23T14:56:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.813+0000 7f9f503a1740 1 --2- 172.21.15.83:0/1385496070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2c2b30 0x5575ff2c2100 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.813+0000 7f9f503a1740 1 --2- 172.21.15.83:0/1385496070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2b8460 0x5575ff2b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.813+0000 7f9f503a1740 1 --2- 172.21.15.83:0/1385496070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b7b40 0x5575ff2ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.813+0000 7f9f503a1740 1 -- 172.21.15.83:0/1385496070 >> 172.21.15.83:0/1385496070 conn(0x5575ff2b3e60 msgr2=0x5575ff2be190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.813+0000 7f9f503a1740 1 -- 172.21.15.83:0/1385496070 shutdown_connections 2024-04-23T14:56:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.813+0000 7f9f503a1740 1 -- 172.21.15.83:0/1385496070 wait complete. 2024-04-23T14:56:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.813+0000 7f9f503a1740 1 -- start start 2024-04-23T14:56:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f503a1740 1 Processor -- start 2024-04-23T14:56:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f503a1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 0x5575ff2c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f503a1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b8460 0x5575ff2c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f503a1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2c2b30 0x5575ff2c3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f503a1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481b27e0 con 0x5575ff2b7b40 2024-04-23T14:56:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f503a1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481b3c50 con 0x5575ff2b8460 2024-04-23T14:56:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f503a1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481b2eb0 con 0x5575ff2c2b30 2024-04-23T14:56:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f503a1740 10 client.0 _ll_register_callbacks cb 0x7f9f30004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f46a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 0x5575ff2c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f46a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 0x5575ff2c2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33218/0 (socket says 172.21.15.83:33218) 2024-04-23T14:56:23.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f46a00640 1 -- 172.21.15.83:0/178380848 learned_addr learned my addr 172.21.15.83:0/178380848 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f47400640 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2c2b30 0x5575ff2c3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.814+0000 7f9f3e000640 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b8460 0x5575ff2c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.815+0000 7f9f47400640 1 -- 172.21.15.83:0/178380848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b8460 msgr2=0x5575ff2c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.815+0000 7f9f47400640 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b8460 0x5575ff2c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.815+0000 7f9f47400640 1 -- 172.21.15.83:0/178380848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 msgr2=0x5575ff2c2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.815+0000 7f9f47400640 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 0x5575ff2c2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.815+0000 7f9f47400640 1 -- 172.21.15.83:0/178380848 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575feff97d0 con 0x5575ff2c2b30 2024-04-23T14:56:23.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.815+0000 7f9f46a00640 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 0x5575ff2c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:23.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.815+0000 7f9f47400640 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2c2b30 0x5575ff2c3990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9f40011d10 tx=0x7f9f400095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1485a82d53b23c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:23.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.815+0000 7f9f2be00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:56:23.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.815+0000 7f9f2be00640 1 -- 172.21.15.83:0/178380848 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40029030 con 0x5575ff2c2b30 2024-04-23T14:56:23.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.815+0000 7f9f503a1740 1 -- 172.21.15.83:0/178380848 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4001f040 con 0x5575ff2c2b30 2024-04-23T14:56:23.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.815+0000 7f9f2be00640 1 -- 172.21.15.83:0/178380848 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f9f4000e660 con 0x5575ff2c2b30 2024-04-23T14:56:23.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.815+0000 7f9f2be00640 1 -- 172.21.15.83:0/178380848 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f4001e040 con 0x5575ff2c2b30 2024-04-23T14:56:23.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.816+0000 7f9f503a1740 10 client.26767 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:23.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.816+0000 7f9f503a1740 1 -- 172.21.15.83:0/178380848 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5575ff2cae90 con 0x5575ff2c2b30 2024-04-23T14:56:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.816+0000 7f9f2be00640 1 -- 172.21.15.83:0/178380848 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4001f040 con 0x5575ff2c2b30 2024-04-23T14:56:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.816+0000 7f9f2be00640 1 -- 172.21.15.83:0/178380848 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f9f40019430 con 0x5575ff2c2b30 2024-04-23T14:56:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.816+0000 7f9f503a1740 10 client.26767 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.816+0000 7f9f503a1740 10 client.26767 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:56:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.816+0000 7f9f503a1740 20 client.26767 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.816+0000 7f9f503a1740 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5575ff2cb3b0 0x5575ff33b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.816+0000 7f9f503a1740 4 client.26767 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:56:23.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.816+0000 7f9f503a1740 1 -- 172.21.15.83:0/178380848 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575ff1a04d0 con 0x5575ff2cb3b0 2024-04-23T14:56:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.816+0000 7f9f46a00640 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5575ff2cb3b0 0x5575ff33b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f46a00640 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5575ff2cb3b0 0x5575ff33b310 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5fbc88c1a67c36e1 server_cookie=afa16dc9dfdadbbc in_seq=0 out_seq=0 2024-04-23T14:56:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f2be00640 10 client.26767 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:56:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f2be00640 1 -- 172.21.15.83:0/178380848 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575ff1a04d0 con 0x5575ff2cb3b0 2024-04-23T14:56:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f2be00640 10 client.26767 handle_command_reply: tid=0 2024-04-23T14:56:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f503a1740 1 client.26767 shutdown 2024-04-23T14:56:23.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f503a1740 1 -- 172.21.15.83:0/178380848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2c2b30 msgr2=0x5575ff2c3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f503a1740 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2c2b30 0x5575ff2c3990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9f40011d10 tx=0x7f9f400095c0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f503a1740 1 -- 172.21.15.83:0/178380848 shutdown_connections 2024-04-23T14:56:23.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f503a1740 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5575ff2cb3b0 0x5575ff33b310 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f503a1740 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575ff2c2b30 0x5575ff2c3990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f503a1740 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5575ff2b8460 0x5575ff2c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f2be00640 0 client.26767 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:56:23.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f503a1740 1 --2- 172.21.15.83:0/178380848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5575ff2b7b40 0x5575ff2c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.817+0000 7f9f503a1740 1 -- 172.21.15.83:0/178380848 >> 172.21.15.83:0/178380848 conn(0x5575ff2b3e60 msgr2=0x5575ff2b9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f503a1740 1 -- 172.21.15.83:0/178380848 shutdown_connections 2024-04-23T14:56:23.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f503a1740 1 -- 172.21.15.83:0/178380848 wait complete. 2024-04-23T14:56:23.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f503a1740 20 client.26767 trim_cache size 0 max 16384 2024-04-23T14:56:23.830 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:56:23.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f47e00640 1 -- 172.21.15.83:0/1270915002 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9f2c0a3d20 msgr2=0x7f9f2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f47e00640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9f2c0a3d20 0x7f9f2c0a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9f38006010 tx=0x7f9f38000bc0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f47e00640 1 -- 172.21.15.83:0/1270915002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f48108f40 msgr2=0x7f9f4819f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f47e00640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f48108f40 0x7f9f4819f290 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000eb90 tx=0x7f9f3000c6a0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f47e00640 1 -- 172.21.15.83:0/1270915002 shutdown_connections 2024-04-23T14:56:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f47e00640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f4810d500 0x7f9f481a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f47e00640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9f2c0a3d20 0x7f9f2c0a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f47e00640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f48108f40 0x7f9f4819f290 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f47e00640 1 --2- 172.21.15.83:0/1270915002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f48108580 0x7f9f4819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f47e00640 1 -- 172.21.15.83:0/1270915002 >> 172.21.15.83:0/1270915002 conn(0x7f9f48077e20 msgr2=0x7f9f480782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:23.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f47e00640 1 -- 172.21.15.83:0/1270915002 shutdown_connections 2024-04-23T14:56:23.833 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:23.818+0000 7f9f47e00640 1 -- 172.21.15.83:0/1270915002 wait complete. 2024-04-23T14:56:23.836 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:56:24.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.032+0000 7f8eb2000640 1 -- 172.21.15.83:0/23294446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac108f40 msgr2=0x7f8eac1153b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.032+0000 7f8eb2000640 1 --2- 172.21.15.83:0/23294446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac108f40 0x7f8eac1153b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009cd0 tx=0x7f8ea402a8f0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.032+0000 7f8eb2000640 1 -- 172.21.15.83:0/23294446 shutdown_connections 2024-04-23T14:56:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.032+0000 7f8eb2000640 1 --2- 172.21.15.83:0/23294446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac108f40 0x7f8eac1153b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.032+0000 7f8eb2000640 1 --2- 172.21.15.83:0/23294446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eac108580 0x7f8eac108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.032+0000 7f8eb2000640 1 --2- 172.21.15.83:0/23294446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eac10d500 0x7f8eac10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.032+0000 7f8eb2000640 1 -- 172.21.15.83:0/23294446 >> 172.21.15.83:0/23294446 conn(0x7f8eac077e20 msgr2=0x7f8eac106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:24.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.032+0000 7f8eb2000640 1 -- 172.21.15.83:0/23294446 shutdown_connections 2024-04-23T14:56:24.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.032+0000 7f8eb2000640 1 -- 172.21.15.83:0/23294446 wait complete. 2024-04-23T14:56:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8eb2000640 1 Processor -- start 2024-04-23T14:56:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8eb2000640 1 -- start start 2024-04-23T14:56:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8eb2000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac108580 0x7f8eac19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8eb2000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eac108f40 0x7f8eac19f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8eb2000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eac10d500 0x7f8eac1a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8eb2000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac118900 con 0x7f8eac108f40 2024-04-23T14:56:24.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8eb2000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac118780 con 0x7f8eac10d500 2024-04-23T14:56:24.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8eb2000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac118a80 con 0x7f8eac108580 2024-04-23T14:56:24.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8eb1600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eac10d500 0x7f8eac1a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8ea3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eac108f40 0x7f8eac19f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8eb1600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eac10d500 0x7f8eac1a3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:34328/0 (socket says 172.21.15.83:34328) 2024-04-23T14:56:24.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8eb1600640 1 -- 172.21.15.83:0/3478020461 learned_addr learned my addr 172.21.15.83:0/3478020461 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:24.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.034+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac108580 0x7f8eac19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.035+0000 7f8ea3e00640 1 -- 172.21.15.83:0/3478020461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac108580 msgr2=0x7f8eac19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:24.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.035+0000 7f8ea3e00640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac108580 0x7f8eac19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.035+0000 7f8ea3e00640 1 -- 172.21.15.83:0/3478020461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eac10d500 msgr2=0x7f8eac1a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:24.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.035+0000 7f8ea3e00640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eac10d500 0x7f8eac1a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.035+0000 7f8ea3e00640 1 -- 172.21.15.83:0/3478020461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eac1a3d90 con 0x7f8eac108f40 2024-04-23T14:56:24.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.035+0000 7f8eb1600640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eac10d500 0x7f8eac1a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:24.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.035+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac108580 0x7f8eac19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.035+0000 7f8ea3e00640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eac108f40 0x7f8eac19f2e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400e9f0 tx=0x7f8e9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.036+0000 7f8eaa000640 1 -- 172.21.15.83:0/3478020461 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9400cd80 con 0x7f8eac108f40 2024-04-23T14:56:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.036+0000 7f8eaa000640 1 -- 172.21.15.83:0/3478020461 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e940035f0 con 0x7f8eac108f40 2024-04-23T14:56:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.036+0000 7f8eb2000640 1 -- 172.21.15.83:0/3478020461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eac1a4080 con 0x7f8eac108f40 2024-04-23T14:56:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.036+0000 7f8eaa000640 1 -- 172.21.15.83:0/3478020461 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94010670 con 0x7f8eac108f40 2024-04-23T14:56:24.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.036+0000 7f8eb2000640 1 -- 172.21.15.83:0/3478020461 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eac1ab910 con 0x7f8eac108f40 2024-04-23T14:56:24.040 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.039+0000 7f8eb656b740 1 Processor -- start 2024-04-23T14:56:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.039+0000 7f8eb656b740 1 -- start start 2024-04-23T14:56:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.040+0000 7f8eaa000640 1 -- 172.21.15.83:0/3478020461 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e940037a0 con 0x7f8eac108f40 2024-04-23T14:56:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.040+0000 7f8eb656b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c476f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.040+0000 7f8eb656b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 0x56211c4815f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.040+0000 7f8eb656b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c483f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.040+0000 7f8eb656b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56211c3dc1d0 con 0x56211c481b30 2024-04-23T14:56:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.040+0000 7f8eb656b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56211c382450 con 0x56211c476b40 2024-04-23T14:56:24.041 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.040+0000 7f8eb656b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56211c3d8190 con 0x56211c477460 2024-04-23T14:56:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.040+0000 7f8eb1600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c476f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.040+0000 7f8ea3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c483f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.040+0000 7f8eb1600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c476f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:34340/0 (socket says 172.21.15.83:34340) 2024-04-23T14:56:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.040+0000 7f8eb1600640 1 -- 172.21.15.83:0/981751894 learned_addr learned my addr 172.21.15.83:0/981751894 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.040+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/981751894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 0x56211c4815f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.041+0000 7f8eb1600640 1 -- 172.21.15.83:0/981751894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 msgr2=0x56211c4815f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.041+0000 7f8eb1600640 1 --2- 172.21.15.83:0/981751894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 0x56211c4815f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.042 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.041+0000 7f8eb1600640 1 -- 172.21.15.83:0/981751894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 msgr2=0x56211c483f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.041+0000 7f8eb1600640 1 --2- 172.21.15.83:0/981751894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c483f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.041+0000 7f8eb1600640 1 -- 172.21.15.83:0/981751894 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56211c43f310 con 0x56211c476b40 2024-04-23T14:56:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.041+0000 7f8ea3e00640 1 --2- 172.21.15.83:0/981751894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c483f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:24.043 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.041+0000 7f8eb1600640 1 --2- 172.21.15.83:0/981751894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c476f20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009420 tx=0x7f8ea4005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=679a4fd8e978d23e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.041+0000 7f8ea1c00640 1 -- 172.21.15.83:0/981751894 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4003c70 con 0x56211c476b40 2024-04-23T14:56:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.041+0000 7f8ea1c00640 1 -- 172.21.15.83:0/981751894 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea4036030 con 0x56211c476b40 2024-04-23T14:56:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.041+0000 7f8ea1c00640 1 -- 172.21.15.83:0/981751894 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4004830 con 0x56211c476b40 2024-04-23T14:56:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eaa000640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f8e900a3da0 0x7f8e900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eaa000640 1 -- 172.21.15.83:0/3478020461 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f8e940d0ad0 con 0x7f8eac108f40 2024-04-23T14:56:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f8e900a3da0 0x7f8e900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eb656b740 1 -- 172.21.15.83:0/981751894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 msgr2=0x56211c476f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eb656b740 1 --2- 172.21.15.83:0/981751894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c476f20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4009420 tx=0x7f8ea4005ef0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eb656b740 1 -- 172.21.15.83:0/981751894 shutdown_connections 2024-04-23T14:56:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eb656b740 1 --2- 172.21.15.83:0/981751894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c483f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eb656b740 1 --2- 172.21.15.83:0/981751894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 0x56211c4815f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eb656b740 1 --2- 172.21.15.83:0/981751894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c476f20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eb656b740 1 -- 172.21.15.83:0/981751894 >> 172.21.15.83:0/981751894 conn(0x56211c472e60 msgr2=0x56211c473240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eb656b740 1 -- 172.21.15.83:0/981751894 shutdown_connections 2024-04-23T14:56:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f8e900a3da0 0x7f8e900a6260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x56211c478100 tx=0x7f8e9c005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:24.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.042+0000 7f8eb656b740 1 -- 172.21.15.83:0/981751894 wait complete. 2024-04-23T14:56:24.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.043+0000 7f8eb656b740 1 -- start start 2024-04-23T14:56:24.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.043+0000 7f8eb656b740 1 Processor -- start 2024-04-23T14:56:24.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.043+0000 7f8eb656b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c476b40 0x56211c481560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb656b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c477460 0x56211c47e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb656b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c47ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb656b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56211c484960 con 0x56211c481b30 2024-04-23T14:56:24.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb656b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56211c395810 con 0x56211c477460 2024-04-23T14:56:24.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb656b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56211c484ae0 con 0x56211c476b40 2024-04-23T14:56:24.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb656b740 10 client.0 _ll_register_callbacks cb 0x56211c363140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c47ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8ea3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c477460 0x56211c47e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c47ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33266/0 (socket says 172.21.15.83:33266) 2024-04-23T14:56:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb1600640 1 -- 172.21.15.83:0/1343038183 learned_addr learned my addr 172.21.15.83:0/1343038183 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c476b40 0x56211c481560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb1600640 1 -- 172.21.15.83:0/1343038183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c476b40 msgr2=0x56211c481560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb1600640 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c476b40 0x56211c481560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb1600640 1 -- 172.21.15.83:0/1343038183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c477460 msgr2=0x56211c47e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:24.050 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb1600640 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c477460 0x56211c47e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8eb1600640 1 -- 172.21.15.83:0/1343038183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56211c3a63f0 con 0x56211c481b30 2024-04-23T14:56:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.044+0000 7f8ea3e00640 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c477460 0x56211c47e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.045+0000 7f8eb1600640 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c47ef90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4006010 tx=0x7f8ea402b030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249daa74c3cd8c08 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.045+0000 7f8e9a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.045+0000 7f8e9a000640 1 -- 172.21.15.83:0/1343038183 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4053020 con 0x56211c481b30 2024-04-23T14:56:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.045+0000 7f8e9a000640 1 -- 172.21.15.83:0/1343038183 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f8ea4003c80 con 0x56211c481b30 2024-04-23T14:56:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.045+0000 7f8eb656b740 1 -- 172.21.15.83:0/1343038183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea4036030 con 0x56211c481b30 2024-04-23T14:56:24.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.045+0000 7f8e9a000640 1 -- 172.21.15.83:0/1343038183 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ea405e050 con 0x56211c481b30 2024-04-23T14:56:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.045+0000 7f8eb656b740 10 client.16602 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.045+0000 7f8eb656b740 1 -- 172.21.15.83:0/1343038183 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56211c0ee3f0 con 0x56211c481b30 2024-04-23T14:56:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.046+0000 7f8e9a000640 1 -- 172.21.15.83:0/1343038183 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea4036030 con 0x56211c481b30 2024-04-23T14:56:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.046+0000 7f8e9a000640 1 -- 172.21.15.83:0/1343038183 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f8ea4043070 con 0x56211c481b30 2024-04-23T14:56:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.046+0000 7f8eb656b740 10 client.16602 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.046+0000 7f8eb656b740 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.046+0000 7f8eb656b740 20 client.16602 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.046+0000 7f8eb656b740 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56211c491b60 0x56211c493f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.046+0000 7f8eb656b740 4 client.16602 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:56:24.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.046+0000 7f8eb656b740 1 -- 172.21.15.83:0/1343038183 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56211c3e9bf0 con 0x56211c491b60 2024-04-23T14:56:24.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.046+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56211c491b60 0x56211c493f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.047+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56211c491b60 0x56211c493f50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0cb3947d911cd79 server_cookie=d0c30cbf27ead2d6 in_seq=0 out_seq=0 2024-04-23T14:56:24.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.047+0000 7f8e9a000640 10 client.16602 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:24.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8e9a000640 1 -- 172.21.15.83:0/1343038183 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56211c3e9bf0 con 0x56211c491b60 2024-04-23T14:56:24.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8e9a000640 10 client.16602 handle_command_reply: tid=0 2024-04-23T14:56:24.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8eb656b740 1 client.16602 shutdown 2024-04-23T14:56:24.054 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8eb656b740 1 -- 172.21.15.83:0/1343038183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 msgr2=0x56211c47ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8eb656b740 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c47ef90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4006010 tx=0x7f8ea402b030 comp rx=0 tx=0).stop 2024-04-23T14:56:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8eb656b740 1 -- 172.21.15.83:0/1343038183 shutdown_connections 2024-04-23T14:56:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8eb656b740 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56211c491b60 0x56211c493f50 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8eb656b740 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c47ef90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8eb656b740 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c477460 0x56211c47e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8e9a000640 0 client.16602 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8eb656b740 1 --2- 172.21.15.83:0/1343038183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c476b40 0x56211c481560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.055 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8eb656b740 1 -- 172.21.15.83:0/1343038183 >> 172.21.15.83:0/1343038183 conn(0x56211c472e60 msgr2=0x56211c478c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:24.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8eb656b740 1 -- 172.21.15.83:0/1343038183 shutdown_connections 2024-04-23T14:56:24.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8eb656b740 1 -- 172.21.15.83:0/1343038183 wait complete. 2024-04-23T14:56:24.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.048+0000 7f8eb656b740 20 client.16602 trim_cache size 0 max 16384 2024-04-23T14:56:24.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.063+0000 7f8eb656b740 1 Processor -- start 2024-04-23T14:56:24.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.063+0000 7f8eb656b740 1 -- start start 2024-04-23T14:56:24.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.063+0000 7f8eb656b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c48d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.063 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.063+0000 7f8eb656b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 0x56211c47d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.063+0000 7f8eb656b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c481090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.063+0000 7f8eb656b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac1b69f0 con 0x56211c481b30 2024-04-23T14:56:24.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.063+0000 7f8eb656b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac1b42c0 con 0x56211c476b40 2024-04-23T14:56:24.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.063+0000 7f8eb656b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac1b2130 con 0x56211c477460 2024-04-23T14:56:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8eb1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c481090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8eb0c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c48d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8eb0c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c48d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:34360/0 (socket says 172.21.15.83:34360) 2024-04-23T14:56:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8eb1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c481090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33274/0 (socket says 172.21.15.83:33274) 2024-04-23T14:56:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8eb0c00640 1 -- 172.21.15.83:0/1496172788 learned_addr learned my addr 172.21.15.83:0/1496172788 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8ea3e00640 1 --2- 172.21.15.83:0/1496172788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 0x56211c47d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8eb0c00640 1 -- 172.21.15.83:0/1496172788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 msgr2=0x56211c47d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:24.066 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/1496172788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 0x56211c47d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8eb0c00640 1 -- 172.21.15.83:0/1496172788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 msgr2=0x56211c481090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:24.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/1496172788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c481090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8eb0c00640 1 -- 172.21.15.83:0/1496172788 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56211c4942c0 con 0x56211c476b40 2024-04-23T14:56:24.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8eb1600640 1 --2- 172.21.15.83:0/1496172788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c481090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:24.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.064+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/1496172788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c48d880 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x56211c4944c0 tx=0x7f8e9c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df15f9113ad168e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:24.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.065+0000 7f8ea1200640 1 -- 172.21.15.83:0/1496172788 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c02b470 con 0x56211c476b40 2024-04-23T14:56:24.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.065+0000 7f8ea1200640 1 -- 172.21.15.83:0/1496172788 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e9c030070 con 0x56211c476b40 2024-04-23T14:56:24.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.065+0000 7f8ea1200640 1 -- 172.21.15.83:0/1496172788 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c01f690 con 0x56211c476b40 2024-04-23T14:56:24.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.065+0000 7f8eb656b740 1 -- 172.21.15.83:0/1496172788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 msgr2=0x56211c48d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:24.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.065+0000 7f8eb656b740 1 --2- 172.21.15.83:0/1496172788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c48d880 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x56211c4944c0 tx=0x7f8e9c01b000 comp rx=0 tx=0).stop 2024-04-23T14:56:24.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.065+0000 7f8eb656b740 1 -- 172.21.15.83:0/1496172788 shutdown_connections 2024-04-23T14:56:24.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.065+0000 7f8eb656b740 1 --2- 172.21.15.83:0/1496172788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c481090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.065+0000 7f8eb656b740 1 --2- 172.21.15.83:0/1496172788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 0x56211c47d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.065+0000 7f8eb656b740 1 --2- 172.21.15.83:0/1496172788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c48d880 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.065+0000 7f8eb656b740 1 -- 172.21.15.83:0/1496172788 >> 172.21.15.83:0/1496172788 conn(0x56211c472e60 msgr2=0x56211c47d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:24.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.066+0000 7f8eb656b740 1 -- 172.21.15.83:0/1496172788 shutdown_connections 2024-04-23T14:56:24.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.066+0000 7f8eb656b740 1 -- 172.21.15.83:0/1496172788 wait complete. 2024-04-23T14:56:24.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.066+0000 7f8eb656b740 1 -- start start 2024-04-23T14:56:24.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.066+0000 7f8eb656b740 1 Processor -- start 2024-04-23T14:56:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.066+0000 7f8eb656b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c4826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.066+0000 7f8eb656b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 0x56211c482c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.066+0000 7f8eb656b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c488d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.066+0000 7f8eb656b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac1b2780 con 0x56211c481b30 2024-04-23T14:56:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.066+0000 7f8eb656b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac1b3bf0 con 0x56211c476b40 2024-04-23T14:56:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.066+0000 7f8eb656b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac1b2e50 con 0x56211c477460 2024-04-23T14:56:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8eb656b740 10 client.0 _ll_register_callbacks cb 0x7f8e940d4450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8eb1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c488d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8eb1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c488d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33276/0 (socket says 172.21.15.83:33276) 2024-04-23T14:56:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8eb1600640 1 -- 172.21.15.83:0/482158621 learned_addr learned my addr 172.21.15.83:0/482158621 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8ea3e00640 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 0x56211c482c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c4826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8eb1600640 1 -- 172.21.15.83:0/482158621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 msgr2=0x56211c482c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8eb1600640 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 0x56211c482c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8eb1600640 1 -- 172.21.15.83:0/482158621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 msgr2=0x56211c4826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8eb1600640 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c4826e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8eb1600640 1 -- 172.21.15.83:0/482158621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56211c1b87d0 con 0x56211c481b30 2024-04-23T14:56:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8eb1600640 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c488d90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8ea402ae10 tx=0x7f8ea404fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc76cbbe649cc2c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.067+0000 7f8e9a000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.068+0000 7f8e9a000640 1 -- 172.21.15.83:0/482158621 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4053070 con 0x56211c481b30 2024-04-23T14:56:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.068+0000 7f8e9a000640 1 -- 172.21.15.83:0/482158621 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f8ea404a040 con 0x56211c481b30 2024-04-23T14:56:24.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.068+0000 7f8eb656b740 1 -- 172.21.15.83:0/482158621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e9c030070 con 0x56211c481b30 2024-04-23T14:56:24.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.068+0000 7f8e9a000640 1 -- 172.21.15.83:0/482158621 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ea406a050 con 0x56211c481b30 2024-04-23T14:56:24.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.068+0000 7f8eb656b740 10 client.16608 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:24.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.068+0000 7f8eb656b740 1 -- 172.21.15.83:0/482158621 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56211c48a400 con 0x56211c481b30 2024-04-23T14:56:24.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.068+0000 7f8e9a000640 1 -- 172.21.15.83:0/482158621 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e9c030070 con 0x56211c481b30 2024-04-23T14:56:24.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.068+0000 7f8e9a000640 1 -- 172.21.15.83:0/482158621 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f8ea4043040 con 0x56211c481b30 2024-04-23T14:56:24.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.068+0000 7f8eb656b740 10 client.16608 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:24.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.068+0000 7f8eb656b740 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:24.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.068+0000 7f8eb656b740 20 client.16608 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:24.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.068+0000 7f8eb656b740 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56211c4f8090 0x56211c4fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:24.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.069+0000 7f8eb656b740 4 client.16608 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:56:24.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.069+0000 7f8eb656b740 1 -- 172.21.15.83:0/482158621 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56211c3e3a60 con 0x56211c4f8090 2024-04-23T14:56:24.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.069+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56211c4f8090 0x56211c4fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:24.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.069+0000 7f8eb0c00640 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56211c4f8090 0x56211c4fa4b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6072d555a45e1664 server_cookie=e1e94c3faf7842b8 in_seq=0 out_seq=0 2024-04-23T14:56:24.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.069+0000 7f8e9a000640 10 client.16608 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:24.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8e9a000640 1 -- 172.21.15.83:0/482158621 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56211c3e3a60 con 0x56211c4f8090 2024-04-23T14:56:24.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8e9a000640 10 client.16608 handle_command_reply: tid=0 2024-04-23T14:56:24.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8eb656b740 1 client.16608 shutdown 2024-04-23T14:56:24.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8eb656b740 1 -- 172.21.15.83:0/482158621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 msgr2=0x56211c488d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:24.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8eb656b740 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c488d90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8ea402ae10 tx=0x7f8ea404fc60 comp rx=0 tx=0).stop 2024-04-23T14:56:24.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8eb656b740 1 -- 172.21.15.83:0/482158621 shutdown_connections 2024-04-23T14:56:24.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8eb656b740 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56211c4f8090 0x56211c4fa4b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8eb656b740 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56211c481b30 0x56211c488d90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8eb656b740 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56211c477460 0x56211c482c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8e9a000640 0 client.16608 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:24.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8eb656b740 1 --2- 172.21.15.83:0/482158621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56211c476b40 0x56211c4826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8eb656b740 1 -- 172.21.15.83:0/482158621 >> 172.21.15.83:0/482158621 conn(0x56211c472e60 msgr2=0x56211c478290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:24.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8eb656b740 1 -- 172.21.15.83:0/482158621 shutdown_connections 2024-04-23T14:56:24.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8eb656b740 1 -- 172.21.15.83:0/482158621 wait complete. 2024-04-23T14:56:24.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.070+0000 7f8eb656b740 20 client.16608 trim_cache size 0 max 16384 2024-04-23T14:56:24.078 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:56:24.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.071+0000 7f8eb2000640 1 -- 172.21.15.83:0/3478020461 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f8e900a3da0 msgr2=0x7f8e900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:24.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.071+0000 7f8eb2000640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f8e900a3da0 0x7f8e900a6260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x56211c478100 tx=0x7f8e9c005f90 comp rx=0 tx=0).stop 2024-04-23T14:56:24.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.071+0000 7f8eb2000640 1 -- 172.21.15.83:0/3478020461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eac108f40 msgr2=0x7f8eac19f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:24.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.071+0000 7f8eb2000640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eac108f40 0x7f8eac19f2e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400e9f0 tx=0x7f8e9400eeb0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.071+0000 7f8eb2000640 1 -- 172.21.15.83:0/3478020461 shutdown_connections 2024-04-23T14:56:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.071+0000 7f8eb2000640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8eac10d500 0x7f8eac1a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.071+0000 7f8eb2000640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f8e900a3da0 0x7f8e900a6260 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.071+0000 7f8eb2000640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8eac108f40 0x7f8eac19f2e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.071+0000 7f8eb2000640 1 --2- 172.21.15.83:0/3478020461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8eac108580 0x7f8eac19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.071+0000 7f8eb2000640 1 -- 172.21.15.83:0/3478020461 >> 172.21.15.83:0/3478020461 conn(0x7f8eac077e20 msgr2=0x7f8eac0783a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.071+0000 7f8eb2000640 1 -- 172.21.15.83:0/3478020461 shutdown_connections 2024-04-23T14:56:24.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:24.071+0000 7f8eb2000640 1 -- 172.21.15.83:0/3478020461 wait complete. 2024-04-23T14:56:24.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:24 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2785055058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:56:24.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:24 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2785055058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:56:24.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:24 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2785055058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:56:25.088 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:56:25.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:56:25.089 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:56:25.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.312+0000 7ff962e00640 1 -- 172.21.15.83:0/4281492293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c104cc0 msgr2=0x7ff95c108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.312+0000 7ff962e00640 1 --2- 172.21.15.83:0/4281492293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c104cc0 0x7ff95c108d00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff948009900 tx=0x7ff94802a400 comp rx=0 tx=0).stop 2024-04-23T14:56:25.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.313+0000 7ff962e00640 1 -- 172.21.15.83:0/4281492293 shutdown_connections 2024-04-23T14:56:25.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.313+0000 7ff962e00640 1 --2- 172.21.15.83:0/4281492293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c1093e0 0x7ff95c110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.313+0000 7ff962e00640 1 --2- 172.21.15.83:0/4281492293 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c104cc0 0x7ff95c108d00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.313+0000 7ff962e00640 1 --2- 172.21.15.83:0/4281492293 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff95c104310 0x7ff95c1046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.313+0000 7ff962e00640 1 -- 172.21.15.83:0/4281492293 >> 172.21.15.83:0/4281492293 conn(0x7ff95c0ffb10 msgr2=0x7ff95c101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:25.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.313+0000 7ff962e00640 1 -- 172.21.15.83:0/4281492293 shutdown_connections 2024-04-23T14:56:25.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.313+0000 7ff962e00640 1 -- 172.21.15.83:0/4281492293 wait complete. 2024-04-23T14:56:25.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.314+0000 7ff962e00640 1 Processor -- start 2024-04-23T14:56:25.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.314+0000 7ff962e00640 1 -- start start 2024-04-23T14:56:25.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.314+0000 7ff962e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff95c104310 0x7ff95c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.314+0000 7ff962e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c1093e0 0x7ff95c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.314+0000 7ff962e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c19a7c0 0x7ff95c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.314+0000 7ff962e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1144c0 con 0x7ff95c1093e0 2024-04-23T14:56:25.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.314+0000 7ff962e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c114340 con 0x7ff95c19a7c0 2024-04-23T14:56:25.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.314+0000 7ff962e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c114640 con 0x7ff95c104310 2024-04-23T14:56:25.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff961000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c1093e0 0x7ff95c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff962400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c19a7c0 0x7ff95c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff961a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff95c104310 0x7ff95c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff961000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c1093e0 0x7ff95c1a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49560/0 (socket says 172.21.15.83:49560) 2024-04-23T14:56:25.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff961000640 1 -- 172.21.15.83:0/464195903 learned_addr learned my addr 172.21.15.83:0/464195903 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:25.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff962400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c19a7c0 0x7ff95c19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:39152/0 (socket says 172.21.15.83:39152) 2024-04-23T14:56:25.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff962400640 1 -- 172.21.15.83:0/464195903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff95c104310 msgr2=0x7ff95c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff962400640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff95c104310 0x7ff95c1a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff962400640 1 -- 172.21.15.83:0/464195903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c1093e0 msgr2=0x7ff95c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff962400640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c1093e0 0x7ff95c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff962400640 1 -- 172.21.15.83:0/464195903 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c19b3c0 con 0x7ff95c19a7c0 2024-04-23T14:56:25.318 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff961a00640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff95c104310 0x7ff95c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:25.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.315+0000 7ff962400640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c19a7c0 0x7ff95c19ac40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff954009fd0 tx=0x7ff95400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:25.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.316+0000 7ff95a000640 1 -- 172.21.15.83:0/464195903 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95400ce20 con 0x7ff95c19a7c0 2024-04-23T14:56:25.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.316+0000 7ff95a000640 1 -- 172.21.15.83:0/464195903 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff954005020 con 0x7ff95c19a7c0 2024-04-23T14:56:25.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.316+0000 7ff95a000640 1 -- 172.21.15.83:0/464195903 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954010720 con 0x7ff95c19a7c0 2024-04-23T14:56:25.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.316+0000 7ff962e00640 1 -- 172.21.15.83:0/464195903 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95c1a7450 con 0x7ff95c19a7c0 2024-04-23T14:56:25.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.316+0000 7ff962e00640 1 -- 172.21.15.83:0/464195903 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95c1a7800 con 0x7ff95c19a7c0 2024-04-23T14:56:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.319+0000 7ff95a000640 1 -- 172.21.15.83:0/464195903 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff954005540 con 0x7ff95c19a7c0 2024-04-23T14:56:25.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.319+0000 7ff967169740 1 Processor -- start 2024-04-23T14:56:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.319+0000 7ff967169740 1 -- start start 2024-04-23T14:56:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff967169740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 0x56549de77650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff967169740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77b90 0x56549de81d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff967169740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de82260 0x56549de84650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff967169740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56549dddce70 con 0x56549de77270 2024-04-23T14:56:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff967169740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56549dd83380 con 0x56549de77b90 2024-04-23T14:56:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff967169740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56549ddd8e30 con 0x56549de82260 2024-04-23T14:56:25.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff961000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 0x56549de77650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff961a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77b90 0x56549de81d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff961000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 0x56549de77650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49562/0 (socket says 172.21.15.83:49562) 2024-04-23T14:56:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff961000640 1 -- 172.21.15.83:0/1194917669 learned_addr learned my addr 172.21.15.83:0/1194917669 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff961000640 1 --2- 172.21.15.83:0/1194917669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de82260 0x56549de84650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff961a00640 1 -- 172.21.15.83:0/1194917669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de82260 msgr2=0x56549de84650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff961a00640 1 --2- 172.21.15.83:0/1194917669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de82260 0x56549de84650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff961a00640 1 -- 172.21.15.83:0/1194917669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 msgr2=0x56549de77650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff961a00640 1 --2- 172.21.15.83:0/1194917669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 0x56549de77650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.324 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.320+0000 7ff961a00640 1 -- 172.21.15.83:0/1194917669 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549dbb77e0 con 0x56549de77b90 2024-04-23T14:56:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.321+0000 7ff961000640 1 --2- 172.21.15.83:0/1194917669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de82260 0x56549de84650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:56:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.321+0000 7ff95a000640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff9440a3da0 0x7ff9440a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.321+0000 7ff95a000640 1 -- 172.21.15.83:0/464195903 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7ff9540d0de0 con 0x7ff95c19a7c0 2024-04-23T14:56:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.321+0000 7ff961a00640 1 --2- 172.21.15.83:0/1194917669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77b90 0x56549de81d20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff9500103d0 tx=0x7ff950010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7125dfcb16794dfc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.321+0000 7ff961000640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff9440a3da0 0x7ff9440a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.321+0000 7ff94ea00640 1 -- 172.21.15.83:0/1194917669 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950011700 con 0x56549de77b90 2024-04-23T14:56:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.321+0000 7ff94ea00640 1 -- 172.21.15.83:0/1194917669 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff950011d80 con 0x56549de77b90 2024-04-23T14:56:25.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.321+0000 7ff94ea00640 1 -- 172.21.15.83:0/1194917669 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95001b3e0 con 0x56549de77b90 2024-04-23T14:56:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.322+0000 7ff967169740 1 -- 172.21.15.83:0/1194917669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77b90 msgr2=0x56549de81d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.322+0000 7ff961000640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff9440a3da0 0x7ff9440a6260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff948031550 tx=0x7ff948005bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.322+0000 7ff967169740 1 --2- 172.21.15.83:0/1194917669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77b90 0x56549de81d20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff9500103d0 tx=0x7ff950010890 comp rx=0 tx=0).stop 2024-04-23T14:56:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.322+0000 7ff967169740 1 -- 172.21.15.83:0/1194917669 shutdown_connections 2024-04-23T14:56:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.322+0000 7ff967169740 1 --2- 172.21.15.83:0/1194917669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de82260 0x56549de84650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.322+0000 7ff967169740 1 --2- 172.21.15.83:0/1194917669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77b90 0x56549de81d20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.322+0000 7ff967169740 1 --2- 172.21.15.83:0/1194917669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 0x56549de77650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.322+0000 7ff967169740 1 -- 172.21.15.83:0/1194917669 >> 172.21.15.83:0/1194917669 conn(0x56549de73590 msgr2=0x56549de73970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:25.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.322+0000 7ff967169740 1 -- 172.21.15.83:0/1194917669 shutdown_connections 2024-04-23T14:56:25.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.322+0000 7ff967169740 1 -- 172.21.15.83:0/1194917669 wait complete. 2024-04-23T14:56:25.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.323+0000 7ff967169740 1 -- start start 2024-04-23T14:56:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.323+0000 7ff967169740 1 Processor -- start 2024-04-23T14:56:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.323+0000 7ff967169740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de77270 0x56549de7f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.323+0000 7ff967169740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77b90 0x56549de7f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.323+0000 7ff967169740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de82260 0x56549de7fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff967169740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56549de85050 con 0x56549de77b90 2024-04-23T14:56:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff967169740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x56549dd96660 con 0x56549de82260 2024-04-23T14:56:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff967169740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56549de851d0 con 0x56549de77270 2024-04-23T14:56:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff967169740 10 client.0 _ll_register_callbacks cb 0x56549dd84aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff961000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77b90 0x56549de7f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff961000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77b90 0x56549de7f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49576/0 (socket says 172.21.15.83:49576) 2024-04-23T14:56:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff961000640 1 -- 172.21.15.83:0/3228218661 learned_addr learned my addr 172.21.15.83:0/3228218661 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff961a00640 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de77270 0x56549de7f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff962400640 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de82260 0x56549de7fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff961000640 1 -- 172.21.15.83:0/3228218661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de77270 msgr2=0x56549de7f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff961000640 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de77270 0x56549de7f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff961000640 1 -- 172.21.15.83:0/3228218661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de82260 msgr2=0x56549de7fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff961000640 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de82260 0x56549de7fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.324+0000 7ff961000640 1 -- 172.21.15.83:0/3228218661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56549dc17f70 con 0x56549de77b90 2024-04-23T14:56:25.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.325+0000 7ff961000640 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77b90 0x56549de7f680 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff948006010 tx=0x7ff94802a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acdee8e609ba6e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.325+0000 7ff943400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.325+0000 7ff943400640 1 -- 172.21.15.83:0/3228218661 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948035040 con 0x56549de77b90 2024-04-23T14:56:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.325+0000 7ff943400640 1 -- 172.21.15.83:0/3228218661 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7ff948004d70 con 0x56549de77b90 2024-04-23T14:56:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.325+0000 7ff967169740 1 -- 172.21.15.83:0/3228218661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff950011d80 con 0x56549de77b90 2024-04-23T14:56:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.325+0000 7ff943400640 1 -- 172.21.15.83:0/3228218661 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9480325c0 con 0x56549de77b90 2024-04-23T14:56:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.325+0000 7ff967169740 10 client.16620 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.325+0000 7ff967169740 1 -- 172.21.15.83:0/3228218661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56549daaf170 con 0x56549de77b90 2024-04-23T14:56:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.326+0000 7ff943400640 1 -- 172.21.15.83:0/3228218661 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff950011d80 con 0x56549de77b90 2024-04-23T14:56:25.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.326+0000 7ff943400640 1 -- 172.21.15.83:0/3228218661 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ff948003b20 con 0x56549de77b90 2024-04-23T14:56:25.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.326+0000 7ff967169740 10 client.16620 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:25.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.326+0000 7ff967169740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:25.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.326+0000 7ff967169740 20 client.16620 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:25.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.326+0000 7ff967169740 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56549de92510 0x56549de94990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.326+0000 7ff967169740 4 client.16620 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:56:25.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.326+0000 7ff967169740 1 -- 172.21.15.83:0/3228218661 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56549ddea890 con 0x56549de92510 2024-04-23T14:56:25.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.326+0000 7ff961a00640 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56549de92510 0x56549de94990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.327+0000 7ff961a00640 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56549de92510 0x56549de94990 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c32fa57fd1469e server_cookie=eb205efd07faaf73 in_seq=0 out_seq=0 2024-04-23T14:56:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.327+0000 7ff943400640 10 client.16620 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff943400640 1 -- 172.21.15.83:0/3228218661 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56549ddea890 con 0x56549de92510 2024-04-23T14:56:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff943400640 10 client.16620 handle_command_reply: tid=0 2024-04-23T14:56:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff967169740 1 client.16620 shutdown 2024-04-23T14:56:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff967169740 1 -- 172.21.15.83:0/3228218661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77b90 msgr2=0x56549de7f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff967169740 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77b90 0x56549de7f680 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff948006010 tx=0x7ff94802a400 comp rx=0 tx=0).stop 2024-04-23T14:56:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff967169740 1 -- 172.21.15.83:0/3228218661 shutdown_connections 2024-04-23T14:56:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff967169740 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56549de92510 0x56549de94990 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff967169740 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de82260 0x56549de7fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff967169740 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77b90 0x56549de7f680 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff967169740 1 --2- 172.21.15.83:0/3228218661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de77270 0x56549de7f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff943400640 0 client.16620 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff967169740 1 -- 172.21.15.83:0/3228218661 >> 172.21.15.83:0/3228218661 conn(0x56549de73590 msgr2=0x56549de79290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.328+0000 7ff967169740 1 -- 172.21.15.83:0/3228218661 shutdown_connections 2024-04-23T14:56:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.329+0000 7ff967169740 1 -- 172.21.15.83:0/3228218661 wait complete. 2024-04-23T14:56:25.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.329+0000 7ff967169740 20 client.16620 trim_cache size 0 max 16384 2024-04-23T14:56:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff967169740 1 Processor -- start 2024-04-23T14:56:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff967169740 1 -- start start 2024-04-23T14:56:25.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff967169740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77270 0x56549de7e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff967169740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de77b90 0x56549de8df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff967169740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de82260 0x56549de9a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff967169740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1b2500 con 0x56549de82260 2024-04-23T14:56:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff967169740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1afdd0 con 0x56549de77270 2024-04-23T14:56:25.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff967169740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1adc40 con 0x56549de77b90 2024-04-23T14:56:25.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff962400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de82260 0x56549de9a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff961a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77270 0x56549de7e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff962400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de82260 0x56549de9a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49578/0 (socket says 172.21.15.83:49578) 2024-04-23T14:56:25.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff962400640 1 -- 172.21.15.83:0/3089160516 learned_addr learned my addr 172.21.15.83:0/3089160516 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:25.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.348+0000 7ff961000640 1 --2- 172.21.15.83:0/3089160516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de77b90 0x56549de8df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.349+0000 7ff961a00640 1 -- 172.21.15.83:0/3089160516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de77b90 msgr2=0x56549de8df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.349+0000 7ff961a00640 1 --2- 172.21.15.83:0/3089160516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de77b90 0x56549de8df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.349+0000 7ff961a00640 1 -- 172.21.15.83:0/3089160516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de82260 msgr2=0x56549de9a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.349+0000 7ff961a00640 1 --2- 172.21.15.83:0/3089160516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de82260 0x56549de9a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.349+0000 7ff961a00640 1 -- 172.21.15.83:0/3089160516 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56549de780e0 con 0x56549de77270 2024-04-23T14:56:25.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.349+0000 7ff962400640 1 --2- 172.21.15.83:0/3089160516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de82260 0x56549de9a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:25.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.349+0000 7ff961a00640 1 --2- 172.21.15.83:0/3089160516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77270 0x56549de7e080 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff95001a9f0 tx=0x7ff95001a970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf56d2a0dce4aab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:25.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.349+0000 7ff94e000640 1 -- 172.21.15.83:0/3089160516 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95001aa20 con 0x56549de77270 2024-04-23T14:56:25.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.349+0000 7ff94e000640 1 -- 172.21.15.83:0/3089160516 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff950005520 con 0x56549de77270 2024-04-23T14:56:25.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.349+0000 7ff94e000640 1 -- 172.21.15.83:0/3089160516 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95002a630 con 0x56549de77270 2024-04-23T14:56:25.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.350+0000 7ff967169740 1 -- 172.21.15.83:0/3089160516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77270 msgr2=0x56549de7e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.350+0000 7ff967169740 1 --2- 172.21.15.83:0/3089160516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77270 0x56549de7e080 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff95001a9f0 tx=0x7ff95001a970 comp rx=0 tx=0).stop 2024-04-23T14:56:25.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.350+0000 7ff967169740 1 -- 172.21.15.83:0/3089160516 shutdown_connections 2024-04-23T14:56:25.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.350+0000 7ff967169740 1 --2- 172.21.15.83:0/3089160516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de82260 0x56549de9a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.350+0000 7ff967169740 1 --2- 172.21.15.83:0/3089160516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de77b90 0x56549de8df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.350+0000 7ff967169740 1 --2- 172.21.15.83:0/3089160516 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77270 0x56549de7e080 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.350+0000 7ff967169740 1 -- 172.21.15.83:0/3089160516 >> 172.21.15.83:0/3089160516 conn(0x56549de73590 msgr2=0x56549de7f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:25.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.350+0000 7ff967169740 1 -- 172.21.15.83:0/3089160516 shutdown_connections 2024-04-23T14:56:25.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.350+0000 7ff967169740 1 -- 172.21.15.83:0/3089160516 wait complete. 2024-04-23T14:56:25.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.351+0000 7ff967169740 1 -- start start 2024-04-23T14:56:25.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.351+0000 7ff967169740 1 Processor -- start 2024-04-23T14:56:25.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.351+0000 7ff967169740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 0x56549de86790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.351+0000 7ff967169740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77b90 0x56549de86cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.351+0000 7ff967169740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de82260 0x56549def1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.351+0000 7ff967169740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1ae290 con 0x56549de77270 2024-04-23T14:56:25.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.351+0000 7ff967169740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1af700 con 0x56549de77b90 2024-04-23T14:56:25.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.351+0000 7ff967169740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1ae960 con 0x56549de82260 2024-04-23T14:56:25.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.351+0000 7ff967169740 10 client.0 _ll_register_callbacks cb 0x7ff948004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:25.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.351+0000 7ff961a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 0x56549de86790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.352+0000 7ff961a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 0x56549de86790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49584/0 (socket says 172.21.15.83:49584) 2024-04-23T14:56:25.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.352+0000 7ff961a00640 1 -- 172.21.15.83:0/914571604 learned_addr learned my addr 172.21.15.83:0/914571604 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:25.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.352+0000 7ff962400640 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de82260 0x56549def1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.357 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.352+0000 7ff961000640 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77b90 0x56549de86cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.352+0000 7ff961a00640 1 -- 172.21.15.83:0/914571604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de82260 msgr2=0x56549def1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.352+0000 7ff961a00640 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de82260 0x56549def1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.352+0000 7ff961a00640 1 -- 172.21.15.83:0/914571604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77b90 msgr2=0x56549de86cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.352+0000 7ff961a00640 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77b90 0x56549de86cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.352+0000 7ff961a00640 1 -- 172.21.15.83:0/914571604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56549def2100 con 0x56549de77270 2024-04-23T14:56:25.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.352+0000 7ff961000640 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77b90 0x56549de86cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:56:25.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.352+0000 7ff961a00640 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 0x56549de86790 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff950010ef0 tx=0x7ff950010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08006330831ac92 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:25.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.352+0000 7ff943400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:25.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff943400640 1 -- 172.21.15.83:0/914571604 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950005380 con 0x56549de77270 2024-04-23T14:56:25.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff943400640 1 -- 172.21.15.83:0/914571604 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7ff95001a600 con 0x56549de77270 2024-04-23T14:56:25.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff967169740 1 -- 172.21.15.83:0/914571604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff950005520 con 0x56549de77270 2024-04-23T14:56:25.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff943400640 1 -- 172.21.15.83:0/914571604 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff950040050 con 0x56549de77270 2024-04-23T14:56:25.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff967169740 10 client.16626 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:25.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff967169740 1 -- 172.21.15.83:0/914571604 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56549def94e0 con 0x56549de77270 2024-04-23T14:56:25.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff943400640 1 -- 172.21.15.83:0/914571604 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff950005520 con 0x56549de77270 2024-04-23T14:56:25.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff943400640 1 -- 172.21.15.83:0/914571604 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ff9500035f0 con 0x56549de77270 2024-04-23T14:56:25.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff967169740 10 client.16626 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:25.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff967169740 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:25.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff967169740 20 client.16626 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:25.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff967169740 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56549def9a40 0x56549defbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff967169740 4 client.16626 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:56:25.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.353+0000 7ff967169740 1 -- 172.21.15.83:0/914571604 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56549dde4700 con 0x56549def9a40 2024-04-23T14:56:25.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.354+0000 7ff961000640 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56549def9a40 0x56549defbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.354+0000 7ff961000640 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56549def9a40 0x56549defbe60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c537b04e8157d27 server_cookie=9ab9642697d95287 in_seq=0 out_seq=0 2024-04-23T14:56:25.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.354+0000 7ff943400640 10 client.16626 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:25.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.355+0000 7ff943400640 1 -- 172.21.15.83:0/914571604 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56549dde4700 con 0x56549def9a40 2024-04-23T14:56:25.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.355+0000 7ff943400640 10 client.16626 handle_command_reply: tid=0 2024-04-23T14:56:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.355+0000 7ff967169740 1 client.16626 shutdown 2024-04-23T14:56:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.356+0000 7ff967169740 1 -- 172.21.15.83:0/914571604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 msgr2=0x56549de86790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.356+0000 7ff967169740 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 0x56549de86790 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff950010ef0 tx=0x7ff950010f20 comp rx=0 tx=0).stop 2024-04-23T14:56:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.356+0000 7ff967169740 1 -- 172.21.15.83:0/914571604 shutdown_connections 2024-04-23T14:56:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.356+0000 7ff967169740 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56549def9a40 0x56549defbe60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.356+0000 7ff967169740 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56549de82260 0x56549def1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.356+0000 7ff967169740 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56549de77b90 0x56549de86cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.362 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.356+0000 7ff967169740 1 --2- 172.21.15.83:0/914571604 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56549de77270 0x56549de86790 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.356+0000 7ff967169740 1 -- 172.21.15.83:0/914571604 >> 172.21.15.83:0/914571604 conn(0x56549de73590 msgr2=0x56549de789c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:25.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.356+0000 7ff943400640 0 client.16626 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:25.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.356+0000 7ff967169740 1 -- 172.21.15.83:0/914571604 shutdown_connections 2024-04-23T14:56:25.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.356+0000 7ff967169740 1 -- 172.21.15.83:0/914571604 wait complete. 2024-04-23T14:56:25.363 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.356+0000 7ff967169740 20 client.16626 trim_cache size 0 max 16384 2024-04-23T14:56:25.363 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:56:25.364 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:56:25.364 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "026119be-7cf6-43ec-91bd-2a55148a855d", 2024-04-23T14:56:25.364 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:56:25.364 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:56:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.357+0000 7ff962e00640 1 -- 172.21.15.83:0/464195903 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff9440a3da0 msgr2=0x7ff9440a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.357+0000 7ff962e00640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff9440a3da0 0x7ff9440a6260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff948031550 tx=0x7ff948005bc0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.357+0000 7ff962e00640 1 -- 172.21.15.83:0/464195903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c19a7c0 msgr2=0x7ff95c19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.357+0000 7ff962e00640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c19a7c0 0x7ff95c19ac40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff954009fd0 tx=0x7ff95400ef10 comp rx=0 tx=0).stop 2024-04-23T14:56:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.357+0000 7ff962e00640 1 -- 172.21.15.83:0/464195903 shutdown_connections 2024-04-23T14:56:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.357+0000 7ff962e00640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff95c19a7c0 0x7ff95c19ac40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.357+0000 7ff962e00640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff9440a3da0 0x7ff9440a6260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.357+0000 7ff962e00640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c1093e0 0x7ff95c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.365 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.357+0000 7ff962e00640 1 --2- 172.21.15.83:0/464195903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff95c104310 0x7ff95c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.357+0000 7ff962e00640 1 -- 172.21.15.83:0/464195903 >> 172.21.15.83:0/464195903 conn(0x7ff95c0ffb10 msgr2=0x7ff95c101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:25.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.357+0000 7ff962e00640 1 -- 172.21.15.83:0/464195903 shutdown_connections 2024-04-23T14:56:25.366 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.357+0000 7ff962e00640 1 -- 172.21.15.83:0/464195903 wait complete. 2024-04-23T14:56:25.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:56:25.376 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:56:25.442 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:25 smithi083 ceph-mon[27867]: pgmap v246: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:56:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.573+0000 7f6212a00640 1 -- 172.21.15.83:0/2243688394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620c10e8e0 msgr2=0x7f620c110cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.573+0000 7f6212a00640 1 --2- 172.21.15.83:0/2243688394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620c10e8e0 0x7f620c110cd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6204009900 tx=0x7f620402a480 comp rx=0 tx=0).stop 2024-04-23T14:56:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.574+0000 7f6212a00640 1 -- 172.21.15.83:0/2243688394 shutdown_connections 2024-04-23T14:56:25.574 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.574+0000 7f6212a00640 1 --2- 172.21.15.83:0/2243688394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620c10e8e0 0x7f620c110cd0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.574+0000 7f6212a00640 1 --2- 172.21.15.83:0/2243688394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f620c074bf0 0x7f620c10e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.574+0000 7f6212a00640 1 --2- 172.21.15.83:0/2243688394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f620c0742d0 0x7f620c0746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.574+0000 7f6212a00640 1 -- 172.21.15.83:0/2243688394 >> 172.21.15.83:0/2243688394 conn(0x7f620c079450 msgr2=0x7f620c079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.574+0000 7f6212a00640 1 -- 172.21.15.83:0/2243688394 shutdown_connections 2024-04-23T14:56:25.575 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.574+0000 7f6212a00640 1 -- 172.21.15.83:0/2243688394 wait complete. 2024-04-23T14:56:25.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:25 smithi173 ceph-mon[32443]: pgmap v246: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:56:25.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.575+0000 7f6212a00640 1 Processor -- start 2024-04-23T14:56:25.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.575+0000 7f6212a00640 1 -- start start 2024-04-23T14:56:25.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.575+0000 7f6212a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620c0742d0 0x7f620c1a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.577 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.575+0000 7f6212a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f620c074bf0 0x7f620c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.575+0000 7f6212a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f620c10e8e0 0x7f620c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.575+0000 7f6212a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1145f0 con 0x7f620c0742d0 2024-04-23T14:56:25.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.575+0000 7f6212a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f620c114470 con 0x7f620c074bf0 2024-04-23T14:56:25.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.575+0000 7f6212a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f620c114770 con 0x7f620c10e8e0 2024-04-23T14:56:25.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.575+0000 7f620be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620c0742d0 0x7f620c1a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.576+0000 7f620be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620c0742d0 0x7f620c1a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49590/0 (socket says 172.21.15.83:49590) 2024-04-23T14:56:25.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.576+0000 7f620be00640 1 -- 172.21.15.83:0/1260166374 learned_addr learned my addr 172.21.15.83:0/1260166374 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:25.578 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.576+0000 7f6210a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f620c10e8e0 0x7f620c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.576+0000 7f620b400640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f620c074bf0 0x7f620c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.576+0000 7f6210a00640 1 -- 172.21.15.83:0/1260166374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f620c074bf0 msgr2=0x7f620c1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.576+0000 7f6210a00640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f620c074bf0 0x7f620c1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.576+0000 7f6210a00640 1 -- 172.21.15.83:0/1260166374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620c0742d0 msgr2=0x7f620c1a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.576+0000 7f6210a00640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620c0742d0 0x7f620c1a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.576+0000 7f6210a00640 1 -- 172.21.15.83:0/1260166374 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620c19b090 con 0x7f620c10e8e0 2024-04-23T14:56:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.576+0000 7f6210a00640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f620c10e8e0 0x7f620c19a910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6204009230 tx=0x7f6204003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.576+0000 7f6208c00640 1 -- 172.21.15.83:0/1260166374 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620403f070 con 0x7f620c10e8e0 2024-04-23T14:56:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.577+0000 7f6208c00640 1 -- 172.21.15.83:0/1260166374 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6204031c90 con 0x7f620c10e8e0 2024-04-23T14:56:25.579 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.577+0000 7f6212a00640 1 -- 172.21.15.83:0/1260166374 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620c19b320 con 0x7f620c10e8e0 2024-04-23T14:56:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.577+0000 7f6208c00640 1 -- 172.21.15.83:0/1260166374 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204035030 con 0x7f620c10e8e0 2024-04-23T14:56:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.577+0000 7f6212a00640 1 -- 172.21.15.83:0/1260166374 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620c1a7620 con 0x7f620c10e8e0 2024-04-23T14:56:25.580 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.581+0000 7f62159e1740 1 Processor -- start 2024-04-23T14:56:25.581 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.581+0000 7f6208c00640 1 -- 172.21.15.83:0/1260166374 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6204042070 con 0x7f620c10e8e0 2024-04-23T14:56:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.581+0000 7f62159e1740 1 -- start start 2024-04-23T14:56:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.581+0000 7f62159e1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090a9860 0x5651090a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.581+0000 7f62159e1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090aa180 0x5651090b4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f62159e1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090b4850 0x5651090b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f62159e1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56510900f1f0 con 0x5651090a9860 2024-04-23T14:56:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f62159e1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x565108fb5350 con 0x5651090b4850 2024-04-23T14:56:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f62159e1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56510900b1b0 con 0x5651090aa180 2024-04-23T14:56:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f620b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090a9860 0x5651090a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f620b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090a9860 0x5651090a9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:49594/0 (socket says 172.21.15.83:49594) 2024-04-23T14:56:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f620b400640 1 -- 172.21.15.83:0/788731863 learned_addr learned my addr 172.21.15.83:0/788731863 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f620be00640 1 --2- 172.21.15.83:0/788731863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090aa180 0x5651090b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f620b400640 1 --2- 172.21.15.83:0/788731863 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090b4850 0x5651090b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f620be00640 1 -- 172.21.15.83:0/788731863 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090b4850 msgr2=0x5651090b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.584 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f620be00640 1 --2- 172.21.15.83:0/788731863 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090b4850 0x5651090b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f620be00640 1 -- 172.21.15.83:0/788731863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090a9860 msgr2=0x5651090a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f620be00640 1 --2- 172.21.15.83:0/788731863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090a9860 0x5651090a9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f620be00640 1 -- 172.21.15.83:0/788731863 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565109009f50 con 0x5651090aa180 2024-04-23T14:56:25.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.582+0000 7f620b400640 1 --2- 172.21.15.83:0/788731863 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090b4850 0x5651090b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:56:25.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.583+0000 7f6208c00640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f61f00a3d20 0x7f61f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.583+0000 7f620be00640 1 --2- 172.21.15.83:0/788731863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090aa180 0x5651090b4310 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0114c0 tx=0x7f61fc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42085d9fde54f8ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:25.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.583+0000 7f6208c00640 1 -- 172.21.15.83:0/1260166374 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f62040f0d80 con 0x7f620c10e8e0 2024-04-23T14:56:25.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.583+0000 7f620b400640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f61f00a3d20 0x7f61f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.585 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.583+0000 7f6201200640 1 -- 172.21.15.83:0/788731863 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc01e070 con 0x5651090aa180 2024-04-23T14:56:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.583+0000 7f6201200640 1 -- 172.21.15.83:0/788731863 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61fc00fae0 con 0x5651090aa180 2024-04-23T14:56:25.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.583+0000 7f6201200640 1 -- 172.21.15.83:0/788731863 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc019b80 con 0x5651090aa180 2024-04-23T14:56:25.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.584+0000 7f62159e1740 1 -- 172.21.15.83:0/788731863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090aa180 msgr2=0x5651090b4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.584+0000 7f620b400640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f61f00a3d20 0x7f61f00a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009fd0 tx=0x7f61f4015040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:25.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.584+0000 7f62159e1740 1 --2- 172.21.15.83:0/788731863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090aa180 0x5651090b4310 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0114c0 tx=0x7f61fc009f90 comp rx=0 tx=0).stop 2024-04-23T14:56:25.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.584+0000 7f62159e1740 1 -- 172.21.15.83:0/788731863 shutdown_connections 2024-04-23T14:56:25.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.584+0000 7f62159e1740 1 --2- 172.21.15.83:0/788731863 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090b4850 0x5651090b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.584+0000 7f62159e1740 1 --2- 172.21.15.83:0/788731863 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090aa180 0x5651090b4310 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.584+0000 7f62159e1740 1 --2- 172.21.15.83:0/788731863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090a9860 0x5651090a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.584+0000 7f62159e1740 1 -- 172.21.15.83:0/788731863 >> 172.21.15.83:0/788731863 conn(0x5651090a5b80 msgr2=0x5651090a5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.584+0000 7f62159e1740 1 -- 172.21.15.83:0/788731863 shutdown_connections 2024-04-23T14:56:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.584+0000 7f62159e1740 1 -- 172.21.15.83:0/788731863 wait complete. 2024-04-23T14:56:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.584+0000 7f62159e1740 1 -- start start 2024-04-23T14:56:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.585+0000 7f62159e1740 1 Processor -- start 2024-04-23T14:56:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.585+0000 7f62159e1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 0x5651090b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.585+0000 7f62159e1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090aa180 0x5651090b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.585+0000 7f62159e1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.585+0000 7f62159e1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5651090b7640 con 0x5651090aa180 2024-04-23T14:56:25.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.585+0000 7f62159e1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x565108fc8690 con 0x5651090a9860 2024-04-23T14:56:25.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.585+0000 7f62159e1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5651090b77c0 con 0x5651090b4850 2024-04-23T14:56:25.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.585+0000 7f62159e1740 10 client.0 _ll_register_callbacks cb 0x565109004650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:25.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.585+0000 7f6210a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f620b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090aa180 0x5651090b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f620be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 0x5651090b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f6210a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:47546/0 (socket says 172.21.15.83:47546) 2024-04-23T14:56:25.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f6210a00640 1 -- 172.21.15.83:0/97983970 learned_addr learned my addr 172.21.15.83:0/97983970 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:25.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f620be00640 1 -- 172.21.15.83:0/97983970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 msgr2=0x5651090b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f620be00640 1 --2- 172.21.15.83:0/97983970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f620be00640 1 -- 172.21.15.83:0/97983970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090aa180 msgr2=0x5651090b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f620be00640 1 --2- 172.21.15.83:0/97983970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090aa180 0x5651090b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f620be00640 1 -- 172.21.15.83:0/97983970 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565108fc7920 con 0x5651090a9860 2024-04-23T14:56:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f6210a00640 1 --2- 172.21.15.83:0/97983970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f620be00640 1 --2- 172.21.15.83:0/97983970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 0x5651090b16c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f61fc000f80 tx=0x7f61fc00f830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89a9138bcfe26192 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f61f9600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:56:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f61f9600640 1 -- 172.21.15.83:0/97983970 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc026020 con 0x5651090a9860 2024-04-23T14:56:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f61f9600640 1 -- 172.21.15.83:0/97983970 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f61fc004a40 con 0x5651090a9860 2024-04-23T14:56:25.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.586+0000 7f62159e1740 1 -- 172.21.15.83:0/97983970 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61fc00fae0 con 0x5651090a9860 2024-04-23T14:56:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.587+0000 7f61f9600640 1 -- 172.21.15.83:0/97983970 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f61fc01e040 con 0x5651090a9860 2024-04-23T14:56:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.587+0000 7f62159e1740 10 client.26837 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.587+0000 7f62159e1740 1 -- 172.21.15.83:0/97983970 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565108deb7d0 con 0x5651090a9860 2024-04-23T14:56:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.587+0000 7f61f9600640 1 -- 172.21.15.83:0/97983970 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61fc00fae0 con 0x5651090a9860 2024-04-23T14:56:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.587+0000 7f61f9600640 1 -- 172.21.15.83:0/97983970 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f61fc021070 con 0x5651090a9860 2024-04-23T14:56:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.587+0000 7f62159e1740 10 client.26837 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.587+0000 7f62159e1740 10 client.26837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.587+0000 7f62159e1740 20 client.26837 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:25.594 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.587+0000 7f62159e1740 1 --2- 172.21.15.83:0/97983970 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5651090c49b0 0x5651090c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.587+0000 7f62159e1740 4 client.26837 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:56:25.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.587+0000 7f62159e1740 1 -- 172.21.15.83:0/97983970 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56510901cc10 con 0x5651090c49b0 2024-04-23T14:56:25.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.588+0000 7f620b400640 1 --2- 172.21.15.83:0/97983970 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5651090c49b0 0x5651090c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.588+0000 7f620b400640 1 --2- 172.21.15.83:0/97983970 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5651090c49b0 0x5651090c6da0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f90f7d80ebc3296 server_cookie=b5bd071e434b71ed in_seq=0 out_seq=0 2024-04-23T14:56:25.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.588+0000 7f61f9600640 10 client.26837 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:25.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.589+0000 7f61f9600640 1 -- 172.21.15.83:0/97983970 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56510901cc10 con 0x5651090c49b0 2024-04-23T14:56:25.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.589+0000 7f61f9600640 10 client.26837 handle_command_reply: tid=0 2024-04-23T14:56:25.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.589+0000 7f62159e1740 1 client.26837 shutdown 2024-04-23T14:56:25.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.590+0000 7f62159e1740 1 -- 172.21.15.83:0/97983970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 msgr2=0x5651090b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.590+0000 7f62159e1740 1 --2- 172.21.15.83:0/97983970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 0x5651090b16c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f61fc000f80 tx=0x7f61fc00f830 comp rx=0 tx=0).stop 2024-04-23T14:56:25.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.590+0000 7f62159e1740 1 -- 172.21.15.83:0/97983970 shutdown_connections 2024-04-23T14:56:25.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.590+0000 7f62159e1740 1 --2- 172.21.15.83:0/97983970 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5651090c49b0 0x5651090c6da0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.590+0000 7f62159e1740 1 --2- 172.21.15.83:0/97983970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.590+0000 7f62159e1740 1 --2- 172.21.15.83:0/97983970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090aa180 0x5651090b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.590+0000 7f61f9600640 0 client.26837 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:25.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.590+0000 7f62159e1740 1 --2- 172.21.15.83:0/97983970 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 0x5651090b16c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.590+0000 7f62159e1740 1 -- 172.21.15.83:0/97983970 >> 172.21.15.83:0/97983970 conn(0x5651090a5b80 msgr2=0x5651090ab980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:25.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.590+0000 7f62159e1740 1 -- 172.21.15.83:0/97983970 shutdown_connections 2024-04-23T14:56:25.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.590+0000 7f62159e1740 1 -- 172.21.15.83:0/97983970 wait complete. 2024-04-23T14:56:25.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.590+0000 7f62159e1740 20 client.26837 trim_cache size 0 max 16384 2024-04-23T14:56:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.609+0000 7f62159e1740 1 Processor -- start 2024-04-23T14:56:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.609+0000 7f62159e1740 1 -- start start 2024-04-23T14:56:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.609+0000 7f62159e1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 0x5651090c0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.609+0000 7f62159e1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090aa180 0x5651090abb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.609+0000 7f62159e1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.609+0000 7f62159e1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1b2810 con 0x5651090aa180 2024-04-23T14:56:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.609+0000 7f62159e1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1b00e0 con 0x5651090a9860 2024-04-23T14:56:25.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.609+0000 7f62159e1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1adf50 con 0x5651090b4850 2024-04-23T14:56:25.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f620be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 0x5651090c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f620b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090aa180 0x5651090abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f620be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 0x5651090c0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:39224/0 (socket says 172.21.15.83:39224) 2024-04-23T14:56:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f620be00640 1 -- 172.21.15.83:0/3148625814 learned_addr learned my addr 172.21.15.83:0/3148625814 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f6210a00640 1 --2- 172.21.15.83:0/3148625814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f620be00640 1 -- 172.21.15.83:0/3148625814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 msgr2=0x5651090b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f620be00640 1 --2- 172.21.15.83:0/3148625814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f620be00640 1 -- 172.21.15.83:0/3148625814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090aa180 msgr2=0x5651090abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f620be00640 1 --2- 172.21.15.83:0/3148625814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090aa180 0x5651090abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f620be00640 1 -- 172.21.15.83:0/3148625814 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651090a9fb0 con 0x5651090a9860 2024-04-23T14:56:25.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f620be00640 1 --2- 172.21.15.83:0/3148625814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 0x5651090c0500 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f61fc005190 tx=0x7f61fc009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3d6935ea97cec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f61fbe00640 1 -- 172.21.15.83:0/3148625814 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc026040 con 0x5651090a9860 2024-04-23T14:56:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f61fbe00640 1 -- 172.21.15.83:0/3148625814 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61fc016030 con 0x5651090a9860 2024-04-23T14:56:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.610+0000 7f61fbe00640 1 -- 172.21.15.83:0/3148625814 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc019870 con 0x5651090a9860 2024-04-23T14:56:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.611+0000 7f62159e1740 1 -- 172.21.15.83:0/3148625814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 msgr2=0x5651090c0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.611+0000 7f62159e1740 1 --2- 172.21.15.83:0/3148625814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 0x5651090c0500 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f61fc005190 tx=0x7f61fc009500 comp rx=0 tx=0).stop 2024-04-23T14:56:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.611+0000 7f62159e1740 1 -- 172.21.15.83:0/3148625814 shutdown_connections 2024-04-23T14:56:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.611+0000 7f62159e1740 1 --2- 172.21.15.83:0/3148625814 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.611+0000 7f62159e1740 1 --2- 172.21.15.83:0/3148625814 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090aa180 0x5651090abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.611+0000 7f62159e1740 1 --2- 172.21.15.83:0/3148625814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090a9860 0x5651090c0500 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.611+0000 7f62159e1740 1 -- 172.21.15.83:0/3148625814 >> 172.21.15.83:0/3148625814 conn(0x5651090a5b80 msgr2=0x5651090b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.611+0000 7f62159e1740 1 -- 172.21.15.83:0/3148625814 shutdown_connections 2024-04-23T14:56:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.611+0000 7f62159e1740 1 -- 172.21.15.83:0/3148625814 wait complete. 2024-04-23T14:56:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.612+0000 7f62159e1740 1 -- start start 2024-04-23T14:56:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.612+0000 7f62159e1740 1 Processor -- start 2024-04-23T14:56:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.612+0000 7f62159e1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090a9860 0x5651090ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.612+0000 7f62159e1740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090aa180 0x5651090aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.612+0000 7f62159e1740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090af010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.612+0000 7f62159e1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1ae5a0 con 0x5651090a9860 2024-04-23T14:56:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.612+0000 7f62159e1740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1afa10 con 0x5651090aa180 2024-04-23T14:56:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.612+0000 7f62159e1740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f620c1aec70 con 0x5651090b4850 2024-04-23T14:56:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.612+0000 7f62159e1740 10 client.0 _ll_register_callbacks cb 0x7f61fc009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f620b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090aa180 0x5651090aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f620be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090a9860 0x5651090ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f620b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090aa180 0x5651090aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:39234/0 (socket says 172.21.15.83:39234) 2024-04-23T14:56:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f620b400640 1 -- 172.21.15.83:0/269373543 learned_addr learned my addr 172.21.15.83:0/269373543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:25.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f6210a00640 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f620b400640 1 -- 172.21.15.83:0/269373543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 msgr2=0x5651090af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f620b400640 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f620b400640 1 -- 172.21.15.83:0/269373543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090a9860 msgr2=0x5651090ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f620b400640 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090a9860 0x5651090ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f620b400640 1 -- 172.21.15.83:0/269373543 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565108d213f0 con 0x5651090aa180 2024-04-23T14:56:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f620be00640 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090a9860 0x5651090ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f620b400640 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090aa180 0x5651090aead0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5651090a6d10 tx=0x7f61f4009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=428a9bba29c27c9e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f61f9600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:56:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f61f9600640 1 -- 172.21.15.83:0/269373543 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f400fcd0 con 0x5651090aa180 2024-04-23T14:56:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f61f9600640 1 -- 172.21.15.83:0/269373543 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f61f4005020 con 0x5651090aa180 2024-04-23T14:56:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.613+0000 7f62159e1740 1 -- 172.21.15.83:0/269373543 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61fc016030 con 0x5651090aa180 2024-04-23T14:56:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.614+0000 7f61f9600640 1 -- 172.21.15.83:0/269373543 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f61f400fe70 con 0x5651090aa180 2024-04-23T14:56:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.614+0000 7f62159e1740 10 client.26843 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:25.620 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.614+0000 7f62159e1740 1 -- 172.21.15.83:0/269373543 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565108ce10f0 con 0x5651090aa180 2024-04-23T14:56:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.614+0000 7f61f9600640 1 -- 172.21.15.83:0/269373543 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61fc016030 con 0x5651090aa180 2024-04-23T14:56:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.614+0000 7f61f9600640 1 -- 172.21.15.83:0/269373543 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f61f402c2d0 con 0x5651090aa180 2024-04-23T14:56:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.614+0000 7f62159e1740 10 client.26843 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.614+0000 7f62159e1740 10 client.26843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.614+0000 7f62159e1740 20 client.26843 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.614+0000 7f62159e1740 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56510912b790 0x56510912dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.614+0000 7f62159e1740 4 client.26843 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:56:25.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.614+0000 7f62159e1740 1 -- 172.21.15.83:0/269373543 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565109016a80 con 0x56510912b790 2024-04-23T14:56:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f620be00640 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56510912b790 0x56510912dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f620be00640 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56510912b790 0x56510912dbb0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47dcd398c648573 server_cookie=a102493eb626d9e9 in_seq=0 out_seq=0 2024-04-23T14:56:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f61f9600640 10 client.26843 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f61f9600640 1 -- 172.21.15.83:0/269373543 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x565109016a80 con 0x56510912b790 2024-04-23T14:56:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f61f9600640 10 client.26843 handle_command_reply: tid=0 2024-04-23T14:56:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f62159e1740 1 client.26843 shutdown 2024-04-23T14:56:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f62159e1740 1 -- 172.21.15.83:0/269373543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090aa180 msgr2=0x5651090aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f62159e1740 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090aa180 0x5651090aead0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5651090a6d10 tx=0x7f61f4009770 comp rx=0 tx=0).stop 2024-04-23T14:56:25.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f62159e1740 1 -- 172.21.15.83:0/269373543 shutdown_connections 2024-04-23T14:56:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f62159e1740 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x56510912b790 0x56510912dbb0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f62159e1740 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5651090b4850 0x5651090af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f62159e1740 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5651090aa180 0x5651090aead0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f61f9600640 0 client.26843 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f62159e1740 1 --2- 172.21.15.83:0/269373543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5651090a9860 0x5651090ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.615+0000 7f62159e1740 1 -- 172.21.15.83:0/269373543 >> 172.21.15.83:0/269373543 conn(0x5651090a5b80 msgr2=0x5651090b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.616+0000 7f62159e1740 1 -- 172.21.15.83:0/269373543 shutdown_connections 2024-04-23T14:56:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.616+0000 7f62159e1740 1 -- 172.21.15.83:0/269373543 wait complete. 2024-04-23T14:56:25.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.616+0000 7f62159e1740 20 client.26843 trim_cache size 0 max 16384 2024-04-23T14:56:25.624 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:56:25.624 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:56:25.624 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:56:25.624 INFO:teuthology.orchestra.run.smithi083.stdout: "026119be-7cf6-43ec-91bd-2a55148a855d": { 2024-04-23T14:56:25.624 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:56:25.624 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "026119be-7cf6-43ec-91bd-2a55148a855d", 2024-04-23T14:56:25.625 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:56:25.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:56:25.625 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:56:25.625 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:56:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.616+0000 7f6212a00640 1 -- 172.21.15.83:0/1260166374 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f61f00a3d20 msgr2=0x7f61f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.616+0000 7f6212a00640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f61f00a3d20 0x7f61f00a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009fd0 tx=0x7f61f4015040 comp rx=0 tx=0).stop 2024-04-23T14:56:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.616+0000 7f6212a00640 1 -- 172.21.15.83:0/1260166374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f620c10e8e0 msgr2=0x7f620c19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.616+0000 7f6212a00640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f620c10e8e0 0x7f620c19a910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6204009230 tx=0x7f6204003b90 comp rx=0 tx=0).stop 2024-04-23T14:56:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.617+0000 7f6212a00640 1 -- 172.21.15.83:0/1260166374 shutdown_connections 2024-04-23T14:56:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.617+0000 7f6212a00640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f620c10e8e0 0x7f620c19a910 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.617+0000 7f6212a00640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f620c074bf0 0x7f620c1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.617+0000 7f6212a00640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f61f00a3d20 0x7f61f00a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.617+0000 7f6212a00640 1 --2- 172.21.15.83:0/1260166374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620c0742d0 0x7f620c1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:25.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.617+0000 7f6212a00640 1 -- 172.21.15.83:0/1260166374 >> 172.21.15.83:0/1260166374 conn(0x7f620c079450 msgr2=0x7f620c0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:25.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.617+0000 7f6212a00640 1 -- 172.21.15.83:0/1260166374 shutdown_connections 2024-04-23T14:56:25.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:25.617+0000 7f6212a00640 1 -- 172.21.15.83:0/1260166374 wait complete. 2024-04-23T14:56:25.635 INFO:tasks.cephfs.filesystem:scrub status for tag:026119be-7cf6-43ec-91bd-2a55148a855d - {'path': '/', 'tag': '026119be-7cf6-43ec-91bd-2a55148a855d', 'options': 'recursive,force'} 2024-04-23T14:56:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:25 smithi203 ceph-mon[32800]: pgmap v246: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:56:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:56:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:26.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:26.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:26.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:26.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:26.577 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.578 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:26 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:56:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:26.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:26.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:26.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:26.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:26.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:26.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:26.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:26 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:56:26.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:26.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:26.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:26.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:26.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:26.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:26.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:26.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:26.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:26 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:27.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:27 smithi173 ceph-mon[32443]: pgmap v247: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-23T14:56:27.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:27 smithi203 ceph-mon[32800]: pgmap v247: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-23T14:56:27.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:27 smithi083 ceph-mon[27867]: pgmap v247: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-23T14:56:29.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:29 smithi203 ceph-mon[32800]: pgmap v248: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:56:29.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:29 smithi083 ceph-mon[27867]: pgmap v248: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:56:29.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:29 smithi173 ceph-mon[32443]: pgmap v248: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:56:31.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:31 smithi203 ceph-mon[32800]: pgmap v249: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:56:31.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:31 smithi083 ceph-mon[27867]: pgmap v249: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:56:31.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:31 smithi173 ceph-mon[32443]: pgmap v249: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:56:33.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:33 smithi203 ceph-mon[32800]: pgmap v250: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:56:33.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:33 smithi083 ceph-mon[27867]: pgmap v250: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:56:33.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:33 smithi173 ceph-mon[32443]: pgmap v250: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:56:35.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:35 smithi203 ceph-mon[32800]: pgmap v251: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:56:35.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:35 smithi083 ceph-mon[27867]: pgmap v251: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:56:35.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:35 smithi173 ceph-mon[32443]: pgmap v251: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:56:37.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:37 smithi203 ceph-mon[32800]: pgmap v252: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T14:56:37.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:37 smithi083 ceph-mon[27867]: pgmap v252: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T14:56:37.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:37 smithi173 ceph-mon[32443]: pgmap v252: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T14:56:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:39 smithi203 ceph-mon[32800]: pgmap v253: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T14:56:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:39 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:56:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:39 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:56:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:39 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:56:39.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:39 smithi083 ceph-mon[27867]: pgmap v253: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T14:56:39.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:39 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:56:39.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:39 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:56:39.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:39 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:56:39.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:39 smithi173 ceph-mon[32443]: pgmap v253: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T14:56:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:39 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:56:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:39 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:56:39.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:39 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:56:41.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:41 smithi203 ceph-mon[32800]: pgmap v254: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:56:41.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:41 smithi083 ceph-mon[27867]: pgmap v254: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:56:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:41 smithi173 ceph-mon[32443]: pgmap v254: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:56:43.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:43 smithi203 ceph-mon[32800]: pgmap v255: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:56:43.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:43 smithi083 ceph-mon[27867]: pgmap v255: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:56:43.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:43 smithi173 ceph-mon[32443]: pgmap v255: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:56:45.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:45 smithi203 ceph-mon[32800]: pgmap v256: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:56:45.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:45 smithi083 ceph-mon[27867]: pgmap v256: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:56:45.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:45 smithi173 ceph-mon[32443]: pgmap v256: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:56:47.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:47 smithi203 ceph-mon[32800]: pgmap v257: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:56:47.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:47 smithi083 ceph-mon[27867]: pgmap v257: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:56:47.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:47 smithi173 ceph-mon[32443]: pgmap v257: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:56:49.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:49 smithi203 ceph-mon[32800]: pgmap v258: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:56:49.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:49 smithi083 ceph-mon[27867]: pgmap v258: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:56:49.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:49 smithi173 ceph-mon[32443]: pgmap v258: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:56:51.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:51 smithi203 ceph-mon[32800]: pgmap v259: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:56:51.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:51 smithi083 ceph-mon[27867]: pgmap v259: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:56:51.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:51 smithi173 ceph-mon[32443]: pgmap v259: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:56:52.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:52 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:56:52.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:52 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:56:52.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:52 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:56:53.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:53 smithi203 ceph-mon[32800]: pgmap v260: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:56:53.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:53 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:56:53.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:53 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:56:53.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:53 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:56:53.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:53 smithi173 ceph-mon[32443]: pgmap v260: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:56:53.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:53 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:56:53.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:53 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:56:53.726 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:53 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:56:53.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:53 smithi083 ceph-mon[27867]: pgmap v260: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:56:53.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:53 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:56:53.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:53 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:56:53.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:53 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:56:55.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:56:55.636 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:56:55.711 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:55 smithi083 ceph-mon[27867]: pgmap v261: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:56:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:55 smithi203 ceph-mon[32800]: pgmap v261: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:56:55.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:55 smithi173 ceph-mon[32443]: pgmap v261: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:56:55.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.848+0000 7fcd0be00640 1 -- 172.21.15.83:0/1046981431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c10c4d0 msgr2=0x7fcd0c10e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:55.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.848+0000 7fcd0be00640 1 --2- 172.21.15.83:0/1046981431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c10c4d0 0x7fcd0c10e920 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009be0 tx=0x7fcd0402a750 comp rx=0 tx=0).stop 2024-04-23T14:56:55.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.848+0000 7fcd0be00640 1 -- 172.21.15.83:0/1046981431 shutdown_connections 2024-04-23T14:56:55.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.848+0000 7fcd0be00640 1 --2- 172.21.15.83:0/1046981431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c10c4d0 0x7fcd0c10e920 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.848+0000 7fcd0be00640 1 --2- 172.21.15.83:0/1046981431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd0c103a20 0x7fcd0c10bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.848+0000 7fcd0be00640 1 --2- 172.21.15.83:0/1046981431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd0c103100 0x7fcd0c1034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.848+0000 7fcd0be00640 1 -- 172.21.15.83:0/1046981431 >> 172.21.15.83:0/1046981431 conn(0x7fcd0c0fc880 msgr2=0x7fcd0c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:55.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.848+0000 7fcd0be00640 1 -- 172.21.15.83:0/1046981431 shutdown_connections 2024-04-23T14:56:55.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.848+0000 7fcd0be00640 1 -- 172.21.15.83:0/1046981431 wait complete. 2024-04-23T14:56:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.850+0000 7fcd0be00640 1 Processor -- start 2024-04-23T14:56:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.850+0000 7fcd0be00640 1 -- start start 2024-04-23T14:56:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.850+0000 7fcd0be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd0c103100 0x7fcd0c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.850+0000 7fcd0be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd0c103a20 0x7fcd0c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.850+0000 7fcd0be00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c10c4d0 0x7fcd0c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.850+0000 7fcd0be00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c112250 con 0x7fcd0c103a20 2024-04-23T14:56:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.850+0000 7fcd0be00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1120d0 con 0x7fcd0c10c4d0 2024-04-23T14:56:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.850+0000 7fcd0be00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1123d0 con 0x7fcd0c103100 2024-04-23T14:56:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.851+0000 7fcd02000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd0c103a20 0x7fcd0c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.851+0000 7fcd0aa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd0c103100 0x7fcd0c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.851+0000 7fcd02000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd0c103a20 0x7fcd0c1a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58104/0 (socket says 172.21.15.83:58104) 2024-04-23T14:56:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.851+0000 7fcd02000640 1 -- 172.21.15.83:0/2510176711 learned_addr learned my addr 172.21.15.83:0/2510176711 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.851+0000 7fcd0b400640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c10c4d0 0x7fcd0c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.851+0000 7fcd02000640 1 -- 172.21.15.83:0/2510176711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd0c103100 msgr2=0x7fcd0c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:55.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.851+0000 7fcd02000640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd0c103100 0x7fcd0c1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.851+0000 7fcd02000640 1 -- 172.21.15.83:0/2510176711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c10c4d0 msgr2=0x7fcd0c19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:55.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.851+0000 7fcd02000640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c10c4d0 0x7fcd0c19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.851+0000 7fcd02000640 1 -- 172.21.15.83:0/2510176711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0c19b070 con 0x7fcd0c103a20 2024-04-23T14:56:55.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.851+0000 7fcd02000640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd0c103a20 0x7fcd0c1a0b90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fccf400e9f0 tx=0x7fccf400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:55.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.852+0000 7fcd03e00640 1 -- 172.21.15.83:0/2510176711 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf400cd80 con 0x7fcd0c103a20 2024-04-23T14:56:55.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.852+0000 7fcd0be00640 1 -- 172.21.15.83:0/2510176711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0c19b360 con 0x7fcd0c103a20 2024-04-23T14:56:55.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.852+0000 7fcd03e00640 1 -- 172.21.15.83:0/2510176711 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccf4005500 con 0x7fcd0c103a20 2024-04-23T14:56:55.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.852+0000 7fcd03e00640 1 -- 172.21.15.83:0/2510176711 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4010690 con 0x7fcd0c103a20 2024-04-23T14:56:55.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.852+0000 7fcd0be00640 1 -- 172.21.15.83:0/2510176711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd0c1a7510 con 0x7fcd0c103a20 2024-04-23T14:56:55.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.856+0000 7fcd149a2740 1 Processor -- start 2024-04-23T14:56:55.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.856+0000 7fcd149a2740 1 -- start start 2024-04-23T14:56:55.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd03e00640 1 -- 172.21.15.83:0/2510176711 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fccf40046c0 con 0x7fcd0c103a20 2024-04-23T14:56:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd149a2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 0x55d988ddab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd149a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988ddb130 0x55d988dedcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd149a2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dee230 0x55d988df0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd149a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d988d401f0 con 0x55d988ddb130 2024-04-23T14:56:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd149a2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d988ce6350 con 0x55d988dee230 2024-04-23T14:56:55.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd149a2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d988d3c1b0 con 0x55d988dda780 2024-04-23T14:56:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd0aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988ddb130 0x55d988dedcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd0b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 0x55d988ddab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd02000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dee230 0x55d988df0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd0aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988ddb130 0x55d988dedcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58106/0 (socket says 172.21.15.83:58106) 2024-04-23T14:56:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd0b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 0x55d988ddab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59564/0 (socket says 172.21.15.83:59564) 2024-04-23T14:56:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd0aa00640 1 -- 172.21.15.83:0/1834616745 learned_addr learned my addr 172.21.15.83:0/1834616745 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd0b400640 1 -- 172.21.15.83:0/1834616745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dee230 msgr2=0x55d988df0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:55.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd0b400640 1 --2- 172.21.15.83:0/1834616745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dee230 0x55d988df0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd0b400640 1 -- 172.21.15.83:0/1834616745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988ddb130 msgr2=0x55d988dedcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:55.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd0b400640 1 --2- 172.21.15.83:0/1834616745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988ddb130 0x55d988dedcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.857+0000 7fcd0b400640 1 -- 172.21.15.83:0/1834616745 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d988d3af50 con 0x55d988dda780 2024-04-23T14:56:55.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.858+0000 7fcd02000640 1 --2- 172.21.15.83:0/1834616745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dee230 0x55d988df0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:55.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.858+0000 7fcd0b400640 1 --2- 172.21.15.83:0/1834616745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 0x55d988ddab60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009b30 tx=0x7fcd04030aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80b7e89ed134a8e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:55.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.858+0000 7fccfbe00640 1 -- 172.21.15.83:0/1834616745 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04041030 con 0x55d988dda780 2024-04-23T14:56:55.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.858+0000 7fccfbe00640 1 -- 172.21.15.83:0/1834616745 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd04030c50 con 0x55d988dda780 2024-04-23T14:56:55.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.858+0000 7fccfbe00640 1 -- 172.21.15.83:0/1834616745 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04036040 con 0x55d988dda780 2024-04-23T14:56:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.858+0000 7fcd03e00640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fccf00a3d20 0x7fccf00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.859+0000 7fcd03e00640 1 -- 172.21.15.83:0/2510176711 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fccf40d0bf0 con 0x7fcd0c103a20 2024-04-23T14:56:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.859+0000 7fcd0aa00640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fccf00a3d20 0x7fccf00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.859+0000 7fcd149a2740 1 -- 172.21.15.83:0/1834616745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 msgr2=0x55d988ddab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.859+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1834616745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 0x55d988ddab60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009b30 tx=0x7fcd04030aa0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.859+0000 7fcd149a2740 1 -- 172.21.15.83:0/1834616745 shutdown_connections 2024-04-23T14:56:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.859+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1834616745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dee230 0x55d988df0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.859+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1834616745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988ddb130 0x55d988dedcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.859+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1834616745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 0x55d988ddab60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.859+0000 7fcd149a2740 1 -- 172.21.15.83:0/1834616745 >> 172.21.15.83:0/1834616745 conn(0x55d988dd6d50 msgr2=0x55d988dd7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.859+0000 7fcd149a2740 1 -- 172.21.15.83:0/1834616745 shutdown_connections 2024-04-23T14:56:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.859+0000 7fcd149a2740 1 -- 172.21.15.83:0/1834616745 wait complete. 2024-04-23T14:56:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.860+0000 7fcd0aa00640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fccf00a3d20 0x7fccf00a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fccfc009e40 tx=0x7fccfc009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.860+0000 7fcd149a2740 1 -- start start 2024-04-23T14:56:55.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.860+0000 7fcd149a2740 1 Processor -- start 2024-04-23T14:56:55.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd149a2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 0x55d988deb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd149a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988ddb130 0x55d988deb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd149a2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dee230 0x55d988debbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd149a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d988df0f90 con 0x55d988ddb130 2024-04-23T14:56:55.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd149a2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d988cf9690 con 0x55d988dee230 2024-04-23T14:56:55.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd149a2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d988df1110 con 0x55d988dda780 2024-04-23T14:56:55.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd149a2740 10 client.0 _ll_register_callbacks cb 0x55d988d35650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:55.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd0aa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 0x55d988deb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd02000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988ddb130 0x55d988deb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd0aa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 0x55d988deb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59574/0 (socket says 172.21.15.83:59574) 2024-04-23T14:56:55.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd0aa00640 1 -- 172.21.15.83:0/1885966629 learned_addr learned my addr 172.21.15.83:0/1885966629 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:55.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd0b400640 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dee230 0x55d988debbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd0aa00640 1 -- 172.21.15.83:0/1885966629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dee230 msgr2=0x55d988debbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:55.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd0aa00640 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dee230 0x55d988debbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd0aa00640 1 -- 172.21.15.83:0/1885966629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988ddb130 msgr2=0x55d988deb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:55.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd0aa00640 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988ddb130 0x55d988deb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd0aa00640 1 -- 172.21.15.83:0/1885966629 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d988cf8920 con 0x55d988dda780 2024-04-23T14:56:55.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.861+0000 7fcd0b400640 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dee230 0x55d988debbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:56:55.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.862+0000 7fcd0aa00640 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 0x55d988deb150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fccfc01d9c0 tx=0x7fccfc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=314b0b852697194e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:55.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.862+0000 7fccf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:56:55.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.862+0000 7fccf8c00640 1 -- 172.21.15.83:0/1885966629 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc02d060 con 0x55d988dda780 2024-04-23T14:56:55.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.862+0000 7fcd149a2740 1 -- 172.21.15.83:0/1885966629 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd04030c50 con 0x55d988dda780 2024-04-23T14:56:55.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.862+0000 7fccf8c00640 1 -- 172.21.15.83:0/1885966629 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fccfc018030 con 0x55d988dda780 2024-04-23T14:56:55.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.862+0000 7fccf8c00640 1 -- 172.21.15.83:0/1885966629 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fccfc031420 con 0x55d988dda780 2024-04-23T14:56:55.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.862+0000 7fcd149a2740 10 client.26830 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:55.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.862+0000 7fcd149a2740 1 -- 172.21.15.83:0/1885966629 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d988b1c7d0 con 0x55d988dda780 2024-04-23T14:56:55.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.863+0000 7fccf8c00640 1 -- 172.21.15.83:0/1885966629 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd04030c50 con 0x55d988dda780 2024-04-23T14:56:55.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.863+0000 7fccf8c00640 1 -- 172.21.15.83:0/1885966629 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fccfc030580 con 0x55d988dda780 2024-04-23T14:56:55.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.863+0000 7fcd149a2740 10 client.26830 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:55.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.863+0000 7fcd149a2740 10 client.26830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:55.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.863+0000 7fcd149a2740 20 client.26830 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:55.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.863+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d988dfe600 0x55d988e00a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.863+0000 7fcd149a2740 4 client.26830 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:56:55.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.863+0000 7fcd149a2740 1 -- 172.21.15.83:0/1885966629 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d988d4dc10 con 0x55d988dfe600 2024-04-23T14:56:55.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.863+0000 7fcd02000640 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d988dfe600 0x55d988e00a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.864+0000 7fcd02000640 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d988dfe600 0x55d988e00a80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88927e827117b0db server_cookie=4c5b08ffa7ecad46 in_seq=0 out_seq=0 2024-04-23T14:56:55.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.864+0000 7fccf8c00640 10 client.26830 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:55.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fccf8c00640 1 -- 172.21.15.83:0/1885966629 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d988d4dc10 con 0x55d988dfe600 2024-04-23T14:56:55.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fccf8c00640 10 client.26830 handle_command_reply: tid=0 2024-04-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fcd149a2740 1 client.26830 shutdown 2024-04-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fcd149a2740 1 -- 172.21.15.83:0/1885966629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 msgr2=0x55d988deb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 0x55d988deb150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fccfc01d9c0 tx=0x7fccfc004730 comp rx=0 tx=0).stop 2024-04-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fcd149a2740 1 -- 172.21.15.83:0/1885966629 shutdown_connections 2024-04-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d988dfe600 0x55d988e00a80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dee230 0x55d988debbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fccf8c00640 0 client.26830 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988ddb130 0x55d988deb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1885966629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dda780 0x55d988deb150 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fcd149a2740 1 -- 172.21.15.83:0/1885966629 >> 172.21.15.83:0/1885966629 conn(0x55d988dd6d50 msgr2=0x55d988ddcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:55.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fcd149a2740 1 -- 172.21.15.83:0/1885966629 shutdown_connections 2024-04-23T14:56:55.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fcd149a2740 1 -- 172.21.15.83:0/1885966629 wait complete. 2024-04-23T14:56:55.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.865+0000 7fcd149a2740 20 client.26830 trim_cache size 0 max 16384 2024-04-23T14:56:55.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.884+0000 7fcd149a2740 1 Processor -- start 2024-04-23T14:56:55.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.885+0000 7fcd149a2740 1 -- start start 2024-04-23T14:56:55.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.885+0000 7fcd149a2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dda780 0x55d988df9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.885+0000 7fcd149a2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988ddb130 0x55d988e06350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.885+0000 7fcd149a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988dee230 0x55d988e06890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.885+0000 7fcd149a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1ae240 con 0x55d988dee230 2024-04-23T14:56:55.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.885+0000 7fcd149a2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1abb10 con 0x55d988dda780 2024-04-23T14:56:55.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.885+0000 7fcd149a2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1a99c0 con 0x55d988ddb130 2024-04-23T14:56:55.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.885+0000 7fcd02000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988ddb130 0x55d988e06350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.885+0000 7fcd0aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dda780 0x55d988df9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.885+0000 7fcd0b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988dee230 0x55d988e06890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.885+0000 7fcd02000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988ddb130 0x55d988e06350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59588/0 (socket says 172.21.15.83:59588) 2024-04-23T14:56:55.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.885+0000 7fcd02000640 1 -- 172.21.15.83:0/3127163753 learned_addr learned my addr 172.21.15.83:0/3127163753 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:55.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.886+0000 7fcd0aa00640 1 -- 172.21.15.83:0/3127163753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988ddb130 msgr2=0x55d988e06350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:55.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.886+0000 7fcd0aa00640 1 --2- 172.21.15.83:0/3127163753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988ddb130 0x55d988e06350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.886+0000 7fcd0aa00640 1 -- 172.21.15.83:0/3127163753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988dee230 msgr2=0x55d988e06890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:55.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.886+0000 7fcd0aa00640 1 --2- 172.21.15.83:0/3127163753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988dee230 0x55d988e06890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.886+0000 7fcd0aa00640 1 -- 172.21.15.83:0/3127163753 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d988a523f0 con 0x55d988dda780 2024-04-23T14:56:55.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.886+0000 7fcd02000640 1 --2- 172.21.15.83:0/3127163753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988ddb130 0x55d988e06350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:55.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.886+0000 7fcd0aa00640 1 --2- 172.21.15.83:0/3127163753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dda780 0x55d988df9e90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fccfc009530 tx=0x7fccfc004680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62602af7a5cabf32 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:55.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.886+0000 7fccfb400640 1 -- 172.21.15.83:0/3127163753 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc03f070 con 0x55d988dda780 2024-04-23T14:56:55.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.886+0000 7fccfb400640 1 -- 172.21.15.83:0/3127163753 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccfc030c60 con 0x55d988dda780 2024-04-23T14:56:55.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.886+0000 7fccfb400640 1 -- 172.21.15.83:0/3127163753 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc031b90 con 0x55d988dda780 2024-04-23T14:56:55.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.887+0000 7fcd149a2740 1 -- 172.21.15.83:0/3127163753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dda780 msgr2=0x55d988df9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:55.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.887+0000 7fcd149a2740 1 --2- 172.21.15.83:0/3127163753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dda780 0x55d988df9e90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fccfc009530 tx=0x7fccfc004680 comp rx=0 tx=0).stop 2024-04-23T14:56:55.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.887+0000 7fcd149a2740 1 -- 172.21.15.83:0/3127163753 shutdown_connections 2024-04-23T14:56:55.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.887+0000 7fcd149a2740 1 --2- 172.21.15.83:0/3127163753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988dee230 0x55d988e06890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.887+0000 7fcd149a2740 1 --2- 172.21.15.83:0/3127163753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988ddb130 0x55d988e06350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.887+0000 7fcd149a2740 1 --2- 172.21.15.83:0/3127163753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988dda780 0x55d988df9e90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.887+0000 7fcd149a2740 1 -- 172.21.15.83:0/3127163753 >> 172.21.15.83:0/3127163753 conn(0x55d988dd6d50 msgr2=0x55d988ddc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.887+0000 7fcd149a2740 1 -- 172.21.15.83:0/3127163753 shutdown_connections 2024-04-23T14:56:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.887+0000 7fcd149a2740 1 -- 172.21.15.83:0/3127163753 wait complete. 2024-04-23T14:56:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.887+0000 7fcd149a2740 1 -- start start 2024-04-23T14:56:55.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.888+0000 7fcd149a2740 1 Processor -- start 2024-04-23T14:56:55.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.888+0000 7fcd149a2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988dda780 0x55d988df6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.888+0000 7fcd149a2740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988ddb130 0x55d988df18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.888+0000 7fcd149a2740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dee230 0x55d988df1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.888+0000 7fcd149a2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1aa010 con 0x55d988dda780 2024-04-23T14:56:55.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.888+0000 7fcd149a2740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1ab440 con 0x55d988ddb130 2024-04-23T14:56:55.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.888+0000 7fcd149a2740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1aa6a0 con 0x55d988dee230 2024-04-23T14:56:55.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.888+0000 7fcd149a2740 10 client.0 _ll_register_callbacks cb 0x7fccfc009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:55.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.888+0000 7fcd0aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988dda780 0x55d988df6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.888+0000 7fcd0aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988dda780 0x55d988df6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58122/0 (socket says 172.21.15.83:58122) 2024-04-23T14:56:55.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.888+0000 7fcd0aa00640 1 -- 172.21.15.83:0/1633894685 learned_addr learned my addr 172.21.15.83:0/1633894685 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:55.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.888+0000 7fcd0b400640 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dee230 0x55d988df1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.889+0000 7fcd02000640 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988ddb130 0x55d988df18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.889+0000 7fcd0aa00640 1 -- 172.21.15.83:0/1633894685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dee230 msgr2=0x55d988df1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:55.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.889+0000 7fcd0aa00640 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dee230 0x55d988df1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.889+0000 7fcd0aa00640 1 -- 172.21.15.83:0/1633894685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988ddb130 msgr2=0x55d988df18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:55.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.889+0000 7fcd0aa00640 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988ddb130 0x55d988df18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.889+0000 7fcd0aa00640 1 -- 172.21.15.83:0/1633894685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d988a120f0 con 0x55d988dda780 2024-04-23T14:56:55.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.889+0000 7fcd0aa00640 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988dda780 0x55d988df6520 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fccfc000c00 tx=0x7fccfc005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f20c55ce4785786 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:55.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.889+0000 7fccf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:55.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.889+0000 7fccf8c00640 1 -- 172.21.15.83:0/1633894685 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc03f070 con 0x55d988dda780 2024-04-23T14:56:55.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.889+0000 7fccf8c00640 1 -- 172.21.15.83:0/1633894685 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fccfc030580 con 0x55d988dda780 2024-04-23T14:56:55.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.889+0000 7fcd149a2740 1 -- 172.21.15.83:0/1633894685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccfc030c60 con 0x55d988dda780 2024-04-23T14:56:55.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.890+0000 7fccf8c00640 1 -- 172.21.15.83:0/1633894685 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fccfc0307e0 con 0x55d988dda780 2024-04-23T14:56:55.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.890+0000 7fcd149a2740 10 client.16662 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:55.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.890+0000 7fcd149a2740 1 -- 172.21.15.83:0/1633894685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d988d05810 con 0x55d988dda780 2024-04-23T14:56:55.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.890+0000 7fccf8c00640 1 -- 172.21.15.83:0/1633894685 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccfc030c60 con 0x55d988dda780 2024-04-23T14:56:55.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.890+0000 7fccf8c00640 1 -- 172.21.15.83:0/1633894685 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fccfc0315c0 con 0x55d988dda780 2024-04-23T14:56:55.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.890+0000 7fcd149a2740 10 client.16662 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:55.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.890+0000 7fcd149a2740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:55.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.890+0000 7fcd149a2740 20 client.16662 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:55.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.890+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d988e64c30 0x55d988e67050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:55.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.890+0000 7fcd149a2740 4 client.16662 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:56:55.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.890+0000 7fcd149a2740 1 -- 172.21.15.83:0/1633894685 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d988d47a80 con 0x55d988e64c30 2024-04-23T14:56:55.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.891+0000 7fcd02000640 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d988e64c30 0x55d988e67050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:55.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.891+0000 7fcd02000640 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d988e64c30 0x55d988e67050 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=212d3c48bced2bc9 server_cookie=dcfbfc42d2f0a788 in_seq=0 out_seq=0 2024-04-23T14:56:55.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.891+0000 7fccf8c00640 10 client.16662 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:55.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.891+0000 7fccf8c00640 1 -- 172.21.15.83:0/1633894685 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d988d47a80 con 0x55d988e64c30 2024-04-23T14:56:55.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.891+0000 7fccf8c00640 10 client.16662 handle_command_reply: tid=0 2024-04-23T14:56:55.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.891+0000 7fcd149a2740 1 client.16662 shutdown 2024-04-23T14:56:55.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.891+0000 7fcd149a2740 1 -- 172.21.15.83:0/1633894685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988dda780 msgr2=0x55d988df6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:55.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.891+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988dda780 0x55d988df6520 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fccfc000c00 tx=0x7fccfc005860 comp rx=0 tx=0).stop 2024-04-23T14:56:55.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.891+0000 7fcd149a2740 1 -- 172.21.15.83:0/1633894685 shutdown_connections 2024-04-23T14:56:55.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.891+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55d988e64c30 0x55d988e67050 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.892+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d988dee230 0x55d988df1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.892+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d988ddb130 0x55d988df18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.892+0000 7fcd149a2740 1 --2- 172.21.15.83:0/1633894685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d988dda780 0x55d988df6520 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.892+0000 7fccf8c00640 0 client.16662 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.892+0000 7fcd149a2740 1 -- 172.21.15.83:0/1633894685 >> 172.21.15.83:0/1633894685 conn(0x55d988dd6d50 msgr2=0x55d988ddcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.892+0000 7fcd149a2740 1 -- 172.21.15.83:0/1633894685 shutdown_connections 2024-04-23T14:56:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.892+0000 7fcd149a2740 1 -- 172.21.15.83:0/1633894685 wait complete. 2024-04-23T14:56:55.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.892+0000 7fcd149a2740 20 client.16662 trim_cache size 0 max 16384 2024-04-23T14:56:55.901 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:56:55.901 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:56:55.901 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:56:55.901 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:56:55.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.892+0000 7fcd0be00640 1 -- 172.21.15.83:0/2510176711 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fccf00a3d20 msgr2=0x7fccf00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:55.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.892+0000 7fcd0be00640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fccf00a3d20 0x7fccf00a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fccfc009e40 tx=0x7fccfc009260 comp rx=0 tx=0).stop 2024-04-23T14:56:55.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.892+0000 7fcd0be00640 1 -- 172.21.15.83:0/2510176711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd0c103a20 msgr2=0x7fcd0c1a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:55.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.892+0000 7fcd0be00640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd0c103a20 0x7fcd0c1a0b90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fccf400e9f0 tx=0x7fccf400edf0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.893+0000 7fcd0be00640 1 -- 172.21.15.83:0/2510176711 shutdown_connections 2024-04-23T14:56:55.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.893+0000 7fcd0be00640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcd0c10c4d0 0x7fcd0c19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.893+0000 7fcd0be00640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fccf00a3d20 0x7fccf00a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.893+0000 7fcd0be00640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd0c103a20 0x7fcd0c1a0b90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.893+0000 7fcd0be00640 1 --2- 172.21.15.83:0/2510176711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd0c103100 0x7fcd0c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:55.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.893+0000 7fcd0be00640 1 -- 172.21.15.83:0/2510176711 >> 172.21.15.83:0/2510176711 conn(0x7fcd0c0fc880 msgr2=0x7fcd0c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:55.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.893+0000 7fcd0be00640 1 -- 172.21.15.83:0/2510176711 shutdown_connections 2024-04-23T14:56:55.903 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:55.893+0000 7fcd0be00640 1 -- 172.21.15.83:0/2510176711 wait complete. 2024-04-23T14:56:55.912 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:56:55.912 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:56:56.112 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.111+0000 7fb9a3e00640 1 -- 172.21.15.83:0/657498276 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9980319b0 con 0x7fb9a410e6b0 2024-04-23T14:56:56.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.112+0000 7fb9abe00640 1 -- 172.21.15.83:0/657498276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a410e6b0 msgr2=0x7fb9a410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.112+0000 7fb9abe00640 1 --2- 172.21.15.83:0/657498276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a410e6b0 0x7fb9a410ea90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb998009930 tx=0x7fb99802a440 comp rx=0 tx=0).stop 2024-04-23T14:56:56.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.112+0000 7fb9abe00640 1 -- 172.21.15.83:0/657498276 shutdown_connections 2024-04-23T14:56:56.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.112+0000 7fb9abe00640 1 --2- 172.21.15.83:0/657498276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4108fd0 0x7fb9a41154e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.113 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.112+0000 7fb9abe00640 1 --2- 172.21.15.83:0/657498276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9a4108610 0x7fb9a4108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.112+0000 7fb9abe00640 1 --2- 172.21.15.83:0/657498276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a410e6b0 0x7fb9a410ea90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.112+0000 7fb9abe00640 1 -- 172.21.15.83:0/657498276 >> 172.21.15.83:0/657498276 conn(0x7fb9a4075800 msgr2=0x7fb9a4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.112+0000 7fb9abe00640 1 -- 172.21.15.83:0/657498276 shutdown_connections 2024-04-23T14:56:56.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.112+0000 7fb9abe00640 1 -- 172.21.15.83:0/657498276 wait complete. 2024-04-23T14:56:56.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.113+0000 7fb9abe00640 1 Processor -- start 2024-04-23T14:56:56.116 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.113+0000 7fb9abe00640 1 -- start start 2024-04-23T14:56:56.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.113+0000 7fb9abe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a4108610 0x7fb9a419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.113+0000 7fb9abe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4108fd0 0x7fb9a419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.113+0000 7fb9abe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9a410e6b0 0x7fb9a41a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.113+0000 7fb9abe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4118a30 con 0x7fb9a4108610 2024-04-23T14:56:56.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.113+0000 7fb9abe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a41188b0 con 0x7fb9a4108fd0 2024-04-23T14:56:56.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.113+0000 7fb9abe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4118bb0 con 0x7fb9a410e6b0 2024-04-23T14:56:56.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.114+0000 7fb9a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a4108610 0x7fb9a419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.114+0000 7fb9a9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a4108610 0x7fb9a419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58146/0 (socket says 172.21.15.83:58146) 2024-04-23T14:56:56.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.114+0000 7fb9a9400640 1 -- 172.21.15.83:0/4083724222 learned_addr learned my addr 172.21.15.83:0/4083724222 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:56.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.114+0000 7fb9a9e00640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9a410e6b0 0x7fb9a41a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.114+0000 7fb9a8a00640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4108fd0 0x7fb9a419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.114+0000 7fb9a9400640 1 -- 172.21.15.83:0/4083724222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9a410e6b0 msgr2=0x7fb9a41a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.114+0000 7fb9a9400640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9a410e6b0 0x7fb9a41a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.114+0000 7fb9a9400640 1 -- 172.21.15.83:0/4083724222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4108fd0 msgr2=0x7fb9a419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.114+0000 7fb9a9400640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4108fd0 0x7fb9a419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.114+0000 7fb9a9400640 1 -- 172.21.15.83:0/4083724222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a41a3ef0 con 0x7fb9a4108610 2024-04-23T14:56:56.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.114+0000 7fb9a9e00640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9a410e6b0 0x7fb9a41a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:56.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.115+0000 7fb9a9400640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a4108610 0x7fb9a419ef50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb998005940 tx=0x7fb9980040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.115+0000 7fb9a2000640 1 -- 172.21.15.83:0/4083724222 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99803f030 con 0x7fb9a4108610 2024-04-23T14:56:56.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.115+0000 7fb9a2000640 1 -- 172.21.15.83:0/4083724222 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9980044c0 con 0x7fb9a4108610 2024-04-23T14:56:56.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.115+0000 7fb9a2000640 1 -- 172.21.15.83:0/4083724222 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998035070 con 0x7fb9a4108610 2024-04-23T14:56:56.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.115+0000 7fb9abe00640 1 -- 172.21.15.83:0/4083724222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a41a4180 con 0x7fb9a4108610 2024-04-23T14:56:56.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.115+0000 7fb9abe00640 1 -- 172.21.15.83:0/4083724222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a41aba70 con 0x7fb9a4108610 2024-04-23T14:56:56.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.116+0000 7fb9abe00640 1 -- 172.21.15.83:0/4083724222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a4069270 con 0x7fb9a4108610 2024-04-23T14:56:56.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.118+0000 7fb9a2000640 1 -- 172.21.15.83:0/4083724222 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb998042030 con 0x7fb9a4108610 2024-04-23T14:56:56.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.119+0000 7fb9a2000640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb98c0a3da0 0x7fb98c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.119+0000 7fb9a2000640 1 -- 172.21.15.83:0/4083724222 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7fb9980f0b50 con 0x7fb9a4108610 2024-04-23T14:56:56.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.120+0000 7fb9a8a00640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb98c0a3da0 0x7fb98c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.120+0000 7fb9a8a00640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb98c0a3da0 0x7fb98c0a6260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb9a410ddb0 tx=0x7fb990005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.124+0000 7fb9a2000640 1 -- 172.21.15.83:0/4083724222 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9980bd180 con 0x7fb9a4108610 2024-04-23T14:56:56.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.386+0000 7fb9abe00640 1 -- 172.21.15.83:0/4083724222 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9a4110b80 con 0x7fb9a4108610 2024-04-23T14:56:56.425 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.387+0000 7fb9a2000640 1 -- 172.21.15.83:0/4083724222 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7fb9980bca80 con 0x7fb9a4108610 2024-04-23T14:56:56.425 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:56:56.426 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:56:56.427 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:56:56.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.390+0000 7fb9abe00640 1 -- 172.21.15.83:0/4083724222 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb98c0a3da0 msgr2=0x7fb98c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.390+0000 7fb9abe00640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb98c0a3da0 0x7fb98c0a6260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb9a410ddb0 tx=0x7fb990005f90 comp rx=0 tx=0).stop 2024-04-23T14:56:56.427 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.390+0000 7fb9abe00640 1 -- 172.21.15.83:0/4083724222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a4108610 msgr2=0x7fb9a419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.390+0000 7fb9abe00640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a4108610 0x7fb9a419ef50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb998005940 tx=0x7fb9980040c0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.390+0000 7fb9abe00640 1 -- 172.21.15.83:0/4083724222 shutdown_connections 2024-04-23T14:56:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.390+0000 7fb9abe00640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9a410e6b0 0x7fb9a41a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.390+0000 7fb9abe00640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a4108fd0 0x7fb9a419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.390+0000 7fb9abe00640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb98c0a3da0 0x7fb98c0a6260 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.390+0000 7fb9abe00640 1 --2- 172.21.15.83:0/4083724222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb9a4108610 0x7fb9a419ef50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.390+0000 7fb9abe00640 1 -- 172.21.15.83:0/4083724222 >> 172.21.15.83:0/4083724222 conn(0x7fb9a4075800 msgr2=0x7fb9a4076910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.428 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.390+0000 7fb9abe00640 1 -- 172.21.15.83:0/4083724222 shutdown_connections 2024-04-23T14:56:56.429 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.390+0000 7fb9abe00640 1 -- 172.21.15.83:0/4083724222 wait complete. 2024-04-23T14:56:56.429 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:56:56.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.626+0000 7ff259a00640 1 -- 172.21.15.83:0/3726419224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff254108580 msgr2=0x7ff254108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.626+0000 7ff259a00640 1 --2- 172.21.15.83:0/3726419224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff254108580 0x7ff254108a00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009930 tx=0x7ff23c02a420 comp rx=0 tx=0).stop 2024-04-23T14:56:56.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.626+0000 7ff259a00640 1 -- 172.21.15.83:0/3726419224 shutdown_connections 2024-04-23T14:56:56.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.626+0000 7ff259a00640 1 --2- 172.21.15.83:0/3726419224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff254108f40 0x7ff2541153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.626+0000 7ff259a00640 1 --2- 172.21.15.83:0/3726419224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff254108580 0x7ff254108a00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.626+0000 7ff259a00640 1 --2- 172.21.15.83:0/3726419224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff25410d500 0x7ff25410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.626+0000 7ff259a00640 1 -- 172.21.15.83:0/3726419224 >> 172.21.15.83:0/3726419224 conn(0x7ff254077e20 msgr2=0x7ff254106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.626+0000 7ff259a00640 1 -- 172.21.15.83:0/3726419224 shutdown_connections 2024-04-23T14:56:56.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.626+0000 7ff259a00640 1 -- 172.21.15.83:0/3726419224 wait complete. 2024-04-23T14:56:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.627+0000 7ff259a00640 1 Processor -- start 2024-04-23T14:56:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.627+0000 7ff259a00640 1 -- start start 2024-04-23T14:56:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff259a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff254108580 0x7ff25419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff259a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff254108f40 0x7ff25419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff259a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff25410d500 0x7ff2541a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff259a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541188b0 con 0x7ff254108580 2024-04-23T14:56:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff259a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff254118730 con 0x7ff25410d500 2024-04-23T14:56:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff259a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff254118a30 con 0x7ff254108f40 2024-04-23T14:56:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff253e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff254108580 0x7ff25419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff253e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff254108580 0x7ff25419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58160/0 (socket says 172.21.15.83:58160) 2024-04-23T14:56:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff253e00640 1 -- 172.21.15.83:0/2399747982 learned_addr learned my addr 172.21.15.83:0/2399747982 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff259000640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff25410d500 0x7ff2541a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff24b400640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff254108f40 0x7ff25419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff259000640 1 -- 172.21.15.83:0/2399747982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff254108f40 msgr2=0x7ff25419f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff259000640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff254108f40 0x7ff25419f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff259000640 1 -- 172.21.15.83:0/2399747982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff254108580 msgr2=0x7ff25419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff259000640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff254108580 0x7ff25419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff259000640 1 -- 172.21.15.83:0/2399747982 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2541a3da0 con 0x7ff25410d500 2024-04-23T14:56:56.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.628+0000 7ff253e00640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff254108580 0x7ff25419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:56.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.629+0000 7ff259000640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff25410d500 0x7ff2541a3670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff24c005fd0 tx=0x7ff24c00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.629+0000 7ff251600640 1 -- 172.21.15.83:0/2399747982 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00cd80 con 0x7ff25410d500 2024-04-23T14:56:56.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.629+0000 7ff251600640 1 -- 172.21.15.83:0/2399747982 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff24c005020 con 0x7ff25410d500 2024-04-23T14:56:56.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.629+0000 7ff251600640 1 -- 172.21.15.83:0/2399747982 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c010720 con 0x7ff25410d500 2024-04-23T14:56:56.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.629+0000 7ff259a00640 1 -- 172.21.15.83:0/2399747982 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2541a4090 con 0x7ff25410d500 2024-04-23T14:56:56.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.629+0000 7ff259a00640 1 -- 172.21.15.83:0/2399747982 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2541ab920 con 0x7ff25410d500 2024-04-23T14:56:56.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.632+0000 7ff25de38740 1 Processor -- start 2024-04-23T14:56:56.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.632+0000 7ff251600640 1 -- 172.21.15.83:0/2399747982 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff24c0051d0 con 0x7ff25410d500 2024-04-23T14:56:56.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.632+0000 7ff25de38740 1 -- start start 2024-04-23T14:56:56.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff25de38740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dbef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff25de38740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff25de38740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dcbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff25de38740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d7c4d241d0 con 0x55d7c4dc9b30 2024-04-23T14:56:56.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff25de38740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d7c4cca450 con 0x55d7c4dbeb40 2024-04-23T14:56:56.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff25de38740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d7c4d20190 con 0x55d7c4dbf460 2024-04-23T14:56:56.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff24b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dbef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff24b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dcbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff253e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff24b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dcbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58168/0 (socket says 172.21.15.83:58168) 2024-04-23T14:56:56.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff24b400640 1 -- 172.21.15.83:0/1602311794 learned_addr learned my addr 172.21.15.83:0/1602311794 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:56.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff24b400640 1 -- 172.21.15.83:0/1602311794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbf460 msgr2=0x55d7c4dc95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.633+0000 7ff24b400640 1 --2- 172.21.15.83:0/1602311794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.634+0000 7ff24b400640 1 -- 172.21.15.83:0/1602311794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbeb40 msgr2=0x55d7c4dbef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.638 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.634+0000 7ff24b400640 1 --2- 172.21.15.83:0/1602311794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dbef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.634+0000 7ff24b400640 1 -- 172.21.15.83:0/1602311794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c4d87310 con 0x55d7c4dc9b30 2024-04-23T14:56:56.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.634+0000 7ff24b400640 1 --2- 172.21.15.83:0/1602311794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dcbf20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009930 tx=0x7ff23c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683f1bdced81d80f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.634+0000 7ff251600640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff2380a3da0 0x7ff2380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.634+0000 7ff249600640 1 -- 172.21.15.83:0/1602311794 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c035050 con 0x55d7c4dc9b30 2024-04-23T14:56:56.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.634+0000 7ff249600640 1 -- 172.21.15.83:0/1602311794 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff23c0318f0 con 0x55d7c4dc9b30 2024-04-23T14:56:56.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.634+0000 7ff249600640 1 -- 172.21.15.83:0/1602311794 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0047a0 con 0x55d7c4dc9b30 2024-04-23T14:56:56.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.634+0000 7ff251600640 1 -- 172.21.15.83:0/2399747982 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7ff24c0d0de0 con 0x7ff25410d500 2024-04-23T14:56:56.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.634+0000 7ff253e00640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff2380a3da0 0x7ff2380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.635+0000 7ff25de38740 1 -- 172.21.15.83:0/1602311794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 msgr2=0x55d7c4dcbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.635+0000 7ff25de38740 1 --2- 172.21.15.83:0/1602311794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dcbf20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009930 tx=0x7ff23c0058d0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.635+0000 7ff253e00640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff2380a3da0 0x7ff2380a6260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff244009fd0 tx=0x7ff244005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.635+0000 7ff25de38740 1 -- 172.21.15.83:0/1602311794 shutdown_connections 2024-04-23T14:56:56.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.635+0000 7ff25de38740 1 --2- 172.21.15.83:0/1602311794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dcbf20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.635+0000 7ff25de38740 1 --2- 172.21.15.83:0/1602311794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.635+0000 7ff25de38740 1 --2- 172.21.15.83:0/1602311794 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dbef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.635+0000 7ff25de38740 1 -- 172.21.15.83:0/1602311794 >> 172.21.15.83:0/1602311794 conn(0x55d7c4dbae60 msgr2=0x55d7c4dbb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.635+0000 7ff25de38740 1 -- 172.21.15.83:0/1602311794 shutdown_connections 2024-04-23T14:56:56.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.635+0000 7ff25de38740 1 -- 172.21.15.83:0/1602311794 wait complete. 2024-04-23T14:56:56.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.636+0000 7ff25de38740 1 -- start start 2024-04-23T14:56:56.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.636+0000 7ff25de38740 1 Processor -- start 2024-04-23T14:56:56.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.636+0000 7ff25de38740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dc9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.636+0000 7ff25de38740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.636+0000 7ff25de38740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dc6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.636+0000 7ff25de38740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d7c4dcc960 con 0x55d7c4dbf460 2024-04-23T14:56:56.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.636+0000 7ff25de38740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55d7c4cdd810 con 0x55d7c4dc9b30 2024-04-23T14:56:56.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.636+0000 7ff25de38740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55d7c4dccae0 con 0x55d7c4dbeb40 2024-04-23T14:56:56.645 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.636+0000 7ff25de38740 10 client.0 _ll_register_callbacks cb 0x55d7c4cab140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.636+0000 7ff24b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.636+0000 7ff24b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58178/0 (socket says 172.21.15.83:58178) 2024-04-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.636+0000 7ff24b400640 1 -- 172.21.15.83:0/3975123374 learned_addr learned my addr 172.21.15.83:0/3975123374 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.637+0000 7ff259000640 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dc6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.637+0000 7ff253e00640 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dc9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.637+0000 7ff24b400640 1 -- 172.21.15.83:0/3975123374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbeb40 msgr2=0x55d7c4dc9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.637+0000 7ff24b400640 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dc9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.646 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.637+0000 7ff24b400640 1 -- 172.21.15.83:0/3975123374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dc9b30 msgr2=0x55d7c4dc6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.637+0000 7ff24b400640 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dc6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.637+0000 7ff24b400640 1 -- 172.21.15.83:0/3975123374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c4cee3f0 con 0x55d7c4dbf460 2024-04-23T14:56:56.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.637+0000 7ff24b400640 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc6880 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff23c02a460 tx=0x7ff23c009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314af3059a06c5f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.637+0000 7ff242000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:56.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.637+0000 7ff242000640 1 -- 172.21.15.83:0/3975123374 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23c04e020 con 0x55d7c4dbf460 2024-04-23T14:56:56.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.637+0000 7ff242000640 1 -- 172.21.15.83:0/3975123374 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7ff23c0055c0 con 0x55d7c4dbf460 2024-04-23T14:56:56.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.637+0000 7ff25de38740 1 -- 172.21.15.83:0/3975123374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff23c0318f0 con 0x55d7c4dbf460 2024-04-23T14:56:56.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.638+0000 7ff242000640 1 -- 172.21.15.83:0/3975123374 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff23c059050 con 0x55d7c4dbf460 2024-04-23T14:56:56.647 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.638+0000 7ff25de38740 10 client.16680 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:56.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.638+0000 7ff25de38740 1 -- 172.21.15.83:0/3975123374 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7c4a363f0 con 0x55d7c4dbf460 2024-04-23T14:56:56.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.638+0000 7ff242000640 1 -- 172.21.15.83:0/3975123374 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff23c0318f0 con 0x55d7c4dbf460 2024-04-23T14:56:56.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.638+0000 7ff242000640 1 -- 172.21.15.83:0/3975123374 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ff23c003b00 con 0x55d7c4dbf460 2024-04-23T14:56:56.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.638+0000 7ff25de38740 10 client.16680 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:56.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.638+0000 7ff25de38740 10 client.16680 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:56:56.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.638+0000 7ff25de38740 20 client.16680 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:56.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.638+0000 7ff25de38740 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d7c4dd9bb0 0x55d7c4ddbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.638+0000 7ff25de38740 4 client.16680 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:56:56.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.638+0000 7ff25de38740 1 -- 172.21.15.83:0/3975123374 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7c4d31bf0 con 0x55d7c4dd9bb0 2024-04-23T14:56:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.639+0000 7ff253e00640 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d7c4dd9bb0 0x55d7c4ddbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.639+0000 7ff253e00640 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d7c4dd9bb0 0x55d7c4ddbfa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=916c88a7934f97a1 server_cookie=7135b455b4cf9d95 in_seq=0 out_seq=0 2024-04-23T14:56:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.639+0000 7ff242000640 10 client.16680 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:56:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.640+0000 7ff242000640 1 -- 172.21.15.83:0/3975123374 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7c4d31bf0 con 0x55d7c4dd9bb0 2024-04-23T14:56:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.640+0000 7ff242000640 10 client.16680 handle_command_reply: tid=0 2024-04-23T14:56:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.640+0000 7ff25de38740 1 client.16680 shutdown 2024-04-23T14:56:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.640+0000 7ff25de38740 1 -- 172.21.15.83:0/3975123374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbf460 msgr2=0x55d7c4dc6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.640+0000 7ff25de38740 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc6880 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff23c02a460 tx=0x7ff23c009230 comp rx=0 tx=0).stop 2024-04-23T14:56:56.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.640+0000 7ff25de38740 1 -- 172.21.15.83:0/3975123374 shutdown_connections 2024-04-23T14:56:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.640+0000 7ff25de38740 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d7c4dd9bb0 0x55d7c4ddbfa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.640+0000 7ff25de38740 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dc6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.640+0000 7ff25de38740 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc6880 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.640+0000 7ff25de38740 1 --2- 172.21.15.83:0/3975123374 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dc9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.640+0000 7ff242000640 0 client.16680 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:56:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.640+0000 7ff25de38740 1 -- 172.21.15.83:0/3975123374 >> 172.21.15.83:0/3975123374 conn(0x55d7c4dbae60 msgr2=0x55d7c4dc0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.641+0000 7ff25de38740 1 -- 172.21.15.83:0/3975123374 shutdown_connections 2024-04-23T14:56:56.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.641+0000 7ff25de38740 1 -- 172.21.15.83:0/3975123374 wait complete. 2024-04-23T14:56:56.651 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.641+0000 7ff25de38740 20 client.16680 trim_cache size 0 max 16384 2024-04-23T14:56:56.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.655+0000 7ff25de38740 1 Processor -- start 2024-04-23T14:56:56.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.655+0000 7ff25de38740 1 -- start start 2024-04-23T14:56:56.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.655+0000 7ff25de38740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dd5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.655+0000 7ff25de38740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff25de38740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dc90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff25de38740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541b6a70 con 0x55d7c4dc9b30 2024-04-23T14:56:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff25de38740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541b4340 con 0x55d7c4dbf460 2024-04-23T14:56:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff25de38740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541b2190 con 0x55d7c4dbeb40 2024-04-23T14:56:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff259000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dc90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff259000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dc90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58188/0 (socket says 172.21.15.83:58188) 2024-04-23T14:56:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff259000640 1 -- 172.21.15.83:0/2620411843 learned_addr learned my addr 172.21.15.83:0/2620411843 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff24b400640 1 --2- 172.21.15.83:0/2620411843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff253e00640 1 --2- 172.21.15.83:0/2620411843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dd5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff259000640 1 -- 172.21.15.83:0/2620411843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbeb40 msgr2=0x55d7c4dd5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff259000640 1 --2- 172.21.15.83:0/2620411843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dd5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff259000640 1 -- 172.21.15.83:0/2620411843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbf460 msgr2=0x55d7c4dc5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff259000640 1 --2- 172.21.15.83:0/2620411843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff259000640 1 -- 172.21.15.83:0/2620411843 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c4b007d0 con 0x55d7c4dc9b30 2024-04-23T14:56:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff253e00640 1 --2- 172.21.15.83:0/2620411843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dd5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:56:56.659 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff24b400640 1 --2- 172.21.15.83:0/2620411843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:56:56.663 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.656+0000 7ff259000640 1 --2- 172.21.15.83:0/2620411843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dc90c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009fd0 tx=0x7ff24c099060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a209da14e1743ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.657+0000 7ff248c00640 1 -- 172.21.15.83:0/2620411843 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c0b5070 con 0x55d7c4dc9b30 2024-04-23T14:56:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.657+0000 7ff248c00640 1 -- 172.21.15.83:0/2620411843 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff24c0b6260 con 0x55d7c4dc9b30 2024-04-23T14:56:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.657+0000 7ff248c00640 1 -- 172.21.15.83:0/2620411843 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c09cf70 con 0x55d7c4dc9b30 2024-04-23T14:56:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.657+0000 7ff25de38740 1 -- 172.21.15.83:0/2620411843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 msgr2=0x55d7c4dc90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.657+0000 7ff25de38740 1 --2- 172.21.15.83:0/2620411843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dc90c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009fd0 tx=0x7ff24c099060 comp rx=0 tx=0).stop 2024-04-23T14:56:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.657+0000 7ff25de38740 1 -- 172.21.15.83:0/2620411843 shutdown_connections 2024-04-23T14:56:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.657+0000 7ff25de38740 1 --2- 172.21.15.83:0/2620411843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dc90c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.657+0000 7ff25de38740 1 --2- 172.21.15.83:0/2620411843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbf460 0x55d7c4dc5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.657+0000 7ff25de38740 1 --2- 172.21.15.83:0/2620411843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dd5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.657+0000 7ff25de38740 1 -- 172.21.15.83:0/2620411843 >> 172.21.15.83:0/2620411843 conn(0x55d7c4dbae60 msgr2=0x55d7c4dc5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.657+0000 7ff25de38740 1 -- 172.21.15.83:0/2620411843 shutdown_connections 2024-04-23T14:56:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.658+0000 7ff25de38740 1 -- 172.21.15.83:0/2620411843 wait complete. 2024-04-23T14:56:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.658+0000 7ff25de38740 1 -- start start 2024-04-23T14:56:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.658+0000 7ff25de38740 1 Processor -- start 2024-04-23T14:56:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.658+0000 7ff25de38740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dd2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.658+0000 7ff25de38740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbf460 0x55d7c4dca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.658+0000 7ff25de38740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dcab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.658+0000 7ff25de38740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541b2800 con 0x55d7c4dbeb40 2024-04-23T14:56:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.658+0000 7ff25de38740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541b3c70 con 0x55d7c4dbf460 2024-04-23T14:56:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.658+0000 7ff25de38740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541b2ed0 con 0x55d7c4dc9b30 2024-04-23T14:56:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.658+0000 7ff25de38740 10 client.0 _ll_register_callbacks cb 0x7ff23c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff253e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dd2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff253e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dd2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58190/0 (socket says 172.21.15.83:58190) 2024-04-23T14:56:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff253e00640 1 -- 172.21.15.83:0/2867365943 learned_addr learned my addr 172.21.15.83:0/2867365943 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff24b400640 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbf460 0x55d7c4dca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff259000640 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dcab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff253e00640 1 -- 172.21.15.83:0/2867365943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dc9b30 msgr2=0x55d7c4dcab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff253e00640 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dcab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff253e00640 1 -- 172.21.15.83:0/2867365943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbf460 msgr2=0x55d7c4dca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff253e00640 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbf460 0x55d7c4dca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff253e00640 1 -- 172.21.15.83:0/2867365943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c4dcb380 con 0x55d7c4dbeb40 2024-04-23T14:56:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff253e00640 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dd2200 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff24400f990 tx=0x7ff244017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5105eb21730975e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff242000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:56.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.659+0000 7ff242000640 1 -- 172.21.15.83:0/2867365943 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24401d690 con 0x55d7c4dbeb40 2024-04-23T14:56:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff242000640 1 -- 172.21.15.83:0/2867365943 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7ff24401d830 con 0x55d7c4dbeb40 2024-04-23T14:56:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff25de38740 1 -- 172.21.15.83:0/2867365943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff24c0b6260 con 0x55d7c4dbeb40 2024-04-23T14:56:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff242000640 1 -- 172.21.15.83:0/2867365943 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff244038050 con 0x55d7c4dbeb40 2024-04-23T14:56:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff25de38740 10 client.16686 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff25de38740 1 -- 172.21.15.83:0/2867365943 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7c4e3fef0 con 0x55d7c4dbeb40 2024-04-23T14:56:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff242000640 1 -- 172.21.15.83:0/2867365943 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff24c0b6260 con 0x55d7c4dbeb40 2024-04-23T14:56:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff242000640 1 -- 172.21.15.83:0/2867365943 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ff24402a410 con 0x55d7c4dbeb40 2024-04-23T14:56:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff25de38740 10 client.16686 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:56.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff25de38740 10 client.16686 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:56:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff25de38740 20 client.16686 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff25de38740 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d7c4e40660 0x55d7c4e42a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff25de38740 4 client.16686 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:56:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.660+0000 7ff25de38740 1 -- 172.21.15.83:0/2867365943 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7c4d2ba60 con 0x55d7c4e40660 2024-04-23T14:56:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff24b400640 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d7c4e40660 0x55d7c4e42a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff24b400640 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d7c4e40660 0x55d7c4e42a50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3140a8d6534eed1 server_cookie=4c23b9804a4173b2 in_seq=0 out_seq=0 2024-04-23T14:56:56.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff242000640 10 client.16686 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:56:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff242000640 1 -- 172.21.15.83:0/2867365943 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7c4d2ba60 con 0x55d7c4e40660 2024-04-23T14:56:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff242000640 10 client.16686 handle_command_reply: tid=0 2024-04-23T14:56:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff25de38740 1 client.16686 shutdown 2024-04-23T14:56:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff25de38740 1 -- 172.21.15.83:0/2867365943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbeb40 msgr2=0x55d7c4dd2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff25de38740 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dd2200 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff24400f990 tx=0x7ff244017000 comp rx=0 tx=0).stop 2024-04-23T14:56:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff25de38740 1 -- 172.21.15.83:0/2867365943 shutdown_connections 2024-04-23T14:56:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff25de38740 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55d7c4e40660 0x55d7c4e42a50 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff25de38740 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7c4dc9b30 0x55d7c4dcab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff25de38740 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55d7c4dbf460 0x55d7c4dca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff242000640 0 client.16686 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:56:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff25de38740 1 --2- 172.21.15.83:0/2867365943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7c4dbeb40 0x55d7c4dd2200 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.661+0000 7ff25de38740 1 -- 172.21.15.83:0/2867365943 >> 172.21.15.83:0/2867365943 conn(0x55d7c4dbae60 msgr2=0x55d7c4db96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff25de38740 1 -- 172.21.15.83:0/2867365943 shutdown_connections 2024-04-23T14:56:56.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff25de38740 1 -- 172.21.15.83:0/2867365943 wait complete. 2024-04-23T14:56:56.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff25de38740 20 client.16686 trim_cache size 0 max 16384 2024-04-23T14:56:56.672 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:56:56.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff259a00640 1 -- 172.21.15.83:0/2399747982 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff2380a3da0 msgr2=0x7ff2380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff259a00640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff2380a3da0 0x7ff2380a6260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff244009fd0 tx=0x7ff244005f90 comp rx=0 tx=0).stop 2024-04-23T14:56:56.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff259a00640 1 -- 172.21.15.83:0/2399747982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff25410d500 msgr2=0x7ff2541a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff259a00640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff25410d500 0x7ff2541a3670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff24c005fd0 tx=0x7ff24c00ee60 comp rx=0 tx=0).stop 2024-04-23T14:56:56.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff259a00640 1 -- 172.21.15.83:0/2399747982 shutdown_connections 2024-04-23T14:56:56.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff259a00640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff25410d500 0x7ff2541a3670 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.673 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff259a00640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff254108f40 0x7ff25419f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff259a00640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff2380a3da0 0x7ff2380a6260 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff259a00640 1 --2- 172.21.15.83:0/2399747982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff254108580 0x7ff25419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.662+0000 7ff259a00640 1 -- 172.21.15.83:0/2399747982 >> 172.21.15.83:0/2399747982 conn(0x7ff254077e20 msgr2=0x7ff254078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.663+0000 7ff259a00640 1 -- 172.21.15.83:0/2399747982 shutdown_connections 2024-04-23T14:56:56.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.663+0000 7ff259a00640 1 -- 172.21.15.83:0/2399747982 wait complete. 2024-04-23T14:56:56.677 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:56 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/4083724222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:56:56.681 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:56:56.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:56 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/4083724222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:56:56.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:56 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/4083724222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:56:56.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.872+0000 7f0ea9400640 1 -- 172.21.15.83:0/1132722576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4108fd0 msgr2=0x7f0ea4115490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.872+0000 7f0ea9400640 1 --2- 172.21.15.83:0/1132722576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4108fd0 0x7f0ea4115490 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009cd0 tx=0x7f0e9c02a8f0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.872+0000 7f0ea9400640 1 -- 172.21.15.83:0/1132722576 shutdown_connections 2024-04-23T14:56:56.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.872+0000 7f0ea9400640 1 --2- 172.21.15.83:0/1132722576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4108fd0 0x7f0ea4115490 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.872+0000 7f0ea9400640 1 --2- 172.21.15.83:0/1132722576 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ea4108610 0x7f0ea4108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.872+0000 7f0ea9400640 1 --2- 172.21.15.83:0/1132722576 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ea410e660 0x7f0ea410ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.872+0000 7f0ea9400640 1 -- 172.21.15.83:0/1132722576 >> 172.21.15.83:0/1132722576 conn(0x7f0ea4076540 msgr2=0x7f0ea4078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.873+0000 7f0ea9400640 1 -- 172.21.15.83:0/1132722576 shutdown_connections 2024-04-23T14:56:56.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.873+0000 7f0ea9400640 1 -- 172.21.15.83:0/1132722576 wait complete. 2024-04-23T14:56:56.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.873+0000 7f0ea9400640 1 Processor -- start 2024-04-23T14:56:56.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.874+0000 7f0ea9400640 1 -- start start 2024-04-23T14:56:56.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.874+0000 7f0ea9400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ea4108610 0x7f0ea419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.874+0000 7f0ea9400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4108fd0 0x7f0ea419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.874+0000 7f0ea9400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ea410e660 0x7f0ea41a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.874+0000 7f0ea9400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41189a0 con 0x7f0ea4108fd0 2024-04-23T14:56:56.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.874+0000 7f0ea9400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4118820 con 0x7f0ea4108610 2024-04-23T14:56:56.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.874+0000 7f0ea9400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4118b20 con 0x7f0ea410e660 2024-04-23T14:56:56.877 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.874+0000 7f0ea1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4108fd0 0x7f0ea419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.874+0000 7f0ea1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4108fd0 0x7f0ea419f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58206/0 (socket says 172.21.15.83:58206) 2024-04-23T14:56:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.874+0000 7f0ea1e00640 1 -- 172.21.15.83:0/993073929 learned_addr learned my addr 172.21.15.83:0/993073929 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.874+0000 7f0ea3200640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ea410e660 0x7f0ea41a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.874+0000 7f0ea2800640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ea4108610 0x7f0ea419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.875+0000 7f0ea1e00640 1 -- 172.21.15.83:0/993073929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ea410e660 msgr2=0x7f0ea41a36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.875+0000 7f0ea1e00640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ea410e660 0x7f0ea41a36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.875+0000 7f0ea1e00640 1 -- 172.21.15.83:0/993073929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ea4108610 msgr2=0x7f0ea419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.875+0000 7f0ea1e00640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ea4108610 0x7f0ea419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.875+0000 7f0ea1e00640 1 -- 172.21.15.83:0/993073929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea41a3de0 con 0x7f0ea4108fd0 2024-04-23T14:56:56.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.875+0000 7f0ea1e00640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4108fd0 0x7f0ea419f380 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000e9f0 tx=0x7f0e9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.875+0000 7f0e9b400640 1 -- 172.21.15.83:0/993073929 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9000cd80 con 0x7f0ea4108fd0 2024-04-23T14:56:56.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.875+0000 7f0e9b400640 1 -- 172.21.15.83:0/993073929 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e90010910 con 0x7f0ea4108fd0 2024-04-23T14:56:56.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.875+0000 7f0ea9400640 1 -- 172.21.15.83:0/993073929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea41a40d0 con 0x7f0ea4108fd0 2024-04-23T14:56:56.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.875+0000 7f0e9b400640 1 -- 172.21.15.83:0/993073929 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90018990 con 0x7f0ea4108fd0 2024-04-23T14:56:56.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.875+0000 7f0ea9400640 1 -- 172.21.15.83:0/993073929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea41ab960 con 0x7f0ea4108fd0 2024-04-23T14:56:56.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.878+0000 7f0eac231740 1 Processor -- start 2024-04-23T14:56:56.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.878+0000 7f0eac231740 1 -- start start 2024-04-23T14:56:56.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.879+0000 7f0eac231740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 0x558c6949ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.879+0000 7f0eac231740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694a9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.879+0000 7f0eac231740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694abd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.879+0000 7f0eac231740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558c694041d0 con 0x558c694a9940 2024-04-23T14:56:56.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.879+0000 7f0eac231740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558c693aa450 con 0x558c6949f270 2024-04-23T14:56:56.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.879+0000 7f0eac231740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558c69400190 con 0x558c6949e950 2024-04-23T14:56:56.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.879+0000 7f0e9b400640 1 -- 172.21.15.83:0/993073929 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0e90010ac0 con 0x7f0ea4108fd0 2024-04-23T14:56:56.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.879+0000 7f0ea3200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 0x558c6949ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.879+0000 7f0ea2800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.879+0000 7f0ea3200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 0x558c6949ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59704/0 (socket says 172.21.15.83:59704) 2024-04-23T14:56:56.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.879+0000 7f0ea3200640 1 -- 172.21.15.83:0/3081694486 learned_addr learned my addr 172.21.15.83:0/3081694486 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:56.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.879+0000 7f0ea1e00640 1 --2- 172.21.15.83:0/3081694486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.880+0000 7f0ea3200640 1 -- 172.21.15.83:0/3081694486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 msgr2=0x558c694a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.880+0000 7f0ea3200640 1 --2- 172.21.15.83:0/3081694486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694a9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.880+0000 7f0ea3200640 1 -- 172.21.15.83:0/3081694486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 msgr2=0x558c694abd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.880+0000 7f0ea3200640 1 --2- 172.21.15.83:0/3081694486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694abd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.880+0000 7f0ea3200640 1 -- 172.21.15.83:0/3081694486 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c69467310 con 0x558c6949e950 2024-04-23T14:56:56.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.880+0000 7f0ea3200640 1 --2- 172.21.15.83:0/3081694486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 0x558c6949ed30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c004aa0 tx=0x7f0e9c009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b91e85e2215e4c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.880+0000 7f0e8f400640 1 -- 172.21.15.83:0/3081694486 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c030660 con 0x558c6949e950 2024-04-23T14:56:56.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.880+0000 7f0e8f400640 1 -- 172.21.15.83:0/3081694486 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c036030 con 0x558c6949e950 2024-04-23T14:56:56.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.880+0000 7f0e8f400640 1 -- 172.21.15.83:0/3081694486 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0043a0 con 0x558c6949e950 2024-04-23T14:56:56.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.881+0000 7f0e9b400640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0e880a3d20 0x7f0e880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.881+0000 7f0e9b400640 1 -- 172.21.15.83:0/993073929 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f0e900d0b90 con 0x7f0ea4108fd0 2024-04-23T14:56:56.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.881+0000 7f0ea2800640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0e880a3d20 0x7f0e880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.881+0000 7f0eac231740 1 -- 172.21.15.83:0/3081694486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 msgr2=0x558c6949ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.881+0000 7f0eac231740 1 --2- 172.21.15.83:0/3081694486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 0x558c6949ed30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c004aa0 tx=0x7f0e9c009360 comp rx=0 tx=0).stop 2024-04-23T14:56:56.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.881+0000 7f0eac231740 1 -- 172.21.15.83:0/3081694486 shutdown_connections 2024-04-23T14:56:56.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.881+0000 7f0eac231740 1 --2- 172.21.15.83:0/3081694486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694abd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.881+0000 7f0eac231740 1 --2- 172.21.15.83:0/3081694486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.881+0000 7f0eac231740 1 --2- 172.21.15.83:0/3081694486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 0x558c6949ed30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.882+0000 7f0eac231740 1 -- 172.21.15.83:0/3081694486 >> 172.21.15.83:0/3081694486 conn(0x558c6949ac70 msgr2=0x558c6949b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.882+0000 7f0eac231740 1 -- 172.21.15.83:0/3081694486 shutdown_connections 2024-04-23T14:56:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.882+0000 7f0eac231740 1 -- 172.21.15.83:0/3081694486 wait complete. 2024-04-23T14:56:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.882+0000 7f0ea2800640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0e880a3d20 0x7f0e880a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x558c6949ff10 tx=0x7f0e94005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.882+0000 7f0eac231740 1 -- start start 2024-04-23T14:56:56.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.882+0000 7f0eac231740 1 Processor -- start 2024-04-23T14:56:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0eac231740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 0x558c694a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0eac231740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694a6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0eac231740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0eac231740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558c694ac770 con 0x558c694a9940 2024-04-23T14:56:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0eac231740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x558c693bd810 con 0x558c6949f270 2024-04-23T14:56:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0eac231740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x558c694ac8f0 con 0x558c6949e950 2024-04-23T14:56:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0eac231740 10 client.0 _ll_register_callbacks cb 0x558c6938b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0ea3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0ea1e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0ea2800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 0x558c694a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0ea1e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694a6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:42826/0 (socket says 172.21.15.83:42826) 2024-04-23T14:56:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0ea1e00640 1 -- 172.21.15.83:0/1431528672 learned_addr learned my addr 172.21.15.83:0/1431528672 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.883+0000 7f0ea2800640 1 -- 172.21.15.83:0/1431528672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 msgr2=0x558c694a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0ea2800640 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0ea2800640 1 -- 172.21.15.83:0/1431528672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 msgr2=0x558c694a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0ea2800640 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0ea2800640 1 -- 172.21.15.83:0/1431528672 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c693ce3f0 con 0x558c6949e950 2024-04-23T14:56:56.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0ea1e00640 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0ea3200640 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0ea2800640 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 0x558c694a9350 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40792c0 tx=0x7f0e94004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bf1d7301507cce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0e83e00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:56:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0e83e00640 1 -- 172.21.15.83:0/1431528672 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9401f630 con 0x558c6949e950 2024-04-23T14:56:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0eac231740 1 -- 172.21.15.83:0/1431528672 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9c036030 con 0x558c6949e950 2024-04-23T14:56:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0e83e00640 1 -- 172.21.15.83:0/1431528672 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f0e9401f7d0 con 0x558c6949e950 2024-04-23T14:56:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0e83e00640 1 -- 172.21.15.83:0/1431528672 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0e9401f9d0 con 0x558c6949e950 2024-04-23T14:56:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0eac231740 10 client.26872 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:56.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.884+0000 7f0eac231740 1 -- 172.21.15.83:0/1431528672 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558c691163f0 con 0x558c6949e950 2024-04-23T14:56:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.885+0000 7f0e83e00640 1 -- 172.21.15.83:0/1431528672 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c036030 con 0x558c6949e950 2024-04-23T14:56:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.885+0000 7f0e83e00640 1 -- 172.21.15.83:0/1431528672 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0e94027070 con 0x558c6949e950 2024-04-23T14:56:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.885+0000 7f0eac231740 10 client.26872 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.885+0000 7f0eac231740 10 client.26872 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:56:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.885+0000 7f0eac231740 20 client.26872 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.885+0000 7f0eac231740 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x558c694b9ac0 0x558c694bbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.885+0000 7f0eac231740 4 client.26872 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:56:56.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.885+0000 7f0eac231740 1 -- 172.21.15.83:0/1431528672 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c69411bf0 con 0x558c694b9ac0 2024-04-23T14:56:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.886+0000 7f0ea1e00640 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x558c694b9ac0 0x558c694bbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.886+0000 7f0ea1e00640 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x558c694b9ac0 0x558c694bbf10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=180593b2cc31a509 server_cookie=d89393e81f5c295a in_seq=0 out_seq=0 2024-04-23T14:56:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.886+0000 7f0e83e00640 10 client.26872 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:56:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0e83e00640 1 -- 172.21.15.83:0/1431528672 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c69411bf0 con 0x558c694b9ac0 2024-04-23T14:56:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0e83e00640 10 client.26872 handle_command_reply: tid=0 2024-04-23T14:56:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0eac231740 1 client.26872 shutdown 2024-04-23T14:56:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0eac231740 1 -- 172.21.15.83:0/1431528672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 msgr2=0x558c694a9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0eac231740 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 0x558c694a9350 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40792c0 tx=0x7f0e94004730 comp rx=0 tx=0).stop 2024-04-23T14:56:56.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0eac231740 1 -- 172.21.15.83:0/1431528672 shutdown_connections 2024-04-23T14:56:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0eac231740 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0eac231740 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0eac231740 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x558c694b9ac0 0x558c694bbf10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0eac231740 1 --2- 172.21.15.83:0/1431528672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949e950 0x558c694a9350 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0e83e00640 0 client.26872 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:56:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0eac231740 1 -- 172.21.15.83:0/1431528672 >> 172.21.15.83:0/1431528672 conn(0x558c6949ac70 msgr2=0x558c694a0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.887+0000 7f0eac231740 1 -- 172.21.15.83:0/1431528672 shutdown_connections 2024-04-23T14:56:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.888+0000 7f0eac231740 1 -- 172.21.15.83:0/1431528672 wait complete. 2024-04-23T14:56:56.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.888+0000 7f0eac231740 20 client.26872 trim_cache size 0 max 16384 2024-04-23T14:56:56.904 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.902+0000 7f0eac231740 1 Processor -- start 2024-04-23T14:56:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.902+0000 7f0eac231740 1 -- start start 2024-04-23T14:56:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.902+0000 7f0eac231740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949e950 0x558c694b5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.902+0000 7f0eac231740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949f270 0x558c694a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.902+0000 7f0eac231740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694c17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.902+0000 7f0eac231740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b6c30 con 0x558c694a9940 2024-04-23T14:56:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.902+0000 7f0eac231740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b4500 con 0x558c6949e950 2024-04-23T14:56:56.905 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.902+0000 7f0eac231740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b2370 con 0x558c6949f270 2024-04-23T14:56:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.903+0000 7f0ea3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694c17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.903+0000 7f0ea3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694c17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58230/0 (socket says 172.21.15.83:58230) 2024-04-23T14:56:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.903+0000 7f0ea3200640 1 -- 172.21.15.83:0/1893940610 learned_addr learned my addr 172.21.15.83:0/1893940610 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.903+0000 7f0ea1e00640 1 --2- 172.21.15.83:0/1893940610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949f270 0x558c694a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.903+0000 7f0ea2800640 1 --2- 172.21.15.83:0/1893940610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949e950 0x558c694b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.903+0000 7f0ea3200640 1 -- 172.21.15.83:0/1893940610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949f270 msgr2=0x558c694a8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.903+0000 7f0ea3200640 1 --2- 172.21.15.83:0/1893940610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949f270 0x558c694a8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.906 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.903+0000 7f0ea3200640 1 -- 172.21.15.83:0/1893940610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949e950 msgr2=0x558c694b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.903+0000 7f0ea3200640 1 --2- 172.21.15.83:0/1893940610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949e950 0x558c694b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.903+0000 7f0ea3200640 1 -- 172.21.15.83:0/1893940610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c694a6fc0 con 0x558c694a9940 2024-04-23T14:56:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.904+0000 7f0ea3200640 1 --2- 172.21.15.83:0/1893940610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694c17e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c045000 tx=0x7f0e9c02a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb67a2e8148f459 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.904+0000 7f0e8ea00640 1 -- 172.21.15.83:0/1893940610 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c030460 con 0x558c694a9940 2024-04-23T14:56:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.904+0000 7f0e8ea00640 1 -- 172.21.15.83:0/1893940610 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c036050 con 0x558c694a9940 2024-04-23T14:56:56.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.904+0000 7f0e8ea00640 1 -- 172.21.15.83:0/1893940610 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c004570 con 0x558c694a9940 2024-04-23T14:56:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.905+0000 7f0eac231740 1 -- 172.21.15.83:0/1893940610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 msgr2=0x558c694c17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.905+0000 7f0eac231740 1 --2- 172.21.15.83:0/1893940610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694c17e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c045000 tx=0x7f0e9c02a8f0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.905+0000 7f0eac231740 1 -- 172.21.15.83:0/1893940610 shutdown_connections 2024-04-23T14:56:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.905+0000 7f0eac231740 1 --2- 172.21.15.83:0/1893940610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c694a9940 0x558c694c17e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.905+0000 7f0eac231740 1 --2- 172.21.15.83:0/1893940610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c6949f270 0x558c694a8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.905+0000 7f0eac231740 1 --2- 172.21.15.83:0/1893940610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949e950 0x558c694b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.905+0000 7f0eac231740 1 -- 172.21.15.83:0/1893940610 >> 172.21.15.83:0/1893940610 conn(0x558c6949ac70 msgr2=0x558c694a18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.905+0000 7f0eac231740 1 -- 172.21.15.83:0/1893940610 shutdown_connections 2024-04-23T14:56:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.905+0000 7f0eac231740 1 -- 172.21.15.83:0/1893940610 wait complete. 2024-04-23T14:56:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.905+0000 7f0eac231740 1 -- start start 2024-04-23T14:56:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.905+0000 7f0eac231740 1 Processor -- start 2024-04-23T14:56:56.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0eac231740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c6949e950 0x558c694b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0eac231740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694aa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0eac231740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c694a9940 0x558c694aab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0eac231740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b29c0 con 0x558c6949e950 2024-04-23T14:56:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0eac231740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b3e30 con 0x558c6949f270 2024-04-23T14:56:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0eac231740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b3090 con 0x558c694a9940 2024-04-23T14:56:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0eac231740 10 client.0 _ll_register_callbacks cb 0x7f0e94009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0ea2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c6949e950 0x558c694b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0ea2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c6949e950 0x558c694b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58232/0 (socket says 172.21.15.83:58232) 2024-04-23T14:56:56.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0ea2800640 1 -- 172.21.15.83:0/2226639629 learned_addr learned my addr 172.21.15.83:0/2226639629 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:56.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0ea1e00640 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0ea3200640 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c694a9940 0x558c694aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0ea2800640 1 -- 172.21.15.83:0/2226639629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c694a9940 msgr2=0x558c694aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0ea2800640 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c694a9940 0x558c694aab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0ea2800640 1 -- 172.21.15.83:0/2226639629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 msgr2=0x558c694aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:56.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0ea2800640 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694aa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.906+0000 7f0ea2800640 1 -- 172.21.15.83:0/2226639629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c691e07d0 con 0x558c6949e950 2024-04-23T14:56:56.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.907+0000 7f0ea2800640 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c6949e950 0x558c694b2090 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0e94004700 tx=0x7f0e940296f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40871818c862c95e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:56.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.907+0000 7f0e83e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.907+0000 7f0e83e00640 1 -- 172.21.15.83:0/2226639629 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e940043f0 con 0x558c6949e950 2024-04-23T14:56:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.907+0000 7f0e83e00640 1 -- 172.21.15.83:0/2226639629 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f0e9401fd80 con 0x558c6949e950 2024-04-23T14:56:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.907+0000 7f0eac231740 1 -- 172.21.15.83:0/2226639629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9c036050 con 0x558c6949e950 2024-04-23T14:56:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.907+0000 7f0e83e00640 1 -- 172.21.15.83:0/2226639629 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0e940298e0 con 0x558c6949e950 2024-04-23T14:56:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.907+0000 7f0eac231740 10 client.16704 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.907+0000 7f0eac231740 1 -- 172.21.15.83:0/2226639629 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558c69520050 con 0x558c6949e950 2024-04-23T14:56:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.907+0000 7f0e83e00640 1 -- 172.21.15.83:0/2226639629 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c036050 con 0x558c6949e950 2024-04-23T14:56:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.908+0000 7f0e83e00640 1 -- 172.21.15.83:0/2226639629 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0e94038020 con 0x558c6949e950 2024-04-23T14:56:56.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.908+0000 7f0eac231740 10 client.16704 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:56.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.908+0000 7f0eac231740 10 client.16704 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:56:56.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.908+0000 7f0eac231740 20 client.16704 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:56.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.908+0000 7f0eac231740 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x558c69520770 0x558c69522b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:56.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.908+0000 7f0eac231740 4 client.16704 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:56:56.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.908+0000 7f0eac231740 1 -- 172.21.15.83:0/2226639629 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c6940ba60 con 0x558c69520770 2024-04-23T14:56:56.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.908+0000 7f0ea1e00640 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x558c69520770 0x558c69522b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:56.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0e83e00640 10 client.16704 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:56:56.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0ea1e00640 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x558c69520770 0x558c69522b90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20477f2b6abe67d6 server_cookie=eee7ecb0bd2c83b6 in_seq=0 out_seq=0 2024-04-23T14:56:56.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0e83e00640 1 -- 172.21.15.83:0/2226639629 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c6940ba60 con 0x558c69520770 2024-04-23T14:56:56.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0e83e00640 10 client.16704 handle_command_reply: tid=0 2024-04-23T14:56:56.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0eac231740 1 client.16704 shutdown 2024-04-23T14:56:56.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0eac231740 1 -- 172.21.15.83:0/2226639629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c6949e950 msgr2=0x558c694b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0eac231740 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c6949e950 0x558c694b2090 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0e94004700 tx=0x7f0e940296f0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0eac231740 1 -- 172.21.15.83:0/2226639629 shutdown_connections 2024-04-23T14:56:56.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0eac231740 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x558c69520770 0x558c69522b90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0eac231740 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558c694a9940 0x558c694aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0e83e00640 0 client.16704 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:56:56.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0eac231740 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558c6949f270 0x558c694aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0eac231740 1 --2- 172.21.15.83:0/2226639629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558c6949e950 0x558c694b2090 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0eac231740 1 -- 172.21.15.83:0/2226639629 >> 172.21.15.83:0/2226639629 conn(0x558c6949ac70 msgr2=0x558c694a0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0eac231740 1 -- 172.21.15.83:0/2226639629 shutdown_connections 2024-04-23T14:56:56.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0eac231740 1 -- 172.21.15.83:0/2226639629 wait complete. 2024-04-23T14:56:56.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.909+0000 7f0eac231740 20 client.16704 trim_cache size 0 max 16384 2024-04-23T14:56:56.920 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:56:56.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.910+0000 7f0ea9400640 1 -- 172.21.15.83:0/993073929 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0e880a3d20 msgr2=0x7f0e880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.910+0000 7f0ea9400640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0e880a3d20 0x7f0e880a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x558c6949ff10 tx=0x7f0e94005f90 comp rx=0 tx=0).stop 2024-04-23T14:56:56.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.910+0000 7f0ea9400640 1 -- 172.21.15.83:0/993073929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4108fd0 msgr2=0x7f0ea419f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:56.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.910+0000 7f0ea9400640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4108fd0 0x7f0ea419f380 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000e9f0 tx=0x7f0e9000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.910+0000 7f0ea9400640 1 -- 172.21.15.83:0/993073929 shutdown_connections 2024-04-23T14:56:56.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.910+0000 7f0ea9400640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ea410e660 0x7f0ea41a36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.921 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.910+0000 7f0ea9400640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0e880a3d20 0x7f0e880a61e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.910+0000 7f0ea9400640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4108fd0 0x7f0ea419f380 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.910+0000 7f0ea9400640 1 --2- 172.21.15.83:0/993073929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ea4108610 0x7f0ea419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:56.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.910+0000 7f0ea9400640 1 -- 172.21.15.83:0/993073929 >> 172.21.15.83:0/993073929 conn(0x7f0ea4076540 msgr2=0x7f0ea4077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:56.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.910+0000 7f0ea9400640 1 -- 172.21.15.83:0/993073929 shutdown_connections 2024-04-23T14:56:56.922 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:56.910+0000 7f0ea9400640 1 -- 172.21.15.83:0/993073929 wait complete. 2024-04-23T14:56:56.928 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:56:57.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.120+0000 7f0b8d800640 1 -- 172.21.15.83:0/2908116833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b88102ad0 msgr2=0x7f0b88106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:57.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.120+0000 7f0b8d800640 1 --2- 172.21.15.83:0/2908116833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b88102ad0 0x7f0b88106b10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009900 tx=0x7f0b7002a410 comp rx=0 tx=0).stop 2024-04-23T14:56:57.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.120+0000 7f0b8d800640 1 -- 172.21.15.83:0/2908116833 shutdown_connections 2024-04-23T14:56:57.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.120+0000 7f0b8d800640 1 --2- 172.21.15.83:0/2908116833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b881071f0 0x7f0b8810edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.120+0000 7f0b8d800640 1 --2- 172.21.15.83:0/2908116833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b88102ad0 0x7f0b88106b10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.120+0000 7f0b8d800640 1 --2- 172.21.15.83:0/2908116833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b88102120 0x7f0b88102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.120+0000 7f0b8d800640 1 -- 172.21.15.83:0/2908116833 >> 172.21.15.83:0/2908116833 conn(0x7f0b880fd9a0 msgr2=0x7f0b880ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:57.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.120+0000 7f0b8d800640 1 -- 172.21.15.83:0/2908116833 shutdown_connections 2024-04-23T14:56:57.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.120+0000 7f0b8d800640 1 -- 172.21.15.83:0/2908116833 wait complete. 2024-04-23T14:56:57.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.121+0000 7f0b8d800640 1 Processor -- start 2024-04-23T14:56:57.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.122+0000 7f0b8d800640 1 -- start start 2024-04-23T14:56:57.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.122+0000 7f0b8d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b88102120 0x7f0b88074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.122+0000 7f0b8d800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b881071f0 0x7f0b88074840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.122+0000 7f0b8d800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b88074d80 0x7f0b881a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.122+0000 7f0b8d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881122d0 con 0x7f0b88102120 2024-04-23T14:56:57.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.122+0000 7f0b8d800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0b88112150 con 0x7f0b88074d80 2024-04-23T14:56:57.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.122+0000 7f0b8d800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0b88112450 con 0x7f0b881071f0 2024-04-23T14:56:57.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.122+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b88102120 0x7f0b88074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.122+0000 7f0b87400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b881071f0 0x7f0b88074840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.122+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b88102120 0x7f0b88074300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58264/0 (socket says 172.21.15.83:58264) 2024-04-23T14:56:57.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.123+0000 7f0b87e00640 1 -- 172.21.15.83:0/2723237112 learned_addr learned my addr 172.21.15.83:0/2723237112 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:57.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.123+0000 7f0b8ce00640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b88074d80 0x7f0b881a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.123+0000 7f0b87e00640 1 -- 172.21.15.83:0/2723237112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b881071f0 msgr2=0x7f0b88074840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:57.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.123+0000 7f0b87e00640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b881071f0 0x7f0b88074840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.123+0000 7f0b87e00640 1 -- 172.21.15.83:0/2723237112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b88074d80 msgr2=0x7f0b881a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:57.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.123+0000 7f0b87e00640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b88074d80 0x7f0b881a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.123+0000 7f0b87e00640 1 -- 172.21.15.83:0/2723237112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b881a70c0 con 0x7f0b88102120 2024-04-23T14:56:57.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.123+0000 7f0b8ce00640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b88074d80 0x7f0b881a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:56:57.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.123+0000 7f0b87e00640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b88102120 0x7f0b88074300 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800e9f0 tx=0x7f0b7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:57.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.123+0000 7f0b84c00640 1 -- 172.21.15.83:0/2723237112 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7800cd80 con 0x7f0b88102120 2024-04-23T14:56:57.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.124+0000 7f0b8d800640 1 -- 172.21.15.83:0/2723237112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b881a73b0 con 0x7f0b88102120 2024-04-23T14:56:57.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.124+0000 7f0b84c00640 1 -- 172.21.15.83:0/2723237112 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b78005500 con 0x7f0b88102120 2024-04-23T14:56:57.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.124+0000 7f0b84c00640 1 -- 172.21.15.83:0/2723237112 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78010690 con 0x7f0b88102120 2024-04-23T14:56:57.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.124+0000 7f0b8d800640 1 -- 172.21.15.83:0/2723237112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b881a78d0 con 0x7f0b88102120 2024-04-23T14:56:57.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.127+0000 7f0b91c0c740 1 Processor -- start 2024-04-23T14:56:57.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.127+0000 7f0b91c0c740 1 -- start start 2024-04-23T14:56:57.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.127+0000 7f0b84c00640 1 -- 172.21.15.83:0/2723237112 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0b780046c0 con 0x7f0b88102120 2024-04-23T14:56:57.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.127+0000 7f0b91c0c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28563fb40 0x55f28563ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.127+0000 7f0b91c0c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.127+0000 7f0b91c0c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28564ab30 0x55f28564cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.127+0000 7f0b91c0c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f2855a51d0 con 0x55f28564ab30 2024-04-23T14:56:57.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.127+0000 7f0b91c0c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f28554b450 con 0x55f285640460 2024-04-23T14:56:57.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.127+0000 7f0b91c0c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f2855a1190 con 0x55f28563fb40 2024-04-23T14:56:57.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.128+0000 7f0b87400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28564ab30 0x55f28564cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.128+0000 7f0b87400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28564ab30 0x55f28564cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58274/0 (socket says 172.21.15.83:58274) 2024-04-23T14:56:57.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.128+0000 7f0b87400640 1 -- 172.21.15.83:0/1731575349 learned_addr learned my addr 172.21.15.83:0/1731575349 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:57.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.128+0000 7f0b8ce00640 1 --2- 172.21.15.83:0/1731575349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28563fb40 0x55f28563ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.128+0000 7f0b87e00640 1 --2- 172.21.15.83:0/1731575349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.128+0000 7f0b87400640 1 -- 172.21.15.83:0/1731575349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28563fb40 msgr2=0x55f28563ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:57.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.128+0000 7f0b87400640 1 --2- 172.21.15.83:0/1731575349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28563fb40 0x55f28563ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.128+0000 7f0b87400640 1 -- 172.21.15.83:0/1731575349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 msgr2=0x55f28564a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:57.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.128+0000 7f0b87400640 1 --2- 172.21.15.83:0/1731575349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.128+0000 7f0b87400640 1 -- 172.21.15.83:0/1731575349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f285608310 con 0x55f28564ab30 2024-04-23T14:56:57.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.128+0000 7f0b87e00640 1 --2- 172.21.15.83:0/1731575349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:56:57.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.128+0000 7f0b87400640 1 --2- 172.21.15.83:0/1731575349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28564ab30 0x55f28564cf20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009900 tx=0x7f0b70005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acbd3f18c6242a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:57.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.129+0000 7f0b7d600640 1 -- 172.21.15.83:0/1731575349 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b70035040 con 0x55f28564ab30 2024-04-23T14:56:57.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.129+0000 7f0b7d600640 1 -- 172.21.15.83:0/1731575349 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b70005520 con 0x55f28564ab30 2024-04-23T14:56:57.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.129+0000 7f0b7d600640 1 -- 172.21.15.83:0/1731575349 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b700035f0 con 0x55f28564ab30 2024-04-23T14:56:57.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.129+0000 7f0b84c00640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0b6c0a3dd0 0x7f0b6c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.133 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.129+0000 7f0b84c00640 1 -- 172.21.15.83:0/2723237112 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f0b780d0c80 con 0x7f0b88102120 2024-04-23T14:56:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.129+0000 7f0b8ce00640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0b6c0a3dd0 0x7f0b6c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.129+0000 7f0b91c0c740 1 -- 172.21.15.83:0/1731575349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28564ab30 msgr2=0x55f28564cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.129+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/1731575349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28564ab30 0x55f28564cf20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009900 tx=0x7f0b70005de0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.130+0000 7f0b91c0c740 1 -- 172.21.15.83:0/1731575349 shutdown_connections 2024-04-23T14:56:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.130+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/1731575349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28564ab30 0x55f28564cf20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.130+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/1731575349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.130+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/1731575349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28563fb40 0x55f28563ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.130+0000 7f0b91c0c740 1 -- 172.21.15.83:0/1731575349 >> 172.21.15.83:0/1731575349 conn(0x55f28563be60 msgr2=0x55f28563c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:57.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.130+0000 7f0b91c0c740 1 -- 172.21.15.83:0/1731575349 shutdown_connections 2024-04-23T14:56:57.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.130+0000 7f0b91c0c740 1 -- 172.21.15.83:0/1731575349 wait complete. 2024-04-23T14:56:57.135 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.130+0000 7f0b8ce00640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0b6c0a3dd0 0x7f0b6c0a6290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0b80005fd0 tx=0x7f0b80009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:57.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.130+0000 7f0b91c0c740 1 -- start start 2024-04-23T14:56:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.131+0000 7f0b91c0c740 1 Processor -- start 2024-04-23T14:56:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.131+0000 7f0b91c0c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f28564a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.131+0000 7f0b91c0c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f285640460 0x55f285647870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.131+0000 7f0b91c0c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f28564ab30 0x55f285647fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.131+0000 7f0b91c0c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f28564d960 con 0x55f28563fb40 2024-04-23T14:56:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.131+0000 7f0b91c0c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f28555e810 con 0x55f28564ab30 2024-04-23T14:56:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.131+0000 7f0b91c0c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f28564dae0 con 0x55f285640460 2024-04-23T14:56:57.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.131+0000 7f0b91c0c740 10 client.0 _ll_register_callbacks cb 0x55f28552c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.131+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f28564a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.131+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f28564a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58278/0 (socket says 172.21.15.83:58278) 2024-04-23T14:56:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.131+0000 7f0b87e00640 1 -- 172.21.15.83:0/74089543 learned_addr learned my addr 172.21.15.83:0/74089543 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.131+0000 7f0b87400640 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f285640460 0x55f285647870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b8ce00640 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f28564ab30 0x55f285647fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b87e00640 1 -- 172.21.15.83:0/74089543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f285640460 msgr2=0x55f285647870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b87e00640 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f285640460 0x55f285647870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b87e00640 1 -- 172.21.15.83:0/74089543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f28564ab30 msgr2=0x55f285647fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:57.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b87e00640 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f28564ab30 0x55f285647fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b87e00640 1 -- 172.21.15.83:0/74089543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f28556f3f0 con 0x55f28563fb40 2024-04-23T14:56:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b87e00640 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f28564a580 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800e9c0 tx=0x7f0b7809ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79583de0bb2dcac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b76000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b76000640 1 -- 172.21.15.83:0/74089543 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780ce7e0 con 0x55f28563fb40 2024-04-23T14:56:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b76000640 1 -- 172.21.15.83:0/74089543 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f0b780a5390 con 0x55f28563fb40 2024-04-23T14:56:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b91c0c740 1 -- 172.21.15.83:0/74089543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b70005520 con 0x55f28563fb40 2024-04-23T14:56:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b76000640 1 -- 172.21.15.83:0/74089543 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b7809cbb0 con 0x55f28563fb40 2024-04-23T14:56:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.132+0000 7f0b91c0c740 10 client.16716 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:57.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.133+0000 7f0b91c0c740 1 -- 172.21.15.83:0/74089543 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2852b73f0 con 0x55f28563fb40 2024-04-23T14:56:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.133+0000 7f0b76000640 1 -- 172.21.15.83:0/74089543 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b70005520 con 0x55f28563fb40 2024-04-23T14:56:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.133+0000 7f0b76000640 1 -- 172.21.15.83:0/74089543 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0b780d8040 con 0x55f28563fb40 2024-04-23T14:56:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.133+0000 7f0b91c0c740 10 client.16716 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.133+0000 7f0b91c0c740 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.133+0000 7f0b91c0c740 20 client.16716 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.133+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f28565adb0 0x55f28565d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.133+0000 7f0b91c0c740 4 client.16716 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:56:57.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.133+0000 7f0b91c0c740 1 -- 172.21.15.83:0/74089543 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2855b2bf0 con 0x55f28565adb0 2024-04-23T14:56:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.134+0000 7f0b87400640 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f28565adb0 0x55f28565d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.134+0000 7f0b87400640 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f28565adb0 0x55f28565d1a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51eed93b68359857 server_cookie=9d8b50e654adc346 in_seq=0 out_seq=0 2024-04-23T14:56:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.134+0000 7f0b76000640 10 client.16716 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b76000640 1 -- 172.21.15.83:0/74089543 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2855b2bf0 con 0x55f28565adb0 2024-04-23T14:56:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b76000640 10 client.16716 handle_command_reply: tid=0 2024-04-23T14:56:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b91c0c740 1 client.16716 shutdown 2024-04-23T14:56:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b91c0c740 1 -- 172.21.15.83:0/74089543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 msgr2=0x55f28564a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f28564a580 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800e9c0 tx=0x7f0b7809ca40 comp rx=0 tx=0).stop 2024-04-23T14:56:57.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b91c0c740 1 -- 172.21.15.83:0/74089543 shutdown_connections 2024-04-23T14:56:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f28565adb0 0x55f28565d1a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f28564ab30 0x55f285647fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f285640460 0x55f285647870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/74089543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f28564a580 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b76000640 0 client.16716 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b91c0c740 1 -- 172.21.15.83:0/74089543 >> 172.21.15.83:0/74089543 conn(0x55f28563be60 msgr2=0x55f285641c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.135+0000 7f0b91c0c740 1 -- 172.21.15.83:0/74089543 shutdown_connections 2024-04-23T14:56:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.136+0000 7f0b91c0c740 1 -- 172.21.15.83:0/74089543 wait complete. 2024-04-23T14:56:57.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.136+0000 7f0b91c0c740 20 client.16716 trim_cache size 0 max 16384 2024-04-23T14:56:57.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.150+0000 7f0b91c0c740 1 Processor -- start 2024-04-23T14:56:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.150+0000 7f0b91c0c740 1 -- start start 2024-04-23T14:56:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b91c0c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f285656880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b91c0c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b91c0c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28564ab30 0x55f28564a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b91c0c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881b26d0 con 0x55f28563fb40 2024-04-23T14:56:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b91c0c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881affa0 con 0x55f285640460 2024-04-23T14:56:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b91c0c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881ade10 con 0x55f28564ab30 2024-04-23T14:56:57.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f285656880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f285656880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58284/0 (socket says 172.21.15.83:58284) 2024-04-23T14:56:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b87e00640 1 -- 172.21.15.83:0/3170042006 learned_addr learned my addr 172.21.15.83:0/3170042006 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b87400640 1 --2- 172.21.15.83:0/3170042006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b8ce00640 1 --2- 172.21.15.83:0/3170042006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28564ab30 0x55f28564a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b87e00640 1 -- 172.21.15.83:0/3170042006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28564ab30 msgr2=0x55f28564a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b87e00640 1 --2- 172.21.15.83:0/3170042006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28564ab30 0x55f28564a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b87e00640 1 -- 172.21.15.83:0/3170042006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 msgr2=0x55f28564a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b87e00640 1 --2- 172.21.15.83:0/3170042006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.151+0000 7f0b87e00640 1 -- 172.21.15.83:0/3170042006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2853817d0 con 0x55f28563fb40 2024-04-23T14:56:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.152+0000 7f0b87e00640 1 --2- 172.21.15.83:0/3170042006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f285656880 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0b780a52f0 tx=0x7f0b780057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9c522179394d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.152+0000 7f0b7cc00640 1 -- 172.21.15.83:0/3170042006 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780ce7e0 con 0x55f28563fb40 2024-04-23T14:56:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.152+0000 7f0b7cc00640 1 -- 172.21.15.83:0/3170042006 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b780d4200 con 0x55f28563fb40 2024-04-23T14:56:57.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.152+0000 7f0b7cc00640 1 -- 172.21.15.83:0/3170042006 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b780ceab0 con 0x55f28563fb40 2024-04-23T14:56:57.159 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.152+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3170042006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 msgr2=0x55f285656880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.152+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/3170042006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f285656880 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0b780a52f0 tx=0x7f0b780057b0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.153+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3170042006 shutdown_connections 2024-04-23T14:56:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.153+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/3170042006 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28564ab30 0x55f28564a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.153+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/3170042006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.153+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/3170042006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f285656880 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.153+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3170042006 >> 172.21.15.83:0/3170042006 conn(0x55f28563be60 msgr2=0x55f28563c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.153+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3170042006 shutdown_connections 2024-04-23T14:56:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.153+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3170042006 wait complete. 2024-04-23T14:56:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.153+0000 7f0b91c0c740 1 -- start start 2024-04-23T14:56:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.153+0000 7f0b91c0c740 1 Processor -- start 2024-04-23T14:56:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b91c0c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f28564b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b91c0c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b91c0c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28564ab30 0x55f28564c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b91c0c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881ae460 con 0x55f28563fb40 2024-04-23T14:56:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b91c0c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881af8d0 con 0x55f285640460 2024-04-23T14:56:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b91c0c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0b881aeb30 con 0x55f28564ab30 2024-04-23T14:56:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b91c0c740 10 client.0 _ll_register_callbacks cb 0x7f0b78004ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f28564b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b8ce00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28564ab30 0x55f28564c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b87400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b87e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f28564b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58288/0 (socket says 172.21.15.83:58288) 2024-04-23T14:56:57.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b87e00640 1 -- 172.21.15.83:0/3730060878 learned_addr learned my addr 172.21.15.83:0/3730060878 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b87e00640 1 -- 172.21.15.83:0/3730060878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28564ab30 msgr2=0x55f28564c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b87e00640 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28564ab30 0x55f28564c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b87e00640 1 -- 172.21.15.83:0/3730060878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 msgr2=0x55f28564be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b87e00640 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b87e00640 1 -- 172.21.15.83:0/3730060878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f285651200 con 0x55f28563fb40 2024-04-23T14:56:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b87400640 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.154+0000 7f0b8ce00640 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28564ab30 0x55f28564c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:57.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.155+0000 7f0b87e00640 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f28564b940 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0b780049d0 tx=0x7f0b780a55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b946b720bed154 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.155+0000 7f0b76000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:56:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.155+0000 7f0b76000640 1 -- 172.21.15.83:0/3730060878 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78098b00 con 0x55f28563fb40 2024-04-23T14:56:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.155+0000 7f0b76000640 1 -- 172.21.15.83:0/3730060878 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f0b780ceab0 con 0x55f28563fb40 2024-04-23T14:56:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.155+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3730060878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f2855aca60 con 0x55f28563fb40 2024-04-23T14:56:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.155+0000 7f0b76000640 1 -- 172.21.15.83:0/3730060878 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b7809c210 con 0x55f28563fb40 2024-04-23T14:56:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.155+0000 7f0b91c0c740 10 client.16722 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.155+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3730060878 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2856c16e0 con 0x55f28563fb40 2024-04-23T14:56:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.155+0000 7f0b76000640 1 -- 172.21.15.83:0/3730060878 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f2855aca60 con 0x55f28563fb40 2024-04-23T14:56:57.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.155+0000 7f0b76000640 1 -- 172.21.15.83:0/3730060878 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0b780ac060 con 0x55f28563fb40 2024-04-23T14:56:57.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.156+0000 7f0b91c0c740 10 client.16722 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:57.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.156+0000 7f0b91c0c740 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:57.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.156+0000 7f0b91c0c740 20 client.16722 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:57.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.156+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f2856c1a70 0x55f2856c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:57.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.156+0000 7f0b91c0c740 4 client.16722 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:56:57.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.156+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3730060878 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f2855284d0 con 0x55f2856c1a70 2024-04-23T14:56:57.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.156+0000 7f0b87400640 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f2856c1a70 0x55f2856c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:57.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b76000640 10 client.16722 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:57.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b87400640 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f2856c1a70 0x55f2856c3e60 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7026661c118d94e server_cookie=904f90fe6e3484a3 in_seq=0 out_seq=0 2024-04-23T14:56:57.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b76000640 1 -- 172.21.15.83:0/3730060878 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f2855284d0 con 0x55f2856c1a70 2024-04-23T14:56:57.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b76000640 10 client.16722 handle_command_reply: tid=0 2024-04-23T14:56:57.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b91c0c740 1 client.16722 shutdown 2024-04-23T14:56:57.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3730060878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 msgr2=0x55f28564b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:57.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f28564b940 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0b780049d0 tx=0x7f0b780a55c0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3730060878 shutdown_connections 2024-04-23T14:56:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55f2856c1a70 0x55f2856c3e60 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f28564ab30 0x55f28564c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f285640460 0x55f28564be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b76000640 0 client.16722 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b91c0c740 1 --2- 172.21.15.83:0/3730060878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f28563fb40 0x55f28564b940 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3730060878 >> 172.21.15.83:0/3730060878 conn(0x55f28563be60 msgr2=0x55f28563a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3730060878 shutdown_connections 2024-04-23T14:56:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b91c0c740 1 -- 172.21.15.83:0/3730060878 wait complete. 2024-04-23T14:56:57.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.157+0000 7f0b91c0c740 20 client.16722 trim_cache size 0 max 16384 2024-04-23T14:56:57.168 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:56:57.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.158+0000 7f0b8d800640 1 -- 172.21.15.83:0/2723237112 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0b6c0a3dd0 msgr2=0x7f0b6c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:57.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.158+0000 7f0b8d800640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0b6c0a3dd0 0x7f0b6c0a6290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0b80005fd0 tx=0x7f0b80009500 comp rx=0 tx=0).stop 2024-04-23T14:56:57.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.158+0000 7f0b8d800640 1 -- 172.21.15.83:0/2723237112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b88102120 msgr2=0x7f0b88074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:57.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.158+0000 7f0b8d800640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b88102120 0x7f0b88074300 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800e9f0 tx=0x7f0b7800edf0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.158+0000 7f0b8d800640 1 -- 172.21.15.83:0/2723237112 shutdown_connections 2024-04-23T14:56:57.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.158+0000 7f0b8d800640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b88074d80 0x7f0b881a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.158+0000 7f0b8d800640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b881071f0 0x7f0b88074840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.158+0000 7f0b8d800640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0b6c0a3dd0 0x7f0b6c0a6290 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.158+0000 7f0b8d800640 1 --2- 172.21.15.83:0/2723237112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b88102120 0x7f0b88074300 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.158+0000 7f0b8d800640 1 -- 172.21.15.83:0/2723237112 >> 172.21.15.83:0/2723237112 conn(0x7f0b880fd9a0 msgr2=0x7f0b880ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.158+0000 7f0b8d800640 1 -- 172.21.15.83:0/2723237112 shutdown_connections 2024-04-23T14:56:57.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:57.158+0000 7f0b8d800640 1 -- 172.21.15.83:0/2723237112 wait complete. 2024-04-23T14:56:57.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:57 smithi203 ceph-mon[32800]: pgmap v262: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:56:57.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:57 smithi083 ceph-mon[27867]: pgmap v262: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:56:57.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:57 smithi173 ceph-mon[32443]: pgmap v262: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:56:58.177 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:56:58.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:56:58.177 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:56:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.394+0000 7f0057200640 1 -- 172.21.15.83:0/1249528575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0058104cd0 msgr2=0x7f0058108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.394+0000 7f0057200640 1 --2- 172.21.15.83:0/1249528575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0058104cd0 0x7f0058108d10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0040009900 tx=0x7f004002a400 comp rx=0 tx=0).stop 2024-04-23T14:56:58.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.395+0000 7f0057200640 1 -- 172.21.15.83:0/1249528575 shutdown_connections 2024-04-23T14:56:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.395+0000 7f0057200640 1 --2- 172.21.15.83:0/1249528575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00581093f0 0x7f0058110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.395+0000 7f0057200640 1 --2- 172.21.15.83:0/1249528575 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0058104cd0 0x7f0058108d10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.395+0000 7f0057200640 1 --2- 172.21.15.83:0/1249528575 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0058104320 0x7f0058104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.395+0000 7f0057200640 1 -- 172.21.15.83:0/1249528575 >> 172.21.15.83:0/1249528575 conn(0x7f00580ffb40 msgr2=0x7f0058101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.395+0000 7f0057200640 1 -- 172.21.15.83:0/1249528575 shutdown_connections 2024-04-23T14:56:58.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.395+0000 7f0057200640 1 -- 172.21.15.83:0/1249528575 wait complete. 2024-04-23T14:56:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.396+0000 7f0057200640 1 Processor -- start 2024-04-23T14:56:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.396+0000 7f0057200640 1 -- start start 2024-04-23T14:56:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.396+0000 7f0057200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0058104320 0x7f00581a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0057200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00581093f0 0x7f00581a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0057200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f005819a850 0x7f005819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0057200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f00581144b0 con 0x7f0058104320 2024-04-23T14:56:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0057200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0058114330 con 0x7f00581093f0 2024-04-23T14:56:58.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0057200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0058114630 con 0x7f005819a850 2024-04-23T14:56:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0055e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0058104320 0x7f00581a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0055e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0058104320 0x7f00581a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58294/0 (socket says 172.21.15.83:58294) 2024-04-23T14:56:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0055e00640 1 -- 172.21.15.83:0/4184849748 learned_addr learned my addr 172.21.15.83:0/4184849748 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f004fe00640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00581093f0 0x7f00581a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0056800640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f005819a850 0x7f005819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0055e00640 1 -- 172.21.15.83:0/4184849748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f005819a850 msgr2=0x7f005819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.401 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0055e00640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f005819a850 0x7f005819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0055e00640 1 -- 172.21.15.83:0/4184849748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00581093f0 msgr2=0x7f00581a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0055e00640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00581093f0 0x7f00581a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.397+0000 7f0055e00640 1 -- 172.21.15.83:0/4184849748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005819b3e0 con 0x7f0058104320 2024-04-23T14:56:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.398+0000 7f0055e00640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0058104320 0x7f00581a0690 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f004800e9f0 tx=0x7f004800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.398+0000 7f004ea00640 1 -- 172.21.15.83:0/4184849748 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004800cd80 con 0x7f0058104320 2024-04-23T14:56:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.398+0000 7f004ea00640 1 -- 172.21.15.83:0/4184849748 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0048010910 con 0x7f0058104320 2024-04-23T14:56:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.398+0000 7f0057200640 1 -- 172.21.15.83:0/4184849748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005819b6d0 con 0x7f0058104320 2024-04-23T14:56:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.398+0000 7f004ea00640 1 -- 172.21.15.83:0/4184849748 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048018990 con 0x7f0058104320 2024-04-23T14:56:58.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.398+0000 7f0057200640 1 -- 172.21.15.83:0/4184849748 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00581a7830 con 0x7f0058104320 2024-04-23T14:56:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.402+0000 7f005fe7d740 1 Processor -- start 2024-04-23T14:56:58.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.403+0000 7f005fe7d740 1 -- start start 2024-04-23T14:56:58.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.403+0000 7f004ea00640 1 -- 172.21.15.83:0/4184849748 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0048010ac0 con 0x7f0058104320 2024-04-23T14:56:58.405 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.403+0000 7f005fe7d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.403+0000 7f005fe7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.403+0000 7f005fe7d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.403+0000 7f005fe7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559263628e70 con 0x5592636c3b90 2024-04-23T14:56:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.403+0000 7f005fe7d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5592635cf380 con 0x5592636c3270 2024-04-23T14:56:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.403+0000 7f005fe7d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559263624e30 con 0x5592636ce260 2024-04-23T14:56:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.403+0000 7f0055e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.404+0000 7f004fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.404+0000 7f0055e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58298/0 (socket says 172.21.15.83:58298) 2024-04-23T14:56:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.404+0000 7f0055e00640 1 -- 172.21.15.83:0/2039639375 learned_addr learned my addr 172.21.15.83:0/2039639375 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.404+0000 7f0056800640 1 --2- 172.21.15.83:0/2039639375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.404+0000 7f004fe00640 1 -- 172.21.15.83:0/2039639375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 msgr2=0x5592636c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.404+0000 7f004fe00640 1 --2- 172.21.15.83:0/2039639375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.404+0000 7f004fe00640 1 -- 172.21.15.83:0/2039639375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 msgr2=0x5592636cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.404+0000 7f004fe00640 1 --2- 172.21.15.83:0/2039639375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.404+0000 7f004fe00640 1 -- 172.21.15.83:0/2039639375 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592634037e0 con 0x5592636ce260 2024-04-23T14:56:58.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.404+0000 7f004fe00640 1 --2- 172.21.15.83:0/2039639375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636d0650 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f004002a920 tx=0x7f0040005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e81e6117a47ad786 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.405+0000 7f004ea00640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f003c0a3d20 0x7f003c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.405+0000 7f0047400640 1 -- 172.21.15.83:0/2039639375 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0040035040 con 0x5592636ce260 2024-04-23T14:56:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.405+0000 7f0047400640 1 -- 172.21.15.83:0/2039639375 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0040004480 con 0x5592636ce260 2024-04-23T14:56:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.405+0000 7f0047400640 1 -- 172.21.15.83:0/2039639375 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00400035f0 con 0x5592636ce260 2024-04-23T14:56:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.405+0000 7f004ea00640 1 -- 172.21.15.83:0/4184849748 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f00480d0b40 con 0x7f0058104320 2024-04-23T14:56:58.408 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.405+0000 7f0056800640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f003c0a3d20 0x7f003c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.405+0000 7f005fe7d740 1 -- 172.21.15.83:0/2039639375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 msgr2=0x5592636d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.405+0000 7f0056800640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f003c0a3d20 0x7f003c0a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0050005fd0 tx=0x7f0050009600 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.405+0000 7f005fe7d740 1 --2- 172.21.15.83:0/2039639375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636d0650 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f004002a920 tx=0x7f0040005bf0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.406+0000 7f005fe7d740 1 -- 172.21.15.83:0/2039639375 shutdown_connections 2024-04-23T14:56:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.406+0000 7f005fe7d740 1 --2- 172.21.15.83:0/2039639375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636d0650 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.406+0000 7f005fe7d740 1 --2- 172.21.15.83:0/2039639375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.411 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.406+0000 7f005fe7d740 1 --2- 172.21.15.83:0/2039639375 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.406+0000 7f005fe7d740 1 -- 172.21.15.83:0/2039639375 >> 172.21.15.83:0/2039639375 conn(0x5592636bf590 msgr2=0x5592636bf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.406+0000 7f005fe7d740 1 -- 172.21.15.83:0/2039639375 shutdown_connections 2024-04-23T14:56:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.406+0000 7f005fe7d740 1 -- 172.21.15.83:0/2039639375 wait complete. 2024-04-23T14:56:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.406+0000 7f005fe7d740 1 -- start start 2024-04-23T14:56:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.406+0000 7f005fe7d740 1 Processor -- start 2024-04-23T14:56:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.407+0000 7f005fe7d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.407+0000 7f005fe7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636cb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.412 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.407+0000 7f005fe7d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.407+0000 7f005fe7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5592636d0ff0 con 0x5592636c3b90 2024-04-23T14:56:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.407+0000 7f005fe7d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5592635e2660 con 0x5592636c3270 2024-04-23T14:56:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.407+0000 7f005fe7d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5592636d1170 con 0x5592636ce260 2024-04-23T14:56:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.407+0000 7f005fe7d740 10 client.0 _ll_register_callbacks cb 0x5592635d0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.407+0000 7f004fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.407+0000 7f004fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58306/0 (socket says 172.21.15.83:58306) 2024-04-23T14:56:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.407+0000 7f004fe00640 1 -- 172.21.15.83:0/3285814671 learned_addr learned my addr 172.21.15.83:0/3285814671 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.407+0000 7f0055e00640 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.413 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.407+0000 7f0056800640 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.408+0000 7f0055e00640 1 -- 172.21.15.83:0/3285814671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 msgr2=0x5592636cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.408+0000 7f0055e00640 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.408+0000 7f0055e00640 1 -- 172.21.15.83:0/3285814671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 msgr2=0x5592636cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.408+0000 7f0055e00640 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.408+0000 7f0055e00640 1 -- 172.21.15.83:0/3285814671 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559263463f70 con 0x5592636c3270 2024-04-23T14:56:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.408+0000 7f004fe00640 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.408+0000 7f0056800640 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:58.414 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.408+0000 7f0055e00640 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636cb220 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5592636c4830 tx=0x7f0048009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9208346c69350e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.408+0000 7f003be00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:56:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.408+0000 7f003be00640 1 -- 172.21.15.83:0/3285814671 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004809c6c0 con 0x5592636c3270 2024-04-23T14:56:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.408+0000 7f005fe7d740 1 -- 172.21.15.83:0/3285814671 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0040004480 con 0x5592636c3270 2024-04-23T14:56:58.415 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.408+0000 7f003be00640 1 -- 172.21.15.83:0/3285814671 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f0048018b30 con 0x5592636c3270 2024-04-23T14:56:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.412+0000 7f003be00640 1 -- 172.21.15.83:0/3285814671 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f004809c4e0 con 0x5592636c3270 2024-04-23T14:56:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.412+0000 7f005fe7d740 10 client.26933 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.412+0000 7f005fe7d740 1 -- 172.21.15.83:0/3285814671 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592632fb170 con 0x5592636c3270 2024-04-23T14:56:58.417 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.412+0000 7f003be00640 1 -- 172.21.15.83:0/3285814671 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0040004480 con 0x5592636c3270 2024-04-23T14:56:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.412+0000 7f003be00640 1 -- 172.21.15.83:0/3285814671 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f004809bfd0 con 0x5592636c3270 2024-04-23T14:56:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.412+0000 7f005fe7d740 10 client.26933 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.412+0000 7f005fe7d740 10 client.26933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.412+0000 7f005fe7d740 20 client.26933 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.412+0000 7f005fe7d740 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592636de5b0 0x5592636e0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.412+0000 7f005fe7d740 4 client.26933 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:56:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.412+0000 7f005fe7d740 1 -- 172.21.15.83:0/3285814671 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559263636890 con 0x5592636de5b0 2024-04-23T14:56:58.418 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.413+0000 7f004fe00640 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592636de5b0 0x5592636e0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.413+0000 7f004fe00640 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592636de5b0 0x5592636e0a40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9befa40cb43a7e41 server_cookie=1e5804144f880c34 in_seq=0 out_seq=0 2024-04-23T14:56:58.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.413+0000 7f003be00640 10 client.26933 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:58.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.414+0000 7f003be00640 1 -- 172.21.15.83:0/3285814671 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559263636890 con 0x5592636de5b0 2024-04-23T14:56:58.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.414+0000 7f003be00640 10 client.26933 handle_command_reply: tid=0 2024-04-23T14:56:58.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.414+0000 7f005fe7d740 1 client.26933 shutdown 2024-04-23T14:56:58.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.414+0000 7f005fe7d740 1 -- 172.21.15.83:0/3285814671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 msgr2=0x5592636cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.414+0000 7f005fe7d740 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636cb220 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5592636c4830 tx=0x7f0048009700 comp rx=0 tx=0).stop 2024-04-23T14:56:58.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.414+0000 7f005fe7d740 1 -- 172.21.15.83:0/3285814671 shutdown_connections 2024-04-23T14:56:58.419 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.414+0000 7f005fe7d740 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592636de5b0 0x5592636e0a40 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.414+0000 7f005fe7d740 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.414+0000 7f005fe7d740 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.414+0000 7f003be00640 0 client.26933 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:58.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.414+0000 7f005fe7d740 1 --2- 172.21.15.83:0/3285814671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636cb220 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.414+0000 7f005fe7d740 1 -- 172.21.15.83:0/3285814671 >> 172.21.15.83:0/3285814671 conn(0x5592636bf590 msgr2=0x5592636c5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:58.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.415+0000 7f005fe7d740 1 -- 172.21.15.83:0/3285814671 shutdown_connections 2024-04-23T14:56:58.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.415+0000 7f005fe7d740 1 -- 172.21.15.83:0/3285814671 wait complete. 2024-04-23T14:56:58.420 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.415+0000 7f005fe7d740 20 client.26933 trim_cache size 0 max 16384 2024-04-23T14:56:58.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f005fe7d740 1 Processor -- start 2024-04-23T14:56:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f005fe7d740 1 -- start start 2024-04-23T14:56:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f005fe7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3270 0x5592636d9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f005fe7d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636c3b90 0x5592636ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f005fe7d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636ce260 0x5592636be570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f005fe7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f00581b26f0 con 0x5592636c3270 2024-04-23T14:56:58.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f005fe7d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f00581affc0 con 0x5592636ce260 2024-04-23T14:56:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f005fe7d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f00581ade30 con 0x5592636c3b90 2024-04-23T14:56:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f0055e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3270 0x5592636d9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f0055e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3270 0x5592636d9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58320/0 (socket says 172.21.15.83:58320) 2024-04-23T14:56:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f0055e00640 1 -- 172.21.15.83:0/1905475417 learned_addr learned my addr 172.21.15.83:0/1905475417 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f004fe00640 1 --2- 172.21.15.83:0/1905475417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636c3b90 0x5592636ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.434+0000 7f0056800640 1 --2- 172.21.15.83:0/1905475417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636ce260 0x5592636be570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.435+0000 7f0055e00640 1 -- 172.21.15.83:0/1905475417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636c3b90 msgr2=0x5592636ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.435+0000 7f0055e00640 1 --2- 172.21.15.83:0/1905475417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636c3b90 0x5592636ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.435+0000 7f0055e00640 1 -- 172.21.15.83:0/1905475417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636ce260 msgr2=0x5592636be570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.435+0000 7f0055e00640 1 --2- 172.21.15.83:0/1905475417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636ce260 0x5592636be570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.435+0000 7f0055e00640 1 -- 172.21.15.83:0/1905475417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592636c6030 con 0x5592636c3270 2024-04-23T14:56:58.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.435+0000 7f004fe00640 1 --2- 172.21.15.83:0/1905475417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636c3b90 0x5592636ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:58.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.435+0000 7f0055e00640 1 --2- 172.21.15.83:0/1905475417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3270 0x5592636d9eb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f00480ce880 tx=0x7f0048000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aada9f4a42b7a5d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:58.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.435+0000 7f0046a00640 1 -- 172.21.15.83:0/1905475417 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048098e20 con 0x5592636c3270 2024-04-23T14:56:58.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.435+0000 7f0046a00640 1 -- 172.21.15.83:0/1905475417 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00480d4380 con 0x5592636c3270 2024-04-23T14:56:58.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.435+0000 7f0046a00640 1 -- 172.21.15.83:0/1905475417 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00480991f0 con 0x5592636c3270 2024-04-23T14:56:58.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.436+0000 7f005fe7d740 1 -- 172.21.15.83:0/1905475417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3270 msgr2=0x5592636d9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.436+0000 7f005fe7d740 1 --2- 172.21.15.83:0/1905475417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3270 0x5592636d9eb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f00480ce880 tx=0x7f0048000bc0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.436+0000 7f005fe7d740 1 -- 172.21.15.83:0/1905475417 shutdown_connections 2024-04-23T14:56:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.436+0000 7f005fe7d740 1 --2- 172.21.15.83:0/1905475417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636ce260 0x5592636be570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.436+0000 7f005fe7d740 1 --2- 172.21.15.83:0/1905475417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636c3b90 0x5592636ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.436+0000 7f005fe7d740 1 --2- 172.21.15.83:0/1905475417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3270 0x5592636d9eb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.436+0000 7f005fe7d740 1 -- 172.21.15.83:0/1905475417 >> 172.21.15.83:0/1905475417 conn(0x5592636bf590 msgr2=0x5592636bde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.436+0000 7f005fe7d740 1 -- 172.21.15.83:0/1905475417 shutdown_connections 2024-04-23T14:56:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.436+0000 7f005fe7d740 1 -- 172.21.15.83:0/1905475417 wait complete. 2024-04-23T14:56:58.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f005fe7d740 1 -- start start 2024-04-23T14:56:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f005fe7d740 1 Processor -- start 2024-04-23T14:56:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f005fe7d740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636d6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f005fe7d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f005fe7d740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f005fe7d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f00581ae480 con 0x5592636c3b90 2024-04-23T14:56:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f005fe7d740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f00581af8f0 con 0x5592636c3270 2024-04-23T14:56:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f005fe7d740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f00581aeb50 con 0x5592636ce260 2024-04-23T14:56:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f005fe7d740 10 client.0 _ll_register_callbacks cb 0x7f004800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:58.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f004fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f0055e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636d6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f004fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636d1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58322/0 (socket says 172.21.15.83:58322) 2024-04-23T14:56:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.437+0000 7f004fe00640 1 -- 172.21.15.83:0/2427052368 learned_addr learned my addr 172.21.15.83:0/2427052368 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.438+0000 7f0056800640 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.438+0000 7f0055e00640 1 -- 172.21.15.83:0/2427052368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 msgr2=0x5592636d2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.438+0000 7f0055e00640 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636d2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.438+0000 7f0055e00640 1 -- 172.21.15.83:0/2427052368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 msgr2=0x5592636d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.438+0000 7f0055e00640 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.438+0000 7f0055e00640 1 -- 172.21.15.83:0/2427052368 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592636d2820 con 0x5592636c3270 2024-04-23T14:56:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.438+0000 7f004fe00640 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.438+0000 7f0055e00640 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636d6790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f004809cb10 tx=0x7f00480ce330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fb1da0b8bff48c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.438+0000 7f003be00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:56:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.438+0000 7f003be00640 1 -- 172.21.15.83:0/2427052368 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004809cb80 con 0x5592636c3270 2024-04-23T14:56:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.438+0000 7f003be00640 1 -- 172.21.15.83:0/2427052368 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f004809de00 con 0x5592636c3270 2024-04-23T14:56:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.438+0000 7f005fe7d740 1 -- 172.21.15.83:0/2427052368 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00480d4380 con 0x5592636c3270 2024-04-23T14:56:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.439+0000 7f003be00640 1 -- 172.21.15.83:0/2427052368 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00480a0e80 con 0x5592636c3270 2024-04-23T14:56:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.439+0000 7f005fe7d740 10 client.26939 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:58.445 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.439+0000 7f005fe7d740 1 -- 172.21.15.83:0/2427052368 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559263744bd0 con 0x5592636c3270 2024-04-23T14:56:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.439+0000 7f003be00640 1 -- 172.21.15.83:0/2427052368 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00480d4380 con 0x5592636c3270 2024-04-23T14:56:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.439+0000 7f003be00640 1 -- 172.21.15.83:0/2427052368 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0048018c10 con 0x5592636c3270 2024-04-23T14:56:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.439+0000 7f005fe7d740 10 client.26939 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.439+0000 7f005fe7d740 10 client.26939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.439+0000 7f005fe7d740 20 client.26939 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.439+0000 7f005fe7d740 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592637450d0 0x5592637474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.439+0000 7f005fe7d740 4 client.26939 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:56:58.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.439+0000 7f005fe7d740 1 -- 172.21.15.83:0/2427052368 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559263630700 con 0x5592637450d0 2024-04-23T14:56:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.439+0000 7f004fe00640 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592637450d0 0x5592637474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.440+0000 7f004fe00640 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592637450d0 0x5592637474f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf6dbc8cfd37794e server_cookie=7fcca103a78d627e in_seq=0 out_seq=0 2024-04-23T14:56:58.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.440+0000 7f003be00640 10 client.26939 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.441+0000 7f003be00640 1 -- 172.21.15.83:0/2427052368 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559263630700 con 0x5592637450d0 2024-04-23T14:56:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.441+0000 7f003be00640 10 client.26939 handle_command_reply: tid=0 2024-04-23T14:56:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.441+0000 7f005fe7d740 1 client.26939 shutdown 2024-04-23T14:56:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.441+0000 7f005fe7d740 1 -- 172.21.15.83:0/2427052368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 msgr2=0x5592636d6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.441+0000 7f005fe7d740 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636d6790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f004809cb10 tx=0x7f00480ce330 comp rx=0 tx=0).stop 2024-04-23T14:56:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.441+0000 7f005fe7d740 1 -- 172.21.15.83:0/2427052368 shutdown_connections 2024-04-23T14:56:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.441+0000 7f005fe7d740 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5592637450d0 0x5592637474f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.441+0000 7f005fe7d740 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592636ce260 0x5592636d2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.441+0000 7f005fe7d740 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5592636c3b90 0x5592636d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.441+0000 7f005fe7d740 1 --2- 172.21.15.83:0/2427052368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5592636c3270 0x5592636d6790 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.441+0000 7f003be00640 0 client.26939 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.441+0000 7f005fe7d740 1 -- 172.21.15.83:0/2427052368 >> 172.21.15.83:0/2427052368 conn(0x5592636bf590 msgr2=0x5592636cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.442+0000 7f005fe7d740 1 -- 172.21.15.83:0/2427052368 shutdown_connections 2024-04-23T14:56:58.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.442+0000 7f005fe7d740 1 -- 172.21.15.83:0/2427052368 wait complete. 2024-04-23T14:56:58.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.442+0000 7f005fe7d740 20 client.26939 trim_cache size 0 max 16384 2024-04-23T14:56:58.450 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:56:58.450 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:56:58.450 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "ab7482d9-95a8-49e0-b540-ad952c3d73f6", 2024-04-23T14:56:58.450 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:56:58.450 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:56:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.442+0000 7f0057200640 1 -- 172.21.15.83:0/4184849748 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f003c0a3d20 msgr2=0x7f003c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.442+0000 7f0057200640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f003c0a3d20 0x7f003c0a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0050005fd0 tx=0x7f0050009600 comp rx=0 tx=0).stop 2024-04-23T14:56:58.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.442+0000 7f0057200640 1 -- 172.21.15.83:0/4184849748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0058104320 msgr2=0x7f00581a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.442+0000 7f0057200640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0058104320 0x7f00581a0690 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f004800e9f0 tx=0x7f004800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.442+0000 7f0057200640 1 -- 172.21.15.83:0/4184849748 shutdown_connections 2024-04-23T14:56:58.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.442+0000 7f0057200640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f005819a850 0x7f005819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.442+0000 7f0057200640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00581093f0 0x7f00581a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.443+0000 7f0057200640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f003c0a3d20 0x7f003c0a61e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.443+0000 7f0057200640 1 --2- 172.21.15.83:0/4184849748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0058104320 0x7f00581a0690 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.443+0000 7f0057200640 1 -- 172.21.15.83:0/4184849748 >> 172.21.15.83:0/4184849748 conn(0x7f00580ffb40 msgr2=0x7f0058100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:58.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.443+0000 7f0057200640 1 -- 172.21.15.83:0/4184849748 shutdown_connections 2024-04-23T14:56:58.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.443+0000 7f0057200640 1 -- 172.21.15.83:0/4184849748 wait complete. 2024-04-23T14:56:58.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:56:58.464 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:56:58.660 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.660+0000 7f66e1000640 1 -- 172.21.15.83:0/1698594520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66dc10d500 msgr2=0x7f66dc10d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.660+0000 7f66e1000640 1 --2- 172.21.15.83:0/1698594520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66dc10d500 0x7f66dc10d8e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009cd0 tx=0x7f66cc02a8f0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.660+0000 7f66e1000640 1 -- 172.21.15.83:0/1698594520 shutdown_connections 2024-04-23T14:56:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.660+0000 7f66e1000640 1 --2- 172.21.15.83:0/1698594520 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66dc108f40 0x7f66dc1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.660+0000 7f66e1000640 1 --2- 172.21.15.83:0/1698594520 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66dc108580 0x7f66dc108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.660+0000 7f66e1000640 1 --2- 172.21.15.83:0/1698594520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66dc10d500 0x7f66dc10d8e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.660+0000 7f66e1000640 1 -- 172.21.15.83:0/1698594520 >> 172.21.15.83:0/1698594520 conn(0x7f66dc077e20 msgr2=0x7f66dc106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:58.661 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.660+0000 7f66e1000640 1 -- 172.21.15.83:0/1698594520 shutdown_connections 2024-04-23T14:56:58.662 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.660+0000 7f66e1000640 1 -- 172.21.15.83:0/1698594520 wait complete. 2024-04-23T14:56:58.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.661+0000 7f66e1000640 1 Processor -- start 2024-04-23T14:56:58.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.661+0000 7f66e1000640 1 -- start start 2024-04-23T14:56:58.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66e1000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66dc108580 0x7f66dc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66e1000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66dc108f40 0x7f66dc19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66e1000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66dc10d500 0x7f66dc1a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66e1000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc118910 con 0x7f66dc108580 2024-04-23T14:56:58.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66e1000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc118790 con 0x7f66dc10d500 2024-04-23T14:56:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66e1000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc118a90 con 0x7f66dc108f40 2024-04-23T14:56:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66dc108580 0x7f66dc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66dc108580 0x7f66dc19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58342/0 (socket says 172.21.15.83:58342) 2024-04-23T14:56:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66db400640 1 -- 172.21.15.83:0/4262418533 learned_addr learned my addr 172.21.15.83:0/4262418533 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66dbe00640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66dc10d500 0x7f66dc1a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66d2a00640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66dc108f40 0x7f66dc19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66db400640 1 -- 172.21.15.83:0/4262418533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66dc108f40 msgr2=0x7f66dc19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66db400640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66dc108f40 0x7f66dc19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66db400640 1 -- 172.21.15.83:0/4262418533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66dc10d500 msgr2=0x7f66dc1a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66db400640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66dc10d500 0x7f66dc1a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66db400640 1 -- 172.21.15.83:0/4262418533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc1a3d10 con 0x7f66dc108580 2024-04-23T14:56:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.662+0000 7f66dbe00640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66dc10d500 0x7f66dc1a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.663+0000 7f66db400640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66dc108580 0x7f66dc19ed70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f66cc000f80 tx=0x7f66cc003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.663+0000 7f66d8c00640 1 -- 172.21.15.83:0/4262418533 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc030e40 con 0x7f66dc108580 2024-04-23T14:56:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.663+0000 7f66d8c00640 1 -- 172.21.15.83:0/4262418533 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66cc040070 con 0x7f66dc108580 2024-04-23T14:56:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.663+0000 7f66e1000640 1 -- 172.21.15.83:0/4262418533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66dc1a3fa0 con 0x7f66dc108580 2024-04-23T14:56:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.663+0000 7f66d8c00640 1 -- 172.21.15.83:0/4262418533 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc031b10 con 0x7f66dc108580 2024-04-23T14:56:58.666 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.663+0000 7f66e1000640 1 -- 172.21.15.83:0/4262418533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66dc1ab890 con 0x7f66dc108580 2024-04-23T14:56:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.666+0000 7f66e5467740 1 Processor -- start 2024-04-23T14:56:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66d8c00640 1 -- 172.21.15.83:0/4262418533 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66cc031cb0 con 0x7f66dc108580 2024-04-23T14:56:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66e5467740 1 -- start start 2024-04-23T14:56:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66e5467740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53ade10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.674 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66e5467740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66e5467740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53bae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66e5467740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5573b53131f0 con 0x5573b53b8a20 2024-04-23T14:56:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66e5467740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5573b52b9350 con 0x5573b53ae350 2024-04-23T14:56:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66e5467740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5573b530f1b0 con 0x5573b53ada30 2024-04-23T14:56:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66db400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66dbe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66dbe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53ade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59894/0 (socket says 172.21.15.83:59894) 2024-04-23T14:56:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66dbe00640 1 -- 172.21.15.83:0/1275281225 learned_addr learned my addr 172.21.15.83:0/1275281225 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:58.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.667+0000 7f66d2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53bae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.668+0000 7f66db400640 1 -- 172.21.15.83:0/1275281225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 msgr2=0x5573b53ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.668+0000 7f66db400640 1 --2- 172.21.15.83:0/1275281225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.668+0000 7f66db400640 1 -- 172.21.15.83:0/1275281225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 msgr2=0x5573b53bae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.668+0000 7f66db400640 1 --2- 172.21.15.83:0/1275281225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53bae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.668+0000 7f66db400640 1 -- 172.21.15.83:0/1275281225 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b530df50 con 0x5573b53ae350 2024-04-23T14:56:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.668+0000 7f66db400640 1 --2- 172.21.15.83:0/1275281225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b84e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0eefb0 tx=0x7f66cc0f7cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7734dac001393e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.668+0000 7f66d8c00640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f66c00a3d20 0x7f66c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.668+0000 7f66d8c00640 1 -- 172.21.15.83:0/4262418533 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f66cc0f17c0 con 0x7f66dc108580 2024-04-23T14:56:58.676 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.668+0000 7f66d0c00640 1 -- 172.21.15.83:0/1275281225 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0b9aa0 con 0x5573b53ae350 2024-04-23T14:56:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.669+0000 7f66dbe00640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f66c00a3d20 0x7f66c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.669+0000 7f66d0c00640 1 -- 172.21.15.83:0/1275281225 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66cc101020 con 0x5573b53ae350 2024-04-23T14:56:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.669+0000 7f66d0c00640 1 -- 172.21.15.83:0/1275281225 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0bd160 con 0x5573b53ae350 2024-04-23T14:56:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.669+0000 7f66e5467740 1 -- 172.21.15.83:0/1275281225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 msgr2=0x5573b53b84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.669+0000 7f66dbe00640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f66c00a3d20 0x7f66c00a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009fd0 tx=0x7f66d4005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.669+0000 7f66e5467740 1 --2- 172.21.15.83:0/1275281225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b84e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0eefb0 tx=0x7f66cc0f7cb0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.669+0000 7f66e5467740 1 -- 172.21.15.83:0/1275281225 shutdown_connections 2024-04-23T14:56:58.677 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.669+0000 7f66e5467740 1 --2- 172.21.15.83:0/1275281225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53bae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.669+0000 7f66e5467740 1 --2- 172.21.15.83:0/1275281225 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b84e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.669+0000 7f66e5467740 1 --2- 172.21.15.83:0/1275281225 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.669+0000 7f66e5467740 1 -- 172.21.15.83:0/1275281225 >> 172.21.15.83:0/1275281225 conn(0x5573b53a9d50 msgr2=0x5573b53aa130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.670+0000 7f66e5467740 1 -- 172.21.15.83:0/1275281225 shutdown_connections 2024-04-23T14:56:58.678 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.670+0000 7f66e5467740 1 -- 172.21.15.83:0/1275281225 wait complete. 2024-04-23T14:56:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.670+0000 7f66e5467740 1 -- start start 2024-04-23T14:56:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.670+0000 7f66e5467740 1 Processor -- start 2024-04-23T14:56:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66e5467740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53ada30 0x5573b53b5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66e5467740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.679 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66e5467740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53b8a20 0x5573b53b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66e5467740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5573b53bb5d0 con 0x5573b53ada30 2024-04-23T14:56:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66e5467740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5573b52cc690 con 0x5573b53ae350 2024-04-23T14:56:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66e5467740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5573b53bb750 con 0x5573b53b8a20 2024-04-23T14:56:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66e5467740 10 client.0 _ll_register_callbacks cb 0x5573b5308650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53ada30 0x5573b53b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66d2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66d2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:42994/0 (socket says 172.21.15.83:42994) 2024-04-23T14:56:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53ada30 0x5573b53b5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58368/0 (socket says 172.21.15.83:58368) 2024-04-23T14:56:58.680 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66d2a00640 1 -- 172.21.15.83:0/2365072109 learned_addr learned my addr 172.21.15.83:0/2365072109 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66dbe00640 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53b8a20 0x5573b53b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66d2a00640 1 -- 172.21.15.83:0/2365072109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53b8a20 msgr2=0x5573b53b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66d2a00640 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53b8a20 0x5573b53b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66d2a00640 1 -- 172.21.15.83:0/2365072109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53ada30 msgr2=0x5573b53b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66d2a00640 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53ada30 0x5573b53b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.681 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66d2a00640 1 -- 172.21.15.83:0/2365072109 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b52cb920 con 0x5573b53ae350 2024-04-23T14:56:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.671+0000 7f66db400640 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53ada30 0x5573b53b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.672+0000 7f66d2a00640 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b5e60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f66c4012720 tx=0x7f66c4012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fe4516a8da9ac4c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.672+0000 7f66c9600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:56:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.672+0000 7f66c9600640 1 -- 172.21.15.83:0/2365072109 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c400f920 con 0x5573b53ae350 2024-04-23T14:56:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.672+0000 7f66e5467740 1 -- 172.21.15.83:0/2365072109 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66cc101020 con 0x5573b53ae350 2024-04-23T14:56:58.682 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.672+0000 7f66c9600640 1 -- 172.21.15.83:0/2365072109 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f66c400fac0 con 0x5573b53ae350 2024-04-23T14:56:58.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.672+0000 7f66c9600640 1 -- 172.21.15.83:0/2365072109 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f66c400fcc0 con 0x5573b53ae350 2024-04-23T14:56:58.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.672+0000 7f66e5467740 10 client.26951 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:58.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.672+0000 7f66e5467740 1 -- 172.21.15.83:0/2365072109 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573b50ef7d0 con 0x5573b53ae350 2024-04-23T14:56:58.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.672+0000 7f66c9600640 1 -- 172.21.15.83:0/2365072109 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66cc101020 con 0x5573b53ae350 2024-04-23T14:56:58.683 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.673+0000 7f66c9600640 1 -- 172.21.15.83:0/2365072109 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f66c401c420 con 0x5573b53ae350 2024-04-23T14:56:58.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.673+0000 7f66e5467740 10 client.26951 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:58.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.673+0000 7f66e5467740 10 client.26951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:58.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.673+0000 7f66e5467740 20 client.26951 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:58.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.673+0000 7f66e5467740 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5573b53c8a90 0x5573b53caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.673+0000 7f66e5467740 4 client.26951 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:56:58.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.673+0000 7f66e5467740 1 -- 172.21.15.83:0/2365072109 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573b5320c10 con 0x5573b53c8a90 2024-04-23T14:56:58.684 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.673+0000 7f66db400640 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5573b53c8a90 0x5573b53caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.674+0000 7f66c9600640 10 client.26951 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.674+0000 7f66db400640 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5573b53c8a90 0x5573b53caf10 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a43d7b03efbb50d1 server_cookie=3cdb3868453c896e in_seq=0 out_seq=0 2024-04-23T14:56:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.674+0000 7f66c9600640 1 -- 172.21.15.83:0/2365072109 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573b5320c10 con 0x5573b53c8a90 2024-04-23T14:56:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.674+0000 7f66c9600640 10 client.26951 handle_command_reply: tid=0 2024-04-23T14:56:58.685 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.674+0000 7f66e5467740 1 client.26951 shutdown 2024-04-23T14:56:58.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.675+0000 7f66e5467740 1 -- 172.21.15.83:0/2365072109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 msgr2=0x5573b53b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.675+0000 7f66e5467740 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b5e60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f66c4012720 tx=0x7f66c4012be0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.675+0000 7f66e5467740 1 -- 172.21.15.83:0/2365072109 shutdown_connections 2024-04-23T14:56:58.686 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.675+0000 7f66e5467740 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5573b53c8a90 0x5573b53caf10 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.675+0000 7f66e5467740 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53b8a20 0x5573b53b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.675+0000 7f66e5467740 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b5e60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.675+0000 7f66e5467740 1 --2- 172.21.15.83:0/2365072109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53ada30 0x5573b53b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.675+0000 7f66e5467740 1 -- 172.21.15.83:0/2365072109 >> 172.21.15.83:0/2365072109 conn(0x5573b53a9d50 msgr2=0x5573b53afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.675+0000 7f66c9600640 0 client.26951 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.675+0000 7f66e5467740 1 -- 172.21.15.83:0/2365072109 shutdown_connections 2024-04-23T14:56:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.675+0000 7f66e5467740 1 -- 172.21.15.83:0/2365072109 wait complete. 2024-04-23T14:56:58.687 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.675+0000 7f66e5467740 20 client.26951 trim_cache size 0 max 16384 2024-04-23T14:56:58.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.694+0000 7f66e5467740 1 Processor -- start 2024-04-23T14:56:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.694+0000 7f66e5467740 1 -- start start 2024-04-23T14:56:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.694+0000 7f66e5467740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.694+0000 7f66e5467740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.694+0000 7f66e5467740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.694+0000 7f66e5467740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1b6a20 con 0x5573b53b8a20 2024-04-23T14:56:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.694+0000 7f66e5467740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1b42f0 con 0x5573b53ae350 2024-04-23T14:56:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.694+0000 7f66e5467740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1b2160 con 0x5573b53ada30 2024-04-23T14:56:58.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.695+0000 7f66dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.695+0000 7f66db400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.695+0000 7f66db400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59908/0 (socket says 172.21.15.83:59908) 2024-04-23T14:56:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.695+0000 7f66db400640 1 -- 172.21.15.83:0/2962192835 learned_addr learned my addr 172.21.15.83:0/2962192835 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.695+0000 7f66dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:58382/0 (socket says 172.21.15.83:58382) 2024-04-23T14:56:58.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.695+0000 7f66d2a00640 1 --2- 172.21.15.83:0/2962192835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.695+0000 7f66db400640 1 -- 172.21.15.83:0/2962192835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 msgr2=0x5573b53b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.695+0000 7f66db400640 1 --2- 172.21.15.83:0/2962192835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.695+0000 7f66db400640 1 -- 172.21.15.83:0/2962192835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 msgr2=0x5573b53d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.695+0000 7f66db400640 1 --2- 172.21.15.83:0/2962192835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.695+0000 7f66db400640 1 -- 172.21.15.83:0/2962192835 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b50253f0 con 0x5573b53ada30 2024-04-23T14:56:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.695+0000 7f66db400640 1 --2- 172.21.15.83:0/2962192835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53c43e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0b9c60 tx=0x7f66cc003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bfa8b682bc3d898 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.696+0000 7f66cbe00640 1 -- 172.21.15.83:0/2962192835 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0bd340 con 0x5573b53ada30 2024-04-23T14:56:58.700 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.696+0000 7f66cbe00640 1 -- 172.21.15.83:0/2962192835 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66cc036040 con 0x5573b53ada30 2024-04-23T14:56:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.696+0000 7f66cbe00640 1 -- 172.21.15.83:0/2962192835 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0bd5d0 con 0x5573b53ada30 2024-04-23T14:56:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.696+0000 7f66e5467740 1 -- 172.21.15.83:0/2962192835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 msgr2=0x5573b53c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.696+0000 7f66e5467740 1 --2- 172.21.15.83:0/2962192835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53c43e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f66cc0b9c60 tx=0x7f66cc003f90 comp rx=0 tx=0).stop 2024-04-23T14:56:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.696+0000 7f66e5467740 1 -- 172.21.15.83:0/2962192835 shutdown_connections 2024-04-23T14:56:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.696+0000 7f66e5467740 1 --2- 172.21.15.83:0/2962192835 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.696+0000 7f66e5467740 1 --2- 172.21.15.83:0/2962192835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.696+0000 7f66e5467740 1 --2- 172.21.15.83:0/2962192835 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53c43e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.696+0000 7f66e5467740 1 -- 172.21.15.83:0/2962192835 >> 172.21.15.83:0/2962192835 conn(0x5573b53a9d50 msgr2=0x5573b53a8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:58.701 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.697+0000 7f66e5467740 1 -- 172.21.15.83:0/2962192835 shutdown_connections 2024-04-23T14:56:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.697+0000 7f66e5467740 1 -- 172.21.15.83:0/2962192835 wait complete. 2024-04-23T14:56:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.697+0000 7f66e5467740 1 -- start start 2024-04-23T14:56:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.697+0000 7f66e5467740 1 Processor -- start 2024-04-23T14:56:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66e5467740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53c09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66e5467740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53bbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66e5467740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66e5467740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1b27b0 con 0x5573b53b8a20 2024-04-23T14:56:58.702 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66e5467740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1b3c20 con 0x5573b53ae350 2024-04-23T14:56:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66e5467740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1b2e80 con 0x5573b53ada30 2024-04-23T14:56:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66e5467740 10 client.0 _ll_register_callbacks cb 0x7f66c4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:56:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66db400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53c09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66d2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53bbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66d2a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53bbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:43000/0 (socket says 172.21.15.83:43000) 2024-04-23T14:56:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66d2a00640 1 -- 172.21.15.83:0/3996028572 learned_addr learned my addr 172.21.15.83:0/3996028572 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:56:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66db400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53c09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59910/0 (socket says 172.21.15.83:59910) 2024-04-23T14:56:58.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66d2a00640 1 -- 172.21.15.83:0/3996028572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 msgr2=0x5573b53c09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66d2a00640 1 --2- 172.21.15.83:0/3996028572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53c09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66d2a00640 1 -- 172.21.15.83:0/3996028572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 msgr2=0x5573b53bc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:56:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66d2a00640 1 --2- 172.21.15.83:0/3996028572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53bc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66d2a00640 1 -- 172.21.15.83:0/3996028572 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b4fe50f0 con 0x5573b53ae350 2024-04-23T14:56:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.698+0000 7f66db400640 1 --2- 172.21.15.83:0/3996028572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53c09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:56:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.699+0000 7f66d2a00640 1 --2- 172.21.15.83:0/3996028572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53bbe10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f66c4000c00 tx=0x7f66c401c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dee0343b278d00ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:56:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.699+0000 7f66c9600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:56:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.699+0000 7f66c9600640 1 -- 172.21.15.83:0/3996028572 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c400fe60 con 0x5573b53ae350 2024-04-23T14:56:58.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.699+0000 7f66c9600640 1 -- 172.21.15.83:0/3996028572 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8542+0+0 (secure 0 0 0) 0x7f66c401cd10 con 0x5573b53ae350 2024-04-23T14:56:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.699+0000 7f66e5467740 1 -- 172.21.15.83:0/3996028572 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66cc036040 con 0x5573b53ae350 2024-04-23T14:56:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.699+0000 7f66c9600640 1 -- 172.21.15.83:0/3996028572 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f66c4033050 con 0x5573b53ae350 2024-04-23T14:56:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.699+0000 7f66e5467740 10 client.26957 fetch_fsmap learned FSMap version 29 2024-04-23T14:56:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.699+0000 7f66e5467740 1 -- 172.21.15.83:0/3996028572 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573b52d8810 con 0x5573b53ae350 2024-04-23T14:56:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.699+0000 7f66c9600640 1 -- 172.21.15.83:0/3996028572 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66cc036040 con 0x5573b53ae350 2024-04-23T14:56:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.700+0000 7f66c9600640 1 -- 172.21.15.83:0/3996028572 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f66c402b020 con 0x5573b53ae350 2024-04-23T14:56:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.700+0000 7f66e5467740 10 client.26957 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:56:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.700+0000 7f66e5467740 10 client.26957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:56:58.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.700+0000 7f66e5467740 20 client.26957 populate_metadata read hostname 'smithi083' 2024-04-23T14:56:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.700+0000 7f66e5467740 1 --2- 172.21.15.83:0/3996028572 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5573b542f1a0 0x5573b54315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:56:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.700+0000 7f66e5467740 4 client.26957 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:56:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.700+0000 7f66e5467740 1 -- 172.21.15.83:0/3996028572 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573b531aa80 con 0x5573b542f1a0 2024-04-23T14:56:58.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.700+0000 7f66db400640 1 --2- 172.21.15.83:0/3996028572 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5573b542f1a0 0x5573b54315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:56:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.700+0000 7f66db400640 1 --2- 172.21.15.83:0/3996028572 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5573b542f1a0 0x5573b54315c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76e356ffb33de832 server_cookie=cf43a92f89259b52 in_seq=0 out_seq=0 2024-04-23T14:56:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.700+0000 7f66c9600640 10 client.26957 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:58.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66c9600640 1 -- 172.21.15.83:0/3996028572 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5573b531aa80 con 0x5573b542f1a0 2024-04-23T14:56:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66c9600640 10 client.26957 handle_command_reply: tid=0 2024-04-23T14:56:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66e5467740 1 client.26957 shutdown 2024-04-23T14:56:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66e5467740 1 -- 172.21.15.83:0/3996028572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 msgr2=0x5573b53bbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66e5467740 1 --2- 172.21.15.83:0/3996028572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53bbe10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f66c4000c00 tx=0x7f66c401c650 comp rx=0 tx=0).stop 2024-04-23T14:56:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66e5467740 1 -- 172.21.15.83:0/3996028572 shutdown_connections 2024-04-23T14:56:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66e5467740 1 --2- 172.21.15.83:0/3996028572 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5573b542f1a0 0x5573b54315c0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66e5467740 1 --2- 172.21.15.83:0/3996028572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573b53b8a20 0x5573b53bc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66e5467740 1 --2- 172.21.15.83:0/3996028572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5573b53ae350 0x5573b53bbe10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66e5467740 1 --2- 172.21.15.83:0/3996028572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5573b53ada30 0x5573b53c09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66e5467740 1 -- 172.21.15.83:0/3996028572 >> 172.21.15.83:0/3996028572 conn(0x5573b53a9d50 msgr2=0x5573b53b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66c9600640 0 client.26957 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:56:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66e5467740 1 -- 172.21.15.83:0/3996028572 shutdown_connections 2024-04-23T14:56:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66e5467740 1 -- 172.21.15.83:0/3996028572 wait complete. 2024-04-23T14:56:58.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.701+0000 7f66e5467740 20 client.26957 trim_cache size 0 max 16384 2024-04-23T14:56:58.709 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:56:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:56:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:56:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "ab7482d9-95a8-49e0-b540-ad952c3d73f6": { 2024-04-23T14:56:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:56:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "ab7482d9-95a8-49e0-b540-ad952c3d73f6", 2024-04-23T14:56:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:56:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:56:58.710 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:56:58.711 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:56:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.702+0000 7f66e1000640 1 -- 172.21.15.83:0/4262418533 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f66c00a3d20 msgr2=0x7f66c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.702+0000 7f66e1000640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f66c00a3d20 0x7f66c00a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009fd0 tx=0x7f66d4005f90 comp rx=0 tx=0).stop 2024-04-23T14:56:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.702+0000 7f66e1000640 1 -- 172.21.15.83:0/4262418533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66dc108580 msgr2=0x7f66dc19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:56:58.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.702+0000 7f66e1000640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66dc108580 0x7f66dc19ed70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f66cc000f80 tx=0x7f66cc003ea0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.702+0000 7f66e1000640 1 -- 172.21.15.83:0/4262418533 shutdown_connections 2024-04-23T14:56:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.702+0000 7f66e1000640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66dc10d500 0x7f66dc1a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.702+0000 7f66e1000640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66dc108f40 0x7f66dc19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.702+0000 7f66e1000640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f66c00a3d20 0x7f66c00a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.702+0000 7f66e1000640 1 --2- 172.21.15.83:0/4262418533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66dc108580 0x7f66dc19ed70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:56:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.702+0000 7f66e1000640 1 -- 172.21.15.83:0/4262418533 >> 172.21.15.83:0/4262418533 conn(0x7f66dc077e20 msgr2=0x7f66dc078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:56:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.702+0000 7f66e1000640 1 -- 172.21.15.83:0/4262418533 shutdown_connections 2024-04-23T14:56:58.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:56:58.702+0000 7f66e1000640 1 -- 172.21.15.83:0/4262418533 wait complete. 2024-04-23T14:56:58.719 INFO:tasks.cephfs.filesystem:scrub status for tag:ab7482d9-95a8-49e0-b540-ad952c3d73f6 - {'path': '/', 'tag': 'ab7482d9-95a8-49e0-b540-ad952c3d73f6', 'options': 'recursive,force'} 2024-04-23T14:56:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: pgmap v263: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:56:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:56:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:59.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:59.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:59.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:59.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:59.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:56:59.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.723 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:56:59 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: pgmap v263: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:56:59.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:56:59.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:59.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:59.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:59.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:59.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:59.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:56:59.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.759 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:56:59 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: pgmap v263: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:56:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:56:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:59.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:59.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:56:59.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:56:59.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:56:59 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:01.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:01 smithi203 ceph-mon[32800]: pgmap v264: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:57:01.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:01 smithi083 ceph-mon[27867]: pgmap v264: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:57:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:01 smithi173 ceph-mon[32443]: pgmap v264: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:57:03.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:03 smithi203 ceph-mon[32800]: pgmap v265: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-23T14:57:03.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:03 smithi083 ceph-mon[27867]: pgmap v265: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-23T14:57:03.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:03 smithi173 ceph-mon[32443]: pgmap v265: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-23T14:57:05.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:05 smithi203 ceph-mon[32800]: pgmap v266: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:57:05.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:05 smithi083 ceph-mon[27867]: pgmap v266: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:57:05.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:05 smithi173 ceph-mon[32443]: pgmap v266: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:57:07.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:07 smithi203 ceph-mon[32800]: pgmap v267: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-23T14:57:07.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:07 smithi083 ceph-mon[27867]: pgmap v267: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-23T14:57:07.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:07 smithi173 ceph-mon[32443]: pgmap v267: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-23T14:57:09.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:09 smithi203 ceph-mon[32800]: pgmap v268: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-23T14:57:09.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:09 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:57:09.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:09 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:57:09.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:09 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:57:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:09 smithi083 ceph-mon[27867]: pgmap v268: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-23T14:57:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:09 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:57:09.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:09 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:57:09.758 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:09 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:57:09.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:09 smithi173 ceph-mon[32443]: pgmap v268: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2024-04-23T14:57:09.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:09 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:57:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:09 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:57:09.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:09 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:57:10.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:10 smithi203 ceph-mon[32800]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T14:57:10.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:10 smithi083 ceph-mon[27867]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T14:57:10.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:10 smithi173 ceph-mon[32443]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T14:57:11.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:11 smithi203 ceph-mon[32800]: pgmap v270: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T14:57:11.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:11 smithi083 ceph-mon[27867]: pgmap v270: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T14:57:11.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:11 smithi173 ceph-mon[32443]: pgmap v270: 97 pgs: 97 active+clean; 20 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T14:57:13.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:13 smithi083 ceph-mon[27867]: pgmap v271: 97 pgs: 10 active+clean+snaptrim, 65 active+clean, 22 active+clean+snaptrim_wait; 18 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T14:57:13.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:13 smithi173 ceph-mon[32443]: pgmap v271: 97 pgs: 10 active+clean+snaptrim, 65 active+clean, 22 active+clean+snaptrim_wait; 18 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T14:57:13.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:13 smithi203 ceph-mon[32800]: pgmap v271: 97 pgs: 10 active+clean+snaptrim, 65 active+clean, 22 active+clean+snaptrim_wait; 18 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-23T14:57:15.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:15 smithi083 ceph-mon[27867]: pgmap v272: 97 pgs: 10 active+clean+snaptrim, 65 active+clean, 22 active+clean+snaptrim_wait; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-23T14:57:15.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:15 smithi173 ceph-mon[32443]: pgmap v272: 97 pgs: 10 active+clean+snaptrim, 65 active+clean, 22 active+clean+snaptrim_wait; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-23T14:57:15.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:15 smithi203 ceph-mon[32800]: pgmap v272: 97 pgs: 10 active+clean+snaptrim, 65 active+clean, 22 active+clean+snaptrim_wait; 17 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-23T14:57:17.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:17 smithi083 ceph-mon[27867]: pgmap v273: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:57:17.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:17 smithi083 ceph-mon[27867]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T14:57:17.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:17 smithi173 ceph-mon[32443]: pgmap v273: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:57:17.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:17 smithi173 ceph-mon[32443]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T14:57:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:17 smithi203 ceph-mon[32800]: pgmap v273: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:57:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:17 smithi203 ceph-mon[32800]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T14:57:19.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:19 smithi083 ceph-mon[27867]: pgmap v275: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 243 B/s wr, 7 op/s 2024-04-23T14:57:19.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:19 smithi173 ceph-mon[32443]: pgmap v275: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 243 B/s wr, 7 op/s 2024-04-23T14:57:19.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:19 smithi203 ceph-mon[32800]: pgmap v275: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 243 B/s wr, 7 op/s 2024-04-23T14:57:21.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:21 smithi083 ceph-mon[27867]: pgmap v276: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:57:21.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:21 smithi173 ceph-mon[32443]: pgmap v276: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:57:21.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:21 smithi203 ceph-mon[32800]: pgmap v276: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:57:23.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:23 smithi083 ceph-mon[27867]: pgmap v277: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:57:23.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:23 smithi173 ceph-mon[32443]: pgmap v277: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:57:23.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:23 smithi203 ceph-mon[32800]: pgmap v277: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:57:25.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:25 smithi083 ceph-mon[27867]: pgmap v278: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:57:25.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:25 smithi173 ceph-mon[32443]: pgmap v278: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:57:25.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:25 smithi203 ceph-mon[32800]: pgmap v278: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:57:27.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:27 smithi173 ceph-mon[32443]: pgmap v279: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:57:27.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:27 smithi203 ceph-mon[32800]: pgmap v279: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:57:28.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:27 smithi083 ceph-mon[27867]: pgmap v279: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:57:28.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:57:28.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:57:28.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.956+0000 7fb1e2400640 1 -- 172.21.15.83:0/4135076583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1dc102240 msgr2=0x7fb1dc102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:28.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.956+0000 7fb1e2400640 1 --2- 172.21.15.83:0/4135076583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1dc102240 0x7fb1dc102620 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009900 tx=0x7fb1cc02a410 comp rx=0 tx=0).stop 2024-04-23T14:57:28.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.957+0000 7fb1e2400640 1 -- 172.21.15.83:0/4135076583 shutdown_connections 2024-04-23T14:57:28.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.957+0000 7fb1e2400640 1 --2- 172.21.15.83:0/4135076583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1dc107310 0x7fb1dc10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.957+0000 7fb1e2400640 1 --2- 172.21.15.83:0/4135076583 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1dc102bf0 0x7fb1dc106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.957+0000 7fb1e2400640 1 --2- 172.21.15.83:0/4135076583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1dc102240 0x7fb1dc102620 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.957+0000 7fb1e2400640 1 -- 172.21.15.83:0/4135076583 >> 172.21.15.83:0/4135076583 conn(0x7fb1dc079450 msgr2=0x7fb1dc079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:28.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.957+0000 7fb1e2400640 1 -- 172.21.15.83:0/4135076583 shutdown_connections 2024-04-23T14:57:28.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.957+0000 7fb1e2400640 1 -- 172.21.15.83:0/4135076583 wait complete. 2024-04-23T14:57:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.958+0000 7fb1e2400640 1 Processor -- start 2024-04-23T14:57:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.958+0000 7fb1e2400640 1 -- start start 2024-04-23T14:57:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.958+0000 7fb1e2400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1dc102240 0x7fb1dc1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.958+0000 7fb1e2400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1dc107310 0x7fb1dc1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.958+0000 7fb1e2400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1dc19a950 0x7fb1dc19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.958+0000 7fb1e2400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc1123f0 con 0x7fb1dc19a950 2024-04-23T14:57:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.958+0000 7fb1e2400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc112270 con 0x7fb1dc107310 2024-04-23T14:57:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.958+0000 7fb1e2400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc112570 con 0x7fb1dc102240 2024-04-23T14:57:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1dc19a950 0x7fb1dc19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1db400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1dc102240 0x7fb1dc1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1db400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1dc102240 0x7fb1dc1a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59096/0 (socket says 172.21.15.83:59096) 2024-04-23T14:57:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1db400640 1 -- 172.21.15.83:0/1619223029 learned_addr learned my addr 172.21.15.83:0/1619223029 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1dc107310 0x7fb1dc1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.963 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1dbe00640 1 -- 172.21.15.83:0/1619223029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1dc102240 msgr2=0x7fb1dc1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1dbe00640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1dc102240 0x7fb1dc1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1dbe00640 1 -- 172.21.15.83:0/1619223029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1dc107310 msgr2=0x7fb1dc1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1dbe00640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1dc107310 0x7fb1dc1a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1dbe00640 1 -- 172.21.15.83:0/1619223029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1dc19b470 con 0x7fb1dc19a950 2024-04-23T14:57:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1db400640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1dc102240 0x7fb1dc1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:57:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1dc107310 0x7fb1dc1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:57:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.959+0000 7fb1dbe00640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1dc19a950 0x7fb1dc19ae00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009fd0 tx=0x7fb1d400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:28.964 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.960+0000 7fb1d8c00640 1 -- 172.21.15.83:0/1619223029 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400ce20 con 0x7fb1dc19a950 2024-04-23T14:57:28.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.960+0000 7fb1d8c00640 1 -- 172.21.15.83:0/1619223029 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d4005020 con 0x7fb1dc19a950 2024-04-23T14:57:28.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.960+0000 7fb1d8c00640 1 -- 172.21.15.83:0/1619223029 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4010720 con 0x7fb1dc19a950 2024-04-23T14:57:28.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.960+0000 7fb1e2400640 1 -- 172.21.15.83:0/1619223029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1dc19b760 con 0x7fb1dc19a950 2024-04-23T14:57:28.965 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.960+0000 7fb1e2400640 1 -- 172.21.15.83:0/1619223029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1dc1a7910 con 0x7fb1dc19a950 2024-04-23T14:57:28.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.963+0000 7fb1e525a740 1 Processor -- start 2024-04-23T14:57:28.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.963+0000 7fb1d8c00640 1 -- 172.21.15.83:0/1619223029 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb1d4005540 con 0x7fb1dc19a950 2024-04-23T14:57:28.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.964+0000 7fb1e525a740 1 -- start start 2024-04-23T14:57:28.968 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.964+0000 7fb1e525a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386040860 0x55e386040c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.964+0000 7fb1e525a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386041180 0x55e38604b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.964+0000 7fb1e525a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e38604b850 0x55e38604dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.964+0000 7fb1e525a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e385fa61f0 con 0x55e386041180 2024-04-23T14:57:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.964+0000 7fb1e525a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e385f4c350 con 0x55e38604b850 2024-04-23T14:57:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.964+0000 7fb1e525a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e385fa21b0 con 0x55e386040860 2024-04-23T14:57:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.964+0000 7fb1db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386041180 0x55e38604b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.964+0000 7fb1d2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386040860 0x55e386040c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.964+0000 7fb1d2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386040860 0x55e386040c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59106/0 (socket says 172.21.15.83:59106) 2024-04-23T14:57:28.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.964+0000 7fb1d2a00640 1 -- 172.21.15.83:0/125628226 learned_addr learned my addr 172.21.15.83:0/125628226 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.965+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/125628226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e38604b850 0x55e38604dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.965+0000 7fb1db400640 1 -- 172.21.15.83:0/125628226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386040860 msgr2=0x55e386040c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.965+0000 7fb1db400640 1 --2- 172.21.15.83:0/125628226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386040860 0x55e386040c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.965+0000 7fb1db400640 1 -- 172.21.15.83:0/125628226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e38604b850 msgr2=0x55e38604dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.965+0000 7fb1db400640 1 --2- 172.21.15.83:0/125628226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e38604b850 0x55e38604dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.965+0000 7fb1db400640 1 -- 172.21.15.83:0/125628226 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e385fa0f50 con 0x55e386041180 2024-04-23T14:57:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.965+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/125628226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386040860 0x55e386040c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:57:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.965+0000 7fb1d8c00640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb1c00a3da0 0x7fb1c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.965+0000 7fb1d8c00640 1 -- 172.21.15.83:0/1619223029 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fb1d40d0e40 con 0x7fb1dc19a950 2024-04-23T14:57:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.965+0000 7fb1db400640 1 --2- 172.21.15.83:0/125628226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386041180 0x55e38604b310 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0ffb10 tx=0x7fb1cc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d22794ef00cda26 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.965+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb1c00a3da0 0x7fb1c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.966+0000 7fb1d0c00640 1 -- 172.21.15.83:0/125628226 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc040070 con 0x55e386041180 2024-04-23T14:57:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.966+0000 7fb1d0c00640 1 -- 172.21.15.83:0/125628226 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1cc0047a0 con 0x55e386041180 2024-04-23T14:57:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.966+0000 7fb1d0c00640 1 -- 172.21.15.83:0/125628226 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc035040 con 0x55e386041180 2024-04-23T14:57:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.966+0000 7fb1e525a740 1 -- 172.21.15.83:0/125628226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386041180 msgr2=0x55e38604b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.966+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb1c00a3da0 0x7fb1c00a6260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55e38603dd10 tx=0x7fb1c4005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:28.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.966+0000 7fb1e525a740 1 --2- 172.21.15.83:0/125628226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386041180 0x55e38604b310 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0ffb10 tx=0x7fb1cc005dd0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.967+0000 7fb1e525a740 1 -- 172.21.15.83:0/125628226 shutdown_connections 2024-04-23T14:57:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.967+0000 7fb1e525a740 1 --2- 172.21.15.83:0/125628226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e38604b850 0x55e38604dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.967+0000 7fb1e525a740 1 --2- 172.21.15.83:0/125628226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386041180 0x55e38604b310 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.967+0000 7fb1e525a740 1 --2- 172.21.15.83:0/125628226 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386040860 0x55e386040c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.967+0000 7fb1e525a740 1 -- 172.21.15.83:0/125628226 >> 172.21.15.83:0/125628226 conn(0x55e38603cb80 msgr2=0x55e38603cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.967+0000 7fb1e525a740 1 -- 172.21.15.83:0/125628226 shutdown_connections 2024-04-23T14:57:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.967+0000 7fb1e525a740 1 -- 172.21.15.83:0/125628226 wait complete. 2024-04-23T14:57:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.967+0000 7fb1e525a740 1 -- start start 2024-04-23T14:57:28.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.967+0000 7fb1e525a740 1 Processor -- start 2024-04-23T14:57:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1e525a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 0x55e386048800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1e525a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e386041180 0x55e386048d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1e525a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e38604b850 0x55e386049280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1e525a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e38604e5e0 con 0x55e386040860 2024-04-23T14:57:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1e525a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e385f5f690 con 0x55e386041180 2024-04-23T14:57:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1e525a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e38604e760 con 0x55e38604b850 2024-04-23T14:57:28.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1e525a740 10 client.0 _ll_register_callbacks cb 0x55e385f9b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:57:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 0x55e386048800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 0x55e386048800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37432/0 (socket says 172.21.15.83:37432) 2024-04-23T14:57:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1db400640 1 -- 172.21.15.83:0/3066424389 learned_addr learned my addr 172.21.15.83:0/3066424389 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1dbe00640 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e38604b850 0x55e386049280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e386041180 0x55e386048d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1db400640 1 -- 172.21.15.83:0/3066424389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e38604b850 msgr2=0x55e386049280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1db400640 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e38604b850 0x55e386049280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1db400640 1 -- 172.21.15.83:0/3066424389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e386041180 msgr2=0x55e386048d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:28.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.968+0000 7fb1db400640 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e386041180 0x55e386048d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.969+0000 7fb1db400640 1 -- 172.21.15.83:0/3066424389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e385f5e920 con 0x55e386040860 2024-04-23T14:57:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.969+0000 7fb1dbe00640 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e38604b850 0x55e386049280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:57:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.969+0000 7fb1db400640 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 0x55e386048800 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc004770 tx=0x7fb1cc031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c147a8f98a37c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.969+0000 7fb1c9600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:57:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.969+0000 7fb1c9600640 1 -- 172.21.15.83:0/3066424389 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc04c020 con 0x55e386040860 2024-04-23T14:57:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.969+0000 7fb1e525a740 1 -- 172.21.15.83:0/3066424389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1cc0047a0 con 0x55e386040860 2024-04-23T14:57:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.969+0000 7fb1c9600640 1 -- 172.21.15.83:0/3066424389 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fb1cc0035f0 con 0x55e386040860 2024-04-23T14:57:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.969+0000 7fb1c9600640 1 -- 172.21.15.83:0/3066424389 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb1cc057050 con 0x55e386040860 2024-04-23T14:57:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.969+0000 7fb1e525a740 10 client.16770 fetch_fsmap learned FSMap version 29 2024-04-23T14:57:28.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.969+0000 7fb1e525a740 1 -- 172.21.15.83:0/3066424389 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e385d827d0 con 0x55e386040860 2024-04-23T14:57:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.970+0000 7fb1c9600640 1 -- 172.21.15.83:0/3066424389 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1cc0047a0 con 0x55e386040860 2024-04-23T14:57:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.970+0000 7fb1c9600640 1 -- 172.21.15.83:0/3066424389 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fb1cc03c810 con 0x55e386040860 2024-04-23T14:57:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.970+0000 7fb1e525a740 10 client.16770 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:57:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.970+0000 7fb1e525a740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:57:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.970+0000 7fb1e525a740 20 client.16770 populate_metadata read hostname 'smithi083' 2024-04-23T14:57:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.970+0000 7fb1e525a740 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e38605ba20 0x55e38605de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.970+0000 7fb1e525a740 4 client.16770 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:57:28.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.970+0000 7fb1e525a740 1 -- 172.21.15.83:0/3066424389 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e385fb3c10 con 0x55e38605ba20 2024-04-23T14:57:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.971+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e38605ba20 0x55e38605de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.971+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e38605ba20 0x55e38605de10 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a1989d6dceb9c0b server_cookie=272d5d0609b7a138 in_seq=0 out_seq=0 2024-04-23T14:57:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.971+0000 7fb1c9600640 10 client.16770 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1c9600640 1 -- 172.21.15.83:0/3066424389 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e385fb3c10 con 0x55e38605ba20 2024-04-23T14:57:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1c9600640 10 client.16770 handle_command_reply: tid=0 2024-04-23T14:57:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1e525a740 1 client.16770 shutdown 2024-04-23T14:57:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1e525a740 1 -- 172.21.15.83:0/3066424389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 msgr2=0x55e386048800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1e525a740 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 0x55e386048800 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc004770 tx=0x7fb1cc031860 comp rx=0 tx=0).stop 2024-04-23T14:57:28.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1e525a740 1 -- 172.21.15.83:0/3066424389 shutdown_connections 2024-04-23T14:57:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1e525a740 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e38605ba20 0x55e38605de10 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1e525a740 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e38604b850 0x55e386049280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1e525a740 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e386041180 0x55e386048d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1c9600640 0 client.16770 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1e525a740 1 --2- 172.21.15.83:0/3066424389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 0x55e386048800 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1e525a740 1 -- 172.21.15.83:0/3066424389 >> 172.21.15.83:0/3066424389 conn(0x55e38603cb80 msgr2=0x55e386042980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1e525a740 1 -- 172.21.15.83:0/3066424389 shutdown_connections 2024-04-23T14:57:28.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.972+0000 7fb1e525a740 1 -- 172.21.15.83:0/3066424389 wait complete. 2024-04-23T14:57:28.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.973+0000 7fb1e525a740 20 client.16770 trim_cache size 0 max 16384 2024-04-23T14:57:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.991+0000 7fb1e525a740 1 Processor -- start 2024-04-23T14:57:28.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.991+0000 7fb1e525a740 1 -- start start 2024-04-23T14:57:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1e525a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 0x55e3860574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1e525a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386041180 0x55e38603baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1e525a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e38604b850 0x55e3860633e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:28.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1e525a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc1b2780 con 0x55e386040860 2024-04-23T14:57:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1e525a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc1b0050 con 0x55e38604b850 2024-04-23T14:57:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1e525a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc1adec0 con 0x55e386041180 2024-04-23T14:57:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 0x55e3860574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1db400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 0x55e3860574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37440/0 (socket says 172.21.15.83:37440) 2024-04-23T14:57:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1db400640 1 -- 172.21.15.83:0/4135714767 learned_addr learned my addr 172.21.15.83:0/4135714767 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1dbe00640 1 --2- 172.21.15.83:0/4135714767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e38604b850 0x55e3860633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/4135714767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386041180 0x55e38603baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1db400640 1 -- 172.21.15.83:0/4135714767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386041180 msgr2=0x55e38603baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:28.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1db400640 1 --2- 172.21.15.83:0/4135714767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386041180 0x55e38603baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1db400640 1 -- 172.21.15.83:0/4135714767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e38604b850 msgr2=0x55e3860633e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.992+0000 7fb1db400640 1 --2- 172.21.15.83:0/4135714767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e38604b850 0x55e3860633e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.993+0000 7fb1db400640 1 -- 172.21.15.83:0/4135714767 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e386043620 con 0x55e386040860 2024-04-23T14:57:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.993+0000 7fb1db400640 1 --2- 172.21.15.83:0/4135714767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 0x55e3860574a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc03cc00 tx=0x7fb1cc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e7c9a75d3156fd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.993+0000 7fb1cbe00640 1 -- 172.21.15.83:0/4135714767 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc04c040 con 0x55e386040860 2024-04-23T14:57:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.993+0000 7fb1cbe00640 1 -- 172.21.15.83:0/4135714767 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1cc0035f0 con 0x55e386040860 2024-04-23T14:57:28.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.993+0000 7fb1cbe00640 1 -- 172.21.15.83:0/4135714767 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc040070 con 0x55e386040860 2024-04-23T14:57:28.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.994+0000 7fb1e525a740 1 -- 172.21.15.83:0/4135714767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 msgr2=0x55e3860574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.994+0000 7fb1e525a740 1 --2- 172.21.15.83:0/4135714767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 0x55e3860574a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc03cc00 tx=0x7fb1cc005dd0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.994+0000 7fb1e525a740 1 -- 172.21.15.83:0/4135714767 shutdown_connections 2024-04-23T14:57:29.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.994+0000 7fb1e525a740 1 --2- 172.21.15.83:0/4135714767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e38604b850 0x55e3860633e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.994+0000 7fb1e525a740 1 --2- 172.21.15.83:0/4135714767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386041180 0x55e38603baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.000 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.994+0000 7fb1e525a740 1 --2- 172.21.15.83:0/4135714767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e386040860 0x55e3860574a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.994+0000 7fb1e525a740 1 -- 172.21.15.83:0/4135714767 >> 172.21.15.83:0/4135714767 conn(0x55e38603cb80 msgr2=0x55e38603b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.994+0000 7fb1e525a740 1 -- 172.21.15.83:0/4135714767 shutdown_connections 2024-04-23T14:57:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.994+0000 7fb1e525a740 1 -- 172.21.15.83:0/4135714767 wait complete. 2024-04-23T14:57:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.994+0000 7fb1e525a740 1 -- start start 2024-04-23T14:57:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.994+0000 7fb1e525a740 1 Processor -- start 2024-04-23T14:57:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1e525a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e386040860 0x55e386053e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.001 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1e525a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386041180 0x55e3860451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1e525a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e38604b850 0x55e386045730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1e525a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc1ae510 con 0x55e38604b850 2024-04-23T14:57:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1e525a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc1af980 con 0x55e386040860 2024-04-23T14:57:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1e525a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc1aebe0 con 0x55e386041180 2024-04-23T14:57:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1e525a740 10 client.0 _ll_register_callbacks cb 0x7fb1c4011650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:57:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e38604b850 0x55e386045730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1db400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e386040860 0x55e386053e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1d2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386041180 0x55e3860451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1db400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e386040860 0x55e386053e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:39472/0 (socket says 172.21.15.83:39472) 2024-04-23T14:57:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1dbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e38604b850 0x55e386045730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37442/0 (socket says 172.21.15.83:37442) 2024-04-23T14:57:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.995+0000 7fb1db400640 1 -- 172.21.15.83:0/3681073081 learned_addr learned my addr 172.21.15.83:0/3681073081 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1db400640 1 -- 172.21.15.83:0/3681073081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386041180 msgr2=0x55e3860451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1db400640 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386041180 0x55e3860451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1db400640 1 -- 172.21.15.83:0/3681073081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e38604b850 msgr2=0x55e386045730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1db400640 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e38604b850 0x55e386045730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1db400640 1 -- 172.21.15.83:0/3681073081 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e385cb83f0 con 0x55e386040860 2024-04-23T14:57:29.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386041180 0x55e3860451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:57:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1dbe00640 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e38604b850 0x55e386045730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:57:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1db400640 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e386040860 0x55e386053e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0059d0 tx=0x7fb1cc0045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d1a603dc8e09ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1c9600640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:57:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1c9600640 1 -- 172.21.15.83:0/3681073081 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc04c030 con 0x55e386040860 2024-04-23T14:57:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1e525a740 1 -- 172.21.15.83:0/3681073081 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1cc0035f0 con 0x55e386040860 2024-04-23T14:57:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1c9600640 1 -- 172.21.15.83:0/3681073081 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fb1cc03c850 con 0x55e386040860 2024-04-23T14:57:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1c9600640 1 -- 172.21.15.83:0/3681073081 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb1cc040040 con 0x55e386040860 2024-04-23T14:57:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1e525a740 10 client.26975 fetch_fsmap learned FSMap version 29 2024-04-23T14:57:29.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.996+0000 7fb1e525a740 1 -- 172.21.15.83:0/3681073081 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e385c780f0 con 0x55e386040860 2024-04-23T14:57:29.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.997+0000 7fb1c9600640 1 -- 172.21.15.83:0/3681073081 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1cc0035f0 con 0x55e386040860 2024-04-23T14:57:29.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.997+0000 7fb1c9600640 1 -- 172.21.15.83:0/3681073081 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fb1cc06c960 con 0x55e386040860 2024-04-23T14:57:29.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.997+0000 7fb1e525a740 10 client.26975 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:57:29.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.997+0000 7fb1e525a740 10 client.26975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:57:29.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.997+0000 7fb1e525a740 20 client.26975 populate_metadata read hostname 'smithi083' 2024-04-23T14:57:29.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.997+0000 7fb1e525a740 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e3860c1f40 0x55e3860c4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.997+0000 7fb1e525a740 4 client.26975 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:57:29.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.997+0000 7fb1e525a740 1 -- 172.21.15.83:0/3681073081 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e385fada80 con 0x55e3860c1f40 2024-04-23T14:57:29.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.997+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e3860c1f40 0x55e3860c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.998+0000 7fb1d2a00640 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e3860c1f40 0x55e3860c4330 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd856be84ecc9e6c server_cookie=ad8fcc71c131ec87 in_seq=0 out_seq=0 2024-04-23T14:57:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.998+0000 7fb1c9600640 10 client.26975 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.998+0000 7fb1c9600640 1 -- 172.21.15.83:0/3681073081 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e385fada80 con 0x55e3860c1f40 2024-04-23T14:57:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.998+0000 7fb1c9600640 10 client.26975 handle_command_reply: tid=0 2024-04-23T14:57:29.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.998+0000 7fb1e525a740 1 client.26975 shutdown 2024-04-23T14:57:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.998+0000 7fb1e525a740 1 -- 172.21.15.83:0/3681073081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e386040860 msgr2=0x55e386053e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.998+0000 7fb1e525a740 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e386040860 0x55e386053e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0059d0 tx=0x7fb1cc0045b0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e525a740 1 -- 172.21.15.83:0/3681073081 shutdown_connections 2024-04-23T14:57:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e525a740 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e3860c1f40 0x55e3860c4330 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e525a740 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e38604b850 0x55e386045730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e525a740 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e386041180 0x55e3860451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1c9600640 0 client.26975 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e525a740 1 --2- 172.21.15.83:0/3681073081 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e386040860 0x55e386053e10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e525a740 1 -- 172.21.15.83:0/3681073081 >> 172.21.15.83:0/3681073081 conn(0x55e38603cb80 msgr2=0x55e38603bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e525a740 1 -- 172.21.15.83:0/3681073081 shutdown_connections 2024-04-23T14:57:29.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e525a740 1 -- 172.21.15.83:0/3681073081 wait complete. 2024-04-23T14:57:29.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e525a740 20 client.26975 trim_cache size 0 max 16384 2024-04-23T14:57:29.009 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:57:29.009 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:57:29.009 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:57:29.010 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:57:29.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e2400640 1 -- 172.21.15.83:0/1619223029 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb1c00a3da0 msgr2=0x7fb1c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.010 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e2400640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb1c00a3da0 0x7fb1c00a6260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55e38603dd10 tx=0x7fb1c4005f90 comp rx=0 tx=0).stop 2024-04-23T14:57:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e2400640 1 -- 172.21.15.83:0/1619223029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1dc19a950 msgr2=0x7fb1dc19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:28.999+0000 7fb1e2400640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1dc19a950 0x7fb1dc19ae00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009fd0 tx=0x7fb1d400ef10 comp rx=0 tx=0).stop 2024-04-23T14:57:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.000+0000 7fb1e2400640 1 -- 172.21.15.83:0/1619223029 shutdown_connections 2024-04-23T14:57:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.000+0000 7fb1e2400640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb1c00a3da0 0x7fb1c00a6260 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.000+0000 7fb1e2400640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1dc19a950 0x7fb1dc19ae00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.000+0000 7fb1e2400640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1dc107310 0x7fb1dc1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.000+0000 7fb1e2400640 1 --2- 172.21.15.83:0/1619223029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1dc102240 0x7fb1dc1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.000+0000 7fb1e2400640 1 -- 172.21.15.83:0/1619223029 >> 172.21.15.83:0/1619223029 conn(0x7fb1dc079450 msgr2=0x7fb1dc100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.011 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.000+0000 7fb1e2400640 1 -- 172.21.15.83:0/1619223029 shutdown_connections 2024-04-23T14:57:29.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.000+0000 7fb1e2400640 1 -- 172.21.15.83:0/1619223029 wait complete. 2024-04-23T14:57:29.018 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:57:29.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:57:29.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.218+0000 7f9e4c000640 1 -- 172.21.15.83:0/689598389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e44104e00 msgr2=0x7f9e44108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.218+0000 7f9e4c000640 1 --2- 172.21.15.83:0/689598389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e44104e00 0x7f9e44108e40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009860 tx=0x7f9e3002a340 comp rx=0 tx=0).stop 2024-04-23T14:57:29.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.219+0000 7f9e4c000640 1 -- 172.21.15.83:0/689598389 shutdown_connections 2024-04-23T14:57:29.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.219+0000 7f9e4c000640 1 --2- 172.21.15.83:0/689598389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e44109520 0x7f9e441110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.219+0000 7f9e4c000640 1 --2- 172.21.15.83:0/689598389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e44104e00 0x7f9e44108e40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.219+0000 7f9e4c000640 1 --2- 172.21.15.83:0/689598389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e44104450 0x7f9e44104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.219+0000 7f9e4c000640 1 -- 172.21.15.83:0/689598389 >> 172.21.15.83:0/689598389 conn(0x7f9e440ffcf0 msgr2=0x7f9e44102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.219+0000 7f9e4c000640 1 -- 172.21.15.83:0/689598389 shutdown_connections 2024-04-23T14:57:29.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.219+0000 7f9e4c000640 1 -- 172.21.15.83:0/689598389 wait complete. 2024-04-23T14:57:29.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.220+0000 7f9e4c000640 1 Processor -- start 2024-04-23T14:57:29.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.220+0000 7f9e4c000640 1 -- start start 2024-04-23T14:57:29.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.220+0000 7f9e4c000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e44104450 0x7f9e441a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.220+0000 7f9e4c000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e44109520 0x7f9e441a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.220+0000 7f9e4c000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e4419a930 0x7f9e4419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.220+0000 7f9e4c000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44114590 con 0x7f9e4419a930 2024-04-23T14:57:29.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.220+0000 7f9e4c000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44114410 con 0x7f9e44104450 2024-04-23T14:57:29.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.220+0000 7f9e4c000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44114710 con 0x7f9e44109520 2024-04-23T14:57:29.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.221+0000 7f9e49600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e44104450 0x7f9e441a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.221+0000 7f9e49600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e44104450 0x7f9e441a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:39482/0 (socket says 172.21.15.83:39482) 2024-04-23T14:57:29.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.221+0000 7f9e49600640 1 -- 172.21.15.83:0/3470942616 learned_addr learned my addr 172.21.15.83:0/3470942616 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:29.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.221+0000 7f9e48c00640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e44109520 0x7f9e441a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.221+0000 7f9e4a000640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e4419a930 0x7f9e4419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.221+0000 7f9e49600640 1 -- 172.21.15.83:0/3470942616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e44109520 msgr2=0x7f9e441a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.221+0000 7f9e49600640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e44109520 0x7f9e441a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.221+0000 7f9e49600640 1 -- 172.21.15.83:0/3470942616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e4419a930 msgr2=0x7f9e4419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.221+0000 7f9e49600640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e4419a930 0x7f9e4419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.221+0000 7f9e49600640 1 -- 172.21.15.83:0/3470942616 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4419b450 con 0x7f9e44104450 2024-04-23T14:57:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.221+0000 7f9e4a000640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e4419a930 0x7f9e4419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:57:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.221+0000 7f9e49600640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e44104450 0x7f9e441a0780 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800e9f0 tx=0x7f9e3800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.222+0000 7f9e42000640 1 -- 172.21.15.83:0/3470942616 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3800cd80 con 0x7f9e44104450 2024-04-23T14:57:29.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.222+0000 7f9e42000640 1 -- 172.21.15.83:0/3470942616 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e38010910 con 0x7f9e44104450 2024-04-23T14:57:29.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.222+0000 7f9e4c000640 1 -- 172.21.15.83:0/3470942616 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4419b740 con 0x7f9e44104450 2024-04-23T14:57:29.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.222+0000 7f9e42000640 1 -- 172.21.15.83:0/3470942616 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38018990 con 0x7f9e44104450 2024-04-23T14:57:29.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.222+0000 7f9e4c000640 1 -- 172.21.15.83:0/3470942616 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e441a79e0 con 0x7f9e44104450 2024-04-23T14:57:29.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.223+0000 7f9e4c000640 1 -- 172.21.15.83:0/3470942616 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e44069270 con 0x7f9e44104450 2024-04-23T14:57:29.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.225+0000 7f9e42000640 1 -- 172.21.15.83:0/3470942616 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9e38010ac0 con 0x7f9e44104450 2024-04-23T14:57:29.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.226+0000 7f9e42000640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9e2c0a3da0 0x7f9e2c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.227+0000 7f9e42000640 1 -- 172.21.15.83:0/3470942616 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f9e380d0db0 con 0x7f9e44104450 2024-04-23T14:57:29.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.227+0000 7f9e48c00640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9e2c0a3da0 0x7f9e2c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.227+0000 7f9e48c00640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9e2c0a3da0 0x7f9e2c0a6260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009830 tx=0x7f9e300056d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.229+0000 7f9e42000640 1 -- 172.21.15.83:0/3470942616 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e3809d3e0 con 0x7f9e44104450 2024-04-23T14:57:29.468 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.467+0000 7f9e4c000640 1 -- 172.21.15.83:0/3470942616 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e4419be60 con 0x7f9e44104450 2024-04-23T14:57:29.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.468+0000 7f9e42000640 1 -- 172.21.15.83:0/3470942616 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7f9e3809cce0 con 0x7f9e44104450 2024-04-23T14:57:29.469 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:57:29.470 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:57:29.470 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:57:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.472+0000 7f9e4c000640 1 -- 172.21.15.83:0/3470942616 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9e2c0a3da0 msgr2=0x7f9e2c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.472+0000 7f9e4c000640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9e2c0a3da0 0x7f9e2c0a6260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009830 tx=0x7f9e300056d0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.472+0000 7f9e4c000640 1 -- 172.21.15.83:0/3470942616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e44104450 msgr2=0x7f9e441a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.472+0000 7f9e4c000640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e44104450 0x7f9e441a0780 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800e9f0 tx=0x7f9e3800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.472+0000 7f9e4c000640 1 -- 172.21.15.83:0/3470942616 shutdown_connections 2024-04-23T14:57:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.472+0000 7f9e4c000640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9e2c0a3da0 0x7f9e2c0a6260 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.472+0000 7f9e4c000640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e4419a930 0x7f9e4419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.472+0000 7f9e4c000640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9e44109520 0x7f9e441a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.472+0000 7f9e4c000640 1 --2- 172.21.15.83:0/3470942616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e44104450 0x7f9e441a0780 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.472+0000 7f9e4c000640 1 -- 172.21.15.83:0/3470942616 >> 172.21.15.83:0/3470942616 conn(0x7f9e440ffcf0 msgr2=0x7f9e44100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.472+0000 7f9e4c000640 1 -- 172.21.15.83:0/3470942616 shutdown_connections 2024-04-23T14:57:29.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.472+0000 7f9e4c000640 1 -- 172.21.15.83:0/3470942616 wait complete. 2024-04-23T14:57:29.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:57:29.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.689+0000 7ff90f600640 1 -- 172.21.15.83:0/3014311709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff908104e00 msgr2=0x7ff908108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.689+0000 7ff90f600640 1 --2- 172.21.15.83:0/3014311709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff908104e00 0x7ff908108e40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009860 tx=0x7ff8f002a340 comp rx=0 tx=0).stop 2024-04-23T14:57:29.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.690+0000 7ff90f600640 1 -- 172.21.15.83:0/3014311709 shutdown_connections 2024-04-23T14:57:29.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.690+0000 7ff90f600640 1 --2- 172.21.15.83:0/3014311709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff908109520 0x7ff9081110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.691 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.690+0000 7ff90f600640 1 --2- 172.21.15.83:0/3014311709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff908104e00 0x7ff908108e40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.690+0000 7ff90f600640 1 --2- 172.21.15.83:0/3014311709 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff908104450 0x7ff908104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.690+0000 7ff90f600640 1 -- 172.21.15.83:0/3014311709 >> 172.21.15.83:0/3014311709 conn(0x7ff9080ffc70 msgr2=0x7ff908102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.690+0000 7ff90f600640 1 -- 172.21.15.83:0/3014311709 shutdown_connections 2024-04-23T14:57:29.692 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.690+0000 7ff90f600640 1 -- 172.21.15.83:0/3014311709 wait complete. 2024-04-23T14:57:29.694 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.691+0000 7ff90f600640 1 Processor -- start 2024-04-23T14:57:29.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.691+0000 7ff90f600640 1 -- start start 2024-04-23T14:57:29.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.691+0000 7ff90f600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff908104450 0x7ff9081a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.691+0000 7ff90f600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff908109520 0x7ff9081a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90f600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90819a930 0x7ff90819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90f600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff908114590 con 0x7ff90819a930 2024-04-23T14:57:29.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90f600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff908114410 con 0x7ff908109520 2024-04-23T14:57:29.695 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90f600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff908114710 con 0x7ff908104450 2024-04-23T14:57:29.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff908104450 0x7ff9081a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90819a930 0x7ff90819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff908104450 0x7ff9081a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59176/0 (socket says 172.21.15.83:59176) 2024-04-23T14:57:29.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90cc00640 1 -- 172.21.15.83:0/1591556837 learned_addr learned my addr 172.21.15.83:0/1591556837 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:29.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff8ffe00640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff908109520 0x7ff9081a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90cc00640 1 -- 172.21.15.83:0/1591556837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff908109520 msgr2=0x7ff9081a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90cc00640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff908109520 0x7ff9081a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.696 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90cc00640 1 -- 172.21.15.83:0/1591556837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90819a930 msgr2=0x7ff90819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90cc00640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90819a930 0x7ff90819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90cc00640 1 -- 172.21.15.83:0/1591556837 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90819b450 con 0x7ff908104450 2024-04-23T14:57:29.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff8ffe00640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff908109520 0x7ff9081a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:57:29.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.692+0000 7ff90d600640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90819a930 0x7ff90819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:57:29.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.693+0000 7ff90cc00640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff908104450 0x7ff9081a0730 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800e9f0 tx=0x7ff8f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.693+0000 7ff906000640 1 -- 172.21.15.83:0/1591556837 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f800cd80 con 0x7ff908104450 2024-04-23T14:57:29.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.693+0000 7ff906000640 1 -- 172.21.15.83:0/1591556837 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8f8010910 con 0x7ff908104450 2024-04-23T14:57:29.697 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.693+0000 7ff90f600640 1 -- 172.21.15.83:0/1591556837 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff90819b740 con 0x7ff908104450 2024-04-23T14:57:29.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.693+0000 7ff906000640 1 -- 172.21.15.83:0/1591556837 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8018990 con 0x7ff908104450 2024-04-23T14:57:29.698 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.693+0000 7ff90f600640 1 -- 172.21.15.83:0/1591556837 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9081a7940 con 0x7ff908104450 2024-04-23T14:57:29.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.696+0000 7ff906000640 1 -- 172.21.15.83:0/1591556837 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff8f8010ac0 con 0x7ff908104450 2024-04-23T14:57:29.703 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.696+0000 7ff91258b740 1 Processor -- start 2024-04-23T14:57:29.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.696+0000 7ff91258b740 1 -- start start 2024-04-23T14:57:29.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff91258b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb5f950 0x560cdcb5fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff91258b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb60270 0x560cdcb6a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff91258b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb6a940 0x560cdcb6cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff91258b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560cdcac51d0 con 0x560cdcb6a940 2024-04-23T14:57:29.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff91258b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560cdca6b450 con 0x560cdcb5f950 2024-04-23T14:57:29.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff91258b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560cdcac1190 con 0x560cdcb60270 2024-04-23T14:57:29.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff90d600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb5f950 0x560cdcb5fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff90d600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb5f950 0x560cdcb5fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:39500/0 (socket says 172.21.15.83:39500) 2024-04-23T14:57:29.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff90d600640 1 -- 172.21.15.83:0/2502639846 learned_addr learned my addr 172.21.15.83:0/2502639846 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:29.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff90cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb60270 0x560cdcb6a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff8ffe00640 1 --2- 172.21.15.83:0/2502639846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb6a940 0x560cdcb6cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff90d600640 1 -- 172.21.15.83:0/2502639846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb60270 msgr2=0x560cdcb6a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff90d600640 1 --2- 172.21.15.83:0/2502639846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb60270 0x560cdcb6a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff90d600640 1 -- 172.21.15.83:0/2502639846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb6a940 msgr2=0x560cdcb6cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff90d600640 1 --2- 172.21.15.83:0/2502639846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb6a940 0x560cdcb6cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.705 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.697+0000 7ff90d600640 1 -- 172.21.15.83:0/2502639846 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cdcb28310 con 0x560cdcb5f950 2024-04-23T14:57:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.698+0000 7ff906000640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff8ec0a3d20 0x7ff8ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.698+0000 7ff90d600640 1 --2- 172.21.15.83:0/2502639846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb5f950 0x560cdcb5fd30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff90000e870 tx=0x7ff9000095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=732af08bc382ead9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.698+0000 7ff906000640 1 -- 172.21.15.83:0/1591556837 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7ff8f80d0ba0 con 0x7ff908104450 2024-04-23T14:57:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.698+0000 7ff8fdc00640 1 -- 172.21.15.83:0/2502639846 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90000ddd0 con 0x560cdcb5f950 2024-04-23T14:57:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.698+0000 7ff90cc00640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff8ec0a3d20 0x7ff8ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.698+0000 7ff8fdc00640 1 -- 172.21.15.83:0/2502639846 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff90001c070 con 0x560cdcb5f950 2024-04-23T14:57:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.698+0000 7ff8fdc00640 1 -- 172.21.15.83:0/2502639846 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff900016600 con 0x560cdcb5f950 2024-04-23T14:57:29.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.699+0000 7ff91258b740 1 -- 172.21.15.83:0/2502639846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb5f950 msgr2=0x560cdcb5fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.699+0000 7ff91258b740 1 --2- 172.21.15.83:0/2502639846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb5f950 0x560cdcb5fd30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff90000e870 tx=0x7ff9000095c0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.699+0000 7ff90cc00640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff8ec0a3d20 0x7ff8ec0a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x560cdcb60f10 tx=0x7ff8f80d4650 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.699+0000 7ff91258b740 1 -- 172.21.15.83:0/2502639846 shutdown_connections 2024-04-23T14:57:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.699+0000 7ff91258b740 1 --2- 172.21.15.83:0/2502639846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb6a940 0x560cdcb6cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.699+0000 7ff91258b740 1 --2- 172.21.15.83:0/2502639846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb60270 0x560cdcb6a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.699+0000 7ff91258b740 1 --2- 172.21.15.83:0/2502639846 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb5f950 0x560cdcb5fd30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.699+0000 7ff91258b740 1 -- 172.21.15.83:0/2502639846 >> 172.21.15.83:0/2502639846 conn(0x560cdcb5bc70 msgr2=0x560cdcb5c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.699+0000 7ff91258b740 1 -- 172.21.15.83:0/2502639846 shutdown_connections 2024-04-23T14:57:29.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.699+0000 7ff91258b740 1 -- 172.21.15.83:0/2502639846 wait complete. 2024-04-23T14:57:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.699+0000 7ff91258b740 1 -- start start 2024-04-23T14:57:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.699+0000 7ff91258b740 1 Processor -- start 2024-04-23T14:57:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.700+0000 7ff91258b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb5f950 0x560cdcb6a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.700+0000 7ff91258b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb60270 0x560cdcb67670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.700+0000 7ff91258b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb6a940 0x560cdcb67db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.700+0000 7ff91258b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560cdcb6d7d0 con 0x560cdcb5f950 2024-04-23T14:57:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.700+0000 7ff91258b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x560cdca7e810 con 0x560cdcb60270 2024-04-23T14:57:29.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.700+0000 7ff91258b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x560cdcb6d950 con 0x560cdcb6a940 2024-04-23T14:57:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.700+0000 7ff91258b740 10 client.0 _ll_register_callbacks cb 0x560cdca4c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:57:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.700+0000 7ff90d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb5f950 0x560cdcb6a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.700+0000 7ff90d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb5f950 0x560cdcb6a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37492/0 (socket says 172.21.15.83:37492) 2024-04-23T14:57:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.700+0000 7ff90d600640 1 -- 172.21.15.83:0/4001405852 learned_addr learned my addr 172.21.15.83:0/4001405852 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.700+0000 7ff8ffe00640 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb60270 0x560cdcb67670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.700+0000 7ff90d600640 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb6a940 0x560cdcb67db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.701+0000 7ff8ffe00640 1 -- 172.21.15.83:0/4001405852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb6a940 msgr2=0x560cdcb67db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.701+0000 7ff8ffe00640 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb6a940 0x560cdcb67db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.701+0000 7ff8ffe00640 1 -- 172.21.15.83:0/4001405852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb5f950 msgr2=0x560cdcb6a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.701+0000 7ff8ffe00640 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb5f950 0x560cdcb6a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.701+0000 7ff8ffe00640 1 -- 172.21.15.83:0/4001405852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cdca8f3f0 con 0x560cdcb60270 2024-04-23T14:57:29.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.701+0000 7ff90d600640 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb5f950 0x560cdcb6a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:57:29.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.701+0000 7ff8ffe00640 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb60270 0x560cdcb67670 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff8f002a860 tx=0x7ff8f00045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=324b83b0ae381968 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.701+0000 7ff8f6000640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:57:29.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.701+0000 7ff8f6000640 1 -- 172.21.15.83:0/4001405852 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0037040 con 0x560cdcb60270 2024-04-23T14:57:29.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.701+0000 7ff91258b740 1 -- 172.21.15.83:0/4001405852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff90001c070 con 0x560cdcb60270 2024-04-23T14:57:29.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.701+0000 7ff8f6000640 1 -- 172.21.15.83:0/4001405852 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7ff8f0030960 con 0x560cdcb60270 2024-04-23T14:57:29.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.702+0000 7ff8f6000640 1 -- 172.21.15.83:0/4001405852 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff8f004c050 con 0x560cdcb60270 2024-04-23T14:57:29.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.702+0000 7ff91258b740 10 client.26993 fetch_fsmap learned FSMap version 29 2024-04-23T14:57:29.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.702+0000 7ff91258b740 1 -- 172.21.15.83:0/4001405852 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560cdc7d73f0 con 0x560cdcb60270 2024-04-23T14:57:29.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.702+0000 7ff8f6000640 1 -- 172.21.15.83:0/4001405852 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff90001c070 con 0x560cdcb60270 2024-04-23T14:57:29.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.702+0000 7ff8f6000640 1 -- 172.21.15.83:0/4001405852 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ff8f003d6c0 con 0x560cdcb60270 2024-04-23T14:57:29.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.702+0000 7ff91258b740 10 client.26993 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:57:29.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.702+0000 7ff91258b740 10 client.26993 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:57:29.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.702+0000 7ff91258b740 20 client.26993 populate_metadata read hostname 'smithi083' 2024-04-23T14:57:29.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.702+0000 7ff91258b740 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x560cdcb7ab40 0x560cdcb7cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.702+0000 7ff91258b740 4 client.26993 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:57:29.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.702+0000 7ff91258b740 1 -- 172.21.15.83:0/4001405852 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cdcad2bf0 con 0x560cdcb7ab40 2024-04-23T14:57:29.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.703+0000 7ff90d600640 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x560cdcb7ab40 0x560cdcb7cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.703+0000 7ff90d600640 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x560cdcb7ab40 0x560cdcb7cf50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=141c13642e0a1d2f server_cookie=2061a43753424169 in_seq=0 out_seq=0 2024-04-23T14:57:29.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.703+0000 7ff8f6000640 10 client.26993 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:57:29.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.704+0000 7ff8f6000640 1 -- 172.21.15.83:0/4001405852 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560cdcad2bf0 con 0x560cdcb7ab40 2024-04-23T14:57:29.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.704+0000 7ff8f6000640 10 client.26993 handle_command_reply: tid=0 2024-04-23T14:57:29.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.704+0000 7ff91258b740 1 client.26993 shutdown 2024-04-23T14:57:29.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.705+0000 7ff91258b740 1 -- 172.21.15.83:0/4001405852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb60270 msgr2=0x560cdcb67670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.705+0000 7ff91258b740 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb60270 0x560cdcb67670 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff8f002a860 tx=0x7ff8f00045e0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.705+0000 7ff91258b740 1 -- 172.21.15.83:0/4001405852 shutdown_connections 2024-04-23T14:57:29.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.705+0000 7ff91258b740 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x560cdcb7ab40 0x560cdcb7cf50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.705+0000 7ff91258b740 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb6a940 0x560cdcb67db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.705+0000 7ff91258b740 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb60270 0x560cdcb67670 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.705+0000 7ff91258b740 1 --2- 172.21.15.83:0/4001405852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb5f950 0x560cdcb6a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.705+0000 7ff8f6000640 0 client.26993 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:57:29.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.705+0000 7ff91258b740 1 -- 172.21.15.83:0/4001405852 >> 172.21.15.83:0/4001405852 conn(0x560cdcb5bc70 msgr2=0x560cdcb61a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.705+0000 7ff91258b740 1 -- 172.21.15.83:0/4001405852 shutdown_connections 2024-04-23T14:57:29.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.705+0000 7ff91258b740 1 -- 172.21.15.83:0/4001405852 wait complete. 2024-04-23T14:57:29.715 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.705+0000 7ff91258b740 20 client.26993 trim_cache size 0 max 16384 2024-04-23T14:57:29.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.719+0000 7ff91258b740 1 Processor -- start 2024-04-23T14:57:29.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.719+0000 7ff91258b740 1 -- start start 2024-04-23T14:57:29.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.719+0000 7ff91258b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb5f950 0x560cdcb5b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.719+0000 7ff91258b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb60270 0x560cdcb69eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff91258b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb6a940 0x560cdcb6a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff91258b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9081b27b0 con 0x560cdcb6a940 2024-04-23T14:57:29.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff91258b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff9081b0080 con 0x560cdcb5f950 2024-04-23T14:57:29.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff91258b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff9081adef0 con 0x560cdcb60270 2024-04-23T14:57:29.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff90d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb6a940 0x560cdcb6a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff90d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb6a940 0x560cdcb6a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37502/0 (socket says 172.21.15.83:37502) 2024-04-23T14:57:29.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff90d600640 1 -- 172.21.15.83:0/4174421076 learned_addr learned my addr 172.21.15.83:0/4174421076 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:29.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff8ffe00640 1 --2- 172.21.15.83:0/4174421076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb60270 0x560cdcb69eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff90d600640 1 --2- 172.21.15.83:0/4174421076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb5f950 0x560cdcb5b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff90d600640 1 -- 172.21.15.83:0/4174421076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb60270 msgr2=0x560cdcb69eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff90d600640 1 --2- 172.21.15.83:0/4174421076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb60270 0x560cdcb69eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff90d600640 1 -- 172.21.15.83:0/4174421076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb5f950 msgr2=0x560cdcb5b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff90d600640 1 --2- 172.21.15.83:0/4174421076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb5f950 0x560cdcb5b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.720+0000 7ff90d600640 1 -- 172.21.15.83:0/4174421076 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cdc8a17d0 con 0x560cdcb6a940 2024-04-23T14:57:29.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.721+0000 7ff90d600640 1 --2- 172.21.15.83:0/4174421076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb6a940 0x560cdcb6a3f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff90000d7b0 tx=0x7ff900009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f00b4fc52ff8d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.721+0000 7ff8fd200640 1 -- 172.21.15.83:0/4174421076 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff900003970 con 0x560cdcb6a940 2024-04-23T14:57:29.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.721+0000 7ff8fd200640 1 -- 172.21.15.83:0/4174421076 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff90001c040 con 0x560cdcb6a940 2024-04-23T14:57:29.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.721+0000 7ff8fd200640 1 -- 172.21.15.83:0/4174421076 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9000042d0 con 0x560cdcb6a940 2024-04-23T14:57:29.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.721+0000 7ff91258b740 1 -- 172.21.15.83:0/4174421076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb6a940 msgr2=0x560cdcb6a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.721+0000 7ff91258b740 1 --2- 172.21.15.83:0/4174421076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb6a940 0x560cdcb6a3f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff90000d7b0 tx=0x7ff900009780 comp rx=0 tx=0).stop 2024-04-23T14:57:29.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.721+0000 7ff91258b740 1 -- 172.21.15.83:0/4174421076 shutdown_connections 2024-04-23T14:57:29.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.721+0000 7ff91258b740 1 --2- 172.21.15.83:0/4174421076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb6a940 0x560cdcb6a3f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.721+0000 7ff91258b740 1 --2- 172.21.15.83:0/4174421076 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb60270 0x560cdcb69eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.721+0000 7ff91258b740 1 --2- 172.21.15.83:0/4174421076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb5f950 0x560cdcb5b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.721+0000 7ff91258b740 1 -- 172.21.15.83:0/4174421076 >> 172.21.15.83:0/4174421076 conn(0x560cdcb5bc70 msgr2=0x560cdcb62940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.721+0000 7ff91258b740 1 -- 172.21.15.83:0/4174421076 shutdown_connections 2024-04-23T14:57:29.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.722+0000 7ff91258b740 1 -- 172.21.15.83:0/4174421076 wait complete. 2024-04-23T14:57:29.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.722+0000 7ff91258b740 1 -- start start 2024-04-23T14:57:29.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.722+0000 7ff91258b740 1 Processor -- start 2024-04-23T14:57:29.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.722+0000 7ff91258b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb5f950 0x560cdcb73110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.722+0000 7ff91258b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb60270 0x560cdcb649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.722+0000 7ff91258b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb6a940 0x560cdcb650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.722+0000 7ff91258b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9081ae540 con 0x560cdcb60270 2024-04-23T14:57:29.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.722+0000 7ff91258b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7ff9081af9b0 con 0x560cdcb6a940 2024-04-23T14:57:29.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.722+0000 7ff91258b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7ff9081aec10 con 0x560cdcb5f950 2024-04-23T14:57:29.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff91258b740 10 client.0 _ll_register_callbacks cb 0x7ff8f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:57:29.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff8ffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb60270 0x560cdcb649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff8ffe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb60270 0x560cdcb649d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37512/0 (socket says 172.21.15.83:37512) 2024-04-23T14:57:29.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff8ffe00640 1 -- 172.21.15.83:0/2188354847 learned_addr learned my addr 172.21.15.83:0/2188354847 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:29.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff90d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb5f950 0x560cdcb73110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff90d600640 1 --2- 172.21.15.83:0/2188354847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb6a940 0x560cdcb650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff8ffe00640 1 -- 172.21.15.83:0/2188354847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb5f950 msgr2=0x560cdcb73110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff8ffe00640 1 --2- 172.21.15.83:0/2188354847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb5f950 0x560cdcb73110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff8ffe00640 1 -- 172.21.15.83:0/2188354847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb6a940 msgr2=0x560cdcb650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff8ffe00640 1 --2- 172.21.15.83:0/2188354847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb6a940 0x560cdcb650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff8ffe00640 1 -- 172.21.15.83:0/2188354847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cdcb6b540 con 0x560cdcb60270 2024-04-23T14:57:29.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff8ffe00640 1 --2- 172.21.15.83:0/2188354847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb60270 0x560cdcb649d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0001eb0 tx=0x7ff8f003df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91125eb955caae79 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.723+0000 7ff8f6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.724+0000 7ff8f6000640 1 -- 172.21.15.83:0/2188354847 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0037040 con 0x560cdcb60270 2024-04-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.724+0000 7ff8f6000640 1 -- 172.21.15.83:0/2188354847 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7ff8f0030970 con 0x560cdcb60270 2024-04-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.724+0000 7ff91258b740 1 -- 172.21.15.83:0/2188354847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff90001c040 con 0x560cdcb60270 2024-04-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.724+0000 7ff8f6000640 1 -- 172.21.15.83:0/2188354847 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff8f0060050 con 0x560cdcb60270 2024-04-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.724+0000 7ff91258b740 10 client.16800 fetch_fsmap learned FSMap version 29 2024-04-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.724+0000 7ff91258b740 1 -- 172.21.15.83:0/2188354847 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560cdcbe1340 con 0x560cdcb60270 2024-04-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.724+0000 7ff8f6000640 1 -- 172.21.15.83:0/2188354847 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff90001c040 con 0x560cdcb60270 2024-04-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.724+0000 7ff8f6000640 1 -- 172.21.15.83:0/2188354847 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ff8f0032770 con 0x560cdcb60270 2024-04-23T14:57:29.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.725+0000 7ff91258b740 10 client.16800 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.725+0000 7ff91258b740 10 client.16800 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.725+0000 7ff91258b740 20 client.16800 populate_metadata read hostname 'smithi083' 2024-04-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.725+0000 7ff91258b740 1 --2- 172.21.15.83:0/2188354847 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x560cdcbe1b30 0x560cdcbe3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.725+0000 7ff91258b740 4 client.16800 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.725+0000 7ff91258b740 1 -- 172.21.15.83:0/2188354847 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560cdcacca60 con 0x560cdcbe1b30 2024-04-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.725+0000 7ff90d600640 1 --2- 172.21.15.83:0/2188354847 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x560cdcbe1b30 0x560cdcbe3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.725+0000 7ff8f6000640 10 client.16800 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:57:29.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.725+0000 7ff90d600640 1 --2- 172.21.15.83:0/2188354847 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x560cdcbe1b30 0x560cdcbe3f50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6972457ff49a4b68 server_cookie=899985f0d5ed21b4 in_seq=0 out_seq=0 2024-04-23T14:57:29.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff8f6000640 1 -- 172.21.15.83:0/2188354847 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560cdcacca60 con 0x560cdcbe1b30 2024-04-23T14:57:29.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff8f6000640 10 client.16800 handle_command_reply: tid=0 2024-04-23T14:57:29.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff91258b740 1 client.16800 shutdown 2024-04-23T14:57:29.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff91258b740 1 -- 172.21.15.83:0/2188354847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb60270 msgr2=0x560cdcb649d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff91258b740 1 --2- 172.21.15.83:0/2188354847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb60270 0x560cdcb649d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0001eb0 tx=0x7ff8f003df80 comp rx=0 tx=0).stop 2024-04-23T14:57:29.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff91258b740 1 -- 172.21.15.83:0/2188354847 shutdown_connections 2024-04-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff91258b740 1 --2- 172.21.15.83:0/2188354847 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x560cdcbe1b30 0x560cdcbe3f50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff91258b740 1 --2- 172.21.15.83:0/2188354847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x560cdcb6a940 0x560cdcb650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff91258b740 1 --2- 172.21.15.83:0/2188354847 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560cdcb60270 0x560cdcb649d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff91258b740 1 --2- 172.21.15.83:0/2188354847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560cdcb5f950 0x560cdcb73110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff8f6000640 0 client.16800 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:57:29.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff91258b740 1 -- 172.21.15.83:0/2188354847 >> 172.21.15.83:0/2188354847 conn(0x560cdcb5bc70 msgr2=0x560cdcb61070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff91258b740 1 -- 172.21.15.83:0/2188354847 shutdown_connections 2024-04-23T14:57:29.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff91258b740 1 -- 172.21.15.83:0/2188354847 wait complete. 2024-04-23T14:57:29.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.726+0000 7ff91258b740 20 client.16800 trim_cache size 0 max 16384 2024-04-23T14:57:29.736 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:57:29.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.727+0000 7ff90f600640 1 -- 172.21.15.83:0/1591556837 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff8ec0a3d20 msgr2=0x7ff8ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.737 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.727+0000 7ff90f600640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff8ec0a3d20 0x7ff8ec0a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x560cdcb60f10 tx=0x7ff8f80d4650 comp rx=0 tx=0).stop 2024-04-23T14:57:29.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.727+0000 7ff90f600640 1 -- 172.21.15.83:0/1591556837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff908104450 msgr2=0x7ff9081a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.727+0000 7ff90f600640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff908104450 0x7ff9081a0730 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800e9f0 tx=0x7ff8f800eeb0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.727+0000 7ff90f600640 1 -- 172.21.15.83:0/1591556837 shutdown_connections 2024-04-23T14:57:29.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.727+0000 7ff90f600640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7ff8ec0a3d20 0x7ff8ec0a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.727+0000 7ff90f600640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff90819a930 0x7ff90819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.727+0000 7ff90f600640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff908109520 0x7ff9081a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.727+0000 7ff90f600640 1 --2- 172.21.15.83:0/1591556837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff908104450 0x7ff9081a0730 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.738 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.727+0000 7ff90f600640 1 -- 172.21.15.83:0/1591556837 >> 172.21.15.83:0/1591556837 conn(0x7ff9080ffc70 msgr2=0x7ff908101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.727+0000 7ff90f600640 1 -- 172.21.15.83:0/1591556837 shutdown_connections 2024-04-23T14:57:29.739 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.727+0000 7ff90f600640 1 -- 172.21.15.83:0/1591556837 wait complete. 2024-04-23T14:57:29.742 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:29 smithi083 ceph-mon[27867]: pgmap v280: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-23T14:57:29.742 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:29 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3470942616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:57:29.744 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:57:29.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:29 smithi173 ceph-mon[32443]: pgmap v280: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-23T14:57:29.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:29 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3470942616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:57:29.939 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.938+0000 7f1585600640 1 -- 172.21.15.83:0/3428412372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1580104320 msgr2=0x7f1580104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.938+0000 7f1585600640 1 --2- 172.21.15.83:0/3428412372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1580104320 0x7f1580104700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1570009860 tx=0x7f157002a340 comp rx=0 tx=0).stop 2024-04-23T14:57:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.938+0000 7f1585600640 1 -- 172.21.15.83:0/3428412372 shutdown_connections 2024-04-23T14:57:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.938+0000 7f1585600640 1 --2- 172.21.15.83:0/3428412372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15801093f0 0x7f1580110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.938+0000 7f1585600640 1 --2- 172.21.15.83:0/3428412372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1580104cd0 0x7f1580108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.938+0000 7f1585600640 1 --2- 172.21.15.83:0/3428412372 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1580104320 0x7f1580104700 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.938+0000 7f1585600640 1 -- 172.21.15.83:0/3428412372 >> 172.21.15.83:0/3428412372 conn(0x7f15800ffb40 msgr2=0x7f1580101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.940 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.938+0000 7f1585600640 1 -- 172.21.15.83:0/3428412372 shutdown_connections 2024-04-23T14:57:29.941 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.938+0000 7f1585600640 1 -- 172.21.15.83:0/3428412372 wait complete. 2024-04-23T14:57:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.939+0000 7f1585600640 1 Processor -- start 2024-04-23T14:57:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.940+0000 7f1585600640 1 -- start start 2024-04-23T14:57:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.940+0000 7f1585600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1580104320 0x7f15801a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.940+0000 7f1585600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15801093f0 0x7f15801a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.943 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.940+0000 7f1585600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f158019a800 0x7f158019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.940+0000 7f1585600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1580114480 con 0x7f15801093f0 2024-04-23T14:57:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.940+0000 7f1585600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1580114300 con 0x7f1580104320 2024-04-23T14:57:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.940+0000 7f1585600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1580114600 con 0x7f158019a800 2024-04-23T14:57:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.940+0000 7f157f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15801093f0 0x7f15801a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.941+0000 7f157f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15801093f0 0x7f15801a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37530/0 (socket says 172.21.15.83:37530) 2024-04-23T14:57:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.941+0000 7f157f400640 1 -- 172.21.15.83:0/4021835209 learned_addr learned my addr 172.21.15.83:0/4021835209 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.941+0000 7f157fe00640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1580104320 0x7f15801a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.944 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.941+0000 7f1584c00640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f158019a800 0x7f158019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.941+0000 7f157f400640 1 -- 172.21.15.83:0/4021835209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f158019a800 msgr2=0x7f158019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.941+0000 7f157f400640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f158019a800 0x7f158019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.941+0000 7f157f400640 1 -- 172.21.15.83:0/4021835209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1580104320 msgr2=0x7f15801a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.941+0000 7f157f400640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1580104320 0x7f15801a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.941+0000 7f157f400640 1 -- 172.21.15.83:0/4021835209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158019b390 con 0x7f15801093f0 2024-04-23T14:57:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.941+0000 7f157f400640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15801093f0 0x7f15801a0bd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f156c00e9a0 tx=0x7f156c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.942+0000 7f157cc00640 1 -- 172.21.15.83:0/4021835209 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c00cd80 con 0x7f15801093f0 2024-04-23T14:57:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.942+0000 7f1585600640 1 -- 172.21.15.83:0/4021835209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158019b680 con 0x7f15801093f0 2024-04-23T14:57:29.945 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.942+0000 7f157cc00640 1 -- 172.21.15.83:0/4021835209 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f156c0035f0 con 0x7f15801093f0 2024-04-23T14:57:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.942+0000 7f157cc00640 1 -- 172.21.15.83:0/4021835209 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c010670 con 0x7f15801093f0 2024-04-23T14:57:29.946 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.942+0000 7f1585600640 1 -- 172.21.15.83:0/4021835209 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15801a7860 con 0x7f15801093f0 2024-04-23T14:57:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.945+0000 7f15899d0740 1 Processor -- start 2024-04-23T14:57:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.945+0000 7f15899d0740 1 -- start start 2024-04-23T14:57:29.947 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f15899d0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761ab40 0x55f43761af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f15899d0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761b460 0x55f4376255f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f157cc00640 1 -- 172.21.15.83:0/4021835209 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f156c0037a0 con 0x7f15801093f0 2024-04-23T14:57:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f15899d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f437627f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f15899d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f4375801d0 con 0x55f437625b30 2024-04-23T14:57:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f15899d0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f437526450 con 0x55f43761b460 2024-04-23T14:57:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f15899d0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f43757c190 con 0x55f43761ab40 2024-04-23T14:57:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f1584c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761ab40 0x55f43761af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.948 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f157fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761b460 0x55f4376255f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f157f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f437627f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f1584c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761ab40 0x55f43761af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59246/0 (socket says 172.21.15.83:59246) 2024-04-23T14:57:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f1584c00640 1 -- 172.21.15.83:0/2584853011 learned_addr learned my addr 172.21.15.83:0/2584853011 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f1584c00640 1 -- 172.21.15.83:0/2584853011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761b460 msgr2=0x55f4376255f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f1584c00640 1 --2- 172.21.15.83:0/2584853011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761b460 0x55f4376255f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f1584c00640 1 -- 172.21.15.83:0/2584853011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 msgr2=0x55f437627f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f1584c00640 1 --2- 172.21.15.83:0/2584853011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f437627f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.946+0000 7f1584c00640 1 -- 172.21.15.83:0/2584853011 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4375e3310 con 0x55f43761ab40 2024-04-23T14:57:29.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.947+0000 7f1584c00640 1 --2- 172.21.15.83:0/2584853011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761ab40 0x55f43761af20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1578009f50 tx=0x7f15780118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9373167c84051d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.947+0000 7f1575200640 1 -- 172.21.15.83:0/2584853011 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578004d20 con 0x55f43761ab40 2024-04-23T14:57:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.947+0000 7f1575200640 1 -- 172.21.15.83:0/2584853011 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1578004050 con 0x55f43761ab40 2024-04-23T14:57:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.947+0000 7f1575200640 1 -- 172.21.15.83:0/2584853011 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578005240 con 0x55f43761ab40 2024-04-23T14:57:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.947+0000 7f157cc00640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f15640a3d20 0x7f15640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.948+0000 7f157cc00640 1 -- 172.21.15.83:0/4021835209 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f156c0d0ad0 con 0x7f15801093f0 2024-04-23T14:57:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.948+0000 7f157fe00640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f15640a3d20 0x7f15640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.948+0000 7f15899d0740 1 -- 172.21.15.83:0/2584853011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761ab40 msgr2=0x55f43761af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.948+0000 7f15899d0740 1 --2- 172.21.15.83:0/2584853011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761ab40 0x55f43761af20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1578009f50 tx=0x7f15780118d0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.948+0000 7f15899d0740 1 -- 172.21.15.83:0/2584853011 shutdown_connections 2024-04-23T14:57:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.948+0000 7f15899d0740 1 --2- 172.21.15.83:0/2584853011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f437627f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.948+0000 7f15899d0740 1 --2- 172.21.15.83:0/2584853011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761b460 0x55f4376255f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.948+0000 7f15899d0740 1 --2- 172.21.15.83:0/2584853011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761ab40 0x55f43761af20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.948+0000 7f15899d0740 1 -- 172.21.15.83:0/2584853011 >> 172.21.15.83:0/2584853011 conn(0x55f437616e60 msgr2=0x55f437617240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.948+0000 7f15899d0740 1 -- 172.21.15.83:0/2584853011 shutdown_connections 2024-04-23T14:57:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.948+0000 7f15899d0740 1 -- 172.21.15.83:0/2584853011 wait complete. 2024-04-23T14:57:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.948+0000 7f15899d0740 1 -- start start 2024-04-23T14:57:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f157fe00640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f15640a3d20 0x7f15640a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f157002a930 tx=0x7f15700056d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f15899d0740 1 Processor -- start 2024-04-23T14:57:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f15899d0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761ab40 0x55f437625580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f15899d0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761b460 0x55f437622870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f15899d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f437622fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f15899d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f4376289c0 con 0x55f437625b30 2024-04-23T14:57:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f15899d0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55f437539810 con 0x55f43761ab40 2024-04-23T14:57:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f15899d0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55f437628b40 con 0x55f43761b460 2024-04-23T14:57:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f15899d0740 10 client.0 _ll_register_callbacks cb 0x55f437507140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:57:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f157f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761b460 0x55f437622870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f1584c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f437622fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f157fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761ab40 0x55f437625580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f157f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761b460 0x55f437622870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59248/0 (socket says 172.21.15.83:59248) 2024-04-23T14:57:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.949+0000 7f157f400640 1 -- 172.21.15.83:0/2011420808 learned_addr learned my addr 172.21.15.83:0/2011420808 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.950+0000 7f157f400640 1 -- 172.21.15.83:0/2011420808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761ab40 msgr2=0x55f437625580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.950+0000 7f157f400640 1 --2- 172.21.15.83:0/2011420808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761ab40 0x55f437625580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.950+0000 7f157f400640 1 -- 172.21.15.83:0/2011420808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 msgr2=0x55f437622fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.950+0000 7f157f400640 1 --2- 172.21.15.83:0/2011420808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f437622fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.950+0000 7f157f400640 1 -- 172.21.15.83:0/2011420808 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f43754a3f0 con 0x55f43761b460 2024-04-23T14:57:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.950+0000 7f157f400640 1 --2- 172.21.15.83:0/2011420808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761b460 0x55f437622870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f156c009230 tx=0x7f156c009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ff818e292757831 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.950+0000 7f1569600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:57:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.950+0000 7f1569600640 1 -- 172.21.15.83:0/2011420808 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c0d7060 con 0x55f43761b460 2024-04-23T14:57:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.950+0000 7f1569600640 1 -- 172.21.15.83:0/2011420808 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f156c0bbbc0 con 0x55f43761b460 2024-04-23T14:57:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.950+0000 7f15899d0740 1 -- 172.21.15.83:0/2011420808 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1578004050 con 0x55f43761b460 2024-04-23T14:57:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.950+0000 7f1569600640 1 -- 172.21.15.83:0/2011420808 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f156c09c540 con 0x55f43761b460 2024-04-23T14:57:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.951+0000 7f15899d0740 10 client.26986 fetch_fsmap learned FSMap version 29 2024-04-23T14:57:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.951+0000 7f15899d0740 1 -- 172.21.15.83:0/2011420808 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f4372923f0 con 0x55f43761b460 2024-04-23T14:57:29.959 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.951+0000 7f1569600640 1 -- 172.21.15.83:0/2011420808 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1578004050 con 0x55f43761b460 2024-04-23T14:57:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.951+0000 7f1569600640 1 -- 172.21.15.83:0/2011420808 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f156c09be50 con 0x55f43761b460 2024-04-23T14:57:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.951+0000 7f15899d0740 10 client.26986 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:57:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.951+0000 7f15899d0740 10 client.26986 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:57:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.951+0000 7f15899d0740 20 client.26986 populate_metadata read hostname 'smithi083' 2024-04-23T14:57:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.951+0000 7f15899d0740 1 --2- 172.21.15.83:0/2011420808 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f437635d40 0x55f437638130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.951+0000 7f15899d0740 4 client.26986 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:57:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.951+0000 7f15899d0740 1 -- 172.21.15.83:0/2011420808 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f43758dbf0 con 0x55f437635d40 2024-04-23T14:57:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.952+0000 7f157fe00640 1 --2- 172.21.15.83:0/2011420808 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f437635d40 0x55f437638130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.952+0000 7f157fe00640 1 --2- 172.21.15.83:0/2011420808 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f437635d40 0x55f437638130 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=faef347ff1068088 server_cookie=4040b1feadc9f792 in_seq=0 out_seq=0 2024-04-23T14:57:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.952+0000 7f1569600640 10 client.26986 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:57:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f1569600640 1 -- 172.21.15.83:0/2011420808 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f43758dbf0 con 0x55f437635d40 2024-04-23T14:57:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f1569600640 10 client.26986 handle_command_reply: tid=0 2024-04-23T14:57:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f15899d0740 1 client.26986 shutdown 2024-04-23T14:57:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f15899d0740 1 -- 172.21.15.83:0/2011420808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761b460 msgr2=0x55f437622870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f15899d0740 1 --2- 172.21.15.83:0/2011420808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761b460 0x55f437622870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f156c009230 tx=0x7f156c009560 comp rx=0 tx=0).stop 2024-04-23T14:57:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f15899d0740 1 -- 172.21.15.83:0/2011420808 shutdown_connections 2024-04-23T14:57:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f15899d0740 1 --2- 172.21.15.83:0/2011420808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f437622fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.961 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f15899d0740 1 --2- 172.21.15.83:0/2011420808 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f437635d40 0x55f437638130 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f15899d0740 1 --2- 172.21.15.83:0/2011420808 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761b460 0x55f437622870 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f15899d0740 1 --2- 172.21.15.83:0/2011420808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761ab40 0x55f437625580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f15899d0740 1 -- 172.21.15.83:0/2011420808 >> 172.21.15.83:0/2011420808 conn(0x55f437616e60 msgr2=0x55f43761cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f1569600640 0 client.26986 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:57:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f15899d0740 1 -- 172.21.15.83:0/2011420808 shutdown_connections 2024-04-23T14:57:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f15899d0740 1 -- 172.21.15.83:0/2011420808 wait complete. 2024-04-23T14:57:29.962 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.953+0000 7f15899d0740 20 client.26986 trim_cache size 0 max 16384 2024-04-23T14:57:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.968+0000 7f15899d0740 1 Processor -- start 2024-04-23T14:57:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.968+0000 7f15899d0740 1 -- start start 2024-04-23T14:57:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.968+0000 7f15899d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f43761ab40 0x55f437631880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.968+0000 7f15899d0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761b460 0x55f4376250b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.968+0000 7f15899d0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f437625b30 0x55f4376255f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.968+0000 7f15899d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15801b2680 con 0x55f43761ab40 2024-04-23T14:57:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.968+0000 7f15899d0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f15801aff50 con 0x55f43761b460 2024-04-23T14:57:29.969 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.968+0000 7f15899d0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f15801addc0 con 0x55f437625b30 2024-04-23T14:57:29.973 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.968+0000 7f157fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f43761ab40 0x55f437631880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.968+0000 7f157fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f43761ab40 0x55f437631880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37572/0 (socket says 172.21.15.83:37572) 2024-04-23T14:57:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.968+0000 7f157fe00640 1 -- 172.21.15.83:0/879172965 learned_addr learned my addr 172.21.15.83:0/879172965 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.968+0000 7f157f400640 1 --2- 172.21.15.83:0/879172965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761b460 0x55f4376250b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.969+0000 7f1584c00640 1 --2- 172.21.15.83:0/879172965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f437625b30 0x55f4376255f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.969+0000 7f157fe00640 1 -- 172.21.15.83:0/879172965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f437625b30 msgr2=0x55f4376255f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.969+0000 7f157fe00640 1 --2- 172.21.15.83:0/879172965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f437625b30 0x55f4376255f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.969+0000 7f157fe00640 1 -- 172.21.15.83:0/879172965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761b460 msgr2=0x55f4376250b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.969+0000 7f157fe00640 1 --2- 172.21.15.83:0/879172965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761b460 0x55f4376250b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.969+0000 7f157fe00640 1 -- 172.21.15.83:0/879172965 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f43735c7d0 con 0x55f43761ab40 2024-04-23T14:57:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.969+0000 7f157fe00640 1 --2- 172.21.15.83:0/879172965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f43761ab40 0x55f437631880 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1570000c00 tx=0x7f1570005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f408c5e70a4be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.969+0000 7f156be00640 1 -- 172.21.15.83:0/879172965 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570037040 con 0x55f43761ab40 2024-04-23T14:57:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.969+0000 7f156be00640 1 -- 172.21.15.83:0/879172965 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1570054070 con 0x55f43761ab40 2024-04-23T14:57:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.969+0000 7f156be00640 1 -- 172.21.15.83:0/879172965 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15700038e0 con 0x55f43761ab40 2024-04-23T14:57:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.970+0000 7f15899d0740 1 -- 172.21.15.83:0/879172965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f43761ab40 msgr2=0x55f437631880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.970+0000 7f15899d0740 1 --2- 172.21.15.83:0/879172965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f43761ab40 0x55f437631880 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1570000c00 tx=0x7f1570005f90 comp rx=0 tx=0).stop 2024-04-23T14:57:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.970+0000 7f15899d0740 1 -- 172.21.15.83:0/879172965 shutdown_connections 2024-04-23T14:57:29.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.970+0000 7f15899d0740 1 --2- 172.21.15.83:0/879172965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f437625b30 0x55f4376255f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.970+0000 7f15899d0740 1 --2- 172.21.15.83:0/879172965 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761b460 0x55f4376250b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.970+0000 7f15899d0740 1 --2- 172.21.15.83:0/879172965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f43761ab40 0x55f437631880 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.970+0000 7f15899d0740 1 -- 172.21.15.83:0/879172965 >> 172.21.15.83:0/879172965 conn(0x55f437616e60 msgr2=0x55f437617240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.970+0000 7f15899d0740 1 -- 172.21.15.83:0/879172965 shutdown_connections 2024-04-23T14:57:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.970+0000 7f15899d0740 1 -- 172.21.15.83:0/879172965 wait complete. 2024-04-23T14:57:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.970+0000 7f15899d0740 1 -- start start 2024-04-23T14:57:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.971+0000 7f15899d0740 1 Processor -- start 2024-04-23T14:57:29.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.971+0000 7f15899d0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761b460 0x55f43762dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.971+0000 7f15899d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f43762e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.971+0000 7f15899d0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761eb20 0x55f43761efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.971+0000 7f15899d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15801ae410 con 0x55f437625b30 2024-04-23T14:57:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.971+0000 7f15899d0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f15801af880 con 0x55f43761eb20 2024-04-23T14:57:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.971+0000 7f15899d0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f15801aeae0 con 0x55f43761b460 2024-04-23T14:57:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.971+0000 7f15899d0740 10 client.0 _ll_register_callbacks cb 0x7f156c0d4270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:57:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.971+0000 7f157f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f43762e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.977 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.971+0000 7f157f400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f43762e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37580/0 (socket says 172.21.15.83:37580) 2024-04-23T14:57:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.971+0000 7f157f400640 1 -- 172.21.15.83:0/2040873415 learned_addr learned my addr 172.21.15.83:0/2040873415 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.971+0000 7f157fe00640 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761b460 0x55f43762dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f1584c00640 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761eb20 0x55f43761efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f157f400640 1 -- 172.21.15.83:0/2040873415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761b460 msgr2=0x55f43762dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f157f400640 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761b460 0x55f43762dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f157f400640 1 -- 172.21.15.83:0/2040873415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761eb20 msgr2=0x55f43761efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f157f400640 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761eb20 0x55f43761efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f157f400640 1 -- 172.21.15.83:0/2040873415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f43761f5e0 con 0x55f437625b30 2024-04-23T14:57:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f157f400640 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f43762e2d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f156c098e60 tx=0x7f156c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b11bc5b0684b960 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f1569600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:57:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f1569600640 1 -- 172.21.15.83:0/2040873415 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c0d7070 con 0x55f437625b30 2024-04-23T14:57:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f1569600640 1 -- 172.21.15.83:0/2040873415 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f156c0872b0 con 0x55f437625b30 2024-04-23T14:57:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f15899d0740 1 -- 172.21.15.83:0/2040873415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1570054070 con 0x55f437625b30 2024-04-23T14:57:29.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f1569600640 1 -- 172.21.15.83:0/2040873415 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f156c080050 con 0x55f437625b30 2024-04-23T14:57:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f15899d0740 10 client.16818 fetch_fsmap learned FSMap version 29 2024-04-23T14:57:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.972+0000 7f15899d0740 1 -- 172.21.15.83:0/2040873415 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f4376208e0 con 0x55f437625b30 2024-04-23T14:57:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.973+0000 7f1569600640 1 -- 172.21.15.83:0/2040873415 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1570054070 con 0x55f437625b30 2024-04-23T14:57:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.973+0000 7f1569600640 1 -- 172.21.15.83:0/2040873415 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f156c003fb0 con 0x55f437625b30 2024-04-23T14:57:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.973+0000 7f15899d0740 10 client.16818 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:57:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.973+0000 7f15899d0740 10 client.16818 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:57:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.973+0000 7f15899d0740 20 client.16818 populate_metadata read hostname 'smithi083' 2024-04-23T14:57:29.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.973+0000 7f15899d0740 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f43769d720 0x55f43769fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:29.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.973+0000 7f15899d0740 4 client.16818 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:57:29.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.973+0000 7f15899d0740 1 -- 172.21.15.83:0/2040873415 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f437587a60 con 0x55f43769d720 2024-04-23T14:57:29.981 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:29 smithi203 ceph-mon[32800]: pgmap v280: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-23T14:57:29.981 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:29 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3470942616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:57:29.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.973+0000 7f157fe00640 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f43769d720 0x55f43769fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f157fe00640 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f43769d720 0x55f43769fb40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40f3447b425e05c6 server_cookie=d57712f1a3213ec in_seq=0 out_seq=0 2024-04-23T14:57:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f1569600640 10 client.16818 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:57:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f1569600640 1 -- 172.21.15.83:0/2040873415 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f437587a60 con 0x55f43769d720 2024-04-23T14:57:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f1569600640 10 client.16818 handle_command_reply: tid=0 2024-04-23T14:57:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f15899d0740 1 client.16818 shutdown 2024-04-23T14:57:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f15899d0740 1 -- 172.21.15.83:0/2040873415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 msgr2=0x55f43762e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f15899d0740 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f43762e2d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f156c098e60 tx=0x7f156c000bc0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f15899d0740 1 -- 172.21.15.83:0/2040873415 shutdown_connections 2024-04-23T14:57:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f15899d0740 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f43761eb20 0x55f43761efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f15899d0740 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f437625b30 0x55f43762e2d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f15899d0740 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55f43769d720 0x55f43769fb40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f15899d0740 1 --2- 172.21.15.83:0/2040873415 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f43761b460 0x55f43762dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f1569600640 0 client.16818 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:57:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f15899d0740 1 -- 172.21.15.83:0/2040873415 >> 172.21.15.83:0/2040873415 conn(0x55f437616e60 msgr2=0x55f4376156c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f15899d0740 1 -- 172.21.15.83:0/2040873415 shutdown_connections 2024-04-23T14:57:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f15899d0740 1 -- 172.21.15.83:0/2040873415 wait complete. 2024-04-23T14:57:29.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.974+0000 7f15899d0740 20 client.16818 trim_cache size 0 max 16384 2024-04-23T14:57:29.985 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:57:29.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.975+0000 7f1585600640 1 -- 172.21.15.83:0/4021835209 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f15640a3d20 msgr2=0x7f15640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.975+0000 7f1585600640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f15640a3d20 0x7f15640a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f157002a930 tx=0x7f15700056d0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.975+0000 7f1585600640 1 -- 172.21.15.83:0/4021835209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15801093f0 msgr2=0x7f15801a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.975+0000 7f1585600640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15801093f0 0x7f15801a0bd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f156c00e9a0 tx=0x7f156c00ee60 comp rx=0 tx=0).stop 2024-04-23T14:57:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.975+0000 7f1585600640 1 -- 172.21.15.83:0/4021835209 shutdown_connections 2024-04-23T14:57:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.975+0000 7f1585600640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f158019a800 0x7f158019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.975+0000 7f1585600640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f15640a3d20 0x7f15640a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.975+0000 7f1585600640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f15801093f0 0x7f15801a0bd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.975+0000 7f1585600640 1 --2- 172.21.15.83:0/4021835209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1580104320 0x7f15801a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:29.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.975+0000 7f1585600640 1 -- 172.21.15.83:0/4021835209 >> 172.21.15.83:0/4021835209 conn(0x7f15800ffb40 msgr2=0x7f1580100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:29.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.975+0000 7f1585600640 1 -- 172.21.15.83:0/4021835209 shutdown_connections 2024-04-23T14:57:29.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:29.975+0000 7f1585600640 1 -- 172.21.15.83:0/4021835209 wait complete. 2024-04-23T14:57:29.992 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:57:30.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.188+0000 7fba60e00640 1 -- 172.21.15.83:0/2686255191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba5c10e660 msgr2=0x7fba5c10ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:30.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.188+0000 7fba60e00640 1 --2- 172.21.15.83:0/2686255191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba5c10e660 0x7fba5c10ea40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009930 tx=0x7fba4c02a440 comp rx=0 tx=0).stop 2024-04-23T14:57:30.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.188+0000 7fba60e00640 1 -- 172.21.15.83:0/2686255191 shutdown_connections 2024-04-23T14:57:30.190 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.188+0000 7fba60e00640 1 --2- 172.21.15.83:0/2686255191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5c108fd0 0x7fba5c115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.188+0000 7fba60e00640 1 --2- 172.21.15.83:0/2686255191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5c108610 0x7fba5c108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.188+0000 7fba60e00640 1 --2- 172.21.15.83:0/2686255191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba5c10e660 0x7fba5c10ea40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.188+0000 7fba60e00640 1 -- 172.21.15.83:0/2686255191 >> 172.21.15.83:0/2686255191 conn(0x7fba5c076540 msgr2=0x7fba5c078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.189+0000 7fba60e00640 1 -- 172.21.15.83:0/2686255191 shutdown_connections 2024-04-23T14:57:30.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.189+0000 7fba60e00640 1 -- 172.21.15.83:0/2686255191 wait complete. 2024-04-23T14:57:30.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba60e00640 1 Processor -- start 2024-04-23T14:57:30.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba60e00640 1 -- start start 2024-04-23T14:57:30.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba60e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5c108610 0x7fba5c19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba60e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba5c108fd0 0x7fba5c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba60e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5c10e660 0x7fba5c1a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba60e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c118a30 con 0x7fba5c10e660 2024-04-23T14:57:30.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba60e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1188b0 con 0x7fba5c108610 2024-04-23T14:57:30.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba60e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c118bb0 con 0x7fba5c108fd0 2024-04-23T14:57:30.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba59e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5c108610 0x7fba5c19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba59e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5c108610 0x7fba5c19eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:39612/0 (socket says 172.21.15.83:39612) 2024-04-23T14:57:30.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba59e00640 1 -- 172.21.15.83:0/3301932858 learned_addr learned my addr 172.21.15.83:0/3301932858 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:30.194 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba5a800640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5c10e660 0x7fba5c1a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.190+0000 7fba51400640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba5c108fd0 0x7fba5c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.191+0000 7fba59e00640 1 -- 172.21.15.83:0/3301932858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba5c108fd0 msgr2=0x7fba5c19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.191+0000 7fba59e00640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba5c108fd0 0x7fba5c19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.191+0000 7fba59e00640 1 -- 172.21.15.83:0/3301932858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5c10e660 msgr2=0x7fba5c1a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.191+0000 7fba59e00640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5c10e660 0x7fba5c1a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.191+0000 7fba59e00640 1 -- 172.21.15.83:0/3301932858 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5c1a3ea0 con 0x7fba5c108610 2024-04-23T14:57:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.191+0000 7fba59e00640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5c108610 0x7fba5c19eeb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fba4c02a960 tx=0x7fba4c003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.191+0000 7fba53400640 1 -- 172.21.15.83:0/3301932858 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c03f070 con 0x7fba5c108610 2024-04-23T14:57:30.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.191+0000 7fba60e00640 1 -- 172.21.15.83:0/3301932858 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5c1a4130 con 0x7fba5c108610 2024-04-23T14:57:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.192+0000 7fba60e00640 1 -- 172.21.15.83:0/3301932858 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5c1aba20 con 0x7fba5c108610 2024-04-23T14:57:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.192+0000 7fba53400640 1 -- 172.21.15.83:0/3301932858 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4c031c90 con 0x7fba5c108610 2024-04-23T14:57:30.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.192+0000 7fba53400640 1 -- 172.21.15.83:0/3301932858 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c049060 con 0x7fba5c108610 2024-04-23T14:57:30.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.195+0000 7fba53400640 1 -- 172.21.15.83:0/3301932858 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba4c042070 con 0x7fba5c108610 2024-04-23T14:57:30.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.195+0000 7fba63da0740 1 Processor -- start 2024-04-23T14:57:30.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.195+0000 7fba63da0740 1 -- start start 2024-04-23T14:57:30.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.195+0000 7fba63da0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f7950 0x55662b2f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.198 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba63da0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f8270 0x55662b302400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba63da0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b302940 0x55662b304d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba63da0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55662b25d1d0 con 0x55662b302940 2024-04-23T14:57:30.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba63da0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55662b203450 con 0x55662b2f7950 2024-04-23T14:57:30.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba63da0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55662b259190 con 0x55662b2f8270 2024-04-23T14:57:30.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba51400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b302940 0x55662b304d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba59e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f8270 0x55662b302400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba51400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b302940 0x55662b304d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37614/0 (socket says 172.21.15.83:37614) 2024-04-23T14:57:30.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba51400640 1 -- 172.21.15.83:0/3737291469 learned_addr learned my addr 172.21.15.83:0/3737291469 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:30.199 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba5a800640 1 --2- 172.21.15.83:0/3737291469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f7950 0x55662b2f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba51400640 1 -- 172.21.15.83:0/3737291469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f8270 msgr2=0x55662b302400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba51400640 1 --2- 172.21.15.83:0/3737291469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f8270 0x55662b302400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba51400640 1 -- 172.21.15.83:0/3737291469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f7950 msgr2=0x55662b2f7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba51400640 1 --2- 172.21.15.83:0/3737291469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f7950 0x55662b2f7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.196+0000 7fba51400640 1 -- 172.21.15.83:0/3737291469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662b2c0310 con 0x55662b302940 2024-04-23T14:57:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.197+0000 7fba53400640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fba400a3da0 0x7fba400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.197+0000 7fba51400640 1 --2- 172.21.15.83:0/3737291469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b302940 0x55662b304d30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fba44009ca0 tx=0x7fba44011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bd98b9d5f76af3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.197+0000 7fba53400640 1 -- 172.21.15.83:0/3301932858 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fba4c0f09e0 con 0x7fba5c108610 2024-04-23T14:57:30.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.197+0000 7fba5a800640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fba400a3da0 0x7fba400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.197+0000 7fba4b400640 1 -- 172.21.15.83:0/3737291469 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4401e070 con 0x55662b302940 2024-04-23T14:57:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.197+0000 7fba4b400640 1 -- 172.21.15.83:0/3737291469 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba44019b10 con 0x55662b302940 2024-04-23T14:57:30.201 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.197+0000 7fba4b400640 1 -- 172.21.15.83:0/3737291469 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4400eae0 con 0x55662b302940 2024-04-23T14:57:30.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.198+0000 7fba63da0740 1 -- 172.21.15.83:0/3737291469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b302940 msgr2=0x55662b304d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:30.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.198+0000 7fba5a800640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fba400a3da0 0x7fba400a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fba54005fd0 tx=0x7fba54009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:30.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.198+0000 7fba63da0740 1 --2- 172.21.15.83:0/3737291469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b302940 0x55662b304d30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fba44009ca0 tx=0x7fba44011810 comp rx=0 tx=0).stop 2024-04-23T14:57:30.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.198+0000 7fba63da0740 1 -- 172.21.15.83:0/3737291469 shutdown_connections 2024-04-23T14:57:30.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.198+0000 7fba63da0740 1 --2- 172.21.15.83:0/3737291469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b302940 0x55662b304d30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.198+0000 7fba63da0740 1 --2- 172.21.15.83:0/3737291469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f8270 0x55662b302400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.198+0000 7fba63da0740 1 --2- 172.21.15.83:0/3737291469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f7950 0x55662b2f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.198+0000 7fba63da0740 1 -- 172.21.15.83:0/3737291469 >> 172.21.15.83:0/3737291469 conn(0x55662b2f3c70 msgr2=0x55662b2f4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:30.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.198+0000 7fba63da0740 1 -- 172.21.15.83:0/3737291469 shutdown_connections 2024-04-23T14:57:30.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.198+0000 7fba63da0740 1 -- 172.21.15.83:0/3737291469 wait complete. 2024-04-23T14:57:30.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.198+0000 7fba63da0740 1 -- start start 2024-04-23T14:57:30.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba63da0740 1 Processor -- start 2024-04-23T14:57:30.206 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba63da0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b2f7950 0x55662b302340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba63da0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f8270 0x55662b2ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba63da0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b302940 0x55662b2ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba63da0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55662b3057d0 con 0x55662b2f7950 2024-04-23T14:57:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba63da0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55662b216810 con 0x55662b2f8270 2024-04-23T14:57:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba63da0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55662b305950 con 0x55662b302940 2024-04-23T14:57:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba63da0740 10 client.0 _ll_register_callbacks cb 0x55662b1e4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:57:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba59e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b2f7950 0x55662b302340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba59e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b2f7950 0x55662b302340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37622/0 (socket says 172.21.15.83:37622) 2024-04-23T14:57:30.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba59e00640 1 -- 172.21.15.83:0/1128827921 learned_addr learned my addr 172.21.15.83:0/1128827921 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:30.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba5a800640 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b302940 0x55662b2ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.199+0000 7fba51400640 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f8270 0x55662b2ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba59e00640 1 -- 172.21.15.83:0/1128827921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b302940 msgr2=0x55662b2ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:30.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba59e00640 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b302940 0x55662b2ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba59e00640 1 -- 172.21.15.83:0/1128827921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f8270 msgr2=0x55662b2ff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:30.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba59e00640 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f8270 0x55662b2ff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba59e00640 1 -- 172.21.15.83:0/1128827921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662b2273f0 con 0x55662b2f7950 2024-04-23T14:57:30.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba59e00640 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b2f7950 0x55662b302340 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0bc6c0 tx=0x7fba4c031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a00d182ef75c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:30.208 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:57:30.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba3fe00640 1 -- 172.21.15.83:0/1128827921 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0bc500 con 0x55662b2f7950 2024-04-23T14:57:30.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba3fe00640 1 -- 172.21.15.83:0/1128827921 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fba4c0ee340 con 0x55662b2f7950 2024-04-23T14:57:30.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba63da0740 1 -- 172.21.15.83:0/1128827921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba44019b10 con 0x55662b2f7950 2024-04-23T14:57:30.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba3fe00640 1 -- 172.21.15.83:0/1128827921 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba4c0f6020 con 0x55662b2f7950 2024-04-23T14:57:30.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba63da0740 10 client.16830 fetch_fsmap learned FSMap version 29 2024-04-23T14:57:30.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.200+0000 7fba63da0740 1 -- 172.21.15.83:0/1128827921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55662af6f3f0 con 0x55662b2f7950 2024-04-23T14:57:30.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.201+0000 7fba3fe00640 1 -- 172.21.15.83:0/1128827921 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba44019b10 con 0x55662b2f7950 2024-04-23T14:57:30.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.201+0000 7fba3fe00640 1 -- 172.21.15.83:0/1128827921 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fba4c0c50f0 con 0x55662b2f7950 2024-04-23T14:57:30.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.201+0000 7fba63da0740 10 client.16830 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:57:30.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.201+0000 7fba63da0740 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:57:30.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.201+0000 7fba63da0740 20 client.16830 populate_metadata read hostname 'smithi083' 2024-04-23T14:57:30.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.201+0000 7fba63da0740 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55662b312af0 0x55662b314f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.201+0000 7fba63da0740 4 client.16830 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:57:30.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.201+0000 7fba63da0740 1 -- 172.21.15.83:0/1128827921 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55662b26abf0 con 0x55662b312af0 2024-04-23T14:57:30.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.201+0000 7fba51400640 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55662b312af0 0x55662b314f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.202+0000 7fba51400640 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55662b312af0 0x55662b314f20 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=510f0b1721075364 server_cookie=7766a8e50c1bc3ea in_seq=0 out_seq=0 2024-04-23T14:57:30.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.202+0000 7fba3fe00640 10 client.16830 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:30.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba3fe00640 1 -- 172.21.15.83:0/1128827921 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55662b26abf0 con 0x55662b312af0 2024-04-23T14:57:30.210 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba3fe00640 10 client.16830 handle_command_reply: tid=0 2024-04-23T14:57:30.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba63da0740 1 client.16830 shutdown 2024-04-23T14:57:30.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba63da0740 1 -- 172.21.15.83:0/1128827921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b2f7950 msgr2=0x55662b302340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:30.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba63da0740 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b2f7950 0x55662b302340 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0bc6c0 tx=0x7fba4c031490 comp rx=0 tx=0).stop 2024-04-23T14:57:30.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba63da0740 1 -- 172.21.15.83:0/1128827921 shutdown_connections 2024-04-23T14:57:30.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba63da0740 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55662b312af0 0x55662b314f20 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba63da0740 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b302940 0x55662b2ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba63da0740 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f8270 0x55662b2ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba3fe00640 0 client.16830 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:30.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba63da0740 1 --2- 172.21.15.83:0/1128827921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b2f7950 0x55662b302340 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba63da0740 1 -- 172.21.15.83:0/1128827921 >> 172.21.15.83:0/1128827921 conn(0x55662b2f3c70 msgr2=0x55662b2f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba63da0740 1 -- 172.21.15.83:0/1128827921 shutdown_connections 2024-04-23T14:57:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba63da0740 1 -- 172.21.15.83:0/1128827921 wait complete. 2024-04-23T14:57:30.212 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.203+0000 7fba63da0740 20 client.16830 trim_cache size 0 max 16384 2024-04-23T14:57:30.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.218+0000 7fba63da0740 1 Processor -- start 2024-04-23T14:57:30.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.218+0000 7fba63da0740 1 -- start start 2024-04-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.218+0000 7fba63da0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 0x55662b30e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.218+0000 7fba63da0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b2f8270 0x55662b301ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.218+0000 7fba63da0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b302940 0x55662b31a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.218+0000 7fba63da0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1b6aa0 con 0x55662b2f8270 2024-04-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.218+0000 7fba63da0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1b4370 con 0x55662b302940 2024-04-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.218+0000 7fba63da0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1b21e0 con 0x55662b2f7950 2024-04-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.219+0000 7fba59e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 0x55662b30e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.219+0000 7fba59e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 0x55662b30e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59318/0 (socket says 172.21.15.83:59318) 2024-04-23T14:57:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.219+0000 7fba59e00640 1 -- 172.21.15.83:0/797747240 learned_addr learned my addr 172.21.15.83:0/797747240 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.219+0000 7fba5a800640 1 --2- 172.21.15.83:0/797747240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b302940 0x55662b31a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.219+0000 7fba51400640 1 --2- 172.21.15.83:0/797747240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b2f8270 0x55662b301ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.219+0000 7fba59e00640 1 -- 172.21.15.83:0/797747240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b302940 msgr2=0x55662b31a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.219+0000 7fba59e00640 1 --2- 172.21.15.83:0/797747240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b302940 0x55662b31a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.219+0000 7fba59e00640 1 -- 172.21.15.83:0/797747240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b2f8270 msgr2=0x55662b301ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.219+0000 7fba59e00640 1 --2- 172.21.15.83:0/797747240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b2f8270 0x55662b301ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.219+0000 7fba59e00640 1 -- 172.21.15.83:0/797747240 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662b0397d0 con 0x55662b2f7950 2024-04-23T14:57:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.219+0000 7fba59e00640 1 --2- 172.21.15.83:0/797747240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 0x55662b30e6c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0b9390 tx=0x7fba4c03b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4edf9ef33afba2be server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.220+0000 7fba4aa00640 1 -- 172.21.15.83:0/797747240 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0bc320 con 0x55662b2f7950 2024-04-23T14:57:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.220+0000 7fba4aa00640 1 -- 172.21.15.83:0/797747240 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4c0a73a0 con 0x55662b2f7950 2024-04-23T14:57:30.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.220+0000 7fba4aa00640 1 -- 172.21.15.83:0/797747240 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0bc9c0 con 0x55662b2f7950 2024-04-23T14:57:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.220+0000 7fba63da0740 1 -- 172.21.15.83:0/797747240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 msgr2=0x55662b30e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.220+0000 7fba63da0740 1 --2- 172.21.15.83:0/797747240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 0x55662b30e6c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0b9390 tx=0x7fba4c03b6b0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.220+0000 7fba63da0740 1 -- 172.21.15.83:0/797747240 shutdown_connections 2024-04-23T14:57:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.220+0000 7fba63da0740 1 --2- 172.21.15.83:0/797747240 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b302940 0x55662b31a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.220+0000 7fba63da0740 1 --2- 172.21.15.83:0/797747240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b2f8270 0x55662b301ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.220+0000 7fba63da0740 1 --2- 172.21.15.83:0/797747240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 0x55662b30e6c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.220+0000 7fba63da0740 1 -- 172.21.15.83:0/797747240 >> 172.21.15.83:0/797747240 conn(0x55662b2f3c70 msgr2=0x55662b2fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.220+0000 7fba63da0740 1 -- 172.21.15.83:0/797747240 shutdown_connections 2024-04-23T14:57:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.220+0000 7fba63da0740 1 -- 172.21.15.83:0/797747240 wait complete. 2024-04-23T14:57:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.221+0000 7fba63da0740 1 -- start start 2024-04-23T14:57:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.221+0000 7fba63da0740 1 Processor -- start 2024-04-23T14:57:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.221+0000 7fba63da0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 0x55662b30add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.221+0000 7fba63da0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f8270 0x55662b305b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.221+0000 7fba63da0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b302940 0x55662b3060d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.221+0000 7fba63da0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1b2830 con 0x55662b302940 2024-04-23T14:57:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.221+0000 7fba63da0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1b3ca0 con 0x55662b2f8270 2024-04-23T14:57:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.221+0000 7fba63da0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fba5c1b2f00 con 0x55662b2f7950 2024-04-23T14:57:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba63da0740 10 client.0 _ll_register_callbacks cb 0x7fba4c0f6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:57:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba59e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 0x55662b30add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba5a800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b302940 0x55662b3060d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba59e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 0x55662b30add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59332/0 (socket says 172.21.15.83:59332) 2024-04-23T14:57:30.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba59e00640 1 -- 172.21.15.83:0/45443438 learned_addr learned my addr 172.21.15.83:0/45443438 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:30.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba51400640 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f8270 0x55662b305b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba59e00640 1 -- 172.21.15.83:0/45443438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f8270 msgr2=0x55662b305b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:30.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba59e00640 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f8270 0x55662b305b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba59e00640 1 -- 172.21.15.83:0/45443438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b302940 msgr2=0x55662b3060d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:30.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba59e00640 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b302940 0x55662b3060d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba59e00640 1 -- 172.21.15.83:0/45443438 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662b306640 con 0x55662b2f7950 2024-04-23T14:57:30.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba51400640 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f8270 0x55662b305b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:57:30.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba59e00640 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 0x55662b30add0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0f0da0 tx=0x7fba4c031490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6313f5765a6203fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.222+0000 7fba3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:57:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba3fe00640 1 -- 172.21.15.83:0/45443438 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0bc320 con 0x55662b2f7950 2024-04-23T14:57:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba63da0740 1 -- 172.21.15.83:0/45443438 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55662b264a60 con 0x55662b2f7950 2024-04-23T14:57:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba3fe00640 1 -- 172.21.15.83:0/45443438 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fba4c0bbe10 con 0x55662b2f7950 2024-04-23T14:57:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba3fe00640 1 -- 172.21.15.83:0/45443438 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba4c0ee540 con 0x55662b2f7950 2024-04-23T14:57:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba63da0740 10 client.27010 fetch_fsmap learned FSMap version 29 2024-04-23T14:57:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba63da0740 1 -- 172.21.15.83:0/45443438 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55662b306640 con 0x55662b2f7950 2024-04-23T14:57:30.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba3fe00640 1 -- 172.21.15.83:0/45443438 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55662b264a60 con 0x55662b2f7950 2024-04-23T14:57:30.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba3fe00640 1 -- 172.21.15.83:0/45443438 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fba4c0b8710 con 0x55662b2f7950 2024-04-23T14:57:30.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba63da0740 10 client.27010 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:57:30.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba63da0740 10 client.27010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:57:30.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba63da0740 20 client.27010 populate_metadata read hostname 'smithi083' 2024-04-23T14:57:30.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba63da0740 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55662b37a0b0 0x55662b37c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:30.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba63da0740 4 client.27010 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:57:30.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.223+0000 7fba63da0740 1 -- 172.21.15.83:0/45443438 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55662b1e04d0 con 0x55662b37a0b0 2024-04-23T14:57:30.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.224+0000 7fba51400640 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55662b37a0b0 0x55662b37c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:30.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.224+0000 7fba51400640 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55662b37a0b0 0x55662b37c4a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d53cd78dc5fb64e server_cookie=97358ae71b90c26b in_seq=0 out_seq=0 2024-04-23T14:57:30.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.224+0000 7fba3fe00640 10 client.27010 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:30.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba3fe00640 1 -- 172.21.15.83:0/45443438 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55662b1e04d0 con 0x55662b37a0b0 2024-04-23T14:57:30.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba3fe00640 10 client.27010 handle_command_reply: tid=0 2024-04-23T14:57:30.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba63da0740 1 client.27010 shutdown 2024-04-23T14:57:30.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba63da0740 1 -- 172.21.15.83:0/45443438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 msgr2=0x55662b30add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:30.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba63da0740 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 0x55662b30add0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0f0da0 tx=0x7fba4c031490 comp rx=0 tx=0).stop 2024-04-23T14:57:30.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba63da0740 1 -- 172.21.15.83:0/45443438 shutdown_connections 2024-04-23T14:57:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba63da0740 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55662b37a0b0 0x55662b37c4a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba63da0740 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55662b302940 0x55662b3060d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba63da0740 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55662b2f8270 0x55662b305b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba63da0740 1 --2- 172.21.15.83:0/45443438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55662b2f7950 0x55662b30add0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba3fe00640 0 client.27010 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba63da0740 1 -- 172.21.15.83:0/45443438 >> 172.21.15.83:0/45443438 conn(0x55662b2f3c70 msgr2=0x55662b2f2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba63da0740 1 -- 172.21.15.83:0/45443438 shutdown_connections 2024-04-23T14:57:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba63da0740 1 -- 172.21.15.83:0/45443438 wait complete. 2024-04-23T14:57:30.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.225+0000 7fba63da0740 20 client.27010 trim_cache size 0 max 16384 2024-04-23T14:57:30.236 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:57:30.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.226+0000 7fba60e00640 1 -- 172.21.15.83:0/3301932858 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fba400a3da0 msgr2=0x7fba400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:30.236 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.226+0000 7fba60e00640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fba400a3da0 0x7fba400a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fba54005fd0 tx=0x7fba54009500 comp rx=0 tx=0).stop 2024-04-23T14:57:30.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.226+0000 7fba60e00640 1 -- 172.21.15.83:0/3301932858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5c108610 msgr2=0x7fba5c19eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:30.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.226+0000 7fba60e00640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5c108610 0x7fba5c19eeb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fba4c02a960 tx=0x7fba4c003b20 comp rx=0 tx=0).stop 2024-04-23T14:57:30.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.226+0000 7fba60e00640 1 -- 172.21.15.83:0/3301932858 shutdown_connections 2024-04-23T14:57:30.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.226+0000 7fba60e00640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fba400a3da0 0x7fba400a6260 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.226+0000 7fba60e00640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba5c10e660 0x7fba5c1a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.226+0000 7fba60e00640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba5c108fd0 0x7fba5c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.226+0000 7fba60e00640 1 --2- 172.21.15.83:0/3301932858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba5c108610 0x7fba5c19eeb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:30.237 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.226+0000 7fba60e00640 1 -- 172.21.15.83:0/3301932858 >> 172.21.15.83:0/3301932858 conn(0x7fba5c076540 msgr2=0x7fba5c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:30.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.226+0000 7fba60e00640 1 -- 172.21.15.83:0/3301932858 shutdown_connections 2024-04-23T14:57:30.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:30.226+0000 7fba60e00640 1 -- 172.21.15.83:0/3301932858 wait complete. 2024-04-23T14:57:31.245 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:57:31.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:57:31.245 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:57:31.458 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.457+0000 7f4f0da00640 1 -- 172.21.15.83:0/3893459291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f08108580 msgr2=0x7f4f08108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.457+0000 7f4f0da00640 1 --2- 172.21.15.83:0/3893459291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f08108580 0x7f4f08108a00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009930 tx=0x7f4ef002a420 comp rx=0 tx=0).stop 2024-04-23T14:57:31.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.457+0000 7f4f0da00640 1 -- 172.21.15.83:0/3893459291 shutdown_connections 2024-04-23T14:57:31.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.457+0000 7f4f0da00640 1 --2- 172.21.15.83:0/3893459291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f08108f40 0x7f4f081153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.457+0000 7f4f0da00640 1 --2- 172.21.15.83:0/3893459291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f08108580 0x7f4f08108a00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.459 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.457+0000 7f4f0da00640 1 --2- 172.21.15.83:0/3893459291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f0810d500 0x7f4f0810d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.457+0000 7f4f0da00640 1 -- 172.21.15.83:0/3893459291 >> 172.21.15.83:0/3893459291 conn(0x7f4f08077e20 msgr2=0x7f4f08106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.457+0000 7f4f0da00640 1 -- 172.21.15.83:0/3893459291 shutdown_connections 2024-04-23T14:57:31.461 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.457+0000 7f4f0da00640 1 -- 172.21.15.83:0/3893459291 wait complete. 2024-04-23T14:57:31.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.458+0000 7f4f0da00640 1 Processor -- start 2024-04-23T14:57:31.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.458+0000 7f4f0da00640 1 -- start start 2024-04-23T14:57:31.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4f0da00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f08108580 0x7f4f0819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4f0da00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f08108f40 0x7f4f0819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.463 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4f0da00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0810d500 0x7f4f081a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4f0da00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081188d0 con 0x7f4f0810d500 2024-04-23T14:57:31.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4f0da00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08118750 con 0x7f4f08108f40 2024-04-23T14:57:31.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4f0da00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08118a50 con 0x7f4f08108580 2024-04-23T14:57:31.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4f0d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0810d500 0x7f4f081a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4eff400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f08108f40 0x7f4f0819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4f07e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f08108580 0x7f4f0819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.464 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4f0d000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0810d500 0x7f4f081a3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37660/0 (socket says 172.21.15.83:37660) 2024-04-23T14:57:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4f07e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f08108580 0x7f4f0819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59342/0 (socket says 172.21.15.83:59342) 2024-04-23T14:57:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4eff400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f08108f40 0x7f4f0819f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:39674/0 (socket says 172.21.15.83:39674) 2024-04-23T14:57:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.459+0000 7f4f0d000640 1 -- 172.21.15.83:0/1813574290 learned_addr learned my addr 172.21.15.83:0/1813574290 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.460+0000 7f4f07e00640 1 -- 172.21.15.83:0/1813574290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f08108f40 msgr2=0x7f4f0819f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.460+0000 7f4f07e00640 1 --2- 172.21.15.83:0/1813574290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f08108f40 0x7f4f0819f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.460+0000 7f4f07e00640 1 -- 172.21.15.83:0/1813574290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0810d500 msgr2=0x7f4f081a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.460+0000 7f4f07e00640 1 --2- 172.21.15.83:0/1813574290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0810d500 0x7f4f081a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.460+0000 7f4f07e00640 1 -- 172.21.15.83:0/1813574290 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f081a3d50 con 0x7f4f08108580 2024-04-23T14:57:31.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.460+0000 7f4f07e00640 1 --2- 172.21.15.83:0/1813574290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f08108580 0x7f4f0819edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800e9c0 tx=0x7f4ef800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.460+0000 7f4f05600640 1 -- 172.21.15.83:0/1813574290 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800ccc0 con 0x7f4f08108580 2024-04-23T14:57:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.460+0000 7f4f0da00640 1 -- 172.21.15.83:0/1813574290 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f081a4040 con 0x7f4f08108580 2024-04-23T14:57:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.460+0000 7f4f05600640 1 -- 172.21.15.83:0/1813574290 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef80035f0 con 0x7f4f08108580 2024-04-23T14:57:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.460+0000 7f4f05600640 1 -- 172.21.15.83:0/1813574290 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800ce60 con 0x7f4f08108580 2024-04-23T14:57:31.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.460+0000 7f4f0da00640 1 -- 172.21.15.83:0/1813574290 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f081ab8d0 con 0x7f4f08108580 2024-04-23T14:57:31.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.463+0000 7f4f11deb740 1 Processor -- start 2024-04-23T14:57:31.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4f11deb740 1 -- start start 2024-04-23T14:57:31.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4f05600640 1 -- 172.21.15.83:0/1813574290 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ef80037a0 con 0x7f4f08108580 2024-04-23T14:57:31.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4f11deb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 0x5635d4854650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4f11deb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d4854b90 0x5635d485ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4f11deb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d485f260 0x5635d4861650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4f11deb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5635d47b9e70 con 0x5635d4854b90 2024-04-23T14:57:31.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4f11deb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5635d4760380 con 0x5635d485f260 2024-04-23T14:57:31.472 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4f11deb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5635d47b5e30 con 0x5635d4854270 2024-04-23T14:57:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4f07e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d4854b90 0x5635d485ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4f0d000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 0x5635d4854650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4f0d000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 0x5635d4854650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59358/0 (socket says 172.21.15.83:59358) 2024-04-23T14:57:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4f0d000640 1 -- 172.21.15.83:0/4105696327 learned_addr learned my addr 172.21.15.83:0/4105696327 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.464+0000 7f4eff400640 1 --2- 172.21.15.83:0/4105696327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d485f260 0x5635d4861650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.465+0000 7f4f0d000640 1 -- 172.21.15.83:0/4105696327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d485f260 msgr2=0x5635d4861650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.465+0000 7f4f0d000640 1 --2- 172.21.15.83:0/4105696327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d485f260 0x5635d4861650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.465+0000 7f4f0d000640 1 -- 172.21.15.83:0/4105696327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d4854b90 msgr2=0x5635d485ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.465+0000 7f4f0d000640 1 --2- 172.21.15.83:0/4105696327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d4854b90 0x5635d485ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.473 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.465+0000 7f4f0d000640 1 -- 172.21.15.83:0/4105696327 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635d45947e0 con 0x5635d4854270 2024-04-23T14:57:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.465+0000 7f4eff400640 1 --2- 172.21.15.83:0/4105696327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d485f260 0x5635d4861650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:57:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.465+0000 7f4f0d000640 1 --2- 172.21.15.83:0/4105696327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 0x5635d4854650 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4f00011570 tx=0x7f4f00011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87813e65ce4c5060 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.465+0000 7f4f05600640 1 --2- 172.21.15.83:0/1813574290 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4eec0a3d20 0x7f4eec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.465+0000 7f4efd600640 1 -- 172.21.15.83:0/4105696327 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0001e070 con 0x5635d4854270 2024-04-23T14:57:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.465+0000 7f4efd600640 1 -- 172.21.15.83:0/4105696327 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0000faa0 con 0x5635d4854270 2024-04-23T14:57:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.465+0000 7f4efd600640 1 -- 172.21.15.83:0/4105696327 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00019cb0 con 0x5635d4854270 2024-04-23T14:57:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.466+0000 7f4f05600640 1 -- 172.21.15.83:0/1813574290 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f4ef80d0d20 con 0x7f4f08108580 2024-04-23T14:57:31.474 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.466+0000 7f4f07e00640 1 --2- 172.21.15.83:0/1813574290 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4eec0a3d20 0x7f4eec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.466+0000 7f4f11deb740 1 -- 172.21.15.83:0/4105696327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 msgr2=0x5635d4854650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.466+0000 7f4f11deb740 1 --2- 172.21.15.83:0/4105696327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 0x5635d4854650 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4f00011570 tx=0x7f4f00011a30 comp rx=0 tx=0).stop 2024-04-23T14:57:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.466+0000 7f4f11deb740 1 -- 172.21.15.83:0/4105696327 shutdown_connections 2024-04-23T14:57:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.466+0000 7f4f11deb740 1 --2- 172.21.15.83:0/4105696327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d485f260 0x5635d4861650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.466+0000 7f4f11deb740 1 --2- 172.21.15.83:0/4105696327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d4854b90 0x5635d485ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.466+0000 7f4f11deb740 1 --2- 172.21.15.83:0/4105696327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 0x5635d4854650 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.466+0000 7f4f11deb740 1 -- 172.21.15.83:0/4105696327 >> 172.21.15.83:0/4105696327 conn(0x5635d4850590 msgr2=0x5635d4850970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.466+0000 7f4f07e00640 1 --2- 172.21.15.83:0/1813574290 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4eec0a3d20 0x7f4eec0a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x5635d4855830 tx=0x7f4ef80d47d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.466+0000 7f4f11deb740 1 -- 172.21.15.83:0/4105696327 shutdown_connections 2024-04-23T14:57:31.475 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.466+0000 7f4f11deb740 1 -- 172.21.15.83:0/4105696327 wait complete. 2024-04-23T14:57:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.467+0000 7f4f11deb740 1 -- start start 2024-04-23T14:57:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.467+0000 7f4f11deb740 1 Processor -- start 2024-04-23T14:57:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.467+0000 7f4f11deb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854270 0x5635d485c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4f11deb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d4854b90 0x5635d485c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4f11deb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d485f260 0x5635d485ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4f11deb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5635d4862050 con 0x5635d4854b90 2024-04-23T14:57:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4f11deb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5635d4773660 con 0x5635d4854270 2024-04-23T14:57:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4f11deb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5635d48621d0 con 0x5635d485f260 2024-04-23T14:57:31.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4f11deb740 10 client.0 _ll_register_callbacks cb 0x5635d4761aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:57:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4eff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d4854b90 0x5635d485c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4eff400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d4854b90 0x5635d485c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37682/0 (socket says 172.21.15.83:37682) 2024-04-23T14:57:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4eff400640 1 -- 172.21.15.83:0/1998310063 learned_addr learned my addr 172.21.15.83:0/1998310063 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4f0d000640 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854270 0x5635d485c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4f0d000640 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d485f260 0x5635d485ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4eff400640 1 -- 172.21.15.83:0/1998310063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d485f260 msgr2=0x5635d485ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4eff400640 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d485f260 0x5635d485ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4eff400640 1 -- 172.21.15.83:0/1998310063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854270 msgr2=0x5635d485c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4eff400640 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854270 0x5635d485c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4eff400640 1 -- 172.21.15.83:0/1998310063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635d45f4f70 con 0x5635d4854b90 2024-04-23T14:57:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.468+0000 7f4f0d000640 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d485f260 0x5635d485ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:57:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.469+0000 7f4eff400640 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d4854b90 0x5635d485c790 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4ef002a940 tx=0x7f4ef00317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4e6f56efab57c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.469+0000 7f4ef6000640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:57:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.469+0000 7f4ef6000640 1 -- 172.21.15.83:0/1998310063 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0035040 con 0x5635d4854b90 2024-04-23T14:57:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.469+0000 7f4ef6000640 1 -- 172.21.15.83:0/1998310063 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f4ef0004240 con 0x5635d4854b90 2024-04-23T14:57:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.469+0000 7f4f11deb740 1 -- 172.21.15.83:0/1998310063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f0000faa0 con 0x5635d4854b90 2024-04-23T14:57:31.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.469+0000 7f4ef6000640 1 -- 172.21.15.83:0/1998310063 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ef0031a10 con 0x5635d4854b90 2024-04-23T14:57:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.469+0000 7f4f11deb740 10 client.16845 fetch_fsmap learned FSMap version 29 2024-04-23T14:57:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.469+0000 7f4f11deb740 1 -- 172.21.15.83:0/1998310063 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5635d448c170 con 0x5635d4854b90 2024-04-23T14:57:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.470+0000 7f4ef6000640 1 -- 172.21.15.83:0/1998310063 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0000faa0 con 0x5635d4854b90 2024-04-23T14:57:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.470+0000 7f4ef6000640 1 -- 172.21.15.83:0/1998310063 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f4ef00466e0 con 0x5635d4854b90 2024-04-23T14:57:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.470+0000 7f4f11deb740 10 client.16845 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:57:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.470+0000 7f4f11deb740 10 client.16845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:57:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.470+0000 7f4f11deb740 20 client.16845 populate_metadata read hostname 'smithi083' 2024-04-23T14:57:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.470+0000 7f4f11deb740 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5635d486f640 0x5635d4871a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.470+0000 7f4f11deb740 4 client.16845 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:57:31.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.470+0000 7f4f11deb740 1 -- 172.21.15.83:0/1998310063 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635d47c7890 con 0x5635d486f640 2024-04-23T14:57:31.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.470+0000 7f4f0d000640 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5635d486f640 0x5635d4871a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.471+0000 7f4f0d000640 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5635d486f640 0x5635d4871a30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c3fb3568330e232 server_cookie=559af1b188791037 in_seq=0 out_seq=0 2024-04-23T14:57:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.471+0000 7f4ef6000640 10 client.16845 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.472+0000 7f4ef6000640 1 -- 172.21.15.83:0/1998310063 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635d47c7890 con 0x5635d486f640 2024-04-23T14:57:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.472+0000 7f4ef6000640 10 client.16845 handle_command_reply: tid=0 2024-04-23T14:57:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.472+0000 7f4f11deb740 1 client.16845 shutdown 2024-04-23T14:57:31.481 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.472+0000 7f4f11deb740 1 -- 172.21.15.83:0/1998310063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d4854b90 msgr2=0x5635d485c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.472+0000 7f4f11deb740 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d4854b90 0x5635d485c790 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4ef002a940 tx=0x7f4ef00317e0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.472+0000 7f4f11deb740 1 -- 172.21.15.83:0/1998310063 shutdown_connections 2024-04-23T14:57:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.472+0000 7f4f11deb740 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5635d486f640 0x5635d4871a30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.472+0000 7f4f11deb740 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d485f260 0x5635d485ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.472+0000 7f4f11deb740 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d4854b90 0x5635d485c790 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.472+0000 7f4f11deb740 1 --2- 172.21.15.83:0/1998310063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854270 0x5635d485c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.472+0000 7f4ef6000640 0 client.16845 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.472+0000 7f4f11deb740 1 -- 172.21.15.83:0/1998310063 >> 172.21.15.83:0/1998310063 conn(0x5635d4850590 msgr2=0x5635d4856390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:31.482 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.473+0000 7f4f11deb740 1 -- 172.21.15.83:0/1998310063 shutdown_connections 2024-04-23T14:57:31.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.473+0000 7f4f11deb740 1 -- 172.21.15.83:0/1998310063 wait complete. 2024-04-23T14:57:31.483 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.473+0000 7f4f11deb740 20 client.16845 trim_cache size 0 max 16384 2024-04-23T14:57:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.492+0000 7f4f11deb740 1 Processor -- start 2024-04-23T14:57:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.492+0000 7f4f11deb740 1 -- start start 2024-04-23T14:57:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.492+0000 7f4f11deb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 0x5635d486af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.492+0000 7f4f11deb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854b90 0x5635d4877320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.492+0000 7f4f11deb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d485f260 0x5635d4877860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4f11deb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081b6a50 con 0x5635d485f260 2024-04-23T14:57:31.494 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4f11deb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081b4320 con 0x5635d4854b90 2024-04-23T14:57:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4f11deb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081b2190 con 0x5635d4854270 2024-04-23T14:57:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4eff400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854b90 0x5635d4877320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4eff400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854b90 0x5635d4877320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:39700/0 (socket says 172.21.15.83:39700) 2024-04-23T14:57:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4eff400640 1 -- 172.21.15.83:0/3098043502 learned_addr learned my addr 172.21.15.83:0/3098043502 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4f0d000640 1 --2- 172.21.15.83:0/3098043502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 0x5635d486af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.495 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4f0d000640 1 --2- 172.21.15.83:0/3098043502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d485f260 0x5635d4877860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4f0d000640 1 -- 172.21.15.83:0/3098043502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854b90 msgr2=0x5635d4877320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4f0d000640 1 --2- 172.21.15.83:0/3098043502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854b90 0x5635d4877320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.499 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4f0d000640 1 -- 172.21.15.83:0/3098043502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d485f260 msgr2=0x5635d4877860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4f0d000640 1 --2- 172.21.15.83:0/3098043502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d485f260 0x5635d4877860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.493+0000 7f4f0d000640 1 -- 172.21.15.83:0/3098043502 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635d48549c0 con 0x5635d4854270 2024-04-23T14:57:31.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.494+0000 7f4f0d000640 1 --2- 172.21.15.83:0/3098043502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 0x5635d486af10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4f00011540 tx=0x7f4f000198b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d78337727157756 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:31.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.494+0000 7f4efcc00640 1 -- 172.21.15.83:0/3098043502 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00030070 con 0x5635d4854270 2024-04-23T14:57:31.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.494+0000 7f4efcc00640 1 -- 172.21.15.83:0/3098043502 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0001f040 con 0x5635d4854270 2024-04-23T14:57:31.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.494+0000 7f4efcc00640 1 -- 172.21.15.83:0/3098043502 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0001e030 con 0x5635d4854270 2024-04-23T14:57:31.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.495+0000 7f4f11deb740 1 -- 172.21.15.83:0/3098043502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 msgr2=0x5635d486af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.495+0000 7f4f11deb740 1 --2- 172.21.15.83:0/3098043502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 0x5635d486af10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4f00011540 tx=0x7f4f000198b0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.495+0000 7f4f11deb740 1 -- 172.21.15.83:0/3098043502 shutdown_connections 2024-04-23T14:57:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.495+0000 7f4f11deb740 1 --2- 172.21.15.83:0/3098043502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d485f260 0x5635d4877860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.495+0000 7f4f11deb740 1 --2- 172.21.15.83:0/3098043502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854b90 0x5635d4877320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.495+0000 7f4f11deb740 1 --2- 172.21.15.83:0/3098043502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854270 0x5635d486af10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.495+0000 7f4f11deb740 1 -- 172.21.15.83:0/3098043502 >> 172.21.15.83:0/3098043502 conn(0x5635d4850590 msgr2=0x5635d485caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.495+0000 7f4f11deb740 1 -- 172.21.15.83:0/3098043502 shutdown_connections 2024-04-23T14:57:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.495+0000 7f4f11deb740 1 -- 172.21.15.83:0/3098043502 wait complete. 2024-04-23T14:57:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.495+0000 7f4f11deb740 1 -- start start 2024-04-23T14:57:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.495+0000 7f4f11deb740 1 Processor -- start 2024-04-23T14:57:31.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4f11deb740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854270 0x5635d4863470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4f11deb740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854b90 0x5635d48639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4f11deb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d485f260 0x5635d48d4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4f11deb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081b27e0 con 0x5635d485f260 2024-04-23T14:57:31.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4f11deb740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081b3c50 con 0x5635d4854270 2024-04-23T14:57:31.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4f11deb740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081b2eb0 con 0x5635d4854b90 2024-04-23T14:57:31.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4f11deb740 10 client.0 _ll_register_callbacks cb 0x7f4ef0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:57:31.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4eff400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854b90 0x5635d48639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.502 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4eff400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854b90 0x5635d48639b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59378/0 (socket says 172.21.15.83:59378) 2024-04-23T14:57:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4eff400640 1 -- 172.21.15.83:0/507070497 learned_addr learned my addr 172.21.15.83:0/507070497 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4f0d000640 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d485f260 0x5635d48d4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4f0d000640 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854270 0x5635d4863470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4eff400640 1 -- 172.21.15.83:0/507070497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854270 msgr2=0x5635d4863470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4eff400640 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854270 0x5635d4863470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4eff400640 1 -- 172.21.15.83:0/507070497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d485f260 msgr2=0x5635d48d4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4eff400640 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d485f260 0x5635d48d4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.496+0000 7f4eff400640 1 -- 172.21.15.83:0/507070497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635d48d50d0 con 0x5635d4854b90 2024-04-23T14:57:31.503 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.497+0000 7f4f0d000640 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854270 0x5635d4863470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:57:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.497+0000 7f4eff400640 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854b90 0x5635d48639b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0005940 tx=0x7f4ef00466b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cc040b350b10b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.497+0000 7f4ef6000640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:57:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.497+0000 7f4ef6000640 1 -- 172.21.15.83:0/507070497 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0035070 con 0x5635d4854b90 2024-04-23T14:57:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.497+0000 7f4ef6000640 1 -- 172.21.15.83:0/507070497 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f4ef0005230 con 0x5635d4854b90 2024-04-23T14:57:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.497+0000 7f4f11deb740 1 -- 172.21.15.83:0/507070497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f0001f040 con 0x5635d4854b90 2024-04-23T14:57:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.497+0000 7f4ef6000640 1 -- 172.21.15.83:0/507070497 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ef0053070 con 0x5635d4854b90 2024-04-23T14:57:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.497+0000 7f4f11deb740 10 client.27028 fetch_fsmap learned FSMap version 29 2024-04-23T14:57:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.497+0000 7f4f11deb740 1 -- 172.21.15.83:0/507070497 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5635d48d6310 con 0x5635d4854b90 2024-04-23T14:57:31.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.498+0000 7f4ef6000640 1 -- 172.21.15.83:0/507070497 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0001f040 con 0x5635d4854b90 2024-04-23T14:57:31.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.498+0000 7f4ef6000640 1 -- 172.21.15.83:0/507070497 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f4ef00594b0 con 0x5635d4854b90 2024-04-23T14:57:31.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.498+0000 7f4f11deb740 10 client.27028 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:57:31.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.498+0000 7f4f11deb740 10 client.27028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:57:31.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.498+0000 7f4f11deb740 20 client.27028 populate_metadata read hostname 'smithi083' 2024-04-23T14:57:31.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.498+0000 7f4f11deb740 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5635d48d6a90 0x5635d48d8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.498+0000 7f4f11deb740 4 client.27028 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:57:31.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.498+0000 7f4f11deb740 1 -- 172.21.15.83:0/507070497 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5635d47c1700 con 0x5635d48d6a90 2024-04-23T14:57:31.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.498+0000 7f4f0d000640 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5635d48d6a90 0x5635d48d8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.499+0000 7f4ef6000640 10 client.27028 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:31.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.499+0000 7f4f0d000640 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5635d48d6a90 0x5635d48d8eb0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4081205a38e2b0c7 server_cookie=2bb078cfeed9a36 in_seq=0 out_seq=0 2024-04-23T14:57:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4ef6000640 1 -- 172.21.15.83:0/507070497 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5635d47c1700 con 0x5635d48d6a90 2024-04-23T14:57:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4ef6000640 10 client.27028 handle_command_reply: tid=0 2024-04-23T14:57:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4f11deb740 1 client.27028 shutdown 2024-04-23T14:57:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4f11deb740 1 -- 172.21.15.83:0/507070497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854b90 msgr2=0x5635d48639b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4f11deb740 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854b90 0x5635d48639b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0005940 tx=0x7f4ef00466b0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4f11deb740 1 -- 172.21.15.83:0/507070497 shutdown_connections 2024-04-23T14:57:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4f11deb740 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5635d48d6a90 0x5635d48d8eb0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4f11deb740 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5635d485f260 0x5635d48d4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4f11deb740 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5635d4854b90 0x5635d48639b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4ef6000640 0 client.27028 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4f11deb740 1 --2- 172.21.15.83:0/507070497 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5635d4854270 0x5635d4863470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4f11deb740 1 -- 172.21.15.83:0/507070497 >> 172.21.15.83:0/507070497 conn(0x5635d4850590 msgr2=0x5635d484ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:31.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4f11deb740 1 -- 172.21.15.83:0/507070497 shutdown_connections 2024-04-23T14:57:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4f11deb740 1 -- 172.21.15.83:0/507070497 wait complete. 2024-04-23T14:57:31.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.500+0000 7f4f11deb740 20 client.27028 trim_cache size 0 max 16384 2024-04-23T14:57:31.509 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:57:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:57:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "7387dfb5-9c1e-43a9-b907-f51e9199f5e2", 2024-04-23T14:57:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:57:31.510 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:57:31.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.501+0000 7f4f0da00640 1 -- 172.21.15.83:0/1813574290 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4eec0a3d20 msgr2=0x7f4eec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.501+0000 7f4f0da00640 1 --2- 172.21.15.83:0/1813574290 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4eec0a3d20 0x7f4eec0a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x5635d4855830 tx=0x7f4ef80d47d0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.501+0000 7f4f0da00640 1 -- 172.21.15.83:0/1813574290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f08108580 msgr2=0x7f4f0819edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.501+0000 7f4f0da00640 1 --2- 172.21.15.83:0/1813574290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f08108580 0x7f4f0819edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800e9c0 tx=0x7f4ef800ee80 comp rx=0 tx=0).stop 2024-04-23T14:57:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.501+0000 7f4f0da00640 1 -- 172.21.15.83:0/1813574290 shutdown_connections 2024-04-23T14:57:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.501+0000 7f4f0da00640 1 --2- 172.21.15.83:0/1813574290 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4eec0a3d20 0x7f4eec0a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.501+0000 7f4f0da00640 1 --2- 172.21.15.83:0/1813574290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0810d500 0x7f4f081a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.501+0000 7f4f0da00640 1 --2- 172.21.15.83:0/1813574290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f08108f40 0x7f4f0819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.501+0000 7f4f0da00640 1 --2- 172.21.15.83:0/1813574290 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f08108580 0x7f4f0819edb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.501+0000 7f4f0da00640 1 -- 172.21.15.83:0/1813574290 >> 172.21.15.83:0/1813574290 conn(0x7f4f08077e20 msgr2=0x7f4f080783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:31.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.501+0000 7f4f0da00640 1 -- 172.21.15.83:0/1813574290 shutdown_connections 2024-04-23T14:57:31.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.501+0000 7f4f0da00640 1 -- 172.21.15.83:0/1813574290 wait complete. 2024-04-23T14:57:31.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:57:31.522 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:57:31.583 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:31 smithi083 ceph-mon[27867]: pgmap v281: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:57:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.712+0000 7f7937000640 1 -- 172.21.15.83:0/545846131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7930108fd0 msgr2=0x7f7930115490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.712+0000 7f7937000640 1 --2- 172.21.15.83:0/545846131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7930108fd0 0x7f7930115490 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7928009cd0 tx=0x7f792802a8e0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.712+0000 7f7937000640 1 -- 172.21.15.83:0/545846131 shutdown_connections 2024-04-23T14:57:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.712+0000 7f7937000640 1 --2- 172.21.15.83:0/545846131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7930108fd0 0x7f7930115490 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.712+0000 7f7937000640 1 --2- 172.21.15.83:0/545846131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7930108610 0x7f7930108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.712+0000 7f7937000640 1 --2- 172.21.15.83:0/545846131 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f793010e660 0x7f793010ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.712+0000 7f7937000640 1 -- 172.21.15.83:0/545846131 >> 172.21.15.83:0/545846131 conn(0x7f7930076540 msgr2=0x7f7930078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.712+0000 7f7937000640 1 -- 172.21.15.83:0/545846131 shutdown_connections 2024-04-23T14:57:31.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.712+0000 7f7937000640 1 -- 172.21.15.83:0/545846131 wait complete. 2024-04-23T14:57:31.716 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.713+0000 7f7937000640 1 Processor -- start 2024-04-23T14:57:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.713+0000 7f7937000640 1 -- start start 2024-04-23T14:57:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f7937000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7930108610 0x7f793019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f7937000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7930108fd0 0x7f793019f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f7937000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f793010e660 0x7f79301a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f7937000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79301189f0 con 0x7f7930108610 2024-04-23T14:57:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f7937000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7930118870 con 0x7f793010e660 2024-04-23T14:57:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f7937000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7930118b70 con 0x7f7930108fd0 2024-04-23T14:57:31.717 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f792fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7930108610 0x7f793019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f792fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7930108610 0x7f793019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37732/0 (socket says 172.21.15.83:37732) 2024-04-23T14:57:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f792fe00640 1 -- 172.21.15.83:0/1009501756 learned_addr learned my addr 172.21.15.83:0/1009501756 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f7935000640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f793010e660 0x7f79301a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f7927400640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7930108fd0 0x7f793019f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f792fe00640 1 -- 172.21.15.83:0/1009501756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7930108fd0 msgr2=0x7f793019f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f792fe00640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7930108fd0 0x7f793019f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f792fe00640 1 -- 172.21.15.83:0/1009501756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f793010e660 msgr2=0x7f79301a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f792fe00640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f793010e660 0x7f79301a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.718 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.714+0000 7f792fe00640 1 -- 172.21.15.83:0/1009501756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79301a3e00 con 0x7f7930108610 2024-04-23T14:57:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.715+0000 7f792fe00640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7930108610 0x7f793019ee10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f792000e9c0 tx=0x7f792000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.715+0000 7f792d600640 1 -- 172.21.15.83:0/1009501756 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792000cde0 con 0x7f7930108610 2024-04-23T14:57:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.715+0000 7f792d600640 1 -- 172.21.15.83:0/1009501756 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7920010910 con 0x7f7930108610 2024-04-23T14:57:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.715+0000 7f7937000640 1 -- 172.21.15.83:0/1009501756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79301a40f0 con 0x7f7930108610 2024-04-23T14:57:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.715+0000 7f792d600640 1 -- 172.21.15.83:0/1009501756 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920018a00 con 0x7f7930108610 2024-04-23T14:57:31.719 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.715+0000 7f7937000640 1 -- 172.21.15.83:0/1009501756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79301ab980 con 0x7f7930108610 2024-04-23T14:57:31.721 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.718+0000 7f7939f97740 1 Processor -- start 2024-04-23T14:57:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.718+0000 7f792d600640 1 -- 172.21.15.83:0/1009501756 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f792001f020 con 0x7f7930108610 2024-04-23T14:57:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.719+0000 7f7939f97740 1 -- start start 2024-04-23T14:57:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.719+0000 7f7939f97740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dfef860 0x562b1dfefc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.719+0000 7f7939f97740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dffa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.719+0000 7f7939f97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 0x562b1dffcc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.719+0000 7f7939f97740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562b1df551f0 con 0x562b1dff0180 2024-04-23T14:57:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.719+0000 7f7939f97740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562b1defb350 con 0x562b1dfef860 2024-04-23T14:57:31.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.719+0000 7f7939f97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562b1df511b0 con 0x562b1dffa850 2024-04-23T14:57:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.719+0000 7f792fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dffa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.719+0000 7f792fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dffa310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37744/0 (socket says 172.21.15.83:37744) 2024-04-23T14:57:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.719+0000 7f792fe00640 1 -- 172.21.15.83:0/3845550542 learned_addr learned my addr 172.21.15.83:0/3845550542 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.719+0000 7f7927400640 1 --2- 172.21.15.83:0/3845550542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 0x562b1dffcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.719+0000 7f7935000640 1 --2- 172.21.15.83:0/3845550542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dfef860 0x562b1dfefc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.720+0000 7f7935000640 1 -- 172.21.15.83:0/3845550542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 msgr2=0x562b1dffcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.720+0000 7f7935000640 1 --2- 172.21.15.83:0/3845550542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 0x562b1dffcc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.720+0000 7f7935000640 1 -- 172.21.15.83:0/3845550542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 msgr2=0x562b1dffa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.720+0000 7f7935000640 1 --2- 172.21.15.83:0/3845550542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dffa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.720+0000 7f7935000640 1 -- 172.21.15.83:0/3845550542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b1df4ff50 con 0x562b1dfef860 2024-04-23T14:57:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.720+0000 7f7927400640 1 --2- 172.21.15.83:0/3845550542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 0x562b1dffcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:57:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.720+0000 7f792d600640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79140a3d20 0x7f79140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.720+0000 7f7935000640 1 --2- 172.21.15.83:0/3845550542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dfef860 0x562b1dfefc40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7928009ca0 tx=0x7f7928009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=115ebfd4818d8465 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.720+0000 7f792d600640 1 -- 172.21.15.83:0/1009501756 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f79200d0a30 con 0x7f7930108610 2024-04-23T14:57:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.720+0000 7f792fe00640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79140a3d20 0x7f79140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.721+0000 7f7925600640 1 -- 172.21.15.83:0/3845550542 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928049070 con 0x562b1dfef860 2024-04-23T14:57:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.721+0000 7f7925600640 1 -- 172.21.15.83:0/3845550542 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7928036040 con 0x562b1dfef860 2024-04-23T14:57:31.724 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.721+0000 7f7925600640 1 -- 172.21.15.83:0/3845550542 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928004040 con 0x562b1dfef860 2024-04-23T14:57:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.721+0000 7f7939f97740 1 -- 172.21.15.83:0/3845550542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dfef860 msgr2=0x562b1dfefc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.721+0000 7f792fe00640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79140a3d20 0x7f79140a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7920000c00 tx=0x7f79200d44e0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.721+0000 7f7939f97740 1 --2- 172.21.15.83:0/3845550542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dfef860 0x562b1dfefc40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7928009ca0 tx=0x7f7928009a20 comp rx=0 tx=0).stop 2024-04-23T14:57:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.722+0000 7f7939f97740 1 -- 172.21.15.83:0/3845550542 shutdown_connections 2024-04-23T14:57:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.722+0000 7f7939f97740 1 --2- 172.21.15.83:0/3845550542 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 0x562b1dffcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.722+0000 7f7939f97740 1 --2- 172.21.15.83:0/3845550542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dffa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.722+0000 7f7939f97740 1 --2- 172.21.15.83:0/3845550542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dfef860 0x562b1dfefc40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.722+0000 7f7939f97740 1 -- 172.21.15.83:0/3845550542 >> 172.21.15.83:0/3845550542 conn(0x562b1dfebb80 msgr2=0x562b1dfebf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:31.729 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.722+0000 7f7939f97740 1 -- 172.21.15.83:0/3845550542 shutdown_connections 2024-04-23T14:57:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.722+0000 7f7939f97740 1 -- 172.21.15.83:0/3845550542 wait complete. 2024-04-23T14:57:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.722+0000 7f7939f97740 1 -- start start 2024-04-23T14:57:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.722+0000 7f7939f97740 1 Processor -- start 2024-04-23T14:57:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7939f97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dfef860 0x562b1dff7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7939f97740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dff7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7939f97740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dffa850 0x562b1dff82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7939f97740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562b1dffd5e0 con 0x562b1dff0180 2024-04-23T14:57:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7939f97740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562b1df0e690 con 0x562b1dffa850 2024-04-23T14:57:31.730 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7939f97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562b1dffd760 con 0x562b1dfef860 2024-04-23T14:57:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7935000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dfef860 0x562b1dff7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7935000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dfef860 0x562b1dff7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59408/0 (socket says 172.21.15.83:59408) 2024-04-23T14:57:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7935000640 1 -- 172.21.15.83:0/1608768990 learned_addr learned my addr 172.21.15.83:0/1608768990 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7927400640 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dff7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7935000640 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dffa850 0x562b1dff82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7939f97740 10 client.0 _ll_register_callbacks cb 0x562b1df4a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:57:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7935000640 1 -- 172.21.15.83:0/1608768990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dffa850 msgr2=0x562b1dff82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7935000640 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dffa850 0x562b1dff82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.731 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7935000640 1 -- 172.21.15.83:0/1608768990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 msgr2=0x562b1dff7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7935000640 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dff7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.723+0000 7f7935000640 1 -- 172.21.15.83:0/1608768990 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b1df0d920 con 0x562b1dfef860 2024-04-23T14:57:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.724+0000 7f7935000640 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dfef860 0x562b1dff7820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7928003cc0 tx=0x7f7928004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a1dfab163f4aed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.724+0000 7f791e000640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:57:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.724+0000 7f791e000640 1 -- 172.21.15.83:0/1608768990 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928056060 con 0x562b1dfef860 2024-04-23T14:57:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.724+0000 7f7939f97740 1 -- 172.21.15.83:0/1608768990 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7928036040 con 0x562b1dfef860 2024-04-23T14:57:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.724+0000 7f791e000640 1 -- 172.21.15.83:0/1608768990 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f7928031b90 con 0x562b1dfef860 2024-04-23T14:57:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.724+0000 7f791e000640 1 -- 172.21.15.83:0/1608768990 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7928049070 con 0x562b1dfef860 2024-04-23T14:57:31.732 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.724+0000 7f7939f97740 10 client.27040 fetch_fsmap learned FSMap version 29 2024-04-23T14:57:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.724+0000 7f7939f97740 1 -- 172.21.15.83:0/1608768990 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b1dd317d0 con 0x562b1dfef860 2024-04-23T14:57:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.724+0000 7f791e000640 1 -- 172.21.15.83:0/1608768990 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7928036040 con 0x562b1dfef860 2024-04-23T14:57:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.725+0000 7f791e000640 1 -- 172.21.15.83:0/1608768990 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f7928030070 con 0x562b1dfef860 2024-04-23T14:57:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.725+0000 7f7939f97740 10 client.27040 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:57:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.725+0000 7f7939f97740 10 client.27040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:57:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.725+0000 7f7939f97740 20 client.27040 populate_metadata read hostname 'smithi083' 2024-04-23T14:57:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.725+0000 7f7939f97740 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562b1e00aa40 0x562b1e00ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.725+0000 7f7939f97740 4 client.27040 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:57:31.733 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.725+0000 7f7939f97740 1 -- 172.21.15.83:0/1608768990 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b1df62c10 con 0x562b1e00aa40 2024-04-23T14:57:31.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.725+0000 7f7927400640 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562b1e00aa40 0x562b1e00ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.726+0000 7f7927400640 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562b1e00aa40 0x562b1e00ce30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=274b07c3c0a9f49d server_cookie=b1d329aaf55f5006 in_seq=0 out_seq=0 2024-04-23T14:57:31.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.726+0000 7f791e000640 10 client.27040 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:31.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.726+0000 7f791e000640 1 -- 172.21.15.83:0/1608768990 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b1df62c10 con 0x562b1e00aa40 2024-04-23T14:57:31.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.726+0000 7f791e000640 10 client.27040 handle_command_reply: tid=0 2024-04-23T14:57:31.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.726+0000 7f7939f97740 1 client.27040 shutdown 2024-04-23T14:57:31.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.727+0000 7f7939f97740 1 -- 172.21.15.83:0/1608768990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dfef860 msgr2=0x562b1dff7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.734 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.727+0000 7f7939f97740 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dfef860 0x562b1dff7820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7928003cc0 tx=0x7f7928004a40 comp rx=0 tx=0).stop 2024-04-23T14:57:31.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.727+0000 7f7939f97740 1 -- 172.21.15.83:0/1608768990 shutdown_connections 2024-04-23T14:57:31.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.727+0000 7f7939f97740 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562b1e00aa40 0x562b1e00ce30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.727+0000 7f7939f97740 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dffa850 0x562b1dff82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.727+0000 7f7939f97740 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dff7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.727+0000 7f791e000640 0 client.27040 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:31.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.727+0000 7f7939f97740 1 --2- 172.21.15.83:0/1608768990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dfef860 0x562b1dff7820 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.727+0000 7f7939f97740 1 -- 172.21.15.83:0/1608768990 >> 172.21.15.83:0/1608768990 conn(0x562b1dfebb80 msgr2=0x562b1dff1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:31.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.727+0000 7f7939f97740 1 -- 172.21.15.83:0/1608768990 shutdown_connections 2024-04-23T14:57:31.735 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.727+0000 7f7939f97740 1 -- 172.21.15.83:0/1608768990 wait complete. 2024-04-23T14:57:31.736 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.727+0000 7f7939f97740 20 client.27040 trim_cache size 0 max 16384 2024-04-23T14:57:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.746+0000 7f7939f97740 1 Processor -- start 2024-04-23T14:57:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.746+0000 7f7939f97740 1 -- start start 2024-04-23T14:57:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.746+0000 7f7939f97740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dfef860 0x562b1e006270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.746+0000 7f7939f97740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dff1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.746+0000 7f7939f97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 0x562b1dff7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.746+0000 7f7939f97740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79301b6b80 con 0x562b1dff0180 2024-04-23T14:57:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.746+0000 7f7939f97740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f79301b4450 con 0x562b1dfef860 2024-04-23T14:57:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.746+0000 7f7939f97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79301b22c0 con 0x562b1dffa850 2024-04-23T14:57:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7935000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 0x562b1dff7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.749 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7935000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 0x562b1dff7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:59412/0 (socket says 172.21.15.83:59412) 2024-04-23T14:57:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7935000640 1 -- 172.21.15.83:0/3157391449 learned_addr learned my addr 172.21.15.83:0/3157391449 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7927400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dff1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7935000640 1 --2- 172.21.15.83:0/3157391449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dfef860 0x562b1e006270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7935000640 1 -- 172.21.15.83:0/3157391449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dfef860 msgr2=0x562b1e006270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7935000640 1 --2- 172.21.15.83:0/3157391449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dfef860 0x562b1e006270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7935000640 1 -- 172.21.15.83:0/3157391449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 msgr2=0x562b1dff1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7935000640 1 --2- 172.21.15.83:0/3157391449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dff1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7935000640 1 -- 172.21.15.83:0/3157391449 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b1dfeffb0 con 0x562b1dffa850 2024-04-23T14:57:31.750 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7935000640 1 --2- 172.21.15.83:0/3157391449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 0x562b1dff7ef0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7928002230 tx=0x7f7928003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d823d11bce5f344a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7924c00640 1 -- 172.21.15.83:0/3157391449 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928056040 con 0x562b1dffa850 2024-04-23T14:57:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7924c00640 1 -- 172.21.15.83:0/3157391449 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7928036070 con 0x562b1dffa850 2024-04-23T14:57:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.747+0000 7f7924c00640 1 -- 172.21.15.83:0/3157391449 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928004da0 con 0x562b1dffa850 2024-04-23T14:57:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.748+0000 7f7939f97740 1 -- 172.21.15.83:0/3157391449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 msgr2=0x562b1dff7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.748+0000 7f7939f97740 1 --2- 172.21.15.83:0/3157391449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 0x562b1dff7ef0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7928002230 tx=0x7f7928003b80 comp rx=0 tx=0).stop 2024-04-23T14:57:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.748+0000 7f7939f97740 1 -- 172.21.15.83:0/3157391449 shutdown_connections 2024-04-23T14:57:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.748+0000 7f7939f97740 1 --2- 172.21.15.83:0/3157391449 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dffa850 0x562b1dff7ef0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.748+0000 7f7939f97740 1 --2- 172.21.15.83:0/3157391449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dff0180 0x562b1dff1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.748+0000 7f7939f97740 1 --2- 172.21.15.83:0/3157391449 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dfef860 0x562b1e006270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.748+0000 7f7939f97740 1 -- 172.21.15.83:0/3157391449 >> 172.21.15.83:0/3157391449 conn(0x562b1dfebb80 msgr2=0x562b1dff77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.748+0000 7f7939f97740 1 -- 172.21.15.83:0/3157391449 shutdown_connections 2024-04-23T14:57:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.748+0000 7f7939f97740 1 -- 172.21.15.83:0/3157391449 wait complete. 2024-04-23T14:57:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.749+0000 7f7939f97740 1 -- start start 2024-04-23T14:57:31.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.749+0000 7f7939f97740 1 Processor -- start 2024-04-23T14:57:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.749+0000 7f7939f97740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dfef860 0x562b1e010060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.749+0000 7f7939f97740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dff0180 0x562b1e00d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.749+0000 7f7939f97740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dffa850 0x562b1e00d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.749+0000 7f7939f97740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79301b2910 con 0x562b1dffa850 2024-04-23T14:57:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.749+0000 7f7939f97740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f79301b3d80 con 0x562b1dff0180 2024-04-23T14:57:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.749+0000 7f7939f97740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f79301b2fe0 con 0x562b1dfef860 2024-04-23T14:57:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.749+0000 7f7939f97740 10 client.0 _ll_register_callbacks cb 0x7f7928047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:57:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.750+0000 7f7935000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dffa850 0x562b1e00d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.750+0000 7f7927400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dff0180 0x562b1e00d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.756 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.750+0000 7f7935000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dffa850 0x562b1e00d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:37776/0 (socket says 172.21.15.83:37776) 2024-04-23T14:57:31.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.750+0000 7f7935000640 1 -- 172.21.15.83:0/1826803905 learned_addr learned my addr 172.21.15.83:0/1826803905 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:57:31.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.750+0000 7f7935000640 1 --2- 172.21.15.83:0/1826803905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dfef860 0x562b1e010060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.750+0000 7f7927400640 1 -- 172.21.15.83:0/1826803905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dfef860 msgr2=0x562b1e010060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.750+0000 7f7927400640 1 --2- 172.21.15.83:0/1826803905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dfef860 0x562b1e010060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.750+0000 7f7927400640 1 -- 172.21.15.83:0/1826803905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dffa850 msgr2=0x562b1e00d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:57:31.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.750+0000 7f7927400640 1 --2- 172.21.15.83:0/1826803905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dffa850 0x562b1e00d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.750+0000 7f7927400640 1 -- 172.21.15.83:0/1826803905 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b1dc673f0 con 0x562b1dff0180 2024-04-23T14:57:31.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.750+0000 7f7927400640 1 --2- 172.21.15.83:0/1826803905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dff0180 0x562b1e00d440 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x562b1dffd8b0 tx=0x7f7918004100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc46ac12367e553a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:57:31.757 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.750+0000 7f791e000640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:57:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f791e000640 1 -- 172.21.15.83:0/1826803905 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918024030 con 0x562b1dff0180 2024-04-23T14:57:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f791e000640 1 -- 172.21.15.83:0/1826803905 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f79180208e0 con 0x562b1dff0180 2024-04-23T14:57:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f7939f97740 1 -- 172.21.15.83:0/1826803905 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7928036070 con 0x562b1dff0180 2024-04-23T14:57:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f791e000640 1 -- 172.21.15.83:0/1826803905 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f791802d050 con 0x562b1dff0180 2024-04-23T14:57:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f7939f97740 10 client.27071 fetch_fsmap learned FSMap version 29 2024-04-23T14:57:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f7939f97740 1 -- 172.21.15.83:0/1826803905 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b1dc270f0 con 0x562b1dff0180 2024-04-23T14:57:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f791e000640 1 -- 172.21.15.83:0/1826803905 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7928036070 con 0x562b1dff0180 2024-04-23T14:57:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f791e000640 1 -- 172.21.15.83:0/1826803905 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f79180035f0 con 0x562b1dff0180 2024-04-23T14:57:31.758 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f7939f97740 10 client.27071 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:57:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f7939f97740 10 client.27071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:57:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f7939f97740 20 client.27071 populate_metadata read hostname 'smithi083' 2024-04-23T14:57:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f7939f97740 1 --2- 172.21.15.83:0/1826803905 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562b1e070ef0 0x562b1e073310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:57:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f7939f97740 4 client.27071 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:57:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.751+0000 7f7939f97740 1 -- 172.21.15.83:0/1826803905 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b1df5ca80 con 0x562b1e070ef0 2024-04-23T14:57:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.752+0000 7f7935000640 1 --2- 172.21.15.83:0/1826803905 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562b1e070ef0 0x562b1e073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:57:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.752+0000 7f791e000640 10 client.27071 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.752+0000 7f7935000640 1 --2- 172.21.15.83:0/1826803905 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562b1e070ef0 0x562b1e073310 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfea7eed18bf5d44 server_cookie=8e76749d4becabb4 in_seq=0 out_seq=0 2024-04-23T14:57:31.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f791e000640 1 -- 172.21.15.83:0/1826803905 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562b1df5ca80 con 0x562b1e070ef0 2024-04-23T14:57:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f791e000640 10 client.27071 handle_command_reply: tid=0 2024-04-23T14:57:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f7939f97740 1 client.27071 shutdown 2024-04-23T14:57:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f7939f97740 1 -- 172.21.15.83:0/1826803905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dff0180 msgr2=0x562b1e00d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f7939f97740 1 --2- 172.21.15.83:0/1826803905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dff0180 0x562b1e00d440 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x562b1dffd8b0 tx=0x7f7918004100 comp rx=0 tx=0).stop 2024-04-23T14:57:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f7939f97740 1 -- 172.21.15.83:0/1826803905 shutdown_connections 2024-04-23T14:57:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f7939f97740 1 --2- 172.21.15.83:0/1826803905 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x562b1e070ef0 0x562b1e073310 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f7939f97740 1 --2- 172.21.15.83:0/1826803905 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1dffa850 0x562b1e00d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f791e000640 0 client.27071 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:57:31.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f7939f97740 1 --2- 172.21.15.83:0/1826803905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b1dff0180 0x562b1e00d440 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f7939f97740 1 --2- 172.21.15.83:0/1826803905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562b1dfef860 0x562b1e010060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f7939f97740 1 -- 172.21.15.83:0/1826803905 >> 172.21.15.83:0/1826803905 conn(0x562b1dfebb80 msgr2=0x562b1e0060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:31.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f7939f97740 1 -- 172.21.15.83:0/1826803905 shutdown_connections 2024-04-23T14:57:31.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f7939f97740 1 -- 172.21.15.83:0/1826803905 wait complete. 2024-04-23T14:57:31.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.753+0000 7f7939f97740 20 client.27071 trim_cache size 0 max 16384 2024-04-23T14:57:31.761 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:57:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:57:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:57:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: "7387dfb5-9c1e-43a9-b907-f51e9199f5e2": { 2024-04-23T14:57:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:57:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "7387dfb5-9c1e-43a9-b907-f51e9199f5e2", 2024-04-23T14:57:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:57:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:57:31.762 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:57:31.763 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:57:31.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.754+0000 7f7937000640 1 -- 172.21.15.83:0/1009501756 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79140a3d20 msgr2=0x7f79140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.754+0000 7f7937000640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79140a3d20 0x7f79140a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7920000c00 tx=0x7f79200d44e0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.754+0000 7f7937000640 1 -- 172.21.15.83:0/1009501756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7930108610 msgr2=0x7f793019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:57:31.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.754+0000 7f7937000640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7930108610 0x7f793019ee10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f792000e9c0 tx=0x7f792000ee80 comp rx=0 tx=0).stop 2024-04-23T14:57:31.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.754+0000 7f7937000640 1 -- 172.21.15.83:0/1009501756 shutdown_connections 2024-04-23T14:57:31.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.754+0000 7f7937000640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f793010e660 0x7f79301a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.754+0000 7f7937000640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7930108fd0 0x7f793019f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.754+0000 7f7937000640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f79140a3d20 0x7f79140a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.754+0000 7f7937000640 1 --2- 172.21.15.83:0/1009501756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7930108610 0x7f793019ee10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:57:31.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.754+0000 7f7937000640 1 -- 172.21.15.83:0/1009501756 >> 172.21.15.83:0/1009501756 conn(0x7f7930076540 msgr2=0x7f7930077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:57:31.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.754+0000 7f7937000640 1 -- 172.21.15.83:0/1009501756 shutdown_connections 2024-04-23T14:57:31.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:57:31.754+0000 7f7937000640 1 -- 172.21.15.83:0/1009501756 wait complete. 2024-04-23T14:57:31.771 INFO:tasks.cephfs.filesystem:scrub status for tag:7387dfb5-9c1e-43a9-b907-f51e9199f5e2 - {'path': '/', 'tag': '7387dfb5-9c1e-43a9-b907-f51e9199f5e2', 'options': 'recursive,force'} 2024-04-23T14:57:31.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:31 smithi173 ceph-mon[32443]: pgmap v281: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:57:31.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:31 smithi203 ceph-mon[32800]: pgmap v281: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:57:32.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:57:32.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:57:32.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:57:32.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:57:32.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:57:32.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:57:32.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:57:32.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:32 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:57:32.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:57:32.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:57:32.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:57:32.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:57:32.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:57:32.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:57:32.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:32.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:32 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:33.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:57:33.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:33.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:57:33.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:57:33.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:33.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:57:33.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:33.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:57:33.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:33.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:57:33.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:33.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:57:33.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:33.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:32 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:57:33.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:33 smithi173 ceph-mon[32443]: pgmap v282: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:57:33.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:33 smithi203 ceph-mon[32800]: pgmap v282: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:57:34.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:33 smithi083 ceph-mon[27867]: pgmap v282: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:57:35.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:35 smithi173 ceph-mon[32443]: pgmap v283: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T14:57:35.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:35 smithi203 ceph-mon[32800]: pgmap v283: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T14:57:36.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:35 smithi083 ceph-mon[27867]: pgmap v283: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T14:57:37.809 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:37 smithi083 ceph-mon[27867]: pgmap v284: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T14:57:37.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:37 smithi173 ceph-mon[32443]: pgmap v284: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T14:57:37.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:37 smithi203 ceph-mon[32800]: pgmap v284: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T14:57:39.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:39 smithi173 ceph-mon[32443]: pgmap v285: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:57:39.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:39 smithi203 ceph-mon[32800]: pgmap v285: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:57:40.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:39 smithi083 ceph-mon[27867]: pgmap v285: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:57:41.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:41 smithi173 ceph-mon[32443]: pgmap v286: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:57:41.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:41 smithi203 ceph-mon[32800]: pgmap v286: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:57:42.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:41 smithi083 ceph-mon[27867]: pgmap v286: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-04-23T14:57:43.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:43 smithi173 ceph-mon[32443]: pgmap v287: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:57:43.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:43 smithi203 ceph-mon[32800]: pgmap v287: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:57:44.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:43 smithi083 ceph-mon[27867]: pgmap v287: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:57:44.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:44 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:57:44.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:44 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:57:44.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:44 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:57:44.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:44 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:57:44.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:44 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:57:44.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:44 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:57:45.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:44 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:57:45.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:44 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:57:45.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:44 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:57:45.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:45 smithi173 ceph-mon[32443]: pgmap v288: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:57:45.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:45 smithi203 ceph-mon[32800]: pgmap v288: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:57:46.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:45 smithi083 ceph-mon[27867]: pgmap v288: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-23T14:57:47.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:47 smithi173 ceph-mon[32443]: pgmap v289: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-04-23T14:57:47.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:47 smithi203 ceph-mon[32800]: pgmap v289: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-04-23T14:57:48.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:47 smithi083 ceph-mon[27867]: pgmap v289: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-04-23T14:57:49.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:49 smithi203 ceph-mon[32800]: pgmap v290: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:57:50.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:49 smithi083 ceph-mon[27867]: pgmap v290: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:57:50.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:49 smithi173 ceph-mon[32443]: pgmap v290: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:57:51.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:51 smithi203 ceph-mon[32800]: pgmap v291: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:57:52.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:51 smithi083 ceph-mon[27867]: pgmap v291: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:57:52.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:51 smithi173 ceph-mon[32443]: pgmap v291: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:57:53.877 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:53 smithi173 ceph-mon[32443]: pgmap v292: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:57:53.877 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:53 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:57:53.878 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:53 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:57:53.878 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:53 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:57:53.878 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:53 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:57:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:53 smithi203 ceph-mon[32800]: pgmap v292: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:57:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:53 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:57:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:53 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:57:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:53 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:57:53.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:53 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:57:54.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:53 smithi083 ceph-mon[27867]: pgmap v292: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:57:54.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:53 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:57:54.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:53 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:57:54.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:53 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:57:54.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:53 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:57:55.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:55 smithi203 ceph-mon[32800]: pgmap v293: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:57:56.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:55 smithi083 ceph-mon[27867]: pgmap v293: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:57:56.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:55 smithi173 ceph-mon[32443]: pgmap v293: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:57:57.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:57 smithi203 ceph-mon[32800]: pgmap v294: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:57:58.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:57 smithi083 ceph-mon[27867]: pgmap v294: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:57:58.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:57 smithi173 ceph-mon[32443]: pgmap v294: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:57:59.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:57:59 smithi203 ceph-mon[32800]: pgmap v295: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:58:00.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:57:59 smithi083 ceph-mon[27867]: pgmap v295: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:58:00.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:57:59 smithi173 ceph-mon[32443]: pgmap v295: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:58:01.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:58:01.772 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:58:01.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:01 smithi203 ceph-mon[32800]: pgmap v296: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:58:01.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.985+0000 7f066e800640 1 -- 172.21.15.83:0/1333870183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0668103060 msgr2=0x7f0668103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:01.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.985+0000 7f066e800640 1 --2- 172.21.15.83:0/1333870183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0668103060 0x7f0668103440 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f065c009900 tx=0x7f065c02a410 comp rx=0 tx=0).stop 2024-04-23T14:58:01.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.986+0000 7f066e800640 1 -- 172.21.15.83:0/1333870183 shutdown_connections 2024-04-23T14:58:01.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.986+0000 7f066e800640 1 --2- 172.21.15.83:0/1333870183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f066810c4d0 0x7f066810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:01.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.986+0000 7f066e800640 1 --2- 172.21.15.83:0/1333870183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0668103980 0x7f066810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:01.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.986+0000 7f066e800640 1 --2- 172.21.15.83:0/1333870183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0668103060 0x7f0668103440 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:01.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.986+0000 7f066e800640 1 -- 172.21.15.83:0/1333870183 >> 172.21.15.83:0/1333870183 conn(0x7f06680fc860 msgr2=0x7f06680fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:01.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.986+0000 7f066e800640 1 -- 172.21.15.83:0/1333870183 shutdown_connections 2024-04-23T14:58:01.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.986+0000 7f066e800640 1 -- 172.21.15.83:0/1333870183 wait complete. 2024-04-23T14:58:01.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.987+0000 7f066e800640 1 Processor -- start 2024-04-23T14:58:01.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.987+0000 7f066e800640 1 -- start start 2024-04-23T14:58:01.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.987+0000 7f066e800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0668103060 0x7f06681a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:01.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.987+0000 7f066e800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0668103980 0x7f06681a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:01.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.987+0000 7f066e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f066810c4d0 0x7f066819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:01.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.987+0000 7f066e800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06681122d0 con 0x7f066810c4d0 2024-04-23T14:58:01.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.987+0000 7f066e800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0668112150 con 0x7f0668103060 2024-04-23T14:58:01.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.987+0000 7f066e800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0668112450 con 0x7f0668103980 2024-04-23T14:58:01.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.988+0000 7f066ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0668103980 0x7f06681a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:01.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.988+0000 7f066de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f066810c4d0 0x7f066819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:01.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.988+0000 7f066ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0668103980 0x7f06681a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:52384/0 (socket says 172.21.15.83:52384) 2024-04-23T14:58:01.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.988+0000 7f066d400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0668103060 0x7f06681a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:01.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.988+0000 7f066ca00640 1 -- 172.21.15.83:0/3990950182 learned_addr learned my addr 172.21.15.83:0/3990950182 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:01.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.988+0000 7f066ca00640 1 -- 172.21.15.83:0/3990950182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0668103060 msgr2=0x7f06681a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:01.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.988+0000 7f066ca00640 1 --2- 172.21.15.83:0/3990950182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0668103060 0x7f06681a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:01.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.988+0000 7f066ca00640 1 -- 172.21.15.83:0/3990950182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f066810c4d0 msgr2=0x7f066819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:01.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.988+0000 7f066ca00640 1 --2- 172.21.15.83:0/3990950182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f066810c4d0 0x7f066819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:01.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.988+0000 7f066ca00640 1 -- 172.21.15.83:0/3990950182 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066819aff0 con 0x7f0668103980 2024-04-23T14:58:01.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.988+0000 7f066ca00640 1 --2- 172.21.15.83:0/3990950182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0668103980 0x7f06681a0ba0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f065400e9c0 tx=0x7f065400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:01.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.989+0000 7f0666000640 1 -- 172.21.15.83:0/3990950182 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065400cd80 con 0x7f0668103980 2024-04-23T14:58:01.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.989+0000 7f0666000640 1 -- 172.21.15.83:0/3990950182 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06540035f0 con 0x7f0668103980 2024-04-23T14:58:01.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.989+0000 7f066e800640 1 -- 172.21.15.83:0/3990950182 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066819b2e0 con 0x7f0668103980 2024-04-23T14:58:01.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.989+0000 7f0666000640 1 -- 172.21.15.83:0/3990950182 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654010690 con 0x7f0668103980 2024-04-23T14:58:01.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.989+0000 7f066e800640 1 -- 172.21.15.83:0/3990950182 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06681a74b0 con 0x7f0668103980 2024-04-23T14:58:01.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.992+0000 7f0672c1e740 1 Processor -- start 2024-04-23T14:58:01.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.992+0000 7f0672c1e740 1 -- start start 2024-04-23T14:58:01.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.992+0000 7f0672c1e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7d4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:01.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.992+0000 7f0672c1e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7df4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:01.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.992+0000 7f0672c1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:01.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.992+0000 7f0672c1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563fbd73a1f0 con 0x563fbd7dfa20 2024-04-23T14:58:01.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.992+0000 7f0672c1e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563fbd6e0350 con 0x563fbd7d5350 2024-04-23T14:58:01.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.992+0000 7f0672c1e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563fbd7361b0 con 0x563fbd7d4a30 2024-04-23T14:58:01.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.992+0000 7f0666000640 1 -- 172.21.15.83:0/3990950182 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f06540037a0 con 0x7f0668103980 2024-04-23T14:58:01.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.993+0000 7f066de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7d4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:01.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.993+0000 7f066d400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7df4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:01.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.993+0000 7f066d400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7df4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:55632/0 (socket says 172.21.15.83:55632) 2024-04-23T14:58:01.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.993+0000 7f066d400640 1 -- 172.21.15.83:0/2704144438 learned_addr learned my addr 172.21.15.83:0/2704144438 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:01.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.993+0000 7f066ca00640 1 --2- 172.21.15.83:0/2704144438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:01.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.993+0000 7f066de00640 1 -- 172.21.15.83:0/2704144438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 msgr2=0x563fbd7df4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:01.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.993+0000 7f066de00640 1 --2- 172.21.15.83:0/2704144438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7df4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:01.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.993+0000 7f066de00640 1 -- 172.21.15.83:0/2704144438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 msgr2=0x563fbd7e1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:01.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.993+0000 7f066de00640 1 --2- 172.21.15.83:0/2704144438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7e1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:01.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.993+0000 7f066de00640 1 -- 172.21.15.83:0/2704144438 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fbd734f50 con 0x563fbd7d4a30 2024-04-23T14:58:01.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.993+0000 7f066ca00640 1 --2- 172.21.15.83:0/2704144438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7e1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:58:01.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.994+0000 7f066de00640 1 --2- 172.21.15.83:0/2704144438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7d4e10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0660009eb0 tx=0x7f06600107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da50263a9ebeae7b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:01.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.994+0000 7f065a600640 1 -- 172.21.15.83:0/2704144438 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660003b70 con 0x563fbd7d4a30 2024-04-23T14:58:01.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.994+0000 7f065a600640 1 -- 172.21.15.83:0/2704144438 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0660015070 con 0x563fbd7d4a30 2024-04-23T14:58:01.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.994+0000 7f065a600640 1 -- 172.21.15.83:0/2704144438 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066001a940 con 0x563fbd7d4a30 2024-04-23T14:58:01.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.994+0000 7f0666000640 1 --2- 172.21.15.83:0/3990950182 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06500a3da0 0x7f06500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:01.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.994+0000 7f0666000640 1 -- 172.21.15.83:0/3990950182 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f06540d0b00 con 0x7f0668103980 2024-04-23T14:58:01.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.994+0000 7f0672c1e740 1 -- 172.21.15.83:0/2704144438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 msgr2=0x563fbd7d4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:01.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.994+0000 7f0672c1e740 1 --2- 172.21.15.83:0/2704144438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7d4e10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0660009eb0 tx=0x7f06600107a0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.995+0000 7f066d400640 1 --2- 172.21.15.83:0/3990950182 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06500a3da0 0x7f06500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.002 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.995+0000 7f0672c1e740 1 -- 172.21.15.83:0/2704144438 shutdown_connections 2024-04-23T14:58:02.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.995+0000 7f0672c1e740 1 --2- 172.21.15.83:0/2704144438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7e1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.995+0000 7f0672c1e740 1 --2- 172.21.15.83:0/2704144438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7df4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.995+0000 7f0672c1e740 1 --2- 172.21.15.83:0/2704144438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7d4e10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.995+0000 7f0672c1e740 1 -- 172.21.15.83:0/2704144438 >> 172.21.15.83:0/2704144438 conn(0x563fbd7d0d50 msgr2=0x563fbd7d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.995+0000 7f0672c1e740 1 -- 172.21.15.83:0/2704144438 shutdown_connections 2024-04-23T14:58:02.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.995+0000 7f0672c1e740 1 -- 172.21.15.83:0/2704144438 wait complete. 2024-04-23T14:58:02.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.995+0000 7f0672c1e740 1 -- start start 2024-04-23T14:58:02.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.995+0000 7f066d400640 1 --2- 172.21.15.83:0/3990950182 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06500a3da0 0x7f06500a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f065c005c40 tx=0x7f065c005b50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f0672c1e740 1 Processor -- start 2024-04-23T14:58:02.003 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f0672c1e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7dcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f0672c1e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7dd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f0672c1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7dd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f0672c1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563fbd7e2810 con 0x563fbd7dfa20 2024-04-23T14:58:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f0672c1e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563fbd6f3690 con 0x563fbd7d5350 2024-04-23T14:58:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f0672c1e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563fbd7e2990 con 0x563fbd7d4a30 2024-04-23T14:58:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f0672c1e740 10 client.0 _ll_register_callbacks cb 0x563fbd72f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f066de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7dd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f066d400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7dcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.004 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f066de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7dd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46512/0 (socket says 172.21.15.83:46512) 2024-04-23T14:58:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f066de00640 1 -- 172.21.15.83:0/2249832297 learned_addr learned my addr 172.21.15.83:0/2249832297 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.996+0000 7f066ca00640 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7dd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.997+0000 7f066de00640 1 -- 172.21.15.83:0/2249832297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 msgr2=0x563fbd7dcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.997+0000 7f066de00640 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7dcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.997+0000 7f066de00640 1 -- 172.21.15.83:0/2249832297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 msgr2=0x563fbd7dd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.997+0000 7f066de00640 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7dd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.997+0000 7f066de00640 1 -- 172.21.15.83:0/2249832297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fbd6f2920 con 0x563fbd7dfa20 2024-04-23T14:58:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.997+0000 7f066ca00640 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7dd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.997+0000 7f066de00640 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7dd540 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f066001a5e0 tx=0x7f0660003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47d9b3ed2c39813 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.997+0000 7f064ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:02.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.997+0000 7f064ea00640 1 -- 172.21.15.83:0/2249832297 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660027020 con 0x563fbd7dfa20 2024-04-23T14:58:02.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.997+0000 7f0672c1e740 1 -- 172.21.15.83:0/2249832297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0660015070 con 0x563fbd7dfa20 2024-04-23T14:58:02.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.997+0000 7f064ea00640 1 -- 172.21.15.83:0/2249832297 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f0660004460 con 0x563fbd7dfa20 2024-04-23T14:58:02.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.998+0000 7f064ea00640 1 -- 172.21.15.83:0/2249832297 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0660032050 con 0x563fbd7dfa20 2024-04-23T14:58:02.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.998+0000 7f0672c1e740 10 client.16881 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:02.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.998+0000 7f0672c1e740 1 -- 172.21.15.83:0/2249832297 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563fbd5167d0 con 0x563fbd7dfa20 2024-04-23T14:58:02.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.998+0000 7f064ea00640 1 -- 172.21.15.83:0/2249832297 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0660015070 con 0x563fbd7dfa20 2024-04-23T14:58:02.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.998+0000 7f064ea00640 1 -- 172.21.15.83:0/2249832297 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0660023470 con 0x563fbd7dfa20 2024-04-23T14:58:02.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.998+0000 7f0672c1e740 10 client.16881 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.998+0000 7f0672c1e740 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.998+0000 7f0672c1e740 20 client.16881 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.998+0000 7f0672c1e740 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563fbd7efe50 0x563fbd7f2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.998+0000 7f0672c1e740 4 client.16881 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:58:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.998+0000 7f0672c1e740 1 -- 172.21.15.83:0/2249832297 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fbd747c10 con 0x563fbd7efe50 2024-04-23T14:58:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.999+0000 7f066d400640 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563fbd7efe50 0x563fbd7f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.000+0000 7f064ea00640 10 client.16881 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:01.999+0000 7f066d400640 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563fbd7efe50 0x563fbd7f2240 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d656f251f3256e0d server_cookie=43814dabae2081ea in_seq=0 out_seq=0 2024-04-23T14:58:02.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.000+0000 7f064ea00640 1 -- 172.21.15.83:0/2249832297 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fbd747c10 con 0x563fbd7efe50 2024-04-23T14:58:02.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f064ea00640 10 client.16881 handle_command_reply: tid=0 2024-04-23T14:58:02.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f0672c1e740 1 client.16881 shutdown 2024-04-23T14:58:02.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f0672c1e740 1 -- 172.21.15.83:0/2249832297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 msgr2=0x563fbd7dd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f0672c1e740 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7dd540 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f066001a5e0 tx=0x7f0660003c20 comp rx=0 tx=0).stop 2024-04-23T14:58:02.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f0672c1e740 1 -- 172.21.15.83:0/2249832297 shutdown_connections 2024-04-23T14:58:02.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f0672c1e740 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563fbd7efe50 0x563fbd7f2240 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f0672c1e740 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7dd540 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f0672c1e740 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7dd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f064ea00640 0 client.16881 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:02.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f0672c1e740 1 --2- 172.21.15.83:0/2249832297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7dcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f0672c1e740 1 -- 172.21.15.83:0/2249832297 >> 172.21.15.83:0/2249832297 conn(0x563fbd7d0d50 msgr2=0x563fbd7d6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f0672c1e740 1 -- 172.21.15.83:0/2249832297 shutdown_connections 2024-04-23T14:58:02.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f0672c1e740 1 -- 172.21.15.83:0/2249832297 wait complete. 2024-04-23T14:58:02.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.001+0000 7f0672c1e740 20 client.16881 trim_cache size 0 max 16384 2024-04-23T14:58:02.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:01 smithi083 ceph-mon[27867]: pgmap v296: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:58:02.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.020+0000 7f0672c1e740 1 Processor -- start 2024-04-23T14:58:02.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.021+0000 7f0672c1e740 1 -- start start 2024-04-23T14:58:02.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.021+0000 7f0672c1e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.021+0000 7f0672c1e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.021+0000 7f0672c1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7d71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.021+0000 7f0672c1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06681b2670 con 0x563fbd7dfa20 2024-04-23T14:58:02.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.021+0000 7f0672c1e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f06681aff40 con 0x563fbd7d5350 2024-04-23T14:58:02.022 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.021+0000 7f0672c1e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06681addb0 con 0x563fbd7d4a30 2024-04-23T14:58:02.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.021+0000 7f066de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7d71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.024 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.021+0000 7f066ca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.021+0000 7f066d400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.021+0000 7f066de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7d71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46528/0 (socket says 172.21.15.83:46528) 2024-04-23T14:58:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.021+0000 7f066de00640 1 -- 172.21.15.83:0/698784430 learned_addr learned my addr 172.21.15.83:0/698784430 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.022+0000 7f066d400640 1 -- 172.21.15.83:0/698784430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 msgr2=0x563fbd7dd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.022+0000 7f066d400640 1 --2- 172.21.15.83:0/698784430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7dd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.022+0000 7f066d400640 1 -- 172.21.15.83:0/698784430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 msgr2=0x563fbd7d71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.022+0000 7f066d400640 1 --2- 172.21.15.83:0/698784430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7d71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.022+0000 7f066d400640 1 -- 172.21.15.83:0/698784430 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fbd44c3f0 con 0x563fbd7d4a30 2024-04-23T14:58:02.025 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.022+0000 7f066d400640 1 --2- 172.21.15.83:0/698784430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7eb6d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f065c031550 tx=0x7f065c03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df43e962ca12951f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.022+0000 7f0659c00640 1 -- 172.21.15.83:0/698784430 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c035040 con 0x563fbd7d4a30 2024-04-23T14:58:02.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.022+0000 7f0659c00640 1 -- 172.21.15.83:0/698784430 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f065c054810 con 0x563fbd7d4a30 2024-04-23T14:58:02.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.022+0000 7f0659c00640 1 -- 172.21.15.83:0/698784430 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c0325c0 con 0x563fbd7d4a30 2024-04-23T14:58:02.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.023+0000 7f0672c1e740 1 -- 172.21.15.83:0/698784430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 msgr2=0x563fbd7eb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.023+0000 7f0672c1e740 1 --2- 172.21.15.83:0/698784430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7eb6d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f065c031550 tx=0x7f065c03f000 comp rx=0 tx=0).stop 2024-04-23T14:58:02.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.023+0000 7f0672c1e740 1 -- 172.21.15.83:0/698784430 shutdown_connections 2024-04-23T14:58:02.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.023+0000 7f0672c1e740 1 --2- 172.21.15.83:0/698784430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7d71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.026 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.023+0000 7f0672c1e740 1 --2- 172.21.15.83:0/698784430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d5350 0x563fbd7dd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.023+0000 7f0672c1e740 1 --2- 172.21.15.83:0/698784430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d4a30 0x563fbd7eb6d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.023+0000 7f0672c1e740 1 -- 172.21.15.83:0/698784430 >> 172.21.15.83:0/698784430 conn(0x563fbd7d0d50 msgr2=0x563fbd7de2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.023+0000 7f0672c1e740 1 -- 172.21.15.83:0/698784430 shutdown_connections 2024-04-23T14:58:02.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.023+0000 7f0672c1e740 1 -- 172.21.15.83:0/698784430 wait complete. 2024-04-23T14:58:02.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.024+0000 7f0672c1e740 1 -- start start 2024-04-23T14:58:02.027 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.024+0000 7f0672c1e740 1 Processor -- start 2024-04-23T14:58:02.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.024+0000 7f0672c1e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d4a30 0x563fbd7e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.024+0000 7f0672c1e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d5350 0x563fbd7e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.024+0000 7f0672c1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7e3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.024+0000 7f0672c1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06681ae400 con 0x563fbd7dfa20 2024-04-23T14:58:02.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.024+0000 7f0672c1e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f06681af870 con 0x563fbd7d4a30 2024-04-23T14:58:02.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.024+0000 7f0672c1e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06681aead0 con 0x563fbd7d5350 2024-04-23T14:58:02.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.024+0000 7f0672c1e740 10 client.0 _ll_register_callbacks cb 0x7f065c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:02.031 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f066de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7e3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f066de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7e3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46530/0 (socket says 172.21.15.83:46530) 2024-04-23T14:58:02.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f066de00640 1 -- 172.21.15.83:0/1806872802 learned_addr learned my addr 172.21.15.83:0/1806872802 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:02.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f066ca00640 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d5350 0x563fbd7e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f066d400640 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d4a30 0x563fbd7e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f066de00640 1 -- 172.21.15.83:0/1806872802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d5350 msgr2=0x563fbd7e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f066de00640 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d5350 0x563fbd7e8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f066de00640 1 -- 172.21.15.83:0/1806872802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d4a30 msgr2=0x563fbd7e7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f066de00640 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d4a30 0x563fbd7e7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f066de00640 1 -- 172.21.15.83:0/1806872802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fbd40c0f0 con 0x563fbd7dfa20 2024-04-23T14:58:02.032 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f066de00640 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7e3080 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0660005b40 tx=0x7f0660009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9805994ccf4f622 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f064ea00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:02.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.025+0000 7f064ea00640 1 -- 172.21.15.83:0/1806872802 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0660027040 con 0x563fbd7dfa20 2024-04-23T14:58:02.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f064ea00640 1 -- 172.21.15.83:0/1806872802 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f0660023ab0 con 0x563fbd7dfa20 2024-04-23T14:58:02.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f0672c1e740 1 -- 172.21.15.83:0/1806872802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f065c054810 con 0x563fbd7dfa20 2024-04-23T14:58:02.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f064ea00640 1 -- 172.21.15.83:0/1806872802 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0660044050 con 0x563fbd7dfa20 2024-04-23T14:58:02.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f0672c1e740 10 client.16887 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:02.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f0672c1e740 1 -- 172.21.15.83:0/1806872802 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563fbd6ff810 con 0x563fbd7dfa20 2024-04-23T14:58:02.033 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f064ea00640 1 -- 172.21.15.83:0/1806872802 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f065c054810 con 0x563fbd7dfa20 2024-04-23T14:58:02.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f064ea00640 1 -- 172.21.15.83:0/1806872802 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f066001a4e0 con 0x563fbd7dfa20 2024-04-23T14:58:02.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f0672c1e740 10 client.16887 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:02.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f0672c1e740 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:02.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f0672c1e740 20 client.16887 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:02.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f0672c1e740 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563fbd85e710 0x563fbd860b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f0672c1e740 4 client.16887 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:58:02.034 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.026+0000 7f0672c1e740 1 -- 172.21.15.83:0/1806872802 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563fbd741a80 con 0x563fbd85e710 2024-04-23T14:58:02.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.027+0000 7f066d400640 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563fbd85e710 0x563fbd860b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.027+0000 7f066d400640 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563fbd85e710 0x563fbd860b30 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb23b914000c1ca4 server_cookie=e2d3c2526da9dc81 in_seq=0 out_seq=0 2024-04-23T14:58:02.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.027+0000 7f064ea00640 10 client.16887 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:02.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.027+0000 7f064ea00640 1 -- 172.21.15.83:0/1806872802 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563fbd741a80 con 0x563fbd85e710 2024-04-23T14:58:02.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f064ea00640 10 client.16887 handle_command_reply: tid=0 2024-04-23T14:58:02.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f0672c1e740 1 client.16887 shutdown 2024-04-23T14:58:02.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f0672c1e740 1 -- 172.21.15.83:0/1806872802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 msgr2=0x563fbd7e3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f0672c1e740 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7e3080 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0660005b40 tx=0x7f0660009c30 comp rx=0 tx=0).stop 2024-04-23T14:58:02.035 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f0672c1e740 1 -- 172.21.15.83:0/1806872802 shutdown_connections 2024-04-23T14:58:02.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f0672c1e740 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563fbd85e710 0x563fbd860b30 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f0672c1e740 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563fbd7dfa20 0x563fbd7e3080 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f0672c1e740 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563fbd7d5350 0x563fbd7e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f0672c1e740 1 --2- 172.21.15.83:0/1806872802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563fbd7d4a30 0x563fbd7e7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f064ea00640 0 client.16887 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:02.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f0672c1e740 1 -- 172.21.15.83:0/1806872802 >> 172.21.15.83:0/1806872802 conn(0x563fbd7d0d50 msgr2=0x563fbd7ea6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f0672c1e740 1 -- 172.21.15.83:0/1806872802 shutdown_connections 2024-04-23T14:58:02.036 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f0672c1e740 1 -- 172.21.15.83:0/1806872802 wait complete. 2024-04-23T14:58:02.037 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.028+0000 7f0672c1e740 20 client.16887 trim_cache size 0 max 16384 2024-04-23T14:58:02.037 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:58:02.037 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:58:02.037 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:58:02.037 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:58:02.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.029+0000 7f066e800640 1 -- 172.21.15.83:0/3990950182 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06500a3da0 msgr2=0x7f06500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.029+0000 7f066e800640 1 --2- 172.21.15.83:0/3990950182 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06500a3da0 0x7f06500a6260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f065c005c40 tx=0x7f065c005b50 comp rx=0 tx=0).stop 2024-04-23T14:58:02.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.029+0000 7f066e800640 1 -- 172.21.15.83:0/3990950182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0668103980 msgr2=0x7f06681a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.038 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.029+0000 7f066e800640 1 --2- 172.21.15.83:0/3990950182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0668103980 0x7f06681a0ba0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f065400e9c0 tx=0x7f065400ee80 comp rx=0 tx=0).stop 2024-04-23T14:58:02.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.029+0000 7f066e800640 1 -- 172.21.15.83:0/3990950182 shutdown_connections 2024-04-23T14:58:02.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.029+0000 7f066e800640 1 --2- 172.21.15.83:0/3990950182 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06500a3da0 0x7f06500a6260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.029+0000 7f066e800640 1 --2- 172.21.15.83:0/3990950182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f066810c4d0 0x7f066819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.029+0000 7f066e800640 1 --2- 172.21.15.83:0/3990950182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0668103980 0x7f06681a0ba0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.029+0000 7f066e800640 1 --2- 172.21.15.83:0/3990950182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0668103060 0x7f06681a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.029+0000 7f066e800640 1 -- 172.21.15.83:0/3990950182 >> 172.21.15.83:0/3990950182 conn(0x7f06680fc860 msgr2=0x7f0668101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.029+0000 7f066e800640 1 -- 172.21.15.83:0/3990950182 shutdown_connections 2024-04-23T14:58:02.039 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.029+0000 7f066e800640 1 -- 172.21.15.83:0/3990950182 wait complete. 2024-04-23T14:58:02.049 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:58:02.049 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:58:02.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:01 smithi173 ceph-mon[32443]: pgmap v296: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:58:02.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.249+0000 7f9630000640 1 -- 172.21.15.83:0/898606671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628107320 msgr2=0x7f962810eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.249+0000 7f9630000640 1 --2- 172.21.15.83:0/898606671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628107320 0x7f962810eef0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9620009be0 tx=0x7f962002a6c0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.249+0000 7f9630000640 1 -- 172.21.15.83:0/898606671 shutdown_connections 2024-04-23T14:58:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.249+0000 7f9630000640 1 --2- 172.21.15.83:0/898606671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628107320 0x7f962810eef0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.249+0000 7f9630000640 1 --2- 172.21.15.83:0/898606671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9628102c00 0x7f9628106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.249+0000 7f9630000640 1 --2- 172.21.15.83:0/898606671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9628102250 0x7f9628102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.249+0000 7f9630000640 1 -- 172.21.15.83:0/898606671 >> 172.21.15.83:0/898606671 conn(0x7f9628079400 msgr2=0x7f9628079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.249+0000 7f9630000640 1 -- 172.21.15.83:0/898606671 shutdown_connections 2024-04-23T14:58:02.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.249+0000 7f9630000640 1 -- 172.21.15.83:0/898606671 wait complete. 2024-04-23T14:58:02.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.250+0000 7f9630000640 1 Processor -- start 2024-04-23T14:58:02.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.251+0000 7f9630000640 1 -- start start 2024-04-23T14:58:02.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.251+0000 7f9630000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628102250 0x7f96281a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.251+0000 7f9630000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9628107320 0x7f96281a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.251+0000 7f9630000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f962819a940 0x7f962819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.251+0000 7f9630000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9628112390 con 0x7f9628107320 2024-04-23T14:58:02.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.251+0000 7f9630000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9628112210 con 0x7f962819a940 2024-04-23T14:58:02.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.251+0000 7f9630000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9628112510 con 0x7f9628102250 2024-04-23T14:58:02.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.251+0000 7f962cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9628107320 0x7f96281a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.254 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.251+0000 7f962e000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f962819a940 0x7f962819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.251+0000 7f962cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9628107320 0x7f96281a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46548/0 (socket says 172.21.15.83:46548) 2024-04-23T14:58:02.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.251+0000 7f962cc00640 1 -- 172.21.15.83:0/1934335707 learned_addr learned my addr 172.21.15.83:0/1934335707 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:02.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.251+0000 7f962d600640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628102250 0x7f96281a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.252+0000 7f962cc00640 1 -- 172.21.15.83:0/1934335707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628102250 msgr2=0x7f96281a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.252+0000 7f962cc00640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628102250 0x7f96281a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.252+0000 7f962cc00640 1 -- 172.21.15.83:0/1934335707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f962819a940 msgr2=0x7f962819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.252+0000 7f962cc00640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f962819a940 0x7f962819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.252+0000 7f962cc00640 1 -- 172.21.15.83:0/1934335707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962819b460 con 0x7f9628107320 2024-04-23T14:58:02.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.252+0000 7f962e000640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f962819a940 0x7f962819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:02.255 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.252+0000 7f962cc00640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9628107320 0x7f96281a0cd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9c0 tx=0x7f961400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.252+0000 7f9626000640 1 -- 172.21.15.83:0/1934335707 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961400ccc0 con 0x7f9628107320 2024-04-23T14:58:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.252+0000 7f9626000640 1 -- 172.21.15.83:0/1934335707 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9614005500 con 0x7f9628107320 2024-04-23T14:58:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.252+0000 7f9630000640 1 -- 172.21.15.83:0/1934335707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962819b750 con 0x7f9628107320 2024-04-23T14:58:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.252+0000 7f9626000640 1 -- 172.21.15.83:0/1934335707 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614010690 con 0x7f9628107320 2024-04-23T14:58:02.256 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.252+0000 7f9630000640 1 -- 172.21.15.83:0/1934335707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96281a7950 con 0x7f9628107320 2024-04-23T14:58:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.253+0000 7f9630000640 1 -- 172.21.15.83:0/1934335707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9628068e20 con 0x7f9628107320 2024-04-23T14:58:02.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.256+0000 7f9626000640 1 -- 172.21.15.83:0/1934335707 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f96140046c0 con 0x7f9628107320 2024-04-23T14:58:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.257+0000 7f9626000640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f96100a3d20 0x7f96100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.258+0000 7f9626000640 1 -- 172.21.15.83:0/1934335707 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f96140d0a40 con 0x7f9628107320 2024-04-23T14:58:02.258 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.258+0000 7f962d600640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f96100a3d20 0x7f96100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.259 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.258+0000 7f962d600640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f96100a3d20 0x7f96100a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f962819ba80 tx=0x7f961c005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.262+0000 7f9626000640 1 -- 172.21.15.83:0/1934335707 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f961409d070 con 0x7f9628107320 2024-04-23T14:58:02.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.505+0000 7f9630000640 1 -- 172.21.15.83:0/1934335707 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f962819be70 con 0x7f9628107320 2024-04-23T14:58:02.508 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.507+0000 7f9626000640 1 -- 172.21.15.83:0/1934335707 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7f962819be70 con 0x7f9628107320 2024-04-23T14:58:02.509 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:58:02.509 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:58:02.509 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:58:02.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.510+0000 7f9630000640 1 -- 172.21.15.83:0/1934335707 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f96100a3d20 msgr2=0x7f96100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.510+0000 7f9630000640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f96100a3d20 0x7f96100a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f962819ba80 tx=0x7f961c005f90 comp rx=0 tx=0).stop 2024-04-23T14:58:02.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.510+0000 7f9630000640 1 -- 172.21.15.83:0/1934335707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9628107320 msgr2=0x7f96281a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.510+0000 7f9630000640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9628107320 0x7f96281a0cd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f961400e9c0 tx=0x7f961400ee80 comp rx=0 tx=0).stop 2024-04-23T14:58:02.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.510+0000 7f9630000640 1 -- 172.21.15.83:0/1934335707 shutdown_connections 2024-04-23T14:58:02.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.510+0000 7f9630000640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f962819a940 0x7f962819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.510+0000 7f9630000640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f96100a3d20 0x7f96100a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.510+0000 7f9630000640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9628107320 0x7f96281a0cd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.510+0000 7f9630000640 1 --2- 172.21.15.83:0/1934335707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628102250 0x7f96281a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.510+0000 7f9630000640 1 -- 172.21.15.83:0/1934335707 >> 172.21.15.83:0/1934335707 conn(0x7f9628079400 msgr2=0x7f96280ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.510+0000 7f9630000640 1 -- 172.21.15.83:0/1934335707 shutdown_connections 2024-04-23T14:58:02.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.511+0000 7f9630000640 1 -- 172.21.15.83:0/1934335707 wait complete. 2024-04-23T14:58:02.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:58:02.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.741+0000 7f72bd600640 1 -- 172.21.15.83:0/2068013231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b8102aa0 msgr2=0x7f72b8106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.741+0000 7f72bd600640 1 --2- 172.21.15.83:0/2068013231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b8102aa0 0x7f72b8106ae0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009900 tx=0x7f72a402a410 comp rx=0 tx=0).stop 2024-04-23T14:58:02.742 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.741+0000 7f72bd600640 1 -- 172.21.15.83:0/2068013231 shutdown_connections 2024-04-23T14:58:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.741+0000 7f72bd600640 1 --2- 172.21.15.83:0/2068013231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b81071c0 0x7f72b810ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.741+0000 7f72bd600640 1 --2- 172.21.15.83:0/2068013231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b8102aa0 0x7f72b8106ae0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.741+0000 7f72bd600640 1 --2- 172.21.15.83:0/2068013231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b81020f0 0x7f72b81024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.741+0000 7f72bd600640 1 -- 172.21.15.83:0/2068013231 >> 172.21.15.83:0/2068013231 conn(0x7f72b80fd9d0 msgr2=0x7f72b80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.741+0000 7f72bd600640 1 -- 172.21.15.83:0/2068013231 shutdown_connections 2024-04-23T14:58:02.743 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.741+0000 7f72bd600640 1 -- 172.21.15.83:0/2068013231 wait complete. 2024-04-23T14:58:02.745 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.742+0000 7f72bd600640 1 Processor -- start 2024-04-23T14:58:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.742+0000 7f72bd600640 1 -- start start 2024-04-23T14:58:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.742+0000 7f72bd600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b81020f0 0x7f72b81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.742+0000 7f72bd600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b81071c0 0x7f72b81a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.742+0000 7f72bd600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b819a8b0 0x7f72b819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.742+0000 7f72bd600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72b8112250 con 0x7f72b81071c0 2024-04-23T14:58:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.742+0000 7f72bd600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81120d0 con 0x7f72b81020f0 2024-04-23T14:58:02.746 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.742+0000 7f72bd600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81123d0 con 0x7f72b819a8b0 2024-04-23T14:58:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.743+0000 7f72b7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b81071c0 0x7f72b81a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.743+0000 7f72b7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b81071c0 0x7f72b81a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46562/0 (socket says 172.21.15.83:46562) 2024-04-23T14:58:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.743+0000 7f72b7400640 1 -- 172.21.15.83:0/3035215681 learned_addr learned my addr 172.21.15.83:0/3035215681 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.743+0000 7f72b7e00640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b81020f0 0x7f72b81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.743+0000 7f72bcc00640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b819a8b0 0x7f72b819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.743+0000 7f72b7400640 1 -- 172.21.15.83:0/3035215681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b819a8b0 msgr2=0x7f72b819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.743+0000 7f72b7400640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b819a8b0 0x7f72b819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.743+0000 7f72b7400640 1 -- 172.21.15.83:0/3035215681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b81020f0 msgr2=0x7f72b81a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.747 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.743+0000 7f72b7400640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b81020f0 0x7f72b81a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.743+0000 7f72b7400640 1 -- 172.21.15.83:0/3035215681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b819b3d0 con 0x7f72b81071c0 2024-04-23T14:58:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.743+0000 7f72b7e00640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b81020f0 0x7f72b81a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.744+0000 7f72b7400640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b81071c0 0x7f72b81a0ba0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f72a4005940 tx=0x7f72a40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.744+0000 7f72b4c00640 1 -- 172.21.15.83:0/3035215681 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a403f030 con 0x7f72b81071c0 2024-04-23T14:58:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.744+0000 7f72b4c00640 1 -- 172.21.15.83:0/3035215681 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a40044c0 con 0x7f72b81071c0 2024-04-23T14:58:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.744+0000 7f72b4c00640 1 -- 172.21.15.83:0/3035215681 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4035070 con 0x7f72b81071c0 2024-04-23T14:58:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.744+0000 7f72bd600640 1 -- 172.21.15.83:0/3035215681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b819b660 con 0x7f72b81071c0 2024-04-23T14:58:02.748 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.744+0000 7f72bd600640 1 -- 172.21.15.83:0/3035215681 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b81a7810 con 0x7f72b81071c0 2024-04-23T14:58:02.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.747+0000 7f72c19f7740 1 Processor -- start 2024-04-23T14:58:02.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72c19f7740 1 -- start start 2024-04-23T14:58:02.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72c19f7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b8af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.751 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72c19f7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72c19f7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b97f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72c19f7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562a99af01d0 con 0x562a99b95b30 2024-04-23T14:58:02.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72b4c00640 1 -- 172.21.15.83:0/3035215681 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f72a4042030 con 0x7f72b81071c0 2024-04-23T14:58:02.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72c19f7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562a99a96450 con 0x562a99b8b460 2024-04-23T14:58:02.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72c19f7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562a99aec190 con 0x562a99b8ab40 2024-04-23T14:58:02.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72bcc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b8af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.752 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72bcc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b8af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:52454/0 (socket says 172.21.15.83:52454) 2024-04-23T14:58:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72bcc00640 1 -- 172.21.15.83:0/3220269642 learned_addr learned my addr 172.21.15.83:0/3220269642 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72b7e00640 1 --2- 172.21.15.83:0/3220269642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.748+0000 7f72b7400640 1 --2- 172.21.15.83:0/3220269642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b97f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.749+0000 7f72bcc00640 1 -- 172.21.15.83:0/3220269642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 msgr2=0x562a99b955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.749+0000 7f72bcc00640 1 --2- 172.21.15.83:0/3220269642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.749+0000 7f72bcc00640 1 -- 172.21.15.83:0/3220269642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 msgr2=0x562a99b97f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.749+0000 7f72bcc00640 1 --2- 172.21.15.83:0/3220269642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b97f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.749+0000 7f72bcc00640 1 -- 172.21.15.83:0/3220269642 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a99b53310 con 0x562a99b8ab40 2024-04-23T14:58:02.753 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.749+0000 7f72b7400640 1 --2- 172.21.15.83:0/3220269642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b97f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:58:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.749+0000 7f72b7e00640 1 --2- 172.21.15.83:0/3220269642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.749+0000 7f72bcc00640 1 --2- 172.21.15.83:0/3220269642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b8af20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009f60 tx=0x7f72b0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be0e9a5be93882bc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.749+0000 7f72ad200640 1 -- 172.21.15.83:0/3220269642 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b001e070 con 0x562a99b8ab40 2024-04-23T14:58:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.749+0000 7f72ad200640 1 -- 172.21.15.83:0/3220269642 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b0003c30 con 0x562a99b8ab40 2024-04-23T14:58:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.749+0000 7f72ad200640 1 -- 172.21.15.83:0/3220269642 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b000f9c0 con 0x562a99b8ab40 2024-04-23T14:58:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.750+0000 7f72b4c00640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f729c0a3da0 0x7f729c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.750+0000 7f72b4c00640 1 -- 172.21.15.83:0/3035215681 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f72a40f0b20 con 0x7f72b81071c0 2024-04-23T14:58:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.750+0000 7f72c19f7740 1 -- 172.21.15.83:0/3220269642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 msgr2=0x562a99b8af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.750+0000 7f72b7e00640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f729c0a3da0 0x7f729c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.750+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3220269642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b8af20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009f60 tx=0x7f72b0009f90 comp rx=0 tx=0).stop 2024-04-23T14:58:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.750+0000 7f72c19f7740 1 -- 172.21.15.83:0/3220269642 shutdown_connections 2024-04-23T14:58:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.750+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3220269642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b97f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.750+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3220269642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.750+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3220269642 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b8af20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.750+0000 7f72c19f7740 1 -- 172.21.15.83:0/3220269642 >> 172.21.15.83:0/3220269642 conn(0x562a99b86e60 msgr2=0x562a99b87240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.750+0000 7f72c19f7740 1 -- 172.21.15.83:0/3220269642 shutdown_connections 2024-04-23T14:58:02.755 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.750+0000 7f72c19f7740 1 -- 172.21.15.83:0/3220269642 wait complete. 2024-04-23T14:58:02.759 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.751+0000 7f72b7e00640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f729c0a3da0 0x7f729c0a6260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009d80 tx=0x7f72a8005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.751+0000 7f72c19f7740 1 -- start start 2024-04-23T14:58:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.751+0000 7f72c19f7740 1 Processor -- start 2024-04-23T14:58:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.751+0000 7f72c19f7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.751+0000 7f72c19f7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.751+0000 7f72c19f7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b93010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.751+0000 7f72c19f7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562a99b989c0 con 0x562a99b95b30 2024-04-23T14:58:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.751+0000 7f72c19f7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x562a99aa9810 con 0x562a99b8b460 2024-04-23T14:58:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.751+0000 7f72c19f7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x562a99b98b40 con 0x562a99b8ab40 2024-04-23T14:58:02.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.752+0000 7f72c19f7740 10 client.0 _ll_register_callbacks cb 0x562a99a77140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.752+0000 7f72bcc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b93010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.752+0000 7f72bcc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b93010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46570/0 (socket says 172.21.15.83:46570) 2024-04-23T14:58:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.752+0000 7f72bcc00640 1 -- 172.21.15.83:0/3956182603 learned_addr learned my addr 172.21.15.83:0/3956182603 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.752+0000 7f72b7e00640 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.752+0000 7f72b7400640 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.752+0000 7f72bcc00640 1 -- 172.21.15.83:0/3956182603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 msgr2=0x562a99b955e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.752+0000 7f72bcc00640 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b955e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.752+0000 7f72bcc00640 1 -- 172.21.15.83:0/3956182603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 msgr2=0x562a99b928d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.752+0000 7f72bcc00640 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b928d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.752+0000 7f72bcc00640 1 -- 172.21.15.83:0/3956182603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a99aba3f0 con 0x562a99b95b30 2024-04-23T14:58:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.753+0000 7f72bcc00640 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b93010 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f72b0000f80 tx=0x7f72b00048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bbba48ba99f8a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.753+0000 7f72a1600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.753+0000 7f72a1600640 1 -- 172.21.15.83:0/3956182603 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0027020 con 0x562a99b95b30 2024-04-23T14:58:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.753+0000 7f72a1600640 1 -- 172.21.15.83:0/3956182603 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f72b0004050 con 0x562a99b95b30 2024-04-23T14:58:02.762 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.753+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956182603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b0003c30 con 0x562a99b95b30 2024-04-23T14:58:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.753+0000 7f72a1600640 1 -- 172.21.15.83:0/3956182603 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72b0032050 con 0x562a99b95b30 2024-04-23T14:58:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.753+0000 7f72c19f7740 10 client.16902 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:02.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.753+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956182603 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562a998023f0 con 0x562a99b95b30 2024-04-23T14:58:02.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.753+0000 7f72a1600640 1 -- 172.21.15.83:0/3956182603 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72b0003c30 con 0x562a99b95b30 2024-04-23T14:58:02.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.754+0000 7f72a1600640 1 -- 172.21.15.83:0/3956182603 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f72b00051f0 con 0x562a99b95b30 2024-04-23T14:58:02.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.754+0000 7f72c19f7740 10 client.16902 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:02.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.754+0000 7f72c19f7740 10 client.16902 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:58:02.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.754+0000 7f72c19f7740 20 client.16902 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:02.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.754+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x562a99ba5ca0 0x562a99ba8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.754+0000 7f72c19f7740 4 client.16902 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:58:02.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.754+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956182603 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a99afdbf0 con 0x562a99ba5ca0 2024-04-23T14:58:02.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.754+0000 7f72b7e00640 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x562a99ba5ca0 0x562a99ba8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.755+0000 7f72b7e00640 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x562a99ba5ca0 0x562a99ba8090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a77f8284aaa840cd server_cookie=b28713933a12e774 in_seq=0 out_seq=0 2024-04-23T14:58:02.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.755+0000 7f72a1600640 10 client.16902 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:58:02.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72a1600640 1 -- 172.21.15.83:0/3956182603 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a99afdbf0 con 0x562a99ba5ca0 2024-04-23T14:58:02.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72a1600640 10 client.16902 handle_command_reply: tid=0 2024-04-23T14:58:02.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72c19f7740 1 client.16902 shutdown 2024-04-23T14:58:02.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956182603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 msgr2=0x562a99b93010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b93010 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f72b0000f80 tx=0x7f72b00048a0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956182603 shutdown_connections 2024-04-23T14:58:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x562a99ba5ca0 0x562a99ba8090 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b93010 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b928d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3956182603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b955e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72a1600640 0 client.16902 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:58:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956182603 >> 172.21.15.83:0/3956182603 conn(0x562a99b86e60 msgr2=0x562a99b8cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956182603 shutdown_connections 2024-04-23T14:58:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.756+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956182603 wait complete. 2024-04-23T14:58:02.768 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.757+0000 7f72c19f7740 20 client.16902 trim_cache size 0 max 16384 2024-04-23T14:58:02.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.771+0000 7f72c19f7740 1 Processor -- start 2024-04-23T14:58:02.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.771+0000 7f72c19f7740 1 -- start start 2024-04-23T14:58:02.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.771+0000 7f72c19f7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b8ab40 0x562a99ba18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.771+0000 7f72c19f7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.771+0000 7f72c19f7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b95b30 0x562a99b95110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.771+0000 7f72c19f7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81b2700 con 0x562a99b8ab40 2024-04-23T14:58:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.771+0000 7f72c19f7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81affd0 con 0x562a99b8b460 2024-04-23T14:58:02.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.771+0000 7f72c19f7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81ade40 con 0x562a99b95b30 2024-04-23T14:58:02.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.772+0000 7f72b7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b8ab40 0x562a99ba18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.772+0000 7f72b7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b8ab40 0x562a99ba18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46586/0 (socket says 172.21.15.83:46586) 2024-04-23T14:58:02.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.772+0000 7f72b7e00640 1 -- 172.21.15.83:0/2963052711 learned_addr learned my addr 172.21.15.83:0/2963052711 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:02.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.772+0000 7f72b7400640 1 --2- 172.21.15.83:0/2963052711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.772+0000 7f72bcc00640 1 --2- 172.21.15.83:0/2963052711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b95b30 0x562a99b95110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.772+0000 7f72b7e00640 1 -- 172.21.15.83:0/2963052711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b95b30 msgr2=0x562a99b95110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.772+0000 7f72b7e00640 1 --2- 172.21.15.83:0/2963052711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b95b30 0x562a99b95110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.772+0000 7f72b7e00640 1 -- 172.21.15.83:0/2963052711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 msgr2=0x562a99b918b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.772+0000 7f72b7e00640 1 --2- 172.21.15.83:0/2963052711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b918b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.772+0000 7f72b7e00640 1 -- 172.21.15.83:0/2963052711 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a998cc7d0 con 0x562a99b8ab40 2024-04-23T14:58:02.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.772+0000 7f72bcc00640 1 --2- 172.21.15.83:0/2963052711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b95b30 0x562a99b95110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:58:02.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.772+0000 7f72b7e00640 1 --2- 172.21.15.83:0/2963052711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b8ab40 0x562a99ba18b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009fd0 tx=0x7f72a801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a61476607ae289 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.773+0000 7f72a3e00640 1 -- 172.21.15.83:0/2963052711 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a802b530 con 0x562a99b8ab40 2024-04-23T14:58:02.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.773+0000 7f72a3e00640 1 -- 172.21.15.83:0/2963052711 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a8030070 con 0x562a99b8ab40 2024-04-23T14:58:02.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.773+0000 7f72a3e00640 1 -- 172.21.15.83:0/2963052711 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a801f690 con 0x562a99b8ab40 2024-04-23T14:58:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.773+0000 7f72c19f7740 1 -- 172.21.15.83:0/2963052711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b8ab40 msgr2=0x562a99ba18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.773+0000 7f72c19f7740 1 --2- 172.21.15.83:0/2963052711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b8ab40 0x562a99ba18b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009fd0 tx=0x7f72a801b000 comp rx=0 tx=0).stop 2024-04-23T14:58:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.773+0000 7f72c19f7740 1 -- 172.21.15.83:0/2963052711 shutdown_connections 2024-04-23T14:58:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.773+0000 7f72c19f7740 1 --2- 172.21.15.83:0/2963052711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b95b30 0x562a99b95110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.773+0000 7f72c19f7740 1 --2- 172.21.15.83:0/2963052711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b918b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.773+0000 7f72c19f7740 1 --2- 172.21.15.83:0/2963052711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b8ab40 0x562a99ba18b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.773+0000 7f72c19f7740 1 -- 172.21.15.83:0/2963052711 >> 172.21.15.83:0/2963052711 conn(0x562a99b86e60 msgr2=0x562a99b911a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.774+0000 7f72c19f7740 1 -- 172.21.15.83:0/2963052711 shutdown_connections 2024-04-23T14:58:02.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.774+0000 7f72c19f7740 1 -- 172.21.15.83:0/2963052711 wait complete. 2024-04-23T14:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.774+0000 7f72c19f7740 1 -- start start 2024-04-23T14:58:02.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.774+0000 7f72c19f7740 1 Processor -- start 2024-04-23T14:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72c19f7740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b96320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72c19f7740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b96860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72c19f7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b96de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72c19f7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81ae490 con 0x562a99b95b30 2024-04-23T14:58:02.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72c19f7740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81af900 con 0x562a99b8b460 2024-04-23T14:58:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72c19f7740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81aeb60 con 0x562a99b8ab40 2024-04-23T14:58:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72c19f7740 10 client.0 _ll_register_callbacks cb 0x7f72b001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72bcc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b96de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72b7e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b96320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72b7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b96860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72b7e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b96320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:52460/0 (socket says 172.21.15.83:52460) 2024-04-23T14:58:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72bcc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b96de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46588/0 (socket says 172.21.15.83:46588) 2024-04-23T14:58:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72b7e00640 1 -- 172.21.15.83:0/3956501843 learned_addr learned my addr 172.21.15.83:0/3956501843 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:02.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72b7e00640 1 -- 172.21.15.83:0/3956501843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 msgr2=0x562a99b96860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72b7e00640 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b96860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72b7e00640 1 -- 172.21.15.83:0/3956501843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 msgr2=0x562a99b96de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72b7e00640 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b96de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72b7e00640 1 -- 172.21.15.83:0/3956501843 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a99b9d100 con 0x562a99b8ab40 2024-04-23T14:58:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72bcc00640 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b96de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.775+0000 7f72b7400640 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b96860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.776+0000 7f72b7e00640 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b96320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f72a8004250 tx=0x7f72a800fe50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c9c7165f340aa8b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.776+0000 7f72a1600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:58:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.776+0000 7f72a1600640 1 -- 172.21.15.83:0/3956501843 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a802b870 con 0x562a99b8ab40 2024-04-23T14:58:02.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.776+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956501843 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72a8030070 con 0x562a99b8ab40 2024-04-23T14:58:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.776+0000 7f72a1600640 1 -- 172.21.15.83:0/3956501843 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f72a800f4e0 con 0x562a99b8ab40 2024-04-23T14:58:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.777+0000 7f72a1600640 1 -- 172.21.15.83:0/3956501843 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72a800f6e0 con 0x562a99b8ab40 2024-04-23T14:58:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.777+0000 7f72c19f7740 10 client.27085 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.777+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956501843 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562a99b9e310 con 0x562a99b8ab40 2024-04-23T14:58:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.777+0000 7f72a1600640 1 -- 172.21.15.83:0/3956501843 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a8030070 con 0x562a99b8ab40 2024-04-23T14:58:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.777+0000 7f72a1600640 1 -- 172.21.15.83:0/3956501843 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f72a803e410 con 0x562a99b8ab40 2024-04-23T14:58:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.777+0000 7f72c19f7740 10 client.27085 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:02.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.777+0000 7f72c19f7740 10 client.27085 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.777+0000 7f72c19f7740 20 client.27085 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.777+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x562a99c0c220 0x562a99c0e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.777+0000 7f72c19f7740 4 client.27085 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.777+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956501843 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a99af7a60 con 0x562a99c0c220 2024-04-23T14:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.778+0000 7f72b7400640 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x562a99c0c220 0x562a99c0e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.778+0000 7f72a1600640 10 client.27085 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:58:02.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.778+0000 7f72b7400640 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x562a99c0c220 0x562a99c0e610 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55d51af6dc678f0a server_cookie=addb624b04b98a34 in_seq=0 out_seq=0 2024-04-23T14:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.778+0000 7f72a1600640 1 -- 172.21.15.83:0/3956501843 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a99af7a60 con 0x562a99c0c220 2024-04-23T14:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.778+0000 7f72a1600640 10 client.27085 handle_command_reply: tid=0 2024-04-23T14:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.778+0000 7f72c19f7740 1 client.27085 shutdown 2024-04-23T14:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956501843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 msgr2=0x562a99b96320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b96320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f72a8004250 tx=0x7f72a800fe50 comp rx=0 tx=0).stop 2024-04-23T14:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956501843 shutdown_connections 2024-04-23T14:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x562a99c0c220 0x562a99c0e610 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562a99b95b30 0x562a99b96de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.786 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562a99b8b460 0x562a99b96860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72c19f7740 1 --2- 172.21.15.83:0/3956501843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562a99b8ab40 0x562a99b96320 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72a1600640 0 client.27085 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:58:02.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956501843 >> 172.21.15.83:0/3956501843 conn(0x562a99b86e60 msgr2=0x562a99ba08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956501843 shutdown_connections 2024-04-23T14:58:02.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72c19f7740 1 -- 172.21.15.83:0/3956501843 wait complete. 2024-04-23T14:58:02.787 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72c19f7740 20 client.27085 trim_cache size 0 max 16384 2024-04-23T14:58:02.788 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:58:02.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72bd600640 1 -- 172.21.15.83:0/3035215681 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f729c0a3da0 msgr2=0x7f729c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.788 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72bd600640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f729c0a3da0 0x7f729c0a6260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009d80 tx=0x7f72a8005f90 comp rx=0 tx=0).stop 2024-04-23T14:58:02.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72bd600640 1 -- 172.21.15.83:0/3035215681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b81071c0 msgr2=0x7f72b81a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.779+0000 7f72bd600640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b81071c0 0x7f72b81a0ba0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f72a4005940 tx=0x7f72a40040c0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.780+0000 7f72bd600640 1 -- 172.21.15.83:0/3035215681 shutdown_connections 2024-04-23T14:58:02.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.780+0000 7f72bd600640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72b819a8b0 0x7f72b819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.780+0000 7f72bd600640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f729c0a3da0 0x7f729c0a6260 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.780+0000 7f72bd600640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72b81071c0 0x7f72b81a0ba0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.780+0000 7f72bd600640 1 --2- 172.21.15.83:0/3035215681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72b81020f0 0x7f72b81a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.780+0000 7f72bd600640 1 -- 172.21.15.83:0/3035215681 >> 172.21.15.83:0/3035215681 conn(0x7f72b80fd9d0 msgr2=0x7f72b80fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.780+0000 7f72bd600640 1 -- 172.21.15.83:0/3035215681 shutdown_connections 2024-04-23T14:58:02.789 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.780+0000 7f72bd600640 1 -- 172.21.15.83:0/3035215681 wait complete. 2024-04-23T14:58:02.795 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:02 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1934335707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:58:02.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:58:02.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:02 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1934335707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:58:02.974 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.973+0000 7fad88c00640 1 -- 172.21.15.83:0/1246579519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad84104cd0 msgr2=0x7fad84108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.973+0000 7fad88c00640 1 --2- 172.21.15.83:0/1246579519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad84104cd0 0x7fad84108d10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009900 tx=0x7fad6c02a410 comp rx=0 tx=0).stop 2024-04-23T14:58:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.973+0000 7fad88c00640 1 -- 172.21.15.83:0/1246579519 shutdown_connections 2024-04-23T14:58:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.973+0000 7fad88c00640 1 --2- 172.21.15.83:0/1246579519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad841093f0 0x7fad84110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.973+0000 7fad88c00640 1 --2- 172.21.15.83:0/1246579519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad84104cd0 0x7fad84108d10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.973+0000 7fad88c00640 1 --2- 172.21.15.83:0/1246579519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad84104320 0x7fad84104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.973+0000 7fad88c00640 1 -- 172.21.15.83:0/1246579519 >> 172.21.15.83:0/1246579519 conn(0x7fad840ffb40 msgr2=0x7fad84101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.975 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.974+0000 7fad88c00640 1 -- 172.21.15.83:0/1246579519 shutdown_connections 2024-04-23T14:58:02.976 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.974+0000 7fad88c00640 1 -- 172.21.15.83:0/1246579519 wait complete. 2024-04-23T14:58:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.974+0000 7fad88c00640 1 Processor -- start 2024-04-23T14:58:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad88c00640 1 -- start start 2024-04-23T14:58:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad88c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad84104320 0x7fad841a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad88c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad841093f0 0x7fad841a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.978 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad88c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8419a840 0x7fad8419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad88c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad84114460 con 0x7fad8419a840 2024-04-23T14:58:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad88c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fad841142e0 con 0x7fad84104320 2024-04-23T14:58:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad88c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fad841145e0 con 0x7fad841093f0 2024-04-23T14:58:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8419a840 0x7fad8419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad7be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad841093f0 0x7fad841a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad7be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad841093f0 0x7fad841a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:52474/0 (socket says 172.21.15.83:52474) 2024-04-23T14:58:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad83400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad84104320 0x7fad841a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.979 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad7be00640 1 -- 172.21.15.83:0/1207084692 learned_addr learned my addr 172.21.15.83:0/1207084692 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:02.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.975+0000 7fad83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8419a840 0x7fad8419acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46596/0 (socket says 172.21.15.83:46596) 2024-04-23T14:58:02.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.976+0000 7fad83400640 1 -- 172.21.15.83:0/1207084692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad841093f0 msgr2=0x7fad841a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.976+0000 7fad83400640 1 --2- 172.21.15.83:0/1207084692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad841093f0 0x7fad841a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.976+0000 7fad83400640 1 -- 172.21.15.83:0/1207084692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8419a840 msgr2=0x7fad8419acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.976+0000 7fad83400640 1 --2- 172.21.15.83:0/1207084692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8419a840 0x7fad8419acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.976+0000 7fad83400640 1 -- 172.21.15.83:0/1207084692 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8419b3d0 con 0x7fad84104320 2024-04-23T14:58:02.980 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.976+0000 7fad83400640 1 --2- 172.21.15.83:0/1207084692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad84104320 0x7fad841a0680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fad7400e9f0 tx=0x7fad7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.976+0000 7fad80c00640 1 -- 172.21.15.83:0/1207084692 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7400ccc0 con 0x7fad84104320 2024-04-23T14:58:02.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.976+0000 7fad88c00640 1 -- 172.21.15.83:0/1207084692 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8419b6c0 con 0x7fad84104320 2024-04-23T14:58:02.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.976+0000 7fad80c00640 1 -- 172.21.15.83:0/1207084692 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad74010910 con 0x7fad84104320 2024-04-23T14:58:02.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.976+0000 7fad80c00640 1 -- 172.21.15.83:0/1207084692 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74018960 con 0x7fad84104320 2024-04-23T14:58:02.981 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.976+0000 7fad88c00640 1 -- 172.21.15.83:0/1207084692 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad841a78a0 con 0x7fad84104320 2024-04-23T14:58:02.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.979+0000 7fad8cf84740 1 Processor -- start 2024-04-23T14:58:02.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.979+0000 7fad8cf84740 1 -- start start 2024-04-23T14:58:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.979+0000 7fad80c00640 1 -- 172.21.15.83:0/1207084692 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fad74010ac0 con 0x7fad84104320 2024-04-23T14:58:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad8cf84740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 0x55ffe7af5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad8cf84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af6460 0x55ffe7b005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad8cf84740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7b00b30 0x55ffe7b02f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad8cf84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7a5b1d0 con 0x55ffe7af6460 2024-04-23T14:58:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad8cf84740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7a01450 con 0x55ffe7af5b40 2024-04-23T14:58:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad8cf84740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7a57190 con 0x55ffe7b00b30 2024-04-23T14:58:02.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad83e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 0x55ffe7af5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad83400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af6460 0x55ffe7b005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad83e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 0x55ffe7af5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:55744/0 (socket says 172.21.15.83:55744) 2024-04-23T14:58:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad83e00640 1 -- 172.21.15.83:0/4135626653 learned_addr learned my addr 172.21.15.83:0/4135626653 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad7be00640 1 --2- 172.21.15.83:0/4135626653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7b00b30 0x55ffe7b02f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad83e00640 1 -- 172.21.15.83:0/4135626653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7b00b30 msgr2=0x55ffe7b02f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad83e00640 1 --2- 172.21.15.83:0/4135626653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7b00b30 0x55ffe7b02f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad83e00640 1 -- 172.21.15.83:0/4135626653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af6460 msgr2=0x55ffe7b005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad83e00640 1 --2- 172.21.15.83:0/4135626653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af6460 0x55ffe7b005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.980+0000 7fad83e00640 1 -- 172.21.15.83:0/4135626653 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe7abe310 con 0x55ffe7af5b40 2024-04-23T14:58:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.981+0000 7fad83e00640 1 --2- 172.21.15.83:0/4135626653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 0x55ffe7af5f20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009f90 tx=0x7fad7c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb47b006b1cff8da server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.981+0000 7fad73e00640 1 -- 172.21.15.83:0/4135626653 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c011700 con 0x55ffe7af5b40 2024-04-23T14:58:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.981+0000 7fad80c00640 1 --2- 172.21.15.83:0/1207084692 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fad680a3d20 0x7fad680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.981+0000 7fad73e00640 1 -- 172.21.15.83:0/4135626653 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad7c011d80 con 0x55ffe7af5b40 2024-04-23T14:58:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.981+0000 7fad73e00640 1 -- 172.21.15.83:0/4135626653 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c019e40 con 0x55ffe7af5b40 2024-04-23T14:58:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.981+0000 7fad80c00640 1 -- 172.21.15.83:0/1207084692 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fad740d0b60 con 0x7fad84104320 2024-04-23T14:58:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.981+0000 7fad83400640 1 --2- 172.21.15.83:0/1207084692 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fad680a3d20 0x7fad680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.982+0000 7fad8cf84740 1 -- 172.21.15.83:0/4135626653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 msgr2=0x55ffe7af5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.982+0000 7fad8cf84740 1 --2- 172.21.15.83:0/4135626653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 0x55ffe7af5f20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009f90 tx=0x7fad7c010860 comp rx=0 tx=0).stop 2024-04-23T14:58:02.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.982+0000 7fad8cf84740 1 -- 172.21.15.83:0/4135626653 shutdown_connections 2024-04-23T14:58:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.982+0000 7fad8cf84740 1 --2- 172.21.15.83:0/4135626653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7b00b30 0x55ffe7b02f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.982+0000 7fad8cf84740 1 --2- 172.21.15.83:0/4135626653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af6460 0x55ffe7b005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.982+0000 7fad8cf84740 1 --2- 172.21.15.83:0/4135626653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 0x55ffe7af5f20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.982+0000 7fad8cf84740 1 -- 172.21.15.83:0/4135626653 >> 172.21.15.83:0/4135626653 conn(0x55ffe7af1e60 msgr2=0x55ffe7af2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.982+0000 7fad8cf84740 1 -- 172.21.15.83:0/4135626653 shutdown_connections 2024-04-23T14:58:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.982+0000 7fad8cf84740 1 -- 172.21.15.83:0/4135626653 wait complete. 2024-04-23T14:58:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.982+0000 7fad83400640 1 --2- 172.21.15.83:0/1207084692 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fad680a3d20 0x7fad680a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fad740ce350 tx=0x7fad740d46b0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.982+0000 7fad8cf84740 1 -- start start 2024-04-23T14:58:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.982+0000 7fad8cf84740 1 Processor -- start 2024-04-23T14:58:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.983+0000 7fad8cf84740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 0x55ffe7b005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.983+0000 7fad8cf84740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af6460 0x55ffe7afd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.983+0000 7fad8cf84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7b00b30 0x55ffe7afe020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.983+0000 7fad8cf84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7b039c0 con 0x55ffe7b00b30 2024-04-23T14:58:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.983+0000 7fad8cf84740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7a14810 con 0x55ffe7af5b40 2024-04-23T14:58:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.983+0000 7fad8cf84740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ffe7b03b40 con 0x55ffe7af6460 2024-04-23T14:58:02.991 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.983+0000 7fad8cf84740 10 client.0 _ll_register_callbacks cb 0x55ffe79e2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.983+0000 7fad83e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 0x55ffe7b005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.983+0000 7fad83e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 0x55ffe7b005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:55752/0 (socket says 172.21.15.83:55752) 2024-04-23T14:58:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.983+0000 7fad83e00640 1 -- 172.21.15.83:0/755089430 learned_addr learned my addr 172.21.15.83:0/755089430 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.983+0000 7fad7be00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af6460 0x55ffe7afd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.983+0000 7fad83e00640 1 --2- 172.21.15.83:0/755089430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7b00b30 0x55ffe7afe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.984+0000 7fad7be00640 1 -- 172.21.15.83:0/755089430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 msgr2=0x55ffe7b005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.984+0000 7fad7be00640 1 --2- 172.21.15.83:0/755089430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 0x55ffe7b005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.984+0000 7fad7be00640 1 -- 172.21.15.83:0/755089430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7b00b30 msgr2=0x55ffe7afe020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.984+0000 7fad7be00640 1 --2- 172.21.15.83:0/755089430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7b00b30 0x55ffe7afe020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.984+0000 7fad7be00640 1 -- 172.21.15.83:0/755089430 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe7a253f0 con 0x55ffe7af6460 2024-04-23T14:58:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.984+0000 7fad7be00640 1 --2- 172.21.15.83:0/755089430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af6460 0x55ffe7afd8e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fad6c000c00 tx=0x7fad6c031850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d425f922c1859fc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.984+0000 7fad70c00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:58:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.984+0000 7fad70c00640 1 -- 172.21.15.83:0/755089430 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c035040 con 0x55ffe7af6460 2024-04-23T14:58:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.984+0000 7fad70c00640 1 -- 172.21.15.83:0/755089430 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fad6c004240 con 0x55ffe7af6460 2024-04-23T14:58:02.993 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.984+0000 7fad8cf84740 1 -- 172.21.15.83:0/755089430 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad7c011d80 con 0x55ffe7af6460 2024-04-23T14:58:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.985+0000 7fad70c00640 1 -- 172.21.15.83:0/755089430 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad6c004440 con 0x55ffe7af6460 2024-04-23T14:58:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.985+0000 7fad8cf84740 10 client.27097 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.985+0000 7fad8cf84740 1 -- 172.21.15.83:0/755089430 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffe776d3f0 con 0x55ffe7af6460 2024-04-23T14:58:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.985+0000 7fad70c00640 1 -- 172.21.15.83:0/755089430 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad7c011d80 con 0x55ffe7af6460 2024-04-23T14:58:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.985+0000 7fad70c00640 1 -- 172.21.15.83:0/755089430 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fad6c046620 con 0x55ffe7af6460 2024-04-23T14:58:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.985+0000 7fad8cf84740 10 client.27097 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.985+0000 7fad8cf84740 10 client.27097 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:58:02.994 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.985+0000 7fad8cf84740 20 client.27097 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.985+0000 7fad8cf84740 1 --2- 172.21.15.83:0/755089430 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ffe7b10d60 0x55ffe7b13170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.985+0000 7fad8cf84740 4 client.27097 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:58:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.985+0000 7fad8cf84740 1 -- 172.21.15.83:0/755089430 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffe7a68bf0 con 0x55ffe7b10d60 2024-04-23T14:58:02.995 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.986+0000 7fad83e00640 1 --2- 172.21.15.83:0/755089430 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ffe7b10d60 0x55ffe7b13170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.986+0000 7fad83e00640 1 --2- 172.21.15.83:0/755089430 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ffe7b10d60 0x55ffe7b13170 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=404c7262b4a2c086 server_cookie=578391579f692a28 in_seq=0 out_seq=0 2024-04-23T14:58:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.986+0000 7fad70c00640 10 client.27097 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:58:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad70c00640 1 -- 172.21.15.83:0/755089430 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ffe7a68bf0 con 0x55ffe7b10d60 2024-04-23T14:58:02.996 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad70c00640 10 client.27097 handle_command_reply: tid=0 2024-04-23T14:58:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad8cf84740 1 client.27097 shutdown 2024-04-23T14:58:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad8cf84740 1 -- 172.21.15.83:0/755089430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af6460 msgr2=0x55ffe7afd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad8cf84740 1 --2- 172.21.15.83:0/755089430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af6460 0x55ffe7afd8e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fad6c000c00 tx=0x7fad6c031850 comp rx=0 tx=0).stop 2024-04-23T14:58:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad8cf84740 1 -- 172.21.15.83:0/755089430 shutdown_connections 2024-04-23T14:58:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad8cf84740 1 --2- 172.21.15.83:0/755089430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7b00b30 0x55ffe7afe020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad8cf84740 1 --2- 172.21.15.83:0/755089430 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ffe7b10d60 0x55ffe7b13170 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.997 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad8cf84740 1 --2- 172.21.15.83:0/755089430 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af6460 0x55ffe7afd8e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad70c00640 0 client.27097 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:58:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad8cf84740 1 --2- 172.21.15.83:0/755089430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7af5b40 0x55ffe7b005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad8cf84740 1 -- 172.21.15.83:0/755089430 >> 172.21.15.83:0/755089430 conn(0x55ffe7af1e60 msgr2=0x55ffe7af7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad8cf84740 1 -- 172.21.15.83:0/755089430 shutdown_connections 2024-04-23T14:58:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.987+0000 7fad8cf84740 1 -- 172.21.15.83:0/755089430 wait complete. 2024-04-23T14:58:02.998 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:02.988+0000 7fad8cf84740 20 client.27097 trim_cache size 0 max 16384 2024-04-23T14:58:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad8cf84740 1 Processor -- start 2024-04-23T14:58:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad8cf84740 1 -- start start 2024-04-23T14:58:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad8cf84740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af5b40 0x55ffe7afd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad8cf84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af6460 0x55ffe7b0c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad8cf84740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7b00b30 0x55ffe7b00120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.005 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad8cf84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad841b26c0 con 0x55ffe7af6460 2024-04-23T14:58:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad8cf84740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fad841aff90 con 0x55ffe7b00b30 2024-04-23T14:58:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad8cf84740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fad841ade00 con 0x55ffe7af5b40 2024-04-23T14:58:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad7be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af6460 0x55ffe7b0c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad83e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af5b40 0x55ffe7afd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad7be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af6460 0x55ffe7b0c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46616/0 (socket says 172.21.15.83:46616) 2024-04-23T14:58:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad7be00640 1 -- 172.21.15.83:0/517041248 learned_addr learned my addr 172.21.15.83:0/517041248 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.002+0000 7fad83e00640 1 --2- 172.21.15.83:0/517041248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7b00b30 0x55ffe7b00120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.006 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.003+0000 7fad83e00640 1 -- 172.21.15.83:0/517041248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af5b40 msgr2=0x55ffe7afd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.003+0000 7fad83e00640 1 --2- 172.21.15.83:0/517041248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af5b40 0x55ffe7afd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.003+0000 7fad83e00640 1 -- 172.21.15.83:0/517041248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af6460 msgr2=0x55ffe7b0c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.003+0000 7fad83e00640 1 --2- 172.21.15.83:0/517041248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af6460 0x55ffe7b0c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.003+0000 7fad83e00640 1 -- 172.21.15.83:0/517041248 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe78377d0 con 0x55ffe7b00b30 2024-04-23T14:58:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.003+0000 7fad7be00640 1 --2- 172.21.15.83:0/517041248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af6460 0x55ffe7b0c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.003+0000 7fad83e00640 1 --2- 172.21.15.83:0/517041248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7b00b30 0x55ffe7b00120 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55ffe7b139e0 tx=0x7fad7c010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b1b3540021de46d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.003+0000 7fad73400640 1 -- 172.21.15.83:0/517041248 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c019490 con 0x55ffe7b00b30 2024-04-23T14:58:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.003+0000 7fad73400640 1 -- 172.21.15.83:0/517041248 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad7c020040 con 0x55ffe7b00b30 2024-04-23T14:58:03.007 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.003+0000 7fad73400640 1 -- 172.21.15.83:0/517041248 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0055e0 con 0x55ffe7b00b30 2024-04-23T14:58:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.004+0000 7fad8cf84740 1 -- 172.21.15.83:0/517041248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7b00b30 msgr2=0x55ffe7b00120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.004+0000 7fad8cf84740 1 --2- 172.21.15.83:0/517041248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7b00b30 0x55ffe7b00120 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55ffe7b139e0 tx=0x7fad7c010fd0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.004+0000 7fad8cf84740 1 -- 172.21.15.83:0/517041248 shutdown_connections 2024-04-23T14:58:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.004+0000 7fad8cf84740 1 --2- 172.21.15.83:0/517041248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7b00b30 0x55ffe7b00120 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.004+0000 7fad8cf84740 1 --2- 172.21.15.83:0/517041248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af6460 0x55ffe7b0c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.004+0000 7fad8cf84740 1 --2- 172.21.15.83:0/517041248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af5b40 0x55ffe7afd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.004+0000 7fad8cf84740 1 -- 172.21.15.83:0/517041248 >> 172.21.15.83:0/517041248 conn(0x55ffe7af1e60 msgr2=0x55ffe7afd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:03.008 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.004+0000 7fad8cf84740 1 -- 172.21.15.83:0/517041248 shutdown_connections 2024-04-23T14:58:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.004+0000 7fad8cf84740 1 -- 172.21.15.83:0/517041248 wait complete. 2024-04-23T14:58:03.009 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.004+0000 7fad8cf84740 1 -- start start 2024-04-23T14:58:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad8cf84740 1 Processor -- start 2024-04-23T14:58:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad8cf84740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af5b40 0x55ffe7b01720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad8cf84740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af6460 0x55ffe7b01c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad8cf84740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7b00b30 0x55ffe7b07d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad8cf84740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad841ae450 con 0x55ffe7af5b40 2024-04-23T14:58:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad8cf84740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fad841af8c0 con 0x55ffe7b00b30 2024-04-23T14:58:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad8cf84740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fad841aeb20 con 0x55ffe7af6460 2024-04-23T14:58:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad8cf84740 10 client.0 _ll_register_callbacks cb 0x7fad6c005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af5b40 0x55ffe7b01720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad83e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af5b40 0x55ffe7b01720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46630/0 (socket says 172.21.15.83:46630) 2024-04-23T14:58:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad83e00640 1 -- 172.21.15.83:0/2635196218 learned_addr learned my addr 172.21.15.83:0/2635196218 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:03.013 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad7be00640 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af6460 0x55ffe7b01c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.005+0000 7fad83e00640 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7b00b30 0x55ffe7b07d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad83e00640 1 -- 172.21.15.83:0/2635196218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af6460 msgr2=0x55ffe7b01c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad83e00640 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af6460 0x55ffe7b01c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad83e00640 1 -- 172.21.15.83:0/2635196218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7b00b30 msgr2=0x55ffe7b07d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad83e00640 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7b00b30 0x55ffe7b07d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad83e00640 1 -- 172.21.15.83:0/2635196218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe7b082d0 con 0x55ffe7af5b40 2024-04-23T14:58:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad83e00640 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af5b40 0x55ffe7b01720 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fad7c010d60 tx=0x7fad7c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11212bb0ecf5dce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad70c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:03.014 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad70c00640 1 -- 172.21.15.83:0/2635196218 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c019590 con 0x55ffe7af5b40 2024-04-23T14:58:03.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad70c00640 1 -- 172.21.15.83:0/2635196218 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fad7c01f070 con 0x55ffe7af5b40 2024-04-23T14:58:03.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad8cf84740 1 -- 172.21.15.83:0/2635196218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad7c020040 con 0x55ffe7af5b40 2024-04-23T14:58:03.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad70c00640 1 -- 172.21.15.83:0/2635196218 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad7c03f050 con 0x55ffe7af5b40 2024-04-23T14:58:03.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad8cf84740 10 client.16926 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:03.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.006+0000 7fad8cf84740 1 -- 172.21.15.83:0/2635196218 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffe7b09550 con 0x55ffe7af5b40 2024-04-23T14:58:03.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.007+0000 7fad70c00640 1 -- 172.21.15.83:0/2635196218 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad7c020040 con 0x55ffe7af5b40 2024-04-23T14:58:03.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.007+0000 7fad70c00640 1 -- 172.21.15.83:0/2635196218 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fad7c005640 con 0x55ffe7af5b40 2024-04-23T14:58:03.015 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.007+0000 7fad8cf84740 10 client.16926 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.007+0000 7fad8cf84740 10 client.16926 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:58:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.007+0000 7fad8cf84740 20 client.16926 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.007+0000 7fad8cf84740 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ffe7b77460 0x55ffe7b79850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.007+0000 7fad8cf84740 4 client.16926 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:58:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.007+0000 7fad8cf84740 1 -- 172.21.15.83:0/2635196218 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffe7a62a60 con 0x55ffe7b77460 2024-04-23T14:58:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.007+0000 7fad7be00640 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ffe7b77460 0x55ffe7b79850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad7be00640 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ffe7b77460 0x55ffe7b79850 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=678a92eed74036e4 server_cookie=4d491825be7d8136 in_seq=0 out_seq=0 2024-04-23T14:58:03.016 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad70c00640 10 client.16926 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:58:03.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad70c00640 1 -- 172.21.15.83:0/2635196218 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffe7a62a60 con 0x55ffe7b77460 2024-04-23T14:58:03.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad70c00640 10 client.16926 handle_command_reply: tid=0 2024-04-23T14:58:03.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad8cf84740 1 client.16926 shutdown 2024-04-23T14:58:03.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad8cf84740 1 -- 172.21.15.83:0/2635196218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af5b40 msgr2=0x55ffe7b01720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad8cf84740 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af5b40 0x55ffe7b01720 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fad7c010d60 tx=0x7fad7c000f40 comp rx=0 tx=0).stop 2024-04-23T14:58:03.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad8cf84740 1 -- 172.21.15.83:0/2635196218 shutdown_connections 2024-04-23T14:58:03.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad8cf84740 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ffe7b00b30 0x55ffe7b07d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.017 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad8cf84740 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55ffe7b77460 0x55ffe7b79850 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad8cf84740 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ffe7af6460 0x55ffe7b01c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad8cf84740 1 --2- 172.21.15.83:0/2635196218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffe7af5b40 0x55ffe7b01720 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad70c00640 0 client.16926 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:58:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.008+0000 7fad8cf84740 1 -- 172.21.15.83:0/2635196218 >> 172.21.15.83:0/2635196218 conn(0x55ffe7af1e60 msgr2=0x55ffe7af06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad8cf84740 1 -- 172.21.15.83:0/2635196218 shutdown_connections 2024-04-23T14:58:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad8cf84740 1 -- 172.21.15.83:0/2635196218 wait complete. 2024-04-23T14:58:03.018 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad8cf84740 20 client.16926 trim_cache size 0 max 16384 2024-04-23T14:58:03.019 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:58:03.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad88c00640 1 -- 172.21.15.83:0/1207084692 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fad680a3d20 msgr2=0x7fad680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.019 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad88c00640 1 --2- 172.21.15.83:0/1207084692 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fad680a3d20 0x7fad680a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fad740ce350 tx=0x7fad740d46b0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad88c00640 1 -- 172.21.15.83:0/1207084692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad84104320 msgr2=0x7fad841a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad88c00640 1 --2- 172.21.15.83:0/1207084692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad84104320 0x7fad841a0680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fad7400e9f0 tx=0x7fad7400eeb0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad88c00640 1 -- 172.21.15.83:0/1207084692 shutdown_connections 2024-04-23T14:58:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad88c00640 1 --2- 172.21.15.83:0/1207084692 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fad680a3d20 0x7fad680a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad88c00640 1 --2- 172.21.15.83:0/1207084692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad8419a840 0x7fad8419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad88c00640 1 --2- 172.21.15.83:0/1207084692 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fad841093f0 0x7fad841a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad88c00640 1 --2- 172.21.15.83:0/1207084692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad84104320 0x7fad841a0680 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.020 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad88c00640 1 -- 172.21.15.83:0/1207084692 >> 172.21.15.83:0/1207084692 conn(0x7fad840ffb40 msgr2=0x7fad84100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:03.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad88c00640 1 -- 172.21.15.83:0/1207084692 shutdown_connections 2024-04-23T14:58:03.021 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.009+0000 7fad88c00640 1 -- 172.21.15.83:0/1207084692 wait complete. 2024-04-23T14:58:03.027 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:58:03.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:02 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1934335707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:58:03.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.227+0000 7fe803e00640 1 -- 172.21.15.83:0/134026698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe804108580 msgr2=0x7fe804108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.227+0000 7fe803e00640 1 --2- 172.21.15.83:0/134026698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe804108580 0x7fe804108a00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009930 tx=0x7fe7ec02a420 comp rx=0 tx=0).stop 2024-04-23T14:58:03.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.228+0000 7fe803e00640 1 -- 172.21.15.83:0/134026698 shutdown_connections 2024-04-23T14:58:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.228+0000 7fe803e00640 1 --2- 172.21.15.83:0/134026698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe804108f40 0x7fe8041153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.228+0000 7fe803e00640 1 --2- 172.21.15.83:0/134026698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe804108580 0x7fe804108a00 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.228+0000 7fe803e00640 1 --2- 172.21.15.83:0/134026698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe80410d500 0x7fe80410d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.228+0000 7fe803e00640 1 -- 172.21.15.83:0/134026698 >> 172.21.15.83:0/134026698 conn(0x7fe804077e20 msgr2=0x7fe804106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.228+0000 7fe803e00640 1 -- 172.21.15.83:0/134026698 shutdown_connections 2024-04-23T14:58:03.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.228+0000 7fe803e00640 1 -- 172.21.15.83:0/134026698 wait complete. 2024-04-23T14:58:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.229+0000 7fe803e00640 1 Processor -- start 2024-04-23T14:58:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.229+0000 7fe803e00640 1 -- start start 2024-04-23T14:58:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.229+0000 7fe803e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe804108580 0x7fe80419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.229+0000 7fe803e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe804108f40 0x7fe80419f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.229+0000 7fe803e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe80410d500 0x7fe8041a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.229+0000 7fe803e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041188b0 con 0x7fe804108f40 2024-04-23T14:58:03.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.229+0000 7fe803e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe804118730 con 0x7fe804108580 2024-04-23T14:58:03.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.229+0000 7fe803e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe804118a30 con 0x7fe80410d500 2024-04-23T14:58:03.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.230+0000 7fe7fa000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe804108f40 0x7fe80419f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.230+0000 7fe7fa000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe804108f40 0x7fe80419f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46644/0 (socket says 172.21.15.83:46644) 2024-04-23T14:58:03.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.230+0000 7fe7fa000640 1 -- 172.21.15.83:0/3225858094 learned_addr learned my addr 172.21.15.83:0/3225858094 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:03.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.230+0000 7fe803400640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe80410d500 0x7fe8041a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.230+0000 7fe802a00640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe804108580 0x7fe80419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.230+0000 7fe7fa000640 1 -- 172.21.15.83:0/3225858094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe80410d500 msgr2=0x7fe8041a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.230+0000 7fe7fa000640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe80410d500 0x7fe8041a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.230+0000 7fe7fa000640 1 -- 172.21.15.83:0/3225858094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe804108580 msgr2=0x7fe80419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.230+0000 7fe7fa000640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe804108580 0x7fe80419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.230+0000 7fe7fa000640 1 -- 172.21.15.83:0/3225858094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8041a3da0 con 0x7fe804108f40 2024-04-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.230+0000 7fe802a00640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe804108580 0x7fe80419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:03.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.230+0000 7fe7fa000640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe804108f40 0x7fe80419f2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009230 tx=0x7fe7ec004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.231+0000 7fe7fbe00640 1 -- 172.21.15.83:0/3225858094 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec03f030 con 0x7fe804108f40 2024-04-23T14:58:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.231+0000 7fe803e00640 1 -- 172.21.15.83:0/3225858094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8041a4030 con 0x7fe804108f40 2024-04-23T14:58:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.231+0000 7fe7fbe00640 1 -- 172.21.15.83:0/3225858094 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7ec004440 con 0x7fe804108f40 2024-04-23T14:58:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.231+0000 7fe7fbe00640 1 -- 172.21.15.83:0/3225858094 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec035070 con 0x7fe804108f40 2024-04-23T14:58:03.235 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.231+0000 7fe803e00640 1 -- 172.21.15.83:0/3225858094 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8041ab920 con 0x7fe804108f40 2024-04-23T14:58:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.234+0000 7fe80c377740 1 Processor -- start 2024-04-23T14:58:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.234+0000 7fe80c377740 1 -- start start 2024-04-23T14:58:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.234+0000 7fe7fbe00640 1 -- 172.21.15.83:0/3225858094 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe7ec042030 con 0x7fe804108f40 2024-04-23T14:58:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.234+0000 7fe80c377740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20130b40 0x561a20130f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.234+0000 7fe80c377740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20131460 0x561a2013b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.238 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.234+0000 7fe80c377740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a2013bb30 0x561a2013df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.234+0000 7fe80c377740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561a200961d0 con 0x561a20131460 2024-04-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.234+0000 7fe80c377740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561a2003c450 con 0x561a20130b40 2024-04-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.234+0000 7fe80c377740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561a20092190 con 0x561a2013bb30 2024-04-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.235+0000 7fe802a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20131460 0x561a2013b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.235+0000 7fe802a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20131460 0x561a2013b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46646/0 (socket says 172.21.15.83:46646) 2024-04-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.235+0000 7fe802a00640 1 -- 172.21.15.83:0/403495212 learned_addr learned my addr 172.21.15.83:0/403495212 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.235+0000 7fe7fa000640 1 --2- 172.21.15.83:0/403495212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a2013bb30 0x561a2013df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.235+0000 7fe803400640 1 --2- 172.21.15.83:0/403495212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20130b40 0x561a20130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.235+0000 7fe803400640 1 -- 172.21.15.83:0/403495212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a2013bb30 msgr2=0x561a2013df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.235+0000 7fe803400640 1 --2- 172.21.15.83:0/403495212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a2013bb30 0x561a2013df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.235+0000 7fe803400640 1 -- 172.21.15.83:0/403495212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20131460 msgr2=0x561a2013b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.235+0000 7fe803400640 1 --2- 172.21.15.83:0/403495212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20131460 0x561a2013b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.235+0000 7fe803400640 1 -- 172.21.15.83:0/403495212 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a200f9310 con 0x561a20130b40 2024-04-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.235+0000 7fe802a00640 1 --2- 172.21.15.83:0/403495212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20131460 0x561a2013b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.235+0000 7fe803400640 1 --2- 172.21.15.83:0/403495212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20130b40 0x561a20130f20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009ef0 tx=0x7fe7fc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1edac3e55d7b7224 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe7f3400640 1 -- 172.21.15.83:0/403495212 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc004d20 con 0x561a20130b40 2024-04-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe7f3400640 1 -- 172.21.15.83:0/403495212 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7fc004030 con 0x561a20130b40 2024-04-23T14:58:03.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe7f3400640 1 -- 172.21.15.83:0/403495212 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc005260 con 0x561a20130b40 2024-04-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe7fbe00640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe7e80a3da0 0x7fe7e80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe80c377740 1 -- 172.21.15.83:0/403495212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20130b40 msgr2=0x561a20130f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe80c377740 1 --2- 172.21.15.83:0/403495212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20130b40 0x561a20130f20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc009ef0 tx=0x7fe7fc011810 comp rx=0 tx=0).stop 2024-04-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe7fbe00640 1 -- 172.21.15.83:0/3225858094 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fe7ec0f0ab0 con 0x7fe804108f40 2024-04-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe80c377740 1 -- 172.21.15.83:0/403495212 shutdown_connections 2024-04-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe80c377740 1 --2- 172.21.15.83:0/403495212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a2013bb30 0x561a2013df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe80c377740 1 --2- 172.21.15.83:0/403495212 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20131460 0x561a2013b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe80c377740 1 --2- 172.21.15.83:0/403495212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20130b40 0x561a20130f20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe80c377740 1 -- 172.21.15.83:0/403495212 >> 172.21.15.83:0/403495212 conn(0x561a2012ce60 msgr2=0x561a2012d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe802a00640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe7e80a3da0 0x7fe7e80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.236+0000 7fe80c377740 1 -- 172.21.15.83:0/403495212 shutdown_connections 2024-04-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.237+0000 7fe80c377740 1 -- 172.21.15.83:0/403495212 wait complete. 2024-04-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.237+0000 7fe80c377740 1 -- start start 2024-04-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.237+0000 7fe802a00640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe7e80a3da0 0x7fe7e80a6260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x561a20132100 tx=0x7fe7f4005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:03.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.237+0000 7fe80c377740 1 Processor -- start 2024-04-23T14:58:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe80c377740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20130b40 0x561a2013b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe80c377740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20131460 0x561a201388e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe80c377740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a2013bb30 0x561a20139020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.246 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe80c377740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561a2013e9c0 con 0x561a20130b40 2024-04-23T14:58:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe80c377740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561a2004f810 con 0x561a20131460 2024-04-23T14:58:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe80c377740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x561a2013eb40 con 0x561a2013bb30 2024-04-23T14:58:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe80c377740 10 client.0 _ll_register_callbacks cb 0x561a2001d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe802a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20130b40 0x561a2013b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe802a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20130b40 0x561a2013b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46652/0 (socket says 172.21.15.83:46652) 2024-04-23T14:58:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe802a00640 1 -- 172.21.15.83:0/2482926167 learned_addr learned my addr 172.21.15.83:0/2482926167 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe7fa000640 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20131460 0x561a201388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe803400640 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a2013bb30 0x561a20139020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.247 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe802a00640 1 -- 172.21.15.83:0/2482926167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a2013bb30 msgr2=0x561a20139020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe802a00640 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a2013bb30 0x561a20139020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.238+0000 7fe802a00640 1 -- 172.21.15.83:0/2482926167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20131460 msgr2=0x561a201388e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.239+0000 7fe802a00640 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20131460 0x561a201388e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.239+0000 7fe802a00640 1 -- 172.21.15.83:0/2482926167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a200603f0 con 0x561a20130b40 2024-04-23T14:58:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.239+0000 7fe803400640 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a2013bb30 0x561a20139020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:58:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.239+0000 7fe7fa000640 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20131460 0x561a201388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.239+0000 7fe802a00640 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20130b40 0x561a2013b5f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe7f401eaa0 tx=0x7fe7f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363f973cb8c6a994 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.239+0000 7fe7e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:03.248 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.239+0000 7fe7e7e00640 1 -- 172.21.15.83:0/2482926167 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f402d060 con 0x561a20130b40 2024-04-23T14:58:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.239+0000 7fe7e7e00640 1 -- 172.21.15.83:0/2482926167 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fe7f4018030 con 0x561a20130b40 2024-04-23T14:58:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.239+0000 7fe80c377740 1 -- 172.21.15.83:0/2482926167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7fc004030 con 0x561a20130b40 2024-04-23T14:58:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.239+0000 7fe7e7e00640 1 -- 172.21.15.83:0/2482926167 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe7f401e320 con 0x561a20130b40 2024-04-23T14:58:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.239+0000 7fe80c377740 10 client.16938 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.239+0000 7fe80c377740 1 -- 172.21.15.83:0/2482926167 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561a1fda83f0 con 0x561a20130b40 2024-04-23T14:58:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.240+0000 7fe7e7e00640 1 -- 172.21.15.83:0/2482926167 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7fc004030 con 0x561a20130b40 2024-04-23T14:58:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.240+0000 7fe7e7e00640 1 -- 172.21.15.83:0/2482926167 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fe7f401d850 con 0x561a20130b40 2024-04-23T14:58:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.240+0000 7fe80c377740 10 client.16938 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:03.249 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.240+0000 7fe80c377740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.240+0000 7fe80c377740 20 client.16938 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.240+0000 7fe80c377740 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561a2014be20 0x561a2014e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.240+0000 7fe80c377740 4 client.16938 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:58:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.240+0000 7fe80c377740 1 -- 172.21.15.83:0/2482926167 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a200a3bf0 con 0x561a2014be20 2024-04-23T14:58:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.240+0000 7fe7fa000640 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561a2014be20 0x561a2014e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.241+0000 7fe7fa000640 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561a2014be20 0x561a2014e270 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7f80a86fb1babb server_cookie=740354f7bd7336ef in_seq=0 out_seq=0 2024-04-23T14:58:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.241+0000 7fe7e7e00640 10 client.16938 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:03.250 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe7e7e00640 1 -- 172.21.15.83:0/2482926167 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a200a3bf0 con 0x561a2014be20 2024-04-23T14:58:03.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe7e7e00640 10 client.16938 handle_command_reply: tid=0 2024-04-23T14:58:03.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe80c377740 1 client.16938 shutdown 2024-04-23T14:58:03.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe80c377740 1 -- 172.21.15.83:0/2482926167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20130b40 msgr2=0x561a2013b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe80c377740 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20130b40 0x561a2013b5f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe7f401eaa0 tx=0x7fe7f4004730 comp rx=0 tx=0).stop 2024-04-23T14:58:03.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe80c377740 1 -- 172.21.15.83:0/2482926167 shutdown_connections 2024-04-23T14:58:03.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe80c377740 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561a2014be20 0x561a2014e270 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe80c377740 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a2013bb30 0x561a20139020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe80c377740 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20131460 0x561a201388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.251 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe80c377740 1 --2- 172.21.15.83:0/2482926167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20130b40 0x561a2013b5f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe80c377740 1 -- 172.21.15.83:0/2482926167 >> 172.21.15.83:0/2482926167 conn(0x561a2012ce60 msgr2=0x561a2012bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe7e7e00640 0 client.16938 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe80c377740 1 -- 172.21.15.83:0/2482926167 shutdown_connections 2024-04-23T14:58:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.242+0000 7fe80c377740 1 -- 172.21.15.83:0/2482926167 wait complete. 2024-04-23T14:58:03.252 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.243+0000 7fe80c377740 20 client.16938 trim_cache size 0 max 16384 2024-04-23T14:58:03.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.256+0000 7fe80c377740 1 Processor -- start 2024-04-23T14:58:03.257 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.256+0000 7fe80c377740 1 -- start start 2024-04-23T14:58:03.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe80c377740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20130b40 0x561a201478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe80c377740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20131460 0x561a2013b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe80c377740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a2013bb30 0x561a20153b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe80c377740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041b6a70 con 0x561a20131460 2024-04-23T14:58:03.260 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe80c377740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041b4340 con 0x561a2013bb30 2024-04-23T14:58:03.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe80c377740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041b21b0 con 0x561a20130b40 2024-04-23T14:58:03.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe802a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20130b40 0x561a201478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe802a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20130b40 0x561a201478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:52558/0 (socket says 172.21.15.83:52558) 2024-04-23T14:58:03.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe802a00640 1 -- 172.21.15.83:0/1865909781 learned_addr learned my addr 172.21.15.83:0/1865909781 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:03.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe7fa000640 1 --2- 172.21.15.83:0/1865909781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20131460 0x561a2013b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe803400640 1 --2- 172.21.15.83:0/1865909781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a2013bb30 0x561a20153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe802a00640 1 -- 172.21.15.83:0/1865909781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a2013bb30 msgr2=0x561a20153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:03.261 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe802a00640 1 --2- 172.21.15.83:0/1865909781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a2013bb30 0x561a20153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe802a00640 1 -- 172.21.15.83:0/1865909781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20131460 msgr2=0x561a2013b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:03.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe802a00640 1 --2- 172.21.15.83:0/1865909781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20131460 0x561a2013b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.257+0000 7fe802a00640 1 -- 172.21.15.83:0/1865909781 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a1fe727d0 con 0x561a20130b40 2024-04-23T14:58:03.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.258+0000 7fe802a00640 1 --2- 172.21.15.83:0/1865909781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20130b40 0x561a201478e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe7f401efd0 tx=0x7fe7f4012d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27cdc538cd6797f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:03.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.258+0000 7fe7f2a00640 1 -- 172.21.15.83:0/1865909781 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4005020 con 0x561a20130b40 2024-04-23T14:58:03.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.258+0000 7fe7f2a00640 1 -- 172.21.15.83:0/1865909781 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f401d410 con 0x561a20130b40 2024-04-23T14:58:03.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.258+0000 7fe7f2a00640 1 -- 172.21.15.83:0/1865909781 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4004b20 con 0x561a20130b40 2024-04-23T14:58:03.262 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.259+0000 7fe80c377740 1 -- 172.21.15.83:0/1865909781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20130b40 msgr2=0x561a201478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.259+0000 7fe80c377740 1 --2- 172.21.15.83:0/1865909781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20130b40 0x561a201478e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe7f401efd0 tx=0x7fe7f4012d80 comp rx=0 tx=0).stop 2024-04-23T14:58:03.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.259+0000 7fe80c377740 1 -- 172.21.15.83:0/1865909781 shutdown_connections 2024-04-23T14:58:03.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.259+0000 7fe80c377740 1 --2- 172.21.15.83:0/1865909781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a2013bb30 0x561a20153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.259+0000 7fe80c377740 1 --2- 172.21.15.83:0/1865909781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a20131460 0x561a2013b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.259+0000 7fe80c377740 1 --2- 172.21.15.83:0/1865909781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20130b40 0x561a201478e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.263 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.259+0000 7fe80c377740 1 -- 172.21.15.83:0/1865909781 >> 172.21.15.83:0/1865909781 conn(0x561a2012ce60 msgr2=0x561a20133bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:03.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.259+0000 7fe80c377740 1 -- 172.21.15.83:0/1865909781 shutdown_connections 2024-04-23T14:58:03.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.259+0000 7fe80c377740 1 -- 172.21.15.83:0/1865909781 wait complete. 2024-04-23T14:58:03.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.259+0000 7fe80c377740 1 -- start start 2024-04-23T14:58:03.264 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.259+0000 7fe80c377740 1 Processor -- start 2024-04-23T14:58:03.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe80c377740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20130b40 0x561a20143df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe80c377740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20131460 0x561a20144330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe80c377740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a2013bb30 0x561a20135b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe80c377740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041b2800 con 0x561a2013bb30 2024-04-23T14:58:03.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe80c377740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041b3c70 con 0x561a20130b40 2024-04-23T14:58:03.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe80c377740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041b2ed0 con 0x561a20131460 2024-04-23T14:58:03.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe80c377740 10 client.0 _ll_register_callbacks cb 0x7fe7f4012f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:03.268 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe803400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a2013bb30 0x561a20135b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe7fa000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20131460 0x561a20144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe7fa000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20131460 0x561a20144330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:52560/0 (socket says 172.21.15.83:52560) 2024-04-23T14:58:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe803400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a2013bb30 0x561a20135b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46680/0 (socket says 172.21.15.83:46680) 2024-04-23T14:58:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe7fa000640 1 -- 172.21.15.83:0/1987043118 learned_addr learned my addr 172.21.15.83:0/1987043118 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.260+0000 7fe802a00640 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20130b40 0x561a20143df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe7fa000640 1 -- 172.21.15.83:0/1987043118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20130b40 msgr2=0x561a20143df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe7fa000640 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20130b40 0x561a20143df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe7fa000640 1 -- 172.21.15.83:0/1987043118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a2013bb30 msgr2=0x561a20135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe7fa000640 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a2013bb30 0x561a20135b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.269 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe7fa000640 1 -- 172.21.15.83:0/1987043118 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a201360a0 con 0x561a20131460 2024-04-23T14:58:03.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe803400640 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a2013bb30 0x561a20135b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:03.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe7fa000640 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20131460 0x561a20144330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0bca20 tx=0x7fe7ec0092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2441e91435534445 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:03.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe7e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:58:03.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe7e7e00640 1 -- 172.21.15.83:0/1987043118 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec049040 con 0x561a20131460 2024-04-23T14:58:03.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe7e7e00640 1 -- 172.21.15.83:0/1987043118 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7fe7ec0ee410 con 0x561a20131460 2024-04-23T14:58:03.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe80c377740 1 -- 172.21.15.83:0/1987043118 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f401d410 con 0x561a20131460 2024-04-23T14:58:03.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe7e7e00640 1 -- 172.21.15.83:0/1987043118 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe7ec0bc860 con 0x561a20131460 2024-04-23T14:58:03.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe80c377740 10 client.27121 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:03.270 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.261+0000 7fe80c377740 1 -- 172.21.15.83:0/1987043118 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561a2013c6e0 con 0x561a20131460 2024-04-23T14:58:03.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.262+0000 7fe7e7e00640 1 -- 172.21.15.83:0/1987043118 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f401d410 con 0x561a20131460 2024-04-23T14:58:03.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.262+0000 7fe7e7e00640 1 -- 172.21.15.83:0/1987043118 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fe7ec0c51c0 con 0x561a20131460 2024-04-23T14:58:03.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.262+0000 7fe80c377740 10 client.27121 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:03.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.262+0000 7fe80c377740 10 client.27121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:03.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.262+0000 7fe80c377740 20 client.27121 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:03.271 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.262+0000 7fe80c377740 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561a201b23b0 0x561a201b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.262+0000 7fe80c377740 4 client.27121 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:58:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.262+0000 7fe80c377740 1 -- 172.21.15.83:0/1987043118 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a2009da60 con 0x561a201b23b0 2024-04-23T14:58:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.262+0000 7fe802a00640 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561a201b23b0 0x561a201b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe802a00640 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561a201b23b0 0x561a201b47a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c530b848f19783 server_cookie=d90f614a9d3eefd2 in_seq=0 out_seq=0 2024-04-23T14:58:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe7e7e00640 10 client.27121 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe7e7e00640 1 -- 172.21.15.83:0/1987043118 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a2009da60 con 0x561a201b23b0 2024-04-23T14:58:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe7e7e00640 10 client.27121 handle_command_reply: tid=0 2024-04-23T14:58:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe80c377740 1 client.27121 shutdown 2024-04-23T14:58:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe80c377740 1 -- 172.21.15.83:0/1987043118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20131460 msgr2=0x561a20144330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.272 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe80c377740 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20131460 0x561a20144330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec0bca20 tx=0x7fe7ec0092e0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe80c377740 1 -- 172.21.15.83:0/1987043118 shutdown_connections 2024-04-23T14:58:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe80c377740 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x561a201b23b0 0x561a201b47a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe80c377740 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a2013bb30 0x561a20135b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe7e7e00640 0 client.27121 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe80c377740 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561a20131460 0x561a20144330 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe80c377740 1 --2- 172.21.15.83:0/1987043118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561a20130b40 0x561a20143df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.263+0000 7fe80c377740 1 -- 172.21.15.83:0/1987043118 >> 172.21.15.83:0/1987043118 conn(0x561a2012ce60 msgr2=0x561a201371b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.264+0000 7fe80c377740 1 -- 172.21.15.83:0/1987043118 shutdown_connections 2024-04-23T14:58:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.264+0000 7fe80c377740 1 -- 172.21.15.83:0/1987043118 wait complete. 2024-04-23T14:58:03.273 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.264+0000 7fe80c377740 20 client.27121 trim_cache size 0 max 16384 2024-04-23T14:58:03.274 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:58:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.264+0000 7fe803e00640 1 -- 172.21.15.83:0/3225858094 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe7e80a3da0 msgr2=0x7fe7e80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.264+0000 7fe803e00640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe7e80a3da0 0x7fe7e80a6260 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x561a20132100 tx=0x7fe7f4005f90 comp rx=0 tx=0).stop 2024-04-23T14:58:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.264+0000 7fe803e00640 1 -- 172.21.15.83:0/3225858094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe804108f40 msgr2=0x7fe80419f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.264+0000 7fe803e00640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe804108f40 0x7fe80419f2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009230 tx=0x7fe7ec004040 comp rx=0 tx=0).stop 2024-04-23T14:58:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.264+0000 7fe803e00640 1 -- 172.21.15.83:0/3225858094 shutdown_connections 2024-04-23T14:58:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.265+0000 7fe803e00640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe80410d500 0x7fe8041a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.265+0000 7fe803e00640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe7e80a3da0 0x7fe7e80a6260 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.265+0000 7fe803e00640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe804108f40 0x7fe80419f2f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.275 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.265+0000 7fe803e00640 1 --2- 172.21.15.83:0/3225858094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe804108580 0x7fe80419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.265+0000 7fe803e00640 1 -- 172.21.15.83:0/3225858094 >> 172.21.15.83:0/3225858094 conn(0x7fe804077e20 msgr2=0x7fe804078440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.265+0000 7fe803e00640 1 -- 172.21.15.83:0/3225858094 shutdown_connections 2024-04-23T14:58:03.276 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:03.265+0000 7fe803e00640 1 -- 172.21.15.83:0/3225858094 wait complete. 2024-04-23T14:58:03.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:03 smithi203 ceph-mon[32800]: pgmap v297: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-23T14:58:04.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:03 smithi083 ceph-mon[27867]: pgmap v297: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-23T14:58:04.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:03 smithi173 ceph-mon[32443]: pgmap v297: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-23T14:58:04.282 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:58:04.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:58:04.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:58:04.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.499+0000 7f99ca400640 1 -- 172.21.15.83:0/2159035543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99c4104cd0 msgr2=0x7f99c4108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.499+0000 7f99ca400640 1 --2- 172.21.15.83:0/2159035543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99c4104cd0 0x7f99c4108d10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009900 tx=0x7f99b002a410 comp rx=0 tx=0).stop 2024-04-23T14:58:04.500 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.499+0000 7f99ca400640 1 -- 172.21.15.83:0/2159035543 shutdown_connections 2024-04-23T14:58:04.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.499+0000 7f99ca400640 1 --2- 172.21.15.83:0/2159035543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99c41093f0 0x7f99c4110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.499+0000 7f99ca400640 1 --2- 172.21.15.83:0/2159035543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99c4104cd0 0x7f99c4108d10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.499+0000 7f99ca400640 1 --2- 172.21.15.83:0/2159035543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c4104320 0x7f99c4104700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.499+0000 7f99ca400640 1 -- 172.21.15.83:0/2159035543 >> 172.21.15.83:0/2159035543 conn(0x7f99c40ffb40 msgr2=0x7f99c4101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:04.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.499+0000 7f99ca400640 1 -- 172.21.15.83:0/2159035543 shutdown_connections 2024-04-23T14:58:04.501 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.499+0000 7f99ca400640 1 -- 172.21.15.83:0/2159035543 wait complete. 2024-04-23T14:58:04.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.500+0000 7f99ca400640 1 Processor -- start 2024-04-23T14:58:04.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.500+0000 7f99ca400640 1 -- start start 2024-04-23T14:58:04.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.500+0000 7f99ca400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99c4104320 0x7f99c41a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.500+0000 7f99ca400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99c41093f0 0x7f99c41a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.504 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99ca400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c419a7e0 0x7f99c419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99ca400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4114480 con 0x7f99c4104320 2024-04-23T14:58:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99ca400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4114300 con 0x7f99c419a7e0 2024-04-23T14:58:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99ca400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99c4114600 con 0x7f99c41093f0 2024-04-23T14:58:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99c9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99c4104320 0x7f99c41a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99c9000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99c4104320 0x7f99c41a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46694/0 (socket says 172.21.15.83:46694) 2024-04-23T14:58:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99c9000640 1 -- 172.21.15.83:0/3313546426 learned_addr learned my addr 172.21.15.83:0/3313546426 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99c3e00640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99c41093f0 0x7f99c41a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99c9a00640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c419a7e0 0x7f99c419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99c9000640 1 -- 172.21.15.83:0/3313546426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99c41093f0 msgr2=0x7f99c41a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99c9000640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99c41093f0 0x7f99c41a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99c9000640 1 -- 172.21.15.83:0/3313546426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c419a7e0 msgr2=0x7f99c419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99c9000640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c419a7e0 0x7f99c419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99c9000640 1 -- 172.21.15.83:0/3313546426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c419b370 con 0x7f99c4104320 2024-04-23T14:58:04.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99c3e00640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99c41093f0 0x7f99c41a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:58:04.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.501+0000 7f99c9a00640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c419a7e0 0x7f99c419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:58:04.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.502+0000 7f99c9000640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99c4104320 0x7f99c41a0670 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f99b800ea60 tx=0x7f99b800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:04.506 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.502+0000 7f99c1600640 1 -- 172.21.15.83:0/3313546426 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b800cde0 con 0x7f99c4104320 2024-04-23T14:58:04.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.502+0000 7f99c1600640 1 -- 172.21.15.83:0/3313546426 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99b80035f0 con 0x7f99c4104320 2024-04-23T14:58:04.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.502+0000 7f99ca400640 1 -- 172.21.15.83:0/3313546426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c419b660 con 0x7f99c4104320 2024-04-23T14:58:04.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.502+0000 7f99c1600640 1 -- 172.21.15.83:0/3313546426 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b8010670 con 0x7f99c4104320 2024-04-23T14:58:04.507 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.502+0000 7f99ca400640 1 -- 172.21.15.83:0/3313546426 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c41a7860 con 0x7f99c4104320 2024-04-23T14:58:04.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.506+0000 7f99ce871740 1 Processor -- start 2024-04-23T14:58:04.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.506+0000 7f99ce871740 1 -- start start 2024-04-23T14:58:04.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.507+0000 7f99ce871740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e310650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.507+0000 7f99ce871740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e310b90 0x55cd0e31ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.507+0000 7f99ce871740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e31b260 0x55cd0e31d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.507+0000 7f99ce871740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd0e275e70 con 0x55cd0e31b260 2024-04-23T14:58:04.509 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.507+0000 7f99ce871740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cd0e21c380 con 0x55cd0e310b90 2024-04-23T14:58:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.507+0000 7f99ce871740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cd0e271e30 con 0x55cd0e310270 2024-04-23T14:58:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.507+0000 7f99c1600640 1 -- 172.21.15.83:0/3313546426 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f99b80037a0 con 0x7f99c4104320 2024-04-23T14:58:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.507+0000 7f99c9a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e310650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.507+0000 7f99c3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e31b260 0x55cd0e31d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.507+0000 7f99c9a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e310650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:52586/0 (socket says 172.21.15.83:52586) 2024-04-23T14:58:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.507+0000 7f99c9a00640 1 -- 172.21.15.83:0/3431288525 learned_addr learned my addr 172.21.15.83:0/3431288525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.507+0000 7f99c9000640 1 --2- 172.21.15.83:0/3431288525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e310b90 0x55cd0e31ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.508+0000 7f99c9a00640 1 -- 172.21.15.83:0/3431288525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e310b90 msgr2=0x55cd0e31ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.510 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.508+0000 7f99c9a00640 1 --2- 172.21.15.83:0/3431288525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e310b90 0x55cd0e31ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.508+0000 7f99c9a00640 1 -- 172.21.15.83:0/3431288525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e31b260 msgr2=0x55cd0e31d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.508+0000 7f99c9a00640 1 --2- 172.21.15.83:0/3431288525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e31b260 0x55cd0e31d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.508+0000 7f99c9a00640 1 -- 172.21.15.83:0/3431288525 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd0e0507e0 con 0x55cd0e310270 2024-04-23T14:58:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.508+0000 7f99c9a00640 1 --2- 172.21.15.83:0/3431288525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e310650 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009fd0 tx=0x7f99bc010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=614921c98b6f8cc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.508+0000 7f99b6000640 1 -- 172.21.15.83:0/3431288525 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc011700 con 0x55cd0e310270 2024-04-23T14:58:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.508+0000 7f99b6000640 1 -- 172.21.15.83:0/3431288525 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99bc011d80 con 0x55cd0e310270 2024-04-23T14:58:04.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.508+0000 7f99b6000640 1 -- 172.21.15.83:0/3431288525 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00e3c0 con 0x55cd0e310270 2024-04-23T14:58:04.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99c1600640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f99ac0a3da0 0x7f99ac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99c1600640 1 -- 172.21.15.83:0/3313546426 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f99b80d0c20 con 0x7f99c4104320 2024-04-23T14:58:04.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99c9000640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f99ac0a3da0 0x7f99ac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99ce871740 1 -- 172.21.15.83:0/3431288525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 msgr2=0x55cd0e310650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99ce871740 1 --2- 172.21.15.83:0/3431288525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e310650 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009fd0 tx=0x7f99bc010860 comp rx=0 tx=0).stop 2024-04-23T14:58:04.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99ce871740 1 -- 172.21.15.83:0/3431288525 shutdown_connections 2024-04-23T14:58:04.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99ce871740 1 --2- 172.21.15.83:0/3431288525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e31b260 0x55cd0e31d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99ce871740 1 --2- 172.21.15.83:0/3431288525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e310b90 0x55cd0e31ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99ce871740 1 --2- 172.21.15.83:0/3431288525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e310650 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99ce871740 1 -- 172.21.15.83:0/3431288525 >> 172.21.15.83:0/3431288525 conn(0x55cd0e30c590 msgr2=0x55cd0e30c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:04.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99ce871740 1 -- 172.21.15.83:0/3431288525 shutdown_connections 2024-04-23T14:58:04.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99ce871740 1 -- 172.21.15.83:0/3431288525 wait complete. 2024-04-23T14:58:04.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.509+0000 7f99c9000640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f99ac0a3da0 0x7f99ac0a6260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55cd0e311830 tx=0x7f99b80d46d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:04.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.510+0000 7f99ce871740 1 -- start start 2024-04-23T14:58:04.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.510+0000 7f99ce871740 1 Processor -- start 2024-04-23T14:58:04.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.510+0000 7f99ce871740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310270 0x55cd0e318220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.510+0000 7f99ce871740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e310b90 0x55cd0e318760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99ce871740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e31b260 0x55cd0e318ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99ce871740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd0e31e050 con 0x55cd0e310270 2024-04-23T14:58:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99ce871740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55cd0e22f660 con 0x55cd0e310b90 2024-04-23T14:58:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99ce871740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55cd0e31e1d0 con 0x55cd0e31b260 2024-04-23T14:58:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99ce871740 10 client.0 _ll_register_callbacks cb 0x55cd0e21daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99c9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310270 0x55cd0e318220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99c9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310270 0x55cd0e318220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46698/0 (socket says 172.21.15.83:46698) 2024-04-23T14:58:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99c9a00640 1 -- 172.21.15.83:0/4076505541 learned_addr learned my addr 172.21.15.83:0/4076505541 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99c3e00640 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e310b90 0x55cd0e318760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99c9a00640 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e31b260 0x55cd0e318ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99c9a00640 1 -- 172.21.15.83:0/4076505541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e31b260 msgr2=0x55cd0e318ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99c9a00640 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e31b260 0x55cd0e318ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99c9a00640 1 -- 172.21.15.83:0/4076505541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e310b90 msgr2=0x55cd0e318760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99c9a00640 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e310b90 0x55cd0e318760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99c9a00640 1 -- 172.21.15.83:0/4076505541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd0e0b0f70 con 0x55cd0e310270 2024-04-23T14:58:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99c9a00640 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310270 0x55cd0e318220 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f99bc011fd0 tx=0x7f99bc01bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e46eb870b08d991 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.511+0000 7f99aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.512+0000 7f99aaa00640 1 -- 172.21.15.83:0/4076505541 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc011700 con 0x55cd0e310270 2024-04-23T14:58:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.512+0000 7f99aaa00640 1 -- 172.21.15.83:0/4076505541 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f99bc0118a0 con 0x55cd0e310270 2024-04-23T14:58:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.512+0000 7f99ce871740 1 -- 172.21.15.83:0/4076505541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99bc011d80 con 0x55cd0e310270 2024-04-23T14:58:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.512+0000 7f99aaa00640 1 -- 172.21.15.83:0/4076505541 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99bc011aa0 con 0x55cd0e310270 2024-04-23T14:58:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.512+0000 7f99ce871740 10 client.16956 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.512+0000 7f99ce871740 1 -- 172.21.15.83:0/4076505541 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd0df48170 con 0x55cd0e310270 2024-04-23T14:58:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.512+0000 7f99aaa00640 1 -- 172.21.15.83:0/4076505541 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99bc011d80 con 0x55cd0e310270 2024-04-23T14:58:04.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.513+0000 7f99aaa00640 1 -- 172.21.15.83:0/4076505541 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f99bc029020 con 0x55cd0e310270 2024-04-23T14:58:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.513+0000 7f99ce871740 10 client.16956 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.513+0000 7f99ce871740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.513+0000 7f99ce871740 20 client.16956 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.513+0000 7f99ce871740 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd0e32b540 0x55cd0e32d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.513+0000 7f99ce871740 4 client.16956 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:58:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.513+0000 7f99ce871740 1 -- 172.21.15.83:0/4076505541 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd0e283890 con 0x55cd0e32b540 2024-04-23T14:58:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.513+0000 7f99c3e00640 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd0e32b540 0x55cd0e32d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.514+0000 7f99c3e00640 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd0e32b540 0x55cd0e32d930 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d2bbcac5039221 server_cookie=79e3ee6adee6b98 in_seq=0 out_seq=0 2024-04-23T14:58:04.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.514+0000 7f99aaa00640 10 client.16956 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.514+0000 7f99aaa00640 1 -- 172.21.15.83:0/4076505541 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd0e283890 con 0x55cd0e32b540 2024-04-23T14:58:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.514+0000 7f99aaa00640 10 client.16956 handle_command_reply: tid=0 2024-04-23T14:58:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.514+0000 7f99ce871740 1 client.16956 shutdown 2024-04-23T14:58:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.515+0000 7f99ce871740 1 -- 172.21.15.83:0/4076505541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310270 msgr2=0x55cd0e318220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.515+0000 7f99ce871740 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310270 0x55cd0e318220 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f99bc011fd0 tx=0x7f99bc01bc30 comp rx=0 tx=0).stop 2024-04-23T14:58:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.515+0000 7f99ce871740 1 -- 172.21.15.83:0/4076505541 shutdown_connections 2024-04-23T14:58:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.515+0000 7f99ce871740 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd0e32b540 0x55cd0e32d930 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.515+0000 7f99ce871740 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e31b260 0x55cd0e318ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.515+0000 7f99ce871740 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e310b90 0x55cd0e318760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.515+0000 7f99aaa00640 0 client.16956 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:04.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.515+0000 7f99ce871740 1 --2- 172.21.15.83:0/4076505541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310270 0x55cd0e318220 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.515+0000 7f99ce871740 1 -- 172.21.15.83:0/4076505541 >> 172.21.15.83:0/4076505541 conn(0x55cd0e30c590 msgr2=0x55cd0e312390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:04.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.515+0000 7f99ce871740 1 -- 172.21.15.83:0/4076505541 shutdown_connections 2024-04-23T14:58:04.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.515+0000 7f99ce871740 1 -- 172.21.15.83:0/4076505541 wait complete. 2024-04-23T14:58:04.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.515+0000 7f99ce871740 20 client.16956 trim_cache size 0 max 16384 2024-04-23T14:58:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.535+0000 7f99ce871740 1 Processor -- start 2024-04-23T14:58:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.535+0000 7f99ce871740 1 -- start start 2024-04-23T14:58:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.535+0000 7f99ce871740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e317b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.535+0000 7f99ce871740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310b90 0x55cd0e30b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.535+0000 7f99ce871740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e31b260 0x55cd0e330950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.535+0000 7f99ce871740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41b2770 con 0x55cd0e310b90 2024-04-23T14:58:04.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.535+0000 7f99ce871740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41b0040 con 0x55cd0e31b260 2024-04-23T14:58:04.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.535+0000 7f99ce871740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41adeb0 con 0x55cd0e310270 2024-04-23T14:58:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.536+0000 7f99c9a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e317b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.536+0000 7f99c9a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e317b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:52600/0 (socket says 172.21.15.83:52600) 2024-04-23T14:58:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.536+0000 7f99c9a00640 1 -- 172.21.15.83:0/913721413 learned_addr learned my addr 172.21.15.83:0/913721413 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.536+0000 7f99c9a00640 1 --2- 172.21.15.83:0/913721413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e31b260 0x55cd0e330950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.536+0000 7f99c3e00640 1 --2- 172.21.15.83:0/913721413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310b90 0x55cd0e30b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.536+0000 7f99c9a00640 1 -- 172.21.15.83:0/913721413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 msgr2=0x55cd0e317b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.536+0000 7f99c9a00640 1 --2- 172.21.15.83:0/913721413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e317b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.536+0000 7f99c9a00640 1 -- 172.21.15.83:0/913721413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310b90 msgr2=0x55cd0e30b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.536+0000 7f99c9a00640 1 --2- 172.21.15.83:0/913721413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310b90 0x55cd0e30b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.536+0000 7f99c9a00640 1 -- 172.21.15.83:0/913721413 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd0e3109c0 con 0x55cd0e31b260 2024-04-23T14:58:04.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.537+0000 7f99c9a00640 1 --2- 172.21.15.83:0/913721413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e31b260 0x55cd0e330950 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005ee0 tx=0x7f99bc009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b9b79912a280d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:04.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.537+0000 7f99b5600640 1 -- 172.21.15.83:0/913721413 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc037030 con 0x55cd0e31b260 2024-04-23T14:58:04.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.537+0000 7f99b5600640 1 -- 172.21.15.83:0/913721413 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99bc038040 con 0x55cd0e31b260 2024-04-23T14:58:04.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.537+0000 7f99b5600640 1 -- 172.21.15.83:0/913721413 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc005220 con 0x55cd0e31b260 2024-04-23T14:58:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.538+0000 7f99ce871740 1 -- 172.21.15.83:0/913721413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e31b260 msgr2=0x55cd0e330950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.538+0000 7f99ce871740 1 --2- 172.21.15.83:0/913721413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e31b260 0x55cd0e330950 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005ee0 tx=0x7f99bc009fd0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.538+0000 7f99ce871740 1 -- 172.21.15.83:0/913721413 shutdown_connections 2024-04-23T14:58:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.538+0000 7f99ce871740 1 --2- 172.21.15.83:0/913721413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e31b260 0x55cd0e330950 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.538+0000 7f99ce871740 1 --2- 172.21.15.83:0/913721413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310b90 0x55cd0e30b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.538+0000 7f99ce871740 1 --2- 172.21.15.83:0/913721413 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e317b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.538+0000 7f99ce871740 1 -- 172.21.15.83:0/913721413 >> 172.21.15.83:0/913721413 conn(0x55cd0e30c590 msgr2=0x55cd0e30ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:04.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.538+0000 7f99ce871740 1 -- 172.21.15.83:0/913721413 shutdown_connections 2024-04-23T14:58:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.538+0000 7f99ce871740 1 -- 172.21.15.83:0/913721413 wait complete. 2024-04-23T14:58:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.538+0000 7f99ce871740 1 -- start start 2024-04-23T14:58:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.539+0000 7f99ce871740 1 Processor -- start 2024-04-23T14:58:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.539+0000 7f99ce871740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e330d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.539+0000 7f99ce871740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310b90 0x55cd0e32e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.539+0000 7f99ce871740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e31b260 0x55cd0e32e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.539+0000 7f99ce871740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41ae500 con 0x55cd0e310b90 2024-04-23T14:58:04.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.539+0000 7f99ce871740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41af970 con 0x55cd0e31b260 2024-04-23T14:58:04.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.539+0000 7f99ce871740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f99c41aebd0 con 0x55cd0e310270 2024-04-23T14:58:04.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.539+0000 7f99ce871740 10 client.0 _ll_register_callbacks cb 0x7f99b8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:04.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.539+0000 7f99c9a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e330d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.540+0000 7f99c9a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e330d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:52604/0 (socket says 172.21.15.83:52604) 2024-04-23T14:58:04.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.540+0000 7f99c9a00640 1 -- 172.21.15.83:0/1955843715 learned_addr learned my addr 172.21.15.83:0/1955843715 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:04.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.540+0000 7f99c9a00640 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e31b260 0x55cd0e32e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.540+0000 7f99c3e00640 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310b90 0x55cd0e32e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.540+0000 7f99c9a00640 1 -- 172.21.15.83:0/1955843715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e31b260 msgr2=0x55cd0e32e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.540+0000 7f99c9a00640 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e31b260 0x55cd0e32e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.540+0000 7f99c9a00640 1 -- 172.21.15.83:0/1955843715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310b90 msgr2=0x55cd0e32e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.540+0000 7f99c9a00640 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310b90 0x55cd0e32e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.540+0000 7f99c9a00640 1 -- 172.21.15.83:0/1955843715 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd0e32eeb0 con 0x55cd0e310270 2024-04-23T14:58:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.540+0000 7f99c9a00640 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e330d60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009aa0 tx=0x7f99bc005cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf1471b6b23fb8a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.540+0000 7f99aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:58:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.540+0000 7f99aaa00640 1 -- 172.21.15.83:0/1955843715 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc037070 con 0x55cd0e310270 2024-04-23T14:58:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99ce871740 1 -- 172.21.15.83:0/1955843715 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99bc038040 con 0x55cd0e310270 2024-04-23T14:58:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99aaa00640 1 -- 172.21.15.83:0/1955843715 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f99bc01b960 con 0x55cd0e310270 2024-04-23T14:58:04.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99aaa00640 1 -- 172.21.15.83:0/1955843715 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f99bc02b970 con 0x55cd0e310270 2024-04-23T14:58:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99ce871740 10 client.27139 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99ce871740 1 -- 172.21.15.83:0/1955843715 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cd0e3917c0 con 0x55cd0e310270 2024-04-23T14:58:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99aaa00640 1 -- 172.21.15.83:0/1955843715 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99bc038040 con 0x55cd0e310270 2024-04-23T14:58:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99aaa00640 1 -- 172.21.15.83:0/1955843715 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f99bc03d020 con 0x55cd0e310270 2024-04-23T14:58:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99ce871740 10 client.27139 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99ce871740 10 client.27139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99ce871740 20 client.27139 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99ce871740 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd0e391dd0 0x55cd0e3941c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99ce871740 4 client.27139 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:58:04.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.541+0000 7f99ce871740 1 -- 172.21.15.83:0/1955843715 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cd0e27d700 con 0x55cd0e391dd0 2024-04-23T14:58:04.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.542+0000 7f99c3e00640 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd0e391dd0 0x55cd0e3941c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.542+0000 7f99c3e00640 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd0e391dd0 0x55cd0e3941c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f7d74f2ad1e4453 server_cookie=24492382ea50b3bd in_seq=0 out_seq=0 2024-04-23T14:58:04.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.542+0000 7f99aaa00640 10 client.27139 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:04.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99aaa00640 1 -- 172.21.15.83:0/1955843715 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cd0e27d700 con 0x55cd0e391dd0 2024-04-23T14:58:04.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99aaa00640 10 client.27139 handle_command_reply: tid=0 2024-04-23T14:58:04.549 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99ce871740 1 client.27139 shutdown 2024-04-23T14:58:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99ce871740 1 -- 172.21.15.83:0/1955843715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 msgr2=0x55cd0e330d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99ce871740 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e330d60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009aa0 tx=0x7f99bc005cf0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99ce871740 1 -- 172.21.15.83:0/1955843715 shutdown_connections 2024-04-23T14:58:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99ce871740 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55cd0e391dd0 0x55cd0e3941c0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99ce871740 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cd0e31b260 0x55cd0e32e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99ce871740 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd0e310b90 0x55cd0e32e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99ce871740 1 --2- 172.21.15.83:0/1955843715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cd0e310270 0x55cd0e330d60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99aaa00640 0 client.27139 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:04.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99ce871740 1 -- 172.21.15.83:0/1955843715 >> 172.21.15.83:0/1955843715 conn(0x55cd0e30c590 msgr2=0x55cd0e30b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:04.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99ce871740 1 -- 172.21.15.83:0/1955843715 shutdown_connections 2024-04-23T14:58:04.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99ce871740 1 -- 172.21.15.83:0/1955843715 wait complete. 2024-04-23T14:58:04.551 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.544+0000 7f99ce871740 20 client.27139 trim_cache size 0 max 16384 2024-04-23T14:58:04.551 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:58:04.551 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:58:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "c234f018-3bec-4982-b652-5e65c8a6c36a", 2024-04-23T14:58:04.552 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:58:04.552 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:58:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.545+0000 7f99ca400640 1 -- 172.21.15.83:0/3313546426 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f99ac0a3da0 msgr2=0x7f99ac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.545+0000 7f99ca400640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f99ac0a3da0 0x7f99ac0a6260 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55cd0e311830 tx=0x7f99b80d46d0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.545+0000 7f99ca400640 1 -- 172.21.15.83:0/3313546426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99c4104320 msgr2=0x7f99c41a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.545+0000 7f99ca400640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99c4104320 0x7f99c41a0670 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f99b800ea60 tx=0x7f99b800ef20 comp rx=0 tx=0).stop 2024-04-23T14:58:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.545+0000 7f99ca400640 1 -- 172.21.15.83:0/3313546426 shutdown_connections 2024-04-23T14:58:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.545+0000 7f99ca400640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c419a7e0 0x7f99c419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.545+0000 7f99ca400640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99c41093f0 0x7f99c41a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.545+0000 7f99ca400640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f99ac0a3da0 0x7f99ac0a6260 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.545+0000 7f99ca400640 1 --2- 172.21.15.83:0/3313546426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99c4104320 0x7f99c41a0670 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.545+0000 7f99ca400640 1 -- 172.21.15.83:0/3313546426 >> 172.21.15.83:0/3313546426 conn(0x7f99c40ffb40 msgr2=0x7f99c4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.545+0000 7f99ca400640 1 -- 172.21.15.83:0/3313546426 shutdown_connections 2024-04-23T14:58:04.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.545+0000 7f99ca400640 1 -- 172.21.15.83:0/3313546426 wait complete. 2024-04-23T14:58:04.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:58:04.565 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:58:04.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.759+0000 7f9190c00640 1 -- 172.21.15.83:0/3031933634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f918c108580 msgr2=0x7f918c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.760 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.759+0000 7f9190c00640 1 --2- 172.21.15.83:0/3031933634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f918c108580 0x7f918c108a00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9174009930 tx=0x7f917402a420 comp rx=0 tx=0).stop 2024-04-23T14:58:04.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.759+0000 7f9190c00640 1 -- 172.21.15.83:0/3031933634 shutdown_connections 2024-04-23T14:58:04.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.759+0000 7f9190c00640 1 --2- 172.21.15.83:0/3031933634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f918c108f40 0x7f918c1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.759+0000 7f9190c00640 1 --2- 172.21.15.83:0/3031933634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f918c108580 0x7f918c108a00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.759+0000 7f9190c00640 1 --2- 172.21.15.83:0/3031933634 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c10d500 0x7f918c10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.759+0000 7f9190c00640 1 -- 172.21.15.83:0/3031933634 >> 172.21.15.83:0/3031933634 conn(0x7f918c077e20 msgr2=0x7f918c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:04.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.759+0000 7f9190c00640 1 -- 172.21.15.83:0/3031933634 shutdown_connections 2024-04-23T14:58:04.761 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.759+0000 7f9190c00640 1 -- 172.21.15.83:0/3031933634 wait complete. 2024-04-23T14:58:04.763 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.760+0000 7f9190c00640 1 Processor -- start 2024-04-23T14:58:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.760+0000 7f9190c00640 1 -- start start 2024-04-23T14:58:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.760+0000 7f9190c00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f918c108580 0x7f918c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.760+0000 7f9190c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c108f40 0x7f918c19f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.760+0000 7f9190c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f918c10d500 0x7f918c1a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.760+0000 7f9190c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f918c118920 con 0x7f918c108f40 2024-04-23T14:58:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.760+0000 7f9190c00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f918c1187a0 con 0x7f918c108580 2024-04-23T14:58:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.760+0000 7f9190c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f918c118aa0 con 0x7f918c10d500 2024-04-23T14:58:04.764 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.761+0000 7f9182a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c108f40 0x7f918c19f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.761+0000 7f9182a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c108f40 0x7f918c19f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46736/0 (socket says 172.21.15.83:46736) 2024-04-23T14:58:04.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.761+0000 7f918b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f918c108580 0x7f918c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.761+0000 7f9182a00640 1 -- 172.21.15.83:0/1017581518 learned_addr learned my addr 172.21.15.83:0/1017581518 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:04.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.761+0000 7f918be00640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f918c10d500 0x7f918c1a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.761+0000 7f918b400640 1 -- 172.21.15.83:0/1017581518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f918c10d500 msgr2=0x7f918c1a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.761+0000 7f918b400640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f918c10d500 0x7f918c1a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.761+0000 7f918b400640 1 -- 172.21.15.83:0/1017581518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c108f40 msgr2=0x7f918c19f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.761+0000 7f918b400640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c108f40 0x7f918c19f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.761+0000 7f918b400640 1 -- 172.21.15.83:0/1017581518 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c1a3d40 con 0x7f918c108580 2024-04-23T14:58:04.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.761+0000 7f9182a00640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c108f40 0x7f918c19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:04.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.761+0000 7f918b400640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f918c108580 0x7f918c19eda0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f917c00e9c0 tx=0x7f917c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:04.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.762+0000 7f9188c00640 1 -- 172.21.15.83:0/1017581518 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c00ccc0 con 0x7f918c108580 2024-04-23T14:58:04.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.762+0000 7f9188c00640 1 -- 172.21.15.83:0/1017581518 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f917c0035f0 con 0x7f918c108580 2024-04-23T14:58:04.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.762+0000 7f9190c00640 1 -- 172.21.15.83:0/1017581518 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c1a4030 con 0x7f918c108580 2024-04-23T14:58:04.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.762+0000 7f9188c00640 1 -- 172.21.15.83:0/1017581518 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c00ce60 con 0x7f918c108580 2024-04-23T14:58:04.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.762+0000 7f9190c00640 1 -- 172.21.15.83:0/1017581518 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c1ab8c0 con 0x7f918c108580 2024-04-23T14:58:04.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.766+0000 7f9194f95740 1 Processor -- start 2024-04-23T14:58:04.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.766+0000 7f9194f95740 1 -- start start 2024-04-23T14:58:04.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.766+0000 7f9188c00640 1 -- 172.21.15.83:0/1017581518 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f917c0037a0 con 0x7f918c108580 2024-04-23T14:58:04.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.767+0000 7f9194f95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 0x55e3e96cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.767+0000 7f9194f95740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 0x55e3e96d64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.767+0000 7f9194f95740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96d6a20 0x55e3e96d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.767+0000 7f9194f95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e3e96311f0 con 0x55e3e96cba30 2024-04-23T14:58:04.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.767+0000 7f9194f95740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e3e95d7350 con 0x55e3e96d6a20 2024-04-23T14:58:04.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.767+0000 7f9194f95740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e3e962d1b0 con 0x55e3e96cc350 2024-04-23T14:58:04.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.767+0000 7f918be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 0x55e3e96cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.767+0000 7f918b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 0x55e3e96d64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.767+0000 7f918be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 0x55e3e96cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46746/0 (socket says 172.21.15.83:46746) 2024-04-23T14:58:04.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.767+0000 7f918be00640 1 -- 172.21.15.83:0/1327577199 learned_addr learned my addr 172.21.15.83:0/1327577199 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:04.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.767+0000 7f9182a00640 1 --2- 172.21.15.83:0/1327577199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96d6a20 0x55e3e96d8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.767+0000 7f9182a00640 1 -- 172.21.15.83:0/1327577199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 msgr2=0x55e3e96d64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f9182a00640 1 --2- 172.21.15.83:0/1327577199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 0x55e3e96d64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f9182a00640 1 -- 172.21.15.83:0/1327577199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 msgr2=0x55e3e96cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f9182a00640 1 --2- 172.21.15.83:0/1327577199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 0x55e3e96cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f9182a00640 1 -- 172.21.15.83:0/1327577199 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3e962bf50 con 0x55e3e96d6a20 2024-04-23T14:58:04.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f918b400640 1 --2- 172.21.15.83:0/1327577199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 0x55e3e96d64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:58:04.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f918be00640 1 --2- 172.21.15.83:0/1327577199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 0x55e3e96cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:04.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f9182a00640 1 --2- 172.21.15.83:0/1327577199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96d6a20 0x55e3e96d8e10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9174000c00 tx=0x7f9174005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f08c90e4f6cada13 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:04.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f9188c00640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f91700a3d20 0x7f91700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f917be00640 1 -- 172.21.15.83:0/1327577199 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174035040 con 0x55e3e96d6a20 2024-04-23T14:58:04.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f917be00640 1 -- 172.21.15.83:0/1327577199 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9174004760 con 0x55e3e96d6a20 2024-04-23T14:58:04.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f917be00640 1 -- 172.21.15.83:0/1327577199 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91740038a0 con 0x55e3e96d6a20 2024-04-23T14:58:04.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f9188c00640 1 -- 172.21.15.83:0/1017581518 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f917c0d0c50 con 0x7f918c108580 2024-04-23T14:58:04.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.768+0000 7f918be00640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f91700a3d20 0x7f91700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.769+0000 7f9194f95740 1 -- 172.21.15.83:0/1327577199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96d6a20 msgr2=0x55e3e96d8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.769+0000 7f9194f95740 1 --2- 172.21.15.83:0/1327577199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96d6a20 0x55e3e96d8e10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9174000c00 tx=0x7f9174005de0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.769+0000 7f9194f95740 1 -- 172.21.15.83:0/1327577199 shutdown_connections 2024-04-23T14:58:04.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.769+0000 7f9194f95740 1 --2- 172.21.15.83:0/1327577199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96d6a20 0x55e3e96d8e10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.769+0000 7f9194f95740 1 --2- 172.21.15.83:0/1327577199 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 0x55e3e96d64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.769+0000 7f9194f95740 1 --2- 172.21.15.83:0/1327577199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 0x55e3e96cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.769+0000 7f9194f95740 1 -- 172.21.15.83:0/1327577199 >> 172.21.15.83:0/1327577199 conn(0x55e3e96c7d50 msgr2=0x55e3e96c8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:04.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.769+0000 7f9194f95740 1 -- 172.21.15.83:0/1327577199 shutdown_connections 2024-04-23T14:58:04.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.769+0000 7f918be00640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f91700a3d20 0x7f91700a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9184011870 tx=0x7f9184009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:04.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.769+0000 7f9194f95740 1 -- 172.21.15.83:0/1327577199 wait complete. 2024-04-23T14:58:04.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.769+0000 7f9194f95740 1 -- start start 2024-04-23T14:58:04.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.770+0000 7f9194f95740 1 Processor -- start 2024-04-23T14:58:04.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.770+0000 7f9194f95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 0x55e3e96d3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.770+0000 7f9194f95740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cc350 0x55e3e96d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.770+0000 7f9194f95740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96d6a20 0x55e3e96d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.770+0000 7f9194f95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e3e96d97b0 con 0x55e3e96cba30 2024-04-23T14:58:04.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.770+0000 7f9194f95740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55e3e95ea690 con 0x55e3e96cc350 2024-04-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.770+0000 7f9194f95740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55e3e96d9930 con 0x55e3e96d6a20 2024-04-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.770+0000 7f9194f95740 10 client.0 _ll_register_callbacks cb 0x55e3e9626650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.770+0000 7f918b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 0x55e3e96d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f9182a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cc350 0x55e3e96d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f9182a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cc350 0x55e3e96d3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:55938/0 (socket says 172.21.15.83:55938) 2024-04-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f9182a00640 1 -- 172.21.15.83:0/3507101763 learned_addr learned my addr 172.21.15.83:0/3507101763 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f918b400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 0x55e3e96d3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46748/0 (socket says 172.21.15.83:46748) 2024-04-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f918be00640 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96d6a20 0x55e3e96d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f9182a00640 1 -- 172.21.15.83:0/3507101763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96d6a20 msgr2=0x55e3e96d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f9182a00640 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96d6a20 0x55e3e96d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f9182a00640 1 -- 172.21.15.83:0/3507101763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 msgr2=0x55e3e96d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f9182a00640 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 0x55e3e96d3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f9182a00640 1 -- 172.21.15.83:0/3507101763 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3e95e9920 con 0x55e3e96cc350 2024-04-23T14:58:04.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f918b400640 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 0x55e3e96d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:04.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f9182a00640 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cc350 0x55e3e96d3ec0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9174003890 tx=0x7f91740038c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68a166c9564244ab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:04.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f9178c00640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:58:04.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.771+0000 7f9178c00640 1 -- 172.21.15.83:0/3507101763 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174052060 con 0x55e3e96cc350 2024-04-23T14:58:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9194f95740 1 -- 172.21.15.83:0/3507101763 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9174004760 con 0x55e3e96cc350 2024-04-23T14:58:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9178c00640 1 -- 172.21.15.83:0/3507101763 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f9174003cf0 con 0x55e3e96cc350 2024-04-23T14:58:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9178c00640 1 -- 172.21.15.83:0/3507101763 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9174035050 con 0x55e3e96cc350 2024-04-23T14:58:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9194f95740 10 client.27179 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9194f95740 1 -- 172.21.15.83:0/3507101763 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3e940d7d0 con 0x55e3e96cc350 2024-04-23T14:58:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9178c00640 1 -- 172.21.15.83:0/3507101763 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9174004760 con 0x55e3e96cc350 2024-04-23T14:58:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9178c00640 1 -- 172.21.15.83:0/3507101763 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f917403c6a0 con 0x55e3e96cc350 2024-04-23T14:58:04.781 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9194f95740 10 client.27179 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9194f95740 10 client.27179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9194f95740 20 client.27179 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9194f95740 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e3e96e6b90 0x55e3e96e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9194f95740 4 client.27179 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:58:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.772+0000 7f9194f95740 1 -- 172.21.15.83:0/3507101763 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3e963ec10 con 0x55e3e96e6b90 2024-04-23T14:58:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.773+0000 7f918b400640 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e3e96e6b90 0x55e3e96e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.773+0000 7f918b400640 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e3e96e6b90 0x55e3e96e8f80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=211c1c42f49926e server_cookie=f446e0a361746677 in_seq=0 out_seq=0 2024-04-23T14:58:04.782 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.773+0000 7f9178c00640 10 client.27179 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:04.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9178c00640 1 -- 172.21.15.83:0/3507101763 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3e963ec10 con 0x55e3e96e6b90 2024-04-23T14:58:04.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9178c00640 10 client.27179 handle_command_reply: tid=0 2024-04-23T14:58:04.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9194f95740 1 client.27179 shutdown 2024-04-23T14:58:04.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9194f95740 1 -- 172.21.15.83:0/3507101763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cc350 msgr2=0x55e3e96d3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9194f95740 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cc350 0x55e3e96d3ec0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9174003890 tx=0x7f91740038c0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9194f95740 1 -- 172.21.15.83:0/3507101763 shutdown_connections 2024-04-23T14:58:04.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9194f95740 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e3e96e6b90 0x55e3e96e8f80 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.783 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9194f95740 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96d6a20 0x55e3e96d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9194f95740 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cc350 0x55e3e96d3ec0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9178c00640 0 client.27179 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9194f95740 1 --2- 172.21.15.83:0/3507101763 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cba30 0x55e3e96d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9194f95740 1 -- 172.21.15.83:0/3507101763 >> 172.21.15.83:0/3507101763 conn(0x55e3e96c7d50 msgr2=0x55e3e96c6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9194f95740 1 -- 172.21.15.83:0/3507101763 shutdown_connections 2024-04-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.774+0000 7f9194f95740 1 -- 172.21.15.83:0/3507101763 wait complete. 2024-04-23T14:58:04.784 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.775+0000 7f9194f95740 20 client.27179 trim_cache size 0 max 16384 2024-04-23T14:58:04.794 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.793+0000 7f9194f95740 1 Processor -- start 2024-04-23T14:58:04.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.793+0000 7f9194f95740 1 -- start start 2024-04-23T14:58:04.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.794+0000 7f9194f95740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cba30 0x55e3e96e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.794+0000 7f9194f95740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 0x55e3e96ee550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.794+0000 7f9194f95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96d6a20 0x55e3e96eea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.794+0000 7f9194f95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f918c1b6a40 con 0x55e3e96d6a20 2024-04-23T14:58:04.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.794+0000 7f9194f95740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f918c1b4310 con 0x55e3e96cba30 2024-04-23T14:58:04.795 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.794+0000 7f9194f95740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f918c1b2180 con 0x55e3e96cc350 2024-04-23T14:58:04.797 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.794+0000 7f918be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96d6a20 0x55e3e96eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.794+0000 7f9182a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 0x55e3e96ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.794+0000 7f918b400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cba30 0x55e3e96e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.794+0000 7f918be00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96d6a20 0x55e3e96eea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46758/0 (socket says 172.21.15.83:46758) 2024-04-23T14:58:04.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.794+0000 7f9182a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 0x55e3e96ee550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:52632/0 (socket says 172.21.15.83:52632) 2024-04-23T14:58:04.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.794+0000 7f918be00640 1 -- 172.21.15.83:0/2343277078 learned_addr learned my addr 172.21.15.83:0/2343277078 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:04.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.795+0000 7f918b400640 1 -- 172.21.15.83:0/2343277078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 msgr2=0x55e3e96ee550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.795+0000 7f918b400640 1 --2- 172.21.15.83:0/2343277078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 0x55e3e96ee550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.795+0000 7f918b400640 1 -- 172.21.15.83:0/2343277078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96d6a20 msgr2=0x55e3e96eea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.795+0000 7f918b400640 1 --2- 172.21.15.83:0/2343277078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96d6a20 0x55e3e96eea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.795+0000 7f918b400640 1 -- 172.21.15.83:0/2343277078 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3e93433f0 con 0x55e3e96cba30 2024-04-23T14:58:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.795+0000 7f9182a00640 1 --2- 172.21.15.83:0/2343277078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 0x55e3e96ee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.795+0000 7f918b400640 1 --2- 172.21.15.83:0/2343277078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cba30 0x55e3e96e2670 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f917c0ce390 tx=0x7f917c0d3160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e908e544e9ec2ee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.795+0000 7f917b400640 1 -- 172.21.15.83:0/2343277078 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c0a0f90 con 0x55e3e96cba30 2024-04-23T14:58:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.795+0000 7f917b400640 1 -- 172.21.15.83:0/2343277078 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f917c0a5850 con 0x55e3e96cba30 2024-04-23T14:58:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.795+0000 7f917b400640 1 -- 172.21.15.83:0/2343277078 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c09c030 con 0x55e3e96cba30 2024-04-23T14:58:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.796+0000 7f9194f95740 1 -- 172.21.15.83:0/2343277078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cba30 msgr2=0x55e3e96e2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.796+0000 7f9194f95740 1 --2- 172.21.15.83:0/2343277078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cba30 0x55e3e96e2670 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f917c0ce390 tx=0x7f917c0d3160 comp rx=0 tx=0).stop 2024-04-23T14:58:04.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.796+0000 7f9194f95740 1 -- 172.21.15.83:0/2343277078 shutdown_connections 2024-04-23T14:58:04.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.796+0000 7f9194f95740 1 --2- 172.21.15.83:0/2343277078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96d6a20 0x55e3e96eea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.796+0000 7f9194f95740 1 --2- 172.21.15.83:0/2343277078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cc350 0x55e3e96ee550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.796+0000 7f9194f95740 1 --2- 172.21.15.83:0/2343277078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96cba30 0x55e3e96e2670 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.796+0000 7f9194f95740 1 -- 172.21.15.83:0/2343277078 >> 172.21.15.83:0/2343277078 conn(0x55e3e96c7d50 msgr2=0x55e3e96d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:04.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.796+0000 7f9194f95740 1 -- 172.21.15.83:0/2343277078 shutdown_connections 2024-04-23T14:58:04.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.796+0000 7f9194f95740 1 -- 172.21.15.83:0/2343277078 wait complete. 2024-04-23T14:58:04.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.796+0000 7f9194f95740 1 -- start start 2024-04-23T14:58:04.801 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f9194f95740 1 Processor -- start 2024-04-23T14:58:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f9194f95740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cba30 0x55e3e96d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f9194f95740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cc350 0x55e3e96d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f9194f95740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96d6a20 0x55e3e96d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f9194f95740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f918c1b27d0 con 0x55e3e96cc350 2024-04-23T14:58:04.804 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f9194f95740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f918c1b3c40 con 0x55e3e96d6a20 2024-04-23T14:58:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f9194f95740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f918c1b2ea0 con 0x55e3e96cba30 2024-04-23T14:58:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f9194f95740 10 client.0 _ll_register_callbacks cb 0x7f9184009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f9182a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cc350 0x55e3e96d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f9182a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cc350 0x55e3e96d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:46762/0 (socket says 172.21.15.83:46762) 2024-04-23T14:58:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f9182a00640 1 -- 172.21.15.83:0/4205573534 learned_addr learned my addr 172.21.15.83:0/4205573534 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f918b400640 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cba30 0x55e3e96d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.797+0000 7f918be00640 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96d6a20 0x55e3e96d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.798+0000 7f918b400640 1 -- 172.21.15.83:0/4205573534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96d6a20 msgr2=0x55e3e96d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.805 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.798+0000 7f918b400640 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96d6a20 0x55e3e96d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.798+0000 7f918b400640 1 -- 172.21.15.83:0/4205573534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cc350 msgr2=0x55e3e96d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.798+0000 7f918b400640 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cc350 0x55e3e96d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.798+0000 7f918b400640 1 -- 172.21.15.83:0/4205573534 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3e93030f0 con 0x55e3e96cba30 2024-04-23T14:58:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.798+0000 7f9182a00640 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cc350 0x55e3e96d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.798+0000 7f918b400640 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cba30 0x55e3e96d0930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f917c003db0 tx=0x7f917c0b6230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d3705f799f6014f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.798+0000 7f9178c00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:58:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.798+0000 7f9178c00640 1 -- 172.21.15.83:0/4205573534 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c003b10 con 0x55e3e96cba30 2024-04-23T14:58:04.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.798+0000 7f9178c00640 1 -- 172.21.15.83:0/4205573534 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8542+0+0 (secure 0 0 0) 0x7f917c099250 con 0x55e3e96cba30 2024-04-23T14:58:04.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.798+0000 7f9194f95740 1 -- 172.21.15.83:0/4205573534 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f917c0a5850 con 0x55e3e96cba30 2024-04-23T14:58:04.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.798+0000 7f9178c00640 1 -- 172.21.15.83:0/4205573534 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f917c0bd0b0 con 0x55e3e96cba30 2024-04-23T14:58:04.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.799+0000 7f9194f95740 10 client.27157 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:04.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.799+0000 7f9194f95740 1 -- 172.21.15.83:0/4205573534 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3e95f6810 con 0x55e3e96cba30 2024-04-23T14:58:04.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.799+0000 7f9178c00640 1 -- 172.21.15.83:0/4205573534 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f917c0a5850 con 0x55e3e96cba30 2024-04-23T14:58:04.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.799+0000 7f9178c00640 1 -- 172.21.15.83:0/4205573534 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f917c098a50 con 0x55e3e96cba30 2024-04-23T14:58:04.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.799+0000 7f9194f95740 10 client.27157 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:04.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.799+0000 7f9194f95740 10 client.27157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:04.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.799+0000 7f9194f95740 20 client.27157 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:04.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.799+0000 7f9194f95740 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e3e974d570 0x55e3e974f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.799+0000 7f9194f95740 4 client.27157 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:58:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.799+0000 7f9194f95740 1 -- 172.21.15.83:0/4205573534 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e3e9638a80 con 0x55e3e974d570 2024-04-23T14:58:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.799+0000 7f9182a00640 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e3e974d570 0x55e3e974f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.800+0000 7f9182a00640 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e3e974d570 0x55e3e974f960 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65fb1ea47b46b9e7 server_cookie=2068aaf2edbc2623 in_seq=0 out_seq=0 2024-04-23T14:58:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.800+0000 7f9178c00640 10 client.27157 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.800+0000 7f9178c00640 1 -- 172.21.15.83:0/4205573534 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e3e9638a80 con 0x55e3e974d570 2024-04-23T14:58:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.800+0000 7f9178c00640 10 client.27157 handle_command_reply: tid=0 2024-04-23T14:58:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.800+0000 7f9194f95740 1 client.27157 shutdown 2024-04-23T14:58:04.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.800+0000 7f9194f95740 1 -- 172.21.15.83:0/4205573534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cba30 msgr2=0x55e3e96d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.800+0000 7f9194f95740 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cba30 0x55e3e96d0930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f917c003db0 tx=0x7f917c0b6230 comp rx=0 tx=0).stop 2024-04-23T14:58:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.800+0000 7f9194f95740 1 -- 172.21.15.83:0/4205573534 shutdown_connections 2024-04-23T14:58:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.800+0000 7f9194f95740 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55e3e974d570 0x55e3e974f960 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.800+0000 7f9194f95740 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55e3e96d6a20 0x55e3e96d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.800+0000 7f9178c00640 0 client.27157 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.800+0000 7f9194f95740 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e3e96cc350 0x55e3e96d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.801+0000 7f9194f95740 1 --2- 172.21.15.83:0/4205573534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e3e96cba30 0x55e3e96d0930 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.801+0000 7f9194f95740 1 -- 172.21.15.83:0/4205573534 >> 172.21.15.83:0/4205573534 conn(0x55e3e96c7d50 msgr2=0x55e3e96cd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.801+0000 7f9194f95740 1 -- 172.21.15.83:0/4205573534 shutdown_connections 2024-04-23T14:58:04.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.801+0000 7f9194f95740 1 -- 172.21.15.83:0/4205573534 wait complete. 2024-04-23T14:58:04.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.801+0000 7f9194f95740 20 client.27157 trim_cache size 0 max 16384 2024-04-23T14:58:04.813 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:58:04.813 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:58:04.813 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:58:04.813 INFO:teuthology.orchestra.run.smithi083.stdout: "c234f018-3bec-4982-b652-5e65c8a6c36a": { 2024-04-23T14:58:04.813 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:58:04.814 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "c234f018-3bec-4982-b652-5e65c8a6c36a", 2024-04-23T14:58:04.814 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:58:04.814 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:58:04.814 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:58:04.814 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:58:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.801+0000 7f9190c00640 1 -- 172.21.15.83:0/1017581518 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f91700a3d20 msgr2=0x7f91700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.801+0000 7f9190c00640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f91700a3d20 0x7f91700a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9184011870 tx=0x7f9184009500 comp rx=0 tx=0).stop 2024-04-23T14:58:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.801+0000 7f9190c00640 1 -- 172.21.15.83:0/1017581518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f918c108580 msgr2=0x7f918c19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.801+0000 7f9190c00640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f918c108580 0x7f918c19eda0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f917c00e9c0 tx=0x7f917c00ee80 comp rx=0 tx=0).stop 2024-04-23T14:58:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.802+0000 7f9190c00640 1 -- 172.21.15.83:0/1017581518 shutdown_connections 2024-04-23T14:58:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.802+0000 7f9190c00640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f918c10d500 0x7f918c1a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.802+0000 7f9190c00640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f91700a3d20 0x7f91700a61e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.802+0000 7f9190c00640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f918c108f40 0x7f918c19f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.802+0000 7f9190c00640 1 --2- 172.21.15.83:0/1017581518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f918c108580 0x7f918c19eda0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:04.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.802+0000 7f9190c00640 1 -- 172.21.15.83:0/1017581518 >> 172.21.15.83:0/1017581518 conn(0x7f918c077e20 msgr2=0x7f918c078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:04.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.802+0000 7f9190c00640 1 -- 172.21.15.83:0/1017581518 shutdown_connections 2024-04-23T14:58:04.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:04.802+0000 7f9190c00640 1 -- 172.21.15.83:0/1017581518 wait complete. 2024-04-23T14:58:04.819 INFO:tasks.cephfs.filesystem:scrub status for tag:c234f018-3bec-4982-b652-5e65c8a6c36a - {'path': '/', 'tag': 'c234f018-3bec-4982-b652-5e65c8a6c36a', 'options': 'recursive,force'} 2024-04-23T14:58:05.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: pgmap v298: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:58:05.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:58:05.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:05.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:58:05.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:58:05.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:05.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:58:05.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:05.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:58:05.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:05.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:58:05.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:05.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:58:05.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:05.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:05 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: pgmap v298: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:58:06.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:58:06.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:58:06.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:58:06.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:58:06.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:58:06.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:58:06.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:58:06.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:05 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: pgmap v298: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:58:06.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:58:06.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:58:06.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:58:06.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:58:06.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:58:06.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:58:06.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:58:06.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:06.077 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:05 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:07.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:07 smithi203 ceph-mon[32800]: pgmap v299: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-23T14:58:08.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:07 smithi083 ceph-mon[27867]: pgmap v299: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-23T14:58:08.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:07 smithi173 ceph-mon[32443]: pgmap v299: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-23T14:58:09.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:09 smithi203 ceph-mon[32800]: pgmap v300: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-23T14:58:10.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:09 smithi083 ceph-mon[27867]: pgmap v300: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-23T14:58:10.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:09 smithi173 ceph-mon[32443]: pgmap v300: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-23T14:58:10.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:10 smithi203 ceph-mon[32800]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T14:58:11.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:10 smithi083 ceph-mon[27867]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T14:58:11.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:10 smithi173 ceph-mon[32443]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T14:58:11.974 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:11 smithi203 ceph-mon[32800]: pgmap v302: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-23T14:58:12.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:11 smithi083 ceph-mon[27867]: pgmap v302: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-23T14:58:12.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:11 smithi173 ceph-mon[32443]: pgmap v302: 97 pgs: 97 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-23T14:58:13.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:13 smithi203 ceph-mon[32800]: pgmap v303: 97 pgs: 14 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 55 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T14:58:14.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:13 smithi083 ceph-mon[27867]: pgmap v303: 97 pgs: 14 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 55 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T14:58:14.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:13 smithi173 ceph-mon[32443]: pgmap v303: 97 pgs: 14 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 55 active+clean; 16 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T14:58:15.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:15 smithi203 ceph-mon[32800]: pgmap v304: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:58:16.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:15 smithi083 ceph-mon[27867]: pgmap v304: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:58:16.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:15 smithi173 ceph-mon[32443]: pgmap v304: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 15 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:58:17.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:17 smithi203 ceph-mon[32800]: pgmap v305: 97 pgs: 19 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 42 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:58:18.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:17 smithi083 ceph-mon[27867]: pgmap v305: 97 pgs: 19 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 42 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:58:18.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:17 smithi173 ceph-mon[32443]: pgmap v305: 97 pgs: 19 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 42 active+clean; 14 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:58:19.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:19 smithi203 ceph-mon[32800]: pgmap v306: 97 pgs: 14 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 55 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:58:19.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:19 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:58:19.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:19 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:58:19.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:19 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:58:20.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:19 smithi083 ceph-mon[27867]: pgmap v306: 97 pgs: 14 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 55 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:58:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:19 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:58:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:19 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:58:20.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:19 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:58:20.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:19 smithi173 ceph-mon[32443]: pgmap v306: 97 pgs: 14 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 55 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:58:20.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:19 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:58:20.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:19 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:58:20.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:19 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:58:21.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:21 smithi203 ceph-mon[32800]: pgmap v307: 97 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 61 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 200 B/s wr, 6 op/s 2024-04-23T14:58:22.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:21 smithi083 ceph-mon[27867]: pgmap v307: 97 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 61 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 200 B/s wr, 6 op/s 2024-04-23T14:58:22.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:21 smithi173 ceph-mon[32443]: pgmap v307: 97 pgs: 12 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 61 active+clean; 13 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 200 B/s wr, 6 op/s 2024-04-23T14:58:23.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:23 smithi203 ceph-mon[32800]: pgmap v308: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-23T14:58:24.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:23 smithi083 ceph-mon[27867]: pgmap v308: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-23T14:58:24.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:23 smithi173 ceph-mon[32443]: pgmap v308: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-23T14:58:25.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:25 smithi203 ceph-mon[32800]: pgmap v309: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-23T14:58:26.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:25 smithi083 ceph-mon[27867]: pgmap v309: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-23T14:58:26.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:25 smithi173 ceph-mon[32443]: pgmap v309: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-23T14:58:27.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:27 smithi203 ceph-mon[32800]: pgmap v310: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-23T14:58:27.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:27 smithi203 ceph-mon[32800]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T14:58:28.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:27 smithi083 ceph-mon[27867]: pgmap v310: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-23T14:58:28.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:27 smithi083 ceph-mon[27867]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T14:58:28.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:27 smithi173 ceph-mon[32443]: pgmap v310: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-23T14:58:28.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:27 smithi173 ceph-mon[32443]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T14:58:29.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:29 smithi203 ceph-mon[32800]: pgmap v312: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:58:30.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:29 smithi083 ceph-mon[27867]: pgmap v312: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:58:30.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:29 smithi173 ceph-mon[32443]: pgmap v312: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:58:31.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:31 smithi203 ceph-mon[32800]: pgmap v313: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:32.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:31 smithi083 ceph-mon[27867]: pgmap v313: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:32.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:31 smithi173 ceph-mon[32443]: pgmap v313: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:33.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:32 smithi083 ceph-mon[27867]: pgmap v314: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:33.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:32 smithi173 ceph-mon[32443]: pgmap v314: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:33.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:32 smithi203 ceph-mon[32800]: pgmap v314: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:34.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:58:34.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:58:35.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.077+0000 7f4150000640 1 -- 172.21.15.83:0/3490589888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f414810e660 msgr2=0x7f414810ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.077+0000 7f4150000640 1 --2- 172.21.15.83:0/3490589888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f414810e660 0x7f414810ea40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f413c009900 tx=0x7f413c02a3f0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.077+0000 7f4150000640 1 -- 172.21.15.83:0/3490589888 shutdown_connections 2024-04-23T14:58:35.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.077+0000 7f4150000640 1 --2- 172.21.15.83:0/3490589888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4148108fd0 0x7f4148115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.077+0000 7f4150000640 1 --2- 172.21.15.83:0/3490589888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4148108610 0x7f4148108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.077+0000 7f4150000640 1 --2- 172.21.15.83:0/3490589888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f414810e660 0x7f414810ea40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.077+0000 7f4150000640 1 -- 172.21.15.83:0/3490589888 >> 172.21.15.83:0/3490589888 conn(0x7f4148076540 msgr2=0x7f4148078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.077+0000 7f4150000640 1 -- 172.21.15.83:0/3490589888 shutdown_connections 2024-04-23T14:58:35.078 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.077+0000 7f4150000640 1 -- 172.21.15.83:0/3490589888 wait complete. 2024-04-23T14:58:35.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.078+0000 7f4150000640 1 Processor -- start 2024-04-23T14:58:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.078+0000 7f4150000640 1 -- start start 2024-04-23T14:58:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f4150000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4148108610 0x7f414819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f4150000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4148108fd0 0x7f414819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f4150000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f414810e660 0x7f41481a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f4150000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4148118a30 con 0x7f4148108610 2024-04-23T14:58:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f4150000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f41481188b0 con 0x7f414810e660 2024-04-23T14:58:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f4150000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4148118bb0 con 0x7f4148108fd0 2024-04-23T14:58:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f414d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4148108610 0x7f414819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f414d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4148108610 0x7f414819ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52634/0 (socket says 172.21.15.83:52634) 2024-04-23T14:58:35.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f414d600640 1 -- 172.21.15.83:0/3519478989 learned_addr learned my addr 172.21.15.83:0/3519478989 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f414cc00640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4148108fd0 0x7f414819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f414e000640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f414810e660 0x7f41481a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f414d600640 1 -- 172.21.15.83:0/3519478989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4148108fd0 msgr2=0x7f414819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f414d600640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4148108fd0 0x7f414819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f414d600640 1 -- 172.21.15.83:0/3519478989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f414810e660 msgr2=0x7f41481a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f414d600640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f414810e660 0x7f41481a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.079+0000 7f414d600640 1 -- 172.21.15.83:0/3519478989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41481a3e10 con 0x7f4148108610 2024-04-23T14:58:35.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.080+0000 7f414d600640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4148108610 0x7f414819ee70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f413c02a910 tx=0x7f413c003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.080+0000 7f4146000640 1 -- 172.21.15.83:0/3519478989 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c03f070 con 0x7f4148108610 2024-04-23T14:58:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.080+0000 7f4150000640 1 -- 172.21.15.83:0/3519478989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41481a40a0 con 0x7f4148108610 2024-04-23T14:58:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.080+0000 7f4146000640 1 -- 172.21.15.83:0/3519478989 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413c031c20 con 0x7f4148108610 2024-04-23T14:58:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.080+0000 7f4146000640 1 -- 172.21.15.83:0/3519478989 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c035030 con 0x7f4148108610 2024-04-23T14:58:35.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.080+0000 7f4150000640 1 -- 172.21.15.83:0/3519478989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41481ab990 con 0x7f4148108610 2024-04-23T14:58:35.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f4152e2e740 1 Processor -- start 2024-04-23T14:58:35.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f4152e2e740 1 -- start start 2024-04-23T14:58:35.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f4146000640 1 -- 172.21.15.83:0/3519478989 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f413c042070 con 0x7f4148108610 2024-04-23T14:58:35.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f4152e2e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa4860 0x5613e7aa4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f4152e2e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aa5180 0x5613e7aaf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f4152e2e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aaf850 0x5613e7ab1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f4152e2e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5613e7a0a1f0 con 0x5613e7aa5180 2024-04-23T14:58:35.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f4152e2e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5613e79b0350 con 0x5613e7aa4860 2024-04-23T14:58:35.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f4152e2e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5613e7a061b0 con 0x5613e7aaf850 2024-04-23T14:58:35.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f414cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aaf850 0x5613e7ab1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f414d600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aa5180 0x5613e7aaf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f414cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aaf850 0x5613e7ab1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41048/0 (socket says 172.21.15.83:41048) 2024-04-23T14:58:35.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.084+0000 7f414cc00640 1 -- 172.21.15.83:0/2567726580 learned_addr learned my addr 172.21.15.83:0/2567726580 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:35.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.085+0000 7f414e000640 1 --2- 172.21.15.83:0/2567726580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa4860 0x5613e7aa4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.085+0000 7f414cc00640 1 -- 172.21.15.83:0/2567726580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa4860 msgr2=0x5613e7aa4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.085+0000 7f414cc00640 1 --2- 172.21.15.83:0/2567726580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa4860 0x5613e7aa4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.085+0000 7f414cc00640 1 -- 172.21.15.83:0/2567726580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aa5180 msgr2=0x5613e7aaf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.085+0000 7f414cc00640 1 --2- 172.21.15.83:0/2567726580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aa5180 0x5613e7aaf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.085+0000 7f414cc00640 1 -- 172.21.15.83:0/2567726580 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613e7a04f50 con 0x5613e7aaf850 2024-04-23T14:58:35.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.085+0000 7f414cc00640 1 --2- 172.21.15.83:0/2567726580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aaf850 0x5613e7ab1c40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4134009c90 tx=0x7f4134011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=604af20aaf87018d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.085+0000 7f4146000640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f41300a3da0 0x7f41300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f413a600640 1 -- 172.21.15.83:0/2567726580 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413401e070 con 0x5613e7aaf850 2024-04-23T14:58:35.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f413a600640 1 -- 172.21.15.83:0/2567726580 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4134019b10 con 0x5613e7aaf850 2024-04-23T14:58:35.088 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f413a600640 1 -- 172.21.15.83:0/2567726580 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413400ea60 con 0x5613e7aaf850 2024-04-23T14:58:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f4146000640 1 -- 172.21.15.83:0/3519478989 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f413c0f0a30 con 0x7f4148108610 2024-04-23T14:58:35.089 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f414e000640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f41300a3da0 0x7f41300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f4152e2e740 1 -- 172.21.15.83:0/2567726580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aaf850 msgr2=0x5613e7ab1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f4152e2e740 1 --2- 172.21.15.83:0/2567726580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aaf850 0x5613e7ab1c40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4134009c90 tx=0x7f4134011810 comp rx=0 tx=0).stop 2024-04-23T14:58:35.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f414e000640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f41300a3da0 0x7f41300a6260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4140005fd0 tx=0x7f4140009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f4152e2e740 1 -- 172.21.15.83:0/2567726580 shutdown_connections 2024-04-23T14:58:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f4152e2e740 1 --2- 172.21.15.83:0/2567726580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aaf850 0x5613e7ab1c40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f4152e2e740 1 --2- 172.21.15.83:0/2567726580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aa5180 0x5613e7aaf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f4152e2e740 1 --2- 172.21.15.83:0/2567726580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa4860 0x5613e7aa4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.086+0000 7f4152e2e740 1 -- 172.21.15.83:0/2567726580 >> 172.21.15.83:0/2567726580 conn(0x5613e7aa0b80 msgr2=0x5613e7aa0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.087+0000 7f4152e2e740 1 -- 172.21.15.83:0/2567726580 shutdown_connections 2024-04-23T14:58:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.087+0000 7f4152e2e740 1 -- 172.21.15.83:0/2567726580 wait complete. 2024-04-23T14:58:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.087+0000 7f4152e2e740 1 -- start start 2024-04-23T14:58:35.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.087+0000 7f4152e2e740 1 Processor -- start 2024-04-23T14:58:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f4152e2e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 0x5613e7aac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f4152e2e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa5180 0x5613e7aacc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f4152e2e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7aad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f4152e2e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5613e7ab2640 con 0x5613e7aaf850 2024-04-23T14:58:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f4152e2e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x5613e79c3690 con 0x5613e7aa5180 2024-04-23T14:58:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f4152e2e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5613e7ab27c0 con 0x5613e7aa4860 2024-04-23T14:58:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f4152e2e740 10 client.0 _ll_register_callbacks cb 0x5613e79ff650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f414d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 0x5613e7aac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.095 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f414cc00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa5180 0x5613e7aacc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f414e000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7aad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f414d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 0x5613e7aac730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41060/0 (socket says 172.21.15.83:41060) 2024-04-23T14:58:35.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f414d600640 1 -- 172.21.15.83:0/978331125 learned_addr learned my addr 172.21.15.83:0/978331125 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:35.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f414d600640 1 -- 172.21.15.83:0/978331125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa5180 msgr2=0x5613e7aacc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f414d600640 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa5180 0x5613e7aacc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f414d600640 1 -- 172.21.15.83:0/978331125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 msgr2=0x5613e7aad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f414d600640 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7aad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f414d600640 1 -- 172.21.15.83:0/978331125 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613e79c2920 con 0x5613e7aa4860 2024-04-23T14:58:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f414cc00640 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa5180 0x5613e7aacc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.088+0000 7f414e000640 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7aad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.089+0000 7f414d600640 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 0x5613e7aac730 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f413c0bc9a0 tx=0x7f413c02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e2a4606da0e339e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.089+0000 7f412ea00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:58:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.089+0000 7f412ea00640 1 -- 172.21.15.83:0/978331125 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c049070 con 0x5613e7aa4860 2024-04-23T14:58:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.089+0000 7f4152e2e740 1 -- 172.21.15.83:0/978331125 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4134019b10 con 0x5613e7aa4860 2024-04-23T14:58:35.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.089+0000 7f412ea00640 1 -- 172.21.15.83:0/978331125 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f413c0b9080 con 0x5613e7aa4860 2024-04-23T14:58:35.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.089+0000 7f412ea00640 1 -- 172.21.15.83:0/978331125 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f413c0bc550 con 0x5613e7aa4860 2024-04-23T14:58:35.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.089+0000 7f4152e2e740 10 client.27169 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:35.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.089+0000 7f4152e2e740 1 -- 172.21.15.83:0/978331125 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5613e77e67d0 con 0x5613e7aa4860 2024-04-23T14:58:35.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.089+0000 7f412ea00640 1 -- 172.21.15.83:0/978331125 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4134019b10 con 0x5613e7aa4860 2024-04-23T14:58:35.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.090+0000 7f412ea00640 1 -- 172.21.15.83:0/978331125 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f413c04c060 con 0x5613e7aa4860 2024-04-23T14:58:35.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.090+0000 7f4152e2e740 10 client.27169 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:35.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.090+0000 7f4152e2e740 10 client.27169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:35.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.090+0000 7f4152e2e740 20 client.27169 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:35.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.090+0000 7f4152e2e740 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613e7abfa90 0x5613e7ac1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.090+0000 7f4152e2e740 4 client.27169 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:58:35.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.090+0000 7f4152e2e740 1 -- 172.21.15.83:0/978331125 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613e7a17c10 con 0x5613e7abfa90 2024-04-23T14:58:35.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.090+0000 7f414cc00640 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613e7abfa90 0x5613e7ac1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.091+0000 7f414cc00640 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613e7abfa90 0x5613e7ac1ee0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91de461be822f943 server_cookie=a822040101e464f1 in_seq=0 out_seq=0 2024-04-23T14:58:35.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.091+0000 7f412ea00640 10 client.27169 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:35.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f412ea00640 1 -- 172.21.15.83:0/978331125 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613e7a17c10 con 0x5613e7abfa90 2024-04-23T14:58:35.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f412ea00640 10 client.27169 handle_command_reply: tid=0 2024-04-23T14:58:35.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f4152e2e740 1 client.27169 shutdown 2024-04-23T14:58:35.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f4152e2e740 1 -- 172.21.15.83:0/978331125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 msgr2=0x5613e7aac730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f4152e2e740 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 0x5613e7aac730 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f413c0bc9a0 tx=0x7f413c02a3f0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f4152e2e740 1 -- 172.21.15.83:0/978331125 shutdown_connections 2024-04-23T14:58:35.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f4152e2e740 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613e7abfa90 0x5613e7ac1ee0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f4152e2e740 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7aad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f4152e2e740 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa5180 0x5613e7aacc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f4152e2e740 1 --2- 172.21.15.83:0/978331125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 0x5613e7aac730 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f412ea00640 0 client.27169 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:35.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f4152e2e740 1 -- 172.21.15.83:0/978331125 >> 172.21.15.83:0/978331125 conn(0x5613e7aa0b80 msgr2=0x5613e7aa6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f4152e2e740 1 -- 172.21.15.83:0/978331125 shutdown_connections 2024-04-23T14:58:35.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.092+0000 7f4152e2e740 1 -- 172.21.15.83:0/978331125 wait complete. 2024-04-23T14:58:35.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.093+0000 7f4152e2e740 20 client.27169 trim_cache size 0 max 16384 2024-04-23T14:58:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.111+0000 7f4152e2e740 1 Processor -- start 2024-04-23T14:58:35.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.112+0000 7f4152e2e740 1 -- start start 2024-04-23T14:58:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.112+0000 7f4152e2e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 0x5613e7abb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.112+0000 7f4152e2e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa5180 0x5613e7aa6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.112+0000 7f4152e2e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7ac77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.112+0000 7f4152e2e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41481b6ba0 con 0x5613e7aaf850 2024-04-23T14:58:35.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.112+0000 7f4152e2e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f41481b4470 con 0x5613e7aa5180 2024-04-23T14:58:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.112+0000 7f4152e2e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41481b22e0 con 0x5613e7aa4860 2024-04-23T14:58:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.112+0000 7f414d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 0x5613e7abb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.112+0000 7f414d600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 0x5613e7abb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41074/0 (socket says 172.21.15.83:41074) 2024-04-23T14:58:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.112+0000 7f414d600640 1 -- 172.21.15.83:0/827688292 learned_addr learned my addr 172.21.15.83:0/827688292 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.112+0000 7f414e000640 1 --2- 172.21.15.83:0/827688292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7ac77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.112+0000 7f414cc00640 1 --2- 172.21.15.83:0/827688292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa5180 0x5613e7aa6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.113+0000 7f414d600640 1 -- 172.21.15.83:0/827688292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa5180 msgr2=0x5613e7aa6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.113+0000 7f414d600640 1 --2- 172.21.15.83:0/827688292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa5180 0x5613e7aa6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.113+0000 7f414d600640 1 -- 172.21.15.83:0/827688292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 msgr2=0x5613e7ac77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.113+0000 7f414d600640 1 --2- 172.21.15.83:0/827688292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7ac77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.113+0000 7f414d600640 1 -- 172.21.15.83:0/827688292 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613e771c3f0 con 0x5613e7aa4860 2024-04-23T14:58:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.113+0000 7f414d600640 1 --2- 172.21.15.83:0/827688292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 0x5613e7abb530 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f413c0b8e70 tx=0x7f413c03b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8979ff352b155f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.113+0000 7f4139c00640 1 -- 172.21.15.83:0/827688292 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c049070 con 0x5613e7aa4860 2024-04-23T14:58:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.113+0000 7f4139c00640 1 -- 172.21.15.83:0/827688292 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413c0c55d0 con 0x5613e7aa4860 2024-04-23T14:58:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.113+0000 7f4139c00640 1 -- 172.21.15.83:0/827688292 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c0bc370 con 0x5613e7aa4860 2024-04-23T14:58:35.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.114+0000 7f4152e2e740 1 -- 172.21.15.83:0/827688292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 msgr2=0x5613e7abb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.114+0000 7f4152e2e740 1 --2- 172.21.15.83:0/827688292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 0x5613e7abb530 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f413c0b8e70 tx=0x7f413c03b720 comp rx=0 tx=0).stop 2024-04-23T14:58:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.114+0000 7f4152e2e740 1 -- 172.21.15.83:0/827688292 shutdown_connections 2024-04-23T14:58:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.114+0000 7f4152e2e740 1 --2- 172.21.15.83:0/827688292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7ac77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.114+0000 7f4152e2e740 1 --2- 172.21.15.83:0/827688292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa5180 0x5613e7aa6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.114+0000 7f4152e2e740 1 --2- 172.21.15.83:0/827688292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa4860 0x5613e7abb530 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.114+0000 7f4152e2e740 1 -- 172.21.15.83:0/827688292 >> 172.21.15.83:0/827688292 conn(0x5613e7aa0b80 msgr2=0x5613e7aab000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.114+0000 7f4152e2e740 1 -- 172.21.15.83:0/827688292 shutdown_connections 2024-04-23T14:58:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.114+0000 7f4152e2e740 1 -- 172.21.15.83:0/827688292 wait complete. 2024-04-23T14:58:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.114+0000 7f4152e2e740 1 -- start start 2024-04-23T14:58:35.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f4152e2e740 1 Processor -- start 2024-04-23T14:58:35.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f4152e2e740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa4860 0x5613e7ab7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f4152e2e740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa5180 0x5613e7ab7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f4152e2e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7aa8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f4152e2e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41481b2930 con 0x5613e7aaf850 2024-04-23T14:58:35.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f4152e2e740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f41481b3da0 con 0x5613e7aa4860 2024-04-23T14:58:35.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f4152e2e740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f41481b3000 con 0x5613e7aa5180 2024-04-23T14:58:35.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f4152e2e740 10 client.0 _ll_register_callbacks cb 0x7f413c0f41d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:35.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f414cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa5180 0x5613e7ab7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f414cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa5180 0x5613e7ab7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41080/0 (socket says 172.21.15.83:41080) 2024-04-23T14:58:35.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f414cc00640 1 -- 172.21.15.83:0/3813265532 learned_addr learned my addr 172.21.15.83:0/3813265532 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:35.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f414e000640 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7aa8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.115+0000 7f414d600640 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa4860 0x5613e7ab7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.116+0000 7f414cc00640 1 -- 172.21.15.83:0/3813265532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa4860 msgr2=0x5613e7ab7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.116+0000 7f414cc00640 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa4860 0x5613e7ab7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.116+0000 7f414cc00640 1 -- 172.21.15.83:0/3813265532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 msgr2=0x5613e7aa8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.116+0000 7f414cc00640 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7aa8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.116+0000 7f414cc00640 1 -- 172.21.15.83:0/3813265532 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613e76dc0f0 con 0x5613e7aa5180 2024-04-23T14:58:35.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.116+0000 7f414cc00640 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa5180 0x5613e7ab7f80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f413400e6b0 tx=0x7f413400e5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=740b0aad31a99dee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.116+0000 7f412ea00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:58:35.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.116+0000 7f412ea00640 1 -- 172.21.15.83:0/3813265532 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413401e040 con 0x5613e7aa5180 2024-04-23T14:58:35.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.116+0000 7f412ea00640 1 -- 172.21.15.83:0/3813265532 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f4134005040 con 0x5613e7aa5180 2024-04-23T14:58:35.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.116+0000 7f4152e2e740 1 -- 172.21.15.83:0/3813265532 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f413c0c55d0 con 0x5613e7aa5180 2024-04-23T14:58:35.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.117+0000 7f412ea00640 1 -- 172.21.15.83:0/3813265532 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f413400e3b0 con 0x5613e7aa5180 2024-04-23T14:58:35.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.117+0000 7f4152e2e740 10 client.27175 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:35.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.117+0000 7f4152e2e740 1 -- 172.21.15.83:0/3813265532 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5613e79cf810 con 0x5613e7aa5180 2024-04-23T14:58:35.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.117+0000 7f412ea00640 1 -- 172.21.15.83:0/3813265532 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f413c0c55d0 con 0x5613e7aa5180 2024-04-23T14:58:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.117+0000 7f412ea00640 1 -- 172.21.15.83:0/3813265532 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f413400eb00 con 0x5613e7aa5180 2024-04-23T14:58:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.117+0000 7f4152e2e740 10 client.27175 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.117+0000 7f4152e2e740 10 client.27175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.117+0000 7f4152e2e740 20 client.27175 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.117+0000 7f4152e2e740 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613e7b25f20 0x5613e7b28340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.117+0000 7f4152e2e740 4 client.27175 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:58:35.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.117+0000 7f4152e2e740 1 -- 172.21.15.83:0/3813265532 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5613e7a11a80 con 0x5613e7b25f20 2024-04-23T14:58:35.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.118+0000 7f414d600640 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613e7b25f20 0x5613e7b28340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.118+0000 7f414d600640 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613e7b25f20 0x5613e7b28340 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b56f36c65b30302d server_cookie=b4f00f58a5417fde in_seq=0 out_seq=0 2024-04-23T14:58:35.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.118+0000 7f412ea00640 10 client.27175 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:35.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.118+0000 7f412ea00640 1 -- 172.21.15.83:0/3813265532 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5613e7a11a80 con 0x5613e7b25f20 2024-04-23T14:58:35.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.118+0000 7f412ea00640 10 client.27175 handle_command_reply: tid=0 2024-04-23T14:58:35.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.118+0000 7f4152e2e740 1 client.27175 shutdown 2024-04-23T14:58:35.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.118+0000 7f4152e2e740 1 -- 172.21.15.83:0/3813265532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa5180 msgr2=0x5613e7ab7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.118+0000 7f4152e2e740 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa5180 0x5613e7ab7f80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f413400e6b0 tx=0x7f413400e5a0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.119+0000 7f4152e2e740 1 -- 172.21.15.83:0/3813265532 shutdown_connections 2024-04-23T14:58:35.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.119+0000 7f4152e2e740 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x5613e7b25f20 0x5613e7b28340 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.119+0000 7f4152e2e740 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5613e7aaf850 0x5613e7aa8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.119+0000 7f4152e2e740 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5613e7aa5180 0x5613e7ab7f80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.119+0000 7f412ea00640 0 client.27175 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:35.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.119+0000 7f4152e2e740 1 --2- 172.21.15.83:0/3813265532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5613e7aa4860 0x5613e7ab7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.119+0000 7f4152e2e740 1 -- 172.21.15.83:0/3813265532 >> 172.21.15.83:0/3813265532 conn(0x5613e7aa0b80 msgr2=0x5613e7a9f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.119+0000 7f4152e2e740 1 -- 172.21.15.83:0/3813265532 shutdown_connections 2024-04-23T14:58:35.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.119+0000 7f4152e2e740 1 -- 172.21.15.83:0/3813265532 wait complete. 2024-04-23T14:58:35.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.119+0000 7f4152e2e740 20 client.27175 trim_cache size 0 max 16384 2024-04-23T14:58:35.128 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:58:35.128 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:58:35.128 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:58:35.128 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:58:35.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.119+0000 7f4150000640 1 -- 172.21.15.83:0/3519478989 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f41300a3da0 msgr2=0x7f41300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.119+0000 7f4150000640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f41300a3da0 0x7f41300a6260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4140005fd0 tx=0x7f4140009500 comp rx=0 tx=0).stop 2024-04-23T14:58:35.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.120+0000 7f4150000640 1 -- 172.21.15.83:0/3519478989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4148108610 msgr2=0x7f414819ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.120+0000 7f4150000640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4148108610 0x7f414819ee70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f413c02a910 tx=0x7f413c003b20 comp rx=0 tx=0).stop 2024-04-23T14:58:35.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.120+0000 7f4150000640 1 -- 172.21.15.83:0/3519478989 shutdown_connections 2024-04-23T14:58:35.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.120+0000 7f4150000640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f414810e660 0x7f41481a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.120+0000 7f4150000640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4148108fd0 0x7f414819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.120+0000 7f4150000640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f41300a3da0 0x7f41300a6260 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.120+0000 7f4150000640 1 --2- 172.21.15.83:0/3519478989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4148108610 0x7f414819ee70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.120+0000 7f4150000640 1 -- 172.21.15.83:0/3519478989 >> 172.21.15.83:0/3519478989 conn(0x7f4148076540 msgr2=0x7f4148077b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.120+0000 7f4150000640 1 -- 172.21.15.83:0/3519478989 shutdown_connections 2024-04-23T14:58:35.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.120+0000 7f4150000640 1 -- 172.21.15.83:0/3519478989 wait complete. 2024-04-23T14:58:35.137 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:58:35.137 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:58:35.200 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:34 smithi083 ceph-mon[27867]: pgmap v315: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:35.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:34 smithi203 ceph-mon[32800]: pgmap v315: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:35.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:34 smithi173 ceph-mon[32443]: pgmap v315: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:35.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.334+0000 7f8bd3e00640 1 -- 172.21.15.83:0/3066945903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd4104260 msgr2=0x7f8bd4104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.334+0000 7f8bd3e00640 1 --2- 172.21.15.83:0/3066945903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd4104260 0x7f8bd4104640 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009900 tx=0x7f8bc402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.334+0000 7f8bd3e00640 1 -- 172.21.15.83:0/3066945903 shutdown_connections 2024-04-23T14:58:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.334+0000 7f8bd3e00640 1 --2- 172.21.15.83:0/3066945903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd4109330 0x7f8bd4110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.334+0000 7f8bd3e00640 1 --2- 172.21.15.83:0/3066945903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bd4104c10 0x7f8bd4108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.334+0000 7f8bd3e00640 1 --2- 172.21.15.83:0/3066945903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd4104260 0x7f8bd4104640 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.334+0000 7f8bd3e00640 1 -- 172.21.15.83:0/3066945903 >> 172.21.15.83:0/3066945903 conn(0x7f8bd40ffaa0 msgr2=0x7f8bd4101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.335+0000 7f8bd3e00640 1 -- 172.21.15.83:0/3066945903 shutdown_connections 2024-04-23T14:58:35.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.335+0000 7f8bd3e00640 1 -- 172.21.15.83:0/3066945903 wait complete. 2024-04-23T14:58:35.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.335+0000 7f8bd3e00640 1 Processor -- start 2024-04-23T14:58:35.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.336+0000 7f8bd3e00640 1 -- start start 2024-04-23T14:58:35.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.336+0000 7f8bd3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd4104260 0x7f8bd41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.336+0000 7f8bd3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bd4109330 0x7f8bd41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.336+0000 7f8bd3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd419a750 0x7f8bd419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.336+0000 7f8bd3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4114410 con 0x7f8bd4104260 2024-04-23T14:58:35.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.336+0000 7f8bd3e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4114290 con 0x7f8bd4109330 2024-04-23T14:58:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.336+0000 7f8bd3e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4114590 con 0x7f8bd419a750 2024-04-23T14:58:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.336+0000 7f8bd2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd4104260 0x7f8bd41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.336+0000 7f8bd2a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd4104260 0x7f8bd41a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52670/0 (socket says 172.21.15.83:52670) 2024-04-23T14:58:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.336+0000 7f8bd2a00640 1 -- 172.21.15.83:0/2365167563 learned_addr learned my addr 172.21.15.83:0/2365167563 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.336+0000 7f8bd3400640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd419a750 0x7f8bd419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.336+0000 7f8bcbe00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bd4109330 0x7f8bd41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.337+0000 7f8bd2a00640 1 -- 172.21.15.83:0/2365167563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd419a750 msgr2=0x7f8bd419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.337+0000 7f8bd2a00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd419a750 0x7f8bd419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.337+0000 7f8bd2a00640 1 -- 172.21.15.83:0/2365167563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bd4109330 msgr2=0x7f8bd41a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.337+0000 7f8bd2a00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bd4109330 0x7f8bd41a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.337+0000 7f8bd2a00640 1 -- 172.21.15.83:0/2365167563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd419b350 con 0x7f8bd4104260 2024-04-23T14:58:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.337+0000 7f8bcbe00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bd4109330 0x7f8bd41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.337+0000 7f8bd2a00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd4104260 0x7f8bd41a0690 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4004770 tx=0x7f8bc4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.337+0000 7f8bcb400640 1 -- 172.21.15.83:0/2365167563 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc403f070 con 0x7f8bd4104260 2024-04-23T14:58:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.337+0000 7f8bcb400640 1 -- 172.21.15.83:0/2365167563 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc4031ca0 con 0x7f8bd4104260 2024-04-23T14:58:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.337+0000 7f8bd3e00640 1 -- 172.21.15.83:0/2365167563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd419b5e0 con 0x7f8bd4104260 2024-04-23T14:58:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.337+0000 7f8bcb400640 1 -- 172.21.15.83:0/2365167563 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4035030 con 0x7f8bd4104260 2024-04-23T14:58:35.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.338+0000 7f8bd3e00640 1 -- 172.21.15.83:0/2365167563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd41a7720 con 0x7f8bd4104260 2024-04-23T14:58:35.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.339+0000 7f8bd3e00640 1 -- 172.21.15.83:0/2365167563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd4105730 con 0x7f8bd4104260 2024-04-23T14:58:35.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.341+0000 7f8bcb400640 1 -- 172.21.15.83:0/2365167563 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8bc4042070 con 0x7f8bd4104260 2024-04-23T14:58:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.343+0000 7f8bcb400640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f8bb80a3d20 0x7f8bb80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.343+0000 7f8bcb400640 1 -- 172.21.15.83:0/2365167563 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f8bc40f0e20 con 0x7f8bd4104260 2024-04-23T14:58:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.343+0000 7f8bcbe00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f8bb80a3d20 0x7f8bb80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.344+0000 7f8bcbe00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f8bb80a3d20 0x7f8bb80a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8bd419bfa0 tx=0x7f8bbc005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.348+0000 7f8bcb400640 1 -- 172.21.15.83:0/2365167563 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8bc40bd450 con 0x7f8bd4104260 2024-04-23T14:58:35.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.587+0000 7f8bd3e00640 1 -- 172.21.15.83:0/2365167563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bd410c590 con 0x7f8bd4104260 2024-04-23T14:58:35.589 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.589+0000 7f8bcb400640 1 -- 172.21.15.83:0/2365167563 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7f8bc40bcd50 con 0x7f8bd4104260 2024-04-23T14:58:35.589 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:58:35.590 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:58:35.590 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:58:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.592+0000 7f8bd3e00640 1 -- 172.21.15.83:0/2365167563 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f8bb80a3d20 msgr2=0x7f8bb80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.592+0000 7f8bd3e00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f8bb80a3d20 0x7f8bb80a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8bd419bfa0 tx=0x7f8bbc005f90 comp rx=0 tx=0).stop 2024-04-23T14:58:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.592+0000 7f8bd3e00640 1 -- 172.21.15.83:0/2365167563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd4104260 msgr2=0x7f8bd41a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.592+0000 7f8bd3e00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd4104260 0x7f8bd41a0690 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4004770 tx=0x7f8bc4003c10 comp rx=0 tx=0).stop 2024-04-23T14:58:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.592+0000 7f8bd3e00640 1 -- 172.21.15.83:0/2365167563 shutdown_connections 2024-04-23T14:58:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.592+0000 7f8bd3e00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bd419a750 0x7f8bd419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.592+0000 7f8bd3e00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8bd4109330 0x7f8bd41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.592+0000 7f8bd3e00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f8bb80a3d20 0x7f8bb80a61e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.592+0000 7f8bd3e00640 1 --2- 172.21.15.83:0/2365167563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bd4104260 0x7f8bd41a0690 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.592+0000 7f8bd3e00640 1 -- 172.21.15.83:0/2365167563 >> 172.21.15.83:0/2365167563 conn(0x7f8bd40ffaa0 msgr2=0x7f8bd4100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.592+0000 7f8bd3e00640 1 -- 172.21.15.83:0/2365167563 shutdown_connections 2024-04-23T14:58:35.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.593+0000 7f8bd3e00640 1 -- 172.21.15.83:0/2365167563 wait complete. 2024-04-23T14:58:35.610 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:58:35.806 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.806+0000 7fcf1ca00640 1 -- 172.21.15.83:0/3032503699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf18108fd0 msgr2=0x7fcf18115490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.806+0000 7fcf1ca00640 1 --2- 172.21.15.83:0/3032503699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf18108fd0 0x7fcf18115490 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009930 tx=0x7fcf1002a440 comp rx=0 tx=0).stop 2024-04-23T14:58:35.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.806+0000 7fcf1ca00640 1 -- 172.21.15.83:0/3032503699 shutdown_connections 2024-04-23T14:58:35.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.806+0000 7fcf1ca00640 1 --2- 172.21.15.83:0/3032503699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf18108fd0 0x7fcf18115490 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.806+0000 7fcf1ca00640 1 --2- 172.21.15.83:0/3032503699 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18108610 0x7fcf18108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.806+0000 7fcf1ca00640 1 --2- 172.21.15.83:0/3032503699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf1810e660 0x7fcf1810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.806+0000 7fcf1ca00640 1 -- 172.21.15.83:0/3032503699 >> 172.21.15.83:0/3032503699 conn(0x7fcf18076540 msgr2=0x7fcf18078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.806+0000 7fcf1ca00640 1 -- 172.21.15.83:0/3032503699 shutdown_connections 2024-04-23T14:58:35.807 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.806+0000 7fcf1ca00640 1 -- 172.21.15.83:0/3032503699 wait complete. 2024-04-23T14:58:35.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.807+0000 7fcf1ca00640 1 Processor -- start 2024-04-23T14:58:35.809 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.807+0000 7fcf1ca00640 1 -- start start 2024-04-23T14:58:35.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.807+0000 7fcf1ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf18108610 0x7fcf1819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.807+0000 7fcf1ca00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18108fd0 0x7fcf1819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf1ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1810e660 0x7fcf181a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf1ca00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181189f0 con 0x7fcf1810e660 2024-04-23T14:58:35.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf1ca00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcf18118870 con 0x7fcf18108fd0 2024-04-23T14:58:35.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf1ca00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf18118b70 con 0x7fcf18108610 2024-04-23T14:58:35.810 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf16800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1810e660 0x7fcf181a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf15e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf18108610 0x7fcf1819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf16800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1810e660 0x7fcf181a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52676/0 (socket says 172.21.15.83:52676) 2024-04-23T14:58:35.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf16800640 1 -- 172.21.15.83:0/539636455 learned_addr learned my addr 172.21.15.83:0/539636455 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:35.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf15e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf18108610 0x7fcf1819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41128/0 (socket says 172.21.15.83:41128) 2024-04-23T14:58:35.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf0d400640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18108fd0 0x7fcf1819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf16800640 1 -- 172.21.15.83:0/539636455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf18108610 msgr2=0x7fcf1819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf16800640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf18108610 0x7fcf1819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf16800640 1 -- 172.21.15.83:0/539636455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18108fd0 msgr2=0x7fcf1819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.811 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf16800640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18108fd0 0x7fcf1819f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf16800640 1 -- 172.21.15.83:0/539636455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf181a3e00 con 0x7fcf1810e660 2024-04-23T14:58:35.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.808+0000 7fcf0d400640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18108fd0 0x7fcf1819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:35.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.809+0000 7fcf16800640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1810e660 0x7fcf181a36d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcf10004770 tx=0x7fcf10003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.809+0000 7fcf0f400640 1 -- 172.21.15.83:0/539636455 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1003f070 con 0x7fcf1810e660 2024-04-23T14:58:35.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.809+0000 7fcf1ca00640 1 -- 172.21.15.83:0/539636455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf181a4090 con 0x7fcf1810e660 2024-04-23T14:58:35.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.809+0000 7fcf0f400640 1 -- 172.21.15.83:0/539636455 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf10031c90 con 0x7fcf1810e660 2024-04-23T14:58:35.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.809+0000 7fcf0f400640 1 -- 172.21.15.83:0/539636455 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf10035030 con 0x7fcf1810e660 2024-04-23T14:58:35.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.809+0000 7fcf1ca00640 1 -- 172.21.15.83:0/539636455 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf181ab980 con 0x7fcf1810e660 2024-04-23T14:58:35.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.812+0000 7fcf0f400640 1 -- 172.21.15.83:0/539636455 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcf10042070 con 0x7fcf1810e660 2024-04-23T14:58:35.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.812+0000 7fcf1f881740 1 Processor -- start 2024-04-23T14:58:35.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.813+0000 7fcf1f881740 1 -- start start 2024-04-23T14:58:35.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.813+0000 7fcf1f881740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e72a950 0x55ed3e72ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.813+0000 7fcf1f881740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e735400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.813+0000 7fcf1f881740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 0x55ed3e737d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.813+0000 7fcf1f881740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ed3e6901d0 con 0x55ed3e735940 2024-04-23T14:58:35.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.813+0000 7fcf1f881740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ed3e636450 con 0x55ed3e72a950 2024-04-23T14:58:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.813+0000 7fcf1f881740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ed3e68c190 con 0x55ed3e72b270 2024-04-23T14:58:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.813+0000 7fcf0d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 0x55ed3e737d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf15e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e735400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf0d400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 0x55ed3e737d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52684/0 (socket says 172.21.15.83:52684) 2024-04-23T14:58:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf0d400640 1 -- 172.21.15.83:0/208872712 learned_addr learned my addr 172.21.15.83:0/208872712 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf0d400640 1 --2- 172.21.15.83:0/208872712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e72a950 0x55ed3e72ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf15e00640 1 -- 172.21.15.83:0/208872712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e72a950 msgr2=0x55ed3e72ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf15e00640 1 --2- 172.21.15.83:0/208872712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e72a950 0x55ed3e72ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf15e00640 1 -- 172.21.15.83:0/208872712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 msgr2=0x55ed3e737d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf15e00640 1 --2- 172.21.15.83:0/208872712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 0x55ed3e737d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf15e00640 1 -- 172.21.15.83:0/208872712 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed3e6f3310 con 0x55ed3e72b270 2024-04-23T14:58:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf0d400640 1 --2- 172.21.15.83:0/208872712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 0x55ed3e737d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf0f400640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcefc0a3da0 0x7fcefc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf15e00640 1 --2- 172.21.15.83:0/208872712 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e735400 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcf08005fd0 tx=0x7fcf08010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd8c5da25e80ac37 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.814+0000 7fcf0f400640 1 -- 172.21.15.83:0/539636455 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fcf100f0e20 con 0x7fcf1810e660 2024-04-23T14:58:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.815+0000 7fcf0d400640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcefc0a3da0 0x7fcefc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.815+0000 7fcf07000640 1 -- 172.21.15.83:0/208872712 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08016070 con 0x55ed3e72b270 2024-04-23T14:58:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.815+0000 7fcf07000640 1 -- 172.21.15.83:0/208872712 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf08012bf0 con 0x55ed3e72b270 2024-04-23T14:58:35.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.815+0000 7fcf07000640 1 -- 172.21.15.83:0/208872712 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08011ca0 con 0x55ed3e72b270 2024-04-23T14:58:35.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.815+0000 7fcf1f881740 1 -- 172.21.15.83:0/208872712 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 msgr2=0x55ed3e735400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.815+0000 7fcf0d400640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcefc0a3da0 0x7fcefc0a6260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55ed3e727e00 tx=0x7fcf00009770 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.815+0000 7fcf1f881740 1 --2- 172.21.15.83:0/208872712 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e735400 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcf08005fd0 tx=0x7fcf08010800 comp rx=0 tx=0).stop 2024-04-23T14:58:35.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.815+0000 7fcf1f881740 1 -- 172.21.15.83:0/208872712 shutdown_connections 2024-04-23T14:58:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.815+0000 7fcf1f881740 1 --2- 172.21.15.83:0/208872712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 0x55ed3e737d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.816+0000 7fcf1f881740 1 --2- 172.21.15.83:0/208872712 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e735400 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.816+0000 7fcf1f881740 1 --2- 172.21.15.83:0/208872712 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e72a950 0x55ed3e72ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.816+0000 7fcf1f881740 1 -- 172.21.15.83:0/208872712 >> 172.21.15.83:0/208872712 conn(0x55ed3e726c70 msgr2=0x55ed3e727050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.816+0000 7fcf1f881740 1 -- 172.21.15.83:0/208872712 shutdown_connections 2024-04-23T14:58:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.816+0000 7fcf1f881740 1 -- 172.21.15.83:0/208872712 wait complete. 2024-04-23T14:58:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.816+0000 7fcf1f881740 1 -- start start 2024-04-23T14:58:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.816+0000 7fcf1f881740 1 Processor -- start 2024-04-23T14:58:35.823 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf1f881740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 0x55ed3e735390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf1f881740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e732680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf1f881740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e735940 0x55ed3e732dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf1f881740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ed3e7387d0 con 0x55ed3e72a950 2024-04-23T14:58:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf1f881740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55ed3e649810 con 0x55ed3e735940 2024-04-23T14:58:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf1f881740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55ed3e738950 con 0x55ed3e72b270 2024-04-23T14:58:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf1f881740 10 client.0 _ll_register_callbacks cb 0x55ed3e617140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf15e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 0x55ed3e735390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf0d400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e732680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf15e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 0x55ed3e735390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52690/0 (socket says 172.21.15.83:52690) 2024-04-23T14:58:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf15e00640 1 -- 172.21.15.83:0/724345682 learned_addr learned my addr 172.21.15.83:0/724345682 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf16800640 1 --2- 172.21.15.83:0/724345682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e735940 0x55ed3e732dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf15e00640 1 -- 172.21.15.83:0/724345682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 msgr2=0x55ed3e732680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf15e00640 1 --2- 172.21.15.83:0/724345682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e732680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf15e00640 1 -- 172.21.15.83:0/724345682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e735940 msgr2=0x55ed3e732dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.817+0000 7fcf15e00640 1 --2- 172.21.15.83:0/724345682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e735940 0x55ed3e732dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.818+0000 7fcf15e00640 1 -- 172.21.15.83:0/724345682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed3e65a3f0 con 0x55ed3e72a950 2024-04-23T14:58:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.818+0000 7fcf15e00640 1 --2- 172.21.15.83:0/724345682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 0x55ed3e735390 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcf08011010 tx=0x7fcf08011450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f912bbca9592023 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.818+0000 7fcefb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:35.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.818+0000 7fcefb400640 1 -- 172.21.15.83:0/724345682 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08016040 con 0x55ed3e72a950 2024-04-23T14:58:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.818+0000 7fcefb400640 1 -- 172.21.15.83:0/724345682 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fcf08011be0 con 0x55ed3e72a950 2024-04-23T14:58:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.818+0000 7fcf1f881740 1 -- 172.21.15.83:0/724345682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf08012bf0 con 0x55ed3e72a950 2024-04-23T14:58:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.818+0000 7fcefb400640 1 -- 172.21.15.83:0/724345682 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf08031050 con 0x55ed3e72a950 2024-04-23T14:58:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.818+0000 7fcf1f881740 10 client.17016 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.818+0000 7fcf1f881740 1 -- 172.21.15.83:0/724345682 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed3e3a23f0 con 0x55ed3e72a950 2024-04-23T14:58:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.819+0000 7fcefb400640 1 -- 172.21.15.83:0/724345682 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf08012bf0 con 0x55ed3e72a950 2024-04-23T14:58:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.819+0000 7fcefb400640 1 -- 172.21.15.83:0/724345682 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fcf0802c030 con 0x55ed3e72a950 2024-04-23T14:58:35.826 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.819+0000 7fcf1f881740 10 client.17016 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.819+0000 7fcf1f881740 10 client.17016 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:58:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.819+0000 7fcf1f881740 20 client.17016 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.819+0000 7fcf1f881740 1 --2- 172.21.15.83:0/724345682 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ed3e745b10 0x55ed3e747f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.819+0000 7fcf1f881740 4 client.17016 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:58:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.819+0000 7fcf1f881740 1 -- 172.21.15.83:0/724345682 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed3e69dbf0 con 0x55ed3e745b10 2024-04-23T14:58:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.819+0000 7fcf0d400640 1 --2- 172.21.15.83:0/724345682 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ed3e745b10 0x55ed3e747f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.820+0000 7fcf0d400640 1 --2- 172.21.15.83:0/724345682 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ed3e745b10 0x55ed3e747f00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67945e2edd16b76e server_cookie=6eda6d1bd0fa980c in_seq=0 out_seq=0 2024-04-23T14:58:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.820+0000 7fcefb400640 10 client.17016 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:58:35.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.821+0000 7fcefb400640 1 -- 172.21.15.83:0/724345682 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed3e69dbf0 con 0x55ed3e745b10 2024-04-23T14:58:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.821+0000 7fcefb400640 10 client.17016 handle_command_reply: tid=0 2024-04-23T14:58:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.821+0000 7fcf1f881740 1 client.17016 shutdown 2024-04-23T14:58:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.821+0000 7fcf1f881740 1 -- 172.21.15.83:0/724345682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 msgr2=0x55ed3e735390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.821+0000 7fcf1f881740 1 --2- 172.21.15.83:0/724345682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 0x55ed3e735390 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcf08011010 tx=0x7fcf08011450 comp rx=0 tx=0).stop 2024-04-23T14:58:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.821+0000 7fcf1f881740 1 -- 172.21.15.83:0/724345682 shutdown_connections 2024-04-23T14:58:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.821+0000 7fcf1f881740 1 --2- 172.21.15.83:0/724345682 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ed3e745b10 0x55ed3e747f00 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.821+0000 7fcf1f881740 1 --2- 172.21.15.83:0/724345682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e735940 0x55ed3e732dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.821+0000 7fcefb400640 0 client.17016 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:58:35.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.821+0000 7fcf1f881740 1 --2- 172.21.15.83:0/724345682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e732680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.821+0000 7fcf1f881740 1 --2- 172.21.15.83:0/724345682 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 0x55ed3e735390 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.821+0000 7fcf1f881740 1 -- 172.21.15.83:0/724345682 >> 172.21.15.83:0/724345682 conn(0x55ed3e726c70 msgr2=0x55ed3e72ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.822+0000 7fcf1f881740 1 -- 172.21.15.83:0/724345682 shutdown_connections 2024-04-23T14:58:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.822+0000 7fcf1f881740 1 -- 172.21.15.83:0/724345682 wait complete. 2024-04-23T14:58:35.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.822+0000 7fcf1f881740 20 client.17016 trim_cache size 0 max 16384 2024-04-23T14:58:35.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.836+0000 7fcf1f881740 1 Processor -- start 2024-04-23T14:58:35.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.836+0000 7fcf1f881740 1 -- start start 2024-04-23T14:58:35.836 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.836+0000 7fcf1f881740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e72a950 0x55ed3e7416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.836+0000 7fcf1f881740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e734ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf1f881740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 0x55ed3e735400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf1f881740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181b6c20 con 0x55ed3e735940 2024-04-23T14:58:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf1f881740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181b44f0 con 0x55ed3e72a950 2024-04-23T14:58:35.837 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf1f881740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181b2360 con 0x55ed3e72b270 2024-04-23T14:58:35.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf16800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 0x55ed3e735400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf15e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e72a950 0x55ed3e7416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf16800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 0x55ed3e735400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52694/0 (socket says 172.21.15.83:52694) 2024-04-23T14:58:35.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf16800640 1 -- 172.21.15.83:0/1348068229 learned_addr learned my addr 172.21.15.83:0/1348068229 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:35.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf0d400640 1 --2- 172.21.15.83:0/1348068229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e734ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf16800640 1 -- 172.21.15.83:0/1348068229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 msgr2=0x55ed3e734ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf16800640 1 --2- 172.21.15.83:0/1348068229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e734ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf16800640 1 -- 172.21.15.83:0/1348068229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e72a950 msgr2=0x55ed3e7416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf16800640 1 --2- 172.21.15.83:0/1348068229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e72a950 0x55ed3e7416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.837+0000 7fcf16800640 1 -- 172.21.15.83:0/1348068229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed3e46c7d0 con 0x55ed3e735940 2024-04-23T14:58:35.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.838+0000 7fcf16800640 1 --2- 172.21.15.83:0/1348068229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 0x55ed3e735400 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcf100bcb00 tx=0x7fcf1002a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3ad9a0526076dd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.838+0000 7fcf06600640 1 -- 172.21.15.83:0/1348068229 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf10049040 con 0x55ed3e735940 2024-04-23T14:58:35.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.838+0000 7fcf06600640 1 -- 172.21.15.83:0/1348068229 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf100c59c0 con 0x55ed3e735940 2024-04-23T14:58:35.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.838+0000 7fcf06600640 1 -- 172.21.15.83:0/1348068229 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1003b460 con 0x55ed3e735940 2024-04-23T14:58:35.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.838+0000 7fcf1f881740 1 -- 172.21.15.83:0/1348068229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 msgr2=0x55ed3e735400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.838+0000 7fcf1f881740 1 --2- 172.21.15.83:0/1348068229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 0x55ed3e735400 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcf100bcb00 tx=0x7fcf1002a440 comp rx=0 tx=0).stop 2024-04-23T14:58:35.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.838+0000 7fcf1f881740 1 -- 172.21.15.83:0/1348068229 shutdown_connections 2024-04-23T14:58:35.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.838+0000 7fcf1f881740 1 --2- 172.21.15.83:0/1348068229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e735940 0x55ed3e735400 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.838+0000 7fcf1f881740 1 --2- 172.21.15.83:0/1348068229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e734ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.838+0000 7fcf1f881740 1 --2- 172.21.15.83:0/1348068229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e72a950 0x55ed3e7416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.838+0000 7fcf1f881740 1 -- 172.21.15.83:0/1348068229 >> 172.21.15.83:0/1348068229 conn(0x55ed3e726c70 msgr2=0x55ed3e727050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.839+0000 7fcf1f881740 1 -- 172.21.15.83:0/1348068229 shutdown_connections 2024-04-23T14:58:35.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.839+0000 7fcf1f881740 1 -- 172.21.15.83:0/1348068229 wait complete. 2024-04-23T14:58:35.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.839+0000 7fcf1f881740 1 -- start start 2024-04-23T14:58:35.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.839+0000 7fcf1f881740 1 Processor -- start 2024-04-23T14:58:35.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf1f881740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 0x55ed3e73dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf1f881740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e73e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf1f881740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e735940 0x55ed3e72f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf1f881740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181b29b0 con 0x55ed3e72a950 2024-04-23T14:58:35.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf1f881740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181b3e20 con 0x55ed3e735940 2024-04-23T14:58:35.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf1f881740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcf181b3080 con 0x55ed3e72b270 2024-04-23T14:58:35.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf1f881740 10 client.0 _ll_register_callbacks cb 0x7fcf00009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:35.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf15e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 0x55ed3e73dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf15e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 0x55ed3e73dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52696/0 (socket says 172.21.15.83:52696) 2024-04-23T14:58:35.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf15e00640 1 -- 172.21.15.83:0/387305137 learned_addr learned my addr 172.21.15.83:0/387305137 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:35.846 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf0d400640 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e73e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf16800640 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e735940 0x55ed3e72f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf15e00640 1 -- 172.21.15.83:0/387305137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 msgr2=0x55ed3e73e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf15e00640 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e73e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf15e00640 1 -- 172.21.15.83:0/387305137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e735940 msgr2=0x55ed3e72f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:35.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf15e00640 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e735940 0x55ed3e72f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf15e00640 1 -- 172.21.15.83:0/387305137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed3e730100 con 0x55ed3e72a950 2024-04-23T14:58:35.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf0d400640 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e73e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:58:35.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.840+0000 7fcf15e00640 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 0x55ed3e73dc00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcf08000f80 tx=0x7fcf08011270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4fb8ca049e9e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:35.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.841+0000 7fcefb400640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:35.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.841+0000 7fcefb400640 1 -- 172.21.15.83:0/387305137 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08039020 con 0x55ed3e72a950 2024-04-23T14:58:35.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.841+0000 7fcefb400640 1 -- 172.21.15.83:0/387305137 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fcf08005200 con 0x55ed3e72a950 2024-04-23T14:58:35.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.841+0000 7fcf1f881740 1 -- 172.21.15.83:0/387305137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf100c59c0 con 0x55ed3e72a950 2024-04-23T14:58:35.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.841+0000 7fcefb400640 1 -- 172.21.15.83:0/387305137 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcf08044050 con 0x55ed3e72a950 2024-04-23T14:58:35.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.841+0000 7fcf1f881740 10 client.17022 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:35.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.841+0000 7fcf1f881740 1 -- 172.21.15.83:0/387305137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed3e7367f0 con 0x55ed3e72a950 2024-04-23T14:58:35.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.841+0000 7fcefb400640 1 -- 172.21.15.83:0/387305137 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf100c59c0 con 0x55ed3e72a950 2024-04-23T14:58:35.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.841+0000 7fcefb400640 1 -- 172.21.15.83:0/387305137 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fcf0802c040 con 0x55ed3e72a950 2024-04-23T14:58:35.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.842+0000 7fcf1f881740 10 client.17022 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:35.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.842+0000 7fcf1f881740 10 client.17022 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:58:35.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.842+0000 7fcf1f881740 20 client.17022 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:35.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.842+0000 7fcf1f881740 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ed3e7abfb0 0x55ed3e7ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:35.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.842+0000 7fcf1f881740 4 client.17022 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:58:35.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.842+0000 7fcf1f881740 1 -- 172.21.15.83:0/387305137 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed3e697a60 con 0x55ed3e7abfb0 2024-04-23T14:58:35.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.842+0000 7fcf0d400640 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ed3e7abfb0 0x55ed3e7ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:35.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.842+0000 7fcefb400640 10 client.17022 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:58:35.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.842+0000 7fcf0d400640 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ed3e7abfb0 0x55ed3e7ae3d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1cfcc4a9b9ee6cc server_cookie=3d0760a9ea99b10a in_seq=0 out_seq=0 2024-04-23T14:58:35.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcefb400640 1 -- 172.21.15.83:0/387305137 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed3e697a60 con 0x55ed3e7abfb0 2024-04-23T14:58:35.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcefb400640 10 client.17022 handle_command_reply: tid=0 2024-04-23T14:58:35.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcf1f881740 1 client.17022 shutdown 2024-04-23T14:58:35.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcf1f881740 1 -- 172.21.15.83:0/387305137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 msgr2=0x55ed3e73dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcf1f881740 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 0x55ed3e73dc00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcf08000f80 tx=0x7fcf08011270 comp rx=0 tx=0).stop 2024-04-23T14:58:35.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcf1f881740 1 -- 172.21.15.83:0/387305137 shutdown_connections 2024-04-23T14:58:35.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcf1f881740 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x55ed3e7abfb0 0x55ed3e7ae3d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcf1f881740 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ed3e735940 0x55ed3e72f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcf1f881740 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ed3e72b270 0x55ed3e73e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcefb400640 0 client.17022 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:58:35.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcf1f881740 1 --2- 172.21.15.83:0/387305137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ed3e72a950 0x55ed3e73dc00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcf1f881740 1 -- 172.21.15.83:0/387305137 >> 172.21.15.83:0/387305137 conn(0x55ed3e726c70 msgr2=0x55ed3e730eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcf1f881740 1 -- 172.21.15.83:0/387305137 shutdown_connections 2024-04-23T14:58:35.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcf1f881740 1 -- 172.21.15.83:0/387305137 wait complete. 2024-04-23T14:58:35.851 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.843+0000 7fcf1f881740 20 client.17022 trim_cache size 0 max 16384 2024-04-23T14:58:35.852 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:58:35.852 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.844+0000 7fcf1ca00640 1 -- 172.21.15.83:0/539636455 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcefc0a3da0 msgr2=0x7fcefc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.844+0000 7fcf1ca00640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcefc0a3da0 0x7fcefc0a6260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55ed3e727e00 tx=0x7fcf00009770 comp rx=0 tx=0).stop 2024-04-23T14:58:35.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.844+0000 7fcf1ca00640 1 -- 172.21.15.83:0/539636455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1810e660 msgr2=0x7fcf181a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:35.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.844+0000 7fcf1ca00640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1810e660 0x7fcf181a36d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcf10004770 tx=0x7fcf10003b90 comp rx=0 tx=0).stop 2024-04-23T14:58:35.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.844+0000 7fcf1ca00640 1 -- 172.21.15.83:0/539636455 shutdown_connections 2024-04-23T14:58:35.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.844+0000 7fcf1ca00640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcefc0a3da0 0x7fcefc0a6260 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.844+0000 7fcf1ca00640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf1810e660 0x7fcf181a36d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.844+0000 7fcf1ca00640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18108fd0 0x7fcf1819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.844+0000 7fcf1ca00640 1 --2- 172.21.15.83:0/539636455 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf18108610 0x7fcf1819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:35.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.844+0000 7fcf1ca00640 1 -- 172.21.15.83:0/539636455 >> 172.21.15.83:0/539636455 conn(0x7fcf18076540 msgr2=0x7fcf18077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:35.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.844+0000 7fcf1ca00640 1 -- 172.21.15.83:0/539636455 shutdown_connections 2024-04-23T14:58:35.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:35.844+0000 7fcf1ca00640 1 -- 172.21.15.83:0/539636455 wait complete. 2024-04-23T14:58:35.861 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:58:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.064+0000 7fe52ec00640 1 -- 172.21.15.83:0/4167656296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5281044b0 msgr2=0x7fe528104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.064+0000 7fe52ec00640 1 --2- 172.21.15.83:0/4167656296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5281044b0 0x7fe528104890 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe518009860 tx=0x7fe51802a340 comp rx=0 tx=0).stop 2024-04-23T14:58:36.064 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.064+0000 7fe52ec00640 1 -- 172.21.15.83:0/4167656296 shutdown_connections 2024-04-23T14:58:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.064+0000 7fe52ec00640 1 --2- 172.21.15.83:0/4167656296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe528109580 0x7fe528111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.064+0000 7fe52ec00640 1 --2- 172.21.15.83:0/4167656296 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe528104e60 0x7fe528108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.064+0000 7fe52ec00640 1 --2- 172.21.15.83:0/4167656296 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5281044b0 0x7fe528104890 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.064+0000 7fe52ec00640 1 -- 172.21.15.83:0/4167656296 >> 172.21.15.83:0/4167656296 conn(0x7fe5280ffd90 msgr2=0x7fe5281021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.064+0000 7fe52ec00640 1 -- 172.21.15.83:0/4167656296 shutdown_connections 2024-04-23T14:58:36.065 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.064+0000 7fe52ec00640 1 -- 172.21.15.83:0/4167656296 wait complete. 2024-04-23T14:58:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.065+0000 7fe52ec00640 1 Processor -- start 2024-04-23T14:58:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.065+0000 7fe52ec00640 1 -- start start 2024-04-23T14:58:36.067 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.065+0000 7fe52ec00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5281044b0 0x7fe5281a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.065+0000 7fe52ec00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe528109580 0x7fe5281a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.065+0000 7fe52ec00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe52819aa40 0x7fe52819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.065+0000 7fe52ec00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe528114610 con 0x7fe52819aa40 2024-04-23T14:58:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.065+0000 7fe52ec00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe528114490 con 0x7fe528109580 2024-04-23T14:58:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.065+0000 7fe52ec00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe528114790 con 0x7fe5281044b0 2024-04-23T14:58:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.066+0000 7fe527e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5281044b0 0x7fe5281a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.066+0000 7fe527e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5281044b0 0x7fe5281a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41174/0 (socket says 172.21.15.83:41174) 2024-04-23T14:58:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.066+0000 7fe527e00640 1 -- 172.21.15.83:0/2727250346 learned_addr learned my addr 172.21.15.83:0/2727250346 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:36.068 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.066+0000 7fe52cc00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe52819aa40 0x7fe52819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.066+0000 7fe527400640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe528109580 0x7fe5281a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.066+0000 7fe527e00640 1 -- 172.21.15.83:0/2727250346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe528109580 msgr2=0x7fe5281a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.066+0000 7fe527e00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe528109580 0x7fe5281a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.066+0000 7fe527e00640 1 -- 172.21.15.83:0/2727250346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe52819aa40 msgr2=0x7fe52819aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.066+0000 7fe527e00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe52819aa40 0x7fe52819aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.066+0000 7fe527e00640 1 -- 172.21.15.83:0/2727250346 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52819b560 con 0x7fe5281044b0 2024-04-23T14:58:36.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.067+0000 7fe527e00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5281044b0 0x7fe5281a0840 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe518035040 tx=0x7fe518033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:36.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.067+0000 7fe524c00640 1 -- 172.21.15.83:0/2727250346 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518037040 con 0x7fe5281044b0 2024-04-23T14:58:36.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.067+0000 7fe52ec00640 1 -- 172.21.15.83:0/2727250346 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52819b7f0 con 0x7fe5281044b0 2024-04-23T14:58:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.067+0000 7fe52ec00640 1 -- 172.21.15.83:0/2727250346 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5281a79f0 con 0x7fe5281044b0 2024-04-23T14:58:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.067+0000 7fe524c00640 1 -- 172.21.15.83:0/2727250346 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe518040070 con 0x7fe5281044b0 2024-04-23T14:58:36.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.067+0000 7fe524c00640 1 -- 172.21.15.83:0/2727250346 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518003900 con 0x7fe5281044b0 2024-04-23T14:58:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.070+0000 7fe531a00740 1 Processor -- start 2024-04-23T14:58:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.070+0000 7fe524c00640 1 -- 172.21.15.83:0/2727250346 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe518030550 con 0x7fe5281044b0 2024-04-23T14:58:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.070+0000 7fe531a00740 1 -- start start 2024-04-23T14:58:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe531a00740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187755a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe531a00740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187756050 0x561187768c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.072 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe531a00740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187769150 0x56118776b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe531a00740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5611876bb1d0 con 0x561187756050 2024-04-23T14:58:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe531a00740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561187661450 con 0x561187769150 2024-04-23T14:58:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe531a00740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x5611876b7190 con 0x5611877556a0 2024-04-23T14:58:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe52cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187755a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe527e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187756050 0x561187768c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe527400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187769150 0x56118776b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe52cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187755a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41190/0 (socket says 172.21.15.83:41190) 2024-04-23T14:58:36.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe52cc00640 1 -- 172.21.15.83:0/1180660369 learned_addr learned my addr 172.21.15.83:0/1180660369 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe527400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187769150 0x56118776b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:47432/0 (socket says 172.21.15.83:47432) 2024-04-23T14:58:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe52cc00640 1 -- 172.21.15.83:0/1180660369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187769150 msgr2=0x56118776b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe52cc00640 1 --2- 172.21.15.83:0/1180660369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187769150 0x56118776b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe52cc00640 1 -- 172.21.15.83:0/1180660369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187756050 msgr2=0x561187768c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe52cc00640 1 --2- 172.21.15.83:0/1180660369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187756050 0x561187768c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.071+0000 7fe52cc00640 1 -- 172.21.15.83:0/1180660369 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118771e310 con 0x5611877556a0 2024-04-23T14:58:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.072+0000 7fe52cc00640 1 --2- 172.21.15.83:0/1180660369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187755a80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe520009f50 tx=0x7fe5200118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772fe5e2767d1d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.072+0000 7fe51d200640 1 -- 172.21.15.83:0/1180660369 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520004d20 con 0x5611877556a0 2024-04-23T14:58:36.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.072+0000 7fe51d200640 1 -- 172.21.15.83:0/1180660369 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe520004050 con 0x5611877556a0 2024-04-23T14:58:36.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.072+0000 7fe51d200640 1 -- 172.21.15.83:0/1180660369 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520005240 con 0x5611877556a0 2024-04-23T14:58:36.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.072+0000 7fe524c00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe50c0a3d20 0x7fe50c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.072+0000 7fe527e00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe50c0a3d20 0x7fe50c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.072+0000 7fe524c00640 1 -- 172.21.15.83:0/2727250346 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fe5180f0460 con 0x7fe5281044b0 2024-04-23T14:58:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.073+0000 7fe531a00740 1 -- 172.21.15.83:0/1180660369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 msgr2=0x561187755a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.073+0000 7fe531a00740 1 --2- 172.21.15.83:0/1180660369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187755a80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe520009f50 tx=0x7fe5200118d0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.073+0000 7fe527e00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe50c0a3d20 0x7fe50c0a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe5180b8990 tx=0x7fe5180f3f10 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.073+0000 7fe531a00740 1 -- 172.21.15.83:0/1180660369 shutdown_connections 2024-04-23T14:58:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.073+0000 7fe531a00740 1 --2- 172.21.15.83:0/1180660369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187769150 0x56118776b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.073+0000 7fe531a00740 1 --2- 172.21.15.83:0/1180660369 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187756050 0x561187768c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.073+0000 7fe531a00740 1 --2- 172.21.15.83:0/1180660369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187755a80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.073+0000 7fe531a00740 1 -- 172.21.15.83:0/1180660369 >> 172.21.15.83:0/1180660369 conn(0x561187751c70 msgr2=0x561187752050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.073+0000 7fe531a00740 1 -- 172.21.15.83:0/1180660369 shutdown_connections 2024-04-23T14:58:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.073+0000 7fe531a00740 1 -- 172.21.15.83:0/1180660369 wait complete. 2024-04-23T14:58:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.073+0000 7fe531a00740 1 -- start start 2024-04-23T14:58:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe531a00740 1 Processor -- start 2024-04-23T14:58:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe531a00740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611877556a0 0x561187768b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe531a00740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561187756050 0x561187765e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe531a00740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187769150 0x561187766420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe531a00740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56118776bfb0 con 0x561187769150 2024-04-23T14:58:36.081 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe531a00740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x561187674810 con 0x5611877556a0 2024-04-23T14:58:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe531a00740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x56118776c130 con 0x561187756050 2024-04-23T14:58:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe531a00740 10 client.0 _ll_register_callbacks cb 0x561187642140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe52cc00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187769150 0x561187766420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe527400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561187756050 0x561187765e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe527400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561187756050 0x561187765e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41200/0 (socket says 172.21.15.83:41200) 2024-04-23T14:58:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe527400640 1 -- 172.21.15.83:0/21348518 learned_addr learned my addr 172.21.15.83:0/21348518 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.074+0000 7fe52cc00640 1 --2- 172.21.15.83:0/21348518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611877556a0 0x561187768b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.075+0000 7fe527400640 1 -- 172.21.15.83:0/21348518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611877556a0 msgr2=0x561187768b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.075+0000 7fe527400640 1 --2- 172.21.15.83:0/21348518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611877556a0 0x561187768b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.075+0000 7fe527400640 1 -- 172.21.15.83:0/21348518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187769150 msgr2=0x561187766420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.075+0000 7fe527400640 1 --2- 172.21.15.83:0/21348518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187769150 0x561187766420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.075+0000 7fe527400640 1 -- 172.21.15.83:0/21348518 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611876853f0 con 0x561187756050 2024-04-23T14:58:36.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.075+0000 7fe527400640 1 --2- 172.21.15.83:0/21348518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561187756050 0x561187765e10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe52806b750 tx=0x7fe514011b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fad574afbc6d9e4e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:36.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.075+0000 7fe511600640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:58:36.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.075+0000 7fe511600640 1 -- 172.21.15.83:0/21348518 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51401e060 con 0x561187756050 2024-04-23T14:58:36.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.075+0000 7fe511600640 1 -- 172.21.15.83:0/21348518 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fe51400fbc0 con 0x561187756050 2024-04-23T14:58:36.083 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.075+0000 7fe531a00740 1 -- 172.21.15.83:0/21348518 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe520004050 con 0x561187756050 2024-04-23T14:58:36.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.076+0000 7fe511600640 1 -- 172.21.15.83:0/21348518 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe51400fe00 con 0x561187756050 2024-04-23T14:58:36.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.076+0000 7fe531a00740 10 client.27211 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:36.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.076+0000 7fe531a00740 1 -- 172.21.15.83:0/21348518 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5611873cd3f0 con 0x561187756050 2024-04-23T14:58:36.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.076+0000 7fe511600640 1 -- 172.21.15.83:0/21348518 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe520004050 con 0x561187756050 2024-04-23T14:58:36.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.076+0000 7fe511600640 1 -- 172.21.15.83:0/21348518 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fe514019070 con 0x561187756050 2024-04-23T14:58:36.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.076+0000 7fe531a00740 10 client.27211 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:36.084 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.076+0000 7fe531a00740 10 client.27211 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:58:36.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.076+0000 7fe531a00740 20 client.27211 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:36.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.076+0000 7fe531a00740 1 --2- 172.21.15.83:0/21348518 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x561187779430 0x56118777b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.076+0000 7fe531a00740 4 client.27211 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:58:36.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.076+0000 7fe531a00740 1 -- 172.21.15.83:0/21348518 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611876c8bf0 con 0x561187779430 2024-04-23T14:58:36.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.076+0000 7fe52cc00640 1 --2- 172.21.15.83:0/21348518 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x561187779430 0x56118777b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.077+0000 7fe52cc00640 1 --2- 172.21.15.83:0/21348518 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x561187779430 0x56118777b870 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70a779254368f124 server_cookie=2dd23c75d3b68e69 in_seq=0 out_seq=0 2024-04-23T14:58:36.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.077+0000 7fe511600640 10 client.27211 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:58:36.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe511600640 1 -- 172.21.15.83:0/21348518 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611876c8bf0 con 0x561187779430 2024-04-23T14:58:36.085 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe511600640 10 client.27211 handle_command_reply: tid=0 2024-04-23T14:58:36.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe531a00740 1 client.27211 shutdown 2024-04-23T14:58:36.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe531a00740 1 -- 172.21.15.83:0/21348518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561187756050 msgr2=0x561187765e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe531a00740 1 --2- 172.21.15.83:0/21348518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561187756050 0x561187765e10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe52806b750 tx=0x7fe514011b30 comp rx=0 tx=0).stop 2024-04-23T14:58:36.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe531a00740 1 -- 172.21.15.83:0/21348518 shutdown_connections 2024-04-23T14:58:36.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe531a00740 1 --2- 172.21.15.83:0/21348518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187769150 0x561187766420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe531a00740 1 --2- 172.21.15.83:0/21348518 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x561187779430 0x56118777b870 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe531a00740 1 --2- 172.21.15.83:0/21348518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561187756050 0x561187765e10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.086 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe531a00740 1 --2- 172.21.15.83:0/21348518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5611877556a0 0x561187768b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe511600640 0 client.27211 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:58:36.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe531a00740 1 -- 172.21.15.83:0/21348518 >> 172.21.15.83:0/21348518 conn(0x561187751c70 msgr2=0x561187757a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:36.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe531a00740 1 -- 172.21.15.83:0/21348518 shutdown_connections 2024-04-23T14:58:36.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe531a00740 1 -- 172.21.15.83:0/21348518 wait complete. 2024-04-23T14:58:36.087 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.078+0000 7fe531a00740 20 client.27211 trim_cache size 0 max 16384 2024-04-23T14:58:36.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.092+0000 7fe531a00740 1 Processor -- start 2024-04-23T14:58:36.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.092+0000 7fe531a00740 1 -- start start 2024-04-23T14:58:36.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe531a00740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187774ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe531a00740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187756050 0x56118776b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe531a00740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187769150 0x561187768650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe531a00740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281ae4d0 con 0x561187769150 2024-04-23T14:58:36.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe531a00740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281abda0 con 0x561187756050 2024-04-23T14:58:36.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe531a00740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281a9c10 con 0x5611877556a0 2024-04-23T14:58:36.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe52cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187774ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe52cc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187774ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41204/0 (socket says 172.21.15.83:41204) 2024-04-23T14:58:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe52cc00640 1 -- 172.21.15.83:0/2766269335 learned_addr learned my addr 172.21.15.83:0/2766269335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe52cc00640 1 --2- 172.21.15.83:0/2766269335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187769150 0x561187768650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe527400640 1 --2- 172.21.15.83:0/2766269335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187756050 0x56118776b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe52cc00640 1 -- 172.21.15.83:0/2766269335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187756050 msgr2=0x56118776b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe52cc00640 1 --2- 172.21.15.83:0/2766269335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187756050 0x56118776b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe52cc00640 1 -- 172.21.15.83:0/2766269335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187769150 msgr2=0x561187768650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe52cc00640 1 --2- 172.21.15.83:0/2766269335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187769150 0x561187768650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.093+0000 7fe52cc00640 1 -- 172.21.15.83:0/2766269335 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561187755df0 con 0x5611877556a0 2024-04-23T14:58:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.094+0000 7fe52cc00640 1 --2- 172.21.15.83:0/2766269335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187774ee0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe520009ef0 tx=0x7fe520009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9003af3283b9db5d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.094+0000 7fe513e00640 1 -- 172.21.15.83:0/2766269335 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520004300 con 0x5611877556a0 2024-04-23T14:58:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.094+0000 7fe513e00640 1 -- 172.21.15.83:0/2766269335 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5200044a0 con 0x5611877556a0 2024-04-23T14:58:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.094+0000 7fe513e00640 1 -- 172.21.15.83:0/2766269335 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520027970 con 0x5611877556a0 2024-04-23T14:58:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.095+0000 7fe531a00740 1 -- 172.21.15.83:0/2766269335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 msgr2=0x561187774ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.095+0000 7fe531a00740 1 --2- 172.21.15.83:0/2766269335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187774ee0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe520009ef0 tx=0x7fe520009500 comp rx=0 tx=0).stop 2024-04-23T14:58:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.095+0000 7fe531a00740 1 -- 172.21.15.83:0/2766269335 shutdown_connections 2024-04-23T14:58:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.095+0000 7fe531a00740 1 --2- 172.21.15.83:0/2766269335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187769150 0x561187768650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.095+0000 7fe531a00740 1 --2- 172.21.15.83:0/2766269335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187756050 0x56118776b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.095+0000 7fe531a00740 1 --2- 172.21.15.83:0/2766269335 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187774ee0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.095+0000 7fe531a00740 1 -- 172.21.15.83:0/2766269335 >> 172.21.15.83:0/2766269335 conn(0x561187751c70 msgr2=0x5611877646e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:36.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.095+0000 7fe531a00740 1 -- 172.21.15.83:0/2766269335 shutdown_connections 2024-04-23T14:58:36.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.095+0000 7fe531a00740 1 -- 172.21.15.83:0/2766269335 wait complete. 2024-04-23T14:58:36.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.095+0000 7fe531a00740 1 -- start start 2024-04-23T14:58:36.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.095+0000 7fe531a00740 1 Processor -- start 2024-04-23T14:58:36.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe531a00740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187769d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe531a00740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187756050 0x56118776a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe531a00740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187769150 0x5611877703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe531a00740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281aa260 con 0x561187756050 2024-04-23T14:58:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe531a00740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281ab6d0 con 0x561187769150 2024-04-23T14:58:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe531a00740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe5281aa930 con 0x5611877556a0 2024-04-23T14:58:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe531a00740 10 client.0 _ll_register_callbacks cb 0x7fe514009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe527400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187756050 0x56118776a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe527400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187756050 0x56118776a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52744/0 (socket says 172.21.15.83:52744) 2024-04-23T14:58:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe527400640 1 -- 172.21.15.83:0/3901624479 learned_addr learned my addr 172.21.15.83:0/3901624479 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:36.101 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe52cc00640 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187769d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe52cc00640 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187769150 0x5611877703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe527400640 1 -- 172.21.15.83:0/3901624479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 msgr2=0x561187769d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe527400640 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187769d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe527400640 1 -- 172.21.15.83:0/3901624479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187769150 msgr2=0x5611877703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe527400640 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187769150 0x5611877703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe527400640 1 -- 172.21.15.83:0/3901624479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611874977d0 con 0x561187756050 2024-04-23T14:58:36.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.096+0000 7fe52cc00640 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187769150 0x5611877703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:58:36.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.097+0000 7fe527400640 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187756050 0x56118776a270 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe51401a560 tx=0x7fe51400f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f7563ea2a44104 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:36.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.097+0000 7fe511600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:36.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.097+0000 7fe511600640 1 -- 172.21.15.83:0/3901624479 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514005040 con 0x561187756050 2024-04-23T14:58:36.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.097+0000 7fe511600640 1 -- 172.21.15.83:0/3901624479 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fe5140051e0 con 0x561187756050 2024-04-23T14:58:36.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.097+0000 7fe531a00740 1 -- 172.21.15.83:0/3901624479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5200044a0 con 0x561187756050 2024-04-23T14:58:36.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.097+0000 7fe511600640 1 -- 172.21.15.83:0/3901624479 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe514038050 con 0x561187756050 2024-04-23T14:58:36.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.097+0000 7fe531a00740 10 client.17040 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:36.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.097+0000 7fe531a00740 1 -- 172.21.15.83:0/3901624479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561187771990 con 0x561187756050 2024-04-23T14:58:36.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.097+0000 7fe511600640 1 -- 172.21.15.83:0/3901624479 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5200044a0 con 0x561187756050 2024-04-23T14:58:36.103 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.097+0000 7fe511600640 1 -- 172.21.15.83:0/3901624479 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fe514033030 con 0x561187756050 2024-04-23T14:58:36.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.097+0000 7fe531a00740 10 client.17040 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:36.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.097+0000 7fe531a00740 10 client.17040 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:58:36.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.098+0000 7fe531a00740 20 client.17040 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:36.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.098+0000 7fe531a00740 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5611877dfc30 0x5611877e2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.098+0000 7fe531a00740 4 client.17040 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:58:36.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.098+0000 7fe531a00740 1 -- 172.21.15.83:0/3901624479 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611876c2a60 con 0x5611877dfc30 2024-04-23T14:58:36.104 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.098+0000 7fe52cc00640 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5611877dfc30 0x5611877e2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.098+0000 7fe52cc00640 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5611877dfc30 0x5611877e2050 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8e162d0e6bae1f5 server_cookie=1385a2aa86de9a76 in_seq=0 out_seq=0 2024-04-23T14:58:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.098+0000 7fe511600640 10 client.17040 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:58:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe511600640 1 -- 172.21.15.83:0/3901624479 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611876c2a60 con 0x5611877dfc30 2024-04-23T14:58:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe511600640 10 client.17040 handle_command_reply: tid=0 2024-04-23T14:58:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe531a00740 1 client.17040 shutdown 2024-04-23T14:58:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe531a00740 1 -- 172.21.15.83:0/3901624479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187756050 msgr2=0x56118776a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe531a00740 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187756050 0x56118776a270 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe51401a560 tx=0x7fe51400f7b0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe531a00740 1 -- 172.21.15.83:0/3901624479 shutdown_connections 2024-04-23T14:58:36.106 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe531a00740 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561187769150 0x5611877703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe531a00740 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561187756050 0x56118776a270 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe531a00740 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x5611877dfc30 0x5611877e2050 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe531a00740 1 --2- 172.21.15.83:0/3901624479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5611877556a0 0x561187769d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe511600640 0 client.17040 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:58:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe531a00740 1 -- 172.21.15.83:0/3901624479 >> 172.21.15.83:0/3901624479 conn(0x561187751c70 msgr2=0x561187764bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe531a00740 1 -- 172.21.15.83:0/3901624479 shutdown_connections 2024-04-23T14:58:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe531a00740 1 -- 172.21.15.83:0/3901624479 wait complete. 2024-04-23T14:58:36.107 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.099+0000 7fe531a00740 20 client.17040 trim_cache size 0 max 16384 2024-04-23T14:58:36.108 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:58:36.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.100+0000 7fe52ec00640 1 -- 172.21.15.83:0/2727250346 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe50c0a3d20 msgr2=0x7fe50c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.108 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.100+0000 7fe52ec00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe50c0a3d20 0x7fe50c0a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe5180b8990 tx=0x7fe5180f3f10 comp rx=0 tx=0).stop 2024-04-23T14:58:36.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.100+0000 7fe52ec00640 1 -- 172.21.15.83:0/2727250346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5281044b0 msgr2=0x7fe5281a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.100+0000 7fe52ec00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5281044b0 0x7fe5281a0840 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe518035040 tx=0x7fe518033b60 comp rx=0 tx=0).stop 2024-04-23T14:58:36.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.100+0000 7fe52ec00640 1 -- 172.21.15.83:0/2727250346 shutdown_connections 2024-04-23T14:58:36.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.100+0000 7fe52ec00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe50c0a3d20 0x7fe50c0a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.100+0000 7fe52ec00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe52819aa40 0x7fe52819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.100+0000 7fe52ec00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe528109580 0x7fe5281a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.100+0000 7fe52ec00640 1 --2- 172.21.15.83:0/2727250346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5281044b0 0x7fe5281a0840 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.109 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.100+0000 7fe52ec00640 1 -- 172.21.15.83:0/2727250346 >> 172.21.15.83:0/2727250346 conn(0x7fe5280ffd90 msgr2=0x7fe5281010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.100+0000 7fe52ec00640 1 -- 172.21.15.83:0/2727250346 shutdown_connections 2024-04-23T14:58:36.110 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.100+0000 7fe52ec00640 1 -- 172.21.15.83:0/2727250346 wait complete. 2024-04-23T14:58:36.113 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:35 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2365167563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:58:36.117 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:58:36.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:35 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2365167563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:58:36.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.309+0000 7f9fe6400640 1 -- 172.21.15.83:0/177690301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe0112d40 msgr2=0x7f9fe0115130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.309+0000 7f9fe6400640 1 --2- 172.21.15.83:0/177690301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe0112d40 0x7f9fe0115130 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009be0 tx=0x7f9fd802a6c0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.310+0000 7f9fe6400640 1 -- 172.21.15.83:0/177690301 shutdown_connections 2024-04-23T14:58:36.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.310+0000 7f9fe6400640 1 --2- 172.21.15.83:0/177690301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe0112d40 0x7f9fe0115130 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.310 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.310+0000 7f9fe6400640 1 --2- 172.21.15.83:0/177690301 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe0076e50 0x7f9fe0112800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.310+0000 7f9fe6400640 1 --2- 172.21.15.83:0/177690301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe0076530 0x7f9fe0076910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.310+0000 7f9fe6400640 1 -- 172.21.15.83:0/177690301 >> 172.21.15.83:0/177690301 conn(0x7f9fe00ffb70 msgr2=0x7f9fe0101f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.310+0000 7f9fe6400640 1 -- 172.21.15.83:0/177690301 shutdown_connections 2024-04-23T14:58:36.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.310+0000 7f9fe6400640 1 -- 172.21.15.83:0/177690301 wait complete. 2024-04-23T14:58:36.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fe6400640 1 Processor -- start 2024-04-23T14:58:36.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fe6400640 1 -- start start 2024-04-23T14:58:36.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fe6400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe0076530 0x7f9fe019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fe6400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe0076e50 0x7f9fe019f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.313 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fe6400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe0112d40 0x7f9fe01a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fe6400640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe0118810 con 0x7f9fe0076530 2024-04-23T14:58:36.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fe6400640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe0118690 con 0x7f9fe0112d40 2024-04-23T14:58:36.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fe6400640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe0118990 con 0x7f9fe0076e50 2024-04-23T14:58:36.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fe5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe0076530 0x7f9fe019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fe5000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe0076530 0x7f9fe019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52750/0 (socket says 172.21.15.83:52750) 2024-04-23T14:58:36.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fe5000640 1 -- 172.21.15.83:0/3493583941 learned_addr learned my addr 172.21.15.83:0/3493583941 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:36.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fdfe00640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe0076e50 0x7f9fe019f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.314 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.311+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe0112d40 0x7f9fe01a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.312+0000 7f9fe5000640 1 -- 172.21.15.83:0/3493583941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe0076e50 msgr2=0x7f9fe019f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.312+0000 7f9fe5000640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe0076e50 0x7f9fe019f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.312+0000 7f9fe5000640 1 -- 172.21.15.83:0/3493583941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe0112d40 msgr2=0x7f9fe01a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.312+0000 7f9fe5000640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe0112d40 0x7f9fe01a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.312+0000 7f9fe5000640 1 -- 172.21.15.83:0/3493583941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe01a3db0 con 0x7f9fe0076530 2024-04-23T14:58:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.312+0000 7f9fe5000640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe0076530 0x7f9fe019ee10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400e9f0 tx=0x7f9fd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.312+0000 7f9fdd600640 1 -- 172.21.15.83:0/3493583941 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400ccc0 con 0x7f9fe0076530 2024-04-23T14:58:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.312+0000 7f9fdd600640 1 -- 172.21.15.83:0/3493583941 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd4010910 con 0x7f9fe0076530 2024-04-23T14:58:36.315 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.312+0000 7f9fdd600640 1 -- 172.21.15.83:0/3493583941 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40188a0 con 0x7f9fe0076530 2024-04-23T14:58:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.312+0000 7f9fe6400640 1 -- 172.21.15.83:0/3493583941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe01a40a0 con 0x7f9fe0076530 2024-04-23T14:58:36.316 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.313+0000 7f9fe6400640 1 -- 172.21.15.83:0/3493583941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe01ab930 con 0x7f9fe0076530 2024-04-23T14:58:36.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.316+0000 7f9fea9c0740 1 Processor -- start 2024-04-23T14:58:36.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.316+0000 7f9fea9c0740 1 -- start start 2024-04-23T14:58:36.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.316+0000 7f9fea9c0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4cfb40 0x555d2b4cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.316+0000 7f9fea9c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4d0460 0x555d2b4da5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.316+0000 7f9fea9c0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4dab30 0x555d2b4dcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.319 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.316+0000 7f9fea9c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555d2b4351d0 con 0x555d2b4d0460 2024-04-23T14:58:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.316+0000 7f9fea9c0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555d2b3db450 con 0x555d2b4cfb40 2024-04-23T14:58:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.316+0000 7f9fea9c0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555d2b431190 con 0x555d2b4dab30 2024-04-23T14:58:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.317+0000 7f9fdd600640 1 -- 172.21.15.83:0/3493583941 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9fd4010ac0 con 0x7f9fe0076530 2024-04-23T14:58:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.317+0000 7f9fe5a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4cfb40 0x555d2b4cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.317+0000 7f9fe5a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4cfb40 0x555d2b4cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:47492/0 (socket says 172.21.15.83:47492) 2024-04-23T14:58:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.317+0000 7f9fe5a00640 1 -- 172.21.15.83:0/3205165767 learned_addr learned my addr 172.21.15.83:0/3205165767 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.317+0000 7f9fdfe00640 1 --2- 172.21.15.83:0/3205165767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4dab30 0x555d2b4dcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.317+0000 7f9fe5a00640 1 -- 172.21.15.83:0/3205165767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4dab30 msgr2=0x555d2b4dcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.317+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/3205165767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4dab30 0x555d2b4dcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.317+0000 7f9fe5a00640 1 -- 172.21.15.83:0/3205165767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4d0460 msgr2=0x555d2b4da5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.317+0000 7f9fe5000640 1 --2- 172.21.15.83:0/3205165767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4d0460 0x555d2b4da5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.317+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/3205165767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4d0460 0x555d2b4da5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.317+0000 7f9fe5a00640 1 -- 172.21.15.83:0/3205165767 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d2b498310 con 0x555d2b4cfb40 2024-04-23T14:58:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.317+0000 7f9fe5000640 1 --2- 172.21.15.83:0/3205165767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4d0460 0x555d2b4da5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:58:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.318+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/3205165767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4cfb40 0x555d2b4cff20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009be0 tx=0x7f9fd80303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2c0892b31dc7291 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.318+0000 7f9fd2000640 1 -- 172.21.15.83:0/3205165767 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8042070 con 0x555d2b4cfb40 2024-04-23T14:58:36.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.318+0000 7f9fd2000640 1 -- 172.21.15.83:0/3205165767 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd8037040 con 0x555d2b4cfb40 2024-04-23T14:58:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.318+0000 7f9fd2000640 1 -- 172.21.15.83:0/3205165767 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8036070 con 0x555d2b4cfb40 2024-04-23T14:58:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.318+0000 7f9fdd600640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9fc80a3d20 0x7f9fc80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.318+0000 7f9fdd600640 1 -- 172.21.15.83:0/3493583941 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f9fd40d0ca0 con 0x7f9fe0076530 2024-04-23T14:58:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.318+0000 7f9fe5000640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9fc80a3d20 0x7f9fc80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.319+0000 7f9fea9c0740 1 -- 172.21.15.83:0/3205165767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4cfb40 msgr2=0x555d2b4cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.319+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/3205165767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4cfb40 0x555d2b4cff20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009be0 tx=0x7f9fd80303a0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.319+0000 7f9fea9c0740 1 -- 172.21.15.83:0/3205165767 shutdown_connections 2024-04-23T14:58:36.322 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.319+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/3205165767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4dab30 0x555d2b4dcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.319+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/3205165767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4d0460 0x555d2b4da5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.319+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/3205165767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4cfb40 0x555d2b4cff20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.319+0000 7f9fea9c0740 1 -- 172.21.15.83:0/3205165767 >> 172.21.15.83:0/3205165767 conn(0x555d2b4cbe60 msgr2=0x555d2b4cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:36.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.319+0000 7f9fea9c0740 1 -- 172.21.15.83:0/3205165767 shutdown_connections 2024-04-23T14:58:36.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.319+0000 7f9fea9c0740 1 -- 172.21.15.83:0/3205165767 wait complete. 2024-04-23T14:58:36.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.319+0000 7f9fe5000640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9fc80a3d20 0x7f9fc80a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x555d2b4d1100 tx=0x7f9fd4009700 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:36.323 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.319+0000 7f9fea9c0740 1 -- start start 2024-04-23T14:58:36.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fea9c0740 1 Processor -- start 2024-04-23T14:58:36.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fea9c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 0x555d2b4da5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fea9c0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4d78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fea9c0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4dab30 0x555d2b4d8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fea9c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555d2b4dd960 con 0x555d2b4cfb40 2024-04-23T14:58:36.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fea9c0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x555d2b3ee810 con 0x555d2b4dab30 2024-04-23T14:58:36.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fea9c0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x555d2b4ddae0 con 0x555d2b4d0460 2024-04-23T14:58:36.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fea9c0740 10 client.0 _ll_register_callbacks cb 0x555d2b3bc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:36.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fe5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 0x555d2b4da5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fdfe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4d78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fe5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 0x555d2b4da5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52762/0 (socket says 172.21.15.83:52762) 2024-04-23T14:58:36.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fe5a00640 1 -- 172.21.15.83:0/1135790930 learned_addr learned my addr 172.21.15.83:0/1135790930 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:36.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.320+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/1135790930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4dab30 0x555d2b4d8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.321+0000 7f9fdfe00640 1 -- 172.21.15.83:0/1135790930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4dab30 msgr2=0x555d2b4d8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.321+0000 7f9fdfe00640 1 --2- 172.21.15.83:0/1135790930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4dab30 0x555d2b4d8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.321+0000 7f9fdfe00640 1 -- 172.21.15.83:0/1135790930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 msgr2=0x555d2b4da5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.321+0000 7f9fdfe00640 1 --2- 172.21.15.83:0/1135790930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 0x555d2b4da5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.321+0000 7f9fdfe00640 1 -- 172.21.15.83:0/1135790930 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d2b3ff3f0 con 0x555d2b4d0460 2024-04-23T14:58:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.321+0000 7f9fdfe00640 1 --2- 172.21.15.83:0/1135790930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4d78d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00791a0 tx=0x7f9fcc012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12249965f914a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.321+0000 7f9fc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:58:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.321+0000 7f9fc6a00640 1 -- 172.21.15.83:0/1135790930 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00f8f0 con 0x555d2b4d0460 2024-04-23T14:58:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.321+0000 7f9fc6a00640 1 -- 172.21.15.83:0/1135790930 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f9fcc00fa90 con 0x555d2b4d0460 2024-04-23T14:58:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.321+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1135790930 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fd8037040 con 0x555d2b4d0460 2024-04-23T14:58:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.321+0000 7f9fc6a00640 1 -- 172.21.15.83:0/1135790930 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9fcc00fcd0 con 0x555d2b4d0460 2024-04-23T14:58:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.322+0000 7f9fea9c0740 10 client.27229 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.322+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1135790930 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555d2b1473f0 con 0x555d2b4d0460 2024-04-23T14:58:36.329 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.322+0000 7f9fc6a00640 1 -- 172.21.15.83:0/1135790930 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd8037040 con 0x555d2b4d0460 2024-04-23T14:58:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.322+0000 7f9fc6a00640 1 -- 172.21.15.83:0/1135790930 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f9fcc01c420 con 0x555d2b4d0460 2024-04-23T14:58:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.322+0000 7f9fea9c0740 10 client.27229 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.322+0000 7f9fea9c0740 10 client.27229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.322+0000 7f9fea9c0740 20 client.27229 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.322+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/1135790930 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x555d2b4ead00 0x555d2b4ed160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.322+0000 7f9fea9c0740 4 client.27229 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:58:36.330 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.322+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1135790930 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d2b442bf0 con 0x555d2b4ead00 2024-04-23T14:58:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.323+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/1135790930 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x555d2b4ead00 0x555d2b4ed160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.323+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/1135790930 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x555d2b4ead00 0x555d2b4ed160 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c046e42ddb44935e server_cookie=dda65a733262b3bc in_seq=0 out_seq=0 2024-04-23T14:58:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.323+0000 7f9fc6a00640 10 client.27229 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fc6a00640 1 -- 172.21.15.83:0/1135790930 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d2b442bf0 con 0x555d2b4ead00 2024-04-23T14:58:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fc6a00640 10 client.27229 handle_command_reply: tid=0 2024-04-23T14:58:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fea9c0740 1 client.27229 shutdown 2024-04-23T14:58:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1135790930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 msgr2=0x555d2b4d78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/1135790930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4d78d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00791a0 tx=0x7f9fcc012b60 comp rx=0 tx=0).stop 2024-04-23T14:58:36.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1135790930 shutdown_connections 2024-04-23T14:58:36.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/1135790930 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x555d2b4ead00 0x555d2b4ed160 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/1135790930 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4dab30 0x555d2b4d8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/1135790930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4d78d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/1135790930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 0x555d2b4da5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fc6a00640 0 client.27229 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:36.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1135790930 >> 172.21.15.83:0/1135790930 conn(0x555d2b4cbe60 msgr2=0x555d2b4d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:36.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1135790930 shutdown_connections 2024-04-23T14:58:36.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.324+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1135790930 wait complete. 2024-04-23T14:58:36.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.325+0000 7f9fea9c0740 20 client.27229 trim_cache size 0 max 16384 2024-04-23T14:58:36.333 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:35 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2365167563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:58:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.339+0000 7f9fea9c0740 1 Processor -- start 2024-04-23T14:58:36.339 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.339+0000 7f9fea9c0740 1 -- start start 2024-04-23T14:58:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.339+0000 7f9fea9c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 0x555d2b4d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.339+0000 7f9fea9c0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4da110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.339+0000 7f9fea9c0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4dab30 0x555d2b4f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.339+0000 7f9fea9c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b6a30 con 0x555d2b4cfb40 2024-04-23T14:58:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.339+0000 7f9fea9c0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b4300 con 0x555d2b4dab30 2024-04-23T14:58:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.339+0000 7f9fea9c0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b2170 con 0x555d2b4d0460 2024-04-23T14:58:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.339+0000 7f9fe5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 0x555d2b4d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.339+0000 7f9fe5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 0x555d2b4d22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52772/0 (socket says 172.21.15.83:52772) 2024-04-23T14:58:36.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.339+0000 7f9fe5a00640 1 -- 172.21.15.83:0/161521909 learned_addr learned my addr 172.21.15.83:0/161521909 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.340+0000 7f9fdfe00640 1 --2- 172.21.15.83:0/161521909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4da110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.340+0000 7f9fe5a00640 1 -- 172.21.15.83:0/161521909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 msgr2=0x555d2b4da110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.340+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/161521909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4da110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.340+0000 7f9fe5a00640 1 -- 172.21.15.83:0/161521909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4dab30 msgr2=0x555d2b4f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.340+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/161521909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4dab30 0x555d2b4f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.340+0000 7f9fe5a00640 1 -- 172.21.15.83:0/161521909 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d2b2117d0 con 0x555d2b4cfb40 2024-04-23T14:58:36.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.340+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/161521909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 0x555d2b4d22a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8003af0 tx=0x7f9fd8009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efe16547ae119f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.340+0000 7f9fd1600640 1 -- 172.21.15.83:0/161521909 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8042040 con 0x555d2b4cfb40 2024-04-23T14:58:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.340+0000 7f9fd1600640 1 -- 172.21.15.83:0/161521909 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd8030730 con 0x555d2b4cfb40 2024-04-23T14:58:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.340+0000 7f9fd1600640 1 -- 172.21.15.83:0/161521909 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8036040 con 0x555d2b4cfb40 2024-04-23T14:58:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.341+0000 7f9fea9c0740 1 -- 172.21.15.83:0/161521909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 msgr2=0x555d2b4d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.341+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/161521909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 0x555d2b4d22a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8003af0 tx=0x7f9fd8009330 comp rx=0 tx=0).stop 2024-04-23T14:58:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.341+0000 7f9fea9c0740 1 -- 172.21.15.83:0/161521909 shutdown_connections 2024-04-23T14:58:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.341+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/161521909 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4dab30 0x555d2b4f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.341+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/161521909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4da110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.341+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/161521909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4cfb40 0x555d2b4d22a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.341+0000 7f9fea9c0740 1 -- 172.21.15.83:0/161521909 >> 172.21.15.83:0/161521909 conn(0x555d2b4cbe60 msgr2=0x555d2b4ca6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.341+0000 7f9fea9c0740 1 -- 172.21.15.83:0/161521909 shutdown_connections 2024-04-23T14:58:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.341+0000 7f9fea9c0740 1 -- 172.21.15.83:0/161521909 wait complete. 2024-04-23T14:58:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.341+0000 7f9fea9c0740 1 -- start start 2024-04-23T14:58:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.341+0000 7f9fea9c0740 1 Processor -- start 2024-04-23T14:58:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.342+0000 7f9fea9c0740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4cfb40 0x555d2b4e2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.342+0000 7f9fea9c0740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4e32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.347 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.342+0000 7f9fea9c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4dab30 0x555d2b4ddd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.342+0000 7f9fea9c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b27c0 con 0x555d2b4dab30 2024-04-23T14:58:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.342+0000 7f9fea9c0740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b3c30 con 0x555d2b4cfb40 2024-04-23T14:58:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.342+0000 7f9fea9c0740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01b2e90 con 0x555d2b4d0460 2024-04-23T14:58:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.342+0000 7f9fea9c0740 10 client.0 _ll_register_callbacks cb 0x7f9fcc009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.342+0000 7f9fe5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4dab30 0x555d2b4ddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.342+0000 7f9fe5a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4dab30 0x555d2b4ddd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52784/0 (socket says 172.21.15.83:52784) 2024-04-23T14:58:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.342+0000 7f9fe5a00640 1 -- 172.21.15.83:0/1198016510 learned_addr learned my addr 172.21.15.83:0/1198016510 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.342+0000 7f9fdfe00640 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4e32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.348 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.342+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4cfb40 0x555d2b4e2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fe5a00640 1 -- 172.21.15.83:0/1198016510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 msgr2=0x555d2b4e32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4e32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fe5a00640 1 -- 172.21.15.83:0/1198016510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4cfb40 msgr2=0x555d2b4e2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4cfb40 0x555d2b4e2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fe5a00640 1 -- 172.21.15.83:0/1198016510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d2b4de3e0 con 0x555d2b4dab30 2024-04-23T14:58:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fe5a00640 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4dab30 0x555d2b4ddd20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009500 tx=0x7f9fd8033040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27da8ccc5b72b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fc6a00640 1 -- 172.21.15.83:0/1198016510 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8036070 con 0x555d2b4dab30 2024-04-23T14:58:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fc6a00640 1 -- 172.21.15.83:0/1198016510 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f9fd8055440 con 0x555d2b4dab30 2024-04-23T14:58:36.349 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1198016510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fd8030730 con 0x555d2b4dab30 2024-04-23T14:58:36.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fc6a00640 1 -- 172.21.15.83:0/1198016510 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9fd8042040 con 0x555d2b4dab30 2024-04-23T14:58:36.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fea9c0740 10 client.17055 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:36.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1198016510 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555d2b551be0 con 0x555d2b4dab30 2024-04-23T14:58:36.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.343+0000 7f9fc6a00640 1 -- 172.21.15.83:0/1198016510 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd8030730 con 0x555d2b4dab30 2024-04-23T14:58:36.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.344+0000 7f9fc6a00640 1 -- 172.21.15.83:0/1198016510 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f9fd8055ce0 con 0x555d2b4dab30 2024-04-23T14:58:36.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.344+0000 7f9fea9c0740 10 client.17055 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:36.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.344+0000 7f9fea9c0740 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:36.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.344+0000 7f9fea9c0740 20 client.17055 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.344+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x555d2b551db0 0x555d2b5541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.344+0000 7f9fea9c0740 4 client.17055 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:58:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.344+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1198016510 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d2b43ca60 con 0x555d2b551db0 2024-04-23T14:58:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.344+0000 7f9fdfe00640 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x555d2b551db0 0x555d2b5541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fdfe00640 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x555d2b551db0 0x555d2b5541a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53a36324c929b3f server_cookie=aa69f91f7d1fdd99 in_seq=0 out_seq=0 2024-04-23T14:58:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fc6a00640 10 client.17055 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fc6a00640 1 -- 172.21.15.83:0/1198016510 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d2b43ca60 con 0x555d2b551db0 2024-04-23T14:58:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fc6a00640 10 client.17055 handle_command_reply: tid=0 2024-04-23T14:58:36.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fea9c0740 1 client.17055 shutdown 2024-04-23T14:58:36.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1198016510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4dab30 msgr2=0x555d2b4ddd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4dab30 0x555d2b4ddd20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009500 tx=0x7f9fd8033040 comp rx=0 tx=0).stop 2024-04-23T14:58:36.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1198016510 shutdown_connections 2024-04-23T14:58:36.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x555d2b551db0 0x555d2b5541a0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555d2b4dab30 0x555d2b4ddd20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555d2b4d0460 0x555d2b4e32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fc6a00640 0 client.17055 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:36.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fea9c0740 1 --2- 172.21.15.83:0/1198016510 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555d2b4cfb40 0x555d2b4e2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1198016510 >> 172.21.15.83:0/1198016510 conn(0x555d2b4cbe60 msgr2=0x555d2b4caf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:36.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1198016510 shutdown_connections 2024-04-23T14:58:36.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fea9c0740 1 -- 172.21.15.83:0/1198016510 wait complete. 2024-04-23T14:58:36.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.345+0000 7f9fea9c0740 20 client.17055 trim_cache size 0 max 16384 2024-04-23T14:58:36.353 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:58:36.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.346+0000 7f9fe6400640 1 -- 172.21.15.83:0/3493583941 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9fc80a3d20 msgr2=0x7f9fc80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.346+0000 7f9fe6400640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9fc80a3d20 0x7f9fc80a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x555d2b4d1100 tx=0x7f9fd4009700 comp rx=0 tx=0).stop 2024-04-23T14:58:36.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.346+0000 7f9fe6400640 1 -- 172.21.15.83:0/3493583941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe0076530 msgr2=0x7f9fe019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:36.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.346+0000 7f9fe6400640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe0076530 0x7f9fe019ee10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400e9f0 tx=0x7f9fd400edf0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.346+0000 7f9fe6400640 1 -- 172.21.15.83:0/3493583941 shutdown_connections 2024-04-23T14:58:36.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.346+0000 7f9fe6400640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fe0112d40 0x7f9fe01a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.346+0000 7f9fe6400640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fe0076e50 0x7f9fe019f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.346+0000 7f9fe6400640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f9fc80a3d20 0x7f9fc80a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.346+0000 7f9fe6400640 1 --2- 172.21.15.83:0/3493583941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9fe0076530 0x7f9fe019ee10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:36.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.346+0000 7f9fe6400640 1 -- 172.21.15.83:0/3493583941 >> 172.21.15.83:0/3493583941 conn(0x7f9fe00ffb70 msgr2=0x7f9fe0101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:36.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.346+0000 7f9fe6400640 1 -- 172.21.15.83:0/3493583941 shutdown_connections 2024-04-23T14:58:36.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:36.346+0000 7f9fe6400640 1 -- 172.21.15.83:0/3493583941 wait complete. 2024-04-23T14:58:37.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:36 smithi203 ceph-mon[32800]: pgmap v316: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:37.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:36 smithi083 ceph-mon[27867]: pgmap v316: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:37.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:36 smithi173 ceph-mon[32443]: pgmap v316: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:37.365 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-23T14:58:37.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:58:37.365 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-23T14:58:37.586 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.586+0000 7faca7e00640 1 -- 172.21.15.83:0/372249435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8074cc0 msgr2=0x7faca8112970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.586+0000 7faca7e00640 1 --2- 172.21.15.83:0/372249435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8074cc0 0x7faca8112970 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fac94009860 tx=0x7fac9402a3d0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.586+0000 7faca7e00640 1 -- 172.21.15.83:0/372249435 shutdown_connections 2024-04-23T14:58:37.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.586+0000 7faca7e00640 1 --2- 172.21.15.83:0/372249435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faca8112eb0 0x7faca81152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.586+0000 7faca7e00640 1 --2- 172.21.15.83:0/372249435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8074cc0 0x7faca8112970 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.586+0000 7faca7e00640 1 --2- 172.21.15.83:0/372249435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca80743a0 0x7faca8074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.586+0000 7faca7e00640 1 -- 172.21.15.83:0/372249435 >> 172.21.15.83:0/372249435 conn(0x7faca80fd9d0 msgr2=0x7faca80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:37.587 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.586+0000 7faca7e00640 1 -- 172.21.15.83:0/372249435 shutdown_connections 2024-04-23T14:58:37.588 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.586+0000 7faca7e00640 1 -- 172.21.15.83:0/372249435 wait complete. 2024-04-23T14:58:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.587+0000 7faca7e00640 1 Processor -- start 2024-04-23T14:58:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.587+0000 7faca7e00640 1 -- start start 2024-04-23T14:58:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.588+0000 7faca7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faca80743a0 0x7faca819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.588+0000 7faca7e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8074cc0 0x7faca819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.588+0000 7faca7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca8112eb0 0x7faca81a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.588+0000 7faca7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faca8118890 con 0x7faca8112eb0 2024-04-23T14:58:37.590 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.588+0000 7faca7e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faca8118710 con 0x7faca80743a0 2024-04-23T14:58:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.588+0000 7faca7e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faca8118a10 con 0x7faca8074cc0 2024-04-23T14:58:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.588+0000 7faca6000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8074cc0 0x7faca819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.588+0000 7faca7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca8112eb0 0x7faca81a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.588+0000 7faca6000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8074cc0 0x7faca819f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41280/0 (socket says 172.21.15.83:41280) 2024-04-23T14:58:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.588+0000 7faca6000640 1 -- 172.21.15.83:0/1246450235 learned_addr learned my addr 172.21.15.83:0/1246450235 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.588+0000 7faca6a00640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faca80743a0 0x7faca819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.589+0000 7faca6000640 1 -- 172.21.15.83:0/1246450235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faca80743a0 msgr2=0x7faca819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.589+0000 7faca6000640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faca80743a0 0x7faca819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.591 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.589+0000 7faca6000640 1 -- 172.21.15.83:0/1246450235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca8112eb0 msgr2=0x7faca81a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.589+0000 7faca6000640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca8112eb0 0x7faca81a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.589+0000 7faca6000640 1 -- 172.21.15.83:0/1246450235 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca81a3df0 con 0x7faca8074cc0 2024-04-23T14:58:37.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.589+0000 7faca7400640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca8112eb0 0x7faca81a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:37.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.589+0000 7faca6000640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8074cc0 0x7faca819f390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fac9402a8f0 tx=0x7fac94030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:37.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.589+0000 7facaca00640 1 -- 172.21.15.83:0/1246450235 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94040020 con 0x7faca8074cc0 2024-04-23T14:58:37.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.589+0000 7faca7e00640 1 -- 172.21.15.83:0/1246450235 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca81a4080 con 0x7faca8074cc0 2024-04-23T14:58:37.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.589+0000 7facaca00640 1 -- 172.21.15.83:0/1246450235 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac94005510 con 0x7faca8074cc0 2024-04-23T14:58:37.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.589+0000 7facaca00640 1 -- 172.21.15.83:0/1246450235 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940046c0 con 0x7faca8074cc0 2024-04-23T14:58:37.592 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.590+0000 7faca7e00640 1 -- 172.21.15.83:0/1246450235 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca81ab970 con 0x7faca8074cc0 2024-04-23T14:58:37.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.594+0000 7facb0c39740 1 Processor -- start 2024-04-23T14:58:37.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.594+0000 7facb0c39740 1 -- start start 2024-04-23T14:58:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.594+0000 7facb0c39740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 0x563b30d1d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.594+0000 7facb0c39740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d1db90 0x563b30d27d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.594+0000 7facb0c39740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d28260 0x563b30d2a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.594+0000 7facb0c39740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563b30c82e70 con 0x563b30d1d270 2024-04-23T14:58:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.594+0000 7facb0c39740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563b30c29380 con 0x563b30d28260 2024-04-23T14:58:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.594+0000 7facb0c39740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563b30c7ee30 con 0x563b30d1db90 2024-04-23T14:58:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.594+0000 7facaca00640 1 -- 172.21.15.83:0/1246450235 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fac94043020 con 0x7faca8074cc0 2024-04-23T14:58:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.595+0000 7faca7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 0x563b30d1d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.595+0000 7faca7400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 0x563b30d1d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52814/0 (socket says 172.21.15.83:52814) 2024-04-23T14:58:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.595+0000 7faca7400640 1 -- 172.21.15.83:0/2500512020 learned_addr learned my addr 172.21.15.83:0/2500512020 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.595+0000 7faca6a00640 1 --2- 172.21.15.83:0/2500512020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d1db90 0x563b30d27d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.596 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.595+0000 7faca6000640 1 --2- 172.21.15.83:0/2500512020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d28260 0x563b30d2a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.595+0000 7faca7400640 1 -- 172.21.15.83:0/2500512020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d1db90 msgr2=0x563b30d27d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.595+0000 7faca7400640 1 --2- 172.21.15.83:0/2500512020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d1db90 0x563b30d27d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.595+0000 7faca7400640 1 -- 172.21.15.83:0/2500512020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d28260 msgr2=0x563b30d2a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.595+0000 7faca7400640 1 --2- 172.21.15.83:0/2500512020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d28260 0x563b30d2a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.595+0000 7faca7400640 1 -- 172.21.15.83:0/2500512020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b30a5d7e0 con 0x563b30d1d270 2024-04-23T14:58:37.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.595+0000 7faca6000640 1 --2- 172.21.15.83:0/2500512020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d28260 0x563b30d2a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:58:37.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.595+0000 7faca7400640 1 --2- 172.21.15.83:0/2500512020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 0x563b30d1d650 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faca0011760 tx=0x7faca0011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90cfb7b8274a563 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:37.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.596+0000 7fac93e00640 1 -- 172.21.15.83:0/2500512020 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0004d20 con 0x563b30d1d270 2024-04-23T14:58:37.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.596+0000 7fac93e00640 1 -- 172.21.15.83:0/2500512020 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faca000f9a0 con 0x563b30d1d270 2024-04-23T14:58:37.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.596+0000 7fac93e00640 1 -- 172.21.15.83:0/2500512020 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0004520 con 0x563b30d1d270 2024-04-23T14:58:37.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.596+0000 7facaca00640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fac8c0a3da0 0x7fac8c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.596+0000 7facb0c39740 1 -- 172.21.15.83:0/2500512020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 msgr2=0x563b30d1d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.596+0000 7facb0c39740 1 --2- 172.21.15.83:0/2500512020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 0x563b30d1d650 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faca0011760 tx=0x7faca0011c20 comp rx=0 tx=0).stop 2024-04-23T14:58:37.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.597+0000 7facaca00640 1 -- 172.21.15.83:0/1246450235 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fac940f1630 con 0x7faca8074cc0 2024-04-23T14:58:37.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.597+0000 7facb0c39740 1 -- 172.21.15.83:0/2500512020 shutdown_connections 2024-04-23T14:58:37.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.597+0000 7faca6a00640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fac8c0a3da0 0x7fac8c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.597+0000 7facb0c39740 1 --2- 172.21.15.83:0/2500512020 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d28260 0x563b30d2a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.597+0000 7facb0c39740 1 --2- 172.21.15.83:0/2500512020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d1db90 0x563b30d27d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.597+0000 7facb0c39740 1 --2- 172.21.15.83:0/2500512020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 0x563b30d1d650 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.597+0000 7facb0c39740 1 -- 172.21.15.83:0/2500512020 >> 172.21.15.83:0/2500512020 conn(0x563b30d19590 msgr2=0x563b30d19970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.597+0000 7facb0c39740 1 -- 172.21.15.83:0/2500512020 shutdown_connections 2024-04-23T14:58:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.597+0000 7facb0c39740 1 -- 172.21.15.83:0/2500512020 wait complete. 2024-04-23T14:58:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.597+0000 7faca6a00640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fac8c0a3da0 0x7fac8c0a6260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x563b30d1e830 tx=0x7fac98014040 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.597+0000 7facb0c39740 1 -- start start 2024-04-23T14:58:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.598+0000 7facb0c39740 1 Processor -- start 2024-04-23T14:58:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.598+0000 7facb0c39740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 0x563b30d251b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.598+0000 7facb0c39740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d1db90 0x563b30d256f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.604 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.598+0000 7facb0c39740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d28260 0x563b30d25c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.598+0000 7facb0c39740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563b30d2b050 con 0x563b30d1d270 2024-04-23T14:58:37.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.598+0000 7facb0c39740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x563b30c3c660 con 0x563b30d28260 2024-04-23T14:58:37.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.598+0000 7facb0c39740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x563b30d2b1d0 con 0x563b30d1db90 2024-04-23T14:58:37.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.598+0000 7facb0c39740 10 client.0 _ll_register_callbacks cb 0x563b30c2aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:37.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.598+0000 7faca6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 0x563b30d251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.598+0000 7faca6000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d1db90 0x563b30d256f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.598+0000 7faca6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 0x563b30d251b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52828/0 (socket says 172.21.15.83:52828) 2024-04-23T14:58:37.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.598+0000 7faca6a00640 1 -- 172.21.15.83:0/2653035915 learned_addr learned my addr 172.21.15.83:0/2653035915 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:37.605 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.599+0000 7faca7400640 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d28260 0x563b30d25c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.599+0000 7faca6000640 1 -- 172.21.15.83:0/2653035915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d28260 msgr2=0x563b30d25c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.599+0000 7faca6000640 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d28260 0x563b30d25c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.599+0000 7faca6000640 1 -- 172.21.15.83:0/2653035915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 msgr2=0x563b30d251b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.599+0000 7faca6000640 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 0x563b30d251b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.599+0000 7faca6000640 1 -- 172.21.15.83:0/2653035915 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b30abdf70 con 0x563b30d1db90 2024-04-23T14:58:37.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.599+0000 7faca6a00640 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 0x563b30d251b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:37.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.599+0000 7faca6000640 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d1db90 0x563b30d256f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fac940eed70 tx=0x7fac940b9260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7108bdd2a5ab5289 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:37.606 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.599+0000 7fac90c00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:58:37.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.599+0000 7fac90c00640 1 -- 172.21.15.83:0/2653035915 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94004470 con 0x563b30d1db90 2024-04-23T14:58:37.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.599+0000 7fac90c00640 1 -- 172.21.15.83:0/2653035915 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fac940b9c30 con 0x563b30d1db90 2024-04-23T14:58:37.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.599+0000 7facb0c39740 1 -- 172.21.15.83:0/2653035915 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faca000f9a0 con 0x563b30d1db90 2024-04-23T14:58:37.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.600+0000 7fac90c00640 1 -- 172.21.15.83:0/2653035915 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac94031400 con 0x563b30d1db90 2024-04-23T14:58:37.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.600+0000 7facb0c39740 10 client.27247 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:37.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.600+0000 7facb0c39740 1 -- 172.21.15.83:0/2653035915 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b30955170 con 0x563b30d1db90 2024-04-23T14:58:37.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.600+0000 7fac90c00640 1 -- 172.21.15.83:0/2653035915 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faca000f9a0 con 0x563b30d1db90 2024-04-23T14:58:37.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.600+0000 7fac90c00640 1 -- 172.21.15.83:0/2653035915 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fac940eeff0 con 0x563b30d1db90 2024-04-23T14:58:37.607 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.600+0000 7facb0c39740 10 client.27247 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:37.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.600+0000 7facb0c39740 10 client.27247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:37.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.600+0000 7facb0c39740 20 client.27247 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:37.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.600+0000 7facb0c39740 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563b30d38430 0x563b30d3a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.600+0000 7facb0c39740 4 client.27247 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:58:37.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.600+0000 7facb0c39740 1 -- 172.21.15.83:0/2653035915 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b30c90890 con 0x563b30d38430 2024-04-23T14:58:37.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.601+0000 7faca6a00640 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563b30d38430 0x563b30d3a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.601+0000 7faca6a00640 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563b30d38430 0x563b30d3a8a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83d7dce592a256e server_cookie=105c8d941733fb37 in_seq=0 out_seq=0 2024-04-23T14:58:37.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.601+0000 7fac90c00640 10 client.27247 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:37.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7fac90c00640 1 -- 172.21.15.83:0/2653035915 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b30c90890 con 0x563b30d38430 2024-04-23T14:58:37.608 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7fac90c00640 10 client.27247 handle_command_reply: tid=0 2024-04-23T14:58:37.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7facb0c39740 1 client.27247 shutdown 2024-04-23T14:58:37.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7facb0c39740 1 -- 172.21.15.83:0/2653035915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d1db90 msgr2=0x563b30d256f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7facb0c39740 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d1db90 0x563b30d256f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fac940eed70 tx=0x7fac940b9260 comp rx=0 tx=0).stop 2024-04-23T14:58:37.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7facb0c39740 1 -- 172.21.15.83:0/2653035915 shutdown_connections 2024-04-23T14:58:37.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7facb0c39740 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563b30d38430 0x563b30d3a8a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7facb0c39740 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d28260 0x563b30d25c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7facb0c39740 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d1db90 0x563b30d256f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7facb0c39740 1 --2- 172.21.15.83:0/2653035915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1d270 0x563b30d251b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.609 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7fac90c00640 0 client.27247 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:37.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7facb0c39740 1 -- 172.21.15.83:0/2653035915 >> 172.21.15.83:0/2653035915 conn(0x563b30d19590 msgr2=0x563b30d1f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:37.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.602+0000 7facb0c39740 1 -- 172.21.15.83:0/2653035915 shutdown_connections 2024-04-23T14:58:37.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.603+0000 7facb0c39740 1 -- 172.21.15.83:0/2653035915 wait complete. 2024-04-23T14:58:37.610 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.603+0000 7facb0c39740 20 client.27247 trim_cache size 0 max 16384 2024-04-23T14:58:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.622+0000 7facb0c39740 1 Processor -- start 2024-04-23T14:58:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.622+0000 7facb0c39740 1 -- start start 2024-04-23T14:58:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.622+0000 7facb0c39740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d1d270 0x563b30d33f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.622+0000 7facb0c39740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 0x563b30d1fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.622+0000 7facb0c39740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d28260 0x563b30d18570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.622+0000 7facb0c39740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faca81b6a00 con 0x563b30d1db90 2024-04-23T14:58:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.622+0000 7facb0c39740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faca81b42d0 con 0x563b30d1d270 2024-04-23T14:58:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.622+0000 7facb0c39740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faca81b2140 con 0x563b30d28260 2024-04-23T14:58:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.622+0000 7faca6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 0x563b30d1fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.622+0000 7faca6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 0x563b30d1fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52834/0 (socket says 172.21.15.83:52834) 2024-04-23T14:58:37.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.622+0000 7faca6000640 1 -- 172.21.15.83:0/3801250661 learned_addr learned my addr 172.21.15.83:0/3801250661 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:37.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.622+0000 7faca7400640 1 --2- 172.21.15.83:0/3801250661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d28260 0x563b30d18570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.623+0000 7faca6a00640 1 --2- 172.21.15.83:0/3801250661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d1d270 0x563b30d33f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.623+0000 7faca6000640 1 -- 172.21.15.83:0/3801250661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d28260 msgr2=0x563b30d18570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.623+0000 7faca6000640 1 --2- 172.21.15.83:0/3801250661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d28260 0x563b30d18570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.623+0000 7faca6000640 1 -- 172.21.15.83:0/3801250661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d1d270 msgr2=0x563b30d33f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.623+0000 7faca6000640 1 --2- 172.21.15.83:0/3801250661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d1d270 0x563b30d33f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.623+0000 7faca6000640 1 -- 172.21.15.83:0/3801250661 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b30d24160 con 0x563b30d1db90 2024-04-23T14:58:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.623+0000 7faca6a00640 1 --2- 172.21.15.83:0/3801250661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d1d270 0x563b30d33f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:58:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.623+0000 7faca6000640 1 --2- 172.21.15.83:0/3801250661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 0x563b30d1fa10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fac940b9a20 tx=0x7fac94005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1839a78a8166a809 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.623+0000 7fac93400640 1 -- 172.21.15.83:0/3801250661 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940b9860 con 0x563b30d1db90 2024-04-23T14:58:37.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.624+0000 7fac93400640 1 -- 172.21.15.83:0/3801250661 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac94004470 con 0x563b30d1db90 2024-04-23T14:58:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.624+0000 7fac93400640 1 -- 172.21.15.83:0/3801250661 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940b9c30 con 0x563b30d1db90 2024-04-23T14:58:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.624+0000 7facb0c39740 1 -- 172.21.15.83:0/3801250661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 msgr2=0x563b30d1fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.624+0000 7facb0c39740 1 --2- 172.21.15.83:0/3801250661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 0x563b30d1fa10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fac940b9a20 tx=0x7fac94005f90 comp rx=0 tx=0).stop 2024-04-23T14:58:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.624+0000 7facb0c39740 1 -- 172.21.15.83:0/3801250661 shutdown_connections 2024-04-23T14:58:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.624+0000 7facb0c39740 1 --2- 172.21.15.83:0/3801250661 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d28260 0x563b30d18570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.624+0000 7facb0c39740 1 --2- 172.21.15.83:0/3801250661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 0x563b30d1fa10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.624+0000 7facb0c39740 1 --2- 172.21.15.83:0/3801250661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d1d270 0x563b30d33f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.624+0000 7facb0c39740 1 -- 172.21.15.83:0/3801250661 >> 172.21.15.83:0/3801250661 conn(0x563b30d19590 msgr2=0x563b30d17e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:37.629 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.624+0000 7facb0c39740 1 -- 172.21.15.83:0/3801250661 shutdown_connections 2024-04-23T14:58:37.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.624+0000 7facb0c39740 1 -- 172.21.15.83:0/3801250661 wait complete. 2024-04-23T14:58:37.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.625+0000 7facb0c39740 1 -- start start 2024-04-23T14:58:37.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.625+0000 7facb0c39740 1 Processor -- start 2024-04-23T14:58:37.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.625+0000 7facb0c39740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d1d270 0x563b30d220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.625+0000 7facb0c39740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 0x563b30d22610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.625+0000 7facb0c39740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d28260 0x563b30d22b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.625+0000 7facb0c39740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faca81b2790 con 0x563b30d1db90 2024-04-23T14:58:37.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.625+0000 7facb0c39740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7faca81b3c00 con 0x563b30d1d270 2024-04-23T14:58:37.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.625+0000 7facb0c39740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7faca81b2e60 con 0x563b30d28260 2024-04-23T14:58:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.625+0000 7facb0c39740 10 client.0 _ll_register_callbacks cb 0x7fac940097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.626+0000 7faca6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 0x563b30d22610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.626+0000 7faca6a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d1d270 0x563b30d220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.626+0000 7faca6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 0x563b30d22610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52846/0 (socket says 172.21.15.83:52846) 2024-04-23T14:58:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.626+0000 7faca6000640 1 -- 172.21.15.83:0/1661831229 learned_addr learned my addr 172.21.15.83:0/1661831229 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.626+0000 7faca7400640 1 --2- 172.21.15.83:0/1661831229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d28260 0x563b30d22b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.626+0000 7faca6000640 1 -- 172.21.15.83:0/1661831229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d28260 msgr2=0x563b30d22b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.631 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.626+0000 7faca6000640 1 --2- 172.21.15.83:0/1661831229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d28260 0x563b30d22b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.626+0000 7faca6000640 1 -- 172.21.15.83:0/1661831229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d1d270 msgr2=0x563b30d220d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.626+0000 7faca6000640 1 --2- 172.21.15.83:0/1661831229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d1d270 0x563b30d220d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.626+0000 7faca6000640 1 -- 172.21.15.83:0/1661831229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b30d9dbe0 con 0x563b30d1db90 2024-04-23T14:58:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.626+0000 7faca6000640 1 --2- 172.21.15.83:0/1661831229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 0x563b30d22610 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fac940bd2c0 tx=0x7fac940315e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c472b5737c1c375 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.626+0000 7fac90c00640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7fac90c00640 1 -- 172.21.15.83:0/1661831229 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940b9860 con 0x563b30d1db90 2024-04-23T14:58:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7fac90c00640 1 -- 172.21.15.83:0/1661831229 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fac940bcc10 con 0x563b30d1db90 2024-04-23T14:58:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7facb0c39740 1 -- 172.21.15.83:0/1661831229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563b30c8a700 con 0x563b30d1db90 2024-04-23T14:58:37.632 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7fac90c00640 1 -- 172.21.15.83:0/1661831229 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fac940bd100 con 0x563b30d1db90 2024-04-23T14:58:37.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7facb0c39740 10 client.17073 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:37.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7facb0c39740 1 -- 172.21.15.83:0/1661831229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b30d9dbe0 con 0x563b30d1db90 2024-04-23T14:58:37.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7fac90c00640 1 -- 172.21.15.83:0/1661831229 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563b30c8a700 con 0x563b30d1db90 2024-04-23T14:58:37.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7fac90c00640 1 -- 172.21.15.83:0/1661831229 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fac940f11a0 con 0x563b30d1db90 2024-04-23T14:58:37.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7facb0c39740 10 client.17073 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:37.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7facb0c39740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:37.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7facb0c39740 20 client.17073 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:37.633 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7facb0c39740 1 --2- 172.21.15.83:0/1661831229 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563b30d9f100 0x563b30da14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7facb0c39740 4 client.17073 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-23T14:58:37.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.627+0000 7facb0c39740 1 -- 172.21.15.83:0/1661831229 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563b30c06980 con 0x563b30d9f100 2024-04-23T14:58:37.634 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.628+0000 7faca6a00640 1 --2- 172.21.15.83:0/1661831229 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563b30d9f100 0x563b30da14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.628+0000 7faca6a00640 1 --2- 172.21.15.83:0/1661831229 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563b30d9f100 0x563b30da14f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fffa277a0086f19e server_cookie=3f182d789c42d3ff in_seq=0 out_seq=0 2024-04-23T14:58:37.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.628+0000 7fac90c00640 10 client.17073 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.629+0000 7fac90c00640 1 -- 172.21.15.83:0/1661831229 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563b30c06980 con 0x563b30d9f100 2024-04-23T14:58:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.629+0000 7fac90c00640 10 client.17073 handle_command_reply: tid=0 2024-04-23T14:58:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.629+0000 7facb0c39740 1 client.17073 shutdown 2024-04-23T14:58:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.629+0000 7facb0c39740 1 -- 172.21.15.83:0/1661831229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 msgr2=0x563b30d22610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.629+0000 7facb0c39740 1 --2- 172.21.15.83:0/1661831229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 0x563b30d22610 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fac940bd2c0 tx=0x7fac940315e0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.630+0000 7facb0c39740 1 -- 172.21.15.83:0/1661831229 shutdown_connections 2024-04-23T14:58:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.630+0000 7facb0c39740 1 --2- 172.21.15.83:0/1661831229 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x563b30d9f100 0x563b30da14f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.630+0000 7facb0c39740 1 --2- 172.21.15.83:0/1661831229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b30d28260 0x563b30d22b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.630+0000 7facb0c39740 1 --2- 172.21.15.83:0/1661831229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563b30d1db90 0x563b30d22610 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.630+0000 7facb0c39740 1 --2- 172.21.15.83:0/1661831229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x563b30d1d270 0x563b30d220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.630+0000 7facb0c39740 1 -- 172.21.15.83:0/1661831229 >> 172.21.15.83:0/1661831229 conn(0x563b30d19590 msgr2=0x563b30d186f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:37.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.630+0000 7fac90c00640 0 client.17073 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:37.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.630+0000 7facb0c39740 1 -- 172.21.15.83:0/1661831229 shutdown_connections 2024-04-23T14:58:37.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.630+0000 7facb0c39740 1 -- 172.21.15.83:0/1661831229 wait complete. 2024-04-23T14:58:37.637 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.630+0000 7facb0c39740 20 client.17073 trim_cache size 0 max 16384 2024-04-23T14:58:37.638 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:58:37.638 INFO:teuthology.orchestra.run.smithi083.stdout: "return_code": 0, 2024-04-23T14:58:37.638 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_tag": "7422c98c-1547-430b-8c92-fab47f63925b", 2024-04-23T14:58:37.638 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": "asynchronous" 2024-04-23T14:58:37.638 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:58:37.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.630+0000 7faca7e00640 1 -- 172.21.15.83:0/1246450235 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fac8c0a3da0 msgr2=0x7fac8c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.630+0000 7faca7e00640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fac8c0a3da0 0x7fac8c0a6260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x563b30d1e830 tx=0x7fac98014040 comp rx=0 tx=0).stop 2024-04-23T14:58:37.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.631+0000 7faca7e00640 1 -- 172.21.15.83:0/1246450235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8074cc0 msgr2=0x7faca819f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.631+0000 7faca7e00640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8074cc0 0x7faca819f390 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fac9402a8f0 tx=0x7fac94030650 comp rx=0 tx=0).stop 2024-04-23T14:58:37.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.631+0000 7faca7e00640 1 -- 172.21.15.83:0/1246450235 shutdown_connections 2024-04-23T14:58:37.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.631+0000 7faca7e00640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fac8c0a3da0 0x7fac8c0a6260 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.631+0000 7faca7e00640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faca8112eb0 0x7faca81a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.631+0000 7faca7e00640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faca8074cc0 0x7faca819f390 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.631+0000 7faca7e00640 1 --2- 172.21.15.83:0/1246450235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faca80743a0 0x7faca819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.631+0000 7faca7e00640 1 -- 172.21.15.83:0/1246450235 >> 172.21.15.83:0/1246450235 conn(0x7faca80fd9d0 msgr2=0x7faca80ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:37.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.631+0000 7faca7e00640 1 -- 172.21.15.83:0/1246450235 shutdown_connections 2024-04-23T14:58:37.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.631+0000 7faca7e00640 1 -- 172.21.15.83:0/1246450235 wait complete. 2024-04-23T14:58:37.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:58:37.649 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:58:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.844+0000 7f6da6800640 1 -- 172.21.15.83:0/312650710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da010e8e0 msgr2=0x7f6da0110cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.844+0000 7f6da6800640 1 --2- 172.21.15.83:0/312650710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da010e8e0 0x7f6da0110cd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009ca0 tx=0x7f6d9802a950 comp rx=0 tx=0).stop 2024-04-23T14:58:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.844+0000 7f6da6800640 1 -- 172.21.15.83:0/312650710 shutdown_connections 2024-04-23T14:58:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.844+0000 7f6da6800640 1 --2- 172.21.15.83:0/312650710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da010e8e0 0x7f6da0110cd0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.844+0000 7f6da6800640 1 --2- 172.21.15.83:0/312650710 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6da0074bf0 0x7f6da010e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.844+0000 7f6da6800640 1 --2- 172.21.15.83:0/312650710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6da00742d0 0x7f6da00746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.844+0000 7f6da6800640 1 -- 172.21.15.83:0/312650710 >> 172.21.15.83:0/312650710 conn(0x7f6da0079450 msgr2=0x7f6da0079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.844+0000 7f6da6800640 1 -- 172.21.15.83:0/312650710 shutdown_connections 2024-04-23T14:58:37.845 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.844+0000 7f6da6800640 1 -- 172.21.15.83:0/312650710 wait complete. 2024-04-23T14:58:37.847 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.845+0000 7f6da6800640 1 Processor -- start 2024-04-23T14:58:37.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.845+0000 7f6da6800640 1 -- start start 2024-04-23T14:58:37.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6da6800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6da00742d0 0x7f6da01a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6da6800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6da0074bf0 0x7f6da01a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6da6800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da010e8e0 0x7f6da019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6da6800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01145c0 con 0x7f6da010e8e0 2024-04-23T14:58:37.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6da6800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6da0114440 con 0x7f6da0074bf0 2024-04-23T14:58:37.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6da6800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6da0114740 con 0x7f6da00742d0 2024-04-23T14:58:37.848 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6d9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da010e8e0 0x7f6da019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6d9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da010e8e0 0x7f6da019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52860/0 (socket says 172.21.15.83:52860) 2024-04-23T14:58:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6d9fe00640 1 -- 172.21.15.83:0/3399277525 learned_addr learned my addr 172.21.15.83:0/3399277525 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6d9f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6da00742d0 0x7f6da01a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6d9ea00640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6da0074bf0 0x7f6da01a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6d9fe00640 1 -- 172.21.15.83:0/3399277525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6da00742d0 msgr2=0x7f6da01a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6d9fe00640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6da00742d0 0x7f6da01a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6d9fe00640 1 -- 172.21.15.83:0/3399277525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6da0074bf0 msgr2=0x7f6da01a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6d9fe00640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6da0074bf0 0x7f6da01a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.849 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6d9fe00640 1 -- 172.21.15.83:0/3399277525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da019b080 con 0x7f6da010e8e0 2024-04-23T14:58:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6d9f400640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6da00742d0 0x7f6da01a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.846+0000 7f6d9ea00640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6da0074bf0 0x7f6da01a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.847+0000 7f6d9fe00640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da010e8e0 0x7f6da019a900 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6d98004af0 tx=0x7f6d9802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.847+0000 7f6d97e00640 1 -- 172.21.15.83:0/3399277525 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98034040 con 0x7f6da010e8e0 2024-04-23T14:58:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.847+0000 7f6d97e00640 1 -- 172.21.15.83:0/3399277525 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d98030920 con 0x7f6da010e8e0 2024-04-23T14:58:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.847+0000 7f6da6800640 1 -- 172.21.15.83:0/3399277525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da019b310 con 0x7f6da010e8e0 2024-04-23T14:58:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.847+0000 7f6d97e00640 1 -- 172.21.15.83:0/3399277525 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d98004640 con 0x7f6da010e8e0 2024-04-23T14:58:37.850 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.847+0000 7f6da6800640 1 -- 172.21.15.83:0/3399277525 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da01a7610 con 0x7f6da010e8e0 2024-04-23T14:58:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.850+0000 7f6da9637740 1 Processor -- start 2024-04-23T14:58:37.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.850+0000 7f6d97e00640 1 -- 172.21.15.83:0/3399277525 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d98037020 con 0x7f6da010e8e0 2024-04-23T14:58:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.851+0000 7f6da9637740 1 -- start start 2024-04-23T14:58:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.851+0000 7f6da9637740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c964860 0x55846c964c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.851+0000 7f6da9637740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c965180 0x55846c96f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.851+0000 7f6da9637740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c96f850 0x55846c971c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.851+0000 7f6da9637740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55846c8ca1f0 con 0x55846c96f850 2024-04-23T14:58:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.851+0000 7f6da9637740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55846c870350 con 0x55846c964860 2024-04-23T14:58:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.851+0000 7f6da9637740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55846c8c61b0 con 0x55846c965180 2024-04-23T14:58:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.851+0000 7f6d9ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c964860 0x55846c964c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.851+0000 7f6d9f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c965180 0x55846c96f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.851+0000 7f6d9f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c965180 0x55846c96f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41334/0 (socket says 172.21.15.83:41334) 2024-04-23T14:58:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.851+0000 7f6d9f400640 1 -- 172.21.15.83:0/2944046247 learned_addr learned my addr 172.21.15.83:0/2944046247 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.851+0000 7f6d9ea00640 1 --2- 172.21.15.83:0/2944046247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c96f850 0x55846c971c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.852+0000 7f6d9f400640 1 -- 172.21.15.83:0/2944046247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c964860 msgr2=0x55846c964c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.852+0000 7f6d9f400640 1 --2- 172.21.15.83:0/2944046247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c964860 0x55846c964c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.852+0000 7f6d9f400640 1 -- 172.21.15.83:0/2944046247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c96f850 msgr2=0x55846c971c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.852+0000 7f6d9f400640 1 --2- 172.21.15.83:0/2944046247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c96f850 0x55846c971c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.855 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.852+0000 7f6d9f400640 1 -- 172.21.15.83:0/2944046247 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846c8c4f50 con 0x55846c965180 2024-04-23T14:58:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.852+0000 7f6d9f400640 1 --2- 172.21.15.83:0/2944046247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c965180 0x55846c96f310 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6d900114f0 tx=0x7f6d900119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abe6b4dabf3ed42d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.852+0000 7f6d97e00640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f6d840a3d20 0x7f6d840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.852+0000 7f6d94c00640 1 -- 172.21.15.83:0/2944046247 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90011250 con 0x55846c965180 2024-04-23T14:58:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.852+0000 7f6d97e00640 1 -- 172.21.15.83:0/3399277525 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f6d980f1440 con 0x7f6da010e8e0 2024-04-23T14:58:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.853+0000 7f6d94c00640 1 -- 172.21.15.83:0/2944046247 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d90005500 con 0x55846c965180 2024-04-23T14:58:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.853+0000 7f6d9ea00640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f6d840a3d20 0x7f6d840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.853+0000 7f6d94c00640 1 -- 172.21.15.83:0/2944046247 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000f820 con 0x55846c965180 2024-04-23T14:58:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.853+0000 7f6da9637740 1 -- 172.21.15.83:0/2944046247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c965180 msgr2=0x55846c96f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.853+0000 7f6d9ea00640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f6d840a3d20 0x7f6d840a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55846c961d10 tx=0x7f6d8c005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.853+0000 7f6da9637740 1 --2- 172.21.15.83:0/2944046247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c965180 0x55846c96f310 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6d900114f0 tx=0x7f6d900119b0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.853+0000 7f6da9637740 1 -- 172.21.15.83:0/2944046247 shutdown_connections 2024-04-23T14:58:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.853+0000 7f6da9637740 1 --2- 172.21.15.83:0/2944046247 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c96f850 0x55846c971c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.853+0000 7f6da9637740 1 --2- 172.21.15.83:0/2944046247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c965180 0x55846c96f310 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.853+0000 7f6da9637740 1 --2- 172.21.15.83:0/2944046247 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c964860 0x55846c964c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.854+0000 7f6da9637740 1 -- 172.21.15.83:0/2944046247 >> 172.21.15.83:0/2944046247 conn(0x55846c960b80 msgr2=0x55846c960f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.854+0000 7f6da9637740 1 -- 172.21.15.83:0/2944046247 shutdown_connections 2024-04-23T14:58:37.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.854+0000 7f6da9637740 1 -- 172.21.15.83:0/2944046247 wait complete. 2024-04-23T14:58:37.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.854+0000 7f6da9637740 1 -- start start 2024-04-23T14:58:37.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.854+0000 7f6da9637740 1 Processor -- start 2024-04-23T14:58:37.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6da9637740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c964860 0x55846c96c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6da9637740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c965180 0x55846c96cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6da9637740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c96f850 0x55846c96d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6da9637740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55846c972640 con 0x55846c965180 2024-04-23T14:58:37.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6da9637740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55846c883690 con 0x55846c96f850 2024-04-23T14:58:37.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6da9637740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55846c9727c0 con 0x55846c964860 2024-04-23T14:58:37.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6da9637740 10 client.0 _ll_register_callbacks cb 0x55846c8bf650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:37.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6d9ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c965180 0x55846c96cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6d9f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c964860 0x55846c96c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6d9f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c964860 0x55846c96c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:41336/0 (socket says 172.21.15.83:41336) 2024-04-23T14:58:37.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6d9f400640 1 -- 172.21.15.83:0/211493052 learned_addr learned my addr 172.21.15.83:0/211493052 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:37.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6d9ea00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c965180 0x55846c96cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52876/0 (socket says 172.21.15.83:52876) 2024-04-23T14:58:37.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.855+0000 7f6d9fe00640 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c96f850 0x55846c96d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.856+0000 7f6d9ea00640 1 -- 172.21.15.83:0/211493052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c964860 msgr2=0x55846c96c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.856+0000 7f6d9ea00640 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c964860 0x55846c96c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.856+0000 7f6d9ea00640 1 -- 172.21.15.83:0/211493052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c96f850 msgr2=0x55846c96d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.856+0000 7f6d9ea00640 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c96f850 0x55846c96d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.856+0000 7f6d9ea00640 1 -- 172.21.15.83:0/211493052 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846c882920 con 0x55846c965180 2024-04-23T14:58:37.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.856+0000 7f6d9fe00640 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c96f850 0x55846c96d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.856+0000 7f6d9f400640 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c964860 0x55846c96c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:58:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.856+0000 7f6d9ea00640 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c965180 0x55846c96cce0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009710 tx=0x7f6d8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208c681941de9ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.856+0000 7f6d89600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.856+0000 7f6d89600640 1 -- 172.21.15.83:0/211493052 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8c005020 con 0x55846c965180 2024-04-23T14:58:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.856+0000 7f6d89600640 1 -- 172.21.15.83:0/211493052 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f6d8c01a050 con 0x55846c965180 2024-04-23T14:58:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.856+0000 7f6da9637740 1 -- 172.21.15.83:0/211493052 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d90005500 con 0x55846c965180 2024-04-23T14:58:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.857+0000 7f6d89600640 1 -- 172.21.15.83:0/211493052 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d8c0051c0 con 0x55846c965180 2024-04-23T14:58:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.857+0000 7f6da9637740 10 client.17085 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:37.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.857+0000 7f6da9637740 1 -- 172.21.15.83:0/211493052 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55846c6a67d0 con 0x55846c965180 2024-04-23T14:58:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.857+0000 7f6d89600640 1 -- 172.21.15.83:0/211493052 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d90005500 con 0x55846c965180 2024-04-23T14:58:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.857+0000 7f6d89600640 1 -- 172.21.15.83:0/211493052 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f6d8c022570 con 0x55846c965180 2024-04-23T14:58:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.857+0000 7f6da9637740 10 client.17085 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.857+0000 7f6da9637740 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.857+0000 7f6da9637740 20 client.17085 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.857+0000 7f6da9637740 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55846c97fb80 0x55846c981ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.857+0000 7f6da9637740 4 client.17085 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:58:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.857+0000 7f6da9637740 1 -- 172.21.15.83:0/211493052 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55846c8d7c10 con 0x55846c97fb80 2024-04-23T14:58:37.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.858+0000 7f6d9f400640 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55846c97fb80 0x55846c981ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.858+0000 7f6d9f400640 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55846c97fb80 0x55846c981ff0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1d7305285fd007 server_cookie=2bef304918da0077 in_seq=0 out_seq=0 2024-04-23T14:58:37.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.858+0000 7f6d89600640 10 client.17085 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:37.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.859+0000 7f6d89600640 1 -- 172.21.15.83:0/211493052 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55846c8d7c10 con 0x55846c97fb80 2024-04-23T14:58:37.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.859+0000 7f6d89600640 10 client.17085 handle_command_reply: tid=0 2024-04-23T14:58:37.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.859+0000 7f6da9637740 1 client.17085 shutdown 2024-04-23T14:58:37.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.860+0000 7f6da9637740 1 -- 172.21.15.83:0/211493052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c965180 msgr2=0x55846c96cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.860+0000 7f6da9637740 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c965180 0x55846c96cce0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009710 tx=0x7f6d8c004730 comp rx=0 tx=0).stop 2024-04-23T14:58:37.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.860+0000 7f6da9637740 1 -- 172.21.15.83:0/211493052 shutdown_connections 2024-04-23T14:58:37.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.860+0000 7f6da9637740 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55846c97fb80 0x55846c981ff0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.860+0000 7f6da9637740 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c96f850 0x55846c96d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.860+0000 7f6da9637740 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c965180 0x55846c96cce0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.860+0000 7f6d89600640 0 client.17085 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:37.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.860+0000 7f6da9637740 1 --2- 172.21.15.83:0/211493052 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c964860 0x55846c96c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.860+0000 7f6da9637740 1 -- 172.21.15.83:0/211493052 >> 172.21.15.83:0/211493052 conn(0x55846c960b80 msgr2=0x55846c966980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:37.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.860+0000 7f6da9637740 1 -- 172.21.15.83:0/211493052 shutdown_connections 2024-04-23T14:58:37.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.860+0000 7f6da9637740 1 -- 172.21.15.83:0/211493052 wait complete. 2024-04-23T14:58:37.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.860+0000 7f6da9637740 20 client.17085 trim_cache size 0 max 16384 2024-04-23T14:58:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.879+0000 7f6da9637740 1 Processor -- start 2024-04-23T14:58:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.879+0000 7f6da9637740 1 -- start start 2024-04-23T14:58:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.879+0000 7f6da9637740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c964860 0x55846c97b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.879+0000 7f6da9637740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c965180 0x55846c9878c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.879+0000 7f6da9637740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c96f850 0x55846c987e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.879+0000 7f6da9637740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01b2840 con 0x55846c965180 2024-04-23T14:58:37.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.879+0000 7f6da9637740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01b0110 con 0x55846c964860 2024-04-23T14:58:37.880 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.879+0000 7f6da9637740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01adf80 con 0x55846c96f850 2024-04-23T14:58:37.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.880+0000 7f6d9f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c964860 0x55846c97b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.880+0000 7f6d9fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c96f850 0x55846c987e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.880+0000 7f6d9f400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c964860 0x55846c97b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:47626/0 (socket says 172.21.15.83:47626) 2024-04-23T14:58:37.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.880+0000 7f6d9f400640 1 -- 172.21.15.83:0/353989035 learned_addr learned my addr 172.21.15.83:0/353989035 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:37.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.880+0000 7f6d9ea00640 1 --2- 172.21.15.83:0/353989035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c965180 0x55846c9878c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.880+0000 7f6d9f400640 1 -- 172.21.15.83:0/353989035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c96f850 msgr2=0x55846c987e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.880+0000 7f6d9f400640 1 --2- 172.21.15.83:0/353989035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c96f850 0x55846c987e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.880+0000 7f6d9f400640 1 -- 172.21.15.83:0/353989035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c965180 msgr2=0x55846c9878c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.880+0000 7f6d9f400640 1 --2- 172.21.15.83:0/353989035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c965180 0x55846c9878c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.880+0000 7f6d9f400640 1 -- 172.21.15.83:0/353989035 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846c5dc3f0 con 0x55846c964860 2024-04-23T14:58:37.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.880+0000 7f6d9f400640 1 --2- 172.21.15.83:0/353989035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c964860 0x55846c97b500 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d90004c60 tx=0x7f6d90004d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80e01a1dee62f68 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:37.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.881+0000 7f6d8be00640 1 -- 172.21.15.83:0/353989035 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d900110d0 con 0x55846c964860 2024-04-23T14:58:37.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.881+0000 7f6d8be00640 1 -- 172.21.15.83:0/353989035 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d90011270 con 0x55846c964860 2024-04-23T14:58:37.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.881+0000 7f6d8be00640 1 -- 172.21.15.83:0/353989035 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d900052d0 con 0x55846c964860 2024-04-23T14:58:37.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.881+0000 7f6da9637740 1 -- 172.21.15.83:0/353989035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c964860 msgr2=0x55846c97b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.881+0000 7f6da9637740 1 --2- 172.21.15.83:0/353989035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c964860 0x55846c97b500 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d90004c60 tx=0x7f6d90004d20 comp rx=0 tx=0).stop 2024-04-23T14:58:37.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.881+0000 7f6da9637740 1 -- 172.21.15.83:0/353989035 shutdown_connections 2024-04-23T14:58:37.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.881+0000 7f6da9637740 1 --2- 172.21.15.83:0/353989035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c96f850 0x55846c987e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.881+0000 7f6da9637740 1 --2- 172.21.15.83:0/353989035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c965180 0x55846c9878c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.881+0000 7f6da9637740 1 --2- 172.21.15.83:0/353989035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c964860 0x55846c97b500 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.881+0000 7f6da9637740 1 -- 172.21.15.83:0/353989035 >> 172.21.15.83:0/353989035 conn(0x55846c960b80 msgr2=0x55846c96b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:37.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.881+0000 7f6da9637740 1 -- 172.21.15.83:0/353989035 shutdown_connections 2024-04-23T14:58:37.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.881+0000 7f6da9637740 1 -- 172.21.15.83:0/353989035 wait complete. 2024-04-23T14:58:37.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.882+0000 7f6da9637740 1 -- start start 2024-04-23T14:58:37.884 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.882+0000 7f6da9637740 1 Processor -- start 2024-04-23T14:58:37.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6da9637740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c964860 0x55846c9729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6da9637740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c965180 0x55846c972f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6da9637740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c96f850 0x55846c9e4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6da9637740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01ae5d0 con 0x55846c96f850 2024-04-23T14:58:37.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6da9637740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01afa40 con 0x55846c965180 2024-04-23T14:58:37.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6da9637740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f6da01aeca0 con 0x55846c964860 2024-04-23T14:58:37.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6da9637740 10 client.0 _ll_register_callbacks cb 0x7f6d8c011e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:58:37.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6d9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c96f850 0x55846c9e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6d9f400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c964860 0x55846c9729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6d9fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c96f850 0x55846c9e4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52898/0 (socket says 172.21.15.83:52898) 2024-04-23T14:58:37.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6d9fe00640 1 -- 172.21.15.83:0/2719163959 learned_addr learned my addr 172.21.15.83:0/2719163959 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:37.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6d9ea00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c965180 0x55846c972f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6d9fe00640 1 -- 172.21.15.83:0/2719163959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c964860 msgr2=0x55846c9729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6d9fe00640 1 --2- 172.21.15.83:0/2719163959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c964860 0x55846c9729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6d9fe00640 1 -- 172.21.15.83:0/2719163959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c965180 msgr2=0x55846c972f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:58:37.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6d9fe00640 1 --2- 172.21.15.83:0/2719163959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c965180 0x55846c972f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.883+0000 7f6d9fe00640 1 -- 172.21.15.83:0/2719163959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846c59c0f0 con 0x55846c96f850 2024-04-23T14:58:37.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.884+0000 7f6d9fe00640 1 --2- 172.21.15.83:0/2719163959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c96f850 0x55846c9e4120 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6d980b9e50 tx=0x7f6d980bd310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6db220addc5eee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:37.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.884+0000 7f6d89600640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-04-23T14:58:37.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.884+0000 7f6d89600640 1 -- 172.21.15.83:0/2719163959 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d980db060 con 0x55846c96f850 2024-04-23T14:58:37.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.884+0000 7f6da9637740 1 -- 172.21.15.83:0/2719163959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d90011270 con 0x55846c96f850 2024-04-23T14:58:37.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.884+0000 7f6d89600640 1 -- 172.21.15.83:0/2719163959 <== mon.0 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f6d980c5c60 con 0x55846c96f850 2024-04-23T14:58:37.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.884+0000 7f6d89600640 1 -- 172.21.15.83:0/2719163959 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6d980bdb80 con 0x55846c96f850 2024-04-23T14:58:37.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.884+0000 7f6da9637740 10 client.17091 fetch_fsmap learned FSMap version 29 2024-04-23T14:58:37.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.884+0000 7f6da9637740 1 -- 172.21.15.83:0/2719163959 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55846c88f810 con 0x55846c96f850 2024-04-23T14:58:37.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.884+0000 7f6d89600640 1 -- 172.21.15.83:0/2719163959 <== mon.0 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d90011270 con 0x55846c96f850 2024-04-23T14:58:37.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.884+0000 7f6d89600640 1 -- 172.21.15.83:0/2719163959 <== mon.0 v2:172.21.15.83:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f6d9809e030 con 0x55846c96f850 2024-04-23T14:58:37.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.885+0000 7f6da9637740 10 client.17091 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:58:37.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.885+0000 7f6da9637740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:58:37.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.885+0000 7f6da9637740 20 client.17091 populate_metadata read hostname 'smithi083' 2024-04-23T14:58:37.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.885+0000 7f6da9637740 1 --2- 172.21.15.83:0/2719163959 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55846c9e5b70 0x55846c9e7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:37.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.885+0000 7f6da9637740 4 client.17091 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:58:37.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.885+0000 7f6da9637740 1 -- 172.21.15.83:0/2719163959 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55846c8d1a80 con 0x55846c9e5b70 2024-04-23T14:58:37.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.885+0000 7f6d9f400640 1 --2- 172.21.15.83:0/2719163959 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55846c9e5b70 0x55846c9e7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:37.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6d9f400640 1 --2- 172.21.15.83:0/2719163959 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55846c9e5b70 0x55846c9e7f60 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a9746d9f8fd5c20 server_cookie=584a1639eecb7bed in_seq=0 out_seq=0 2024-04-23T14:58:37.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6d89600640 10 client.17091 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:37.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6d89600640 1 -- 172.21.15.83:0/2719163959 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55846c8d1a80 con 0x55846c9e5b70 2024-04-23T14:58:37.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6d89600640 10 client.17091 handle_command_reply: tid=0 2024-04-23T14:58:37.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6da9637740 1 client.17091 shutdown 2024-04-23T14:58:37.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6da9637740 1 -- 172.21.15.83:0/2719163959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c96f850 msgr2=0x55846c9e4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6da9637740 1 --2- 172.21.15.83:0/2719163959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c96f850 0x55846c9e4120 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6d980b9e50 tx=0x7f6d980bd310 comp rx=0 tx=0).stop 2024-04-23T14:58:37.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6da9637740 1 -- 172.21.15.83:0/2719163959 shutdown_connections 2024-04-23T14:58:37.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6da9637740 1 --2- 172.21.15.83:0/2719163959 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x55846c9e5b70 0x55846c9e7f60 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6da9637740 1 --2- 172.21.15.83:0/2719163959 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55846c96f850 0x55846c9e4120 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6da9637740 1 --2- 172.21.15.83:0/2719163959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55846c965180 0x55846c972f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6da9637740 1 --2- 172.21.15.83:0/2719163959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55846c964860 0x55846c9729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6da9637740 1 -- 172.21.15.83:0/2719163959 >> 172.21.15.83:0/2719163959 conn(0x55846c960b80 msgr2=0x55846c96c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:37.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6d89600640 0 client.17091 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:58:37.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6da9637740 1 -- 172.21.15.83:0/2719163959 shutdown_connections 2024-04-23T14:58:37.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.886+0000 7f6da9637740 1 -- 172.21.15.83:0/2719163959 wait complete. 2024-04-23T14:58:37.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.887+0000 7f6da9637740 20 client.17091 trim_cache size 0 max 16384 2024-04-23T14:58:37.894 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:58:37.894 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-23T14:58:37.895 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": { 2024-04-23T14:58:37.895 INFO:teuthology.orchestra.run.smithi083.stdout: "7422c98c-1547-430b-8c92-fab47f63925b": { 2024-04-23T14:58:37.895 INFO:teuthology.orchestra.run.smithi083.stdout: "path": "/", 2024-04-23T14:58:37.895 INFO:teuthology.orchestra.run.smithi083.stdout: "tag": "7422c98c-1547-430b-8c92-fab47f63925b", 2024-04-23T14:58:37.895 INFO:teuthology.orchestra.run.smithi083.stdout: "options": "recursive,force" 2024-04-23T14:58:37.895 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:58:37.895 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:58:37.895 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:58:37.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.887+0000 7f6da6800640 1 -- 172.21.15.83:0/3399277525 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f6d840a3d20 msgr2=0x7f6d840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.887+0000 7f6da6800640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f6d840a3d20 0x7f6d840a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55846c961d10 tx=0x7f6d8c005f90 comp rx=0 tx=0).stop 2024-04-23T14:58:37.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.887+0000 7f6da6800640 1 -- 172.21.15.83:0/3399277525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da010e8e0 msgr2=0x7f6da019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:37.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.887+0000 7f6da6800640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da010e8e0 0x7f6da019a900 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6d98004af0 tx=0x7f6d9802f3a0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.887+0000 7f6da6800640 1 -- 172.21.15.83:0/3399277525 shutdown_connections 2024-04-23T14:58:37.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.887+0000 7f6da6800640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f6d840a3d20 0x7f6d840a61e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.887+0000 7f6da6800640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6da010e8e0 0x7f6da019a900 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.887+0000 7f6da6800640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6da0074bf0 0x7f6da01a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.887+0000 7f6da6800640 1 --2- 172.21.15.83:0/3399277525 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6da00742d0 0x7f6da01a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:37.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.887+0000 7f6da6800640 1 -- 172.21.15.83:0/3399277525 >> 172.21.15.83:0/3399277525 conn(0x7f6da0079450 msgr2=0x7f6da00fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:37.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.888+0000 7f6da6800640 1 -- 172.21.15.83:0/3399277525 shutdown_connections 2024-04-23T14:58:37.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:37.888+0000 7f6da6800640 1 -- 172.21.15.83:0/3399277525 wait complete. 2024-04-23T14:58:37.906 INFO:tasks.cephfs.filesystem:scrub status for tag:7422c98c-1547-430b-8c92-fab47f63925b - {'path': '/', 'tag': '7422c98c-1547-430b-8c92-fab47f63925b', 'options': 'recursive,force'} 2024-04-23T14:58:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub queued for path: / 2024-04-23T14:58:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:58:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:58:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:58:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:58:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:58:38.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: active paths [/] 2024-04-23T14:58:38.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:37 smithi083 ceph-mon[27867]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub queued for path: / 2024-04-23T14:58:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:58:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:58:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:58:38.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:58:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:58:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: active paths [/] 2024-04-23T14:58:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:37 smithi173 ceph-mon[32443]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub queued for path: / 2024-04-23T14:58:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:58:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:58:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:58:38.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:58:38.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:58:38.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: active paths [/] 2024-04-23T14:58:38.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:38.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:37 smithi203 ceph-mon[32800]: scrub summary: idle+waiting paths [/] 2024-04-23T14:58:39.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:38 smithi083 ceph-mon[27867]: pgmap v317: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:58:39.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:38 smithi173 ceph-mon[32443]: pgmap v317: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:58:39.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:38 smithi203 ceph-mon[32800]: pgmap v317: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-23T14:58:39.621 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-23T14:58:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.823+0000 7f0d21600640 1 -- 172.21.15.83:0/145947829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1c10e690 msgr2=0x7f0d1c10ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.823+0000 7f0d21600640 1 --2- 172.21.15.83:0/145947829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1c10e690 0x7f0d1c10ea70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009930 tx=0x7f0d0c02a420 comp rx=0 tx=0).stop 2024-04-23T14:58:39.824 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.823+0000 7f0d21600640 1 -- 172.21.15.83:0/145947829 shutdown_connections 2024-04-23T14:58:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.823+0000 7f0d21600640 1 --2- 172.21.15.83:0/145947829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c108fb0 0x7f0d1c1154c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.823+0000 7f0d21600640 1 --2- 172.21.15.83:0/145947829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d1c1085f0 0x7f0d1c108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.823+0000 7f0d21600640 1 --2- 172.21.15.83:0/145947829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1c10e690 0x7f0d1c10ea70 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.823+0000 7f0d21600640 1 -- 172.21.15.83:0/145947829 >> 172.21.15.83:0/145947829 conn(0x7f0d1c076590 msgr2=0x7f0d1c0789b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.823+0000 7f0d21600640 1 -- 172.21.15.83:0/145947829 shutdown_connections 2024-04-23T14:58:39.825 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.824+0000 7f0d21600640 1 -- 172.21.15.83:0/145947829 wait complete. 2024-04-23T14:58:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d21600640 1 Processor -- start 2024-04-23T14:58:39.827 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d21600640 1 -- start start 2024-04-23T14:58:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d21600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1c1085f0 0x7f0d1c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d21600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c108fb0 0x7f0d1c19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d21600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d1c10e690 0x7f0d1c1a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d21600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c118a10 con 0x7f0d1c1085f0 2024-04-23T14:58:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d21600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c118890 con 0x7f0d1c10e690 2024-04-23T14:58:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d21600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c118b90 con 0x7f0d1c108fb0 2024-04-23T14:58:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d1aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1c1085f0 0x7f0d1c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:39.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d1aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1c1085f0 0x7f0d1c19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52910/0 (socket says 172.21.15.83:52910) 2024-04-23T14:58:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d1aa00640 1 -- 172.21.15.83:0/1282260779 learned_addr learned my addr 172.21.15.83:0/1282260779 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d1b400640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d1c10e690 0x7f0d1c1a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.825+0000 7f0d1a000640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c108fb0 0x7f0d1c19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.826+0000 7f0d1aa00640 1 -- 172.21.15.83:0/1282260779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c108fb0 msgr2=0x7f0d1c19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.826+0000 7f0d1aa00640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c108fb0 0x7f0d1c19f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.826+0000 7f0d1aa00640 1 -- 172.21.15.83:0/1282260779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d1c10e690 msgr2=0x7f0d1c1a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.826+0000 7f0d1aa00640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d1c10e690 0x7f0d1c1a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:39.829 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.826+0000 7f0d1aa00640 1 -- 172.21.15.83:0/1282260779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c1a3e20 con 0x7f0d1c1085f0 2024-04-23T14:58:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.826+0000 7f0d1aa00640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1c1085f0 0x7f0d1c19ee30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005940 tx=0x7f0d0c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.826+0000 7f0d13400640 1 -- 172.21.15.83:0/1282260779 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c03f030 con 0x7f0d1c1085f0 2024-04-23T14:58:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.827+0000 7f0d13400640 1 -- 172.21.15.83:0/1282260779 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d0c0044c0 con 0x7f0d1c1085f0 2024-04-23T14:58:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.827+0000 7f0d21600640 1 -- 172.21.15.83:0/1282260779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1c1a40b0 con 0x7f0d1c1085f0 2024-04-23T14:58:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.827+0000 7f0d13400640 1 -- 172.21.15.83:0/1282260779 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c035070 con 0x7f0d1c1085f0 2024-04-23T14:58:39.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.827+0000 7f0d21600640 1 -- 172.21.15.83:0/1282260779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1c1ab9a0 con 0x7f0d1c1085f0 2024-04-23T14:58:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.828+0000 7f0d21600640 1 -- 172.21.15.83:0/1282260779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1c04fd90 con 0x7f0d1c1085f0 2024-04-23T14:58:39.831 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.830+0000 7f0d13400640 1 -- 172.21.15.83:0/1282260779 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0d0c042030 con 0x7f0d1c1085f0 2024-04-23T14:58:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.831+0000 7f0d13400640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0d000a3dc0 0x7f0d000a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.831+0000 7f0d13400640 1 -- 172.21.15.83:0/1282260779 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f0d0c0f0c00 con 0x7f0d1c1085f0 2024-04-23T14:58:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.831+0000 7f0d1a000640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0d000a3dc0 0x7f0d000a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:39.832 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.832+0000 7f0d1a000640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0d000a3dc0 0x7f0d000a6280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c10dca0 tx=0x7f0d08005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:39.835 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:39.836+0000 7f0d13400640 1 -- 172.21.15.83:0/1282260779 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d0c0bcbe0 con 0x7f0d1c1085f0 2024-04-23T14:58:40.062 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.062+0000 7f0d21600640 1 -- 172.21.15.83:0/1282260779 --> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f0d1c062440 con 0x7f0d000a3dc0 2024-04-23T14:58:40.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.076+0000 7f0d13400640 1 -- 172.21.15.83:0/1282260779 <== mgr.15114 v2:172.21.15.83:6800/1667121447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f0d1c062440 con 0x7f0d000a3dc0 2024-04-23T14:58:40.076 INFO:teuthology.orchestra.run.smithi083.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-23T00:00:00", "created": "2024-04-23T14:48:37", "first": "2024-04-23T14:49:01", "last": "2024-04-23T14:58:00", "last_pruned": "2024-04-23T14:58:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-23T14:58:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.079+0000 7f0d21600640 1 -- 172.21.15.83:0/1282260779 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0d000a3dc0 msgr2=0x7f0d000a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.079+0000 7f0d21600640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0d000a3dc0 0x7f0d000a6280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c10dca0 tx=0x7f0d08005f90 comp rx=0 tx=0).stop 2024-04-23T14:58:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.079+0000 7f0d21600640 1 -- 172.21.15.83:0/1282260779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1c1085f0 msgr2=0x7f0d1c19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.079+0000 7f0d21600640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1c1085f0 0x7f0d1c19ee30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c005940 tx=0x7f0d0c0040c0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.079+0000 7f0d21600640 1 -- 172.21.15.83:0/1282260779 shutdown_connections 2024-04-23T14:58:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.079+0000 7f0d21600640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d1c10e690 0x7f0d1c1a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.079 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.079+0000 7f0d21600640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c108fb0 0x7f0d1c19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.079+0000 7f0d21600640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f0d000a3dc0 0x7f0d000a6280 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.079+0000 7f0d21600640 1 --2- 172.21.15.83:0/1282260779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1c1085f0 0x7f0d1c19ee30 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.079+0000 7f0d21600640 1 -- 172.21.15.83:0/1282260779 >> 172.21.15.83:0/1282260779 conn(0x7f0d1c076590 msgr2=0x7f0d1c077b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.079+0000 7f0d21600640 1 -- 172.21.15.83:0/1282260779 shutdown_connections 2024-04-23T14:58:40.080 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.079+0000 7f0d21600640 1 -- 172.21.15.83:0/1282260779 wait complete. 2024-04-23T14:58:40.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-23T14:58:40.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.299+0000 7f6283e00640 1 -- 172.21.15.83:0/1261078442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284074d50 msgr2=0x7f628410e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:40.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.299+0000 7f6283e00640 1 --2- 172.21.15.83:0/1261078442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284074d50 0x7f628410e100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f626c009900 tx=0x7f626c02a480 comp rx=0 tx=0).stop 2024-04-23T14:58:40.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.299+0000 7f6283e00640 1 -- 172.21.15.83:0/1261078442 shutdown_connections 2024-04-23T14:58:40.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.299+0000 7f6283e00640 1 --2- 172.21.15.83:0/1261078442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f628410e640 0x7f6284110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.299+0000 7f6283e00640 1 --2- 172.21.15.83:0/1261078442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284074d50 0x7f628410e100 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.299+0000 7f6283e00640 1 --2- 172.21.15.83:0/1261078442 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6284074430 0x7f6284074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.300 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.299+0000 7f6283e00640 1 -- 172.21.15.83:0/1261078442 >> 172.21.15.83:0/1261078442 conn(0x7f62840fd950 msgr2=0x7f62840ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:40.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.300+0000 7f6283e00640 1 -- 172.21.15.83:0/1261078442 shutdown_connections 2024-04-23T14:58:40.301 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.300+0000 7f6283e00640 1 -- 172.21.15.83:0/1261078442 wait complete. 2024-04-23T14:58:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.301+0000 7f6283e00640 1 Processor -- start 2024-04-23T14:58:40.303 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.301+0000 7f6283e00640 1 -- start start 2024-04-23T14:58:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.301+0000 7f6283e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284074430 0x7f62841a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.301+0000 7f6283e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6284074d50 0x7f62841a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.301+0000 7f6283e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f628410e640 0x7f628419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.301+0000 7f6283e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6284114420 con 0x7f6284074430 2024-04-23T14:58:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.301+0000 7f6283e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f62841142a0 con 0x7f6284074d50 2024-04-23T14:58:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.301+0000 7f6283e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f62841145a0 con 0x7f628410e640 2024-04-23T14:58:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.301+0000 7f6282a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284074430 0x7f62841a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:40.304 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.301+0000 7f6282a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284074430 0x7f62841a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:52928/0 (socket says 172.21.15.83:52928) 2024-04-23T14:58:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.301+0000 7f6282a00640 1 -- 172.21.15.83:0/485681064 learned_addr learned my addr 172.21.15.83:0/485681064 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:58:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f6283400640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f628410e640 0x7f628419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f627be00640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6284074d50 0x7f62841a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f6282a00640 1 -- 172.21.15.83:0/485681064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f628410e640 msgr2=0x7f628419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f6282a00640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f628410e640 0x7f628419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f6282a00640 1 -- 172.21.15.83:0/485681064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6284074d50 msgr2=0x7f62841a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f6282a00640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6284074d50 0x7f62841a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f6282a00640 1 -- 172.21.15.83:0/485681064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628419afc0 con 0x7f6284074430 2024-04-23T14:58:40.305 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f6283400640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f628410e640 0x7f628419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:58:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f6282a00640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284074430 0x7f62841a0640 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f627400ea60 tx=0x7f627400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f627b400640 1 -- 172.21.15.83:0/485681064 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627400cde0 con 0x7f6284074430 2024-04-23T14:58:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f627b400640 1 -- 172.21.15.83:0/485681064 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62740035f0 con 0x7f6284074430 2024-04-23T14:58:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f627b400640 1 -- 172.21.15.83:0/485681064 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274010670 con 0x7f6284074430 2024-04-23T14:58:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.302+0000 7f6283e00640 1 -- 172.21.15.83:0/485681064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628419b2b0 con 0x7f6284074430 2024-04-23T14:58:40.306 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.303+0000 7f6283e00640 1 -- 172.21.15.83:0/485681064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62841a75c0 con 0x7f6284074430 2024-04-23T14:58:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.304+0000 7f6283e00640 1 -- 172.21.15.83:0/485681064 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6284075ae0 con 0x7f6284074430 2024-04-23T14:58:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.306+0000 7f627b400640 1 -- 172.21.15.83:0/485681064 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62740037a0 con 0x7f6284074430 2024-04-23T14:58:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.307+0000 7f627b400640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f62680a3d90 0x7f62680a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:58:40.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.308+0000 7f627b400640 1 -- 172.21.15.83:0/485681064 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f62740d0c10 con 0x7f6284074430 2024-04-23T14:58:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.308+0000 7f627be00640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f62680a3d90 0x7f62680a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:58:40.308 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.309+0000 7f627be00640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f62680a3d90 0x7f62680a6250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f626c000c00 tx=0x7f626c005a20 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:58:40.311 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.312+0000 7f627b400640 1 -- 172.21.15.83:0/485681064 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f627409d240 con 0x7f6284074430 2024-04-23T14:58:40.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.536+0000 7f6283e00640 1 -- 172.21.15.83:0/485681064 --> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f628419b550 con 0x7f62680a3d90 2024-04-23T14:58:40.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.550+0000 7f627b400640 1 -- 172.21.15.83:0/485681064 <== mgr.15114 v2:172.21.15.83:6800/1667121447 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f628419b550 con 0x7f62680a3d90 2024-04-23T14:58:40.550 INFO:teuthology.orchestra.run.smithi083.stdout:/ 1m 6m3h 2024-04-23T14:58:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.553+0000 7f6283e00640 1 -- 172.21.15.83:0/485681064 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f62680a3d90 msgr2=0x7f62680a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.553+0000 7f6283e00640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f62680a3d90 0x7f62680a6250 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f626c000c00 tx=0x7f626c005a20 comp rx=0 tx=0).stop 2024-04-23T14:58:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.553+0000 7f6283e00640 1 -- 172.21.15.83:0/485681064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284074430 msgr2=0x7f62841a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:58:40.553 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.553+0000 7f6283e00640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284074430 0x7f62841a0640 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f627400ea60 tx=0x7f627400ef20 comp rx=0 tx=0).stop 2024-04-23T14:58:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.553+0000 7f6283e00640 1 -- 172.21.15.83:0/485681064 shutdown_connections 2024-04-23T14:58:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.553+0000 7f6283e00640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f628410e640 0x7f628419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.553+0000 7f6283e00640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6284074d50 0x7f62841a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.553+0000 7f6283e00640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f62680a3d90 0x7f62680a6250 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.553+0000 7f6283e00640 1 --2- 172.21.15.83:0/485681064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6284074430 0x7f62841a0640 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:58:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.553+0000 7f6283e00640 1 -- 172.21.15.83:0/485681064 >> 172.21.15.83:0/485681064 conn(0x7f62840fd950 msgr2=0x7f62840fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:58:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.553+0000 7f6283e00640 1 -- 172.21.15.83:0/485681064 shutdown_connections 2024-04-23T14:58:40.554 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:58:40.553+0000 7f6283e00640 1 -- 172.21.15.83:0/485681064 wait complete. 2024-04-23T14:58:40.572 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-23T14:58:40.583 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-23T14:58:41.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:40 smithi083 ceph-mon[27867]: pgmap v318: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:41.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:40 smithi083 ceph-mon[27867]: from='client.17097 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:58:41.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:40 smithi083 ceph-mon[27867]: from='client.17103 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:58:41.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:40 smithi173 ceph-mon[32443]: pgmap v318: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:41.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:40 smithi173 ceph-mon[32443]: from='client.17097 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:58:41.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:40 smithi173 ceph-mon[32443]: from='client.17103 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:58:41.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:40 smithi203 ceph-mon[32800]: pgmap v318: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:58:41.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:40 smithi203 ceph-mon[32800]: from='client.17097 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:58:41.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:40 smithi203 ceph-mon[32800]: from='client.17103 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T14:58:43.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:42 smithi083 ceph-mon[27867]: pgmap v319: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:58:43.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:42 smithi173 ceph-mon[32443]: pgmap v319: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:58:43.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:42 smithi203 ceph-mon[32800]: pgmap v319: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-04-23T14:58:45.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:44 smithi083 ceph-mon[27867]: pgmap v320: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T14:58:45.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:44 smithi173 ceph-mon[32443]: pgmap v320: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T14:58:45.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:44 smithi203 ceph-mon[32800]: pgmap v320: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T14:58:47.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:46 smithi083 ceph-mon[27867]: pgmap v321: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T14:58:47.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:46 smithi173 ceph-mon[32443]: pgmap v321: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T14:58:47.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:46 smithi203 ceph-mon[32800]: pgmap v321: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T14:58:49.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:49 smithi083 ceph-mon[27867]: pgmap v322: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T14:58:49.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:49 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:58:49.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:49 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:58:49.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:49 smithi083 ceph-mon[27867]: scrub summary: idle 2024-04-23T14:58:49.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:49 smithi173 ceph-mon[32443]: pgmap v322: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T14:58:49.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:49 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:58:49.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:49 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:58:49.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:49 smithi173 ceph-mon[32443]: scrub summary: idle 2024-04-23T14:58:49.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:49 smithi203 ceph-mon[32800]: pgmap v322: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T14:58:49.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:49 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:58:49.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:49 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:58:49.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:49 smithi203 ceph-mon[32800]: scrub summary: idle 2024-04-23T14:58:51.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:51 smithi083 ceph-mon[27867]: pgmap v323: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-23T14:58:51.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:51 smithi173 ceph-mon[32443]: pgmap v323: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-23T14:58:51.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:51 smithi203 ceph-mon[32800]: pgmap v323: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-23T14:58:53.663 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:53 smithi083 ceph-mon[27867]: pgmap v324: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T14:58:53.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:53 smithi203 ceph-mon[32800]: pgmap v324: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T14:58:53.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:53 smithi173 ceph-mon[32443]: pgmap v324: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T14:58:54.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:54 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:58:54.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:54 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:54.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:54 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:54.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:54 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:54.722 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:54 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:54.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:54 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:58:54.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:54 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:54 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:54 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:54.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:54 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:54.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:54 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T14:58:54.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:54 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:54.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:54 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:54.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:54 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:54.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:54 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:55.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:55 smithi173 ceph-mon[32443]: pgmap v325: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T14:58:55.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:55 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:55.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:55 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:55 smithi203 ceph-mon[32800]: pgmap v325: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T14:58:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:55 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:55.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:55 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:55.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:55 smithi083 ceph-mon[27867]: pgmap v325: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T14:58:55.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:55 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:55.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:55 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: pgmap v326: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:58:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.258 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.259 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.260 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:57 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: pgmap v326: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:58:57.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.328 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:57 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: pgmap v326: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:58:57.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:57.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.473 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:57.474 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:57 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:58:58.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:58 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-23T14:58:58.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:58 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-04-23T14:58:58.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:58 smithi173 ceph-mon[32443]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-23T14:58:58.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:58 smithi173 ceph-mon[32443]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-04-23T14:58:58.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:58 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-23T14:58:58.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:58 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-04-23T14:58:58.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:58 smithi203 ceph-mon[32800]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-23T14:58:58.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:58 smithi203 ceph-mon[32800]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-04-23T14:58:58.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:58 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi173 to 176.2M 2024-04-23T14:58:58.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:58 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi203 to 176.1M 2024-04-23T14:58:58.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:58 smithi083 ceph-mon[27867]: Unable to set osd_memory_target on smithi173 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-23T14:58:58.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:58 smithi083 ceph-mon[27867]: Unable to set osd_memory_target on smithi203 to 184758784: error parsing value: Value '184758784' is below minimum 939524096 2024-04-23T14:58:59.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:59 smithi173 ceph-mon[32443]: pgmap v327: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:58:59.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:59 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:59.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:58:59 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:59.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:59 smithi203 ceph-mon[32800]: pgmap v327: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:58:59.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:59 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:59.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:58:59 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:59.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:59 smithi083 ceph-mon[27867]: pgmap v327: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:58:59.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:59 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:58:59.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:58:59 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:59:01.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: pgmap v328: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:59:01.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-23T14:59:01.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-23T14:59:01.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:59:01.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:59:01.327 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:01 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:59:01.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: pgmap v328: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:59:01.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-23T14:59:01.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-23T14:59:01.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:59:01.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:59:01.472 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:01 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:59:01.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: pgmap v328: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:59:01.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T14:59:01.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: Adjusting osd_memory_target on smithi083 to 176.4M 2024-04-23T14:59:01.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: Unable to set osd_memory_target on smithi083 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-23T14:59:01.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T14:59:01.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T14:59:01.508 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:01 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T14:59:03.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:03 smithi173 ceph-mon[32443]: pgmap v329: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-23T14:59:03.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:03 smithi203 ceph-mon[32800]: pgmap v329: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-23T14:59:03.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:03 smithi083 ceph-mon[27867]: pgmap v329: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-23T14:59:05.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:05 smithi173 ceph-mon[32443]: pgmap v330: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:59:05.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:05 smithi203 ceph-mon[32800]: pgmap v330: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:59:05.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:05 smithi083 ceph-mon[27867]: pgmap v330: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T14:59:07.289 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:07 smithi203 ceph-mon[32800]: pgmap v331: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-23T14:59:07.310 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:07 smithi083 ceph-mon[27867]: pgmap v331: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-23T14:59:07.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:07 smithi173 ceph-mon[32443]: pgmap v331: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-23T14:59:07.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-23T14:59:07.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-23T14:59:08.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.125+0000 7fb6e8a00640 1 -- 172.21.15.83:0/2154060735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e4102ac0 msgr2=0x7fb6e4106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.125+0000 7fb6e8a00640 1 --2- 172.21.15.83:0/2154060735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e4102ac0 0x7fb6e4106b00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009900 tx=0x7fb6d002a410 comp rx=0 tx=0).stop 2024-04-23T14:59:08.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.125+0000 7fb6e8a00640 1 -- 172.21.15.83:0/2154060735 shutdown_connections 2024-04-23T14:59:08.125 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.125+0000 7fb6e8a00640 1 --2- 172.21.15.83:0/2154060735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e41071e0 0x7fb6e410edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.125+0000 7fb6e8a00640 1 --2- 172.21.15.83:0/2154060735 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e4102ac0 0x7fb6e4106b00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.125+0000 7fb6e8a00640 1 --2- 172.21.15.83:0/2154060735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6e4102110 0x7fb6e41024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.125+0000 7fb6e8a00640 1 -- 172.21.15.83:0/2154060735 >> 172.21.15.83:0/2154060735 conn(0x7fb6e40fd9d0 msgr2=0x7fb6e40ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.125+0000 7fb6e8a00640 1 -- 172.21.15.83:0/2154060735 shutdown_connections 2024-04-23T14:59:08.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.125+0000 7fb6e8a00640 1 -- 172.21.15.83:0/2154060735 wait complete. 2024-04-23T14:59:08.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.126+0000 7fb6e8a00640 1 Processor -- start 2024-04-23T14:59:08.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.126+0000 7fb6e8a00640 1 -- start start 2024-04-23T14:59:08.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.126+0000 7fb6e8a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6e4102110 0x7fb6e41a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.126+0000 7fb6e8a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e41071e0 0x7fb6e41a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.126+0000 7fb6e8a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e419a8b0 0x7fb6e419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.126+0000 7fb6e8a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e4112270 con 0x7fb6e419a8b0 2024-04-23T14:59:08.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.126+0000 7fb6e8a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41120f0 con 0x7fb6e4102110 2024-04-23T14:59:08.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.126+0000 7fb6e8a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41123f0 con 0x7fb6e41071e0 2024-04-23T14:59:08.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.127+0000 7fb6e1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e41071e0 0x7fb6e41a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.127+0000 7fb6e3200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e419a8b0 0x7fb6e419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.127+0000 7fb6e1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e41071e0 0x7fb6e41a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49654/0 (socket says 172.21.15.83:49654) 2024-04-23T14:59:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.127+0000 7fb6e1e00640 1 -- 172.21.15.83:0/876404897 learned_addr learned my addr 172.21.15.83:0/876404897 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.127+0000 7fb6e2800640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6e4102110 0x7fb6e41a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.127+0000 7fb6e3200640 1 -- 172.21.15.83:0/876404897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e41071e0 msgr2=0x7fb6e41a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.127+0000 7fb6e3200640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e41071e0 0x7fb6e41a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.127+0000 7fb6e3200640 1 -- 172.21.15.83:0/876404897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6e4102110 msgr2=0x7fb6e41a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.127+0000 7fb6e3200640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6e4102110 0x7fb6e41a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.127+0000 7fb6e3200640 1 -- 172.21.15.83:0/876404897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e419b3d0 con 0x7fb6e419a8b0 2024-04-23T14:59:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.127+0000 7fb6e1e00640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e41071e0 0x7fb6e41a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:59:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.128+0000 7fb6e3200640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e419a8b0 0x7fb6e419ad60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009fd0 tx=0x7fb6dc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.128+0000 7fb6db400640 1 -- 172.21.15.83:0/876404897 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00ce20 con 0x7fb6e419a8b0 2024-04-23T14:59:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.128+0000 7fb6e8a00640 1 -- 172.21.15.83:0/876404897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e419b6c0 con 0x7fb6e419a8b0 2024-04-23T14:59:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.128+0000 7fb6db400640 1 -- 172.21.15.83:0/876404897 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6dc005020 con 0x7fb6e419a8b0 2024-04-23T14:59:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.128+0000 7fb6db400640 1 -- 172.21.15.83:0/876404897 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc010700 con 0x7fb6e419a8b0 2024-04-23T14:59:08.131 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.128+0000 7fb6e8a00640 1 -- 172.21.15.83:0/876404897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e41a78c0 con 0x7fb6e419a8b0 2024-04-23T14:59:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.131+0000 7fb6ec26c740 1 Processor -- start 2024-04-23T14:59:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.131+0000 7fb6db400640 1 -- 172.21.15.83:0/876404897 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb6dc005540 con 0x7fb6e419a8b0 2024-04-23T14:59:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.131+0000 7fb6ec26c740 1 -- start start 2024-04-23T14:59:08.138 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6ec26c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f4be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6ec26c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 0x565184f564e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6ec26c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f58e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6ec26c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565184eb11f0 con 0x565184f4c350 2024-04-23T14:59:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6ec26c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x565184e57350 con 0x565184f56a20 2024-04-23T14:59:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6ec26c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x565184ead1b0 con 0x565184f4ba30 2024-04-23T14:59:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6e1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f4be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6e2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 0x565184f564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6e1e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f4be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49660/0 (socket says 172.21.15.83:49660) 2024-04-23T14:59:08.139 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6e1e00640 1 -- 172.21.15.83:0/2716409900 learned_addr learned my addr 172.21.15.83:0/2716409900 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6e1e00640 1 --2- 172.21.15.83:0/2716409900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f58e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6e1e00640 1 -- 172.21.15.83:0/2716409900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 msgr2=0x565184f58e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6e1e00640 1 --2- 172.21.15.83:0/2716409900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f58e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6e1e00640 1 -- 172.21.15.83:0/2716409900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 msgr2=0x565184f564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6e1e00640 1 --2- 172.21.15.83:0/2716409900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 0x565184f564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.132+0000 7fb6e1e00640 1 -- 172.21.15.83:0/2716409900 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565184eabf50 con 0x565184f4ba30 2024-04-23T14:59:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.133+0000 7fb6e1e00640 1 --2- 172.21.15.83:0/2716409900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f4be10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0004770 tx=0x7fb6d00314e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc156b83f43901fc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.140 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.133+0000 7fb6cf400640 1 -- 172.21.15.83:0/2716409900 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0044070 con 0x565184f4ba30 2024-04-23T14:59:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.133+0000 7fb6cf400640 1 -- 172.21.15.83:0/2716409900 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d0005020 con 0x565184f4ba30 2024-04-23T14:59:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.133+0000 7fb6cf400640 1 -- 172.21.15.83:0/2716409900 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0035040 con 0x565184f4ba30 2024-04-23T14:59:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.133+0000 7fb6db400640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb6c80a3d20 0x7fb6c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.134+0000 7fb6db400640 1 -- 172.21.15.83:0/876404897 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fb6dc0d0ea0 con 0x7fb6e419a8b0 2024-04-23T14:59:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.134+0000 7fb6ec26c740 1 -- 172.21.15.83:0/2716409900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 msgr2=0x565184f4be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.134+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/2716409900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f4be10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0004770 tx=0x7fb6d00314e0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.134+0000 7fb6e2800640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb6c80a3d20 0x7fb6c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.141 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.134+0000 7fb6ec26c740 1 -- 172.21.15.83:0/2716409900 shutdown_connections 2024-04-23T14:59:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.134+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/2716409900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f58e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.134+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/2716409900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 0x565184f564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.134+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/2716409900 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f4be10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.134+0000 7fb6ec26c740 1 -- 172.21.15.83:0/2716409900 >> 172.21.15.83:0/2716409900 conn(0x565184f47d50 msgr2=0x565184f48130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.134+0000 7fb6ec26c740 1 -- 172.21.15.83:0/2716409900 shutdown_connections 2024-04-23T14:59:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.134+0000 7fb6ec26c740 1 -- 172.21.15.83:0/2716409900 wait complete. 2024-04-23T14:59:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.134+0000 7fb6e2800640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb6c80a3d20 0x7fb6c80a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x565184f4cff0 tx=0x7fb6d4005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.142 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.135+0000 7fb6ec26c740 1 -- start start 2024-04-23T14:59:08.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.135+0000 7fb6ec26c740 1 Processor -- start 2024-04-23T14:59:08.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.135+0000 7fb6ec26c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.135+0000 7fb6ec26c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 0x565184f53f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.135+0000 7fb6ec26c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f54450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.135+0000 7fb6ec26c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x565184f597b0 con 0x565184f4c350 2024-04-23T14:59:08.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.135+0000 7fb6ec26c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x565184e6a690 con 0x565184f56a20 2024-04-23T14:59:08.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.135+0000 7fb6ec26c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x565184f59930 con 0x565184f4ba30 2024-04-23T14:59:08.143 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.135+0000 7fb6ec26c740 10 client.0 _ll_register_callbacks cb 0x565184ea6650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:59:08.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.135+0000 7fb6e2800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6e1e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 0x565184f53f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6e2800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49666/0 (socket says 172.21.15.83:49666) 2024-04-23T14:59:08.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6e2800640 1 -- 172.21.15.83:0/1017943282 learned_addr learned my addr 172.21.15.83:0/1017943282 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:08.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6e3200640 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f54450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6e2800640 1 -- 172.21.15.83:0/1017943282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 msgr2=0x565184f54450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6e2800640 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f54450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.144 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6e2800640 1 -- 172.21.15.83:0/1017943282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 msgr2=0x565184f53f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6e2800640 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 0x565184f53f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6e2800640 1 -- 172.21.15.83:0/1017943282 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565184e69920 con 0x565184f4ba30 2024-04-23T14:59:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6e3200640 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f54450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:59:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6e2800640 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f539d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb6d401ea00 tx=0x7fb6d4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6877501a407c28ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:59:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.136+0000 7fb6c3e00640 1 -- 172.21.15.83:0/1017943282 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d402e060 con 0x565184f4ba30 2024-04-23T14:59:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6ec26c740 1 -- 172.21.15.83:0/1017943282 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6d0005020 con 0x565184f4ba30 2024-04-23T14:59:08.145 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6c3e00640 1 -- 172.21.15.83:0/1017943282 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fb6d4018030 con 0x565184f4ba30 2024-04-23T14:59:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6c3e00640 1 -- 172.21.15.83:0/1017943282 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb6d4032420 con 0x565184f4ba30 2024-04-23T14:59:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6ec26c740 10 client.27295 fetch_fsmap learned FSMap version 29 2024-04-23T14:59:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6ec26c740 1 -- 172.21.15.83:0/1017943282 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565184c8d7d0 con 0x565184f4ba30 2024-04-23T14:59:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6c3e00640 1 -- 172.21.15.83:0/1017943282 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d0005020 con 0x565184f4ba30 2024-04-23T14:59:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6c3e00640 1 -- 172.21.15.83:0/1017943282 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fb6d4031600 con 0x565184f4ba30 2024-04-23T14:59:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6ec26c740 10 client.27295 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:59:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6ec26c740 10 client.27295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:59:08.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6ec26c740 20 client.27295 populate_metadata read hostname 'smithi083' 2024-04-23T14:59:08.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x565184f66d60 0x565184f69200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6ec26c740 4 client.27295 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:59:08.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.137+0000 7fb6ec26c740 1 -- 172.21.15.83:0/1017943282 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565184ebec10 con 0x565184f66d60 2024-04-23T14:59:08.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.138+0000 7fb6e1e00640 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x565184f66d60 0x565184f69200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.138+0000 7fb6e1e00640 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x565184f66d60 0x565184f69200 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c344a5fe92e04ed server_cookie=5576aae46dc276d in_seq=0 out_seq=0 2024-04-23T14:59:08.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.138+0000 7fb6c3e00640 10 client.27295 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:59:08.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.139+0000 7fb6c3e00640 1 -- 172.21.15.83:0/1017943282 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565184ebec10 con 0x565184f66d60 2024-04-23T14:59:08.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.139+0000 7fb6c3e00640 10 client.27295 handle_command_reply: tid=0 2024-04-23T14:59:08.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.139+0000 7fb6ec26c740 1 client.27295 shutdown 2024-04-23T14:59:08.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.139+0000 7fb6ec26c740 1 -- 172.21.15.83:0/1017943282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 msgr2=0x565184f539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.139+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f539d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb6d401ea00 tx=0x7fb6d4004730 comp rx=0 tx=0).stop 2024-04-23T14:59:08.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.140+0000 7fb6ec26c740 1 -- 172.21.15.83:0/1017943282 shutdown_connections 2024-04-23T14:59:08.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.140+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x565184f66d60 0x565184f69200 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.140+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f54450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.140+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 0x565184f53f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.140+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/1017943282 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f539d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.140+0000 7fb6c3e00640 0 client.27295 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:59:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.140+0000 7fb6ec26c740 1 -- 172.21.15.83:0/1017943282 >> 172.21.15.83:0/1017943282 conn(0x565184f47d50 msgr2=0x565184f4db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.140+0000 7fb6ec26c740 1 -- 172.21.15.83:0/1017943282 shutdown_connections 2024-04-23T14:59:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.140+0000 7fb6ec26c740 1 -- 172.21.15.83:0/1017943282 wait complete. 2024-04-23T14:59:08.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.140+0000 7fb6ec26c740 20 client.27295 trim_cache size 0 max 16384 2024-04-23T14:59:08.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.159+0000 7fb6ec26c740 1 Processor -- start 2024-04-23T14:59:08.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6ec26c740 1 -- start start 2024-04-23T14:59:08.160 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6ec26c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f62670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6ec26c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 0x565184f46f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6ec26c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f4e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6ec26c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41b25b0 con 0x565184f4c350 2024-04-23T14:59:08.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6ec26c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41afe80 con 0x565184f56a20 2024-04-23T14:59:08.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6ec26c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41adcf0 con 0x565184f4ba30 2024-04-23T14:59:08.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6e2800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f62670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6e2800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f62670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49672/0 (socket says 172.21.15.83:49672) 2024-04-23T14:59:08.161 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6e2800640 1 -- 172.21.15.83:0/2229619827 learned_addr learned my addr 172.21.15.83:0/2229619827 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:08.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6e1e00640 1 --2- 172.21.15.83:0/2229619827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 0x565184f46f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6e3200640 1 --2- 172.21.15.83:0/2229619827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f4e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6e2800640 1 -- 172.21.15.83:0/2229619827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 msgr2=0x565184f4e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6e2800640 1 --2- 172.21.15.83:0/2229619827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f4e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6e2800640 1 -- 172.21.15.83:0/2229619827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 msgr2=0x565184f46f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6e2800640 1 --2- 172.21.15.83:0/2229619827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 0x565184f46f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.160+0000 7fb6e2800640 1 -- 172.21.15.83:0/2229619827 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565184bc33f0 con 0x565184f4ba30 2024-04-23T14:59:08.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.161+0000 7fb6e2800640 1 --2- 172.21.15.83:0/2229619827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f62670 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4019040 tx=0x7fb6d4004620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9c1434d9765ae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.161+0000 7fb6cea00640 1 -- 172.21.15.83:0/2229619827 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4031c90 con 0x565184f4ba30 2024-04-23T14:59:08.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.161+0000 7fb6cea00640 1 -- 172.21.15.83:0/2229619827 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d4031e30 con 0x565184f4ba30 2024-04-23T14:59:08.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.161+0000 7fb6cea00640 1 -- 172.21.15.83:0/2229619827 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4004a00 con 0x565184f4ba30 2024-04-23T14:59:08.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.162+0000 7fb6ec26c740 1 -- 172.21.15.83:0/2229619827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 msgr2=0x565184f62670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.162+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/2229619827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f62670 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4019040 tx=0x7fb6d4004620 comp rx=0 tx=0).stop 2024-04-23T14:59:08.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.162+0000 7fb6ec26c740 1 -- 172.21.15.83:0/2229619827 shutdown_connections 2024-04-23T14:59:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.162+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/2229619827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f56a20 0x565184f4e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.162+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/2229619827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4c350 0x565184f46f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.162+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/2229619827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f4ba30 0x565184f62670 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.162+0000 7fb6ec26c740 1 -- 172.21.15.83:0/2229619827 >> 172.21.15.83:0/2229619827 conn(0x565184f47d50 msgr2=0x565184f55200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.162+0000 7fb6ec26c740 1 -- 172.21.15.83:0/2229619827 shutdown_connections 2024-04-23T14:59:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.162+0000 7fb6ec26c740 1 -- 172.21.15.83:0/2229619827 wait complete. 2024-04-23T14:59:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.162+0000 7fb6ec26c740 1 -- start start 2024-04-23T14:59:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.163+0000 7fb6ec26c740 1 Processor -- start 2024-04-23T14:59:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.163+0000 7fb6ec26c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4ba30 0x565184f5a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.163+0000 7fb6ec26c740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f4c350 0x565184f5aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.163+0000 7fb6ec26c740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f56a20 0x565184f5b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.163+0000 7fb6ec26c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41ae340 con 0x565184f4ba30 2024-04-23T14:59:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.163+0000 7fb6ec26c740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41af7b0 con 0x565184f4c350 2024-04-23T14:59:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.163+0000 7fb6ec26c740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e41aea10 con 0x565184f56a20 2024-04-23T14:59:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.163+0000 7fb6ec26c740 10 client.0 _ll_register_callbacks cb 0x7fb6d4012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:59:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.163+0000 7fb6e3200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f56a20 0x565184f5b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.163+0000 7fb6e2800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4ba30 0x565184f5a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.163+0000 7fb6e3200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f56a20 0x565184f5b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49678/0 (socket says 172.21.15.83:49678) 2024-04-23T14:59:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.163+0000 7fb6e3200640 1 -- 172.21.15.83:0/947278016 learned_addr learned my addr 172.21.15.83:0/947278016 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.164+0000 7fb6e1e00640 1 --2- 172.21.15.83:0/947278016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f4c350 0x565184f5aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.164+0000 7fb6e3200640 1 -- 172.21.15.83:0/947278016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f4c350 msgr2=0x565184f5aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.164+0000 7fb6e3200640 1 --2- 172.21.15.83:0/947278016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f4c350 0x565184f5aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.164+0000 7fb6e3200640 1 -- 172.21.15.83:0/947278016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4ba30 msgr2=0x565184f5a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.164+0000 7fb6e3200640 1 --2- 172.21.15.83:0/947278016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4ba30 0x565184f5a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.164+0000 7fb6e3200640 1 -- 172.21.15.83:0/947278016 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565184b830f0 con 0x565184f56a20 2024-04-23T14:59:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.164+0000 7fb6e3200640 1 --2- 172.21.15.83:0/947278016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f56a20 0x565184f5b030 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0ce5e0 tx=0x7fb6dc09ce90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55dd5ef11996b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.164+0000 7fb6c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:59:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.164+0000 7fb6c3e00640 1 -- 172.21.15.83:0/947278016 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0b5070 con 0x565184f56a20 2024-04-23T14:59:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.164+0000 7fb6c3e00640 1 -- 172.21.15.83:0/947278016 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7fb6dc099440 con 0x565184f56a20 2024-04-23T14:59:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.164+0000 7fb6ec26c740 1 -- 172.21.15.83:0/947278016 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6d4031e30 con 0x565184f56a20 2024-04-23T14:59:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.165+0000 7fb6c3e00640 1 -- 172.21.15.83:0/947278016 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb6dc0a0040 con 0x565184f56a20 2024-04-23T14:59:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.165+0000 7fb6ec26c740 10 client.27301 fetch_fsmap learned FSMap version 29 2024-04-23T14:59:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.165+0000 7fb6ec26c740 1 -- 172.21.15.83:0/947278016 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565184e76810 con 0x565184f56a20 2024-04-23T14:59:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.165+0000 7fb6c3e00640 1 -- 172.21.15.83:0/947278016 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d4031e30 con 0x565184f56a20 2024-04-23T14:59:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.165+0000 7fb6c3e00640 1 -- 172.21.15.83:0/947278016 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fb6dc09c220 con 0x565184f56a20 2024-04-23T14:59:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.165+0000 7fb6ec26c740 10 client.27301 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:59:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.165+0000 7fb6ec26c740 10 client.27301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:59:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.165+0000 7fb6ec26c740 20 client.27301 populate_metadata read hostname 'smithi083' 2024-04-23T14:59:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.165+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/947278016 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x565184fcda50 0x565184fcfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.165+0000 7fb6ec26c740 4 client.27301 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-23T14:59:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.165+0000 7fb6ec26c740 1 -- 172.21.15.83:0/947278016 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565184eb8a80 con 0x565184fcda50 2024-04-23T14:59:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.165+0000 7fb6e2800640 1 --2- 172.21.15.83:0/947278016 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x565184fcda50 0x565184fcfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6c3e00640 10 client.27301 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:59:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6e2800640 1 --2- 172.21.15.83:0/947278016 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x565184fcda50 0x565184fcfe70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef5c786d8277c16 server_cookie=f98d6616fe2119c3 in_seq=0 out_seq=0 2024-04-23T14:59:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6c3e00640 1 -- 172.21.15.83:0/947278016 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x565184eb8a80 con 0x565184fcda50 2024-04-23T14:59:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6c3e00640 10 client.27301 handle_command_reply: tid=0 2024-04-23T14:59:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6ec26c740 1 client.27301 shutdown 2024-04-23T14:59:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6ec26c740 1 -- 172.21.15.83:0/947278016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f56a20 msgr2=0x565184f5b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/947278016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f56a20 0x565184f5b030 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0ce5e0 tx=0x7fb6dc09ce90 comp rx=0 tx=0).stop 2024-04-23T14:59:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6ec26c740 1 -- 172.21.15.83:0/947278016 shutdown_connections 2024-04-23T14:59:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/947278016 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x565184fcda50 0x565184fcfe70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/947278016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x565184f56a20 0x565184f5b030 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/947278016 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x565184f4c350 0x565184f5aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6c3e00640 0 client.27301 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:59:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6ec26c740 1 --2- 172.21.15.83:0/947278016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x565184f4ba30 0x565184f5a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.166+0000 7fb6ec26c740 1 -- 172.21.15.83:0/947278016 >> 172.21.15.83:0/947278016 conn(0x565184f47d50 msgr2=0x565184f55a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.167+0000 7fb6ec26c740 1 -- 172.21.15.83:0/947278016 shutdown_connections 2024-04-23T14:59:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.167+0000 7fb6ec26c740 1 -- 172.21.15.83:0/947278016 wait complete. 2024-04-23T14:59:08.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.167+0000 7fb6ec26c740 20 client.27301 trim_cache size 0 max 16384 2024-04-23T14:59:08.174 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:59:08.175 INFO:teuthology.orchestra.run.smithi083.stdout: "status": "no active scrubs running", 2024-04-23T14:59:08.175 INFO:teuthology.orchestra.run.smithi083.stdout: "scrubs": {} 2024-04-23T14:59:08.175 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:59:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.167+0000 7fb6e8a00640 1 -- 172.21.15.83:0/876404897 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb6c80a3d20 msgr2=0x7fb6c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.167+0000 7fb6e8a00640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb6c80a3d20 0x7fb6c80a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x565184f4cff0 tx=0x7fb6d4005f90 comp rx=0 tx=0).stop 2024-04-23T14:59:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.167+0000 7fb6e8a00640 1 -- 172.21.15.83:0/876404897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e419a8b0 msgr2=0x7fb6e419ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.167+0000 7fb6e8a00640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e419a8b0 0x7fb6e419ad60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009fd0 tx=0x7fb6dc00ef10 comp rx=0 tx=0).stop 2024-04-23T14:59:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.168+0000 7fb6e8a00640 1 -- 172.21.15.83:0/876404897 shutdown_connections 2024-04-23T14:59:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.168+0000 7fb6e8a00640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb6c80a3d20 0x7fb6c80a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.168+0000 7fb6e8a00640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6e419a8b0 0x7fb6e419ad60 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.168+0000 7fb6e8a00640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6e41071e0 0x7fb6e41a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.168+0000 7fb6e8a00640 1 --2- 172.21.15.83:0/876404897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6e4102110 0x7fb6e41a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.168+0000 7fb6e8a00640 1 -- 172.21.15.83:0/876404897 >> 172.21.15.83:0/876404897 conn(0x7fb6e40fd9d0 msgr2=0x7fb6e40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.168+0000 7fb6e8a00640 1 -- 172.21.15.83:0/876404897 shutdown_connections 2024-04-23T14:59:08.177 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.168+0000 7fb6e8a00640 1 -- 172.21.15.83:0/876404897 wait complete. 2024-04-23T14:59:08.185 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-23T14:59:08.186 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:59:08.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.382+0000 7f5fb9e00640 1 -- 172.21.15.83:0/1858564378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fb4108fd0 msgr2=0x7f5fb41154e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.382 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.382+0000 7f5fb9e00640 1 --2- 172.21.15.83:0/1858564378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fb4108fd0 0x7f5fb41154e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009cd0 tx=0x7f5fac02a8f0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.382+0000 7f5fb9e00640 1 -- 172.21.15.83:0/1858564378 shutdown_connections 2024-04-23T14:59:08.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.382+0000 7f5fb9e00640 1 --2- 172.21.15.83:0/1858564378 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fb4108fd0 0x7f5fb41154e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.382+0000 7f5fb9e00640 1 --2- 172.21.15.83:0/1858564378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb4108610 0x7f5fb4108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.382+0000 7f5fb9e00640 1 --2- 172.21.15.83:0/1858564378 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fb410e6b0 0x7f5fb410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.382+0000 7f5fb9e00640 1 -- 172.21.15.83:0/1858564378 >> 172.21.15.83:0/1858564378 conn(0x7f5fb4075800 msgr2=0x7f5fb4106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.383+0000 7f5fb9e00640 1 -- 172.21.15.83:0/1858564378 shutdown_connections 2024-04-23T14:59:08.383 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.383+0000 7f5fb9e00640 1 -- 172.21.15.83:0/1858564378 wait complete. 2024-04-23T14:59:08.385 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.383+0000 7f5fb9e00640 1 Processor -- start 2024-04-23T14:59:08.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.384+0000 7f5fb9e00640 1 -- start start 2024-04-23T14:59:08.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.384+0000 7f5fb9e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fb4108610 0x7f5fb419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.384+0000 7f5fb9e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb4108fd0 0x7f5fb419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.384+0000 7f5fb9e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fb410e6b0 0x7f5fb41a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.384+0000 7f5fb9e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb4118a40 con 0x7f5fb4108fd0 2024-04-23T14:59:08.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.384+0000 7f5fb9e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb41188c0 con 0x7f5fb4108610 2024-04-23T14:59:08.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.384+0000 7f5fb9e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb4118bc0 con 0x7f5fb410e6b0 2024-04-23T14:59:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.384+0000 7f5faa000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb4108fd0 0x7f5fb419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.384+0000 7f5fb3400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fb410e6b0 0x7f5fb41a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.384+0000 7f5fb3400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fb410e6b0 0x7f5fb41a3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49694/0 (socket says 172.21.15.83:49694) 2024-04-23T14:59:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.384+0000 7f5fb3400640 1 -- 172.21.15.83:0/2464647400 learned_addr learned my addr 172.21.15.83:0/2464647400 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.385+0000 7f5fb2a00640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fb4108610 0x7f5fb419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.385+0000 7f5fb3400640 1 -- 172.21.15.83:0/2464647400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fb4108610 msgr2=0x7f5fb419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.385+0000 7f5fb3400640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fb4108610 0x7f5fb419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.385+0000 7f5fb3400640 1 -- 172.21.15.83:0/2464647400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb4108fd0 msgr2=0x7f5fb419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.385+0000 7f5fb3400640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb4108fd0 0x7f5fb419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.385+0000 7f5fb3400640 1 -- 172.21.15.83:0/2464647400 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb41a3e30 con 0x7f5fb410e6b0 2024-04-23T14:59:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.385+0000 7f5fb3400640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fb410e6b0 0x7f5fb41a3700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5fac005e30 tx=0x7f5fac003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.385+0000 7f5fb8a00640 1 -- 172.21.15.83:0/2464647400 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac030dc0 con 0x7f5fb410e6b0 2024-04-23T14:59:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.385+0000 7f5fb8a00640 1 -- 172.21.15.83:0/2464647400 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fac040070 con 0x7f5fb410e6b0 2024-04-23T14:59:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.385+0000 7f5fb9e00640 1 -- 172.21.15.83:0/2464647400 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb41a40c0 con 0x7f5fb410e6b0 2024-04-23T14:59:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.385+0000 7f5fb8a00640 1 -- 172.21.15.83:0/2464647400 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac045ad0 con 0x7f5fb410e6b0 2024-04-23T14:59:08.388 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.386+0000 7f5fb9e00640 1 -- 172.21.15.83:0/2464647400 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb41ab9b0 con 0x7f5fb410e6b0 2024-04-23T14:59:08.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.387+0000 7f5fb9e00640 1 -- 172.21.15.83:0/2464647400 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb4069270 con 0x7f5fb410e6b0 2024-04-23T14:59:08.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.389+0000 7f5fb8a00640 1 -- 172.21.15.83:0/2464647400 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5fac004700 con 0x7f5fb410e6b0 2024-04-23T14:59:08.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.390+0000 7f5fb8a00640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5f980a3d20 0x7f5f980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.390+0000 7f5fb8a00640 1 -- 172.21.15.83:0/2464647400 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f5fac0f1860 con 0x7f5fb410e6b0 2024-04-23T14:59:08.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.390+0000 7f5fb2a00640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5f980a3d20 0x7f5f980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.391+0000 7f5fb2a00640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5f980a3d20 0x7f5f980a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40095f0 tx=0x7f5fa4005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.392+0000 7f5fb8a00640 1 -- 172.21.15.83:0/2464647400 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fac0be030 con 0x7f5fb410e6b0 2024-04-23T14:59:08.635 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.635+0000 7f5fb9e00640 1 -- 172.21.15.83:0/2464647400 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fb41a0190 con 0x7f5fb410e6b0 2024-04-23T14:59:08.636 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.636+0000 7f5fb8a00640 1 -- 172.21.15.83:0/2464647400 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7f5fb41a0190 con 0x7f5fb410e6b0 2024-04-23T14:59:08.637 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:59:08.637 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:59:08.637 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:59:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.639+0000 7f5fb9e00640 1 -- 172.21.15.83:0/2464647400 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5f980a3d20 msgr2=0x7f5f980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.639+0000 7f5fb9e00640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5f980a3d20 0x7f5f980a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40095f0 tx=0x7f5fa4005f90 comp rx=0 tx=0).stop 2024-04-23T14:59:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.639+0000 7f5fb9e00640 1 -- 172.21.15.83:0/2464647400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fb410e6b0 msgr2=0x7f5fb41a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.639+0000 7f5fb9e00640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fb410e6b0 0x7f5fb41a3700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5fac005e30 tx=0x7f5fac003f10 comp rx=0 tx=0).stop 2024-04-23T14:59:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.640+0000 7f5fb9e00640 1 -- 172.21.15.83:0/2464647400 shutdown_connections 2024-04-23T14:59:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.640+0000 7f5fb9e00640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5fb410e6b0 0x7f5fb41a3700 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.640+0000 7f5fb9e00640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5f980a3d20 0x7f5f980a61e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.640 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.640+0000 7f5fb9e00640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fb4108fd0 0x7f5fb419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.640+0000 7f5fb9e00640 1 --2- 172.21.15.83:0/2464647400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5fb4108610 0x7f5fb419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.640+0000 7f5fb9e00640 1 -- 172.21.15.83:0/2464647400 >> 172.21.15.83:0/2464647400 conn(0x7f5fb4075800 msgr2=0x7f5fb4076740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.640+0000 7f5fb9e00640 1 -- 172.21.15.83:0/2464647400 shutdown_connections 2024-04-23T14:59:08.641 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.640+0000 7f5fb9e00640 1 -- 172.21.15.83:0/2464647400 wait complete. 2024-04-23T14:59:08.659 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-23T14:59:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.853+0000 7f29c4a00640 1 -- 172.21.15.83:0/735731292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c0103ba0 msgr2=0x7f29c010c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.853+0000 7f29c4a00640 1 --2- 172.21.15.83:0/735731292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c0103ba0 0x7f29c010c080 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009900 tx=0x7f29ac02a510 comp rx=0 tx=0).stop 2024-04-23T14:59:08.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.853+0000 7f29c4a00640 1 -- 172.21.15.83:0/735731292 shutdown_connections 2024-04-23T14:59:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.853+0000 7f29c4a00640 1 --2- 172.21.15.83:0/735731292 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c010c5c0 0x7f29c010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.853+0000 7f29c4a00640 1 --2- 172.21.15.83:0/735731292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c0103ba0 0x7f29c010c080 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.853+0000 7f29c4a00640 1 --2- 172.21.15.83:0/735731292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c0103280 0x7f29c0103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.853+0000 7f29c4a00640 1 -- 172.21.15.83:0/735731292 >> 172.21.15.83:0/735731292 conn(0x7f29c0078000 msgr2=0x7f29c0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.853+0000 7f29c4a00640 1 -- 172.21.15.83:0/735731292 shutdown_connections 2024-04-23T14:59:08.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.853+0000 7f29c4a00640 1 -- 172.21.15.83:0/735731292 wait complete. 2024-04-23T14:59:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.854+0000 7f29c4a00640 1 Processor -- start 2024-04-23T14:59:08.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.854+0000 7f29c4a00640 1 -- start start 2024-04-23T14:59:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.855+0000 7f29c4a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c0103280 0x7f29c019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.855+0000 7f29c4a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c0103ba0 0x7f29c019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.855+0000 7f29c4a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c010c5c0 0x7f29c0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.855+0000 7f29c4a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01123a0 con 0x7f29c010c5c0 2024-04-23T14:59:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.855+0000 7f29c4a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29c0112220 con 0x7f29c0103280 2024-04-23T14:59:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.855+0000 7f29c4a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f29c0112520 con 0x7f29c0103ba0 2024-04-23T14:59:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.855+0000 7f29bd400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c0103ba0 0x7f29c019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.855+0000 7f29bde00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c0103280 0x7f29c019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.855+0000 7f29be800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c010c5c0 0x7f29c0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.855+0000 7f29bd400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c0103ba0 0x7f29c019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49710/0 (socket says 172.21.15.83:49710) 2024-04-23T14:59:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.855+0000 7f29bd400640 1 -- 172.21.15.83:0/824115671 learned_addr learned my addr 172.21.15.83:0/824115671 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29bde00640 1 -- 172.21.15.83:0/824115671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c0103ba0 msgr2=0x7f29c019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29bde00640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c0103ba0 0x7f29c019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29bde00640 1 -- 172.21.15.83:0/824115671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c010c5c0 msgr2=0x7f29c0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29bde00640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c010c5c0 0x7f29c0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29bde00640 1 -- 172.21.15.83:0/824115671 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c0198fe0 con 0x7f29c0103280 2024-04-23T14:59:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29bd400640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c0103ba0 0x7f29c019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:59:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29be800640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c010c5c0 0x7f29c0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:59:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29bde00640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c0103280 0x7f29c019e660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f29b000e9f0 tx=0x7f29b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29b6a00640 1 -- 172.21.15.83:0/824115671 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b000ccc0 con 0x7f29c0103280 2024-04-23T14:59:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29c4a00640 1 -- 172.21.15.83:0/824115671 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c01992d0 con 0x7f29c0103280 2024-04-23T14:59:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29b6a00640 1 -- 172.21.15.83:0/824115671 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29b0005020 con 0x7f29c0103280 2024-04-23T14:59:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29b6a00640 1 -- 172.21.15.83:0/824115671 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00049e0 con 0x7f29c0103280 2024-04-23T14:59:08.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.856+0000 7f29c4a00640 1 -- 172.21.15.83:0/824115671 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c01a55e0 con 0x7f29c0103280 2024-04-23T14:59:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.860+0000 7f29c796b740 1 Processor -- start 2024-04-23T14:59:08.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.861+0000 7f29c796b740 1 -- start start 2024-04-23T14:59:08.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.861+0000 7f29c796b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab7950 0x559b53ab7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.861+0000 7f29c796b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab8270 0x559b53ac2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.861+0000 7f29c796b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ac2940 0x559b53ac4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.861+0000 7f29c796b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b53a1d1d0 con 0x559b53ab7950 2024-04-23T14:59:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.861+0000 7f29c796b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559b539c3450 con 0x559b53ab8270 2024-04-23T14:59:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.861+0000 7f29c796b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559b53a19190 con 0x559b53ac2940 2024-04-23T14:59:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.861+0000 7f29be800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab7950 0x559b53ab7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.861+0000 7f29b6a00640 1 -- 172.21.15.83:0/824115671 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f29b0005560 con 0x7f29c0103280 2024-04-23T14:59:08.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.862+0000 7f29be800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab7950 0x559b53ab7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55876/0 (socket says 172.21.15.83:55876) 2024-04-23T14:59:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.862+0000 7f29be800640 1 -- 172.21.15.83:0/2983118679 learned_addr learned my addr 172.21.15.83:0/2983118679 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.862+0000 7f29bd400640 1 --2- 172.21.15.83:0/2983118679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ac2940 0x559b53ac4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.862+0000 7f29bde00640 1 --2- 172.21.15.83:0/2983118679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab8270 0x559b53ac2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.862+0000 7f29be800640 1 -- 172.21.15.83:0/2983118679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ac2940 msgr2=0x559b53ac4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.862+0000 7f29be800640 1 --2- 172.21.15.83:0/2983118679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ac2940 0x559b53ac4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.862+0000 7f29be800640 1 -- 172.21.15.83:0/2983118679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab8270 msgr2=0x559b53ac2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.862+0000 7f29be800640 1 --2- 172.21.15.83:0/2983118679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab8270 0x559b53ac2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.862+0000 7f29be800640 1 -- 172.21.15.83:0/2983118679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b53a80310 con 0x559b53ab7950 2024-04-23T14:59:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.862+0000 7f29bd400640 1 --2- 172.21.15.83:0/2983118679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ac2940 0x559b53ac4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:59:08.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.862+0000 7f29bde00640 1 --2- 172.21.15.83:0/2983118679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab8270 0x559b53ac2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:59:08.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.862+0000 7f29be800640 1 --2- 172.21.15.83:0/2983118679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab7950 0x559b53ab7d30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f29b8011790 tx=0x7f29b8011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806dcc6e4f5bb113 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.863+0000 7f29ab000640 1 -- 172.21.15.83:0/2983118679 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8003c30 con 0x559b53ab7950 2024-04-23T14:59:08.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.863+0000 7f29ab000640 1 -- 172.21.15.83:0/2983118679 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29b800fe80 con 0x559b53ab7950 2024-04-23T14:59:08.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.863+0000 7f29ab000640 1 -- 172.21.15.83:0/2983118679 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b80193e0 con 0x559b53ab7950 2024-04-23T14:59:08.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.863+0000 7f29b6a00640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f29a40a3da0 0x7f29a40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.863+0000 7f29b6a00640 1 -- 172.21.15.83:0/824115671 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f29b00d0770 con 0x7f29c0103280 2024-04-23T14:59:08.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.863+0000 7f29c796b740 1 -- 172.21.15.83:0/2983118679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab7950 msgr2=0x559b53ab7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.863+0000 7f29c796b740 1 --2- 172.21.15.83:0/2983118679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab7950 0x559b53ab7d30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f29b8011790 tx=0x7f29b8011c50 comp rx=0 tx=0).stop 2024-04-23T14:59:08.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.863+0000 7f29bde00640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f29a40a3da0 0x7f29a40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.864+0000 7f29c796b740 1 -- 172.21.15.83:0/2983118679 shutdown_connections 2024-04-23T14:59:08.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.864+0000 7f29c796b740 1 --2- 172.21.15.83:0/2983118679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ac2940 0x559b53ac4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.864+0000 7f29c796b740 1 --2- 172.21.15.83:0/2983118679 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab8270 0x559b53ac2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.864+0000 7f29c796b740 1 --2- 172.21.15.83:0/2983118679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab7950 0x559b53ab7d30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.864+0000 7f29c796b740 1 -- 172.21.15.83:0/2983118679 >> 172.21.15.83:0/2983118679 conn(0x559b53ab3c70 msgr2=0x559b53ab4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.864+0000 7f29c796b740 1 -- 172.21.15.83:0/2983118679 shutdown_connections 2024-04-23T14:59:08.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.864+0000 7f29c796b740 1 -- 172.21.15.83:0/2983118679 wait complete. 2024-04-23T14:59:08.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.864+0000 7f29bde00640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f29a40a3da0 0x7f29a40a6260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x559b53ab8f10 tx=0x7f29b00096e0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.864+0000 7f29c796b740 1 -- start start 2024-04-23T14:59:08.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.864+0000 7f29c796b740 1 Processor -- start 2024-04-23T14:59:08.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.865+0000 7f29c796b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab7950 0x559b53ac2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.865+0000 7f29c796b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab8270 0x559b53abf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.865+0000 7f29c796b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ac2940 0x559b53abfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.865+0000 7f29c796b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b53ac57d0 con 0x559b53ab8270 2024-04-23T14:59:08.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.865+0000 7f29c796b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559b539d6810 con 0x559b53ac2940 2024-04-23T14:59:08.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.865+0000 7f29c796b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559b53ac5950 con 0x559b53ab7950 2024-04-23T14:59:08.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.865+0000 7f29c796b740 10 client.0 _ll_register_callbacks cb 0x559b539a4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:59:08.871 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.865+0000 7f29be800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab7950 0x559b53ac2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.865+0000 7f29bd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab8270 0x559b53abf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.865+0000 7f29be800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab7950 0x559b53ac2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49742/0 (socket says 172.21.15.83:49742) 2024-04-23T14:59:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.865+0000 7f29be800640 1 -- 172.21.15.83:0/697934201 learned_addr learned my addr 172.21.15.83:0/697934201 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.865+0000 7f29be800640 1 --2- 172.21.15.83:0/697934201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ac2940 0x559b53abfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.866+0000 7f29be800640 1 -- 172.21.15.83:0/697934201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ac2940 msgr2=0x559b53abfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.866+0000 7f29be800640 1 --2- 172.21.15.83:0/697934201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ac2940 0x559b53abfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.866+0000 7f29be800640 1 -- 172.21.15.83:0/697934201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab8270 msgr2=0x559b53abf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.866+0000 7f29be800640 1 --2- 172.21.15.83:0/697934201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab8270 0x559b53abf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.866+0000 7f29be800640 1 -- 172.21.15.83:0/697934201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b539e73f0 con 0x559b53ab7950 2024-04-23T14:59:08.872 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.866+0000 7f29be800640 1 --2- 172.21.15.83:0/697934201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab7950 0x559b53ac2390 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f29b8005f40 tx=0x7f29b80045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d3f9d6a61f35cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.866+0000 7f299f400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:59:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.866+0000 7f299f400640 1 -- 172.21.15.83:0/697934201 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8004c50 con 0x559b53ab7950 2024-04-23T14:59:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.866+0000 7f299f400640 1 -- 172.21.15.83:0/697934201 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f29b8004df0 con 0x559b53ab7950 2024-04-23T14:59:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.866+0000 7f29c796b740 1 -- 172.21.15.83:0/697934201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559b53a2abf0 con 0x559b53ab7950 2024-04-23T14:59:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.866+0000 7f299f400640 1 -- 172.21.15.83:0/697934201 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29b8011020 con 0x559b53ab7950 2024-04-23T14:59:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.866+0000 7f29c796b740 10 client.27319 fetch_fsmap learned FSMap version 29 2024-04-23T14:59:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.867+0000 7f29c796b740 1 -- 172.21.15.83:0/697934201 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b5372f3f0 con 0x559b53ab7950 2024-04-23T14:59:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.867+0000 7f299f400640 1 -- 172.21.15.83:0/697934201 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559b53a2abf0 con 0x559b53ab7950 2024-04-23T14:59:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.867+0000 7f299f400640 1 -- 172.21.15.83:0/697934201 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f29b8018070 con 0x559b53ab7950 2024-04-23T14:59:08.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.867+0000 7f29c796b740 10 client.27319 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:59:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.867+0000 7f29c796b740 10 client.27319 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:59:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.867+0000 7f29c796b740 20 client.27319 populate_metadata read hostname 'smithi083' 2024-04-23T14:59:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.867+0000 7f29c796b740 1 --2- 172.21.15.83:0/697934201 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x559b53ad2a50 0x559b53ad4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.867+0000 7f29c796b740 4 client.27319 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:59:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.867+0000 7f29c796b740 1 -- 172.21.15.83:0/697934201 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b53a24a60 con 0x559b53ad2a50 2024-04-23T14:59:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.867+0000 7f29bd400640 1 --2- 172.21.15.83:0/697934201 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x559b53ad2a50 0x559b53ad4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.868+0000 7f29bd400640 1 --2- 172.21.15.83:0/697934201 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x559b53ad2a50 0x559b53ad4e40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=376dad25254ef97f server_cookie=d88df0623b0d5298 in_seq=0 out_seq=0 2024-04-23T14:59:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.868+0000 7f299f400640 10 client.27319 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:59:08.874 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f299f400640 1 -- 172.21.15.83:0/697934201 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b53a24a60 con 0x559b53ad2a50 2024-04-23T14:59:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f299f400640 10 client.27319 handle_command_reply: tid=0 2024-04-23T14:59:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f29c796b740 1 client.27319 shutdown 2024-04-23T14:59:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f29c796b740 1 -- 172.21.15.83:0/697934201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab7950 msgr2=0x559b53ac2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f29c796b740 1 --2- 172.21.15.83:0/697934201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab7950 0x559b53ac2390 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f29b8005f40 tx=0x7f29b80045c0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f29c796b740 1 -- 172.21.15.83:0/697934201 shutdown_connections 2024-04-23T14:59:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f29c796b740 1 --2- 172.21.15.83:0/697934201 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x559b53ad2a50 0x559b53ad4e40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f29c796b740 1 --2- 172.21.15.83:0/697934201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ac2940 0x559b53abfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f29c796b740 1 --2- 172.21.15.83:0/697934201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab8270 0x559b53abf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f299f400640 0 client.27319 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:59:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f29c796b740 1 --2- 172.21.15.83:0/697934201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab7950 0x559b53ac2390 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f29c796b740 1 -- 172.21.15.83:0/697934201 >> 172.21.15.83:0/697934201 conn(0x559b53ab3c70 msgr2=0x559b53ab9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f29c796b740 1 -- 172.21.15.83:0/697934201 shutdown_connections 2024-04-23T14:59:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f29c796b740 1 -- 172.21.15.83:0/697934201 wait complete. 2024-04-23T14:59:08.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.869+0000 7f29c796b740 20 client.27319 trim_cache size 0 max 16384 2024-04-23T14:59:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.884+0000 7f29c796b740 1 Processor -- start 2024-04-23T14:59:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.884+0000 7f29c796b740 1 -- start start 2024-04-23T14:59:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.884+0000 7f29c796b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab7950 0x559b53ace6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.884+0000 7f29c796b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab8270 0x559b53ac1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.884+0000 7f29c796b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ac2940 0x559b53ac2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.884+0000 7f29c796b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01b0750 con 0x559b53ab8270 2024-04-23T14:59:08.885 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.884+0000 7f29c796b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01ae020 con 0x559b53ab7950 2024-04-23T14:59:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.884+0000 7f29c796b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01abe90 con 0x559b53ac2940 2024-04-23T14:59:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.885+0000 7f29bd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab8270 0x559b53ac1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.885+0000 7f29bd400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab8270 0x559b53ac1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55894/0 (socket says 172.21.15.83:55894) 2024-04-23T14:59:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.885+0000 7f29bd400640 1 -- 172.21.15.83:0/788844833 learned_addr learned my addr 172.21.15.83:0/788844833 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.885+0000 7f29be800640 1 --2- 172.21.15.83:0/788844833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab7950 0x559b53ace6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.885+0000 7f29be800640 1 --2- 172.21.15.83:0/788844833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ac2940 0x559b53ac2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.885+0000 7f29bd400640 1 -- 172.21.15.83:0/788844833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ac2940 msgr2=0x559b53ac2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.885+0000 7f29bd400640 1 --2- 172.21.15.83:0/788844833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ac2940 0x559b53ac2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.885+0000 7f29bd400640 1 -- 172.21.15.83:0/788844833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab7950 msgr2=0x559b53ace6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.885+0000 7f29bd400640 1 --2- 172.21.15.83:0/788844833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab7950 0x559b53ace6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.885+0000 7f29bd400640 1 -- 172.21.15.83:0/788844833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b53ab80a0 con 0x559b53ab8270 2024-04-23T14:59:08.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.885+0000 7f29be800640 1 --2- 172.21.15.83:0/788844833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ac2940 0x559b53ac2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:59:08.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.885+0000 7f29bd400640 1 --2- 172.21.15.83:0/788844833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab8270 0x559b53ac1ec0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009a20 tx=0x7f29ac005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68aaabee17523067 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.886+0000 7f29aa600640 1 -- 172.21.15.83:0/788844833 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac035040 con 0x559b53ab8270 2024-04-23T14:59:08.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.886+0000 7f29aa600640 1 -- 172.21.15.83:0/788844833 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29ac032a40 con 0x559b53ab8270 2024-04-23T14:59:08.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.886+0000 7f29aa600640 1 -- 172.21.15.83:0/788844833 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0052a0 con 0x559b53ab8270 2024-04-23T14:59:08.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.887+0000 7f29c796b740 1 -- 172.21.15.83:0/788844833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab8270 msgr2=0x559b53ac1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.887+0000 7f29c796b740 1 --2- 172.21.15.83:0/788844833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab8270 0x559b53ac1ec0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009a20 tx=0x7f29ac005a20 comp rx=0 tx=0).stop 2024-04-23T14:59:08.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.887+0000 7f29c796b740 1 -- 172.21.15.83:0/788844833 shutdown_connections 2024-04-23T14:59:08.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.887+0000 7f29c796b740 1 --2- 172.21.15.83:0/788844833 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ac2940 0x559b53ac2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.887+0000 7f29c796b740 1 --2- 172.21.15.83:0/788844833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ab8270 0x559b53ac1ec0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.887+0000 7f29c796b740 1 --2- 172.21.15.83:0/788844833 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab7950 0x559b53ace6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.887+0000 7f29c796b740 1 -- 172.21.15.83:0/788844833 >> 172.21.15.83:0/788844833 conn(0x559b53ab3c70 msgr2=0x559b53aba940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.887+0000 7f29c796b740 1 -- 172.21.15.83:0/788844833 shutdown_connections 2024-04-23T14:59:08.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.887+0000 7f29c796b740 1 -- 172.21.15.83:0/788844833 wait complete. 2024-04-23T14:59:08.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.887+0000 7f29c796b740 1 -- start start 2024-04-23T14:59:08.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.887+0000 7f29c796b740 1 Processor -- start 2024-04-23T14:59:08.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29c796b740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab7950 0x559b53ac30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29c796b740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab8270 0x559b53ac3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29c796b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ac2940 0x559b53ac9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29c796b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01ac4e0 con 0x559b53ac2940 2024-04-23T14:59:08.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29c796b740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01ad950 con 0x559b53ab7950 2024-04-23T14:59:08.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29c796b740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f29c01acbb0 con 0x559b53ab8270 2024-04-23T14:59:08.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29c796b740 10 client.0 _ll_register_callbacks cb 0x7f29b0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:59:08.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29be800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab7950 0x559b53ac30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.892 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29bd400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab8270 0x559b53ac3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29bd400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab8270 0x559b53ac3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49754/0 (socket says 172.21.15.83:49754) 2024-04-23T14:59:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29bd400640 1 -- 172.21.15.83:0/2354934033 learned_addr learned my addr 172.21.15.83:0/2354934033 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29be800640 1 --2- 172.21.15.83:0/2354934033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ac2940 0x559b53ac9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29bd400640 1 -- 172.21.15.83:0/2354934033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab7950 msgr2=0x559b53ac30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29bd400640 1 --2- 172.21.15.83:0/2354934033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab7950 0x559b53ac30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29bd400640 1 -- 172.21.15.83:0/2354934033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ac2940 msgr2=0x559b53ac9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29bd400640 1 --2- 172.21.15.83:0/2354934033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ac2940 0x559b53ac9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.888+0000 7f29bd400640 1 -- 172.21.15.83:0/2354934033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b537f97d0 con 0x559b53ab8270 2024-04-23T14:59:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.889+0000 7f29bd400640 1 --2- 172.21.15.83:0/2354934033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab8270 0x559b53ac3630 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009230 tx=0x7f29ac005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c84f1160dcc2c559 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:08.893 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.889+0000 7f299f400640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:59:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.889+0000 7f299f400640 1 -- 172.21.15.83:0/2354934033 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac035040 con 0x559b53ab8270 2024-04-23T14:59:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.889+0000 7f29c796b740 1 -- 172.21.15.83:0/2354934033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29ac032a40 con 0x559b53ab8270 2024-04-23T14:59:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.889+0000 7f299f400640 1 -- 172.21.15.83:0/2354934033 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f29ac004240 con 0x559b53ab8270 2024-04-23T14:59:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.889+0000 7f299f400640 1 -- 172.21.15.83:0/2354934033 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29ac05e050 con 0x559b53ab8270 2024-04-23T14:59:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.889+0000 7f29c796b740 10 client.27325 fetch_fsmap learned FSMap version 29 2024-04-23T14:59:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.889+0000 7f29c796b740 1 -- 172.21.15.83:0/2354934033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b53acb020 con 0x559b53ab8270 2024-04-23T14:59:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.889+0000 7f299f400640 1 -- 172.21.15.83:0/2354934033 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29ac032a40 con 0x559b53ab8270 2024-04-23T14:59:08.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.890+0000 7f299f400640 1 -- 172.21.15.83:0/2354934033 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f29b80041d0 con 0x559b53ab8270 2024-04-23T14:59:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.890+0000 7f29c796b740 10 client.27325 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:59:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.890+0000 7f29c796b740 10 client.27325 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-23T14:59:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.890+0000 7f29c796b740 20 client.27325 populate_metadata read hostname 'smithi083' 2024-04-23T14:59:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.890+0000 7f29c796b740 1 --2- 172.21.15.83:0/2354934033 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x559b53b38eb0 0x559b53b3b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.890+0000 7f29c796b740 4 client.27325 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:59:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.890+0000 7f29c796b740 1 -- 172.21.15.83:0/2354934033 --> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b539a04d0 con 0x559b53b38eb0 2024-04-23T14:59:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.890+0000 7f29be800640 1 --2- 172.21.15.83:0/2354934033 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x559b53b38eb0 0x559b53b3b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.890+0000 7f29be800640 1 --2- 172.21.15.83:0/2354934033 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x559b53b38eb0 0x559b53b3b2a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5aaf2dffdb6a2e1 server_cookie=7b7405628e4cbcd9 in_seq=0 out_seq=0 2024-04-23T14:59:08.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.890+0000 7f299f400640 10 client.27325 ms_handle_connect on v2:172.21.15.203:6838/2628587422 2024-04-23T14:59:08.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f299f400640 1 -- 172.21.15.83:0/2354934033 <== mds.2 v2:172.21.15.203:6838/2628587422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b539a04d0 con 0x559b53b38eb0 2024-04-23T14:59:08.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f299f400640 10 client.27325 handle_command_reply: tid=0 2024-04-23T14:59:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f29c796b740 1 client.27325 shutdown 2024-04-23T14:59:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f29c796b740 1 -- 172.21.15.83:0/2354934033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab8270 msgr2=0x559b53ac3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f29c796b740 1 --2- 172.21.15.83:0/2354934033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab8270 0x559b53ac3630 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009230 tx=0x7f29ac005fd0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f29c796b740 1 -- 172.21.15.83:0/2354934033 shutdown_connections 2024-04-23T14:59:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f29c796b740 1 --2- 172.21.15.83:0/2354934033 >> [v2:172.21.15.203:6838/2628587422,v1:172.21.15.203:6839/2628587422] conn(0x559b53b38eb0 0x559b53b3b2a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f29c796b740 1 --2- 172.21.15.83:0/2354934033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b53ac2940 0x559b53ac9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f29c796b740 1 --2- 172.21.15.83:0/2354934033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b53ab8270 0x559b53ac3630 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f29c796b740 1 --2- 172.21.15.83:0/2354934033 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b53ab7950 0x559b53ac30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f299f400640 0 client.27325 ms_handle_reset on v2:172.21.15.203:6838/2628587422 2024-04-23T14:59:08.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f29c796b740 1 -- 172.21.15.83:0/2354934033 >> 172.21.15.83:0/2354934033 conn(0x559b53ab3c70 msgr2=0x559b53abdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f29c796b740 1 -- 172.21.15.83:0/2354934033 shutdown_connections 2024-04-23T14:59:08.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f29c796b740 1 -- 172.21.15.83:0/2354934033 wait complete. 2024-04-23T14:59:08.898 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.891+0000 7f29c796b740 20 client.27325 trim_cache size 0 max 16384 2024-04-23T14:59:08.898 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:59:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.892+0000 7f29c4a00640 1 -- 172.21.15.83:0/824115671 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f29a40a3da0 msgr2=0x7f29a40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.892+0000 7f29c4a00640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f29a40a3da0 0x7f29a40a6260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x559b53ab8f10 tx=0x7f29b00096e0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.892+0000 7f29c4a00640 1 -- 172.21.15.83:0/824115671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c0103280 msgr2=0x7f29c019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.892+0000 7f29c4a00640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c0103280 0x7f29c019e660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f29b000e9f0 tx=0x7f29b000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.892+0000 7f29c4a00640 1 -- 172.21.15.83:0/824115671 shutdown_connections 2024-04-23T14:59:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.892+0000 7f29c4a00640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f29a40a3da0 0x7f29a40a6260 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.892+0000 7f29c4a00640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c010c5c0 0x7f29c0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.892+0000 7f29c4a00640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29c0103ba0 0x7f29c019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.899 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.892+0000 7f29c4a00640 1 --2- 172.21.15.83:0/824115671 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29c0103280 0x7f29c019e660 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.892+0000 7f29c4a00640 1 -- 172.21.15.83:0/824115671 >> 172.21.15.83:0/824115671 conn(0x7f29c0078000 msgr2=0x7f29c00fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.892+0000 7f29c4a00640 1 -- 172.21.15.83:0/824115671 shutdown_connections 2024-04-23T14:59:08.900 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:08.892+0000 7f29c4a00640 1 -- 172.21.15.83:0/824115671 wait complete. 2024-04-23T14:59:08.911 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-23T14:59:09.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.148+0000 7f53d9a00640 1 -- 172.21.15.83:0/2490956555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53d4103060 msgr2=0x7f53d4103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.148+0000 7f53d9a00640 1 --2- 172.21.15.83:0/2490956555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53d4103060 0x7f53d4103440 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009900 tx=0x7f53c402a480 comp rx=0 tx=0).stop 2024-04-23T14:59:09.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.148+0000 7f53d9a00640 1 -- 172.21.15.83:0/2490956555 shutdown_connections 2024-04-23T14:59:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.148+0000 7f53d9a00640 1 --2- 172.21.15.83:0/2490956555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53d410c480 0x7f53d410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.148+0000 7f53d9a00640 1 --2- 172.21.15.83:0/2490956555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d4103980 0x7f53d410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.148+0000 7f53d9a00640 1 --2- 172.21.15.83:0/2490956555 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53d4103060 0x7f53d4103440 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.148+0000 7f53d9a00640 1 -- 172.21.15.83:0/2490956555 >> 172.21.15.83:0/2490956555 conn(0x7f53d40fc880 msgr2=0x7f53d40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.149+0000 7f53d9a00640 1 -- 172.21.15.83:0/2490956555 shutdown_connections 2024-04-23T14:59:09.149 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.149+0000 7f53d9a00640 1 -- 172.21.15.83:0/2490956555 wait complete. 2024-04-23T14:59:09.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.150+0000 7f53d9a00640 1 Processor -- start 2024-04-23T14:59:09.150 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.151+0000 7f53d9a00640 1 -- start start 2024-04-23T14:59:09.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.151+0000 7f53d9a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53d4103060 0x7f53d419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.151+0000 7f53d9a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d4103980 0x7f53d419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.151 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.151+0000 7f53d9a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53d410c480 0x7f53d41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.151+0000 7f53d9a00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4112290 con 0x7f53d4103980 2024-04-23T14:59:09.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.151+0000 7f53d9a00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4112110 con 0x7f53d410c480 2024-04-23T14:59:09.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.151+0000 7f53d9a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4112410 con 0x7f53d4103060 2024-04-23T14:59:09.152 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.152+0000 7f53d3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53d4103060 0x7f53d419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.152+0000 7f53d3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53d4103060 0x7f53d419e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49762/0 (socket says 172.21.15.83:49762) 2024-04-23T14:59:09.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.152+0000 7f53d3e00640 1 -- 172.21.15.83:0/483011706 learned_addr learned my addr 172.21.15.83:0/483011706 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:09.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.152+0000 7f53d9000640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53d410c480 0x7f53d41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.153+0000 7f53d3400640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d4103980 0x7f53d419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.153+0000 7f53d3e00640 1 -- 172.21.15.83:0/483011706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53d410c480 msgr2=0x7f53d41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.153 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.153+0000 7f53d3e00640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53d410c480 0x7f53d41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.153+0000 7f53d3e00640 1 -- 172.21.15.83:0/483011706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d4103980 msgr2=0x7f53d419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.153+0000 7f53d3e00640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d4103980 0x7f53d419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.154+0000 7f53d3e00640 1 -- 172.21.15.83:0/483011706 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d4198d30 con 0x7f53d4103060 2024-04-23T14:59:09.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.154+0000 7f53d9000640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53d410c480 0x7f53d41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:59:09.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.154+0000 7f53d3400640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d4103980 0x7f53d419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T14:59:09.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.155+0000 7f53d3e00640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53d4103060 0x7f53d419e400 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f53c402a9f0 tx=0x7f53c4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:09.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.155+0000 7f53d0c00640 1 -- 172.21.15.83:0/483011706 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c403f070 con 0x7f53d4103060 2024-04-23T14:59:09.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.155+0000 7f53d9a00640 1 -- 172.21.15.83:0/483011706 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d4198fc0 con 0x7f53d4103060 2024-04-23T14:59:09.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.155+0000 7f53d0c00640 1 -- 172.21.15.83:0/483011706 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53c4031c90 con 0x7f53d4103060 2024-04-23T14:59:09.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.155+0000 7f53d0c00640 1 -- 172.21.15.83:0/483011706 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4035030 con 0x7f53d4103060 2024-04-23T14:59:09.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.156+0000 7f53d9a00640 1 -- 172.21.15.83:0/483011706 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d41a52c0 con 0x7f53d4103060 2024-04-23T14:59:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.167+0000 7f53d0c00640 1 -- 172.21.15.83:0/483011706 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f53c4042070 con 0x7f53d4103060 2024-04-23T14:59:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.170+0000 7f53d0c00640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f53b80a3e60 0x7f53b80a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.170+0000 7f53d0c00640 1 -- 172.21.15.83:0/483011706 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8586+0+0 (secure 0 0 0) 0x7f53c40f11e0 con 0x7f53d4103060 2024-04-23T14:59:09.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.171+0000 7f53dde40740 1 Processor -- start 2024-04-23T14:59:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.172+0000 7f53dde40740 1 -- start start 2024-04-23T14:59:09.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.172+0000 7f53d3400640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f53b80a3e60 0x7f53b80a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.173+0000 7f53dde40740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d71307f0 0x55c0d7130bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.173+0000 7f53dde40740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d7131110 0x55c0d71441d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.173+0000 7f53dde40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7144710 0x55c0d7146b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.173+0000 7f53dde40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c0d70961d0 con 0x55c0d7144710 2024-04-23T14:59:09.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.173+0000 7f53dde40740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c0d703c450 con 0x55c0d71307f0 2024-04-23T14:59:09.174 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.173+0000 7f53dde40740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c0d7092190 con 0x55c0d7131110 2024-04-23T14:59:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.174+0000 7f53d3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d71307f0 0x55c0d7130bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.174+0000 7f53d3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d71307f0 0x55c0d7130bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:44886/0 (socket says 172.21.15.83:44886) 2024-04-23T14:59:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.174+0000 7f53d3e00640 1 -- 172.21.15.83:0/3763865812 learned_addr learned my addr 172.21.15.83:0/3763865812 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.174+0000 7f53d9000640 1 --2- 172.21.15.83:0/3763865812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7144710 0x55c0d7146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.174+0000 7f53d3e00640 1 -- 172.21.15.83:0/3763865812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d7131110 msgr2=0x55c0d71441d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:59:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.174+0000 7f53d3e00640 1 --2- 172.21.15.83:0/3763865812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d7131110 0x55c0d71441d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.175 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.174+0000 7f53d3e00640 1 -- 172.21.15.83:0/3763865812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7144710 msgr2=0x55c0d7146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.174+0000 7f53d3e00640 1 --2- 172.21.15.83:0/3763865812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7144710 0x55c0d7146b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.174+0000 7f53d3e00640 1 -- 172.21.15.83:0/3763865812 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d70f9310 con 0x55c0d71307f0 2024-04-23T14:59:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.175+0000 7f53d3e00640 1 --2- 172.21.15.83:0/3763865812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d71307f0 0x55c0d7130bd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f53c40c5820 tx=0x7f53c402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3648560434ede5c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.175+0000 7f53c9600640 1 -- 172.21.15.83:0/3763865812 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c40c1490 con 0x55c0d71307f0 2024-04-23T14:59:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.175+0000 7f53c9600640 1 -- 172.21.15.83:0/3763865812 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53c40c5d50 con 0x55c0d71307f0 2024-04-23T14:59:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.175+0000 7f53c9600640 1 -- 172.21.15.83:0/3763865812 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c40eeb10 con 0x55c0d71307f0 2024-04-23T14:59:09.176 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.175+0000 7f53d3400640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f53b80a3e60 0x7f53b80a6320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55c0d71318c0 tx=0x7f53c0009670 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.176+0000 7f53dde40740 1 -- 172.21.15.83:0/3763865812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d71307f0 msgr2=0x55c0d7130bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.176+0000 7f53dde40740 1 --2- 172.21.15.83:0/3763865812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d71307f0 0x55c0d7130bd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f53c40c5820 tx=0x7f53c402a480 comp rx=0 tx=0).stop 2024-04-23T14:59:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.180+0000 7f53dde40740 1 -- 172.21.15.83:0/3763865812 shutdown_connections 2024-04-23T14:59:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.180+0000 7f53dde40740 1 --2- 172.21.15.83:0/3763865812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7144710 0x55c0d7146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.180+0000 7f53dde40740 1 --2- 172.21.15.83:0/3763865812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d7131110 0x55c0d71441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.180+0000 7f53dde40740 1 --2- 172.21.15.83:0/3763865812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d71307f0 0x55c0d7130bd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.180+0000 7f53dde40740 1 -- 172.21.15.83:0/3763865812 >> 172.21.15.83:0/3763865812 conn(0x55c0d712ce60 msgr2=0x55c0d712d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.182+0000 7f53dde40740 1 -- 172.21.15.83:0/3763865812 shutdown_connections 2024-04-23T14:59:09.181 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.182+0000 7f53dde40740 1 -- 172.21.15.83:0/3763865812 wait complete. 2024-04-23T14:59:09.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.182+0000 7f53dde40740 1 -- start start 2024-04-23T14:59:09.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.183+0000 7f53dde40740 1 Processor -- start 2024-04-23T14:59:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.183+0000 7f53dde40740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d71307f0 0x55c0d7144180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.183 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.183+0000 7f53dde40740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 0x55c0d7141470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.184+0000 7f53dde40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7144710 0x55c0d7141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.184+0000 7f53dde40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c0d703c450 con 0x55c0d7144710 2024-04-23T14:59:09.184 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.184+0000 7f53dde40740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x55c0d704f810 con 0x55c0d7131110 2024-04-23T14:59:09.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.184+0000 7f53dde40740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x55c0d7147040 con 0x55c0d71307f0 2024-04-23T14:59:09.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.184+0000 7f53dde40740 10 client.0 _ll_register_callbacks cb 0x55c0d701d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:59:09.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.184+0000 7f53d3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 0x55c0d7141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.184+0000 7f53d3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 0x55c0d7141470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:44892/0 (socket says 172.21.15.83:44892) 2024-04-23T14:59:09.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.184+0000 7f53d3400640 1 -- 172.21.15.83:0/3334417349 learned_addr learned my addr 172.21.15.83:0/3334417349 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:09.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.184+0000 7f53d3e00640 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d71307f0 0x55c0d7144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.184+0000 7f53d9000640 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7144710 0x55c0d7141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.185+0000 7f53d3400640 1 -- 172.21.15.83:0/3334417349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d71307f0 msgr2=0x55c0d7144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.185+0000 7f53d3400640 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d71307f0 0x55c0d7144180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.185+0000 7f53d3400640 1 -- 172.21.15.83:0/3334417349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7144710 msgr2=0x55c0d7141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.185+0000 7f53d3400640 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7144710 0x55c0d7141bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.185+0000 7f53d3400640 1 -- 172.21.15.83:0/3334417349 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d70603f0 con 0x55c0d7131110 2024-04-23T14:59:09.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.186+0000 7f53d3400640 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 0x55c0d7141470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f53c00116d0 tx=0x7f53c00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1dfa8ed7e1bb482 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:09.187 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.187+0000 7f53be000640 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2024-04-23T14:59:09.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.187+0000 7f53be000640 1 -- 172.21.15.83:0/3334417349 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0022020 con 0x55c0d7131110 2024-04-23T14:59:09.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.187+0000 7f53be000640 1 -- 172.21.15.83:0/3334417349 <== mon.2 v2:172.21.15.173:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8586+0+0 (secure 0 0 0) 0x7f53c0010810 con 0x55c0d7131110 2024-04-23T14:59:09.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.187+0000 7f53dde40740 1 -- 172.21.15.83:0/3334417349 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c40c5d50 con 0x55c0d7131110 2024-04-23T14:59:09.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.188+0000 7f53be000640 1 -- 172.21.15.83:0/3334417349 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53c002d050 con 0x55c0d7131110 2024-04-23T14:59:09.188 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.188+0000 7f53dde40740 10 client.27353 fetch_fsmap learned FSMap version 29 2024-04-23T14:59:09.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.188+0000 7f53dde40740 1 -- 172.21.15.83:0/3334417349 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0d6da83f0 con 0x55c0d7131110 2024-04-23T14:59:09.189 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.188+0000 7f53be000640 1 -- 172.21.15.83:0/3334417349 <== mon.2 v2:172.21.15.173:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53c40c5d50 con 0x55c0d7131110 2024-04-23T14:59:09.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.191+0000 7f53be000640 1 -- 172.21.15.83:0/3334417349 <== mon.2 v2:172.21.15.173:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f53c000f040 con 0x55c0d7131110 2024-04-23T14:59:09.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.191+0000 7f53dde40740 10 client.27353 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:59:09.191 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.191+0000 7f53dde40740 10 client.27353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:59:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.192+0000 7f53dde40740 20 client.27353 populate_metadata read hostname 'smithi083' 2024-04-23T14:59:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.192+0000 7f53dde40740 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55c0d7154330 0x55c0d7156790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.192+0000 7f53dde40740 4 client.27353 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:59:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.192+0000 7f53dde40740 1 -- 172.21.15.83:0/3334417349 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0d70a3bf0 con 0x55c0d7154330 2024-04-23T14:59:09.192 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.192+0000 7f53d3e00640 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55c0d7154330 0x55c0d7156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.193+0000 7f53d3e00640 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55c0d7154330 0x55c0d7156790 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=140f41e59b21d311 server_cookie=4c8f045bd093abb9 in_seq=0 out_seq=0 2024-04-23T14:59:09.193 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.193+0000 7f53be000640 10 client.27353 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:59:09.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.195+0000 7f53be000640 1 -- 172.21.15.83:0/3334417349 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0d70a3bf0 con 0x55c0d7154330 2024-04-23T14:59:09.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.195+0000 7f53be000640 10 client.27353 handle_command_reply: tid=0 2024-04-23T14:59:09.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.195+0000 7f53dde40740 1 client.27353 shutdown 2024-04-23T14:59:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.196+0000 7f53dde40740 1 -- 172.21.15.83:0/3334417349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 msgr2=0x55c0d7141470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.196+0000 7f53dde40740 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 0x55c0d7141470 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f53c00116d0 tx=0x7f53c00046c0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.196+0000 7f53dde40740 1 -- 172.21.15.83:0/3334417349 shutdown_connections 2024-04-23T14:59:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.196+0000 7f53dde40740 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7144710 0x55c0d7141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.196+0000 7f53dde40740 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 0x55c0d7141470 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.196 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.196+0000 7f53dde40740 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55c0d7154330 0x55c0d7156790 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.196+0000 7f53dde40740 1 --2- 172.21.15.83:0/3334417349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d71307f0 0x55c0d7144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.196+0000 7f53dde40740 1 -- 172.21.15.83:0/3334417349 >> 172.21.15.83:0/3334417349 conn(0x55c0d712ce60 msgr2=0x55c0d7132cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.196+0000 7f53be000640 0 client.27353 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:59:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.196+0000 7f53dde40740 1 -- 172.21.15.83:0/3334417349 shutdown_connections 2024-04-23T14:59:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.196+0000 7f53dde40740 1 -- 172.21.15.83:0/3334417349 wait complete. 2024-04-23T14:59:09.197 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.197+0000 7f53dde40740 20 client.27353 trim_cache size 0 max 16384 2024-04-23T14:59:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.225+0000 7f53dde40740 1 Processor -- start 2024-04-23T14:59:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.225+0000 7f53dde40740 1 -- start start 2024-04-23T14:59:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.225+0000 7f53dde40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d71307f0 0x55c0d714fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.225+0000 7f53dde40740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 0x55c0d7143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.225+0000 7f53dde40740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d7144710 0x55c0d715c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.225+0000 7f53dde40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41ae4b0 con 0x55c0d71307f0 2024-04-23T14:59:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.225+0000 7f53dde40740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41abd80 con 0x55c0d7131110 2024-04-23T14:59:09.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.225+0000 7f53dde40740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41a9bf0 con 0x55c0d7144710 2024-04-23T14:59:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.226+0000 7f53d9000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d7144710 0x55c0d715c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.226+0000 7f53d9000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d7144710 0x55c0d715c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49788/0 (socket says 172.21.15.83:49788) 2024-04-23T14:59:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.226+0000 7f53d9000640 1 -- 172.21.15.83:0/3796322313 learned_addr learned my addr 172.21.15.83:0/3796322313 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.226+0000 7f53d3400640 1 --2- 172.21.15.83:0/3796322313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 0x55c0d7143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.226+0000 7f53d3e00640 1 --2- 172.21.15.83:0/3796322313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d71307f0 0x55c0d714fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.227+0000 7f53d9000640 1 -- 172.21.15.83:0/3796322313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 msgr2=0x55c0d7143cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.227+0000 7f53d9000640 1 --2- 172.21.15.83:0/3796322313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 0x55c0d7143cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.227+0000 7f53d9000640 1 -- 172.21.15.83:0/3796322313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d71307f0 msgr2=0x55c0d714fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.227+0000 7f53d9000640 1 --2- 172.21.15.83:0/3796322313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d71307f0 0x55c0d714fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.227+0000 7f53d9000640 1 -- 172.21.15.83:0/3796322313 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d6e727d0 con 0x55c0d7144710 2024-04-23T14:59:09.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.227+0000 7f53d3400640 1 --2- 172.21.15.83:0/3796322313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 0x55c0d7143cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:59:09.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.227+0000 7f53d9000640 1 --2- 172.21.15.83:0/3796322313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d7144710 0x55c0d715c060 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f53cc012770 tx=0x7f53cc012b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=282d738e5f45bd8e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:09.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.228+0000 7f53c8c00640 1 -- 172.21.15.83:0/3796322313 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc003b70 con 0x55c0d7144710 2024-04-23T14:59:09.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.228+0000 7f53c8c00640 1 -- 172.21.15.83:0/3796322313 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53cc010db0 con 0x55c0d7144710 2024-04-23T14:59:09.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.228+0000 7f53c8c00640 1 -- 172.21.15.83:0/3796322313 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc01a430 con 0x55c0d7144710 2024-04-23T14:59:09.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.229+0000 7f53dde40740 1 -- 172.21.15.83:0/3796322313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d7144710 msgr2=0x55c0d715c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.229+0000 7f53dde40740 1 --2- 172.21.15.83:0/3796322313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d7144710 0x55c0d715c060 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f53cc012770 tx=0x7f53cc012b70 comp rx=0 tx=0).stop 2024-04-23T14:59:09.229 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.230+0000 7f53dde40740 1 -- 172.21.15.83:0/3796322313 shutdown_connections 2024-04-23T14:59:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.230+0000 7f53dde40740 1 --2- 172.21.15.83:0/3796322313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d7144710 0x55c0d715c060 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.230+0000 7f53dde40740 1 --2- 172.21.15.83:0/3796322313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7131110 0x55c0d7143cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.230+0000 7f53dde40740 1 --2- 172.21.15.83:0/3796322313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d71307f0 0x55c0d714fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.230+0000 7f53dde40740 1 -- 172.21.15.83:0/3796322313 >> 172.21.15.83:0/3796322313 conn(0x55c0d712ce60 msgr2=0x55c0d71401b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:09.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.230+0000 7f53dde40740 1 -- 172.21.15.83:0/3796322313 shutdown_connections 2024-04-23T14:59:09.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.230+0000 7f53dde40740 1 -- 172.21.15.83:0/3796322313 wait complete. 2024-04-23T14:59:09.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.231+0000 7f53dde40740 1 -- start start 2024-04-23T14:59:09.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.231+0000 7f53dde40740 1 Processor -- start 2024-04-23T14:59:09.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.232+0000 7f53dde40740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d71307f0 0x55c0d714c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.232+0000 7f53dde40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7131110 0x55c0d7135ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.232+0000 7f53dde40740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7144710 0x55c0d7136550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.232 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.232+0000 7f53dde40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41aa240 con 0x55c0d7131110 2024-04-23T14:59:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.232+0000 7f53dde40740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41ab6b0 con 0x55c0d7144710 2024-04-23T14:59:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.232+0000 7f53dde40740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f53d41aa910 con 0x55c0d71307f0 2024-04-23T14:59:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.232+0000 7f53dde40740 10 client.0 _ll_register_callbacks cb 0x7f53d406a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:59:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.232+0000 7f53d9000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7144710 0x55c0d7136550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.232+0000 7f53d3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d71307f0 0x55c0d714c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.232+0000 7f53d9000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7144710 0x55c0d7136550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:44902/0 (socket says 172.21.15.83:44902) 2024-04-23T14:59:09.233 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.233+0000 7f53d9000640 1 -- 172.21.15.83:0/1634339190 learned_addr learned my addr 172.21.15.83:0/1634339190 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.233+0000 7f53d3400640 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7131110 0x55c0d7135ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.233+0000 7f53d3e00640 1 -- 172.21.15.83:0/1634339190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7144710 msgr2=0x55c0d7136550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.233+0000 7f53d3e00640 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7144710 0x55c0d7136550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.233+0000 7f53d3e00640 1 -- 172.21.15.83:0/1634339190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7131110 msgr2=0x55c0d7135ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.233+0000 7f53d3e00640 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7131110 0x55c0d7135ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.233+0000 7f53d3e00640 1 -- 172.21.15.83:0/1634339190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d71458b0 con 0x55c0d71307f0 2024-04-23T14:59:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.233+0000 7f53d9000640 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7144710 0x55c0d7136550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:59:09.239 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.234+0000 7f53d3e00640 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d71307f0 0x55c0d714c7f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f53c4005ab0 tx=0x7f53c4005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e67a15d7987b5557 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.234+0000 7f53be000640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:59:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.234+0000 7f53be000640 1 -- 172.21.15.83:0/1634339190 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c40bb3e0 con 0x55c0d71307f0 2024-04-23T14:59:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.234+0000 7f53be000640 1 -- 172.21.15.83:0/1634339190 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8586+0+0 (secure 0 0 0) 0x7f53c40c1130 con 0x55c0d71307f0 2024-04-23T14:59:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.234+0000 7f53dde40740 1 -- 172.21.15.83:0/1634339190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53cc010db0 con 0x55c0d71307f0 2024-04-23T14:59:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.234+0000 7f53be000640 1 -- 172.21.15.83:0/1634339190 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53c403b460 con 0x55c0d71307f0 2024-04-23T14:59:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.234+0000 7f53dde40740 10 client.27340 fetch_fsmap learned FSMap version 29 2024-04-23T14:59:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.235+0000 7f53dde40740 1 -- 172.21.15.83:0/1634339190 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0d71cbdd0 con 0x55c0d71307f0 2024-04-23T14:59:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.235+0000 7f53be000640 1 -- 172.21.15.83:0/1634339190 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53cc010db0 con 0x55c0d71307f0 2024-04-23T14:59:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.235+0000 7f53be000640 1 -- 172.21.15.83:0/1634339190 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f53c404c060 con 0x55c0d71307f0 2024-04-23T14:59:09.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.235+0000 7f53dde40740 10 client.27340 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:59:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.235+0000 7f53dde40740 10 client.27340 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-04-23T14:59:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.235+0000 7f53dde40740 20 client.27340 populate_metadata read hostname 'smithi083' 2024-04-23T14:59:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.235+0000 7f53dde40740 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55c0d71cc170 0x55c0d71ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.235+0000 7f53dde40740 4 client.27340 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:59:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.235+0000 7f53dde40740 1 -- 172.21.15.83:0/1634339190 --> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0d709da60 con 0x55c0d71cc170 2024-04-23T14:59:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.236+0000 7f53d3400640 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55c0d71cc170 0x55c0d71ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.236+0000 7f53d3400640 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55c0d71cc170 0x55c0d71ce560 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a64221efe6868be server_cookie=1534face3da88299 in_seq=0 out_seq=0 2024-04-23T14:59:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.236+0000 7f53be000640 10 client.27340 ms_handle_connect on v2:172.21.15.173:6836/3951862333 2024-04-23T14:59:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.236+0000 7f53be000640 1 -- 172.21.15.83:0/1634339190 <== mds.1 v2:172.21.15.173:6836/3951862333 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0d709da60 con 0x55c0d71cc170 2024-04-23T14:59:09.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.236+0000 7f53be000640 10 client.27340 handle_command_reply: tid=0 2024-04-23T14:59:09.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.236+0000 7f53dde40740 1 client.27340 shutdown 2024-04-23T14:59:09.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.237+0000 7f53dde40740 1 -- 172.21.15.83:0/1634339190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d71307f0 msgr2=0x55c0d714c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.237+0000 7f53dde40740 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d71307f0 0x55c0d714c7f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f53c4005ab0 tx=0x7f53c4005d50 comp rx=0 tx=0).stop 2024-04-23T14:59:09.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.237+0000 7f53dde40740 1 -- 172.21.15.83:0/1634339190 shutdown_connections 2024-04-23T14:59:09.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.237+0000 7f53dde40740 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55c0d7144710 0x55c0d7136550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.237+0000 7f53dde40740 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c0d7131110 0x55c0d7135ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.237+0000 7f53dde40740 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.173:6836/3951862333,v1:172.21.15.173:6837/3951862333] conn(0x55c0d71cc170 0x55c0d71ce560 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.237+0000 7f53dde40740 1 --2- 172.21.15.83:0/1634339190 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c0d71307f0 0x55c0d714c7f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.237+0000 7f53dde40740 1 -- 172.21.15.83:0/1634339190 >> 172.21.15.83:0/1634339190 conn(0x55c0d712ce60 msgr2=0x55c0d71323f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:09.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.237+0000 7f53be000640 0 client.27340 ms_handle_reset on v2:172.21.15.173:6836/3951862333 2024-04-23T14:59:09.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.237+0000 7f53dde40740 1 -- 172.21.15.83:0/1634339190 shutdown_connections 2024-04-23T14:59:09.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.237+0000 7f53dde40740 1 -- 172.21.15.83:0/1634339190 wait complete. 2024-04-23T14:59:09.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.237+0000 7f53dde40740 20 client.27340 trim_cache size 0 max 16384 2024-04-23T14:59:09.243 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:59:09.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.238+0000 7f53d9a00640 1 -- 172.21.15.83:0/483011706 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f53b80a3e60 msgr2=0x7f53b80a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.238+0000 7f53d9a00640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f53b80a3e60 0x7f53b80a6320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55c0d71318c0 tx=0x7f53c0009670 comp rx=0 tx=0).stop 2024-04-23T14:59:09.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.238+0000 7f53d9a00640 1 -- 172.21.15.83:0/483011706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53d4103060 msgr2=0x7f53d419e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.238+0000 7f53d9a00640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53d4103060 0x7f53d419e400 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f53c402a9f0 tx=0x7f53c4003b90 comp rx=0 tx=0).stop 2024-04-23T14:59:09.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.238+0000 7f53d9a00640 1 -- 172.21.15.83:0/483011706 shutdown_connections 2024-04-23T14:59:09.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.238+0000 7f53d9a00640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53d410c480 0x7f53d41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.238+0000 7f53d9a00640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f53b80a3e60 0x7f53b80a6320 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.238+0000 7f53d9a00640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f53d4103980 0x7f53d419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.238+0000 7f53d9a00640 1 --2- 172.21.15.83:0/483011706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53d4103060 0x7f53d419e400 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.238+0000 7f53d9a00640 1 -- 172.21.15.83:0/483011706 >> 172.21.15.83:0/483011706 conn(0x7f53d40fc880 msgr2=0x7f53d40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:09.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.239+0000 7f53d9a00640 1 -- 172.21.15.83:0/483011706 shutdown_connections 2024-04-23T14:59:09.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.239+0000 7f53d9a00640 1 -- 172.21.15.83:0/483011706 wait complete. 2024-04-23T14:59:09.254 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:09 smithi083 ceph-mon[27867]: pgmap v332: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-23T14:59:09.254 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:09 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2464647400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:59:09.256 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-23T14:59:09.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:09 smithi173 ceph-mon[32443]: pgmap v332: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-23T14:59:09.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:09 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2464647400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:59:09.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.430+0000 7f0702600640 1 -- 172.21.15.83:0/3413203060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06fc101770 msgr2=0x7f06fc105c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.430 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.430+0000 7f0702600640 1 --2- 172.21.15.83:0/3413203060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06fc101770 0x7f06fc105c30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009900 tx=0x7f06e802a470 comp rx=0 tx=0).stop 2024-04-23T14:59:09.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.430+0000 7f0702600640 1 -- 172.21.15.83:0/3413203060 shutdown_connections 2024-04-23T14:59:09.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.430+0000 7f0702600640 1 --2- 172.21.15.83:0/3413203060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06fc106250 0x7f06fc1066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.430+0000 7f0702600640 1 --2- 172.21.15.83:0/3413203060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06fc101770 0x7f06fc105c30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.430+0000 7f0702600640 1 --2- 172.21.15.83:0/3413203060 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06fc100e50 0x7f06fc101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.430+0000 7f0702600640 1 -- 172.21.15.83:0/3413203060 >> 172.21.15.83:0/3413203060 conn(0x7f06fc0fc880 msgr2=0x7f06fc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:09.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.430+0000 7f0702600640 1 -- 172.21.15.83:0/3413203060 shutdown_connections 2024-04-23T14:59:09.431 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.430+0000 7f0702600640 1 -- 172.21.15.83:0/3413203060 wait complete. 2024-04-23T14:59:09.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.431+0000 7f0702600640 1 Processor -- start 2024-04-23T14:59:09.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.431+0000 7f0702600640 1 -- start start 2024-04-23T14:59:09.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f0702600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06fc100e50 0x7f06fc19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f0702600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06fc101770 0x7f06fc19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f0702600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06fc106250 0x7f06fc1a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f0702600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1188c0 con 0x7f06fc106250 2024-04-23T14:59:09.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f0702600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc118740 con 0x7f06fc101770 2024-04-23T14:59:09.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f0702600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc118a40 con 0x7f06fc100e50 2024-04-23T14:59:09.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f06fbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06fc101770 0x7f06fc19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f06fbe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06fc101770 0x7f06fc19f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:44924/0 (socket says 172.21.15.83:44924) 2024-04-23T14:59:09.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f06fbe00640 1 -- 172.21.15.83:0/3439318255 learned_addr learned my addr 172.21.15.83:0/3439318255 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:09.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f0701200640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06fc100e50 0x7f06fc19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f0701c00640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06fc106250 0x7f06fc1a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f0701200640 1 -- 172.21.15.83:0/3439318255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06fc101770 msgr2=0x7f06fc19f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.432+0000 7f0701200640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06fc101770 0x7f06fc19f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.433+0000 7f0701200640 1 -- 172.21.15.83:0/3439318255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06fc106250 msgr2=0x7f06fc1a3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.433+0000 7f0701200640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06fc106250 0x7f06fc1a3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.433+0000 7f0701200640 1 -- 172.21.15.83:0/3439318255 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc1a3db0 con 0x7f06fc100e50 2024-04-23T14:59:09.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.433+0000 7f0701200640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06fc100e50 0x7f06fc19edc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f06f000e9f0 tx=0x7f06f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:09.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.433+0000 7f06f9600640 1 -- 172.21.15.83:0/3439318255 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f000ccc0 con 0x7f06fc100e50 2024-04-23T14:59:09.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.433+0000 7f06f9600640 1 -- 172.21.15.83:0/3439318255 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f0010910 con 0x7f06fc100e50 2024-04-23T14:59:09.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.433+0000 7f06f9600640 1 -- 172.21.15.83:0/3439318255 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0018960 con 0x7f06fc100e50 2024-04-23T14:59:09.435 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.433+0000 7f0702600640 1 -- 172.21.15.83:0/3439318255 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc1a40a0 con 0x7f06fc100e50 2024-04-23T14:59:09.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.433+0000 7f0702600640 1 -- 172.21.15.83:0/3439318255 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc1ab930 con 0x7f06fc100e50 2024-04-23T14:59:09.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.436+0000 7f0706a0a740 1 Processor -- start 2024-04-23T14:59:09.436 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.436+0000 7f0706a0a740 1 -- start start 2024-04-23T14:59:09.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.437+0000 7f0706a0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 0x559b38a64f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.437+0000 7f0706a0a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a65460 0x559b38a6f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.437+0000 7f0706a0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 0x559b38a71f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.437+0000 7f0706a0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b389ca1d0 con 0x559b38a6fb30 2024-04-23T14:59:09.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.437+0000 7f0706a0a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559b38970450 con 0x559b38a65460 2024-04-23T14:59:09.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.437+0000 7f0706a0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559b389c6190 con 0x559b38a64b40 2024-04-23T14:59:09.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.437+0000 7f06fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 0x559b38a71f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.437+0000 7f0701c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 0x559b38a64f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.437+0000 7f06fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 0x559b38a71f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:55992/0 (socket says 172.21.15.83:55992) 2024-04-23T14:59:09.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.437+0000 7f06fbe00640 1 -- 172.21.15.83:0/3566376825 learned_addr learned my addr 172.21.15.83:0/3566376825 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:09.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.438+0000 7f0701c00640 1 -- 172.21.15.83:0/3566376825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a65460 msgr2=0x559b38a6f5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T14:59:09.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.438+0000 7f0701c00640 1 --2- 172.21.15.83:0/3566376825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a65460 0x559b38a6f5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.438+0000 7f0701c00640 1 -- 172.21.15.83:0/3566376825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 msgr2=0x559b38a71f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.438+0000 7f0701c00640 1 --2- 172.21.15.83:0/3566376825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 0x559b38a71f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.438+0000 7f0701c00640 1 -- 172.21.15.83:0/3566376825 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b38a2d310 con 0x559b38a64b40 2024-04-23T14:59:09.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.438+0000 7f06fbe00640 1 --2- 172.21.15.83:0/3566376825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 0x559b38a71f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:59:09.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.438+0000 7f06f9600640 1 -- 172.21.15.83:0/3439318255 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f06f0010ac0 con 0x7f06fc100e50 2024-04-23T14:59:09.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.438+0000 7f0701c00640 1 --2- 172.21.15.83:0/3566376825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 0x559b38a64f20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009f00 tx=0x7f06f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6839d36fb814ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:09.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.438+0000 7f06ee000640 1 -- 172.21.15.83:0/3566376825 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4004d20 con 0x559b38a64b40 2024-04-23T14:59:09.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.439+0000 7f06ee000640 1 -- 172.21.15.83:0/3566376825 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f4004050 con 0x559b38a64b40 2024-04-23T14:59:09.440 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.439+0000 7f06ee000640 1 -- 172.21.15.83:0/3566376825 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4005280 con 0x559b38a64b40 2024-04-23T14:59:09.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.439+0000 7f06f9600640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06e40a3da0 0x7f06e40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.441 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.440+0000 7f0706a0a740 1 -- 172.21.15.83:0/3566376825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 msgr2=0x559b38a64f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.440+0000 7f0706a0a740 1 --2- 172.21.15.83:0/3566376825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 0x559b38a64f20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009f00 tx=0x7f06f4011810 comp rx=0 tx=0).stop 2024-04-23T14:59:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.440+0000 7f06f9600640 1 -- 172.21.15.83:0/3439318255 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8586+0+0 (secure 0 0 0) 0x7f06f00d0b80 con 0x7f06fc100e50 2024-04-23T14:59:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.440+0000 7f0701200640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06e40a3da0 0x7f06e40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.440+0000 7f0706a0a740 1 -- 172.21.15.83:0/3566376825 shutdown_connections 2024-04-23T14:59:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.440+0000 7f0706a0a740 1 --2- 172.21.15.83:0/3566376825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 0x559b38a71f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.440+0000 7f0706a0a740 1 --2- 172.21.15.83:0/3566376825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a65460 0x559b38a6f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.440+0000 7f0706a0a740 1 --2- 172.21.15.83:0/3566376825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 0x559b38a64f20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.440+0000 7f0706a0a740 1 -- 172.21.15.83:0/3566376825 >> 172.21.15.83:0/3566376825 conn(0x559b38a60e60 msgr2=0x559b38a61240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.440+0000 7f0706a0a740 1 -- 172.21.15.83:0/3566376825 shutdown_connections 2024-04-23T14:59:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.440+0000 7f0706a0a740 1 -- 172.21.15.83:0/3566376825 wait complete. 2024-04-23T14:59:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.441+0000 7f0706a0a740 1 -- start start 2024-04-23T14:59:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.441+0000 7f0701200640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06e40a3da0 0x7f06e40a6260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x559b38a66100 tx=0x7f06f009c6c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.441+0000 7f0706a0a740 1 Processor -- start 2024-04-23T14:59:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.441+0000 7f0706a0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a64b40 0x559b38a6c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.441+0000 7f0706a0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a65460 0x559b38a6ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.441+0000 7f0706a0a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a6fb30 0x559b38a6d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.443 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.441+0000 7f0706a0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b38a729c0 con 0x559b38a64b40 2024-04-23T14:59:09.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.441+0000 7f0706a0a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x559b38983810 con 0x559b38a6fb30 2024-04-23T14:59:09.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.441+0000 7f0706a0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x559b38a72b40 con 0x559b38a65460 2024-04-23T14:59:09.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.441+0000 7f0706a0a740 10 client.0 _ll_register_callbacks cb 0x559b38951140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:59:09.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.442+0000 7f0701c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a64b40 0x559b38a6c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.442+0000 7f0701c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a64b40 0x559b38a6c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56004/0 (socket says 172.21.15.83:56004) 2024-04-23T14:59:09.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.442+0000 7f0701c00640 1 -- 172.21.15.83:0/1934030953 learned_addr learned my addr 172.21.15.83:0/1934030953 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:09.444 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.442+0000 7f06fbe00640 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a65460 0x559b38a6ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.442+0000 7f0701c00640 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a6fb30 0x559b38a6d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.443+0000 7f06fbe00640 1 -- 172.21.15.83:0/1934030953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a6fb30 msgr2=0x559b38a6d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.443+0000 7f06fbe00640 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a6fb30 0x559b38a6d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.443+0000 7f06fbe00640 1 -- 172.21.15.83:0/1934030953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a64b40 msgr2=0x559b38a6c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.443+0000 7f06fbe00640 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a64b40 0x559b38a6c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.443+0000 7f06fbe00640 1 -- 172.21.15.83:0/1934030953 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b389943f0 con 0x559b38a65460 2024-04-23T14:59:09.446 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.443+0000 7f0701c00640 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a6fb30 0x559b38a6d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:59:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.444+0000 7f06fbe00640 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a65460 0x559b38a6ce60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f06e802aa60 tx=0x7f06e8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4acaf03177be662 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.444+0000 7f06e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:59:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.444+0000 7f06e2a00640 1 -- 172.21.15.83:0/1934030953 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8035040 con 0x559b38a65460 2024-04-23T14:59:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.444+0000 7f06e2a00640 1 -- 172.21.15.83:0/1934030953 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8586+0+0 (secure 0 0 0) 0x7f06e80035f0 con 0x559b38a65460 2024-04-23T14:59:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.445+0000 7f0706a0a740 1 -- 172.21.15.83:0/1934030953 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06f4004050 con 0x559b38a65460 2024-04-23T14:59:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.445+0000 7f06e2a00640 1 -- 172.21.15.83:0/1934030953 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06e80037f0 con 0x559b38a65460 2024-04-23T14:59:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.445+0000 7f0706a0a740 10 client.27352 fetch_fsmap learned FSMap version 29 2024-04-23T14:59:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.445+0000 7f0706a0a740 1 -- 172.21.15.83:0/1934030953 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b386dc3f0 con 0x559b38a65460 2024-04-23T14:59:09.447 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.446+0000 7f06e2a00640 1 -- 172.21.15.83:0/1934030953 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f4004050 con 0x559b38a65460 2024-04-23T14:59:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.446+0000 7f06e2a00640 1 -- 172.21.15.83:0/1934030953 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f06e8031c30 con 0x559b38a65460 2024-04-23T14:59:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.446+0000 7f0706a0a740 10 client.27352 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:59:09.448 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.446+0000 7f0706a0a740 10 client.27352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:59:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.446+0000 7f0706a0a740 20 client.27352 populate_metadata read hostname 'smithi083' 2024-04-23T14:59:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.446+0000 7f0706a0a740 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b38a7fdb0 0x559b38a821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.446+0000 7f0706a0a740 4 client.27352 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T14:59:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.446+0000 7f0706a0a740 1 -- 172.21.15.83:0/1934030953 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b389d7bf0 con 0x559b38a7fdb0 2024-04-23T14:59:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.447+0000 7f0701c00640 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b38a7fdb0 0x559b38a821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.448+0000 7f0701c00640 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b38a7fdb0 0x559b38a821a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d16e2a5369845df3 server_cookie=f034c61870a3951c in_seq=0 out_seq=0 2024-04-23T14:59:09.449 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.448+0000 7f06e2a00640 10 client.27352 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:59:09.450 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.449+0000 7f06e2a00640 1 -- 172.21.15.83:0/1934030953 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b389d7bf0 con 0x559b38a7fdb0 2024-04-23T14:59:09.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.449+0000 7f06e2a00640 10 client.27352 handle_command_reply: tid=0 2024-04-23T14:59:09.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.449+0000 7f0706a0a740 1 client.27352 shutdown 2024-04-23T14:59:09.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.449+0000 7f0706a0a740 1 -- 172.21.15.83:0/1934030953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a65460 msgr2=0x559b38a6ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.449+0000 7f0706a0a740 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a65460 0x559b38a6ce60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f06e802aa60 tx=0x7f06e8004270 comp rx=0 tx=0).stop 2024-04-23T14:59:09.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.450+0000 7f0706a0a740 1 -- 172.21.15.83:0/1934030953 shutdown_connections 2024-04-23T14:59:09.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.450+0000 7f0706a0a740 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b38a7fdb0 0x559b38a821a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.450+0000 7f0706a0a740 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a6fb30 0x559b38a6d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.451 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.450+0000 7f06e2a00640 0 client.27352 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:59:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.450+0000 7f0706a0a740 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a65460 0x559b38a6ce60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.450+0000 7f0706a0a740 1 --2- 172.21.15.83:0/1934030953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a64b40 0x559b38a6c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.450+0000 7f0706a0a740 1 -- 172.21.15.83:0/1934030953 >> 172.21.15.83:0/1934030953 conn(0x559b38a60e60 msgr2=0x559b38a66c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.450+0000 7f0706a0a740 1 -- 172.21.15.83:0/1934030953 shutdown_connections 2024-04-23T14:59:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.450+0000 7f0706a0a740 1 -- 172.21.15.83:0/1934030953 wait complete. 2024-04-23T14:59:09.452 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.450+0000 7f0706a0a740 20 client.27352 trim_cache size 0 max 16384 2024-04-23T14:59:09.469 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.465+0000 7f0706a0a740 1 Processor -- start 2024-04-23T14:59:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.465+0000 7f0706a0a740 1 -- start start 2024-04-23T14:59:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.465+0000 7f0706a0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 0x559b38a7b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.465+0000 7f0706a0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a65460 0x559b38a6ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.465+0000 7f0706a0a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a6fb30 0x559b38a6cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.465+0000 7f0706a0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1b6b30 con 0x559b38a65460 2024-04-23T14:59:09.470 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.465+0000 7f0706a0a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1b4400 con 0x559b38a6fb30 2024-04-23T14:59:09.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.465+0000 7f0706a0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1b2270 con 0x559b38a64b40 2024-04-23T14:59:09.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.466+0000 7f06fbe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a65460 0x559b38a6ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.466+0000 7f0701c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 0x559b38a7b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.466+0000 7f0701c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 0x559b38a7b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49846/0 (socket says 172.21.15.83:49846) 2024-04-23T14:59:09.471 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.466+0000 7f0701c00640 1 -- 172.21.15.83:0/1858693325 learned_addr learned my addr 172.21.15.83:0/1858693325 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.466+0000 7f0701c00640 1 --2- 172.21.15.83:0/1858693325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a6fb30 0x559b38a6cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.466+0000 7f0701c00640 1 -- 172.21.15.83:0/1858693325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a6fb30 msgr2=0x559b38a6cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.466+0000 7f0701c00640 1 --2- 172.21.15.83:0/1858693325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a6fb30 0x559b38a6cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.466+0000 7f0701c00640 1 -- 172.21.15.83:0/1858693325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a65460 msgr2=0x559b38a6ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.523 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.466+0000 7f0701c00640 1 --2- 172.21.15.83:0/1858693325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a65460 0x559b38a6ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.466+0000 7f0701c00640 1 -- 172.21.15.83:0/1858693325 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b387a67d0 con 0x559b38a64b40 2024-04-23T14:59:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.466+0000 7f0701c00640 1 --2- 172.21.15.83:0/1858693325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 0x559b38a7b880 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f06f4006010 tx=0x7f06f4004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44e0bd9469fad607 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.467+0000 7f06ed600640 1 -- 172.21.15.83:0/1858693325 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4030e10 con 0x559b38a64b40 2024-04-23T14:59:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.467+0000 7f06ed600640 1 -- 172.21.15.83:0/1858693325 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f40301d0 con 0x559b38a64b40 2024-04-23T14:59:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.467+0000 7f06ed600640 1 -- 172.21.15.83:0/1858693325 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f40286b0 con 0x559b38a64b40 2024-04-23T14:59:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.467+0000 7f0706a0a740 1 -- 172.21.15.83:0/1858693325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 msgr2=0x559b38a7b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.467+0000 7f0706a0a740 1 --2- 172.21.15.83:0/1858693325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 0x559b38a7b880 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f06f4006010 tx=0x7f06f4004790 comp rx=0 tx=0).stop 2024-04-23T14:59:09.524 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.467+0000 7f0706a0a740 1 -- 172.21.15.83:0/1858693325 shutdown_connections 2024-04-23T14:59:09.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.467+0000 7f0706a0a740 1 --2- 172.21.15.83:0/1858693325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a6fb30 0x559b38a6cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.467+0000 7f0706a0a740 1 --2- 172.21.15.83:0/1858693325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a65460 0x559b38a6ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.467+0000 7f0706a0a740 1 --2- 172.21.15.83:0/1858693325 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a64b40 0x559b38a7b880 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.467+0000 7f0706a0a740 1 -- 172.21.15.83:0/1858693325 >> 172.21.15.83:0/1858693325 conn(0x559b38a60e60 msgr2=0x559b38a6c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:09.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.468+0000 7f0706a0a740 1 -- 172.21.15.83:0/1858693325 shutdown_connections 2024-04-23T14:59:09.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.468+0000 7f0706a0a740 1 -- 172.21.15.83:0/1858693325 wait complete. 2024-04-23T14:59:09.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.468+0000 7f0706a0a740 1 -- start start 2024-04-23T14:59:09.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.468+0000 7f0706a0a740 1 Processor -- start 2024-04-23T14:59:09.525 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f0706a0a740 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a64b40 0x559b38a70700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f0706a0a740 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a65460 0x559b38a70c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f0706a0a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 0x559b38a76d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f0706a0a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1b28c0 con 0x559b38a6fb30 2024-04-23T14:59:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f0706a0a740 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1b3d30 con 0x559b38a64b40 2024-04-23T14:59:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f0706a0a740 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc1b2f90 con 0x559b38a65460 2024-04-23T14:59:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f0706a0a740 10 client.0 _ll_register_callbacks cb 0x7f06f40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T14:59:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f0701c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 0x559b38a76d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f0701c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 0x559b38a76d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56020/0 (socket says 172.21.15.83:56020) 2024-04-23T14:59:09.526 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f0701c00640 1 -- 172.21.15.83:0/782334262 learned_addr learned my addr 172.21.15.83:0/782334262 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f06fbe00640 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a65460 0x559b38a70c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f0701c00640 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a64b40 0x559b38a70700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f06fbe00640 1 -- 172.21.15.83:0/782334262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a64b40 msgr2=0x559b38a70700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f06fbe00640 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a64b40 0x559b38a70700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f06fbe00640 1 -- 172.21.15.83:0/782334262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 msgr2=0x559b38a76d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T14:59:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f06fbe00640 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 0x559b38a76d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.469+0000 7f06fbe00640 1 -- 172.21.15.83:0/782334262 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b38a77350 con 0x559b38a65460 2024-04-23T14:59:09.527 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.470+0000 7f06fbe00640 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a65460 0x559b38a70c40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f06e80096c0 tx=0x7f06e80468e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=755814377dcfff27 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:09.528 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.470+0000 7f06e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.203:3300/0 2024-04-23T14:59:09.528 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:09 smithi203 ceph-mon[32800]: pgmap v332: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-04-23T14:59:09.529 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:09 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2464647400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:59:09.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.470+0000 7f06e2a00640 1 -- 172.21.15.83:0/782334262 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8055070 con 0x559b38a65460 2024-04-23T14:59:09.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.470+0000 7f0706a0a740 1 -- 172.21.15.83:0/782334262 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06f40301d0 con 0x559b38a65460 2024-04-23T14:59:09.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.470+0000 7f06e2a00640 1 -- 172.21.15.83:0/782334262 <== mon.1 v2:172.21.15.203:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8586+0+0 (secure 0 0 0) 0x7f06e8003990 con 0x559b38a65460 2024-04-23T14:59:09.531 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.470+0000 7f06e2a00640 1 -- 172.21.15.83:0/782334262 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06e8035040 con 0x559b38a65460 2024-04-23T14:59:09.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.470+0000 7f0706a0a740 10 client.27358 fetch_fsmap learned FSMap version 29 2024-04-23T14:59:09.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.470+0000 7f0706a0a740 1 -- 172.21.15.83:0/782334262 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b38a78650 con 0x559b38a65460 2024-04-23T14:59:09.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.471+0000 7f06e2a00640 1 -- 172.21.15.83:0/782334262 <== mon.1 v2:172.21.15.203:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06f40301d0 con 0x559b38a65460 2024-04-23T14:59:09.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.471+0000 7f06e2a00640 1 -- 172.21.15.83:0/782334262 <== mon.1 v2:172.21.15.203:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f06e8031a90 con 0x559b38a65460 2024-04-23T14:59:09.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.471+0000 7f0706a0a740 10 client.27358 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T14:59:09.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.471+0000 7f0706a0a740 10 client.27358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-23T14:59:09.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.471+0000 7f0706a0a740 20 client.27358 populate_metadata read hostname 'smithi083' 2024-04-23T14:59:09.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.471+0000 7f0706a0a740 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b38ae6870 0x559b38ae8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:09.532 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.471+0000 7f0706a0a740 4 client.27358 mds_command: new command op to 24599 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-23T14:59:09.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.471+0000 7f0706a0a740 1 -- 172.21.15.83:0/782334262 --> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b389d1a60 con 0x559b38ae6870 2024-04-23T14:59:09.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.471+0000 7f0701c00640 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b38ae6870 0x559b38ae8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:09.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f0701c00640 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b38ae6870 0x559b38ae8c90 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1cc03b30ae1126e server_cookie=d823630995ebeb70 in_seq=0 out_seq=0 2024-04-23T14:59:09.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f06e2a00640 10 client.27358 ms_handle_connect on v2:172.21.15.173:6838/2485025097 2024-04-23T14:59:09.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f06e2a00640 1 -- 172.21.15.83:0/782334262 <== mds.0 v2:172.21.15.173:6838/2485025097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b389d1a60 con 0x559b38ae6870 2024-04-23T14:59:09.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f06e2a00640 10 client.27358 handle_command_reply: tid=0 2024-04-23T14:59:09.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f0706a0a740 1 client.27358 shutdown 2024-04-23T14:59:09.533 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f0706a0a740 1 -- 172.21.15.83:0/782334262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a65460 msgr2=0x559b38a70c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f0706a0a740 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a65460 0x559b38a70c40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f06e80096c0 tx=0x7f06e80468e0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f0706a0a740 1 -- 172.21.15.83:0/782334262 shutdown_connections 2024-04-23T14:59:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f0706a0a740 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.173:6838/2485025097,v1:172.21.15.173:6839/2485025097] conn(0x559b38ae6870 0x559b38ae8c90 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f0706a0a740 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b38a6fb30 0x559b38a76d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f0706a0a740 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559b38a65460 0x559b38a70c40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f06e2a00640 0 client.27358 ms_handle_reset on v2:172.21.15.173:6838/2485025097 2024-04-23T14:59:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f0706a0a740 1 --2- 172.21.15.83:0/782334262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x559b38a64b40 0x559b38a70700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.472+0000 7f0706a0a740 1 -- 172.21.15.83:0/782334262 >> 172.21.15.83:0/782334262 conn(0x559b38a60e60 msgr2=0x559b38a66b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:09.534 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.473+0000 7f0706a0a740 1 -- 172.21.15.83:0/782334262 shutdown_connections 2024-04-23T14:59:09.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.473+0000 7f0706a0a740 1 -- 172.21.15.83:0/782334262 wait complete. 2024-04-23T14:59:09.535 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.473+0000 7f0706a0a740 20 client.27358 trim_cache size 0 max 16384 2024-04-23T14:59:09.535 INFO:teuthology.orchestra.run.smithi083.stdout:[] 2024-04-23T14:59:09.536 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.473+0000 7f0702600640 1 -- 172.21.15.83:0/3439318255 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06e40a3da0 msgr2=0x7f06e40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.473+0000 7f0702600640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06e40a3da0 0x7f06e40a6260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x559b38a66100 tx=0x7f06f009c6c0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.473+0000 7f0702600640 1 -- 172.21.15.83:0/3439318255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06fc100e50 msgr2=0x7f06fc19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:09.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.473+0000 7f0702600640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06fc100e50 0x7f06fc19edc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f06f000e9f0 tx=0x7f06f000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.474+0000 7f0702600640 1 -- 172.21.15.83:0/3439318255 shutdown_connections 2024-04-23T14:59:09.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.474+0000 7f0702600640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f06e40a3da0 0x7f06e40a6260 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.474+0000 7f0702600640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06fc106250 0x7f06fc1a3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.474+0000 7f0702600640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f06fc101770 0x7f06fc19f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.474+0000 7f0702600640 1 --2- 172.21.15.83:0/3439318255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f06fc100e50 0x7f06fc19edc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:09.537 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.474+0000 7f0702600640 1 -- 172.21.15.83:0/3439318255 >> 172.21.15.83:0/3439318255 conn(0x7f06fc0fc880 msgr2=0x7f06fc0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.474+0000 7f0702600640 1 -- 172.21.15.83:0/3439318255 shutdown_connections 2024-04-23T14:59:09.538 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:09.474+0000 7f0702600640 1 -- 172.21.15.83:0/3439318255 wait complete. 2024-04-23T14:59:10.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:10 smithi173 ceph-mon[32443]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T14:59:10.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:10 smithi203 ceph-mon[32800]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T14:59:10.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:10 smithi083 ceph-mon[27867]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T14:59:10.538 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-23T14:59:10.539 INFO:tasks.fwd_scrub:done joining 2024-04-23T14:59:10.539 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-23T14:59:10.550 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:59:10.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.766+0000 7f5016e00640 1 -- 172.21.15.83:0/641672187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5010104450 msgr2=0x7f5010104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:10.766 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.766+0000 7f5016e00640 1 --2- 172.21.15.83:0/641672187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5010104450 0x7f5010104830 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5000009860 tx=0x7f500002a340 comp rx=0 tx=0).stop 2024-04-23T14:59:10.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.766+0000 7f5016e00640 1 -- 172.21.15.83:0/641672187 shutdown_connections 2024-04-23T14:59:10.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.766+0000 7f5016e00640 1 --2- 172.21.15.83:0/641672187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5010109520 0x7f50101110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:10.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.766+0000 7f5016e00640 1 --2- 172.21.15.83:0/641672187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5010104e00 0x7f5010108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:10.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.766+0000 7f5016e00640 1 --2- 172.21.15.83:0/641672187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5010104450 0x7f5010104830 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:10.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.766+0000 7f5016e00640 1 -- 172.21.15.83:0/641672187 >> 172.21.15.83:0/641672187 conn(0x7f50100ffcf0 msgr2=0x7f5010102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:10.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.766+0000 7f5016e00640 1 -- 172.21.15.83:0/641672187 shutdown_connections 2024-04-23T14:59:10.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.766+0000 7f5016e00640 1 -- 172.21.15.83:0/641672187 wait complete. 2024-04-23T14:59:10.769 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.767+0000 7f5016e00640 1 Processor -- start 2024-04-23T14:59:10.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.768+0000 7f5016e00640 1 -- start start 2024-04-23T14:59:10.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.768+0000 7f5016e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5010104450 0x7f50101a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:10.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.768+0000 7f5016e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5010109520 0x7f50101a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:10.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.768+0000 7f5016e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f501019a930 0x7f501019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:10.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.768+0000 7f5016e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50101145b0 con 0x7f5010109520 2024-04-23T14:59:10.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.768+0000 7f5016e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5010114430 con 0x7f5010104450 2024-04-23T14:59:10.770 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.768+0000 7f5016e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5010114730 con 0x7f501019a930 2024-04-23T14:59:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.769+0000 7f500fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5010104450 0x7f50101a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.769+0000 7f5007400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5010109520 0x7f50101a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.769+0000 7f500fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5010104450 0x7f50101a0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:44976/0 (socket says 172.21.15.83:44976) 2024-04-23T14:59:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.769+0000 7f500fe00640 1 -- 172.21.15.83:0/1768882284 learned_addr learned my addr 172.21.15.83:0/1768882284 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.769+0000 7f5014e00640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f501019a930 0x7f501019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.769+0000 7f500fe00640 1 -- 172.21.15.83:0/1768882284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f501019a930 msgr2=0x7f501019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.769+0000 7f500fe00640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f501019a930 0x7f501019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.769+0000 7f500fe00640 1 -- 172.21.15.83:0/1768882284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5010109520 msgr2=0x7f50101a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:10.771 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.769+0000 7f500fe00640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5010109520 0x7f50101a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.769+0000 7f500fe00640 1 -- 172.21.15.83:0/1768882284 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501019b450 con 0x7f5010104450 2024-04-23T14:59:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.769+0000 7f5014e00640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f501019a930 0x7f501019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:59:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.769+0000 7f500fe00640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5010104450 0x7f50101a0780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5000035040 tx=0x7f5000033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.770+0000 7f500d600640 1 -- 172.21.15.83:0/1768882284 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000037040 con 0x7f5010104450 2024-04-23T14:59:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.770+0000 7f500d600640 1 -- 172.21.15.83:0/1768882284 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5000040070 con 0x7f5010104450 2024-04-23T14:59:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.770+0000 7f5016e00640 1 -- 172.21.15.83:0/1768882284 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501019b6e0 con 0x7f5010104450 2024-04-23T14:59:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.770+0000 7f500d600640 1 -- 172.21.15.83:0/1768882284 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50000039b0 con 0x7f5010104450 2024-04-23T14:59:10.772 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.770+0000 7f5016e00640 1 -- 172.21.15.83:0/1768882284 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50101a78d0 con 0x7f5010104450 2024-04-23T14:59:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.771+0000 7f5016e00640 1 -- 172.21.15.83:0/1768882284 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5010069270 con 0x7f5010104450 2024-04-23T14:59:10.773 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.773+0000 7f500d600640 1 -- 172.21.15.83:0/1768882284 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f50000305e0 con 0x7f5010104450 2024-04-23T14:59:10.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.774+0000 7f500d600640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4ff40a3d20 0x7f4ff40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:10.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.775+0000 7f500d600640 1 -- 172.21.15.83:0/1768882284 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8586+0+0 (secure 0 0 0) 0x7f50000f0a40 con 0x7f5010104450 2024-04-23T14:59:10.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.775+0000 7f5007400640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4ff40a3d20 0x7f4ff40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:10.775 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.775+0000 7f5007400640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4ff40a3d20 0x7f4ff40a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f501010aad0 tx=0x7f4ff8009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:10.777 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.777+0000 7f500d600640 1 -- 172.21.15.83:0/1768882284 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50000bd150 con 0x7f5010104450 2024-04-23T14:59:10.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.983+0000 7f5016e00640 1 -- 172.21.15.83:0/1768882284 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5010104e00 con 0x7f5010104450 2024-04-23T14:59:10.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.985+0000 7f500d600640 1 -- 172.21.15.83:0/1768882284 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+23233 (secure 0 0 0) 0x7f50000bca50 con 0x7f5010104450 2024-04-23T14:59:10.985 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:59:10.985 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":98,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:59:09.036404+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/622708991":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/1081276647":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/4179157882":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6801/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/2255156114":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:59:10.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.988+0000 7f5016e00640 1 -- 172.21.15.83:0/1768882284 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4ff40a3d20 msgr2=0x7f4ff40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:10.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.988+0000 7f5016e00640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4ff40a3d20 0x7f4ff40a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f501010aad0 tx=0x7f4ff8009260 comp rx=0 tx=0).stop 2024-04-23T14:59:10.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.988+0000 7f5016e00640 1 -- 172.21.15.83:0/1768882284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5010104450 msgr2=0x7f50101a0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:10.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.988+0000 7f5016e00640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5010104450 0x7f50101a0780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5000035040 tx=0x7f5000033b60 comp rx=0 tx=0).stop 2024-04-23T14:59:10.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.988+0000 7f5016e00640 1 -- 172.21.15.83:0/1768882284 shutdown_connections 2024-04-23T14:59:10.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.988+0000 7f5016e00640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f501019a930 0x7f501019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:10.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.988+0000 7f5016e00640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4ff40a3d20 0x7f4ff40a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:10.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.988+0000 7f5016e00640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5010109520 0x7f50101a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:10.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.988+0000 7f5016e00640 1 --2- 172.21.15.83:0/1768882284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5010104450 0x7f50101a0780 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:10.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.988+0000 7f5016e00640 1 -- 172.21.15.83:0/1768882284 >> 172.21.15.83:0/1768882284 conn(0x7f50100ffcf0 msgr2=0x7f5010101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:10.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.988+0000 7f5016e00640 1 -- 172.21.15.83:0/1768882284 shutdown_connections 2024-04-23T14:59:10.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:10.988+0000 7f5016e00640 1 -- 172.21.15.83:0/1768882284 wait complete. 2024-04-23T14:59:11.007 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:40:41.241992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:45:57.341602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:45:58.654798+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:59:11.007 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:59:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.216+0000 7f4e3e800640 1 -- 172.21.15.83:0/1589118249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e3810d500 msgr2=0x7f4e3810d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:11.216 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.216+0000 7f4e3e800640 1 --2- 172.21.15.83:0/1589118249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e3810d500 0x7f4e3810d8e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009930 tx=0x7f4e2c02a420 comp rx=0 tx=0).stop 2024-04-23T14:59:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.216+0000 7f4e3e800640 1 -- 172.21.15.83:0/1589118249 shutdown_connections 2024-04-23T14:59:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.216+0000 7f4e3e800640 1 --2- 172.21.15.83:0/1589118249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e38108f40 0x7f4e381153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.216+0000 7f4e3e800640 1 --2- 172.21.15.83:0/1589118249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e38108580 0x7f4e38108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.216+0000 7f4e3e800640 1 --2- 172.21.15.83:0/1589118249 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e3810d500 0x7f4e3810d8e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.216+0000 7f4e3e800640 1 -- 172.21.15.83:0/1589118249 >> 172.21.15.83:0/1589118249 conn(0x7f4e38077e20 msgr2=0x7f4e38106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.216+0000 7f4e3e800640 1 -- 172.21.15.83:0/1589118249 shutdown_connections 2024-04-23T14:59:11.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.217+0000 7f4e3e800640 1 -- 172.21.15.83:0/1589118249 wait complete. 2024-04-23T14:59:11.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.218+0000 7f4e3e800640 1 Processor -- start 2024-04-23T14:59:11.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.218+0000 7f4e3e800640 1 -- start start 2024-04-23T14:59:11.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.218+0000 7f4e3e800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e38108580 0x7f4e3819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:11.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.218+0000 7f4e3e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e38108f40 0x7f4e3819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:11.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.218+0000 7f4e3e800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e3810d500 0x7f4e381a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:11.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.218+0000 7f4e3e800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4e38118910 con 0x7f4e38108f40 2024-04-23T14:59:11.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.218+0000 7f4e3e800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4e38118790 con 0x7f4e3810d500 2024-04-23T14:59:11.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.218+0000 7f4e3e800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4e38118a90 con 0x7f4e38108580 2024-04-23T14:59:11.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.219+0000 7f4e3ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e38108f40 0x7f4e3819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:11.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.219+0000 7f4e3ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e38108f40 0x7f4e3819f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56056/0 (socket says 172.21.15.83:56056) 2024-04-23T14:59:11.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.219+0000 7f4e3ca00640 1 -- 172.21.15.83:0/65372398 learned_addr learned my addr 172.21.15.83:0/65372398 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:11.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.219+0000 7f4e3de00640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e3810d500 0x7f4e381a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:11.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.219+0000 7f4e3d400640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e38108580 0x7f4e3819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:11.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.219+0000 7f4e3ca00640 1 -- 172.21.15.83:0/65372398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e38108580 msgr2=0x7f4e3819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:11.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.219+0000 7f4e3ca00640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e38108580 0x7f4e3819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.219+0000 7f4e3ca00640 1 -- 172.21.15.83:0/65372398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e3810d500 msgr2=0x7f4e381a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:11.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.219+0000 7f4e3ca00640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e3810d500 0x7f4e381a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.219+0000 7f4e3ca00640 1 -- 172.21.15.83:0/65372398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e381a3d30 con 0x7f4e38108f40 2024-04-23T14:59:11.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.219+0000 7f4e3ca00640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e38108f40 0x7f4e3819f2d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400e9f0 tx=0x7f4e2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:11.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.220+0000 7f4e36000640 1 -- 172.21.15.83:0/65372398 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2400cd80 con 0x7f4e38108f40 2024-04-23T14:59:11.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.220+0000 7f4e3e800640 1 -- 172.21.15.83:0/65372398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e381a4020 con 0x7f4e38108f40 2024-04-23T14:59:11.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.220+0000 7f4e36000640 1 -- 172.21.15.83:0/65372398 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e24005500 con 0x7f4e38108f40 2024-04-23T14:59:11.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.220+0000 7f4e36000640 1 -- 172.21.15.83:0/65372398 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24018630 con 0x7f4e38108f40 2024-04-23T14:59:11.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.220+0000 7f4e3e800640 1 -- 172.21.15.83:0/65372398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e381ab8b0 con 0x7f4e38108f40 2024-04-23T14:59:11.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.221+0000 7f4e3e800640 1 -- 172.21.15.83:0/65372398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e38069190 con 0x7f4e38108f40 2024-04-23T14:59:11.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.223+0000 7f4e36000640 1 -- 172.21.15.83:0/65372398 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4e240046c0 con 0x7f4e38108f40 2024-04-23T14:59:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.225+0000 7f4e36000640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4e200a3d20 0x7f4e200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.225+0000 7f4e36000640 1 -- 172.21.15.83:0/65372398 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8586+0+0 (secure 0 0 0) 0x7f4e240d0da0 con 0x7f4e38108f40 2024-04-23T14:59:11.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.226+0000 7f4e3d400640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4e200a3d20 0x7f4e200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:11.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.226+0000 7f4e3d400640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4e200a3d20 0x7f4e200a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009900 tx=0x7f4e2c000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:11.230 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.230+0000 7f4e36000640 1 -- 172.21.15.83:0/65372398 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e2409d3a0 con 0x7f4e38108f40 2024-04-23T14:59:11.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:11 smithi083 ceph-mon[27867]: pgmap v334: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T14:59:11.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:11 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1768882284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:59:11.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:11 smithi173 ceph-mon[32443]: pgmap v334: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T14:59:11.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:11 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1768882284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:59:11.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.434+0000 7f4e3e800640 1 -- 172.21.15.83:0/65372398 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4e381104c0 con 0x7f4e38108f40 2024-04-23T14:59:11.434 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.435+0000 7f4e36000640 1 -- 172.21.15.83:0/65372398 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f4e2409cca0 con 0x7f4e38108f40 2024-04-23T14:59:11.435 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-23T14:59:11.437 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.437+0000 7f4e3e800640 1 -- 172.21.15.83:0/65372398 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4e200a3d20 msgr2=0x7f4e200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:11.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.437+0000 7f4e3e800640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4e200a3d20 0x7f4e200a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009900 tx=0x7f4e2c000bc0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.437+0000 7f4e3e800640 1 -- 172.21.15.83:0/65372398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e38108f40 msgr2=0x7f4e3819f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:11.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.437+0000 7f4e3e800640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e38108f40 0x7f4e3819f2d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400e9f0 tx=0x7f4e2400edf0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.437+0000 7f4e3e800640 1 -- 172.21.15.83:0/65372398 shutdown_connections 2024-04-23T14:59:11.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.437+0000 7f4e3e800640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e3810d500 0x7f4e381a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.437+0000 7f4e3e800640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4e200a3d20 0x7f4e200a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.437+0000 7f4e3e800640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e38108f40 0x7f4e3819f2d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.437+0000 7f4e3e800640 1 --2- 172.21.15.83:0/65372398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e38108580 0x7f4e3819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.437+0000 7f4e3e800640 1 -- 172.21.15.83:0/65372398 >> 172.21.15.83:0/65372398 conn(0x7f4e38077e20 msgr2=0x7f4e380783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:11.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.438+0000 7f4e3e800640 1 -- 172.21.15.83:0/65372398 shutdown_connections 2024-04-23T14:59:11.439 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.438+0000 7f4e3e800640 1 -- 172.21.15.83:0/65372398 wait complete. 2024-04-23T14:59:11.457 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:59:11.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:11 smithi203 ceph-mon[32800]: pgmap v334: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T14:59:11.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:11 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1768882284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:59:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.664+0000 7f38e2200640 1 -- 172.21.15.83:0/725801796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc108580 msgr2=0x7f38dc108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.664+0000 7f38e2200640 1 --2- 172.21.15.83:0/725801796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc108580 0x7f38dc108a00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009930 tx=0x7f38c802a420 comp rx=0 tx=0).stop 2024-04-23T14:59:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.664+0000 7f38e2200640 1 -- 172.21.15.83:0/725801796 shutdown_connections 2024-04-23T14:59:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.664+0000 7f38e2200640 1 --2- 172.21.15.83:0/725801796 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc108f40 0x7f38dc1153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.664+0000 7f38e2200640 1 --2- 172.21.15.83:0/725801796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc108580 0x7f38dc108a00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.664+0000 7f38e2200640 1 --2- 172.21.15.83:0/725801796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38dc10d500 0x7f38dc10d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.664+0000 7f38e2200640 1 -- 172.21.15.83:0/725801796 >> 172.21.15.83:0/725801796 conn(0x7f38dc077e20 msgr2=0x7f38dc106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.664+0000 7f38e2200640 1 -- 172.21.15.83:0/725801796 shutdown_connections 2024-04-23T14:59:11.665 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.664+0000 7f38e2200640 1 -- 172.21.15.83:0/725801796 wait complete. 2024-04-23T14:59:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.665+0000 7f38e2200640 1 Processor -- start 2024-04-23T14:59:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.665+0000 7f38e2200640 1 -- start start 2024-04-23T14:59:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.665+0000 7f38e2200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38dc108580 0x7f38dc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.665+0000 7f38e2200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc108f40 0x7f38dc19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.665+0000 7f38e2200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc10d500 0x7f38dc1a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:11.667 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.665+0000 7f38e2200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1188d0 con 0x7f38dc10d500 2024-04-23T14:59:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.665+0000 7f38e2200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc118750 con 0x7f38dc108580 2024-04-23T14:59:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.665+0000 7f38e2200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc118a50 con 0x7f38dc108f40 2024-04-23T14:59:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.666+0000 7f38e1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc10d500 0x7f38dc1a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.666+0000 7f38e1800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc10d500 0x7f38dc1a35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56064/0 (socket says 172.21.15.83:56064) 2024-04-23T14:59:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.666+0000 7f38e1800640 1 -- 172.21.15.83:0/3646805919 learned_addr learned my addr 172.21.15.83:0/3646805919 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.666+0000 7f38dbe00640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc108f40 0x7f38dc19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.666+0000 7f38e0e00640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38dc108580 0x7f38dc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.666+0000 7f38e1800640 1 -- 172.21.15.83:0/3646805919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc108f40 msgr2=0x7f38dc19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.666+0000 7f38e1800640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc108f40 0x7f38dc19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.668 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.666+0000 7f38e1800640 1 -- 172.21.15.83:0/3646805919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38dc108580 msgr2=0x7f38dc19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:11.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.666+0000 7f38e1800640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38dc108580 0x7f38dc19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.666+0000 7f38e1800640 1 -- 172.21.15.83:0/3646805919 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc1a3d10 con 0x7f38dc10d500 2024-04-23T14:59:11.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.666+0000 7f38e1800640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc10d500 0x7f38dc1a35e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f38d400ea60 tx=0x7f38d400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:11.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.667+0000 7f38d9600640 1 -- 172.21.15.83:0/3646805919 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d400ce40 con 0x7f38dc10d500 2024-04-23T14:59:11.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.667+0000 7f38d9600640 1 -- 172.21.15.83:0/3646805919 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38d4005020 con 0x7f38dc10d500 2024-04-23T14:59:11.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.667+0000 7f38d9600640 1 -- 172.21.15.83:0/3646805919 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4010690 con 0x7f38dc10d500 2024-04-23T14:59:11.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.667+0000 7f38e2200640 1 -- 172.21.15.83:0/3646805919 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38dc1a4000 con 0x7f38dc10d500 2024-04-23T14:59:11.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.667+0000 7f38e2200640 1 -- 172.21.15.83:0/3646805919 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38dc1ab890 con 0x7f38dc10d500 2024-04-23T14:59:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.668+0000 7f38e2200640 1 -- 172.21.15.83:0/3646805919 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38dc069190 con 0x7f38dc10d500 2024-04-23T14:59:11.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.670+0000 7f38d9600640 1 -- 172.21.15.83:0/3646805919 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f38d4005520 con 0x7f38dc10d500 2024-04-23T14:59:11.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.671+0000 7f38d9600640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38c40a3da0 0x7f38c40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:11.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.672+0000 7f38d9600640 1 -- 172.21.15.83:0/3646805919 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8586+0+0 (secure 0 0 0) 0x7f38d40d0c80 con 0x7f38dc10d500 2024-04-23T14:59:11.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.672+0000 7f38e0e00640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38c40a3da0 0x7f38c40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:11.672 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.672+0000 7f38e0e00640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38c40a3da0 0x7f38c40a6260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38d00095f0 tx=0x7f38d0005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:11.675 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.676+0000 7f38d9600640 1 -- 172.21.15.83:0/3646805919 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38d409d280 con 0x7f38dc10d500 2024-04-23T14:59:11.878 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.878+0000 7f38e2200640 1 -- 172.21.15.83:0/3646805919 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f38dc110aa0 con 0x7f38dc10d500 2024-04-23T14:59:11.879 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.879+0000 7f38d9600640 1 -- 172.21.15.83:0/3646805919 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7f38d409cb80 con 0x7f38dc10d500 2024-04-23T14:59:11.879 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-23T14:59:11.881 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.882+0000 7f38e2200640 1 -- 172.21.15.83:0/3646805919 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38c40a3da0 msgr2=0x7f38c40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:11.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.882+0000 7f38e2200640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38c40a3da0 0x7f38c40a6260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38d00095f0 tx=0x7f38d0005f90 comp rx=0 tx=0).stop 2024-04-23T14:59:11.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.882+0000 7f38e2200640 1 -- 172.21.15.83:0/3646805919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc10d500 msgr2=0x7f38dc1a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:11.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.882+0000 7f38e2200640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc10d500 0x7f38dc1a35e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f38d400ea60 tx=0x7f38d400ef20 comp rx=0 tx=0).stop 2024-04-23T14:59:11.882 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.882+0000 7f38e2200640 1 -- 172.21.15.83:0/3646805919 shutdown_connections 2024-04-23T14:59:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.882+0000 7f38e2200640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38c40a3da0 0x7f38c40a6260 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.882+0000 7f38e2200640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38dc10d500 0x7f38dc1a35e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.882+0000 7f38e2200640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38dc108f40 0x7f38dc19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.882+0000 7f38e2200640 1 --2- 172.21.15.83:0/3646805919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38dc108580 0x7f38dc19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.882+0000 7f38e2200640 1 -- 172.21.15.83:0/3646805919 >> 172.21.15.83:0/3646805919 conn(0x7f38dc077e20 msgr2=0x7f38dc078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.882+0000 7f38e2200640 1 -- 172.21.15.83:0/3646805919 shutdown_connections 2024-04-23T14:59:11.883 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:11.882+0000 7f38e2200640 1 -- 172.21.15.83:0/3646805919 wait complete. 2024-04-23T14:59:11.901 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:59:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.114+0000 7f93e7e00640 1 -- 172.21.15.83:0/2185054396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8102ac0 msgr2=0x7f93e8106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:12.114 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.114+0000 7f93e7e00640 1 --2- 172.21.15.83:0/2185054396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8102ac0 0x7f93e8106b00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009900 tx=0x7f93d402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.114+0000 7f93e7e00640 1 -- 172.21.15.83:0/2185054396 shutdown_connections 2024-04-23T14:59:12.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.114+0000 7f93e7e00640 1 --2- 172.21.15.83:0/2185054396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93e81071e0 0x7f93e810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.114+0000 7f93e7e00640 1 --2- 172.21.15.83:0/2185054396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e8102ac0 0x7f93e8106b00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.114+0000 7f93e7e00640 1 --2- 172.21.15.83:0/2185054396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8102110 0x7f93e81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.114+0000 7f93e7e00640 1 -- 172.21.15.83:0/2185054396 >> 172.21.15.83:0/2185054396 conn(0x7f93e80fd9d0 msgr2=0x7f93e80ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:12.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.114+0000 7f93e7e00640 1 -- 172.21.15.83:0/2185054396 shutdown_connections 2024-04-23T14:59:12.115 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.114+0000 7f93e7e00640 1 -- 172.21.15.83:0/2185054396 wait complete. 2024-04-23T14:59:12.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.116+0000 7f93e7e00640 1 Processor -- start 2024-04-23T14:59:12.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.116+0000 7f93e7e00640 1 -- start start 2024-04-23T14:59:12.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.116+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8102110 0x7f93e819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:12.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.116+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e81071e0 0x7f93e819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:12.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.116+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93e8198620 0x7f93e8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:12.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.116+0000 7f93e7e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81122c0 con 0x7f93e8102110 2024-04-23T14:59:12.117 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.116+0000 7f93e7e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f93e8112140 con 0x7f93e81071e0 2024-04-23T14:59:12.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.116+0000 7f93e7e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f93e8112440 con 0x7f93e8198620 2024-04-23T14:59:12.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.116+0000 7f93e6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8102110 0x7f93e819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:12.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.116+0000 7f93e6a00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8102110 0x7f93e819e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56078/0 (socket says 172.21.15.83:56078) 2024-04-23T14:59:12.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.116+0000 7f93e6a00640 1 -- 172.21.15.83:0/186190155 learned_addr learned my addr 172.21.15.83:0/186190155 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:12.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.116+0000 7f93e6000640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e81071e0 0x7f93e819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:12.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.117+0000 7f93e7400640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93e8198620 0x7f93e8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:12.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.117+0000 7f93e6a00640 1 -- 172.21.15.83:0/186190155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93e8198620 msgr2=0x7f93e8198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:12.118 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.117+0000 7f93e6a00640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93e8198620 0x7f93e8198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.117+0000 7f93e6a00640 1 -- 172.21.15.83:0/186190155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e81071e0 msgr2=0x7f93e819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:12.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.117+0000 7f93e6a00640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e81071e0 0x7f93e819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.117+0000 7f93e6a00640 1 -- 172.21.15.83:0/186190155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e8199140 con 0x7f93e8102110 2024-04-23T14:59:12.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.117+0000 7f93e6a00640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8102110 0x7f93e819e470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f93e000ebc0 tx=0x7f93e000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:12.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.117+0000 7f93eca00640 1 -- 172.21.15.83:0/186190155 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0010630 con 0x7f93e8102110 2024-04-23T14:59:12.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.118+0000 7f93eca00640 1 -- 172.21.15.83:0/186190155 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e0010cb0 con 0x7f93e8102110 2024-04-23T14:59:12.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.118+0000 7f93e7e00640 1 -- 172.21.15.83:0/186190155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e8199430 con 0x7f93e8102110 2024-04-23T14:59:12.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.118+0000 7f93eca00640 1 -- 172.21.15.83:0/186190155 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0018bd0 con 0x7f93e8102110 2024-04-23T14:59:12.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.118+0000 7f93e7e00640 1 -- 172.21.15.83:0/186190155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e81a56d0 con 0x7f93e8102110 2024-04-23T14:59:12.120 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.119+0000 7f93e7e00640 1 -- 172.21.15.83:0/186190155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e8068d40 con 0x7f93e8102110 2024-04-23T14:59:12.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.121+0000 7f93eca00640 1 -- 172.21.15.83:0/186190155 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f93e00107d0 con 0x7f93e8102110 2024-04-23T14:59:12.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.122+0000 7f93eca00640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f93cc0a3da0 0x7f93cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:12.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.122+0000 7f93eca00640 1 -- 172.21.15.83:0/186190155 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8586+0+0 (secure 0 0 0) 0x7f93e00d0e50 con 0x7f93e8102110 2024-04-23T14:59:12.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.122+0000 7f93e6000640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f93cc0a3da0 0x7f93cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:12.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.123+0000 7f93e6000640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f93cc0a3da0 0x7f93cc0a6260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f93d4006010 tx=0x7f93d4000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:12.126 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.126+0000 7f93eca00640 1 -- 172.21.15.83:0/186190155 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93e009ce00 con 0x7f93e8102110 2024-04-23T14:59:12.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:12 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/65372398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:59:12.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:12 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/3646805919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:59:12.325 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:12 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/65372398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:59:12.326 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:12 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/3646805919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:59:12.331 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.331+0000 7f93e7e00640 1 -- 172.21.15.83:0/186190155 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f93e810a410 con 0x7f93e8102110 2024-04-23T14:59:12.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.332+0000 7f93eca00640 1 -- 172.21.15.83:0/186190155 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f93e00a1100 con 0x7f93e8102110 2024-04-23T14:59:12.332 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-23T14:59:12.335 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.335+0000 7f93e7e00640 1 -- 172.21.15.83:0/186190155 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f93cc0a3da0 msgr2=0x7f93cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.335+0000 7f93e7e00640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f93cc0a3da0 0x7f93cc0a6260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f93d4006010 tx=0x7f93d4000bc0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.335+0000 7f93e7e00640 1 -- 172.21.15.83:0/186190155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8102110 msgr2=0x7f93e819e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.335+0000 7f93e7e00640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8102110 0x7f93e819e470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f93e000ebc0 tx=0x7f93e000c6a0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.335+0000 7f93e7e00640 1 -- 172.21.15.83:0/186190155 shutdown_connections 2024-04-23T14:59:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.335+0000 7f93e7e00640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f93e8198620 0x7f93e8198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.335+0000 7f93e7e00640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93e81071e0 0x7f93e819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.336 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.335+0000 7f93e7e00640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f93cc0a3da0 0x7f93cc0a6260 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.335+0000 7f93e7e00640 1 --2- 172.21.15.83:0/186190155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8102110 0x7f93e819e470 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.335+0000 7f93e7e00640 1 -- 172.21.15.83:0/186190155 >> 172.21.15.83:0/186190155 conn(0x7f93e80fd9d0 msgr2=0x7f93e80ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.335+0000 7f93e7e00640 1 -- 172.21.15.83:0/186190155 shutdown_connections 2024-04-23T14:59:12.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.335+0000 7f93e7e00640 1 -- 172.21.15.83:0/186190155 wait complete. 2024-04-23T14:59:12.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-23T14:59:12.470 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:12 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/65372398' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:59:12.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:12 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/3646805919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:59:12.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.561+0000 7f4c82000640 1 -- 172.21.15.83:0/335517728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c7c0743d0 msgr2=0x7f4c7c0747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:12.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.561+0000 7f4c82000640 1 --2- 172.21.15.83:0/335517728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c7c0743d0 0x7f4c7c0747b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009ca0 tx=0x7f4c6c02a940 comp rx=0 tx=0).stop 2024-04-23T14:59:12.561 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.561+0000 7f4c82000640 1 -- 172.21.15.83:0/335517728 shutdown_connections 2024-04-23T14:59:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.561+0000 7f4c82000640 1 --2- 172.21.15.83:0/335517728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c7c10e870 0x7f4c7c110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.561+0000 7f4c82000640 1 --2- 172.21.15.83:0/335517728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c7c074cf0 0x7f4c7c10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.561+0000 7f4c82000640 1 --2- 172.21.15.83:0/335517728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c7c0743d0 0x7f4c7c0747b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.561+0000 7f4c82000640 1 -- 172.21.15.83:0/335517728 >> 172.21.15.83:0/335517728 conn(0x7f4c7c0793c0 msgr2=0x7f4c7c0797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.561+0000 7f4c82000640 1 -- 172.21.15.83:0/335517728 shutdown_connections 2024-04-23T14:59:12.562 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.561+0000 7f4c82000640 1 -- 172.21.15.83:0/335517728 wait complete. 2024-04-23T14:59:12.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.562+0000 7f4c82000640 1 Processor -- start 2024-04-23T14:59:12.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.562+0000 7f4c82000640 1 -- start start 2024-04-23T14:59:12.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.562+0000 7f4c82000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c7c0743d0 0x7f4c7c1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:12.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.562+0000 7f4c82000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c7c074cf0 0x7f4c7c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:12.564 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.562+0000 7f4c82000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c7c10e870 0x7f4c7c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.562+0000 7f4c82000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c114640 con 0x7f4c7c074cf0 2024-04-23T14:59:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.562+0000 7f4c82000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1144c0 con 0x7f4c7c10e870 2024-04-23T14:59:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.562+0000 7f4c82000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1147c0 con 0x7f4c7c0743d0 2024-04-23T14:59:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.563+0000 7f4c7b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c7c0743d0 0x7f4c7c1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.563+0000 7f4c7aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c7c074cf0 0x7f4c7c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.563+0000 7f4c7b400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c7c0743d0 0x7f4c7c1a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:49942/0 (socket says 172.21.15.83:49942) 2024-04-23T14:59:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.563+0000 7f4c7b400640 1 -- 172.21.15.83:0/835609538 learned_addr learned my addr 172.21.15.83:0/835609538 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.563+0000 7f4c7be00640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c7c10e870 0x7f4c7c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:12.565 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.563+0000 7f4c7b400640 1 -- 172.21.15.83:0/835609538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c7c10e870 msgr2=0x7f4c7c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.563+0000 7f4c7b400640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c7c10e870 0x7f4c7c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.563+0000 7f4c7b400640 1 -- 172.21.15.83:0/835609538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c7c074cf0 msgr2=0x7f4c7c1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.563+0000 7f4c7b400640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c7c074cf0 0x7f4c7c1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.563+0000 7f4c7b400640 1 -- 172.21.15.83:0/835609538 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c19b140 con 0x7f4c7c0743d0 2024-04-23T14:59:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.563+0000 7f4c7b400640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c7c0743d0 0x7f4c7c1a0810 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c004af0 tx=0x7f4c6c02f6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.564+0000 7f4c73e00640 1 -- 172.21.15.83:0/835609538 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c034040 con 0x7f4c7c0743d0 2024-04-23T14:59:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.564+0000 7f4c82000640 1 -- 172.21.15.83:0/835609538 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7c19b3d0 con 0x7f4c7c0743d0 2024-04-23T14:59:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.564+0000 7f4c82000640 1 -- 172.21.15.83:0/835609538 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7c1a76d0 con 0x7f4c7c0743d0 2024-04-23T14:59:12.566 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.564+0000 7f4c73e00640 1 -- 172.21.15.83:0/835609538 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c02fe10 con 0x7f4c7c0743d0 2024-04-23T14:59:12.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.564+0000 7f4c73e00640 1 -- 172.21.15.83:0/835609538 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c0046e0 con 0x7f4c7c0743d0 2024-04-23T14:59:12.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.565+0000 7f4c82000640 1 -- 172.21.15.83:0/835609538 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c7c069270 con 0x7f4c7c0743d0 2024-04-23T14:59:12.567 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.567+0000 7f4c73e00640 1 -- 172.21.15.83:0/835609538 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4c6c037020 con 0x7f4c7c0743d0 2024-04-23T14:59:12.568 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.569+0000 7f4c73e00640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4c600a3da0 0x7f4c600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:12.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.569+0000 7f4c73e00640 1 -- 172.21.15.83:0/835609538 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8586+0+0 (secure 0 0 0) 0x7f4c6c0f14f0 con 0x7f4c7c0743d0 2024-04-23T14:59:12.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.569+0000 7f4c7aa00640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4c600a3da0 0x7f4c600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:12.569 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.570+0000 7f4c7aa00640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4c600a3da0 0x7f4c600a6260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c680095f0 tx=0x7f4c68005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:12.571 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.571+0000 7f4c73e00640 1 -- 172.21.15.83:0/835609538 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c6c0bdc00 con 0x7f4c7c0743d0 2024-04-23T14:59:12.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.828+0000 7f4c82000640 1 -- 172.21.15.83:0/835609538 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f4c7c109a20 con 0x7f4c7c0743d0 2024-04-23T14:59:12.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.840+0000 7f4c73e00640 1 -- 172.21.15.83:0/835609538 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746060 (secure 0 0 0) 0x7f4c6c0bd500 con 0x7f4c7c0743d0 2024-04-23T14:59:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:59:12.853 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:59:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "epoch": 26, 2024-04-23T14:59:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "active_gid": 15114, 2024-04-23T14:59:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "active_name": "x", 2024-04-23T14:59:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addrs": { 2024-04-23T14:59:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-23T14:59:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-23T14:59:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6800", 2024-04-23T14:59:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1667121447 2024-04-23T14:59:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v1", 2024-04-23T14:59:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:6801", 2024-04-23T14:59:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1667121447 2024-04-23T14:59:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-23T14:59:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "active_addr": "172.21.15.83:6801/1667121447", 2024-04-23T14:59:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "active_change": "2024-04-23T14:48:27.715698+0000", 2024-04-23T14:59:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "active_mgr_features": 4540701547738038271, 2024-04-23T14:59:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "available": true, 2024-04-23T14:59:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "standbys": [ 2024-04-23T14:59:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 25195, 2024-04-23T14:59:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "z", 2024-04-23T14:59:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540701547738038271, 2024-04-23T14:59:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-04-23T14:59:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-04-23T14:59:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T14:59:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-04-23T14:59:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-04-23T14:59:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-04-23T14:59:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-04-23T14:59:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-04-23T14:59:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-04-23T14:59:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-04-23T14:59:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-04-23T14:59:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-04-23T14:59:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-04-23T14:59:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-04-23T14:59:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-04-23T14:59:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-04-23T14:59:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-04-23T14:59:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-04-23T14:59:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-04-23T14:59:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-04-23T14:59:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-04-23T14:59:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-04-23T14:59:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-04-23T14:59:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-04-23T14:59:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-04-23T14:59:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T14:59:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-04-23T14:59:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-04-23T14:59:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-04-23T14:59:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-04-23T14:59:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-04-23T14:59:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-04-23T14:59:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T14:59:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-04-23T14:59:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-04-23T14:59:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-04-23T14:59:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T14:59:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T14:59:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-04-23T14:59:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-04-23T14:59:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-23T14:59:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-23T14:59:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T14:59:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T14:59:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-04-23T14:59:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-04-23T14:59:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-04-23T14:59:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-23T14:59:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-04-23T14:59:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T14:59:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-04-23T14:59:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-04-23T14:59:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T14:59:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T14:59:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T14:59:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-04-23T14:59:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-04-23T14:59:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-04-23T14:59:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-04-23T14:59:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-04-23T14:59:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-04-23T14:59:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T14:59:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-04-23T14:59:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-04-23T14:59:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-04-23T14:59:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.890 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T14:59:12.890 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T14:59:12.890 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.890 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.890 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-04-23T14:59:12.890 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-04-23T14:59:12.890 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:12.890 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.890 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.891 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:12.891 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-23T14:59:12.891 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-23T14:59:12.891 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.891 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T14:59:12.891 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T14:59:12.891 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.891 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.892 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:12.892 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:12.892 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.892 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.892 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.892 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.892 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.892 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.892 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:12.892 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:12.893 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:12.893 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:12.893 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:12.893 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:12.893 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:12.893 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:12.893 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:12.893 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.893 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.894 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.894 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:12.894 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:12.894 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.894 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.894 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.894 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.894 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.895 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.895 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.895 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:12.895 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.895 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.895 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.895 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.895 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:12.895 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:12.895 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.896 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.896 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.896 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:12.896 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.896 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.896 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:12.896 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:12.896 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:12.896 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:12.897 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:12.897 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:12.897 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:12.897 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:12.897 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:12.897 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.897 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.897 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.898 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:12.898 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:12.898 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.898 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.898 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.898 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.898 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.899 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.899 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.899 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:12.899 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.899 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.899 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.899 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.899 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-04-23T14:59:12.899 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-04-23T14:59:12.900 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:12.900 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.900 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.900 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:12.900 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.900 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.900 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.900 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T14:59:12.900 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.901 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.901 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.901 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-23T14:59:12.901 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-23T14:59:12.901 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.901 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.901 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.901 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-04-23T14:59:12.901 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.902 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.902 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:12.902 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-04-23T14:59:12.902 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-04-23T14:59:12.902 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-04-23T14:59:12.902 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-04-23T14:59:12.902 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-04-23T14:59:12.902 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:12.902 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-04-23T14:59:12.903 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.903 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.903 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.903 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-04-23T14:59:12.903 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-04-23T14:59:12.903 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.903 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.903 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.903 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.904 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.904 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.904 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.904 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T14:59:12.904 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.904 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.904 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.904 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:12.905 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:12.905 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:12.905 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.905 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.905 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:12.905 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.905 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.905 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.905 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T14:59:12.906 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.906 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.906 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.906 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-04-23T14:59:12.906 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-04-23T14:59:12.906 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:12.906 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.906 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.907 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-23T14:59:12.907 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-23T14:59:12.907 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.907 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.907 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T14:59:12.907 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T14:59:12.907 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.907 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.908 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.908 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-04-23T14:59:12.908 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-04-23T14:59:12.908 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:12.908 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.908 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.908 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:12.908 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.908 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.909 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.909 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T14:59:12.909 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.909 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.909 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.909 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:12.909 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:12.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.909 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:12.910 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-04-23T14:59:12.910 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:12.910 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:12.910 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:12.910 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-04-23T14:59:12.910 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-04-23T14:59:12.910 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:12.910 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.910 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.911 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-23T14:59:12.911 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.911 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.911 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.911 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T14:59:12.911 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.911 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.911 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.911 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.911 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-04-23T14:59:12.912 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-04-23T14:59:12.912 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:12.912 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.912 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.912 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-04-23T14:59:12.912 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.912 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.912 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.912 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T14:59:12.913 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.913 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.913 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.913 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-04-23T14:59:12.913 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-04-23T14:59:12.913 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:12.913 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.913 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.913 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-04-23T14:59:12.914 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.914 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.914 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.914 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T14:59:12.914 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.914 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.915 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-04-23T14:59:12.915 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-04-23T14:59:12.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.915 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.915 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.915 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.915 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.915 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.915 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.916 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T14:59:12.916 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T14:59:12.916 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.916 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.916 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-04-23T14:59:12.916 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-04-23T14:59:12.916 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:12.916 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.917 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.917 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:12.917 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.917 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.917 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.917 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T14:59:12.917 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.917 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.918 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.918 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.918 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-04-23T14:59:12.918 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T14:59:12.918 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:12.918 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.918 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.918 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-04-23T14:59:12.919 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.919 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.919 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.919 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T14:59:12.919 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.919 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.919 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.919 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-04-23T14:59:12.920 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-04-23T14:59:12.920 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.920 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.920 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.920 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.920 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.920 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.920 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:12.920 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-04-23T14:59:12.921 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-04-23T14:59:12.921 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-04-23T14:59:12.921 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:12.921 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T14:59:12.921 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.921 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.921 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.921 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.921 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-04-23T14:59:12.922 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-04-23T14:59:12.922 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.922 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.922 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.922 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:12.922 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.922 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.923 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.923 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T14:59:12.923 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.923 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.923 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.923 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.923 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-04-23T14:59:12.923 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-04-23T14:59:12.923 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.924 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.924 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.924 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.924 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.924 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.924 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.924 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T14:59:12.924 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.925 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.925 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.925 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.925 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-04-23T14:59:12.925 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-04-23T14:59:12.925 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.925 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.925 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.925 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:12.926 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.926 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.926 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.926 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T14:59:12.926 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.926 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.926 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.926 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-04-23T14:59:12.927 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-04-23T14:59:12.927 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.927 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.927 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.927 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T14:59:12.927 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.927 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.927 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.927 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-23T14:59:12.927 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.928 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.928 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.928 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.928 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-04-23T14:59:12.928 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-04-23T14:59:12.928 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.928 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.928 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.928 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T14:59:12.929 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.929 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.929 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.929 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-04-23T14:59:12.929 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.929 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.929 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-04-23T14:59:12.930 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-04-23T14:59:12.930 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.930 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.930 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.930 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T14:59:12.930 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.930 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.930 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.930 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-04-23T14:59:12.931 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.931 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.931 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.931 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-04-23T14:59:12.931 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-04-23T14:59:12.931 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.931 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.931 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.931 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T14:59:12.932 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.932 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.932 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.932 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-23T14:59:12.932 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.932 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.932 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.932 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.932 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-04-23T14:59:12.932 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-04-23T14:59:12.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.933 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.933 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T14:59:12.933 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.933 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.933 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.933 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-04-23T14:59:12.934 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.934 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.934 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.934 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-04-23T14:59:12.934 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-04-23T14:59:12.934 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.934 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.935 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.935 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T14:59:12.935 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.935 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.935 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.935 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-04-23T14:59:12.935 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.935 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.935 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.936 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-04-23T14:59:12.936 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-04-23T14:59:12.936 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.936 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.936 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.936 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T14:59:12.936 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.936 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.936 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.937 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-04-23T14:59:12.937 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.937 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.937 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.937 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-04-23T14:59:12.937 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-04-23T14:59:12.937 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.937 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.938 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T14:59:12.938 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.938 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.938 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.938 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-04-23T14:59:12.938 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.938 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.939 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.939 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-04-23T14:59:12.939 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-04-23T14:59:12.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.939 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.939 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.939 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T14:59:12.939 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.940 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.940 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.940 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-04-23T14:59:12.940 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.940 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.940 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.940 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-04-23T14:59:12.940 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-04-23T14:59:12.940 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.941 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.941 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.941 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T14:59:12.941 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.941 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.941 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.941 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-04-23T14:59:12.941 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.941 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.941 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.942 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.942 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-04-23T14:59:12.942 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-04-23T14:59:12.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.942 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.942 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T14:59:12.942 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.943 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.943 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.943 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-23T14:59:12.943 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.943 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.943 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-04-23T14:59:12.943 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-04-23T14:59:12.944 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.944 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.944 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.944 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T14:59:12.944 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.944 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.944 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.944 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-04-23T14:59:12.944 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.944 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.945 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.945 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-04-23T14:59:12.945 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-04-23T14:59:12.945 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.945 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.945 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.945 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T14:59:12.945 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.946 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.946 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.946 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-23T14:59:12.946 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.946 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.946 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.946 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-04-23T14:59:12.946 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-04-23T14:59:12.947 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.947 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.947 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.947 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T14:59:12.947 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.947 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.947 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.947 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-04-23T14:59:12.947 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.948 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.948 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.948 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-04-23T14:59:12.948 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-04-23T14:59:12.948 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.948 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.948 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.948 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T14:59:12.948 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.949 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.949 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.949 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-04-23T14:59:12.949 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.949 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.949 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.949 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.949 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-04-23T14:59:12.949 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-04-23T14:59:12.950 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.950 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.950 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.950 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T14:59:12.950 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.950 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.950 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.950 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-04-23T14:59:12.951 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.951 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.951 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.951 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-04-23T14:59:12.951 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-04-23T14:59:12.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.951 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.951 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.951 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:12.952 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.952 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.952 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.952 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T14:59:12.952 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.952 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.952 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.953 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-04-23T14:59:12.953 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-04-23T14:59:12.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:12.953 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.953 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.953 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:12.953 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.953 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.954 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.954 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T14:59:12.954 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.954 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.954 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-04-23T14:59:12.954 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T14:59:12.955 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:12.955 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.955 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.955 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-04-23T14:59:12.955 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.955 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.955 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.955 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T14:59:12.956 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.956 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.956 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.956 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.956 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-04-23T14:59:12.956 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-04-23T14:59:12.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.956 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.956 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-04-23T14:59:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T14:59:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.957 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-04-23T14:59:12.958 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-04-23T14:59:12.958 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:12.958 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.958 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.958 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-04-23T14:59:12.958 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.958 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.958 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.958 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-04-23T14:59:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-04-23T14:59:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-04-23T14:59:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.959 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.960 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.960 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.960 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.960 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T14:59:12.960 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.960 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.960 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.961 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-04-23T14:59:12.961 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-04-23T14:59:12.961 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:12.961 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.961 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.961 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:12.961 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.961 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.961 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.962 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-04-23T14:59:12.962 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.962 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.962 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.962 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-04-23T14:59:12.962 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-04-23T14:59:12.962 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.962 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.963 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.963 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T14:59:12.963 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.963 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.963 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.963 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T14:59:12.963 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.963 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.963 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.964 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-04-23T14:59:12.964 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-04-23T14:59:12.964 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:12.964 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.964 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.964 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:12.964 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.964 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.965 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.965 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-04-23T14:59:12.965 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.965 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.965 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.965 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-04-23T14:59:12.965 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-04-23T14:59:12.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.965 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.966 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.966 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.966 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.966 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.966 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.966 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T14:59:12.966 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.966 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.967 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.967 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-04-23T14:59:12.967 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-04-23T14:59:12.967 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.967 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.967 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.967 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.967 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.968 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.968 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.968 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T14:59:12.968 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.968 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.968 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.968 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:12.968 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:12.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.969 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.969 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.969 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.969 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.969 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.969 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:12.969 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:12.969 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:12.970 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:12.970 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:12.970 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:12.970 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:12.970 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:12.970 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:12.970 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.970 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.970 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.971 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.971 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-04-23T14:59:12.971 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-04-23T14:59:12.971 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.971 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.971 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.971 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.971 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.971 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.972 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.972 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T14:59:12.972 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.972 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.972 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.972 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:12.972 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:12.972 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.973 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.973 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.973 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:12.973 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.973 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.973 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.973 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T14:59:12.973 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.973 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.974 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.974 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:12.974 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:12.974 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.974 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.974 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:12.975 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.976 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.977 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:12.977 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.977 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.977 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.977 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.977 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:12.977 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.977 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.977 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.978 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T14:59:12.978 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T14:59:12.978 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.978 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.978 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.978 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.978 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.978 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.979 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.979 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T14:59:12.979 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.979 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.979 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.979 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T14:59:12.979 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T14:59:12.979 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.980 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.980 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.980 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-04-23T14:59:12.980 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.980 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.980 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.980 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T14:59:12.980 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.981 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.981 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.981 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-04-23T14:59:12.981 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-04-23T14:59:12.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:12.981 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.981 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:12.981 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.982 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.982 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.982 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-04-23T14:59:12.982 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.982 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.982 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.982 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-04-23T14:59:12.982 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-04-23T14:59:12.983 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:12.983 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.983 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.983 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:12.983 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.983 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.983 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.983 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T14:59:12.983 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.984 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.984 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.984 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-04-23T14:59:12.984 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-04-23T14:59:12.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:12.984 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.984 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.985 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.985 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.985 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.985 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.985 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-04-23T14:59:12.985 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.985 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.985 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.986 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-23T14:59:12.986 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-23T14:59:12.986 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.986 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.986 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.986 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-04-23T14:59:12.986 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.986 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.986 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:12.987 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-04-23T14:59:12.987 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-04-23T14:59:12.987 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:12.987 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T14:59:12.987 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.987 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.987 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.987 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-04-23T14:59:12.987 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-04-23T14:59:12.988 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.988 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.988 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-04-23T14:59:12.988 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.988 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.988 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.988 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T14:59:12.988 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.989 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.989 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.989 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-04-23T14:59:12.989 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-04-23T14:59:12.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.989 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.989 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T14:59:12.989 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.990 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.990 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.990 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T14:59:12.990 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.990 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.990 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.990 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-04-23T14:59:12.990 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-04-23T14:59:12.991 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.991 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.991 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.991 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.991 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.991 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.991 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.991 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T14:59:12.991 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.992 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.992 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.992 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-04-23T14:59:12.992 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-04-23T14:59:12.992 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.992 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.992 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.992 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.992 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.993 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.993 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.993 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T14:59:12.993 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.993 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.993 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.993 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.993 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-04-23T14:59:12.993 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-04-23T14:59:12.994 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.994 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.994 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.994 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.994 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.994 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.994 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.994 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T14:59:12.994 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.995 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.995 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.995 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-04-23T14:59:12.995 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-04-23T14:59:12.995 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.995 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.995 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.995 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:12.995 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.996 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.996 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.996 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T14:59:12.996 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.996 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.996 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.996 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.996 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-04-23T14:59:12.996 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-04-23T14:59:12.996 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:12.997 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.997 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.997 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:12.997 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.997 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.997 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.997 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T14:59:12.997 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.997 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.998 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.998 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.998 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-04-23T14:59:12.998 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-04-23T14:59:12.998 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:12.998 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:12.998 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:12.998 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-04-23T14:59:12.998 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:12.998 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:12.999 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:12.999 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-04-23T14:59:12.999 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:12.999 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:12.999 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:12.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:12.999 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-04-23T14:59:12.999 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-04-23T14:59:12.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:12.999 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T14:59:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.000 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-04-23T14:59:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-04-23T14:59:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.001 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T14:59:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-04-23T14:59:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-04-23T14:59:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.002 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T14:59:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-04-23T14:59:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-04-23T14:59:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.004 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T14:59:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-04-23T14:59:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-04-23T14:59:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.005 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T14:59:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.006 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-04-23T14:59:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-04-23T14:59:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.007 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T14:59:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-04-23T14:59:13.008 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.009 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.010 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.011 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.011 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.011 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.011 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.011 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.011 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.011 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.011 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.012 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.012 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.012 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.012 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.012 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.012 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.012 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.012 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.013 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.013 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.013 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.013 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.013 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.013 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.013 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.013 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.013 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.014 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.014 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.014 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.014 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.014 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.016 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.016 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.016 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.017 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.017 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.017 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.017 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.017 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.017 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.017 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.017 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.018 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.018 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.018 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.018 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.018 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.018 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.018 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.018 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.019 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.019 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-04-23T14:59:13.019 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-04-23T14:59:13.019 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.019 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.019 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.019 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-04-23T14:59:13.019 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.019 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T14:59:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-04-23T14:59:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-04-23T14:59:13.020 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-04-23T14:59:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T14:59:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.021 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.022 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-04-23T14:59:13.022 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.022 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.022 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.022 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T14:59:13.023 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T14:59:13.023 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.023 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.023 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.023 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:13.023 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.023 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.023 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.023 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.024 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.024 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.024 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.024 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.024 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T14:59:13.024 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T14:59:13.024 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.024 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.024 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.025 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.025 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.025 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.025 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.025 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.025 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.025 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.025 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.026 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T14:59:13.026 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T14:59:13.026 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.026 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.026 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.026 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.026 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.026 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.027 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.027 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.027 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.027 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.027 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-04-23T14:59:13.027 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T14:59:13.027 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.027 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.027 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.028 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.028 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.028 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.028 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.028 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.028 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.028 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.028 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.028 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.029 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T14:59:13.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T14:59:13.029 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.029 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.029 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.029 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.029 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.029 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.029 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.029 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.030 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.030 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.030 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.030 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.030 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T14:59:13.030 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T14:59:13.030 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.030 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.030 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.031 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.031 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.031 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.031 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.031 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.031 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.031 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.031 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.032 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.032 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T14:59:13.032 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T14:59:13.032 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.032 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.032 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.032 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.032 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.032 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.033 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.033 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.033 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.033 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.033 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.033 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.033 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T14:59:13.033 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T14:59:13.033 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.033 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.034 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T14:59:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T14:59:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.035 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.036 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T14:59:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T14:59:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.037 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T14:59:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T14:59:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.038 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.039 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T14:59:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T14:59:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.040 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T14:59:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T14:59:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.041 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.042 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T14:59:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T14:59:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.043 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.044 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.044 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.044 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.044 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T14:59:13.044 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T14:59:13.044 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.044 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.045 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.045 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-23T14:59:13.045 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.045 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.045 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.045 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.045 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.045 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.046 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.046 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.046 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T14:59:13.046 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T14:59:13.046 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.046 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.046 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.046 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.046 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.047 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.047 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.047 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.047 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.047 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.047 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.047 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.047 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-04-23T14:59:13.047 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-04-23T14:59:13.048 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.048 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.048 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.048 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.048 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.048 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.048 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.048 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.048 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.049 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.049 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.049 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T14:59:13.049 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T14:59:13.049 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.049 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.049 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.049 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-23T14:59:13.049 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.050 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.050 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.050 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.050 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.050 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.050 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.050 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.050 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T14:59:13.050 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T14:59:13.051 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.051 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.051 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.051 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.051 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.051 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.051 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.051 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.051 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.051 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.052 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.052 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.052 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T14:59:13.052 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T14:59:13.052 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.052 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.052 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.052 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.053 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.053 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.053 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.053 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.053 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.053 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.053 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.053 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T14:59:13.054 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T14:59:13.054 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.054 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.054 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.054 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.054 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.054 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.054 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.054 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.055 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.055 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.055 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.055 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.055 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T14:59:13.055 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T14:59:13.055 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.055 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.055 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.056 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.056 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.056 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.056 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.056 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.056 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.056 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.056 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.056 INFO:teuthology.orchestra.run.smithi083.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T14:59:13.057 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T14:59:13.057 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.057 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.057 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.057 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "[]", 2024-04-23T14:59:13.057 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.057 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.057 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.057 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.058 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.058 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.058 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.058 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.058 INFO:teuthology.orchestra.run.smithi083.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T14:59:13.058 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T14:59:13.058 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.058 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.058 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.059 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "{}", 2024-04-23T14:59:13.059 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.059 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.059 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.059 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.059 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.059 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.059 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.059 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T14:59:13.060 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T14:59:13.060 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.060 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.060 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.060 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.060 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.060 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.060 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.060 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.061 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.061 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.061 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.061 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.061 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T14:59:13.061 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T14:59:13.061 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.061 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.062 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.062 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.062 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.062 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.062 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.062 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.062 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.063 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T14:59:13.063 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T14:59:13.063 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.063 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.063 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.063 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.063 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.063 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.063 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.064 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.064 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.064 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.064 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.064 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.064 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T14:59:13.064 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T14:59:13.064 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.064 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.064 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.065 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.065 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.065 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.065 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.065 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.065 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.065 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.065 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.066 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T14:59:13.066 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T14:59:13.066 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.066 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.066 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.066 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.066 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.066 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.067 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.067 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.067 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.067 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.067 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.067 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.067 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T14:59:13.067 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T14:59:13.067 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.068 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.068 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.068 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.068 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.068 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.068 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.068 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.068 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.068 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.069 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.069 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.069 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T14:59:13.069 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T14:59:13.069 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.069 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.069 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.069 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.070 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.070 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.070 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.070 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.070 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.070 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.070 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T14:59:13.071 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T14:59:13.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.071 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.071 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.071 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.071 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.071 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.071 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.071 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.071 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.072 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.072 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.072 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T14:59:13.072 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T14:59:13.072 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.072 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.072 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.072 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.073 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.073 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.073 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.073 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.073 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.073 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.073 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.073 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.073 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T14:59:13.074 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T14:59:13.074 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.074 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.074 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.074 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.074 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.074 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.074 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.074 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.075 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.075 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.075 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.075 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.075 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T14:59:13.075 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T14:59:13.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.075 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.076 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.076 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T14:59:13.076 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.076 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.076 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.076 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.076 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.076 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.076 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.077 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.077 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T14:59:13.077 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T14:59:13.077 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.077 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.077 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.077 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:13.077 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.078 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.078 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.078 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.078 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.078 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.078 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.078 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T14:59:13.078 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T14:59:13.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.079 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.079 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.079 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-04-23T14:59:13.079 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.079 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.079 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.079 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.079 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.080 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.080 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.080 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T14:59:13.080 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T14:59:13.080 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.080 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.080 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.080 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-04-23T14:59:13.081 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.081 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.081 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.081 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.081 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.081 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.081 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.081 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.081 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T14:59:13.082 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T14:59:13.082 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.082 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.082 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.082 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.082 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.082 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.082 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.082 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.083 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.083 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.083 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.083 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T14:59:13.083 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T14:59:13.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.083 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.083 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.083 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-23T14:59:13.084 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.084 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.084 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.084 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.084 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.084 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.084 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T14:59:13.084 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T14:59:13.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.085 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.085 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.085 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.085 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.085 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.085 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.085 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.085 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.086 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.086 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.086 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.086 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T14:59:13.086 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T14:59:13.086 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.086 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.086 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.086 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.087 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.087 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.087 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.087 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.087 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.087 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.087 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.087 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T14:59:13.087 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T14:59:13.088 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.088 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.088 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.088 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.088 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.088 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.088 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.088 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.088 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.089 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.089 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.089 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.089 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T14:59:13.089 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T14:59:13.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.089 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.089 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.089 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:13.089 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.090 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.090 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.090 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.090 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.090 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.090 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.090 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.090 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T14:59:13.090 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T14:59:13.091 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.091 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.091 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.091 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:13.091 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.091 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.091 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.091 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.092 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.092 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.092 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.092 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.092 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T14:59:13.092 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T14:59:13.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.092 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.093 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-23T14:59:13.093 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.093 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.093 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.093 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.093 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.093 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.093 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.093 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-04-23T14:59:13.094 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-04-23T14:59:13.094 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.094 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.094 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.094 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.094 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.094 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.094 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.094 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.095 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.095 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.095 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.095 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-04-23T14:59:13.095 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-04-23T14:59:13.095 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.095 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.095 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.096 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.096 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.096 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.096 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.096 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.096 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.096 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.096 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.096 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.096 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-04-23T14:59:13.097 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-04-23T14:59:13.097 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.097 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.097 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.097 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.097 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.097 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.097 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.097 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-04-23T14:59:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-04-23T14:59:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-04-23T14:59:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.098 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-04-23T14:59:13.099 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.099 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.099 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.099 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.099 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.099 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.099 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.099 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-23T14:59:13.100 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-23T14:59:13.100 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.100 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.100 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.100 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.100 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.100 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.100 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.100 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.100 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.101 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.101 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.101 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.101 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.101 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.101 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.101 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.101 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.101 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.101 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.102 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.102 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.102 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.102 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.102 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.102 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.102 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.102 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.102 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.103 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.103 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.103 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.103 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.103 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.103 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.103 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.103 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.103 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.104 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.104 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.104 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.104 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.104 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.104 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.104 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.104 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.104 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.105 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.105 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.105 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.105 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.105 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.105 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.105 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.105 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.105 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.105 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.106 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.106 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.106 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.106 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.106 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.106 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.106 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.106 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.106 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.107 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.107 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.107 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.107 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.107 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.107 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.107 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.107 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.107 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.108 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.108 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.108 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.108 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.108 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.108 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.108 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-04-23T14:59:13.108 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-04-23T14:59:13.109 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.109 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.109 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.109 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.109 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.109 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.109 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.109 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-04-23T14:59:13.110 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.110 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.110 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.110 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-04-23T14:59:13.110 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T14:59:13.110 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.110 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.110 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.111 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.111 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.111 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.111 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.111 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.111 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.111 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.111 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.111 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.111 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-23T14:59:13.112 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-23T14:59:13.112 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.112 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.112 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.112 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-23T14:59:13.112 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.112 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.112 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.112 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.113 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.113 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.113 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.113 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.113 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-23T14:59:13.113 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-23T14:59:13.113 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.113 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.113 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.114 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-04-23T14:59:13.114 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.114 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.114 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.114 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.114 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.114 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.114 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.115 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-23T14:59:13.115 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-23T14:59:13.115 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.115 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.115 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.115 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.115 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.115 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.115 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.115 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.116 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.116 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.117 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.117 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-04-23T14:59:13.118 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-04-23T14:59:13.118 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.118 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.118 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.118 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-04-23T14:59:13.118 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.118 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.118 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.119 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.119 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.119 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.119 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.119 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-23T14:59:13.119 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-23T14:59:13.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.119 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.120 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.120 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-04-23T14:59:13.120 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.120 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.120 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.120 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.120 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-04-23T14:59:13.120 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.120 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.121 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.121 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.121 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.121 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.121 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-23T14:59:13.121 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-23T14:59:13.121 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.121 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.121 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.121 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-23T14:59:13.122 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-23T14:59:13.122 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-23T14:59:13.122 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.122 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.122 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.122 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.122 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.122 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-04-23T14:59:13.123 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-04-23T14:59:13.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.123 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.123 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.123 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.123 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.123 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.123 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.123 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.124 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.124 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.124 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.124 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.124 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.124 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.124 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-04-23T14:59:13.124 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.124 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.125 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.125 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-04-23T14:59:13.125 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-04-23T14:59:13.125 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.125 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.125 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.125 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.125 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.125 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.126 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.126 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-04-23T14:59:13.126 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.126 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.126 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.126 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.126 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.127 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.127 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.127 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.127 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.127 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.127 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.127 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.127 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.127 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.127 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.128 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.128 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.128 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.128 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.128 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.128 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.128 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.128 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.129 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.129 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.129 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.129 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.129 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.129 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.129 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.129 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.129 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.129 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.130 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.130 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.130 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.130 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.130 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.130 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.130 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.130 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.130 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.131 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.131 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.131 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.131 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.131 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.131 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.131 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.131 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.131 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.131 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.132 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.132 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.132 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.132 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.132 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.132 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.132 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.132 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.132 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.133 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.133 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.133 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.133 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.133 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.133 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.133 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.133 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.134 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.134 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.134 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-04-23T14:59:13.134 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-04-23T14:59:13.134 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.134 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.134 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.134 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-04-23T14:59:13.134 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.135 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.135 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.135 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T14:59:13.135 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.135 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.135 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.135 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.135 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-04-23T14:59:13.135 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-04-23T14:59:13.136 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.136 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.136 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.136 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-04-23T14:59:13.136 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.136 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.136 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.136 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T14:59:13.136 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.137 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.137 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.137 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-04-23T14:59:13.137 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-04-23T14:59:13.137 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.137 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.137 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.137 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-04-23T14:59:13.137 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.138 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.138 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.138 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-04-23T14:59:13.138 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.138 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.138 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.138 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.138 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-04-23T14:59:13.138 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-04-23T14:59:13.139 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.139 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.139 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.139 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-23T14:59:13.139 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.139 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.139 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.139 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T14:59:13.140 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.140 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.140 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.140 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-04-23T14:59:13.140 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-04-23T14:59:13.140 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.140 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.140 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.141 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.141 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.141 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.141 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.141 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T14:59:13.141 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.141 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.141 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.141 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.141 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.142 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T14:59:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-04-23T14:59:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-04-23T14:59:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.143 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-04-23T14:59:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T14:59:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.144 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-04-23T14:59:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.145 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.146 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.147 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.148 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.149 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.150 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.152 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-04-23T14:59:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-04-23T14:59:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-23T14:59:13.153 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-04-23T14:59:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-04-23T14:59:13.154 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-04-23T14:59:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.155 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.156 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.157 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-04-23T14:59:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-04-23T14:59:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-04-23T14:59:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-04-23T14:59:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-04-23T14:59:13.158 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-04-23T14:59:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T14:59:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.159 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-04-23T14:59:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-04-23T14:59:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-04-23T14:59:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.160 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T14:59:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-04-23T14:59:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-04-23T14:59:13.161 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-04-23T14:59:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.162 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-04-23T14:59:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-04-23T14:59:13.163 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T14:59:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.164 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.165 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.166 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.168 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.169 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.170 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.171 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-04-23T14:59:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-04-23T14:59:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.172 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-04-23T14:59:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-04-23T14:59:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-04-23T14:59:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.173 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-04-23T14:59:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-04-23T14:59:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.174 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-23T14:59:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-23T14:59:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-04-23T14:59:13.175 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T14:59:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-04-23T14:59:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-04-23T14:59:13.176 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-23T14:59:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-23T14:59:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-04-23T14:59:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T14:59:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.177 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-04-23T14:59:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-04-23T14:59:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.178 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-04-23T14:59:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-04-23T14:59:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-04-23T14:59:13.179 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-04-23T14:59:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T14:59:13.180 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-04-23T14:59:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.181 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.182 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.183 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.184 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.185 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.186 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.187 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.188 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-04-23T14:59:13.189 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.190 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.191 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.192 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.193 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.194 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.195 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.196 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-04-23T14:59:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-04-23T14:59:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-04-23T14:59:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.198 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-04-23T14:59:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T14:59:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.199 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-04-23T14:59:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-04-23T14:59:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-04-23T14:59:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.200 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T14:59:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.201 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.202 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.203 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.204 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.205 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.206 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.207 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.208 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-04-23T14:59:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-04-23T14:59:13.209 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-04-23T14:59:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-04-23T14:59:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-04-23T14:59:13.210 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T14:59:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.211 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.212 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.213 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.214 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.215 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.216 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.217 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.218 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-04-23T14:59:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-04-23T14:59:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.219 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-04-23T14:59:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T14:59:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-04-23T14:59:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-04-23T14:59:13.220 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-04-23T14:59:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-04-23T14:59:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.221 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-04-23T14:59:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-04-23T14:59:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-04-23T14:59:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.222 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T14:59:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-04-23T14:59:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-04-23T14:59:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.223 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-04-23T14:59:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.224 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-04-23T14:59:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-04-23T14:59:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-04-23T14:59:13.225 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T14:59:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T14:59:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.226 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-04-23T14:59:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.227 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.228 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.229 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.230 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.231 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.232 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.233 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.234 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.235 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.235 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.235 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-04-23T14:59:13.235 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.235 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.235 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.235 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.235 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.236 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.236 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.236 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.236 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.236 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.236 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.236 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.237 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.237 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.237 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.237 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.237 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.237 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.237 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.237 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.238 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.238 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.238 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.238 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.238 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.238 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.238 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.238 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.238 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.239 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.239 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.239 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.239 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.239 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.239 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.239 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.240 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.240 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.240 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.240 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.240 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.240 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.240 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.241 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.241 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.241 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.241 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.241 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.241 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.241 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.241 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.241 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.242 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.242 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.242 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.242 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.242 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.242 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.242 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.243 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.243 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.243 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.243 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.243 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.243 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.243 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.243 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.243 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.244 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.244 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.244 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-04-23T14:59:13.244 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.244 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.244 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.244 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.245 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.245 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.245 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.245 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.245 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.245 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.245 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.245 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.245 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.246 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.246 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.246 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.246 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.246 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.246 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.246 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.246 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.247 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.247 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.247 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.247 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.247 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.247 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.247 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.248 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.248 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.248 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.248 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.248 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.248 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.248 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.248 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.248 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.249 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.249 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.249 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.249 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.249 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.249 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.249 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.249 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.250 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.250 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.250 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.250 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.250 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.250 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.250 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.250 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.251 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.251 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.251 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.251 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.251 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.251 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.251 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.251 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.251 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.252 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.252 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.252 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.252 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.252 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.252 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.252 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.253 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.253 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-23T14:59:13.253 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.253 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.253 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.253 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-04-23T14:59:13.253 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-04-23T14:59:13.253 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.253 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.254 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.254 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.254 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.254 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.254 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.254 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T14:59:13.254 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.254 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.255 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.256 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.256 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.256 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.256 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.256 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.256 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.256 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.256 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.257 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.257 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.257 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.257 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.257 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.257 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.257 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.258 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.259 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.260 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.261 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.262 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-04-23T14:59:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-23T14:59:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.263 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-04-23T14:59:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-04-23T14:59:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-04-23T14:59:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-04-23T14:59:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.264 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-04-23T14:59:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.265 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.266 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.267 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.268 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.270 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.271 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.272 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-04-23T14:59:13.273 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.274 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.275 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.276 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.278 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.279 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.280 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.281 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-04-23T14:59:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.282 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.283 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.284 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.285 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.286 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.287 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.288 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.289 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.290 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.291 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-04-23T14:59:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-04-23T14:59:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-23T14:59:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-04-23T14:59:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.292 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-04-23T14:59:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T14:59:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-04-23T14:59:13.293 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-04-23T14:59:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-04-23T14:59:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.294 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T14:59:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-23T14:59:13.295 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-23T14:59:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.296 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.297 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.298 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.299 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.300 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.301 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.302 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.303 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-04-23T14:59:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-04-23T14:59:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.304 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-04-23T14:59:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-04-23T14:59:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-23T14:59:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.306 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-04-23T14:59:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-04-23T14:59:13.307 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-04-23T14:59:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-04-23T14:59:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.308 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-04-23T14:59:13.309 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-04-23T14:59:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.310 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T14:59:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T14:59:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.311 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.312 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.313 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.314 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.316 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.318 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-04-23T14:59:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-04-23T14:59:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.319 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.320 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T14:59:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T14:59:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-04-23T14:59:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.321 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-04-23T14:59:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-04-23T14:59:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.322 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-04-23T14:59:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.323 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-23T14:59:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-23T14:59:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-23T14:59:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.324 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T14:59:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-23T14:59:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-23T14:59:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.325 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-04-23T14:59:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T14:59:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.326 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-04-23T14:59:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-04-23T14:59:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-04-23T14:59:13.327 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-23T14:59:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-23T14:59:13.328 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-04-23T14:59:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-04-23T14:59:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-04-23T14:59:13.329 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-23T14:59:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-23T14:59:13.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-23T14:59:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-23T14:59:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-23T14:59:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.331 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-04-23T14:59:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.332 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.333 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.334 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.335 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.336 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.337 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.338 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.339 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.340 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-04-23T14:59:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-04-23T14:59:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.341 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-04-23T14:59:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T14:59:13.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.343 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-04-23T14:59:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-04-23T14:59:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.344 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.345 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-04-23T14:59:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-04-23T14:59:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-04-23T14:59:13.346 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.347 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-23T14:59:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-23T14:59:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.348 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.349 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.350 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.351 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.352 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.353 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.354 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.355 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.356 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-23T14:59:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-23T14:59:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.357 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.358 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-23T14:59:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-23T14:59:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.359 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-04-23T14:59:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.360 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.361 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.362 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.363 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.364 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.365 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.366 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.367 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.368 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-04-23T14:59:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.369 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.370 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.371 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.372 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.373 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.374 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.375 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.376 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-04-23T14:59:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-04-23T14:59:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-04-23T14:59:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.377 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T14:59:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.378 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-04-23T14:59:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.379 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.380 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.381 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.382 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.383 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.384 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.385 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.386 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-04-23T14:59:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-04-23T14:59:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.387 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-04-23T14:59:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-04-23T14:59:13.388 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-04-23T14:59:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.389 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-04-23T14:59:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-04-23T14:59:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.390 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-04-23T14:59:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-04-23T14:59:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.391 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.392 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.392 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.392 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.392 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.392 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.392 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.392 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.392 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.393 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.393 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-04-23T14:59:13.393 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-04-23T14:59:13.393 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:13.393 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.393 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.393 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.393 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.393 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.394 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.394 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.394 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.394 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.394 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-04-23T14:59:13.394 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-04-23T14:59:13.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.395 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.395 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.395 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.395 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.395 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.395 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.395 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.395 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.396 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.396 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.396 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-04-23T14:59:13.396 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-04-23T14:59:13.396 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.396 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.396 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.396 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.396 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.397 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.397 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.397 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.397 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.397 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.397 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.397 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-04-23T14:59:13.397 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-04-23T14:59:13.398 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.398 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.398 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.398 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.398 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.398 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.398 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.398 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.399 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.399 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.399 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.399 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.399 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-04-23T14:59:13.399 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-04-23T14:59:13.399 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.399 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.399 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.400 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-23T14:59:13.400 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-04-23T14:59:13.400 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.400 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.400 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.400 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.400 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.400 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-04-23T14:59:13.401 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-04-23T14:59:13.401 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.401 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.401 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.401 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.401 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.401 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.401 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.401 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.401 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.402 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.402 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.402 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.402 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-04-23T14:59:13.402 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-04-23T14:59:13.402 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.402 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.402 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.402 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.403 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.403 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.403 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.403 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.403 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.403 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.403 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.403 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-04-23T14:59:13.403 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-04-23T14:59:13.404 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.404 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.404 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.404 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.404 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.404 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.404 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.404 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.404 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.405 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.405 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.405 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.405 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.405 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.405 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-23T14:59:13.405 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.405 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.405 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.406 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-04-23T14:59:13.406 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-04-23T14:59:13.406 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.406 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.406 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.406 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.406 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.406 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.406 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.407 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T14:59:13.407 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.407 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.407 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.407 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-04-23T14:59:13.407 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-04-23T14:59:13.407 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.407 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.407 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.408 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.408 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.408 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.408 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.408 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-04-23T14:59:13.408 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.408 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.408 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.409 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.409 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.409 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.409 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.409 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.409 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.409 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.409 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.409 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.410 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.410 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.410 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.410 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.410 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.410 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.410 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.410 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.410 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.410 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.411 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.411 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.411 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.411 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.411 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.411 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.411 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.411 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.411 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.412 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.412 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.412 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.412 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.412 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.412 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.412 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.412 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.412 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.413 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.413 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.413 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.413 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.413 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.413 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.413 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.414 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.414 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.414 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.414 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.414 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.414 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.414 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.414 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.414 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.415 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.415 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.415 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.415 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.415 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.415 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.415 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.415 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.416 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.416 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.416 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.416 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.416 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.416 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.416 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.416 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-04-23T14:59:13.416 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.417 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.417 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.417 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.417 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.417 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.417 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.417 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.417 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.418 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.418 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.418 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.418 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.418 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.418 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.418 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.418 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.418 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.418 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.419 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.419 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.419 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.419 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.419 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.419 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.420 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.420 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.420 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.420 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.420 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.420 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.420 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.421 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.421 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.421 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.421 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.421 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.421 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.421 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.421 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.421 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.422 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.422 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.422 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.422 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.422 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.422 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.422 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.422 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.422 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.423 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.423 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.423 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.423 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.423 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.423 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.423 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.423 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.423 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.423 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.424 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.424 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.424 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.424 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.424 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.424 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.424 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.424 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.425 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-04-23T14:59:13.425 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.425 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.425 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.425 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.425 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.425 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.425 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.425 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.426 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.426 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.426 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.426 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.426 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.426 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.426 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.426 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.426 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.427 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.427 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.427 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.427 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.427 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.427 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.427 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.427 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.428 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.428 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.428 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.428 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.428 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.428 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.428 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.428 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.428 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.428 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.429 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.429 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.429 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.429 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.429 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.429 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.429 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.429 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.430 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.430 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.430 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.430 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.430 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.430 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.430 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.430 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.430 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.431 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.431 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.431 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.431 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.431 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.431 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.431 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.431 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.432 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.432 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.432 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.432 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.432 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.432 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.432 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.432 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.433 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.433 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.433 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.433 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-04-23T14:59:13.433 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.433 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.433 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.434 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-04-23T14:59:13.434 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-04-23T14:59:13.434 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.434 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.434 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.434 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T14:59:13.434 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.434 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:13.435 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-04-23T14:59:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.436 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.437 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.438 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.438 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.438 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.438 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.438 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.438 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.438 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.438 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.439 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.440 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.440 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.440 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.440 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.440 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.440 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.440 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.440 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.440 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.441 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.441 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.441 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.441 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.441 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.441 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.441 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.441 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.441 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.442 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.442 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.442 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.442 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.442 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.442 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.442 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.442 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.442 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.443 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.443 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.443 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.443 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.443 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.443 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.444 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.444 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.444 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.444 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.444 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.444 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.444 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.444 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.444 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-04-23T14:59:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-04-23T14:59:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-04-23T14:59:13.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.446 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.446 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.446 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.446 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.446 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.446 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T14:59:13.446 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.447 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.447 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.447 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-04-23T14:59:13.447 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-04-23T14:59:13.447 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.447 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.447 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.447 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.448 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.448 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.448 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.448 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T14:59:13.448 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.448 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.449 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-04-23T14:59:13.449 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-04-23T14:59:13.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.449 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.449 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.449 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.449 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.449 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.450 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T14:59:13.450 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.450 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.450 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-04-23T14:59:13.450 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-04-23T14:59:13.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.450 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.451 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.451 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.451 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.451 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.451 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T14:59:13.451 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.452 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.452 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.452 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.452 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-04-23T14:59:13.452 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-04-23T14:59:13.452 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.452 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.452 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.452 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.453 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.453 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.453 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.453 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T14:59:13.453 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.453 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.453 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.453 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-04-23T14:59:13.454 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-04-23T14:59:13.454 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.454 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.454 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.454 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.454 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.454 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.454 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.455 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.455 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.455 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.455 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.455 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.455 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-04-23T14:59:13.455 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-04-23T14:59:13.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.455 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.456 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.456 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.456 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.456 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.456 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.456 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.456 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.456 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.456 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.457 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.457 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-04-23T14:59:13.457 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-04-23T14:59:13.457 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.457 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.457 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.457 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T14:59:13.457 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.457 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.458 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.458 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.458 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.458 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.458 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.458 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-23T14:59:13.458 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-23T14:59:13.459 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.459 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.459 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.459 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.459 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.459 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.459 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.460 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.460 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.460 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.460 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:13.460 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:13.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.460 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.460 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.461 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-04-23T14:59:13.461 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-04-23T14:59:13.461 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.461 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.461 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.461 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.461 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.462 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-04-23T14:59:13.462 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-04-23T14:59:13.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.462 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.462 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.462 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-04-23T14:59:13.462 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.462 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.463 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.463 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.463 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.463 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.463 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.463 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-04-23T14:59:13.463 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-04-23T14:59:13.463 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.464 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.464 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.464 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.464 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.464 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.464 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.464 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.464 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.464 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.465 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.465 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.465 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-04-23T14:59:13.465 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-04-23T14:59:13.465 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.465 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.465 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.465 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.465 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.466 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.466 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.466 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.466 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.466 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.466 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.466 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.466 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.467 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.467 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.467 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.467 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.467 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.467 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.467 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.467 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.467 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.468 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.468 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.468 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.468 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.468 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.468 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.468 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.468 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.468 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.469 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.469 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.469 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.469 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.469 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.469 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.469 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.470 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.470 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.470 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.470 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.470 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.470 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.470 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.470 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.471 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.472 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.473 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.473 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.473 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.473 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.473 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.473 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.473 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.474 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.474 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.474 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.474 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.474 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.474 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.474 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.474 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.475 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-04-23T14:59:13.475 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-04-23T14:59:13.475 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.475 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.475 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.475 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.475 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.475 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.475 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.475 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.476 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.476 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.476 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.476 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-04-23T14:59:13.476 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-04-23T14:59:13.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.476 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.477 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.477 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.477 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.477 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.477 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.477 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.477 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.478 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-04-23T14:59:13.478 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-04-23T14:59:13.478 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.478 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.478 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.478 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T14:59:13.478 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.478 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.478 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.479 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.479 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.479 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.479 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.479 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.479 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.479 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.479 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-04-23T14:59:13.480 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.480 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.480 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.480 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.480 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.480 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.480 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.480 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.480 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.481 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.481 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.481 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.481 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.481 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.481 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.481 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.481 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.481 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.482 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.482 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.482 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.482 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.482 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.482 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.482 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.483 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.483 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.483 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.483 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.483 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.483 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.483 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.483 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.483 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.484 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.484 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.484 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.484 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.484 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.484 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.484 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.484 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.484 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.484 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.485 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.485 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.485 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.485 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.485 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.485 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.485 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.485 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.485 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.486 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.486 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.486 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.486 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.486 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.486 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.486 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.486 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.487 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.487 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.487 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.487 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.487 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.487 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.487 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.487 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.488 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.488 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.488 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.488 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.488 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-04-23T14:59:13.488 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.488 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.488 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.488 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.489 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.489 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.489 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.489 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.489 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.489 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.489 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.489 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.489 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.490 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.490 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.490 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.490 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.490 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.490 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.490 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.490 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.490 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.491 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.491 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.491 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.491 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.491 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.491 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.491 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.491 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.491 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.491 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.492 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.492 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.492 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.492 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.492 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.492 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.492 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.493 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.493 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.493 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.493 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.493 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.493 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.493 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.493 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.493 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.493 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.494 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.494 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.494 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.494 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.494 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.494 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.494 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.494 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.495 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.495 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.495 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.495 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.495 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.495 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.495 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.495 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.495 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.495 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.496 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.496 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.496 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.496 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.496 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-04-23T14:59:13.496 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-04-23T14:59:13.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.496 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.496 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.496 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-04-23T14:59:13.497 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.497 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.497 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.497 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T14:59:13.497 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.497 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.497 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.497 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-04-23T14:59:13.498 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-04-23T14:59:13.498 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.498 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.498 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.498 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.498 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.498 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.498 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.498 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-04-23T14:59:13.498 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.499 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.499 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.499 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.499 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-04-23T14:59:13.499 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-04-23T14:59:13.499 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.499 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.499 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.499 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:13.499 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.500 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.500 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.500 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T14:59:13.500 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.500 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.500 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-04-23T14:59:13.500 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T14:59:13.501 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.501 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.501 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.501 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.501 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.501 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.501 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.501 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T14:59:13.501 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.502 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.502 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.502 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.502 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.502 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.502 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-04-23T14:59:13.502 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.502 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.502 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.503 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-04-23T14:59:13.503 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-04-23T14:59:13.503 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:13.503 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.503 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.503 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-04-23T14:59:13.503 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.503 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.503 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.504 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.504 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.504 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.504 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.504 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-04-23T14:59:13.504 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-04-23T14:59:13.504 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.504 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.505 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.505 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.505 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.505 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.505 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.505 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.505 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.505 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.505 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.506 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:13.506 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:13.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.506 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.506 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.506 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:13.506 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.506 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.507 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.507 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.507 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.507 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.507 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.507 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.507 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.507 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.507 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.508 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.508 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.508 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.508 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.508 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.508 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.508 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.508 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.508 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.509 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.509 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.509 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.509 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.509 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.509 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.509 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.509 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.510 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.510 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.510 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.510 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.510 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.510 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.510 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.510 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.510 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.510 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.511 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.511 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.511 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.511 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.511 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.511 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.511 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.511 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.512 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.512 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.512 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.512 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.512 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.512 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.512 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.512 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.512 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.512 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.513 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.513 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.513 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.513 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.513 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.513 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.513 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.513 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.514 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.514 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.514 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.514 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.514 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.514 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.514 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.514 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.515 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.515 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-04-23T14:59:13.515 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-04-23T14:59:13.515 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.515 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.515 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.515 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.515 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.515 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.516 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.516 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.516 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.516 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.516 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.516 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-04-23T14:59:13.516 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-04-23T14:59:13.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.517 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.517 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.517 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-04-23T14:59:13.517 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.517 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.517 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.517 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.517 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.517 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.517 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.518 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.518 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-04-23T14:59:13.518 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-04-23T14:59:13.518 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.518 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.518 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.518 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T14:59:13.518 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.518 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.519 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.519 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.519 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.519 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.519 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.519 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.519 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.519 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.519 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-23T14:59:13.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.520 INFO:teuthology.orchestra.run.smithi083.stdout: "gid": 25208, 2024-04-23T14:59:13.520 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "y", 2024-04-23T14:59:13.520 INFO:teuthology.orchestra.run.smithi083.stdout: "mgr_features": 4540701547738038271, 2024-04-23T14:59:13.520 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-04-23T14:59:13.520 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.520 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-04-23T14:59:13.520 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.520 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.520 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.521 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:13.521 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:13.521 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.521 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.521 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.521 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:13.521 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.521 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.521 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.522 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T14:59:13.522 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.522 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.522 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.522 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.522 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.522 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.522 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.523 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.523 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.523 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.523 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.523 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.523 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.523 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.523 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.523 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.524 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.524 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.524 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.524 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.524 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.524 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.524 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.524 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.524 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.525 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.525 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.525 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.525 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.525 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.525 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.525 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.525 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.525 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.526 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.526 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.526 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.526 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.526 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.526 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.526 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.526 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.527 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.527 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.527 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.527 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.527 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.527 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.527 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.527 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.528 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.528 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.528 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.528 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.528 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.528 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.528 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.528 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.528 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.529 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.529 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.529 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.529 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.529 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.529 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.529 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.529 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.529 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.530 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.530 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.530 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-04-23T14:59:13.530 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-04-23T14:59:13.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.530 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.530 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.531 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.531 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.531 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.531 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.531 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-04-23T14:59:13.531 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.531 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.531 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.531 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.532 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-04-23T14:59:13.532 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-04-23T14:59:13.532 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.532 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.532 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-04-23T14:59:13.532 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.532 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.532 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.532 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-04-23T14:59:13.533 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.533 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.533 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.533 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.533 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-04-23T14:59:13.533 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-04-23T14:59:13.533 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.533 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.533 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.534 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.534 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.534 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.534 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.534 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-04-23T14:59:13.534 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.534 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.534 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.535 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.535 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-04-23T14:59:13.535 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-04-23T14:59:13.535 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.535 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.535 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.535 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.535 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.535 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.536 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.536 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-04-23T14:59:13.536 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.536 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.536 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.536 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-04-23T14:59:13.536 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-04-23T14:59:13.537 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.537 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.537 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.537 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-04-23T14:59:13.537 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.537 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.537 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.537 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-04-23T14:59:13.538 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.538 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.538 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.538 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-04-23T14:59:13.538 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-04-23T14:59:13.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.539 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.539 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.539 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.539 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.539 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.539 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.539 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-04-23T14:59:13.539 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.539 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.540 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.540 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-04-23T14:59:13.540 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-04-23T14:59:13.540 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.540 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.540 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.540 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.540 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.541 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.541 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.541 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T14:59:13.541 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.541 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.541 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.541 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.541 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-04-23T14:59:13.541 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-04-23T14:59:13.542 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.542 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.542 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.542 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.542 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.542 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.542 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.542 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-04-23T14:59:13.542 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.543 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.543 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.543 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.543 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.543 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.543 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.543 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-04-23T14:59:13.543 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.543 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.544 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.544 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-04-23T14:59:13.544 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-04-23T14:59:13.544 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.544 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.544 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.544 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.544 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.544 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.544 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.545 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T14:59:13.545 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.545 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.545 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.545 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.545 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-04-23T14:59:13.545 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-04-23T14:59:13.545 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.545 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.546 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.546 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-04-23T14:59:13.546 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.546 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.546 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.546 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T14:59:13.546 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T14:59:13.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.546 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.547 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.547 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-04-23T14:59:13.547 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-04-23T14:59:13.547 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:13.547 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.547 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:13.547 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-23T14:59:13.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-23T14:59:13.547 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.548 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T14:59:13.548 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T14:59:13.548 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.548 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.548 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-04-23T14:59:13.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-04-23T14:59:13.548 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:13.549 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.549 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-04-23T14:59:13.549 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-23T14:59:13.549 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-04-23T14:59:13.549 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.549 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T14:59:13.549 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.549 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.550 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.550 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-04-23T14:59:13.550 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-04-23T14:59:13.550 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.550 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.550 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.550 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T14:59:13.551 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.551 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.551 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.551 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T14:59:13.551 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T14:59:13.551 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.551 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.551 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.551 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-04-23T14:59:13.551 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-04-23T14:59:13.552 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:13.552 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.552 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.552 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-04-23T14:59:13.552 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-04-23T14:59:13.552 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-04-23T14:59:13.552 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.552 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-04-23T14:59:13.552 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T14:59:13.552 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.553 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.553 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.553 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-04-23T14:59:13.553 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-04-23T14:59:13.553 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.553 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.553 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.553 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-04-23T14:59:13.553 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.554 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.554 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.554 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T14:59:13.554 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T14:59:13.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.554 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.554 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-04-23T14:59:13.554 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-04-23T14:59:13.555 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:13.555 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.555 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.555 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:13.555 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-23T14:59:13.555 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-23T14:59:13.555 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.555 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T14:59:13.555 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T14:59:13.556 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.556 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.556 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.556 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.556 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.556 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.556 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.557 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.557 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.557 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.557 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.557 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.557 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.557 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.557 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.557 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.557 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.558 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.558 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.558 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.558 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.558 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.558 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.558 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.558 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.559 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.559 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.559 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.559 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.559 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.559 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.559 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.559 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.559 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.560 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.560 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.560 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.560 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.560 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.560 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.560 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.560 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.560 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.561 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.561 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.561 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.561 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.561 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.561 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.561 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.561 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.561 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.562 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.562 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.562 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.562 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.562 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.562 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.562 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.562 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.563 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.563 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.563 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.563 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.563 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.563 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.563 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.563 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.563 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.563 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-04-23T14:59:13.564 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-04-23T14:59:13.564 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:13.564 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.564 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.564 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:13.564 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.564 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.564 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.565 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T14:59:13.565 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.565 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.565 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.565 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.565 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-23T14:59:13.565 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-23T14:59:13.565 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.565 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.566 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-04-23T14:59:13.566 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.566 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.566 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.566 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-04-23T14:59:13.566 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-04-23T14:59:13.566 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-04-23T14:59:13.566 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-04-23T14:59:13.566 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-04-23T14:59:13.567 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.567 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-04-23T14:59:13.567 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.567 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.567 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.567 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.567 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-04-23T14:59:13.567 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-04-23T14:59:13.568 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.568 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.568 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.568 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.568 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.568 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.568 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.568 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T14:59:13.569 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.569 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.569 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.569 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:13.569 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:13.569 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.569 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.569 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.570 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:13.570 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.570 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.570 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.570 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T14:59:13.570 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.570 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.570 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.571 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-04-23T14:59:13.571 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-04-23T14:59:13.571 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.571 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.571 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.571 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-23T14:59:13.571 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-23T14:59:13.571 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.571 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.572 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T14:59:13.572 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T14:59:13.572 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.572 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.572 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-04-23T14:59:13.572 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-04-23T14:59:13.572 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:13.572 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.573 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.573 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:13.573 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.573 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.573 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.573 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T14:59:13.573 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.573 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.573 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.573 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.574 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.574 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-04-23T14:59:13.574 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.574 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.574 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.574 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-04-23T14:59:13.574 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-04-23T14:59:13.575 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:13.575 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.575 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.575 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-23T14:59:13.575 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.575 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.575 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.575 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T14:59:13.575 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.576 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.576 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.576 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.576 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-04-23T14:59:13.576 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-04-23T14:59:13.576 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.576 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.576 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.576 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-04-23T14:59:13.577 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.577 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.577 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.577 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T14:59:13.577 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.577 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.577 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.577 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-04-23T14:59:13.577 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-04-23T14:59:13.578 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.578 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.578 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.578 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-04-23T14:59:13.578 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.578 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.578 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.578 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T14:59:13.579 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.579 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.579 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.579 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.579 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-04-23T14:59:13.579 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-04-23T14:59:13.579 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.579 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.579 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.579 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.580 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.580 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.580 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.580 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T14:59:13.580 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T14:59:13.580 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.580 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.580 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.580 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-04-23T14:59:13.581 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-04-23T14:59:13.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.581 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.581 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.581 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:13.581 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.581 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.581 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.581 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T14:59:13.582 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.582 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.582 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.582 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-04-23T14:59:13.582 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T14:59:13.582 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:13.582 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.582 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.583 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-04-23T14:59:13.583 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.583 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.583 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.583 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T14:59:13.583 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.583 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.583 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.583 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.584 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-04-23T14:59:13.584 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-04-23T14:59:13.584 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.584 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.584 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.584 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.584 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.584 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.584 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.585 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-04-23T14:59:13.585 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-04-23T14:59:13.585 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-04-23T14:59:13.585 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.585 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T14:59:13.585 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.585 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.585 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.585 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-04-23T14:59:13.586 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-04-23T14:59:13.586 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.586 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.586 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.586 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.586 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.586 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.586 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.586 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T14:59:13.587 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.587 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.587 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.587 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-04-23T14:59:13.587 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-04-23T14:59:13.587 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.587 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.588 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.588 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.588 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.588 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.588 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T14:59:13.588 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.588 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.588 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.588 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.588 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-04-23T14:59:13.589 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-04-23T14:59:13.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.589 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.589 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.589 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.589 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.589 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.589 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.590 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T14:59:13.590 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.590 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.590 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.590 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.590 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-04-23T14:59:13.590 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-04-23T14:59:13.590 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.590 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.591 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.591 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T14:59:13.591 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.591 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.591 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.591 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-23T14:59:13.591 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.591 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.591 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.592 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-04-23T14:59:13.592 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-04-23T14:59:13.592 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.592 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.592 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.592 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T14:59:13.592 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.592 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.593 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-04-23T14:59:13.593 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.593 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.593 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.593 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-04-23T14:59:13.593 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-04-23T14:59:13.593 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.593 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.593 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.594 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T14:59:13.594 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.594 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.594 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.594 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-04-23T14:59:13.594 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.594 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.594 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.594 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.595 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-04-23T14:59:13.595 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-04-23T14:59:13.595 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.595 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.595 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.595 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T14:59:13.595 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.595 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.595 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.596 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-23T14:59:13.596 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.596 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.596 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-04-23T14:59:13.596 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-04-23T14:59:13.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.596 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.597 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.597 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T14:59:13.597 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.597 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.597 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.597 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-04-23T14:59:13.597 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.597 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.597 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.598 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-04-23T14:59:13.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-04-23T14:59:13.598 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.598 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.598 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.598 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T14:59:13.598 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.598 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.598 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.599 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-04-23T14:59:13.599 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.599 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.599 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.599 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.599 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-04-23T14:59:13.599 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-04-23T14:59:13.599 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.599 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.599 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.600 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T14:59:13.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.600 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.600 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.600 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-04-23T14:59:13.600 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.600 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.600 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.601 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-04-23T14:59:13.601 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-04-23T14:59:13.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.601 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.601 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.601 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T14:59:13.601 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.601 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.601 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.601 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-04-23T14:59:13.602 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.602 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.602 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.602 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.602 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-04-23T14:59:13.602 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-04-23T14:59:13.602 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.602 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.603 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T14:59:13.603 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.603 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.603 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.603 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-04-23T14:59:13.603 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.603 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.603 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.603 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-04-23T14:59:13.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-04-23T14:59:13.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.604 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.604 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.604 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T14:59:13.604 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.604 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.604 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.604 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-04-23T14:59:13.605 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.605 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.605 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.605 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.605 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-04-23T14:59:13.605 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-04-23T14:59:13.605 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.605 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.605 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T14:59:13.606 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.606 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.606 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.606 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-23T14:59:13.606 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.606 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.606 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.606 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-04-23T14:59:13.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-04-23T14:59:13.607 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.607 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.607 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.607 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T14:59:13.607 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.607 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.607 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.607 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-04-23T14:59:13.607 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.608 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.608 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.608 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-04-23T14:59:13.608 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-04-23T14:59:13.608 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.608 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.608 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.608 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T14:59:13.609 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.609 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.609 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.609 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-23T14:59:13.609 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.609 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.609 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.609 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.609 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-04-23T14:59:13.610 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-04-23T14:59:13.610 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.610 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.610 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.610 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T14:59:13.610 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.610 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.610 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.610 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-04-23T14:59:13.611 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.611 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.611 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.611 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.611 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-04-23T14:59:13.611 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-04-23T14:59:13.611 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.611 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.611 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.612 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T14:59:13.612 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.612 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.612 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.612 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-04-23T14:59:13.612 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.612 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.612 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.613 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-04-23T14:59:13.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-04-23T14:59:13.613 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.613 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.613 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.613 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T14:59:13.613 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.613 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.613 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.613 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-04-23T14:59:13.614 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.614 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.614 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.614 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.614 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-04-23T14:59:13.614 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-04-23T14:59:13.614 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.614 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.614 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.615 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.615 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.615 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.615 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.615 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T14:59:13.615 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.615 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.615 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.615 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-04-23T14:59:13.616 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-04-23T14:59:13.616 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.616 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.616 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.616 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:13.616 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.616 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.616 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.616 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T14:59:13.617 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.617 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.617 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-04-23T14:59:13.617 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T14:59:13.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.617 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.618 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-04-23T14:59:13.618 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.618 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.618 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.618 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T14:59:13.618 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.618 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.618 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.618 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.618 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-04-23T14:59:13.619 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-04-23T14:59:13.619 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.619 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.619 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.619 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-04-23T14:59:13.619 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.619 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.619 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.619 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T14:59:13.620 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.620 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.620 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.620 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.620 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-04-23T14:59:13.620 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-04-23T14:59:13.620 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.620 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.621 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-04-23T14:59:13.621 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.621 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.621 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.621 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-04-23T14:59:13.621 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.621 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.621 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-04-23T14:59:13.622 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-04-23T14:59:13.622 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.622 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.622 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.622 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.622 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.622 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.622 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.622 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T14:59:13.623 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.623 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.623 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.623 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.623 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-04-23T14:59:13.623 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-04-23T14:59:13.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.623 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.623 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:13.624 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.624 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.624 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-04-23T14:59:13.624 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.624 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.624 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.624 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.624 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-04-23T14:59:13.624 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-04-23T14:59:13.625 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.625 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.625 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.625 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T14:59:13.625 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.625 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.625 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.625 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T14:59:13.625 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.626 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.626 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.626 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.626 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-04-23T14:59:13.626 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-04-23T14:59:13.626 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.626 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.626 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.626 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:13.627 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.627 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.627 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.627 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-04-23T14:59:13.627 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.627 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.627 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.627 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-04-23T14:59:13.628 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-04-23T14:59:13.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.628 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.628 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.628 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.628 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.628 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.628 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T14:59:13.629 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.629 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.629 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.629 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.629 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-04-23T14:59:13.629 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-04-23T14:59:13.629 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.629 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.629 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.629 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.630 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.630 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.630 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.630 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T14:59:13.630 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.630 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.631 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.631 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.631 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.631 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.631 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.631 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.632 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.632 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.632 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.632 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.632 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.632 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.632 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.632 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.632 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.633 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.633 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.633 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.633 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-04-23T14:59:13.633 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-04-23T14:59:13.633 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.633 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.633 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.633 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.633 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.634 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.634 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.634 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T14:59:13.634 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.634 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.634 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.635 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.635 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.635 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.635 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.636 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.636 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.636 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T14:59:13.636 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.636 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.636 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.636 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.636 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.637 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.637 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.637 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.637 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.637 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.637 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.637 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.637 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.637 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.637 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.638 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.638 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.638 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.638 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.638 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.638 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.638 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.638 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.638 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.639 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.639 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.639 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.639 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.639 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.639 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.639 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.639 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.639 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.640 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.640 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.640 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.640 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.640 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.640 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T14:59:13.640 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T14:59:13.640 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.640 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.641 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.641 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.641 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.641 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.641 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T14:59:13.641 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.641 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.641 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.642 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T14:59:13.642 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T14:59:13.642 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.642 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.642 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.642 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-04-23T14:59:13.642 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.642 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.642 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.643 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T14:59:13.643 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.643 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.643 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.643 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.643 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-04-23T14:59:13.643 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-04-23T14:59:13.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.643 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.644 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.644 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:13.644 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.644 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.644 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.644 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-04-23T14:59:13.644 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.644 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.644 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.645 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-04-23T14:59:13.645 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-04-23T14:59:13.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.645 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.645 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:13.645 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.645 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.645 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.646 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T14:59:13.646 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.646 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.646 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.646 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-04-23T14:59:13.646 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-04-23T14:59:13.646 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.646 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.647 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.647 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.647 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.647 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.647 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.647 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-04-23T14:59:13.647 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.647 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.647 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.648 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.648 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-23T14:59:13.648 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-23T14:59:13.648 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.648 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.648 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.648 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-04-23T14:59:13.648 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.648 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.649 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.649 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-04-23T14:59:13.649 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-04-23T14:59:13.649 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.649 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T14:59:13.649 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.649 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.649 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.649 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-04-23T14:59:13.650 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-04-23T14:59:13.650 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.650 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.650 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.650 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-04-23T14:59:13.650 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.650 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.651 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.651 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T14:59:13.651 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.651 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.651 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.651 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.651 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-04-23T14:59:13.651 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-04-23T14:59:13.651 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.652 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.652 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.652 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T14:59:13.652 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.652 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.652 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.652 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T14:59:13.652 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.653 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.653 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.653 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-04-23T14:59:13.653 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-04-23T14:59:13.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.653 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.653 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.653 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.654 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.654 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.654 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T14:59:13.654 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.654 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.654 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.654 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-04-23T14:59:13.655 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-04-23T14:59:13.655 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.655 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.655 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.655 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.655 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.655 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.655 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.655 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T14:59:13.656 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.656 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.656 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.656 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-04-23T14:59:13.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-04-23T14:59:13.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.656 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.656 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.656 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.657 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.657 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.657 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T14:59:13.657 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.657 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.657 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.657 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.657 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-04-23T14:59:13.658 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-04-23T14:59:13.658 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.658 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.658 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.658 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.658 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.658 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.658 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.658 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T14:59:13.658 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.659 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.659 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.659 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.659 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-04-23T14:59:13.659 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-04-23T14:59:13.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.659 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.659 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.659 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.660 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.660 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.660 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.660 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T14:59:13.660 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.660 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.660 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.660 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-04-23T14:59:13.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-04-23T14:59:13.661 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.661 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.661 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.661 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-04-23T14:59:13.661 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.661 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.661 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.661 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-04-23T14:59:13.661 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.662 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.662 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.662 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-04-23T14:59:13.662 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-04-23T14:59:13.662 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.662 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.662 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.663 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.663 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.663 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.663 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T14:59:13.663 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.663 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.663 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.663 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.663 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-04-23T14:59:13.663 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-04-23T14:59:13.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.664 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.664 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.664 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.664 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.664 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.664 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.664 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T14:59:13.664 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.665 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.665 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-04-23T14:59:13.665 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-04-23T14:59:13.665 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.665 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.666 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.666 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.666 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.666 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T14:59:13.666 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.666 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.666 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.666 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.666 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-04-23T14:59:13.667 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-04-23T14:59:13.667 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.667 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.667 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.667 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.667 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.667 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.667 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.667 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T14:59:13.667 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.668 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.668 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.668 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-04-23T14:59:13.668 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-04-23T14:59:13.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.668 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.668 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.669 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.669 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.669 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.669 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T14:59:13.669 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.669 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.669 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.669 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-04-23T14:59:13.669 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-04-23T14:59:13.670 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.670 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.670 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.670 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.670 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.670 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.670 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.670 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T14:59:13.671 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.671 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.671 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.671 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.671 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.671 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-04-23T14:59:13.672 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.672 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.672 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.672 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.672 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.672 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.672 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.672 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.672 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.673 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.673 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.673 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.673 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.673 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.673 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.673 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.673 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.673 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.674 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.674 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.674 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.674 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.674 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.674 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.675 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.675 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.675 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.675 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.675 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.675 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.675 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.675 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.676 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.676 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.676 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.676 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.676 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.676 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.676 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.676 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.676 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.676 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.677 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.677 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.677 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.677 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.677 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.677 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.677 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.677 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.677 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.678 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.678 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.678 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.678 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.678 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.678 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.678 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.678 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.679 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.679 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.679 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.679 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.679 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.679 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.679 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.679 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-04-23T14:59:13.679 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-04-23T14:59:13.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.680 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.680 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.680 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-04-23T14:59:13.680 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.680 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.680 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.680 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T14:59:13.680 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.681 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.681 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.681 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.681 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-04-23T14:59:13.681 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-04-23T14:59:13.681 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.681 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.681 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.681 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-04-23T14:59:13.682 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.682 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.682 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.682 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T14:59:13.682 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.682 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.682 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.683 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.683 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-04-23T14:59:13.683 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.683 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.683 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.683 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T14:59:13.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T14:59:13.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.684 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.684 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.684 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:13.684 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.684 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.684 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.684 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.684 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.684 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.684 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.685 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T14:59:13.685 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T14:59:13.685 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.685 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.685 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.685 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.685 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.685 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.685 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.686 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.686 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.686 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.686 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.686 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T14:59:13.686 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T14:59:13.686 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.686 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.687 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.687 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.687 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.687 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.687 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.687 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.687 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.687 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.687 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-04-23T14:59:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T14:59:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.688 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T14:59:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T14:59:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.689 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.690 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T14:59:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T14:59:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.691 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T14:59:13.692 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T14:59:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.693 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T14:59:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T14:59:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.694 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T14:59:13.695 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T14:59:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.696 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T14:59:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T14:59:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.697 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T14:59:13.698 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T14:59:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.699 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T14:59:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T14:59:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.700 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T14:59:13.701 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T14:59:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.702 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T14:59:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T14:59:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.703 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T14:59:13.704 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T14:59:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-23T14:59:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.705 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T14:59:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T14:59:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.706 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-04-23T14:59:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-04-23T14:59:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.708 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.709 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T14:59:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T14:59:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-23T14:59:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.710 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T14:59:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T14:59:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.711 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T14:59:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T14:59:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.713 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T14:59:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T14:59:13.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T14:59:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T14:59:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.716 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T14:59:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T14:59:13.717 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "[]", 2024-04-23T14:59:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.718 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T14:59:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T14:59:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "{}", 2024-04-23T14:59:13.719 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T14:59:13.720 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T14:59:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.721 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T14:59:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T14:59:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.722 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.723 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T14:59:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T14:59:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.724 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T14:59:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T14:59:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.725 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.726 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T14:59:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T14:59:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.727 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T14:59:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T14:59:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.728 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.729 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T14:59:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T14:59:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.730 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T14:59:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T14:59:13.731 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.732 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T14:59:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T14:59:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.733 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T14:59:13.734 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T14:59:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.735 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T14:59:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T14:59:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.736 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T14:59:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.737 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T14:59:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T14:59:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.738 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T14:59:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T14:59:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.739 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-04-23T14:59:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T14:59:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T14:59:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-04-23T14:59:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.741 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.742 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.742 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.742 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.742 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.742 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T14:59:13.742 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T14:59:13.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.742 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.743 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.743 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.743 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.743 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.743 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.743 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.743 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.743 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.743 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T14:59:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T14:59:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-23T14:59:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.744 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T14:59:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T14:59:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.745 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T14:59:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T14:59:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.747 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T14:59:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T14:59:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.748 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T14:59:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T14:59:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.750 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T14:59:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T14:59:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.751 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.752 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.753 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T14:59:13.753 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T14:59:13.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.753 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.753 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.753 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-23T14:59:13.753 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.753 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.753 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.754 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.754 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.754 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.754 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.754 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-04-23T14:59:13.754 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-04-23T14:59:13.754 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.754 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.755 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.755 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.755 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.755 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.755 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.755 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.755 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.755 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.755 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.756 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-04-23T14:59:13.756 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-04-23T14:59:13.756 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.756 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.756 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.756 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.756 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.756 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.756 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.757 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.757 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.757 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.757 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.757 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.757 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-04-23T14:59:13.757 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-04-23T14:59:13.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.757 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.758 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.758 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.758 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.758 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.758 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.758 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-04-23T14:59:13.758 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.758 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.758 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.759 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.759 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-04-23T14:59:13.759 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-04-23T14:59:13.759 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.759 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.759 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.759 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-04-23T14:59:13.759 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.759 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.760 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.760 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.760 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.760 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.760 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.760 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-23T14:59:13.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-23T14:59:13.760 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.761 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.761 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.761 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.761 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.761 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.761 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.761 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.761 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.762 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.762 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.762 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.762 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.762 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.762 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.762 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.763 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.763 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.763 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.763 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.763 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.763 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.763 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.763 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.763 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.764 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.764 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.764 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.764 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.764 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.764 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.764 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.765 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.765 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.765 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.765 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.765 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.765 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.765 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.766 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.766 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.766 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.766 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.766 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.766 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.766 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.766 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.767 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.767 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.767 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.767 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.767 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.767 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.767 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.767 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.767 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.768 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.768 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.768 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.768 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.768 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.768 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.768 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.768 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.769 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.769 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.769 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.769 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.769 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.769 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.769 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.770 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.770 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.770 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.770 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-04-23T14:59:13.770 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-04-23T14:59:13.770 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.770 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.771 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.771 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.771 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.771 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.771 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.771 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-04-23T14:59:13.771 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.772 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.772 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.772 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.772 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-04-23T14:59:13.772 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T14:59:13.772 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.772 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.772 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.772 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.772 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.773 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.773 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.773 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.773 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.773 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.773 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.773 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-23T14:59:13.773 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-23T14:59:13.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.774 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.774 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.774 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-23T14:59:13.774 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.774 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.774 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.774 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.774 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.775 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.775 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.775 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-23T14:59:13.775 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-23T14:59:13.775 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.775 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.775 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.775 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-04-23T14:59:13.776 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.776 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.776 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.776 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.776 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.776 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.776 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.776 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-23T14:59:13.776 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-23T14:59:13.777 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.777 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.777 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.777 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.777 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.777 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.777 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.777 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.778 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.778 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.778 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-04-23T14:59:13.778 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-04-23T14:59:13.778 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.778 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.778 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.779 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-04-23T14:59:13.779 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.779 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.779 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.779 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.779 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.779 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.779 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.779 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.780 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-23T14:59:13.780 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-23T14:59:13.780 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.780 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.780 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.780 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-04-23T14:59:13.780 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.780 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-04-23T14:59:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.781 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.782 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-23T14:59:13.782 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-23T14:59:13.782 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.782 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.782 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.782 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-23T14:59:13.782 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-23T14:59:13.782 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-23T14:59:13.783 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.783 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.783 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.783 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.783 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.783 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-04-23T14:59:13.783 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-04-23T14:59:13.783 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.784 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.784 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.784 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.784 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.784 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.784 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.784 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.785 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.785 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.785 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.785 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.785 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.785 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.785 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-04-23T14:59:13.785 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.786 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.786 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.786 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-04-23T14:59:13.786 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-04-23T14:59:13.786 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.786 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.786 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.786 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.787 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.787 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.787 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.787 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-04-23T14:59:13.787 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.787 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.787 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.787 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.788 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.788 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.788 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.788 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.788 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.788 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.788 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.788 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.789 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.789 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.789 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.789 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.789 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.789 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.789 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.789 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.790 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.790 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.790 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.790 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.790 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.790 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.790 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.791 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.791 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.791 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.791 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.791 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.791 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.791 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.791 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.791 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.792 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.792 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.792 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.792 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.792 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.792 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.792 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.792 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.792 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.793 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.793 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.793 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.793 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.793 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.793 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.793 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.793 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.793 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.794 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.794 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.794 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.794 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.794 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.794 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.794 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.794 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.795 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.795 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.795 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.795 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.795 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.795 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.795 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.795 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-04-23T14:59:13.796 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-04-23T14:59:13.796 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.796 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.796 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.796 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-04-23T14:59:13.796 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.796 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.796 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.796 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T14:59:13.796 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.797 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.797 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.797 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-04-23T14:59:13.797 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-04-23T14:59:13.797 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.797 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.797 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.797 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-04-23T14:59:13.798 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.798 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.798 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.798 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T14:59:13.798 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.798 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.798 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.798 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.798 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-04-23T14:59:13.799 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-04-23T14:59:13.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.799 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.799 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.799 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-04-23T14:59:13.799 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.799 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.799 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.799 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-04-23T14:59:13.800 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.800 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.800 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.800 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.800 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-04-23T14:59:13.800 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-04-23T14:59:13.800 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.800 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.800 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.800 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-23T14:59:13.801 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.801 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.801 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.801 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T14:59:13.801 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.801 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.801 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.801 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-04-23T14:59:13.802 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-04-23T14:59:13.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.802 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.802 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.802 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.802 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.802 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.802 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.802 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T14:59:13.803 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.803 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.803 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.803 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.803 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:13.803 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:13.803 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.803 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.803 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.804 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:13.804 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.804 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.804 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.804 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T14:59:13.804 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.804 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.804 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.804 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-04-23T14:59:13.805 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-04-23T14:59:13.805 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.805 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.805 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.805 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-04-23T14:59:13.805 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.805 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.805 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.805 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T14:59:13.805 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.806 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.806 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.806 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.806 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.806 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.806 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-04-23T14:59:13.806 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.807 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.807 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.807 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.807 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.807 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.807 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.807 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.807 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.808 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.808 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.808 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.808 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.808 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.808 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.808 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.808 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.808 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.808 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.809 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.809 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.809 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.809 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.809 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.809 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.809 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.809 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.809 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.810 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.810 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.810 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.810 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.810 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.810 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.810 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.810 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.811 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.811 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.811 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.811 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.811 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.811 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.811 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.811 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.811 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.812 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.812 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.812 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.812 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.812 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.812 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.812 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.812 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.812 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.813 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.813 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.813 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.813 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.813 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.813 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.813 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.813 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.814 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.814 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.814 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.814 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.814 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.814 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.814 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.815 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-04-23T14:59:13.815 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-04-23T14:59:13.815 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.815 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.815 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.815 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-23T14:59:13.815 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.815 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.815 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.816 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.816 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.816 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.816 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.816 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-04-23T14:59:13.816 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-04-23T14:59:13.816 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.816 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.816 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.817 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-04-23T14:59:13.817 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.817 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.817 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.817 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.817 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.817 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.817 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.817 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.818 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:13.818 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:13.818 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.818 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.818 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.818 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:13.818 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.818 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.818 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.819 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.819 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.819 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.819 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.819 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.819 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.819 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.819 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-04-23T14:59:13.820 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-04-23T14:59:13.820 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-04-23T14:59:13.820 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.820 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-04-23T14:59:13.820 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-04-23T14:59:13.820 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.820 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.820 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.820 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-04-23T14:59:13.820 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.821 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.821 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.821 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T14:59:13.821 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.821 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.821 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.821 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.821 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-04-23T14:59:13.821 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-04-23T14:59:13.822 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.822 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.822 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.822 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-04-23T14:59:13.822 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.822 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.822 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.822 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T14:59:13.823 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.823 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.823 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.823 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-04-23T14:59:13.823 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-04-23T14:59:13.823 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.823 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.823 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.824 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.824 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.824 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.824 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.824 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-04-23T14:59:13.824 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.824 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.824 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.824 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.825 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:13.825 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:13.825 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.825 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.825 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.825 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-04-23T14:59:13.825 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-04-23T14:59:13.825 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.825 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.826 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T14:59:13.826 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.826 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.826 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.826 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.826 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.826 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.826 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.827 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.827 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.827 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.827 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.827 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.827 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.827 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.827 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.827 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.828 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.828 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.828 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.828 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.828 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.828 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.828 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.829 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.829 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.829 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.829 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.829 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.829 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.829 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.829 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.829 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.830 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.830 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.830 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.830 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.830 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.830 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.830 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.830 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.830 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.831 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.831 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.831 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.831 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.831 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.831 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.831 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.831 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.831 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.832 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.832 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.832 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.832 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.832 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.832 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.832 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.832 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.832 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.833 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.833 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.833 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.833 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.833 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.833 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.833 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.833 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.833 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.834 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.834 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.834 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.834 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-04-23T14:59:13.834 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-04-23T14:59:13.834 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.834 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.834 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.835 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.835 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.835 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.835 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.835 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-04-23T14:59:13.835 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.835 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.835 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.836 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-04-23T14:59:13.836 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-04-23T14:59:13.836 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.836 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.836 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.836 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-04-23T14:59:13.836 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.836 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.836 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.837 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-04-23T14:59:13.837 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.837 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.837 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.837 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.837 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-23T14:59:13.837 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-23T14:59:13.837 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.837 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.838 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.838 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-04-23T14:59:13.838 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.838 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.838 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.838 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T14:59:13.838 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.838 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.838 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.839 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-04-23T14:59:13.839 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-04-23T14:59:13.839 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.839 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.839 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.839 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-23T14:59:13.839 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-23T14:59:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-04-23T14:59:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T14:59:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-04-23T14:59:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-04-23T14:59:13.840 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-04-23T14:59:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.841 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-04-23T14:59:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-04-23T14:59:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-04-23T14:59:13.842 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.843 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.843 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.843 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T14:59:13.843 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.843 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.843 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.843 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.843 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.843 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.844 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-04-23T14:59:13.844 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.844 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.844 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.844 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.844 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.844 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.844 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.844 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.845 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.845 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.845 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.845 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.845 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.845 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.845 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.845 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.845 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.845 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.846 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.846 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.846 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.846 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.846 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.846 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.846 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.846 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.847 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.847 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.847 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.847 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.847 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.847 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.847 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.847 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.847 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.848 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.848 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.848 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.848 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.848 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.848 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.848 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.848 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.848 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.849 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.849 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.849 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.849 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.849 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.849 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.849 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.849 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.850 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.850 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.850 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.850 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.850 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.850 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.850 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.850 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.850 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.851 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.851 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.851 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.851 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.851 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.851 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.851 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.851 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.851 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.851 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.852 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.852 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.852 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.852 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-04-23T14:59:13.852 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.852 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.852 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.852 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.852 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.853 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.853 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.853 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.853 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.853 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.853 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.853 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.853 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.853 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.854 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.854 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.854 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.854 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.854 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.854 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.854 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.854 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.854 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.855 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.855 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.855 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.855 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.855 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.855 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.855 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.855 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.856 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.856 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.856 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.856 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.856 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.856 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.856 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.856 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.856 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.857 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.857 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.857 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.857 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.857 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.857 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.857 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.857 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.857 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.858 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.858 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.858 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.858 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.858 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.858 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.858 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.858 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.859 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.859 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.859 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.859 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.859 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.859 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.859 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.860 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.860 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.860 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.860 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.860 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.860 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.860 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-04-23T14:59:13.860 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.860 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.861 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.861 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-04-23T14:59:13.861 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-04-23T14:59:13.861 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.861 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.861 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.861 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-04-23T14:59:13.861 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.861 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.862 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.862 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T14:59:13.862 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.862 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.862 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.862 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-04-23T14:59:13.862 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-04-23T14:59:13.862 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.863 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.863 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.863 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:13.863 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-04-23T14:59:13.863 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.863 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.863 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T14:59:13.863 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.863 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.864 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.864 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.864 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.864 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.864 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.864 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.864 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.864 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.864 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.865 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.865 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.865 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.865 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.865 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.865 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.865 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.865 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.865 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.866 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.866 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.866 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.866 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.866 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.866 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.866 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.867 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.867 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.867 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.867 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.867 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.867 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.867 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.867 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.867 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.868 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.868 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.868 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.868 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.868 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.868 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.868 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.868 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.868 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.869 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.869 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.869 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.869 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.869 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.869 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.869 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.869 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.869 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.869 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.870 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.870 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.870 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.870 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.870 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.870 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.870 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.870 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.871 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.871 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.871 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.871 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.871 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.871 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.871 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.871 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.872 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.872 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-04-23T14:59:13.872 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.872 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.872 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.872 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-04-23T14:59:13.872 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-04-23T14:59:13.872 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.872 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.873 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.873 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-04-23T14:59:13.873 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.873 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.873 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.873 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-04-23T14:59:13.873 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T14:59:13.873 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.873 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.874 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.874 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.874 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.874 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.874 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.874 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.874 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.874 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.874 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.875 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.875 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.875 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.875 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.875 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.875 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.875 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.875 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.875 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.876 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.876 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.876 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.876 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.877 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.877 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.877 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.877 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.877 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.877 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.877 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.877 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.877 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.878 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.878 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.878 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.878 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.878 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.878 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.878 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.878 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.879 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.879 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.879 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.879 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.879 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.879 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.879 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.879 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.879 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.880 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.880 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.880 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.880 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.880 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.880 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.880 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.880 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.880 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.880 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.881 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.881 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.881 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.881 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.881 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.881 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.881 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.881 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.881 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.882 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.882 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-04-23T14:59:13.882 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-04-23T14:59:13.882 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.882 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.882 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.882 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.882 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.883 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.883 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.883 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-04-23T14:59:13.883 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T14:59:13.883 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.883 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.883 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-04-23T14:59:13.883 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-04-23T14:59:13.883 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.884 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.884 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.884 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-04-23T14:59:13.884 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.884 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.884 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.884 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-04-23T14:59:13.884 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.885 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.885 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.885 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-04-23T14:59:13.885 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-04-23T14:59:13.885 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.885 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.885 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.885 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-04-23T14:59:13.886 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.886 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.886 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.886 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T14:59:13.886 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.886 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.886 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.886 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.886 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-04-23T14:59:13.887 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-04-23T14:59:13.887 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.887 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.887 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.887 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.887 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.887 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.887 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.887 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-04-23T14:59:13.888 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.888 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.888 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.888 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-04-23T14:59:13.888 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-04-23T14:59:13.888 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.888 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.888 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.888 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-04-23T14:59:13.889 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.889 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.889 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.889 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T14:59:13.889 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T14:59:13.889 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.889 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.889 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.890 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.890 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.890 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-04-23T14:59:13.890 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.890 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.890 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.890 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.890 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.891 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.891 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.891 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.891 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.891 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.891 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.891 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.891 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.891 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.891 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.892 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.892 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.892 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.892 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.892 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.892 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.892 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.892 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.892 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.893 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.893 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.893 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.893 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.893 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.893 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.893 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.893 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.893 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.893 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.894 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.894 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.894 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.894 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.894 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.894 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.894 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.894 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.894 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.895 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.895 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.895 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.895 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.895 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.895 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.895 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.895 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.895 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.896 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.896 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.896 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.896 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.896 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.896 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.896 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.896 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.896 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.897 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.897 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.897 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.897 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.897 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.897 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.897 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.897 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.898 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.898 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.898 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.898 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.898 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.898 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-04-23T14:59:13.898 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.898 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.898 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.899 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.899 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.899 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.899 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.899 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.899 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.899 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.899 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.899 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.899 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.900 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.900 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.900 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.900 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.900 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.900 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.900 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.900 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.900 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.901 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.901 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.901 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.901 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.901 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.901 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.901 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.901 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.901 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.901 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.902 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.902 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.902 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.902 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.902 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.902 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.902 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.902 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.902 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.903 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.903 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.903 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.903 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.903 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.903 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.903 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.903 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.903 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.903 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.904 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.904 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.904 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.904 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.904 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.904 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.904 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.904 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.904 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.905 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.905 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.905 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.905 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.905 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.905 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.905 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.905 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.905 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.905 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.906 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.906 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.906 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.906 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.906 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.906 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.906 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-04-23T14:59:13.906 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.906 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.907 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.907 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.907 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.907 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.907 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.907 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.907 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.907 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.907 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.908 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.908 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.908 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.908 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.908 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.908 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.908 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.908 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.908 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.908 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.909 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.909 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.909 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.909 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.909 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.909 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.909 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.909 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.909 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.910 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.910 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.910 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.910 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.910 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.910 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.910 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.910 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.910 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.911 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.911 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.911 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.911 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.911 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.911 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.911 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.911 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.911 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.912 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.912 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.912 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.912 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.912 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.912 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.912 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.912 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.912 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.912 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.913 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.913 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.913 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.913 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.913 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.913 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.913 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.913 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.913 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.914 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.914 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.914 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.914 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.914 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.914 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.914 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.914 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.914 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.915 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-23T14:59:13.915 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.915 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.915 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.915 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-04-23T14:59:13.915 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-04-23T14:59:13.915 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.915 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.915 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.916 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.916 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.916 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.916 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.916 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T14:59:13.916 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.916 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.916 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.916 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.916 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.917 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.917 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.917 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.917 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.917 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.917 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.917 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.917 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.917 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.917 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.918 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.918 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.918 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.918 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.918 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.918 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.918 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.918 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.919 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.919 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.919 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.919 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.919 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.919 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.919 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.919 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.919 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.919 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.920 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.920 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.920 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.920 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.920 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.920 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.920 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.920 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.920 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.921 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.921 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.921 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.921 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.921 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.921 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.921 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.921 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.921 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.922 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.922 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.922 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.922 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.922 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.922 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.922 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.922 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.922 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.923 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.923 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.923 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.923 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.923 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.923 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.923 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.923 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.923 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.924 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.924 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.924 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.924 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.924 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.924 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-04-23T14:59:13.924 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-23T14:59:13.924 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.924 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.924 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.925 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.925 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.925 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.925 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.925 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-04-23T14:59:13.925 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-04-23T14:59:13.925 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-04-23T14:59:13.925 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.925 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-04-23T14:59:13.926 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.926 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.926 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.926 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.926 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.926 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.926 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.926 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-04-23T14:59:13.926 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.926 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.927 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.927 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.927 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.927 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.927 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.927 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.927 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.927 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.927 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.928 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.928 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.928 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.928 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.928 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.928 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.928 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.928 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.928 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.929 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.929 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.929 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.929 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.929 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.929 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.929 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.929 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.929 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.930 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.930 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.930 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.930 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.930 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.930 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.930 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.930 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.931 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.931 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.931 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.931 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.931 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.931 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.931 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.931 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.931 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.931 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.932 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.932 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.932 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.932 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.932 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.932 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.932 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.932 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.932 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.933 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.933 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.933 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.933 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.933 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.933 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.933 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.933 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.934 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.934 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.934 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.934 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.934 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.934 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.934 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.934 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.934 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.935 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.935 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-04-23T14:59:13.935 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.935 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.935 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.935 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.935 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.935 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.936 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.936 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.936 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.936 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.936 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.936 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.936 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.936 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.936 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.937 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.937 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.937 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.937 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.937 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.937 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.937 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.937 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.938 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.938 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.938 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.938 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.938 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.938 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.938 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.938 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.938 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.939 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.939 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.939 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.939 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.939 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.939 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.939 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.939 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.940 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.940 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.940 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.940 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.940 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.940 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.940 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.940 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.940 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.941 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.941 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.941 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.941 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.941 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.941 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.941 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.941 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.941 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.941 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.942 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.942 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.942 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.942 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.942 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.942 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.942 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.942 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.942 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.943 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.943 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.943 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.943 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.943 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.943 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.943 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-04-23T14:59:13.943 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.943 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.943 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.944 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.944 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.944 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.944 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.944 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.944 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.944 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.944 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.944 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.945 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.945 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.945 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.945 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.945 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.945 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.945 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.945 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.945 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.945 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.946 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.946 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.946 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.946 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.946 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.946 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.946 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.946 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.947 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.947 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.947 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.947 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.947 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.947 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.947 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.947 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.948 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.948 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.948 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.948 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.948 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.948 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.948 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.948 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.948 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.948 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.949 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.949 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.949 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.949 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.949 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.949 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.949 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.949 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.949 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.950 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.950 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.950 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.950 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.950 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.950 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.950 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.950 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.951 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.951 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.951 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.951 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.951 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.951 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:13.951 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:13.951 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.952 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.952 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.952 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:13.952 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.952 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.952 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.953 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.953 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.953 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.953 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.953 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-04-23T14:59:13.953 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-04-23T14:59:13.953 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:13.953 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.954 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.954 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-23T14:59:13.954 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-04-23T14:59:13.954 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.954 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.954 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-04-23T14:59:13.954 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T14:59:13.954 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.954 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.955 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.955 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.955 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.955 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-04-23T14:59:13.955 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.955 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.955 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.956 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-04-23T14:59:13.956 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-04-23T14:59:13.956 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.956 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.956 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.956 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.956 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.956 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.956 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.957 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T14:59:13.957 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.957 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.957 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.957 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.957 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-23T14:59:13.957 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-23T14:59:13.957 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.957 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.957 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.958 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.958 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.958 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.958 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.958 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.958 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.958 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.958 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.959 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.959 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.959 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.959 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.959 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.959 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.959 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.959 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.959 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.960 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.960 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.960 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.960 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.960 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.960 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.960 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.960 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.960 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.961 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.961 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.961 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.961 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.961 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.961 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.961 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.961 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.962 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.962 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.962 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.962 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.962 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.962 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.962 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.962 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.962 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.963 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.963 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.963 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.963 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.963 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.963 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.963 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.963 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.963 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.964 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.964 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.964 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.964 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.964 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.964 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.964 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.964 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.964 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.965 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.965 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.965 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.965 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.965 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.965 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.965 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.965 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.966 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.966 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.966 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.966 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.966 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.966 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-04-23T14:59:13.966 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-04-23T14:59:13.966 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.967 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.967 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.967 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-04-23T14:59:13.967 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.967 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.967 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.967 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-04-23T14:59:13.967 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.967 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.968 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.968 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:13.968 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:13.968 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:13.968 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.968 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.968 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-23T14:59:13.968 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.968 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.969 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.969 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-04-23T14:59:13.969 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.969 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.969 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.969 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.969 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.969 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.969 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.969 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-04-23T14:59:13.970 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.970 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.970 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.970 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-04-23T14:59:13.970 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-04-23T14:59:13.970 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.970 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.970 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.971 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.971 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.971 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.971 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.971 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.971 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.971 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.971 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.972 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.972 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-04-23T14:59:13.972 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-04-23T14:59:13.972 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.972 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.972 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.972 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:13.972 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.972 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.973 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.973 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T14:59:13.973 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T14:59:13.973 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.973 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.973 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.973 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.973 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.974 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.974 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.974 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.974 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.974 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.974 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.974 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.974 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.975 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.975 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.975 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.975 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.975 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.975 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.975 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.975 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.975 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.976 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.976 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.976 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.976 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.976 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.976 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.976 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.976 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.976 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.977 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.977 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.977 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.977 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.977 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.977 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.977 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.977 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.977 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.978 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.978 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.978 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.978 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.978 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.978 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.978 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.978 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.978 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.978 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.979 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.979 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.979 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.979 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.979 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.979 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.979 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.979 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:13.979 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:13.980 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.980 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.980 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.980 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.980 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.980 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.980 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.980 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.981 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.981 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.981 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.981 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-04-23T14:59:13.981 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-04-23T14:59:13.981 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.981 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.981 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.982 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.982 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.982 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.982 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.982 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.982 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.982 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.982 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.983 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.983 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T14:59:13.983 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T14:59:13.983 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.983 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.983 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.983 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-04-23T14:59:13.983 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.983 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.983 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.984 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.984 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.984 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.984 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.984 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-04-23T14:59:13.984 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-04-23T14:59:13.984 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:13.984 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.985 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.985 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-04-23T14:59:13.985 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.985 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.985 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.985 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.985 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.985 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.985 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.986 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-23T14:59:13.986 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-23T14:59:13.986 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.986 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.986 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.986 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-23T14:59:13.986 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.986 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.987 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.987 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T14:59:13.987 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.987 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.987 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.987 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-23T14:59:13.987 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-23T14:59:13.987 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.988 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.988 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.988 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-04-23T14:59:13.988 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.988 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.988 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.988 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T14:59:13.988 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.988 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.988 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.989 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.989 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-04-23T14:59:13.989 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-04-23T14:59:13.989 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.989 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.989 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:13.989 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-04-23T14:59:13.989 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.989 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.990 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.990 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.990 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.990 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.990 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.990 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.990 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-23T14:59:13.990 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-23T14:59:13.991 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.991 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.991 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.991 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-04-23T14:59:13.991 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.991 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.991 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.991 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-04-23T14:59:13.992 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-04-23T14:59:13.992 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.992 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.992 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.992 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.992 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.992 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.992 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-23T14:59:13.992 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-23T14:59:13.993 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:13.993 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.993 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.993 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-23T14:59:13.993 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-23T14:59:13.993 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-23T14:59:13.993 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.993 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.994 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.994 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.994 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.994 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.994 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:13.994 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.994 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:13.994 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-04-23T14:59:13.994 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:13.995 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:13.995 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:13.995 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:13.995 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:13.995 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.995 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.995 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.995 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:13.995 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.996 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.996 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:13.996 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:13.996 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:13.996 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:13.996 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:13.996 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:13.996 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:13.996 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:13.997 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.997 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.997 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.997 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.997 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.997 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:13.997 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:13.997 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:13.998 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.998 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.998 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:13.998 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:13.998 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:13.998 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:13.998 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:13.998 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:13.998 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:13.999 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:13.999 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:13.999 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:13.999 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:13.999 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:13.999 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:13.999 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:13.999 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:13.999 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.000 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.000 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.000 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.000 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.000 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.000 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.000 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.000 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.001 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.001 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.001 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.001 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.001 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.001 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.001 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.001 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.001 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.002 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.002 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.002 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.002 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.002 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.002 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.002 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.002 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.002 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.003 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.003 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.003 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-04-23T14:59:14.003 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-04-23T14:59:14.003 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.003 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.003 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.003 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:14.003 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.004 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.004 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.004 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.004 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.004 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.004 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.004 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.004 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-04-23T14:59:14.004 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T14:59:14.005 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.005 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.005 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.005 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.005 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.005 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.005 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.005 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.005 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.005 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.006 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.006 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.006 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-04-23T14:59:14.006 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-04-23T14:59:14.006 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.006 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.006 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.006 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.007 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.007 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.007 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.007 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.007 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.007 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.007 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.007 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.007 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.008 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.008 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.008 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-04-23T14:59:14.008 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.008 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.008 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.008 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-04-23T14:59:14.008 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-04-23T14:59:14.008 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.008 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.009 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.009 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.009 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.009 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.009 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.009 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.009 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.009 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.009 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.010 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.010 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-23T14:59:14.010 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-23T14:59:14.010 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.010 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.010 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.010 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.010 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.011 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.011 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.011 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.011 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.011 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.011 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.011 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.011 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.011 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.012 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.012 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.012 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.012 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.012 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.012 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.012 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.012 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.012 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.013 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.013 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.013 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.013 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.013 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.013 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.013 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.013 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.013 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.014 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.014 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.014 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.014 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.014 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.014 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.014 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.014 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.014 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.015 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.015 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.015 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.015 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.015 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.015 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.015 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.015 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.016 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.016 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.016 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.016 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.016 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.016 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.016 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.016 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.016 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.017 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.017 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.017 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.017 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.017 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.017 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.017 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.017 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.017 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.018 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.018 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.018 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.018 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.018 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.018 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.018 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.018 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.018 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.019 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.019 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.019 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.019 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.019 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.019 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.019 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-23T14:59:14.019 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-23T14:59:14.019 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.020 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.020 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.020 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.020 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.020 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.020 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.020 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.020 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.020 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.020 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.021 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.021 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-23T14:59:14.021 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-23T14:59:14.021 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.021 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.021 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.021 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.021 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.021 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.022 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.022 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.022 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.022 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.022 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.022 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.022 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.022 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.023 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-04-23T14:59:14.023 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.023 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.023 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.023 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.023 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.023 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.023 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.023 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.023 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.024 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.024 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.024 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.024 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.024 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.024 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.024 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.024 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.024 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.025 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.025 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.025 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.025 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.025 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.025 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.025 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.025 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.025 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.026 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.026 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.026 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.026 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.026 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.026 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.026 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.026 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.026 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.027 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.027 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.027 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.027 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.027 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.027 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.027 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.027 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.027 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.027 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.028 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.028 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.028 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.028 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.028 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.028 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.028 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.028 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.028 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.029 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.029 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.029 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.029 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.029 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.029 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.029 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.030 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.030 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.030 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.030 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.030 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.030 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.030 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.030 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.031 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.031 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.031 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.031 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.031 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.031 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.031 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-04-23T14:59:14.031 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.031 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.032 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.032 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.032 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.032 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.032 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.032 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.032 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.032 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.032 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.032 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.033 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.033 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.033 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.033 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.033 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.033 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.033 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.033 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.033 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.034 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.034 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.034 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.034 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.034 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.034 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.034 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.034 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.034 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.035 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.035 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.035 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.035 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.035 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.035 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.035 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.035 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.035 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.036 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.036 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.036 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.036 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.036 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.036 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.036 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.036 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.036 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.037 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.037 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.037 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.037 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.037 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.037 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.037 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.037 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.037 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.038 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.038 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.038 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.038 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.038 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.038 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.038 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.038 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.038 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.039 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.039 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.039 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.039 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.039 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.039 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.039 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.039 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-04-23T14:59:14.039 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-04-23T14:59:14.040 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.040 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.040 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.040 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-04-23T14:59:14.040 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.040 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.040 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.040 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T14:59:14.040 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.040 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.041 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.041 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.041 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.041 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.041 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.041 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-04-23T14:59:14.041 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.041 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.042 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.042 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.042 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.042 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.042 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.042 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.042 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.042 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.042 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.043 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.043 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.043 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.043 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.043 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.043 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.043 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.043 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.043 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.044 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.044 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.044 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.044 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.044 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.044 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.044 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.044 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.044 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.045 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.045 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.045 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.045 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.045 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.045 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.045 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.045 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.045 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.045 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.046 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.046 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.046 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.046 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.046 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.046 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.046 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.047 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.047 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.047 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.047 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.047 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.047 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.047 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.047 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.047 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.047 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.048 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.048 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.048 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.048 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.048 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.048 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.048 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.048 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.049 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.049 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.049 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.049 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.049 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.049 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.049 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.049 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.049 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.050 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-04-23T14:59:14.050 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-04-23T14:59:14.050 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.050 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.050 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.050 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.050 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.050 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.051 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.051 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.051 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.051 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.051 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.051 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.051 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-04-23T14:59:14.051 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-04-23T14:59:14.051 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.051 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.052 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.052 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-04-23T14:59:14.052 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.052 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.052 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.052 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.052 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.052 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.052 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.053 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.053 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-04-23T14:59:14.053 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-04-23T14:59:14.053 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.053 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.053 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.053 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.053 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.053 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.054 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.054 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.054 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.054 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.054 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.054 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.054 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-04-23T14:59:14.054 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-04-23T14:59:14.055 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.055 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.055 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.055 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.055 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.055 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.055 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.055 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.055 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.056 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.056 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.056 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.056 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-04-23T14:59:14.056 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-04-23T14:59:14.056 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:14.056 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.056 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.056 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.056 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.057 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.057 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.057 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.057 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.057 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.057 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.057 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.057 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-04-23T14:59:14.057 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-04-23T14:59:14.058 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.058 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.058 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.058 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.058 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.058 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.058 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.058 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.058 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.059 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.059 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.059 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.059 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-04-23T14:59:14.059 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-04-23T14:59:14.059 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.059 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.059 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.059 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.059 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.060 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.060 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.060 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.060 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.060 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.060 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.060 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.060 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-04-23T14:59:14.060 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-04-23T14:59:14.061 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.061 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.061 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.061 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.061 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.061 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.061 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.061 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.061 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.062 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.062 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.062 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.062 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-04-23T14:59:14.062 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-04-23T14:59:14.062 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.062 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.062 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.062 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.063 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-23T14:59:14.063 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-04-23T14:59:14.063 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.063 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.063 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.063 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.063 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.063 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.063 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-04-23T14:59:14.064 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-04-23T14:59:14.064 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.064 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.064 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.064 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.064 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.064 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.064 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.064 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.065 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.065 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.065 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.065 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.065 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-04-23T14:59:14.065 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-04-23T14:59:14.065 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.065 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.065 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.066 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.066 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.066 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.066 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.066 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.066 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.066 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.066 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.066 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.067 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-04-23T14:59:14.067 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-04-23T14:59:14.067 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.067 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.067 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.067 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.067 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.067 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.067 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.067 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.068 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.068 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.068 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.068 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.068 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.068 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.068 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.068 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-23T14:59:14.069 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.069 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.069 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.069 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-04-23T14:59:14.069 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-04-23T14:59:14.069 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.069 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.069 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.069 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.070 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.070 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.070 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.070 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T14:59:14.070 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.070 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.070 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.070 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.070 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-04-23T14:59:14.071 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-04-23T14:59:14.071 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.071 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.071 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.071 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.071 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.071 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.071 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.072 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-04-23T14:59:14.072 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.072 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.072 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.072 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.072 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.072 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.072 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.072 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.073 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.073 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.073 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.073 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.073 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.073 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.073 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.073 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.073 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.073 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.074 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.074 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.074 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.074 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.074 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.074 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.074 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.074 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.074 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.075 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.075 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.075 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.075 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.075 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.075 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.075 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.075 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.075 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.076 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.076 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.076 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.076 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.076 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.076 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.076 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.076 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.076 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.077 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.077 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.077 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.077 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.077 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.077 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.077 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.077 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.077 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.078 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.078 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.078 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.078 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.078 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.078 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.078 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.078 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.079 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.079 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.079 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.079 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.079 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.079 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.079 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.079 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.079 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.079 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.080 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.080 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.080 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.080 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.080 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.080 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-04-23T14:59:14.080 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.080 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.080 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.080 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.081 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.081 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.081 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.081 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.081 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.081 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.081 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.081 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.081 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.082 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.082 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.082 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.082 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.082 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.082 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.082 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.082 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.082 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.083 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.083 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.083 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.083 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.083 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.083 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.083 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.083 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.083 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.083 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.084 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.084 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.084 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.084 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.084 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.084 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.084 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.084 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.085 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.085 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.085 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.085 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.085 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.085 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.085 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.085 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.085 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.086 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.086 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.086 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.086 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.086 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.086 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.086 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.086 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.086 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.087 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.087 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.087 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.087 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.087 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.087 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.087 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.087 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.087 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.088 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.088 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.088 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.088 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.088 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.088 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.088 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.088 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.088 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.088 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-04-23T14:59:14.089 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.089 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.089 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.089 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.089 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.089 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.089 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.089 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.089 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.090 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.090 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.090 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.090 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.090 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.090 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.090 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.090 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.091 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.091 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.091 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.091 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.091 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.091 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.091 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.091 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.092 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.092 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.092 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.092 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.092 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.092 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.092 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.092 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.092 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.092 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.093 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.093 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.093 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.093 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.093 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.093 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.093 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.093 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.093 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.094 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.094 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.094 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.094 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.094 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.094 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.094 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.094 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.094 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.094 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.095 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.095 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.095 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.095 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.095 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.095 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.095 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.095 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.095 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.096 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.096 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.096 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.096 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.096 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.096 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.096 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.096 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.096 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.096 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.097 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.097 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.097 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.097 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-04-23T14:59:14.097 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.097 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.097 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.097 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-04-23T14:59:14.097 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-04-23T14:59:14.098 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.098 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.098 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.098 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T14:59:14.098 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.098 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.098 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.098 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.098 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.099 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.099 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.099 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.099 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:14.099 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:14.099 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.099 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.099 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.099 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-04-23T14:59:14.099 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.100 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.100 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.100 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.100 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.100 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.100 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.100 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.100 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.100 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.100 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.101 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.101 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.101 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.101 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.101 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.101 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.101 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.101 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.101 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.102 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.102 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.102 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.102 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.102 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.102 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.102 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.102 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.102 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.103 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.103 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.103 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.103 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.103 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.103 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.103 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.103 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.104 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.104 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.104 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.104 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.104 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.104 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.104 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.104 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.104 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.104 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.105 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.105 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.105 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.105 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.105 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.105 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.105 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.105 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.105 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.106 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.106 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.106 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.106 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.106 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.106 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.106 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.106 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.106 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.107 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.107 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.107 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.107 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.107 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.107 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.107 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.107 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.108 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.108 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.108 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.108 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.108 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.108 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.108 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.108 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.108 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-04-23T14:59:14.108 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.109 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.109 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.109 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-04-23T14:59:14.109 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-04-23T14:59:14.109 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.109 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.109 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.109 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.110 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.110 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.110 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.110 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T14:59:14.110 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.110 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.110 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.110 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.111 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-04-23T14:59:14.111 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-04-23T14:59:14.111 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.111 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.111 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.111 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.111 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.111 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.112 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.112 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T14:59:14.112 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.112 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.112 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.112 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.112 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-04-23T14:59:14.112 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-04-23T14:59:14.112 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.113 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.113 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.113 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.113 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.113 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.113 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.113 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T14:59:14.113 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.114 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.114 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.114 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.114 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-04-23T14:59:14.114 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-04-23T14:59:14.114 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.114 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.114 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.114 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.115 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.115 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.115 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.115 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T14:59:14.115 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.115 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.115 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.115 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.115 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-04-23T14:59:14.115 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-04-23T14:59:14.116 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.116 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.116 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.116 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.116 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.116 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.116 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.117 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T14:59:14.117 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.117 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.117 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.117 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.117 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-04-23T14:59:14.117 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-04-23T14:59:14.117 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.117 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.118 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.118 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.118 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.118 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.118 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.118 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.118 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.118 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.119 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.119 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.119 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-04-23T14:59:14.119 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-04-23T14:59:14.119 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.119 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.119 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.119 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.119 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.120 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.120 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.120 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.120 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.120 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.120 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.120 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.120 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-04-23T14:59:14.120 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-04-23T14:59:14.120 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.121 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.121 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.121 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T14:59:14.121 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.121 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.121 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.121 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.121 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.121 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.122 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.122 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.122 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-23T14:59:14.122 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-23T14:59:14.122 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.122 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.122 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.122 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.122 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.123 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.123 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.123 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.123 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.123 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.123 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.123 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.123 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:14.123 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:14.123 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.124 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.124 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.124 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-04-23T14:59:14.124 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-04-23T14:59:14.124 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.124 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.124 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.124 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.124 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.124 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.125 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.125 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-04-23T14:59:14.125 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-04-23T14:59:14.125 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.125 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.125 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.125 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-04-23T14:59:14.125 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.125 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.126 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.126 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.126 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.126 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.126 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.126 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.126 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-04-23T14:59:14.126 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-04-23T14:59:14.126 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.126 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.127 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.127 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.127 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.127 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.127 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.127 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.127 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.127 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.127 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.128 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.128 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-04-23T14:59:14.128 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-04-23T14:59:14.128 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.128 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.128 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.128 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.128 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.128 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.128 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.129 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.129 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.129 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.129 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.129 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.129 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.129 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.129 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.129 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.130 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.130 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.130 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.130 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.130 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.130 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.130 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.130 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.131 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.131 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.131 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.131 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.131 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.131 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.131 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.131 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.131 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.132 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.132 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.132 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.132 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.132 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.132 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.132 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.132 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.132 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.133 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.133 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.133 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.133 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.133 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.133 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.133 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.133 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.133 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.134 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.134 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.134 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.134 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.134 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.134 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.134 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.134 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.135 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.135 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.135 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.135 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.135 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.135 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.135 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.135 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.136 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.136 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.136 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.136 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.136 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.136 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.136 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.136 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.136 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.137 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.137 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.137 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.137 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.137 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.137 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.137 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-04-23T14:59:14.137 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-04-23T14:59:14.137 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.138 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.138 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.138 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.138 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.138 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.138 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.138 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.138 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.138 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.139 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.139 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.139 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-04-23T14:59:14.139 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-04-23T14:59:14.139 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.139 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.139 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.139 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.139 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.140 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.140 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.140 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.140 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.140 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.140 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.140 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.140 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-04-23T14:59:14.140 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-04-23T14:59:14.141 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.141 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.141 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.141 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T14:59:14.141 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.141 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.141 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.141 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.141 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.142 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.142 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.142 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.142 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.142 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.142 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.142 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-04-23T14:59:14.142 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.142 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.143 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.143 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.143 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.143 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.143 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.143 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.143 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.143 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.143 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.143 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.144 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.144 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.144 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.144 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.144 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.144 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.144 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.144 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.144 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.145 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.145 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.145 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.145 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.145 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.145 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.145 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.145 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.145 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.145 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.146 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.146 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.146 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.146 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.146 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.146 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.146 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.146 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.147 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.147 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.147 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.147 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.147 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.147 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.147 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.147 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.147 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.147 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.148 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.148 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.148 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.148 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.148 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.148 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.148 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.148 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.148 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.149 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.149 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.149 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.149 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.149 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.149 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.149 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.149 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.149 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.150 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.150 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.150 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.150 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.150 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.150 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.150 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.151 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-04-23T14:59:14.151 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.151 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.151 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.151 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.151 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.151 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.151 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.151 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.152 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.152 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.152 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.152 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.152 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.152 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.152 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.152 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.153 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.153 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.153 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.153 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.153 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.153 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.153 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.153 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.153 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.154 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.154 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.154 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.154 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.154 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.154 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.154 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.154 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.155 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.155 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.155 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.155 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.155 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.155 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.155 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.155 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.155 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.156 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.156 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.156 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.156 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.156 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.156 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.156 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.156 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.156 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.157 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.157 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.157 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.157 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.157 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.157 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.157 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.157 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.157 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.157 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.158 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.158 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.158 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.158 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.158 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.158 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.158 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.158 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.158 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.159 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.159 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.159 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.159 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-04-23T14:59:14.159 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-04-23T14:59:14.159 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.159 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.159 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.159 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-04-23T14:59:14.160 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.160 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.160 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.160 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T14:59:14.160 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.160 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.160 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.160 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.160 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-04-23T14:59:14.161 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-04-23T14:59:14.161 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.161 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.161 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.161 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.161 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.161 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.161 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.161 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-04-23T14:59:14.161 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.162 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.162 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.162 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.162 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-04-23T14:59:14.162 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-04-23T14:59:14.162 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.162 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.162 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.162 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:14.163 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.163 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.163 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.163 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T14:59:14.163 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.163 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.163 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.163 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.163 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-04-23T14:59:14.164 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T14:59:14.164 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.164 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.164 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.164 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.164 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.164 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.164 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.164 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T14:59:14.165 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.165 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.165 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.165 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.165 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.165 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.165 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-04-23T14:59:14.165 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.166 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.166 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.166 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-04-23T14:59:14.166 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-04-23T14:59:14.166 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:14.166 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.166 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.166 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-04-23T14:59:14.166 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.167 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.167 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.167 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.167 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.167 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.167 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.167 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.167 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-04-23T14:59:14.167 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-04-23T14:59:14.167 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.168 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.168 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.168 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.168 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.168 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.168 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.168 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.168 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.168 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.168 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.169 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:14.169 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:14.169 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.169 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.169 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.169 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:14.169 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.169 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.170 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.170 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.170 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.170 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.170 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.170 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.170 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.170 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.170 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.171 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.171 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.171 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.171 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.171 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.171 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.171 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.171 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.171 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.172 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.172 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.172 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.172 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.172 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.172 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.172 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.172 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.173 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.173 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.173 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.173 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.173 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.173 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.173 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.173 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.173 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.174 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.174 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.174 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.174 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.174 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.174 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.174 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.174 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.175 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.175 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.175 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.175 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.175 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.175 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.175 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.175 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.175 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.176 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.176 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.176 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.176 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.176 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.176 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.176 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.176 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.176 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.177 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.177 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.177 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.177 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.177 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.177 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.177 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.177 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.177 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.178 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.178 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.178 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.178 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.178 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.178 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.178 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-04-23T14:59:14.178 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-04-23T14:59:14.178 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.179 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.179 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.179 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.179 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.179 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.179 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.179 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.179 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.179 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.180 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.180 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-04-23T14:59:14.180 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-04-23T14:59:14.180 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.180 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.180 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.180 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-04-23T14:59:14.180 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.180 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.181 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.181 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.181 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.181 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.181 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.181 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.181 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-04-23T14:59:14.181 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-04-23T14:59:14.181 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.182 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.182 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.182 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T14:59:14.182 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.182 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.182 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.182 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.182 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.183 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.183 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.183 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.183 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.183 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.183 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-23T14:59:14.183 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.183 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.183 INFO:teuthology.orchestra.run.smithi083.stdout: "modules": [ 2024-04-23T14:59:14.184 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-04-23T14:59:14.184 INFO:teuthology.orchestra.run.smithi083.stdout: "dashboard", 2024-04-23T14:59:14.184 INFO:teuthology.orchestra.run.smithi083.stdout: "iostat", 2024-04-23T14:59:14.184 INFO:teuthology.orchestra.run.smithi083.stdout: "nfs", 2024-04-23T14:59:14.184 INFO:teuthology.orchestra.run.smithi083.stdout: "restful", 2024-04-23T14:59:14.184 INFO:teuthology.orchestra.run.smithi083.stdout: "snap_schedule" 2024-04-23T14:59:14.184 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.184 INFO:teuthology.orchestra.run.smithi083.stdout: "available_modules": [ 2024-04-23T14:59:14.184 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.184 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "alerts", 2024-04-23T14:59:14.185 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.185 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.185 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.185 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:14.185 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:14.185 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.185 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.185 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.185 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:14.186 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.186 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.186 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.186 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T14:59:14.186 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.186 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.186 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.186 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.187 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.187 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.187 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.187 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.187 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.187 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.187 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.187 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.187 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.188 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.188 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.188 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.188 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.188 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.188 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.188 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.188 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.188 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.188 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.189 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.189 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.189 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.189 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.189 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.189 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.189 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.189 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.189 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.190 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.190 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.190 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.190 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.190 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.190 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.190 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.190 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.191 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.191 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.191 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.191 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.191 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.191 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.191 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.191 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.191 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.191 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.192 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.192 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.192 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.192 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.192 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.192 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.192 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.192 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.192 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.193 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.193 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.193 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.193 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.193 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.193 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.193 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.193 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.193 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.193 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.194 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.194 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.194 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.194 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.194 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.194 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_destination": { 2024-04-23T14:59:14.194 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_destination", 2024-04-23T14:59:14.194 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.194 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.195 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.195 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.195 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.195 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.195 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.195 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email address to send alerts to", 2024-04-23T14:59:14.195 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.195 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.195 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.195 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.196 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_from_name": { 2024-04-23T14:59:14.196 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_from_name", 2024-04-23T14:59:14.196 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.196 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.196 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.196 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "Ceph", 2024-04-23T14:59:14.196 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.196 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.196 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.196 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Email From: name", 2024-04-23T14:59:14.197 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.197 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.197 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.197 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.197 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_host": { 2024-04-23T14:59:14.197 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_host", 2024-04-23T14:59:14.197 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.197 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.197 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.198 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.198 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.198 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.198 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.198 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP server", 2024-04-23T14:59:14.198 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.198 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.198 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.198 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.199 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_password": { 2024-04-23T14:59:14.199 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_password", 2024-04-23T14:59:14.199 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.199 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.199 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.199 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.199 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.199 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.199 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.199 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Password to authenticate with", 2024-04-23T14:59:14.200 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.200 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.200 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.200 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.200 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_port": { 2024-04-23T14:59:14.200 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_port", 2024-04-23T14:59:14.200 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.200 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.200 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.200 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "465", 2024-04-23T14:59:14.201 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.201 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.201 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.201 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP port", 2024-04-23T14:59:14.201 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.201 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.201 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.201 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.201 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_sender": { 2024-04-23T14:59:14.202 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_sender", 2024-04-23T14:59:14.202 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.202 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.202 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.202 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.202 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.202 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.202 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.202 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SMTP envelope sender", 2024-04-23T14:59:14.202 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.203 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.203 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.203 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.203 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_ssl": { 2024-04-23T14:59:14.203 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_ssl", 2024-04-23T14:59:14.203 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.203 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.203 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.203 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.204 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.204 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.204 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.204 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T14:59:14.204 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.204 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.204 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.204 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.204 INFO:teuthology.orchestra.run.smithi083.stdout: "smtp_user": { 2024-04-23T14:59:14.204 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "smtp_user", 2024-04-23T14:59:14.205 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.205 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.205 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.205 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.205 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.205 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.205 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.205 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "User to authenticate as", 2024-04-23T14:59:14.205 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.206 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.206 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.206 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.206 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.206 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.206 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.206 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "balancer", 2024-04-23T14:59:14.206 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.206 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.206 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.207 INFO:teuthology.orchestra.run.smithi083.stdout: "active": { 2024-04-23T14:59:14.207 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "active", 2024-04-23T14:59:14.207 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.207 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.207 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.207 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.207 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.207 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.207 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.208 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T14:59:14.208 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.208 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.208 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.208 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.208 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_time": { 2024-04-23T14:59:14.208 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_time", 2024-04-23T14:59:14.208 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.208 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.208 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.209 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0000", 2024-04-23T14:59:14.209 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.209 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.209 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.209 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T14:59:14.209 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T14:59:14.209 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.209 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.209 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.210 INFO:teuthology.orchestra.run.smithi083.stdout: "begin_weekday": { 2024-04-23T14:59:14.210 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "begin_weekday", 2024-04-23T14:59:14.210 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:14.210 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.210 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.210 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:14.210 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-23T14:59:14.210 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-23T14:59:14.210 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.211 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T14:59:14.211 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T14:59:14.211 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.211 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.211 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.211 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_max_iterations": { 2024-04-23T14:59:14.211 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_max_iterations", 2024-04-23T14:59:14.211 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:14.212 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.212 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.212 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "25", 2024-04-23T14:59:14.212 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-23T14:59:14.212 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "250", 2024-04-23T14:59:14.212 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.212 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T14:59:14.212 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.212 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.212 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.213 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.213 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_metrics": { 2024-04-23T14:59:14.213 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_metrics", 2024-04-23T14:59:14.213 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.213 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.213 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.213 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T14:59:14.213 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.214 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.214 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.214 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T14:59:14.214 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T14:59:14.214 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.214 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.214 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.214 INFO:teuthology.orchestra.run.smithi083.stdout: "crush_compat_step": { 2024-04-23T14:59:14.215 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crush_compat_step", 2024-04-23T14:59:14.215 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:14.215 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.215 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.215 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.5", 2024-04-23T14:59:14.215 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0.001", 2024-04-23T14:59:14.215 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "0.999", 2024-04-23T14:59:14.215 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.215 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "aggressiveness of optimization", 2024-04-23T14:59:14.216 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T14:59:14.216 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.216 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.216 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.216 INFO:teuthology.orchestra.run.smithi083.stdout: "end_time": { 2024-04-23T14:59:14.216 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_time", 2024-04-23T14:59:14.216 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.216 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.216 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.216 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2359", 2024-04-23T14:59:14.217 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.217 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.217 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.217 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T14:59:14.217 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T14:59:14.217 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.217 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.218 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.218 INFO:teuthology.orchestra.run.smithi083.stdout: "end_weekday": { 2024-04-23T14:59:14.218 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "end_weekday", 2024-04-23T14:59:14.218 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:14.218 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.218 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.218 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:14.218 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "0", 2024-04-23T14:59:14.219 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "6", 2024-04-23T14:59:14.219 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.219 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T14:59:14.219 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T14:59:14.219 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.219 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.219 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.219 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.219 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.220 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.220 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.220 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.220 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.220 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.220 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.220 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.220 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.220 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.221 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.221 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.221 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.221 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.221 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.221 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.221 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.221 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.221 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.222 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.222 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.222 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.222 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.222 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.222 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.222 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.222 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.223 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.223 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.223 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.223 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.223 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.223 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.223 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.223 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.223 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.224 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.224 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.224 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.224 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.224 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.224 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.224 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.224 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.224 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.224 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.225 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.225 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.225 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.225 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.225 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.225 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.225 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.225 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.225 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.226 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.226 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.226 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.226 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.226 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.226 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.226 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.227 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.227 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.227 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.227 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.227 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.227 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.227 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.227 INFO:teuthology.orchestra.run.smithi083.stdout: "min_score": { 2024-04-23T14:59:14.227 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_score", 2024-04-23T14:59:14.227 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:14.228 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.228 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.228 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:14.228 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.228 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.228 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.228 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T14:59:14.228 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.228 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.228 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.229 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.229 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-23T14:59:14.229 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-23T14:59:14.229 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.229 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.229 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.229 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "upmap", 2024-04-23T14:59:14.229 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.229 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.230 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.230 INFO:teuthology.orchestra.run.smithi083.stdout: "crush-compat", 2024-04-23T14:59:14.230 INFO:teuthology.orchestra.run.smithi083.stdout: "none", 2024-04-23T14:59:14.230 INFO:teuthology.orchestra.run.smithi083.stdout: "read", 2024-04-23T14:59:14.230 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap", 2024-04-23T14:59:14.230 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap-read" 2024-04-23T14:59:14.230 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.230 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Balancer mode", 2024-04-23T14:59:14.231 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.231 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.231 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.231 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.231 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_ids": { 2024-04-23T14:59:14.231 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_ids", 2024-04-23T14:59:14.231 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.231 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.231 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.232 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.232 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.232 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.232 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.232 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T14:59:14.232 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.232 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.232 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.232 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.232 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:14.233 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:14.233 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.233 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.233 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.233 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:14.233 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.233 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.233 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.233 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T14:59:14.234 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.234 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.234 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.234 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.234 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_deviation": { 2024-04-23T14:59:14.234 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_deviation", 2024-04-23T14:59:14.234 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.234 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.234 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.234 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-23T14:59:14.235 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-23T14:59:14.235 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.235 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.235 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T14:59:14.235 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T14:59:14.235 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.235 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.235 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.236 INFO:teuthology.orchestra.run.smithi083.stdout: "upmap_max_optimizations": { 2024-04-23T14:59:14.236 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "upmap_max_optimizations", 2024-04-23T14:59:14.236 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:14.236 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.236 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.236 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:14.236 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.236 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.236 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.237 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T14:59:14.237 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.237 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.237 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.237 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.237 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.237 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.237 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm", 2024-04-23T14:59:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_down_multiplier": { 2024-04-23T14:59:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_down_multiplier", 2024-04-23T14:59:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.238 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-23T14:59:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T14:59:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.239 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_refresh_rate": { 2024-04-23T14:59:14.240 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_refresh_rate", 2024-04-23T14:59:14.240 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.240 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.240 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.240 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "20", 2024-04-23T14:59:14.240 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.240 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.240 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.240 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T14:59:14.240 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.241 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.241 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.241 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.241 INFO:teuthology.orchestra.run.smithi083.stdout: "agent_starting_port": { 2024-04-23T14:59:14.241 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "agent_starting_port", 2024-04-23T14:59:14.241 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.241 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.241 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.241 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4721", 2024-04-23T14:59:14.241 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.242 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.242 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.242 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T14:59:14.242 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.242 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.242 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.242 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.242 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_ptrace": { 2024-04-23T14:59:14.243 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_ptrace", 2024-04-23T14:59:14.243 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.243 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.243 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.243 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.243 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.243 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.243 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.243 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T14:59:14.244 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T14:59:14.244 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.244 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.244 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.244 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_interval": { 2024-04-23T14:59:14.244 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_interval", 2024-04-23T14:59:14.244 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.244 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.244 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.245 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:14.245 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.245 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.245 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.245 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T14:59:14.245 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.245 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.245 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.245 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.245 INFO:teuthology.orchestra.run.smithi083.stdout: "autotune_memory_target_ratio": { 2024-04-23T14:59:14.246 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T14:59:14.246 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:14.246 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.246 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.246 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0.7", 2024-04-23T14:59:14.246 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.246 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.246 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T14:59:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm_log_destination": { 2024-04-23T14:59:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cephadm_log_destination", 2024-04-23T14:59:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.247 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: "file", 2024-04-23T14:59:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: "file,syslog", 2024-04-23T14:59:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: "syslog" 2024-04-23T14:59:14.248 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.249 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T14:59:14.249 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.249 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.249 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.249 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.249 INFO:teuthology.orchestra.run.smithi083.stdout: "cgroups_split": { 2024-04-23T14:59:14.249 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cgroups_split", 2024-04-23T14:59:14.249 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.249 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.250 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.250 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.250 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.250 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.250 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.250 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T14:59:14.250 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.250 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.251 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.251 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.251 INFO:teuthology.orchestra.run.smithi083.stdout: "config_checks_enabled": { 2024-04-23T14:59:14.251 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_checks_enabled", 2024-04-23T14:59:14.251 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.251 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.251 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.251 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.251 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.251 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.252 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.252 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T14:59:14.252 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.252 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.252 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.252 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.252 INFO:teuthology.orchestra.run.smithi083.stdout: "config_dashboard": { 2024-04-23T14:59:14.252 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_dashboard", 2024-04-23T14:59:14.252 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.253 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.253 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.253 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.253 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.253 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.253 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.253 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T14:59:14.253 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.253 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.253 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.254 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.254 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_alertmanager": { 2024-04-23T14:59:14.254 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_alertmanager", 2024-04-23T14:59:14.254 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.254 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.254 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.254 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T14:59:14.254 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.254 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.255 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.255 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-23T14:59:14.255 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.255 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.255 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.255 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.255 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_base": { 2024-04-23T14:59:14.255 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_base", 2024-04-23T14:59:14.255 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.256 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.256 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.256 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T14:59:14.256 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.256 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.256 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.256 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Container image name, without the tag", 2024-04-23T14:59:14.256 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.256 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.257 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.257 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.257 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_elasticsearch": { 2024-04-23T14:59:14.257 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_elasticsearch", 2024-04-23T14:59:14.257 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.257 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.257 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.257 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T14:59:14.257 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.257 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.258 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.258 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "elasticsearch container image", 2024-04-23T14:59:14.258 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.258 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.258 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.258 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.258 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_grafana": { 2024-04-23T14:59:14.258 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_grafana", 2024-04-23T14:59:14.258 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.259 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.259 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.259 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T14:59:14.259 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.259 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.259 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.259 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-23T14:59:14.259 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.259 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.260 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.260 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.260 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_haproxy": { 2024-04-23T14:59:14.260 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_haproxy", 2024-04-23T14:59:14.260 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.260 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.260 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.260 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T14:59:14.260 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.260 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.261 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.261 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "HAproxy container image", 2024-04-23T14:59:14.261 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.261 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.261 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.261 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.261 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_agent": { 2024-04-23T14:59:14.261 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_agent", 2024-04-23T14:59:14.261 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.262 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.262 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.262 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T14:59:14.262 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.262 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.262 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.262 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger agent container image", 2024-04-23T14:59:14.262 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.262 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.263 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.263 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.263 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_collector": { 2024-04-23T14:59:14.263 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_collector", 2024-04-23T14:59:14.263 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.263 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.263 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.263 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T14:59:14.264 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.264 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.264 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.264 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger collector container image", 2024-04-23T14:59:14.264 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.264 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.264 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.264 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.264 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_jaeger_query": { 2024-04-23T14:59:14.265 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_jaeger_query", 2024-04-23T14:59:14.265 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.265 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.265 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.265 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T14:59:14.265 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.265 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.265 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.265 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Jaeger query container image", 2024-04-23T14:59:14.266 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.266 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.266 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.266 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.266 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_keepalived": { 2024-04-23T14:59:14.266 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_keepalived", 2024-04-23T14:59:14.266 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.266 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.266 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.267 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T14:59:14.267 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.267 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.267 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.267 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Keepalived container image", 2024-04-23T14:59:14.267 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.267 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.267 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.267 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.268 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_loki": { 2024-04-23T14:59:14.268 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_loki", 2024-04-23T14:59:14.268 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.268 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.268 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.268 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T14:59:14.268 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.268 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.268 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.269 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Loki container image", 2024-04-23T14:59:14.269 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.269 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.269 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.269 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.269 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_node_exporter": { 2024-04-23T14:59:14.269 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_node_exporter", 2024-04-23T14:59:14.269 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.269 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.269 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.270 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T14:59:14.270 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.270 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.270 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.270 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-23T14:59:14.270 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.270 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.270 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.270 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.271 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_nvmeof": { 2024-04-23T14:59:14.271 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_nvmeof", 2024-04-23T14:59:14.271 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.271 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.271 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.271 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T14:59:14.271 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.271 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.271 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.272 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Nvme-of container image", 2024-04-23T14:59:14.272 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.272 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.272 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.272 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.272 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_prometheus": { 2024-04-23T14:59:14.272 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_prometheus", 2024-04-23T14:59:14.272 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.272 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.273 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.273 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T14:59:14.273 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.273 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.273 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.273 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Prometheus container image", 2024-04-23T14:59:14.273 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.273 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.273 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.274 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.274 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_promtail": { 2024-04-23T14:59:14.274 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_promtail", 2024-04-23T14:59:14.274 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.274 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.274 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.274 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T14:59:14.274 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.274 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.275 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.275 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Promtail container image", 2024-04-23T14:59:14.275 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.275 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.275 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.275 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.275 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_samba": { 2024-04-23T14:59:14.275 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_samba", 2024-04-23T14:59:14.275 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.276 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.276 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.276 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T14:59:14.276 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.276 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.276 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.276 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Samba/SMB container image", 2024-04-23T14:59:14.276 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.276 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.276 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.277 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.277 INFO:teuthology.orchestra.run.smithi083.stdout: "container_image_snmp_gateway": { 2024-04-23T14:59:14.277 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_image_snmp_gateway", 2024-04-23T14:59:14.277 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.277 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.277 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.277 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T14:59:14.277 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.277 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.278 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.278 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "SNMP Gateway container image", 2024-04-23T14:59:14.278 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.278 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.278 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.278 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.278 INFO:teuthology.orchestra.run.smithi083.stdout: "container_init": { 2024-04-23T14:59:14.278 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "container_init", 2024-04-23T14:59:14.278 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.279 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.279 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.279 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.279 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.279 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.279 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.279 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T14:59:14.279 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.279 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.280 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.280 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.280 INFO:teuthology.orchestra.run.smithi083.stdout: "daemon_cache_timeout": { 2024-04-23T14:59:14.280 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "daemon_cache_timeout", 2024-04-23T14:59:14.280 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.280 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.280 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.280 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:14.280 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.280 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.281 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.281 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T14:59:14.281 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.281 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.281 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.281 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.281 INFO:teuthology.orchestra.run.smithi083.stdout: "default_cephadm_command_timeout": { 2024-04-23T14:59:14.281 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T14:59:14.281 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.282 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.282 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.282 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "900", 2024-04-23T14:59:14.282 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.282 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.282 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.282 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T14:59:14.282 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.282 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.283 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.283 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.283 INFO:teuthology.orchestra.run.smithi083.stdout: "default_registry": { 2024-04-23T14:59:14.283 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "default_registry", 2024-04-23T14:59:14.283 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.283 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.283 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.283 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "docker.io", 2024-04-23T14:59:14.283 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.284 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.284 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.284 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T14:59:14.284 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.284 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.284 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.284 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.284 INFO:teuthology.orchestra.run.smithi083.stdout: "device_cache_timeout": { 2024-04-23T14:59:14.284 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_cache_timeout", 2024-04-23T14:59:14.285 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.285 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.285 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.285 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1800", 2024-04-23T14:59:14.285 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.285 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.285 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.285 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache device inventory", 2024-04-23T14:59:14.285 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.285 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.286 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.286 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.286 INFO:teuthology.orchestra.run.smithi083.stdout: "device_enhanced_scan": { 2024-04-23T14:59:14.286 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_enhanced_scan", 2024-04-23T14:59:14.286 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.286 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.286 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.286 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.287 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.287 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.287 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.287 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T14:59:14.287 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.287 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.287 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.287 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.287 INFO:teuthology.orchestra.run.smithi083.stdout: "facts_cache_timeout": { 2024-04-23T14:59:14.288 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "facts_cache_timeout", 2024-04-23T14:59:14.288 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.288 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.288 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.288 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:14.288 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.288 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.288 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.289 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "seconds to cache host facts data", 2024-04-23T14:59:14.289 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.289 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.289 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.289 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.289 INFO:teuthology.orchestra.run.smithi083.stdout: "grafana_dashboards_path": { 2024-04-23T14:59:14.289 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "grafana_dashboards_path", 2024-04-23T14:59:14.289 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.289 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.289 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.290 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T14:59:14.290 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.290 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.290 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.290 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T14:59:14.290 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.290 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.290 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.290 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.291 INFO:teuthology.orchestra.run.smithi083.stdout: "host_check_interval": { 2024-04-23T14:59:14.291 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "host_check_interval", 2024-04-23T14:59:14.291 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.291 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.291 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.291 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:14.291 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.291 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.291 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.292 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to perform a host check", 2024-04-23T14:59:14.292 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.292 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.292 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.292 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.292 INFO:teuthology.orchestra.run.smithi083.stdout: "hw_monitoring": { 2024-04-23T14:59:14.292 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hw_monitoring", 2024-04-23T14:59:14.292 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.292 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.293 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.293 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.293 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.293 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.293 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.293 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T14:59:14.293 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.293 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.293 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.293 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.294 INFO:teuthology.orchestra.run.smithi083.stdout: "inventory_list_all": { 2024-04-23T14:59:14.294 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "inventory_list_all", 2024-04-23T14:59:14.294 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.294 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.294 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.294 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.294 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.294 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.295 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.295 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T14:59:14.295 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.295 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.295 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.295 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.295 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.295 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.295 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.295 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.296 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.296 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.296 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.296 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.296 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.296 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.296 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.296 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.296 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.297 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.297 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.297 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.297 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.297 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.297 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.297 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.297 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.297 INFO:teuthology.orchestra.run.smithi083.stdout: "log_refresh_metadata": { 2024-04-23T14:59:14.297 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_refresh_metadata", 2024-04-23T14:59:14.298 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.298 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.298 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.298 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.298 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.298 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.298 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.298 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T14:59:14.298 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.299 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.299 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.299 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.299 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.299 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.299 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.299 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.299 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.299 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.300 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.300 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.300 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.300 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T14:59:14.300 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.300 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.300 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.300 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.300 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.301 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.301 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.301 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.301 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.301 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.301 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.301 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.301 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.301 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.301 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.302 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.302 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.302 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.302 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.302 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.302 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.302 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.302 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.302 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.303 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.303 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.303 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.303 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.303 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.303 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.303 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.303 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.303 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.303 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.304 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.304 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.304 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.304 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.304 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.304 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T14:59:14.304 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T14:59:14.304 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.304 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.305 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.305 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.305 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.305 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.305 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.305 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T14:59:14.305 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.305 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.305 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.305 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.306 INFO:teuthology.orchestra.run.smithi083.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T14:59:14.306 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T14:59:14.306 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.306 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.306 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.306 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "*", 2024-04-23T14:59:14.306 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.306 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.306 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.307 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T14:59:14.307 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.307 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.307 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.307 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.307 INFO:teuthology.orchestra.run.smithi083.stdout: "max_count_per_host": { 2024-04-23T14:59:14.307 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_count_per_host", 2024-04-23T14:59:14.307 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.307 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.307 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.308 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:14.308 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.308 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.308 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.308 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of daemons per service per host", 2024-04-23T14:59:14.308 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.308 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.308 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.308 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.309 INFO:teuthology.orchestra.run.smithi083.stdout: "max_osd_draining_count": { 2024-04-23T14:59:14.309 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_osd_draining_count", 2024-04-23T14:59:14.309 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.309 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.309 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.309 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:14.309 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.309 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.309 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.309 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T14:59:14.310 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.310 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.310 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.310 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.310 INFO:teuthology.orchestra.run.smithi083.stdout: "migration_current": { 2024-04-23T14:59:14.310 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "migration_current", 2024-04-23T14:59:14.310 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.310 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.310 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.311 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.311 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.311 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.311 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.311 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "internal - do not modify", 2024-04-23T14:59:14.311 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.311 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.311 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.311 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.312 INFO:teuthology.orchestra.run.smithi083.stdout: "mode": { 2024-04-23T14:59:14.312 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mode", 2024-04-23T14:59:14.312 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.312 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.312 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.312 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "root", 2024-04-23T14:59:14.312 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.312 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.312 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.312 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm-package", 2024-04-23T14:59:14.313 INFO:teuthology.orchestra.run.smithi083.stdout: "root" 2024-04-23T14:59:14.313 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.313 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T14:59:14.313 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.313 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.313 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.313 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.313 INFO:teuthology.orchestra.run.smithi083.stdout: "oob_default_addr": { 2024-04-23T14:59:14.313 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "oob_default_addr", 2024-04-23T14:59:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "169.254.1.1", 2024-04-23T14:59:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.314 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T14:59:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: "prometheus_alerts_path": { 2024-04-23T14:59:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus_alerts_path", 2024-04-23T14:59:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.315 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T14:59:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T14:59:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.316 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_insecure": { 2024-04-23T14:59:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_insecure", 2024-04-23T14:59:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.317 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T14:59:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_password": { 2024-04-23T14:59:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_password", 2024-04-23T14:59:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.318 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.319 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.319 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.319 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.319 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.319 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.319 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T14:59:14.319 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.319 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.319 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.319 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.320 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_url": { 2024-04-23T14:59:14.320 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_url", 2024-04-23T14:59:14.320 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.320 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.320 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.320 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.320 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.320 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.320 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.321 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T14:59:14.321 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.321 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.321 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.321 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.321 INFO:teuthology.orchestra.run.smithi083.stdout: "registry_username": { 2024-04-23T14:59:14.321 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "registry_username", 2024-04-23T14:59:14.321 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.321 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.321 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.322 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.322 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.322 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.322 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.322 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T14:59:14.322 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.322 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.322 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.322 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.323 INFO:teuthology.orchestra.run.smithi083.stdout: "secure_monitoring_stack": { 2024-04-23T14:59:14.323 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "secure_monitoring_stack", 2024-04-23T14:59:14.323 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.323 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.323 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.323 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.323 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.323 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.323 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.324 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T14:59:14.324 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.324 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.324 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.324 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.324 INFO:teuthology.orchestra.run.smithi083.stdout: "service_discovery_port": { 2024-04-23T14:59:14.324 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "service_discovery_port", 2024-04-23T14:59:14.324 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.324 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.325 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.325 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8765", 2024-04-23T14:59:14.325 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.325 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.325 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.325 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "cephadm service discovery port", 2024-04-23T14:59:14.325 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.325 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.325 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.325 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.326 INFO:teuthology.orchestra.run.smithi083.stdout: "ssh_config_file": { 2024-04-23T14:59:14.326 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssh_config_file", 2024-04-23T14:59:14.326 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.326 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.326 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.326 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.326 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.326 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.326 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.327 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T14:59:14.327 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.327 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.327 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.327 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.327 INFO:teuthology.orchestra.run.smithi083.stdout: "use_agent": { 2024-04-23T14:59:14.327 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_agent", 2024-04-23T14:59:14.327 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.327 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.328 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.328 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.328 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.328 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.328 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.328 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T14:59:14.328 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.328 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.328 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.328 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.329 INFO:teuthology.orchestra.run.smithi083.stdout: "use_repo_digest": { 2024-04-23T14:59:14.329 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "use_repo_digest", 2024-04-23T14:59:14.329 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.329 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.329 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.329 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.329 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.329 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.329 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.330 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T14:59:14.330 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.330 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.330 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.330 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.330 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_failed_host_check": { 2024-04-23T14:59:14.330 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_failed_host_check", 2024-04-23T14:59:14.330 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.330 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.331 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.331 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.331 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.331 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.331 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.331 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T14:59:14.331 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.331 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.331 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.332 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.332 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_daemons": { 2024-04-23T14:59:14.332 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_daemons", 2024-04-23T14:59:14.332 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.332 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.332 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.332 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.332 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.332 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.332 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.333 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T14:59:14.333 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.333 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.333 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.333 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.333 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_on_stray_hosts": { 2024-04-23T14:59:14.333 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_on_stray_hosts", 2024-04-23T14:59:14.333 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.333 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.334 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.334 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.334 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.334 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.334 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.334 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T14:59:14.334 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.334 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.335 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.335 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.335 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.335 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.335 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crash", 2024-04-23T14:59:14.335 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.335 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.335 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.336 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.336 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.336 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.336 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.336 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.336 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.336 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.336 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.336 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.337 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.337 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.337 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.337 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.337 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.337 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.337 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.337 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.337 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.338 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.338 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.338 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.338 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.338 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.338 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.338 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.338 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.338 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.339 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.339 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.339 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.339 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.339 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.339 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.339 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.339 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.339 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.340 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.340 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.340 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.340 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.340 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.340 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.340 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.340 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.340 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.341 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.341 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.341 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.341 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.341 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.341 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.341 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.341 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.341 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.341 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.342 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.342 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.342 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.342 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.342 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.342 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.342 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.342 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.342 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.343 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.343 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.343 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.343 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.343 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.343 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.343 INFO:teuthology.orchestra.run.smithi083.stdout: "retain_interval": { 2024-04-23T14:59:14.343 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retain_interval", 2024-04-23T14:59:14.343 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.344 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.344 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.344 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "31536000", 2024-04-23T14:59:14.344 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.344 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.344 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.344 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T14:59:14.344 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.344 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.345 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.345 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_recent_interval": { 2024-04-23T14:59:14.345 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_recent_interval", 2024-04-23T14:59:14.345 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.345 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.345 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.345 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1209600", 2024-04-23T14:59:14.345 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.345 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.346 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.346 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T14:59:14.346 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.346 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.346 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.346 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.346 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.347 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dashboard", 2024-04-23T14:59:14.347 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.347 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.347 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.347 INFO:teuthology.orchestra.run.smithi083.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T14:59:14.347 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T14:59:14.347 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.347 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.347 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.348 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:14.348 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.348 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.348 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.348 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.348 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.348 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.348 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.348 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T14:59:14.349 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T14:59:14.349 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.349 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.349 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.349 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.349 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.349 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.349 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.349 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.350 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.350 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.350 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.350 INFO:teuthology.orchestra.run.smithi083.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T14:59:14.350 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T14:59:14.350 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.350 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.351 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.351 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.351 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.351 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.351 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.351 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.351 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.351 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.351 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.352 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.352 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_ENABLED": { 2024-04-23T14:59:14.352 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T14:59:14.352 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.352 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.352 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.352 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.352 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.352 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.352 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.358 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T14:59:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T14:59:14.359 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.360 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T14:59:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T14:59:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.361 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T14:59:14.362 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T14:59:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.363 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T14:59:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T14:59:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.364 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.365 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T14:59:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T14:59:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.366 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.367 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T14:59:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T14:59:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.368 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T14:59:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T14:59:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.369 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T14:59:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T14:59:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.371 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T14:59:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T14:59:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.372 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.373 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.373 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.373 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.373 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.373 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.373 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.373 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.373 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.374 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.374 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.374 INFO:teuthology.orchestra.run.smithi083.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T14:59:14.374 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T14:59:14.374 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.374 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.374 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.374 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.374 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.375 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.375 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.375 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.375 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.375 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.375 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.375 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.375 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T14:59:14.375 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T14:59:14.375 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.376 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.376 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.376 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-23T14:59:14.376 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.376 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.376 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.376 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.376 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.377 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.377 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.377 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T14:59:14.377 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T14:59:14.377 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.377 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.377 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.377 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.377 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.378 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.378 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.378 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.378 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.378 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.378 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.378 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_URL": { 2024-04-23T14:59:14.378 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_URL", 2024-04-23T14:59:14.379 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.379 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.379 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.379 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.379 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.379 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.379 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.379 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.379 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.380 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.380 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.380 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.380 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T14:59:14.380 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T14:59:14.380 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.380 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.380 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.380 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-23T14:59:14.380 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.381 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.381 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.381 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.381 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.381 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.381 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.381 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.381 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T14:59:14.381 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T14:59:14.382 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.382 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.382 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.382 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.382 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.382 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.382 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.382 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.382 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.383 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.383 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.383 INFO:teuthology.orchestra.run.smithi083.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T14:59:14.383 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T14:59:14.383 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.383 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.383 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.383 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.384 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.384 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.384 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.384 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.384 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.384 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.384 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.384 INFO:teuthology.orchestra.run.smithi083.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T14:59:14.385 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T14:59:14.385 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.385 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.385 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.385 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.385 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.385 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.385 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.385 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.385 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.386 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.386 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.386 INFO:teuthology.orchestra.run.smithi083.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T14:59:14.386 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T14:59:14.386 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.386 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.386 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.387 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.387 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.387 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.387 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.387 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.387 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.387 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.387 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.387 INFO:teuthology.orchestra.run.smithi083.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T14:59:14.388 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T14:59:14.388 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.388 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.388 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.388 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "[]", 2024-04-23T14:59:14.388 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.388 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.388 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.389 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.389 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.389 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.389 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.389 INFO:teuthology.orchestra.run.smithi083.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T14:59:14.389 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T14:59:14.389 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.389 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.389 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.390 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "{}", 2024-04-23T14:59:14.390 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.390 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.390 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.390 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.390 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.390 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.390 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.391 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.391 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T14:59:14.391 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T14:59:14.391 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.391 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.391 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.391 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.391 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.391 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.392 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.392 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.392 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.392 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.392 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.392 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.392 INFO:teuthology.orchestra.run.smithi083.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T14:59:14.392 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T14:59:14.392 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.393 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.393 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.393 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.393 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.393 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.393 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.393 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.393 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.393 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.394 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.394 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.394 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T14:59:14.394 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T14:59:14.394 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.394 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.394 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.394 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.394 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.395 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.395 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.395 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.395 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.395 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.395 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.395 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.395 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T14:59:14.395 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T14:59:14.395 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.396 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.396 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.396 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.396 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.396 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.396 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.396 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.396 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.396 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.397 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.397 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.397 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T14:59:14.397 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T14:59:14.397 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.397 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.397 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.397 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.397 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.398 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.398 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.398 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.398 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.398 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.398 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.398 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.398 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T14:59:14.398 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T14:59:14.399 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.399 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.399 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.399 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.399 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.399 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.399 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.399 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.399 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.400 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.400 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.400 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.400 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T14:59:14.400 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T14:59:14.400 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.400 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.400 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.400 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.401 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.401 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.401 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.401 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.401 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.401 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.401 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.401 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.401 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T14:59:14.402 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T14:59:14.402 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.402 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.402 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.402 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.402 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.402 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.402 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.402 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.403 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.403 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.403 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.403 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.403 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T14:59:14.403 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T14:59:14.403 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.403 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.403 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.404 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.404 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.404 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.404 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.404 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.404 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.404 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.404 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.404 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.405 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T14:59:14.405 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T14:59:14.405 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.405 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.405 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.406 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.406 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.406 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.406 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.406 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.407 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.407 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.407 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.407 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.407 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T14:59:14.408 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T14:59:14.408 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.408 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.408 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.408 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T14:59:14.409 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.409 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.409 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.409 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.409 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.410 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.410 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.410 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.410 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T14:59:14.411 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T14:59:14.411 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.411 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.411 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.411 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:14.411 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.411 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.412 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.412 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.412 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.412 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.412 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.412 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.412 INFO:teuthology.orchestra.run.smithi083.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T14:59:14.412 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T14:59:14.413 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.413 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.413 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.413 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8", 2024-04-23T14:59:14.413 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.413 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.413 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.413 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.414 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.414 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.414 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.414 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.414 INFO:teuthology.orchestra.run.smithi083.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T14:59:14.414 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T14:59:14.414 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.414 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.415 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.415 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "45", 2024-04-23T14:59:14.415 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.415 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.415 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.415 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.415 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.415 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.415 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.416 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.416 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T14:59:14.416 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T14:59:14.416 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.416 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.416 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.416 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.416 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.416 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.417 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.417 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.417 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.417 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.417 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.417 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.417 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T14:59:14.417 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T14:59:14.417 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.417 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.418 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.418 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "admin", 2024-04-23T14:59:14.418 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.418 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.418 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.418 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.418 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.418 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.418 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.419 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.419 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T14:59:14.419 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T14:59:14.419 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.419 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.419 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.419 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.419 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.419 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.420 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.420 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.420 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.420 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.420 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.420 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.420 INFO:teuthology.orchestra.run.smithi083.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T14:59:14.420 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T14:59:14.420 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.421 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.421 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.421 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.421 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.421 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.421 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.421 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.421 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.421 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.422 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.422 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.422 INFO:teuthology.orchestra.run.smithi083.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T14:59:14.422 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T14:59:14.422 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.422 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.422 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.422 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.422 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.423 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.423 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.423 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.423 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.423 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.423 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.424 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.424 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T14:59:14.424 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T14:59:14.424 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.424 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.425 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.425 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:14.425 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.425 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.425 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.426 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.426 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.426 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.426 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.426 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.427 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T14:59:14.427 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T14:59:14.427 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.427 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.427 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.428 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:14.428 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.428 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.428 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.428 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.429 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.429 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.429 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.429 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.429 INFO:teuthology.orchestra.run.smithi083.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T14:59:14.430 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T14:59:14.430 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.430 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.430 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.430 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-23T14:59:14.430 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.430 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.431 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.431 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.431 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.431 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.431 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.431 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.432 INFO:teuthology.orchestra.run.smithi083.stdout: "cross_origin_url": { 2024-04-23T14:59:14.432 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cross_origin_url", 2024-04-23T14:59:14.432 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.432 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.432 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.432 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.432 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.432 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.432 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.433 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.433 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.433 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.433 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.433 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.433 INFO:teuthology.orchestra.run.smithi083.stdout: "crt_file": { 2024-04-23T14:59:14.433 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "crt_file", 2024-04-23T14:59:14.433 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.433 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.433 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.434 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.434 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.434 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.434 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.434 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.434 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.434 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.434 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.435 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.435 INFO:teuthology.orchestra.run.smithi083.stdout: "debug": { 2024-04-23T14:59:14.435 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "debug", 2024-04-23T14:59:14.435 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.435 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.435 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.435 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.435 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.435 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.435 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.436 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Enable/disable debug options", 2024-04-23T14:59:14.436 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.436 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.436 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.436 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.436 INFO:teuthology.orchestra.run.smithi083.stdout: "jwt_token_ttl": { 2024-04-23T14:59:14.436 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "jwt_token_ttl", 2024-04-23T14:59:14.436 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.437 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.437 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.437 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "28800", 2024-04-23T14:59:14.437 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.437 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.437 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.437 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.437 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.437 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.438 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.438 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.438 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-23T14:59:14.438 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-23T14:59:14.438 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.438 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.438 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.438 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.438 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.439 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.439 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.439 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.439 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.439 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.439 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.439 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.439 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.439 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.440 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.440 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.440 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.440 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.440 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.440 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.440 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.440 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.440 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.440 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.441 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.441 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.441 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.441 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.441 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.441 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.441 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.441 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.441 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.442 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.442 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.442 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.442 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.442 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.442 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.442 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.442 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.442 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.443 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.443 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.443 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.443 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.443 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.443 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.443 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.443 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.443 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.443 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.444 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.444 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.444 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.444 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.444 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.444 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.444 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.444 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.444 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.445 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.445 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.445 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.445 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.445 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.445 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.445 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.445 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.445 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.445 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.446 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.446 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.446 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.446 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.446 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.446 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.446 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.446 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.446 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.447 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.447 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.447 INFO:teuthology.orchestra.run.smithi083.stdout: "motd": { 2024-04-23T14:59:14.447 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "motd", 2024-04-23T14:59:14.447 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.447 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.447 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.447 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.447 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.448 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.448 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.448 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "The message of the day", 2024-04-23T14:59:14.448 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.448 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.448 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.448 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.448 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect_resolve_ip_addr": { 2024-04-23T14:59:14.448 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T14:59:14.449 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.449 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.449 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.449 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.449 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.449 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.449 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.449 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.449 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.450 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.450 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.450 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.450 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-23T14:59:14.450 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-23T14:59:14.450 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.450 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.450 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.450 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-23T14:59:14.451 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.451 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.451 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.451 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.451 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.451 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.451 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.451 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.451 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-23T14:59:14.452 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-23T14:59:14.452 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.452 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.452 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.452 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8080", 2024-04-23T14:59:14.452 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.452 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.452 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.452 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.453 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.453 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.453 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.453 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.453 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-23T14:59:14.453 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-23T14:59:14.453 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.453 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.453 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.453 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.454 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.454 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.454 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.454 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.454 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.454 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.454 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.454 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.455 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl_server_port": { 2024-04-23T14:59:14.455 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl_server_port", 2024-04-23T14:59:14.455 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.455 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.455 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.455 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8443", 2024-04-23T14:59:14.455 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.455 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.455 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.456 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.456 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.456 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.456 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.456 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.456 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-23T14:59:14.456 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-23T14:59:14.456 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.456 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.457 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.457 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "redirect", 2024-04-23T14:59:14.457 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.457 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.457 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.457 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.457 INFO:teuthology.orchestra.run.smithi083.stdout: "redirect" 2024-04-23T14:59:14.457 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.457 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.458 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.458 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.458 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.458 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.458 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-23T14:59:14.458 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-23T14:59:14.458 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.458 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.459 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.459 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-23T14:59:14.459 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-23T14:59:14.459 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-23T14:59:14.459 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.459 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.459 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.459 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.459 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.460 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.460 INFO:teuthology.orchestra.run.smithi083.stdout: "url_prefix": { 2024-04-23T14:59:14.460 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url_prefix", 2024-04-23T14:59:14.460 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.460 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.460 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.460 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.460 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.460 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.460 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.461 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.461 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.461 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.461 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.461 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.461 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.461 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.461 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.461 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-04-23T14:59:14.462 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.462 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.462 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.462 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_monitoring": { 2024-04-23T14:59:14.462 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_monitoring", 2024-04-23T14:59:14.462 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.462 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.462 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.462 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.463 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.463 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.463 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.463 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "monitor device health metrics", 2024-04-23T14:59:14.463 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.463 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.463 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.463 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.463 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.464 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.464 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.464 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.464 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.464 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.464 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.464 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.464 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.464 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.465 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.465 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.465 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.465 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.465 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.465 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.465 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.465 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.465 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.466 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.466 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.466 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.466 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.466 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.466 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.466 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.466 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.466 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.466 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.467 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.467 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.467 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.467 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.467 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.467 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.467 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.467 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.467 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.468 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.468 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.468 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.468 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.468 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.468 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.468 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.468 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.468 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.468 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.469 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.469 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.469 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.469 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.469 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.469 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.469 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.469 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.469 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.470 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.470 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.470 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.470 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.470 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.470 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.470 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.470 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.470 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.471 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.471 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.471 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.471 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.471 INFO:teuthology.orchestra.run.smithi083.stdout: "mark_out_threshold": { 2024-04-23T14:59:14.471 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mark_out_threshold", 2024-04-23T14:59:14.471 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.471 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.471 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.471 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "2419200", 2024-04-23T14:59:14.472 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.472 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.472 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.472 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T14:59:14.472 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.472 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.472 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.472 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.472 INFO:teuthology.orchestra.run.smithi083.stdout: "pool_name": { 2024-04-23T14:59:14.473 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pool_name", 2024-04-23T14:59:14.473 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.473 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.473 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.473 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "device_health_metrics", 2024-04-23T14:59:14.473 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.473 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.473 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.473 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T14:59:14.474 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.474 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.474 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.474 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.474 INFO:teuthology.orchestra.run.smithi083.stdout: "retention_period": { 2024-04-23T14:59:14.474 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "retention_period", 2024-04-23T14:59:14.474 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.474 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.474 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.475 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15552000", 2024-04-23T14:59:14.475 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.475 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.475 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.475 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long to retain device health metrics", 2024-04-23T14:59:14.475 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.475 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.476 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.476 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.476 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_frequency": { 2024-04-23T14:59:14.476 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_frequency", 2024-04-23T14:59:14.476 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.476 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.476 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.476 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-23T14:59:14.476 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.477 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.477 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.477 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T14:59:14.477 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.477 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.477 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.477 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.477 INFO:teuthology.orchestra.run.smithi083.stdout: "self_heal": { 2024-04-23T14:59:14.477 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "self_heal", 2024-04-23T14:59:14.477 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.478 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.478 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.478 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.478 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.478 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.478 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.478 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T14:59:14.478 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.478 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.479 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.479 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.479 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:14.479 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:14.479 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.479 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.479 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.479 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:14.479 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.480 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.480 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.480 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T14:59:14.480 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.480 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.480 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.480 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.480 INFO:teuthology.orchestra.run.smithi083.stdout: "warn_threshold": { 2024-04-23T14:59:14.480 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "warn_threshold", 2024-04-23T14:59:14.481 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.481 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.481 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.481 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7257600", 2024-04-23T14:59:14.481 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.481 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.481 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.481 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T14:59:14.481 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.481 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.482 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.482 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.482 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.482 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.482 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.482 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "diskprediction_local", 2024-04-23T14:59:14.482 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.482 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.482 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.483 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.483 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.483 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.483 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.483 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.483 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.483 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.483 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.483 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.484 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.484 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.484 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.484 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.484 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.484 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.484 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.484 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.484 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.485 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.485 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.485 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.485 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.485 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.485 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.485 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.485 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.485 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.486 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.486 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.486 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.486 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.486 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.486 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.486 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.486 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.486 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.487 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.487 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.487 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.487 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.487 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.487 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.487 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.487 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.488 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.488 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.488 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.488 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.488 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.488 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.488 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.488 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.488 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.488 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.489 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.489 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.489 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.489 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.489 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.489 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.489 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.489 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.490 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.490 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.490 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.490 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.490 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.490 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.490 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.490 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.490 INFO:teuthology.orchestra.run.smithi083.stdout: "predict_interval": { 2024-04-23T14:59:14.490 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predict_interval", 2024-04-23T14:59:14.491 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.491 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.491 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.491 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "86400", 2024-04-23T14:59:14.491 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.491 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.491 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.491 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.491 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.492 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.492 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.492 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.492 INFO:teuthology.orchestra.run.smithi083.stdout: "predictor_model": { 2024-04-23T14:59:14.492 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "predictor_model", 2024-04-23T14:59:14.492 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.492 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.492 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.492 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "prophetstor", 2024-04-23T14:59:14.492 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.493 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.493 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.493 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.493 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.493 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.493 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.493 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.493 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:14.493 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:14.494 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.494 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.494 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.494 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "600", 2024-04-23T14:59:14.494 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.494 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.494 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.494 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.494 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.494 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.495 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.495 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.495 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.495 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.495 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "influx", 2024-04-23T14:59:14.495 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": false, 2024-04-23T14:59:14.495 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "influxdb python module not found", 2024-04-23T14:59:14.495 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.496 INFO:teuthology.orchestra.run.smithi083.stdout: "batch_size": { 2024-04-23T14:59:14.496 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "batch_size", 2024-04-23T14:59:14.496 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.496 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.496 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.496 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5000", 2024-04-23T14:59:14.496 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.496 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.496 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.496 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T14:59:14.497 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.497 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.497 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.497 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.497 INFO:teuthology.orchestra.run.smithi083.stdout: "database": { 2024-04-23T14:59:14.497 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "database", 2024-04-23T14:59:14.497 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.497 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.497 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.498 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "ceph", 2024-04-23T14:59:14.498 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.498 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.498 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.498 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T14:59:14.498 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.498 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.498 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.498 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.498 INFO:teuthology.orchestra.run.smithi083.stdout: "hostname": { 2024-04-23T14:59:14.499 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "hostname", 2024-04-23T14:59:14.499 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.499 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.499 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.499 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.499 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.499 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.499 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.499 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server hostname", 2024-04-23T14:59:14.500 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.500 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.500 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.500 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:14.500 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:14.500 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.500 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.500 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.501 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "30", 2024-04-23T14:59:14.501 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "5", 2024-04-23T14:59:14.501 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.501 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.501 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T14:59:14.501 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.501 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.501 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.502 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.502 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.502 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.502 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.502 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.502 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.502 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.502 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.502 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.503 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.503 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.503 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.503 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.503 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.503 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.503 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.503 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.504 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.504 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.504 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.504 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.504 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.504 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.504 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.504 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.504 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.505 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.505 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.505 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.505 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.505 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.505 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.505 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.505 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.505 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.506 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.506 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.506 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.506 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.506 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.506 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.506 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.506 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.506 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.507 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.507 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.507 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.507 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.507 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.507 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.507 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.507 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.507 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.507 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.508 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.508 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.508 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.508 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.508 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.508 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.508 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.508 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.509 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.509 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.509 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.509 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.509 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.509 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.509 INFO:teuthology.orchestra.run.smithi083.stdout: "password": { 2024-04-23T14:59:14.509 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "password", 2024-04-23T14:59:14.510 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.510 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.510 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.510 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.510 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.510 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.510 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.510 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "password of InfluxDB server user", 2024-04-23T14:59:14.510 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.511 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.511 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.511 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.511 INFO:teuthology.orchestra.run.smithi083.stdout: "port": { 2024-04-23T14:59:14.511 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "port", 2024-04-23T14:59:14.511 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.511 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.511 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.511 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "8086", 2024-04-23T14:59:14.512 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.512 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.512 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.512 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "InfluxDB server port", 2024-04-23T14:59:14.512 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.512 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.512 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.512 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.512 INFO:teuthology.orchestra.run.smithi083.stdout: "ssl": { 2024-04-23T14:59:14.513 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ssl", 2024-04-23T14:59:14.513 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.513 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.513 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.513 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "false", 2024-04-23T14:59:14.513 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.513 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.513 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.513 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T14:59:14.513 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.514 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.514 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.514 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.514 INFO:teuthology.orchestra.run.smithi083.stdout: "threads": { 2024-04-23T14:59:14.514 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threads", 2024-04-23T14:59:14.514 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.514 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.514 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.514 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-23T14:59:14.515 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-23T14:59:14.515 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "32", 2024-04-23T14:59:14.515 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.515 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T14:59:14.515 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.515 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.515 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.515 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.516 INFO:teuthology.orchestra.run.smithi083.stdout: "username": { 2024-04-23T14:59:14.516 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "username", 2024-04-23T14:59:14.516 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.516 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.516 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.516 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.516 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.516 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.516 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.516 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "username of InfluxDB server user", 2024-04-23T14:59:14.517 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.517 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.517 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.517 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.517 INFO:teuthology.orchestra.run.smithi083.stdout: "verify_ssl": { 2024-04-23T14:59:14.517 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "verify_ssl", 2024-04-23T14:59:14.517 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.517 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.517 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.517 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "true", 2024-04-23T14:59:14.518 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.518 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.518 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.518 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T14:59:14.518 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.518 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.518 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.519 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.519 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.519 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "insights", 2024-04-23T14:59:14.519 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.519 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.519 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.519 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.519 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.519 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.520 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.520 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.520 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.520 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.520 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.520 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.520 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.520 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.520 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.520 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.521 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.521 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.521 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.521 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.521 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.521 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.521 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.521 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.521 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.522 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.522 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.522 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.522 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.522 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.522 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.522 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.522 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.522 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.522 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.523 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.523 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.523 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.523 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.523 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.523 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.523 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.523 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.523 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.524 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.524 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.524 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.524 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.524 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.524 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.524 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.524 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.524 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.525 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.525 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.525 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.525 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.525 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.525 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.525 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.525 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.525 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.526 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.526 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.526 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.526 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.526 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.526 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.526 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.526 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.527 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.527 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.527 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.527 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.527 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.527 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.527 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "iostat", 2024-04-23T14:59:14.527 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.527 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.528 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.528 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.528 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.528 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.528 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.528 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.528 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.528 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.529 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.529 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.529 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.529 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.529 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.529 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.529 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.529 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.529 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.530 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.530 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.530 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.530 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.530 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.530 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.530 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.530 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.530 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.531 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.531 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.531 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.531 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.531 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.531 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.531 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.531 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.532 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.532 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.532 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.532 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.532 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.532 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.532 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.532 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.532 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.533 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.533 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.533 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.533 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.533 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.533 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.533 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.533 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.533 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.534 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.534 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.534 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.534 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.534 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.534 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.534 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.534 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.534 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.535 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.535 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.535 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.535 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.535 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.535 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.535 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.535 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.536 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.536 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.536 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.536 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.536 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "k8sevents", 2024-04-23T14:59:14.536 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.536 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.536 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.537 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_event_retention_days": { 2024-04-23T14:59:14.537 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "ceph_event_retention_days", 2024-04-23T14:59:14.537 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.537 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.537 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.537 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "7", 2024-04-23T14:59:14.537 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.537 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.537 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.538 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T14:59:14.538 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.538 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.538 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.538 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.538 INFO:teuthology.orchestra.run.smithi083.stdout: "config_check_secs": { 2024-04-23T14:59:14.538 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "config_check_secs", 2024-04-23T14:59:14.538 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.539 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.539 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.539 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:14.539 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "10", 2024-04-23T14:59:14.539 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.539 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.539 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T14:59:14.539 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.540 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.540 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.540 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.540 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.540 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.540 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.540 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.540 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.540 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.541 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.541 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.541 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.541 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.541 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.541 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.541 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.541 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.541 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.542 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.542 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.542 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.542 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.542 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.542 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.542 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.542 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.542 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.543 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.543 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.543 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.543 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.543 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.543 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.543 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.543 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.544 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.544 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.544 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.544 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.544 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.544 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.544 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.544 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.544 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.545 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.545 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.545 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.545 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.545 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.545 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.545 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.545 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.545 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.545 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.546 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.546 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.546 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.546 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.546 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.546 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.546 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.546 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.547 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.547 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.547 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.547 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.547 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.547 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.547 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.547 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.547 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.547 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.548 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.548 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.548 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.548 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "localpool", 2024-04-23T14:59:14.548 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.548 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.548 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.549 INFO:teuthology.orchestra.run.smithi083.stdout: "failure_domain": { 2024-04-23T14:59:14.549 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "failure_domain", 2024-04-23T14:59:14.549 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.549 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.549 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.549 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "host", 2024-04-23T14:59:14.549 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.549 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.549 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.549 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "failure domain for any created local pool", 2024-04-23T14:59:14.550 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T14:59:14.550 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.550 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.550 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.550 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.550 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.550 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.550 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.551 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.551 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.551 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.551 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.551 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.551 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.551 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.551 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.551 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.552 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.552 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.552 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.552 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.552 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.552 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.552 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.552 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.553 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.553 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.553 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.553 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.553 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.553 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.553 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.553 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.553 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.553 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.554 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.554 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.554 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.554 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.554 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.554 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.554 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.554 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.555 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.555 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.555 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.555 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.555 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.555 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.555 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.555 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.555 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.555 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.556 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.556 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.556 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.556 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.556 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.556 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.556 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.556 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.556 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.557 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.557 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.557 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.557 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.557 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.557 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.557 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.557 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.557 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.557 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.558 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.558 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.558 INFO:teuthology.orchestra.run.smithi083.stdout: "min_size": { 2024-04-23T14:59:14.558 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "min_size", 2024-04-23T14:59:14.558 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.558 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.558 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.558 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.559 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.559 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.559 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.559 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default min_size for any created local pool", 2024-04-23T14:59:14.559 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T14:59:14.559 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.559 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.559 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.559 INFO:teuthology.orchestra.run.smithi083.stdout: "num_rep": { 2024-04-23T14:59:14.560 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "num_rep", 2024-04-23T14:59:14.560 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.560 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.560 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.560 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3", 2024-04-23T14:59:14.560 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.560 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.560 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.560 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default replica count for any created local pool", 2024-04-23T14:59:14.560 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.561 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.561 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.561 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.561 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_num": { 2024-04-23T14:59:14.561 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_num", 2024-04-23T14:59:14.561 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.561 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.561 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.562 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "128", 2024-04-23T14:59:14.562 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.562 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.562 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.562 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T14:59:14.562 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.562 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.562 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.562 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.562 INFO:teuthology.orchestra.run.smithi083.stdout: "prefix": { 2024-04-23T14:59:14.563 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prefix", 2024-04-23T14:59:14.563 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.563 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.563 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.563 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.563 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.563 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.563 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.563 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "name prefix for any created local pool", 2024-04-23T14:59:14.564 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.564 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.564 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.564 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.564 INFO:teuthology.orchestra.run.smithi083.stdout: "subtree": { 2024-04-23T14:59:14.564 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "subtree", 2024-04-23T14:59:14.564 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.564 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.565 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.565 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "rack", 2024-04-23T14:59:14.565 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.565 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.565 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.565 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T14:59:14.565 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T14:59:14.565 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.565 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.566 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.566 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.566 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.566 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.566 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mds_autoscaler", 2024-04-23T14:59:14.566 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.566 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.566 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.566 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.567 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.567 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.567 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.567 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.567 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.567 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.567 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.567 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.567 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.567 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.568 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.568 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.568 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.568 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.568 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.568 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.568 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.568 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.568 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.569 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.569 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.569 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.569 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.569 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.569 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.569 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.569 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.569 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.569 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.570 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.570 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.570 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.570 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.570 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.570 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.570 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.570 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.570 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.571 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.571 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.571 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.571 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.571 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.571 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.571 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.571 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.571 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.572 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.572 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.572 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.572 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.572 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.572 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.572 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.572 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.572 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.573 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.573 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.573 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.573 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.573 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.573 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.573 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.573 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.573 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.574 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.574 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.574 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.574 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.574 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.574 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.574 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.574 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirroring", 2024-04-23T14:59:14.574 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.575 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.575 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.575 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.575 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.575 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.575 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.575 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.575 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.575 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.576 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.576 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.576 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.576 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.576 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.576 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.576 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.576 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.576 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.576 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.577 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.577 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.577 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.577 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.577 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.577 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.577 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.577 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.577 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.577 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.578 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.578 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.578 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.578 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.578 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.578 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.578 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.578 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.578 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.579 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.579 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.579 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.579 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.579 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.579 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.579 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.579 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.579 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.580 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.580 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.580 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.580 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.580 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.580 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.580 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.580 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.580 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.580 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.581 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.581 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.581 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.581 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.581 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.581 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.581 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.581 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.581 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.582 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.582 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.582 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.582 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.582 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.582 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.582 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.582 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.582 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "nfs", 2024-04-23T14:59:14.583 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.583 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.583 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.583 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.583 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.583 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.583 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.583 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.584 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.584 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.584 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.584 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.584 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.584 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.584 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.584 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.584 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.585 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.585 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.585 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.585 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.585 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.585 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.585 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.585 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.585 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.586 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.586 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.586 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.586 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.586 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.586 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.586 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.586 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.586 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.587 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.587 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.587 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.587 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.587 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.587 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.587 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.587 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.587 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.588 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.588 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.588 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.588 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.588 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.588 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.588 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.588 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.588 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.589 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.589 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.589 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.589 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.589 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.589 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.589 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.589 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.589 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.590 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.590 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.590 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.590 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.590 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.590 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.590 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.590 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.591 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.591 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.591 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.591 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.591 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.591 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.591 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-23T14:59:14.591 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.591 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.592 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.592 INFO:teuthology.orchestra.run.smithi083.stdout: "fail_fs": { 2024-04-23T14:59:14.592 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "fail_fs", 2024-04-23T14:59:14.592 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.592 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.592 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.592 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.592 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.592 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.593 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.593 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T14:59:14.593 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.593 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.593 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.593 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.593 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.593 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.594 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.594 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.594 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.594 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.594 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.594 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.594 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.594 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.595 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.595 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.595 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.595 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.595 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.595 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.595 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.595 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.596 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.596 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.596 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.596 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.596 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.596 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.596 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.596 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.597 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.597 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.597 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.597 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.597 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.597 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.597 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.597 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.597 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.598 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.598 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.598 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.598 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.598 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.598 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.598 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.598 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.598 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.599 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.599 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.599 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.599 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.599 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.599 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.599 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.599 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.599 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.599 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.600 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.600 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.600 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.600 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.600 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.600 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.600 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.600 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.600 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.601 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.601 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.601 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.601 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.601 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.601 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.601 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.601 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator": { 2024-04-23T14:59:14.601 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "orchestrator", 2024-04-23T14:59:14.601 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.602 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.602 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.602 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.602 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.602 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.602 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.602 INFO:teuthology.orchestra.run.smithi083.stdout: "cephadm", 2024-04-23T14:59:14.602 INFO:teuthology.orchestra.run.smithi083.stdout: "rook", 2024-04-23T14:59:14.602 INFO:teuthology.orchestra.run.smithi083.stdout: "test_orchestrator" 2024-04-23T14:59:14.603 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.603 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Orchestrator backend", 2024-04-23T14:59:14.603 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.603 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.603 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.603 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.603 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.603 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.603 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_perf_query", 2024-04-23T14:59:14.604 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.604 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.604 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.604 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.604 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.604 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.604 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.605 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.605 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.605 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.605 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.605 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.605 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.605 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.605 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.605 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.606 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.606 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.606 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.606 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.606 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.606 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.606 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.606 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.607 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.607 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.607 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.607 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.607 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.607 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.607 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.607 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.607 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.607 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.608 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.608 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.608 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.608 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.608 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.608 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.608 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.608 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.608 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.609 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.609 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.609 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.609 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.609 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.609 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.609 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.609 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.609 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.610 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.610 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.610 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.610 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.610 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.610 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.610 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.610 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.610 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.611 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.611 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.611 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.611 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.611 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.611 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.611 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.611 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.612 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.612 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.612 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.612 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.612 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.612 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.612 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.612 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "osd_support", 2024-04-23T14:59:14.612 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.613 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.613 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.613 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.613 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.613 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.613 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.613 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.613 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.614 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.614 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.614 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.614 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.614 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.614 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.614 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.614 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.614 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.615 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.615 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.615 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.615 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.615 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.615 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.615 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.615 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.615 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.616 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.616 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.616 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.616 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.616 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.616 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.616 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.616 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.617 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.617 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.617 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.617 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.617 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.617 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.617 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.617 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.617 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.618 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.618 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.618 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.618 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.618 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.618 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.618 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.618 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.618 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.619 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.619 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.619 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.619 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.619 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.619 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.619 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.619 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.619 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.620 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.620 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.620 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.620 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.620 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.620 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.620 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.620 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.621 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.621 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.621 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.621 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.621 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.621 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "pg_autoscaler", 2024-04-23T14:59:14.622 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.622 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.622 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.622 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.623 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.623 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.623 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.623 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.624 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.624 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.624 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.624 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.625 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.625 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.625 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.625 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.625 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.626 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.626 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.626 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.626 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.627 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.627 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.627 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.627 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.627 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.628 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.629 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.630 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.630 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.630 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.630 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.631 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.631 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.631 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.631 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.631 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.632 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.632 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.632 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.632 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.632 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.632 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.633 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.633 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.633 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.633 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.633 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.634 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.634 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.634 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.634 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.634 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.634 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.635 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.635 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.635 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.635 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.635 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.636 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.636 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.636 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.636 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.637 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.637 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.637 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.637 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:14.637 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:14.638 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.638 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.638 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.638 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:14.638 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.639 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.639 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.639 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.639 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.639 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.640 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.640 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.640 INFO:teuthology.orchestra.run.smithi083.stdout: "threshold": { 2024-04-23T14:59:14.640 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "threshold", 2024-04-23T14:59:14.640 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:14.640 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "3.0", 2024-04-23T14:59:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1.0", 2024-04-23T14:59:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "scaling threshold", 2024-04-23T14:59:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T14:59:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.641 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "progress", 2024-04-23T14:59:14.642 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.643 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.643 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.643 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_pg_recovery_event": { 2024-04-23T14:59:14.643 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_pg_recovery_event", 2024-04-23T14:59:14.643 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.643 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.643 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.643 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.644 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.644 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.644 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.644 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T14:59:14.644 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.644 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.644 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.644 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.645 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-23T14:59:14.645 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-23T14:59:14.645 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.645 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.645 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.645 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.645 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.645 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.645 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.646 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.646 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.646 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.646 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.646 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.646 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.646 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.646 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.647 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.647 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.647 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.647 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.647 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.647 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.647 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.647 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.648 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.648 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.648 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.648 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.648 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.648 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.648 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.648 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.648 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.649 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.649 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.649 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.649 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.649 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.649 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.649 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.649 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.650 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.650 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.650 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.650 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.650 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.650 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.650 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.650 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.651 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.651 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.651 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.651 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.651 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.651 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.651 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.651 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.651 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.652 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.652 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.652 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.652 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.652 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.652 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.652 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.652 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.653 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.653 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.653 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.653 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.653 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.653 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.653 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.653 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.653 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.654 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.654 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.654 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.654 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.654 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.654 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.654 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.655 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.655 INFO:teuthology.orchestra.run.smithi083.stdout: "max_completed_events": { 2024-04-23T14:59:14.655 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_completed_events", 2024-04-23T14:59:14.655 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.655 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.655 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.655 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "50", 2024-04-23T14:59:14.655 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.656 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.656 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.656 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "number of past completed events to remember", 2024-04-23T14:59:14.656 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.656 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.656 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.656 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.656 INFO:teuthology.orchestra.run.smithi083.stdout: "sleep_interval": { 2024-04-23T14:59:14.656 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "sleep_interval", 2024-04-23T14:59:14.656 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.657 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.657 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.657 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "5", 2024-04-23T14:59:14.657 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.657 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.657 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.657 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "how long the module is going to sleep", 2024-04-23T14:59:14.657 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.658 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.658 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.658 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.658 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.658 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.658 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "prometheus", 2024-04-23T14:59:14.658 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.659 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.659 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.659 INFO:teuthology.orchestra.run.smithi083.stdout: "cache": { 2024-04-23T14:59:14.659 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "cache", 2024-04-23T14:59:14.659 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.659 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.659 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.659 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.659 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.660 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.660 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.660 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.660 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.660 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.660 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.660 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.660 INFO:teuthology.orchestra.run.smithi083.stdout: "exclude_perf_counters": { 2024-04-23T14:59:14.660 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "exclude_perf_counters", 2024-04-23T14:59:14.660 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.661 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.661 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.661 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.661 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.661 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.661 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.661 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T14:59:14.661 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T14:59:14.661 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.662 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.662 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.662 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.662 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.662 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.662 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.662 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.662 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.662 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.663 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.663 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.663 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.663 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.663 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.663 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.663 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.663 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.663 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.663 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.664 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.664 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.664 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.664 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.664 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.664 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.664 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.664 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.665 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.665 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.665 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.665 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.665 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.665 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.665 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.665 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.665 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.665 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.666 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.666 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.666 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.666 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.666 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.666 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.666 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.666 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.666 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.667 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.667 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.667 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.667 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.667 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.667 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.667 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.667 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.667 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.668 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.668 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.668 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.668 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.668 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.668 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.668 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.668 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.668 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.668 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.669 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.669 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.669 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.669 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.669 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.669 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.669 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.669 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools": { 2024-04-23T14:59:14.669 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools", 2024-04-23T14:59:14.669 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.670 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.670 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.670 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.670 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.670 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.670 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.670 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.670 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.670 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.671 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.671 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.671 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T14:59:14.671 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T14:59:14.671 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.671 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.671 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.671 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "300", 2024-04-23T14:59:14.671 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.672 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.672 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.672 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.672 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.672 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.672 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.672 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.672 INFO:teuthology.orchestra.run.smithi083.stdout: "scrape_interval": { 2024-04-23T14:59:14.672 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "scrape_interval", 2024-04-23T14:59:14.672 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:14.673 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.673 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.673 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15.0", 2024-04-23T14:59:14.673 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.673 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.673 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.673 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.673 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.674 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.674 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.674 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.674 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-23T14:59:14.674 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-23T14:59:14.674 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.674 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.674 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.674 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "::", 2024-04-23T14:59:14.675 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.675 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.675 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.675 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T14:59:14.675 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.675 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.675 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.675 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.675 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-23T14:59:14.675 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-23T14:59:14.676 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.676 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.676 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.676 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "9283", 2024-04-23T14:59:14.676 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.676 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.676 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.676 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T14:59:14.676 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.676 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.677 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.677 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.677 INFO:teuthology.orchestra.run.smithi083.stdout: "stale_cache_strategy": { 2024-04-23T14:59:14.677 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stale_cache_strategy", 2024-04-23T14:59:14.677 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.677 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.677 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.677 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "log", 2024-04-23T14:59:14.677 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.678 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.678 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.678 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.678 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.678 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.678 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.678 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.678 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_behaviour": { 2024-04-23T14:59:14.678 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_behaviour", 2024-04-23T14:59:14.679 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.679 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.679 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.679 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "default", 2024-04-23T14:59:14.679 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.679 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.679 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.679 INFO:teuthology.orchestra.run.smithi083.stdout: "default", 2024-04-23T14:59:14.679 INFO:teuthology.orchestra.run.smithi083.stdout: "error" 2024-04-23T14:59:14.680 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.680 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.680 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.680 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.680 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.680 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.680 INFO:teuthology.orchestra.run.smithi083.stdout: "standby_error_status_code": { 2024-04-23T14:59:14.680 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "standby_error_status_code", 2024-04-23T14:59:14.680 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.681 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.681 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.681 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "500", 2024-04-23T14:59:14.681 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "400", 2024-04-23T14:59:14.681 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "599", 2024-04-23T14:59:14.681 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.681 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.681 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.681 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.681 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.682 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.682 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.682 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.682 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-04-23T14:59:14.682 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.682 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.682 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.682 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.683 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.683 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.683 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.683 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.683 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.683 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.683 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.683 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.683 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.684 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.684 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.684 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.684 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.684 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.684 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.684 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.684 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.684 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.685 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.685 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.685 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.685 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.685 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.685 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.685 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.685 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.685 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.686 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.686 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.686 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.686 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.686 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.686 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.686 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.686 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.686 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.686 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.687 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.687 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.687 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.687 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.687 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.687 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.687 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.687 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.687 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.688 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.688 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.688 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.688 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.688 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.688 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.688 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.688 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.689 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.689 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.689 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.689 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.689 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.689 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.689 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.689 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.689 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.690 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.690 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.690 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.690 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.690 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.690 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.690 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_snap_create": { 2024-04-23T14:59:14.690 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_snap_create", 2024-04-23T14:59:14.691 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.691 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.691 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.691 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10", 2024-04-23T14:59:14.691 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.691 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.691 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.691 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.691 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.692 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.692 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.692 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.692 INFO:teuthology.orchestra.run.smithi083.stdout: "mirror_snapshot_schedule": { 2024-04-23T14:59:14.692 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T14:59:14.692 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.692 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.692 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.692 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.693 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.693 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.693 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.693 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.693 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.693 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.693 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.693 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.694 INFO:teuthology.orchestra.run.smithi083.stdout: "trash_purge_schedule": { 2024-04-23T14:59:14.694 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "trash_purge_schedule", 2024-04-23T14:59:14.694 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.694 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.694 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.694 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.694 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.694 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.694 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.695 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.695 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.695 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.695 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.695 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.695 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.695 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.695 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.695 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "restful", 2024-04-23T14:59:14.695 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.696 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.696 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.696 INFO:teuthology.orchestra.run.smithi083.stdout: "enable_auth": { 2024-04-23T14:59:14.696 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enable_auth", 2024-04-23T14:59:14.696 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.696 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.696 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.696 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.696 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.697 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.697 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.697 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.697 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.697 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.697 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.697 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.697 INFO:teuthology.orchestra.run.smithi083.stdout: "key_file": { 2024-04-23T14:59:14.697 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "key_file", 2024-04-23T14:59:14.697 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.698 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.698 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.698 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.698 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.698 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.698 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.698 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.698 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.698 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.699 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.699 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.699 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.699 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.699 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.699 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.699 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.699 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.699 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.699 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.700 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.700 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.700 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.700 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.700 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.700 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.700 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.700 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.700 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.700 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.701 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.701 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.701 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.701 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.701 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.701 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.701 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.701 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.701 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.702 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.702 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.702 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.702 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.702 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.702 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.702 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.702 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.702 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.702 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.703 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.703 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.703 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.703 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.703 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.703 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.703 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.703 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.703 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.704 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.704 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.704 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.704 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.704 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.704 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.704 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.704 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.704 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.704 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.705 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.705 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.705 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.705 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.705 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.705 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.705 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.705 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.705 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.706 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.706 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.706 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.706 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.706 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.706 INFO:teuthology.orchestra.run.smithi083.stdout: "server_addr": { 2024-04-23T14:59:14.706 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_addr", 2024-04-23T14:59:14.706 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.706 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.707 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.707 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.707 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.707 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.707 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.707 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.707 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.707 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.707 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.707 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.708 INFO:teuthology.orchestra.run.smithi083.stdout: "server_port": { 2024-04-23T14:59:14.708 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "server_port", 2024-04-23T14:59:14.708 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.708 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.708 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.708 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.708 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.708 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.708 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.708 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rgw", 2024-04-23T14:59:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.709 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.710 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.711 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.712 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.713 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.714 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.715 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.716 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.717 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rook", 2024-04-23T14:59:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.718 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.719 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.720 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.721 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.722 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.723 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.724 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.725 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "storage_class": { 2024-04-23T14:59:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "storage_class", 2024-04-23T14:59:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "local", 2024-04-23T14:59:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.726 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T14:59:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.727 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "selftest", 2024-04-23T14:59:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.728 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.729 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.730 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.731 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.732 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.733 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.734 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.735 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "roption1": { 2024-04-23T14:59:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption1", 2024-04-23T14:59:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.736 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "roption2": { 2024-04-23T14:59:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "roption2", 2024-04-23T14:59:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.737 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "xyz", 2024-04-23T14:59:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.738 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption1": { 2024-04-23T14:59:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption1", 2024-04-23T14:59:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.739 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption2": { 2024-04-23T14:59:14.740 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption2", 2024-04-23T14:59:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.741 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption3": { 2024-04-23T14:59:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption3", 2024-04-23T14:59:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "float", 2024-04-23T14:59:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.742 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.743 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption4": { 2024-04-23T14:59:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption4", 2024-04-23T14:59:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.744 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption5": { 2024-04-23T14:59:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption5", 2024-04-23T14:59:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.745 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.746 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption6": { 2024-04-23T14:59:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption6", 2024-04-23T14:59:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.747 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "rwoption7": { 2024-04-23T14:59:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rwoption7", 2024-04-23T14:59:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.748 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "1", 2024-04-23T14:59:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "42", 2024-04-23T14:59:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.749 INFO:teuthology.orchestra.run.smithi083.stdout: "testkey": { 2024-04-23T14:59:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testkey", 2024-04-23T14:59:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.750 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "testlkey": { 2024-04-23T14:59:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testlkey", 2024-04-23T14:59:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.751 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.752 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "testnewline": { 2024-04-23T14:59:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "testnewline", 2024-04-23T14:59:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.753 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.754 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-23T14:59:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "allow_m_granularity": { 2024-04-23T14:59:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "allow_m_granularity", 2024-04-23T14:59:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.755 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T14:59:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.756 INFO:teuthology.orchestra.run.smithi083.stdout: "dump_on_update": { 2024-04-23T14:59:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "dump_on_update", 2024-04-23T14:59:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.757 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "dump database to debug log on update", 2024-04-23T14:59:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.758 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.759 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.760 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.761 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.762 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.763 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.764 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.765 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "stats", 2024-04-23T14:59:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.766 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.767 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.768 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.769 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.770 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.771 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.772 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.773 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.774 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.774 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.774 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.774 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.774 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.774 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.774 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.774 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.774 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.775 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.775 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.775 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.775 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.775 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.775 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.775 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "status", 2024-04-23T14:59:14.775 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.776 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.776 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.776 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.776 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.776 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.776 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.776 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.776 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.776 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.777 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.777 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.777 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.777 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.777 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.777 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.777 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.777 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.777 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.778 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.778 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.778 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.778 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.778 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.778 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.779 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.779 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.779 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.779 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.779 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.779 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.779 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.779 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.779 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.780 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.780 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.780 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.780 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.780 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.780 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.780 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.780 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.780 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.781 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.781 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.781 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.781 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.781 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.781 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.781 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.781 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.781 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.781 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.782 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.782 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.782 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.782 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.782 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.782 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.782 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.782 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.783 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.783 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.783 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.783 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.783 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.783 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.783 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.783 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.783 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.783 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.784 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.784 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.784 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.784 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telegraf", 2024-04-23T14:59:14.784 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.784 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.784 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.784 INFO:teuthology.orchestra.run.smithi083.stdout: "address": { 2024-04-23T14:59:14.785 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "address", 2024-04-23T14:59:14.785 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.785 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.785 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.785 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T14:59:14.785 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.785 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.785 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.785 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.785 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.786 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.786 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.786 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.786 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:14.786 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:14.786 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.786 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.786 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.787 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "15", 2024-04-23T14:59:14.787 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.787 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.787 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.787 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.787 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.787 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.787 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.787 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.788 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.788 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.788 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.788 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.788 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.788 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.788 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.788 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.789 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.789 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.789 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.789 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.789 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.789 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.789 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.789 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.789 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.790 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.790 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.790 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.790 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.790 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.790 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.790 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.790 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.791 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.791 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.791 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.791 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.791 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.791 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.791 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.791 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.791 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.792 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.792 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.792 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.792 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.792 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.792 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.792 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.792 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.792 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.792 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.793 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.793 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.793 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.793 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.793 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.793 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.793 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.793 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.793 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.793 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.794 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.794 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.794 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.794 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.794 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.794 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.794 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.794 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.795 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.795 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.795 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.795 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.795 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.795 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.795 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.795 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.795 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.795 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.796 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.796 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "telemetry", 2024-04-23T14:59:14.796 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.796 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.796 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.796 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_basic": { 2024-04-23T14:59:14.796 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_basic", 2024-04-23T14:59:14.796 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.796 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.797 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.797 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.797 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.797 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.797 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.797 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T14:59:14.797 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.797 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.797 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.797 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.798 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_crash": { 2024-04-23T14:59:14.798 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_crash", 2024-04-23T14:59:14.798 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.798 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.798 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.798 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.798 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.798 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.799 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.799 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T14:59:14.799 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.799 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.799 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.799 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.799 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_device": { 2024-04-23T14:59:14.799 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_device", 2024-04-23T14:59:14.799 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.799 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.800 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.800 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.800 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.800 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.800 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.800 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T14:59:14.800 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.800 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.800 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.801 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.801 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_ident": { 2024-04-23T14:59:14.801 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_ident", 2024-04-23T14:59:14.801 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.801 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.801 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.801 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.801 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.801 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.802 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.802 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T14:59:14.802 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.802 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.802 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.802 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.802 INFO:teuthology.orchestra.run.smithi083.stdout: "channel_perf": { 2024-04-23T14:59:14.802 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "channel_perf", 2024-04-23T14:59:14.802 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.803 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.803 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.803 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.803 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.803 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.803 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.803 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T14:59:14.803 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.803 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.804 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.804 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.804 INFO:teuthology.orchestra.run.smithi083.stdout: "contact": { 2024-04-23T14:59:14.804 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "contact", 2024-04-23T14:59:14.804 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.804 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.804 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.804 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.804 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.804 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.805 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.805 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.805 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.805 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.805 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.805 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.805 INFO:teuthology.orchestra.run.smithi083.stdout: "description": { 2024-04-23T14:59:14.805 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "description", 2024-04-23T14:59:14.805 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.806 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.806 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.806 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.806 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.806 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.806 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.806 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.806 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.806 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.807 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.807 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.807 INFO:teuthology.orchestra.run.smithi083.stdout: "device_url": { 2024-04-23T14:59:14.807 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "device_url", 2024-04-23T14:59:14.807 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.807 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.807 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.807 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T14:59:14.807 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.807 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.808 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.808 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.808 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.808 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.808 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.808 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.808 INFO:teuthology.orchestra.run.smithi083.stdout: "enabled": { 2024-04-23T14:59:14.808 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "enabled", 2024-04-23T14:59:14.808 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.809 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.809 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.809 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.809 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.809 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.809 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.809 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.809 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.809 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.810 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.810 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.810 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:14.810 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:14.810 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.810 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.810 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.810 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "24", 2024-04-23T14:59:14.810 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "8", 2024-04-23T14:59:14.811 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.811 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.811 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.811 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.811 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.811 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.811 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.811 INFO:teuthology.orchestra.run.smithi083.stdout: "last_opt_revision": { 2024-04-23T14:59:14.811 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "last_opt_revision", 2024-04-23T14:59:14.812 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.812 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.812 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.812 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "1", 2024-04-23T14:59:14.812 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.812 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.812 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.812 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.812 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.813 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.813 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.813 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.813 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard": { 2024-04-23T14:59:14.813 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard", 2024-04-23T14:59:14.813 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.813 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.813 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.813 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.813 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.814 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.814 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.814 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.814 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.814 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.814 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.814 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.814 INFO:teuthology.orchestra.run.smithi083.stdout: "leaderboard_description": { 2024-04-23T14:59:14.814 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "leaderboard_description", 2024-04-23T14:59:14.815 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.815 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.815 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.815 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.815 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.815 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.815 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.815 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.815 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.816 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.816 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.816 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.816 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.816 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.816 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.816 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.816 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.816 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.816 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.817 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.817 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.817 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.817 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.817 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.817 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.817 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.817 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.817 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.818 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.818 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.818 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.818 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.818 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.818 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.818 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.818 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.818 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.819 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.819 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.819 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.819 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.819 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.819 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.819 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.819 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.819 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.820 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.820 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.820 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.820 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.820 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.820 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.820 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.820 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.820 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.820 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.821 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.821 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.821 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.821 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.821 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.821 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.821 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.821 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.821 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.822 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.822 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.822 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.822 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.822 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.822 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.822 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.822 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.823 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.823 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.823 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.823 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.823 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.823 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.823 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.823 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.823 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.823 INFO:teuthology.orchestra.run.smithi083.stdout: "organization": { 2024-04-23T14:59:14.824 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "organization", 2024-04-23T14:59:14.824 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.824 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.824 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.824 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.824 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.824 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.824 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.824 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.824 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.825 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.825 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.825 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.825 INFO:teuthology.orchestra.run.smithi083.stdout: "proxy": { 2024-04-23T14:59:14.825 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "proxy", 2024-04-23T14:59:14.825 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.825 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.825 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.825 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.826 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.826 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.826 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.826 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.826 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.826 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.826 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.826 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.826 INFO:teuthology.orchestra.run.smithi083.stdout: "url": { 2024-04-23T14:59:14.827 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "url", 2024-04-23T14:59:14.827 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.827 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.827 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.827 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T14:59:14.827 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.827 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.827 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.827 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.828 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.828 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.828 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.828 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.828 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.828 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.828 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.828 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "test_orchestrator", 2024-04-23T14:59:14.828 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.829 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.829 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.829 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.829 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.829 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.829 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.829 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.829 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.829 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.829 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.830 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.830 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.830 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.830 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.830 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.830 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.830 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.830 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.831 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.831 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.831 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.831 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.831 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.831 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.831 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.831 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.831 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.831 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.832 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.832 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.832 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.832 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.832 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.832 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.832 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.832 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.832 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.832 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.833 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.833 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.833 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.833 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.833 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.833 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.833 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.833 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.833 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.834 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.834 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.834 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.834 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.834 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.834 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.834 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.834 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.834 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.835 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.835 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.835 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.835 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.835 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.835 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.835 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.835 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.835 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.836 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.836 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.836 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.836 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.836 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.836 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.836 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.836 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.836 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.836 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.837 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-04-23T14:59:14.837 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.837 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.837 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.837 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.837 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.837 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.837 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.837 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.838 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.838 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.838 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.838 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.838 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.838 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.838 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.838 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.839 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.839 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.839 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.839 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.839 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.839 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.839 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.839 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.839 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.839 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.840 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.840 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.840 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.840 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.840 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.840 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.840 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.840 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.840 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.841 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.841 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.841 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.841 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.841 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.841 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.841 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.841 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.841 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.842 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.842 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.842 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.842 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.842 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.842 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.842 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.842 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.842 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.843 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.843 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.843 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.843 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.843 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.843 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.843 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.843 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.843 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.844 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.844 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.844 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.844 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.844 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.844 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.844 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.844 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.844 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.845 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.845 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.845 INFO:teuthology.orchestra.run.smithi083.stdout: "max_concurrent_clones": { 2024-04-23T14:59:14.845 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "max_concurrent_clones", 2024-04-23T14:59:14.845 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.845 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.845 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.845 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "4", 2024-04-23T14:59:14.845 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.845 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.846 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.846 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T14:59:14.846 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.846 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.846 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.846 INFO:teuthology.orchestra.run.smithi083.stdout: "periodic_async_work": { 2024-04-23T14:59:14.846 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "periodic_async_work", 2024-04-23T14:59:14.846 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.847 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.847 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.847 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.847 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.847 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.847 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.847 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Periodically check for async work", 2024-04-23T14:59:14.847 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.847 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.848 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.848 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_delay": { 2024-04-23T14:59:14.848 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_delay", 2024-04-23T14:59:14.848 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.848 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.848 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.848 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "0", 2024-04-23T14:59:14.848 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.849 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.849 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.849 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T14:59:14.849 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.849 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.849 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.849 INFO:teuthology.orchestra.run.smithi083.stdout: "snapshot_clone_no_wait": { 2024-04-23T14:59:14.849 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T14:59:14.850 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.850 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.850 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.850 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "True", 2024-04-23T14:59:14.850 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.850 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.850 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.851 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T14:59:14.851 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.851 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.851 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.851 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.851 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.851 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.851 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.851 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix", 2024-04-23T14:59:14.852 INFO:teuthology.orchestra.run.smithi083.stdout: "can_run": true, 2024-04-23T14:59:14.852 INFO:teuthology.orchestra.run.smithi083.stdout: "error_string": "", 2024-04-23T14:59:14.852 INFO:teuthology.orchestra.run.smithi083.stdout: "module_options": { 2024-04-23T14:59:14.852 INFO:teuthology.orchestra.run.smithi083.stdout: "discovery_interval": { 2024-04-23T14:59:14.852 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "discovery_interval", 2024-04-23T14:59:14.852 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "uint", 2024-04-23T14:59:14.852 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.852 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.852 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "100", 2024-04-23T14:59:14.853 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.853 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.853 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.853 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.853 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.853 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.853 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.853 INFO:teuthology.orchestra.run.smithi083.stdout: "identifier": { 2024-04-23T14:59:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "identifier", 2024-04-23T14:59:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.854 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "interval": { 2024-04-23T14:59:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "interval", 2024-04-23T14:59:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "secs", 2024-04-23T14:59:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.855 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "60", 2024-04-23T14:59:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.856 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "log_level": { 2024-04-23T14:59:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_level", 2024-04-23T14:59:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.857 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.858 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster": { 2024-04-23T14:59:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster", 2024-04-23T14:59:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.859 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.860 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_cluster_level": { 2024-04-23T14:59:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_cluster_level", 2024-04-23T14:59:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "info", 2024-04-23T14:59:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [ 2024-04-23T14:59:14.861 INFO:teuthology.orchestra.run.smithi083.stdout: "", 2024-04-23T14:59:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "critical", 2024-04-23T14:59:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "debug", 2024-04-23T14:59:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "error", 2024-04-23T14:59:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "info", 2024-04-23T14:59:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "warning" 2024-04-23T14:59:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.862 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "log_to_file": { 2024-04-23T14:59:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "log_to_file", 2024-04-23T14:59:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "bool", 2024-04-23T14:59:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 1, 2024-04-23T14:59:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "False", 2024-04-23T14:59:14.863 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_host": { 2024-04-23T14:59:14.864 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_host", 2024-04-23T14:59:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "", 2024-04-23T14:59:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.865 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_port": { 2024-04-23T14:59:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_port", 2024-04-23T14:59:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "int", 2024-04-23T14:59:14.866 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "10051", 2024-04-23T14:59:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.867 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "zabbix_sender": { 2024-04-23T14:59:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "zabbix_sender", 2024-04-23T14:59:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "str", 2024-04-23T14:59:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "level": "advanced", 2024-04-23T14:59:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "flags": 0, 2024-04-23T14:59:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T14:59:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "min": "", 2024-04-23T14:59:14.868 INFO:teuthology.orchestra.run.smithi083.stdout: "max": "", 2024-04-23T14:59:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "enum_allowed": [], 2024-04-23T14:59:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "desc": "", 2024-04-23T14:59:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "long_desc": "", 2024-04-23T14:59:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "tags": [], 2024-04-23T14:59:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "see_also": [] 2024-04-23T14:59:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.869 INFO:teuthology.orchestra.run.smithi083.stdout: "services": { 2024-04-23T14:59:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "dashboard": "https://172.21.15.83:8443/" 2024-04-23T14:59:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "always_on_modules": { 2024-04-23T14:59:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "octopus": [ 2024-04-23T14:59:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-23T14:59:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-23T14:59:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-23T14:59:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-23T14:59:14.870 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-23T14:59:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-23T14:59:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-23T14:59:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-23T14:59:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-23T14:59:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-23T14:59:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "pacific": [ 2024-04-23T14:59:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-23T14:59:14.871 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-23T14:59:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-23T14:59:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-23T14:59:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-23T14:59:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-23T14:59:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-23T14:59:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-23T14:59:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-23T14:59:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-23T14:59:14.872 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "quincy": [ 2024-04-23T14:59:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-23T14:59:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-23T14:59:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-23T14:59:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-23T14:59:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-23T14:59:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-23T14:59:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-23T14:59:14.873 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-23T14:59:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-23T14:59:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-23T14:59:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "reef": [ 2024-04-23T14:59:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-23T14:59:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-23T14:59:14.874 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-23T14:59:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-23T14:59:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-23T14:59:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-23T14:59:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-23T14:59:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-23T14:59:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-23T14:59:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-23T14:59:14.875 INFO:teuthology.orchestra.run.smithi083.stdout: ], 2024-04-23T14:59:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "squid": [ 2024-04-23T14:59:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "balancer", 2024-04-23T14:59:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "crash", 2024-04-23T14:59:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "devicehealth", 2024-04-23T14:59:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "orchestrator", 2024-04-23T14:59:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "pg_autoscaler", 2024-04-23T14:59:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "progress", 2024-04-23T14:59:14.876 INFO:teuthology.orchestra.run.smithi083.stdout: "rbd_support", 2024-04-23T14:59:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: "status", 2024-04-23T14:59:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: "telemetry", 2024-04-23T14:59:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: "volumes" 2024-04-23T14:59:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-23T14:59:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: "last_failure_osd_epoch": 89, 2024-04-23T14:59:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: "active_clients": [ 2024-04-23T14:59:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.877 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "devicehealth", 2024-04-23T14:59:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-23T14:59:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-23T14:59:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-23T14:59:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1157217363 2024-04-23T14:59:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-23T14:59:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.878 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "libcephsqlite", 2024-04-23T14:59:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-23T14:59:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-23T14:59:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-23T14:59:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 230446960 2024-04-23T14:59:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-23T14:59:14.879 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "rbd_support", 2024-04-23T14:59:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-23T14:59:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-23T14:59:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-23T14:59:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 1629051147 2024-04-23T14:59:14.880 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-23T14:59:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-23T14:59:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-23T14:59:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-23T14:59:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-23T14:59:14.881 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 687639697 2024-04-23T14:59:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-23T14:59:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "snap_schedule", 2024-04-23T14:59:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-23T14:59:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.882 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-23T14:59:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-23T14:59:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 927092118 2024-04-23T14:59:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-23T14:59:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "name": "volumes", 2024-04-23T14:59:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: "addrvec": [ 2024-04-23T14:59:14.883 INFO:teuthology.orchestra.run.smithi083.stdout: { 2024-04-23T14:59:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "type": "v2", 2024-04-23T14:59:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "addr": "172.21.15.83:0", 2024-04-23T14:59:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: "nonce": 2365537387 2024-04-23T14:59:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-23T14:59:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:14.884 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-04-23T14:59:14.884 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:59:14.886 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:13 smithi203 ceph-mon[32800]: pgmap v335: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T14:59:14.886 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:13 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/186190155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:59:14.886 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:13 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/835609538' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T14:59:14.886 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:13 smithi173 ceph-mon[32443]: pgmap v335: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T14:59:14.886 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:13 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/186190155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:59:14.886 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:13 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/835609538' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T14:59:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.851+0000 7f4c82000640 1 -- 172.21.15.83:0/835609538 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4c600a3da0 msgr2=0x7f4c600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.851+0000 7f4c82000640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4c600a3da0 0x7f4c600a6260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4c680095f0 tx=0x7f4c68005f90 comp rx=0 tx=0).stop 2024-04-23T14:59:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.851+0000 7f4c82000640 1 -- 172.21.15.83:0/835609538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c7c0743d0 msgr2=0x7f4c7c1a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:14.887 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.851+0000 7f4c82000640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c7c0743d0 0x7f4c7c1a0810 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c004af0 tx=0x7f4c6c02f6b0 comp rx=0 tx=0).stop 2024-04-23T14:59:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.851+0000 7f4c82000640 1 -- 172.21.15.83:0/835609538 shutdown_connections 2024-04-23T14:59:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.851+0000 7f4c82000640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c7c10e870 0x7f4c7c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.851+0000 7f4c82000640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f4c600a3da0 0x7f4c600a6260 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.851+0000 7f4c82000640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c7c074cf0 0x7f4c7c1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.851+0000 7f4c82000640 1 --2- 172.21.15.83:0/835609538 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c7c0743d0 0x7f4c7c1a0810 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.851+0000 7f4c82000640 1 -- 172.21.15.83:0/835609538 >> 172.21.15.83:0/835609538 conn(0x7f4c7c0793c0 msgr2=0x7f4c7c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.851+0000 7f4c82000640 1 -- 172.21.15.83:0/835609538 shutdown_connections 2024-04-23T14:59:14.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:12.851+0000 7f4c82000640 1 -- 172.21.15.83:0/835609538 wait complete. 2024-04-23T14:59:14.895 DEBUG:tasks.check_counter:Getting stats from a 2024-04-23T14:59:14.895 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-23T14:59:14.896 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:13 smithi083 ceph-mon[27867]: pgmap v335: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T14:59:14.896 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:13 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/186190155' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:59:14.896 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:13 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/835609538' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T14:59:15.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:15 smithi083 ceph-mon[27867]: pgmap v336: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:59:15.353 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:59:15.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:15 smithi203 ceph-mon[32800]: pgmap v336: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:59:15.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:15 smithi173 ceph-mon[32443]: pgmap v336: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:59:17.369 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:17 smithi083 ceph-mon[27867]: pgmap v337: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:59:17.471 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:17 smithi203 ceph-mon[32800]: pgmap v337: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:59:17.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:17 smithi173 ceph-mon[32443]: pgmap v337: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:59:17.768 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:59:17.768 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T14:59:17.769 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-04-23T14:59:17.769 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-04-23T14:59:17.769 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-04-23T14:59:17.769 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-04-23T14:59:17.769 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-23T14:59:17.769 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-04-23T14:59:17.769 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.004059747, 2024-04-23T14:59:17.769 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-04-23T14:59:17.770 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000826672, 2024-04-23T14:59:17.770 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T14:59:17.770 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:17.770 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:17.770 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:17.770 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:17.770 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:17.770 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:17.771 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:17.771 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:17.771 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:17.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:17.771 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T14:59:17.771 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T14:59:17.771 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:17.771 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T14:59:17.772 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 227, 2024-04-23T14:59:17.772 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 216, 2024-04-23T14:59:17.772 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1591264, 2024-04-23T14:59:17.772 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 133024, 2024-04-23T14:59:17.772 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-04-23T14:59:17.772 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-23T14:59:17.772 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.109871259, 2024-04-23T14:59:17.772 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.036657844, 2024-04-23T14:59:17.773 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.066373113, 2024-04-23T14:59:17.773 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.008753656, 2024-04-23T14:59:17.773 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:17.773 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 215, 2024-04-23T14:59:17.773 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.054581490, 2024-04-23T14:59:17.773 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000253867 2024-04-23T14:59:17.773 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:17.773 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:17.773 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:17.774 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:17.774 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:17.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:17.774 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1591264, 2024-04-23T14:59:17.774 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 133024 2024-04-23T14:59:17.774 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:17.774 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T14:59:17.774 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 6, 2024-04-23T14:59:17.775 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 167, 2024-04-23T14:59:17.775 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3344, 2024-04-23T14:59:17.775 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 185792, 2024-04-23T14:59:17.775 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-23T14:59:17.775 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-23T14:59:17.775 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.032858322, 2024-04-23T14:59:17.775 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.022030426, 2024-04-23T14:59:17.775 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.005963138, 2024-04-23T14:59:17.775 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T14:59:17.776 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:17.776 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 165, 2024-04-23T14:59:17.776 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.032416044, 2024-04-23T14:59:17.776 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000196460 2024-04-23T14:59:17.776 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:17.776 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:17.776 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:17.776 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:17.777 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:17.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:17.777 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-23T14:59:17.777 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 185792 2024-04-23T14:59:17.777 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:17.777 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-23T14:59:17.777 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-04-23T14:59:17.777 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-23T14:59:17.778 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:17.778 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-23T14:59:17.778 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-23T14:59:17.778 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-23T14:59:17.778 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T14:59:17.778 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-23T14:59:17.778 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T14:59:17.778 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-23T14:59:17.779 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T14:59:17.779 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T14:59:17.779 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T14:59:17.779 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T14:59:17.779 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T14:59:17.779 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-23T14:59:17.779 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T14:59:17.779 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T14:59:17.780 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-23T14:59:17.780 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-23T14:59:17.780 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-23T14:59:17.780 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-23T14:59:17.780 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T14:59:17.780 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T14:59:17.780 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T14:59:17.780 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T14:59:17.781 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T14:59:17.781 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-23T14:59:17.781 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-23T14:59:17.781 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-23T14:59:17.781 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T14:59:17.781 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T14:59:17.781 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-23T14:59:17.782 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-23T14:59:17.782 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-23T14:59:17.782 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-23T14:59:17.782 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T14:59:17.782 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-23T14:59:17.782 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T14:59:17.782 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-23T14:59:17.782 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148994, 2024-04-23T14:59:17.783 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 31, 2024-04-23T14:59:17.783 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-23T14:59:17.783 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-23T14:59:17.783 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-23T14:59:17.783 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-23T14:59:17.783 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-23T14:59:17.783 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-23T14:59:17.783 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-23T14:59:17.784 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-23T14:59:17.784 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-04-23T14:59:17.784 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-04-23T14:59:17.784 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T14:59:17.784 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-23T14:59:17.784 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-23T14:59:17.784 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-23T14:59:17.784 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-04-23T14:59:17.785 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-04-23T14:59:17.785 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-23T14:59:17.785 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-23T14:59:17.785 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-23T14:59:17.785 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-23T14:59:17.785 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:17.785 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T14:59:17.785 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-23T14:59:17.786 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-23T14:59:17.786 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-23T14:59:17.786 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 229, 2024-04-23T14:59:17.786 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1559249, 2024-04-23T14:59:17.786 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:17.786 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 229, 2024-04-23T14:59:17.786 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-23T14:59:17.786 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-23T14:59:17.787 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 229, 2024-04-23T14:59:17.787 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1559249, 2024-04-23T14:59:17.787 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-23T14:59:17.787 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:17.787 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:17.787 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:17.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:17.787 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:17.787 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:59:18.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:18 smithi203 ceph-mon[32800]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T14:59:18.735 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:18 smithi083 ceph-mon[27867]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T14:59:18.798 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-23T14:59:18.799 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-23T14:59:18.799 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-23T14:59:18.799 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-23T14:59:18.799 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-23T14:59:18.799 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-23T14:59:18.799 DEBUG:tasks.check_counter:Getting stats from d 2024-04-23T14:59:18.799 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-23T14:59:18.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:18 smithi173 ceph-mon[32443]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T14:59:19.284 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:59:19.634 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:19 smithi083 ceph-mon[27867]: pgmap v339: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 230 B/s wr, 7 op/s 2024-04-23T14:59:19.720 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:19 smithi203 ceph-mon[32800]: pgmap v339: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 230 B/s wr, 7 op/s 2024-04-23T14:59:19.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:19 smithi173 ceph-mon[32443]: pgmap v339: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 230 B/s wr, 7 op/s 2024-04-23T14:59:21.498 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:59:21.498 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T14:59:21.498 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 4, 2024-04-23T14:59:21.498 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 4, 2024-04-23T14:59:21.498 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3024, 2024-04-23T14:59:21.498 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 448, 2024-04-23T14:59:21.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-23T14:59:21.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-04-23T14:59:21.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.004284746, 2024-04-23T14:59:21.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000077324, 2024-04-23T14:59:21.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001340435, 2024-04-23T14:59:21.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T14:59:21.499 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:21.499 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-04-23T14:59:21.499 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.001389725, 2024-04-23T14:59:21.499 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000463241 2024-04-23T14:59:21.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:21.500 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:21.500 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:21.500 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:21.500 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:21.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:21.500 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-23T14:59:21.500 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T14:59:21.500 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:21.501 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T14:59:21.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2, 2024-04-23T14:59:21.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 171, 2024-04-23T14:59:21.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 320, 2024-04-23T14:59:21.501 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 186400, 2024-04-23T14:59:21.502 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-23T14:59:21.502 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-23T14:59:21.502 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.034080063, 2024-04-23T14:59:21.503 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.024699605, 2024-04-23T14:59:21.503 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.004076982, 2024-04-23T14:59:21.503 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T14:59:21.503 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:21.504 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 168, 2024-04-23T14:59:21.504 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.037826071, 2024-04-23T14:59:21.504 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000225155 2024-04-23T14:59:21.504 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:21.505 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:21.505 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:21.505 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:21.505 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:21.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:21.506 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T14:59:21.506 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 186400 2024-04-23T14:59:21.506 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:21.506 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T14:59:21.506 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 234, 2024-04-23T14:59:21.506 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 223, 2024-04-23T14:59:21.506 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1593168, 2024-04-23T14:59:21.506 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 137504, 2024-04-23T14:59:21.507 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-04-23T14:59:21.507 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-23T14:59:21.507 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.091205472, 2024-04-23T14:59:21.507 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.020995027, 2024-04-23T14:59:21.507 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.090402713, 2024-04-23T14:59:21.507 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.009376864, 2024-04-23T14:59:21.507 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:21.507 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 222, 2024-04-23T14:59:21.507 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.026146380, 2024-04-23T14:59:21.508 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000117776 2024-04-23T14:59:21.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:21.508 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:21.508 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:21.508 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:21.508 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:21.508 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:21.508 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1593168, 2024-04-23T14:59:21.508 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 137504 2024-04-23T14:59:21.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:21.509 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-23T14:59:21.509 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-04-23T14:59:21.509 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-23T14:59:21.509 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:21.509 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-23T14:59:21.509 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-23T14:59:21.509 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-23T14:59:21.509 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T14:59:21.509 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-23T14:59:21.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T14:59:21.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-23T14:59:21.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T14:59:21.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T14:59:21.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T14:59:21.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T14:59:21.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T14:59:21.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-23T14:59:21.510 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T14:59:21.511 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T14:59:21.511 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-23T14:59:21.511 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-23T14:59:21.511 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-23T14:59:21.511 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-23T14:59:21.511 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T14:59:21.511 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T14:59:21.511 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T14:59:21.511 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T14:59:21.511 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T14:59:21.512 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-23T14:59:21.512 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-23T14:59:21.512 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-23T14:59:21.512 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T14:59:21.512 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T14:59:21.512 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-23T14:59:21.512 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-23T14:59:21.512 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-23T14:59:21.512 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-23T14:59:21.513 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T14:59:21.513 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-23T14:59:21.513 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T14:59:21.513 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-23T14:59:21.513 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 149745, 2024-04-23T14:59:21.513 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 31, 2024-04-23T14:59:21.513 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-23T14:59:21.513 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-23T14:59:21.513 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-23T14:59:21.514 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-23T14:59:21.514 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-23T14:59:21.514 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-23T14:59:21.514 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-23T14:59:21.514 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-23T14:59:21.514 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-04-23T14:59:21.514 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-04-23T14:59:21.514 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T14:59:21.514 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-23T14:59:21.515 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-23T14:59:21.515 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-23T14:59:21.515 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-04-23T14:59:21.515 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-04-23T14:59:21.515 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-23T14:59:21.515 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-23T14:59:21.515 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-23T14:59:21.515 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-23T14:59:21.516 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:21.516 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T14:59:21.516 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-23T14:59:21.516 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-23T14:59:21.516 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-23T14:59:21.516 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 236, 2024-04-23T14:59:21.516 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1560159, 2024-04-23T14:59:21.516 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:21.517 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 236, 2024-04-23T14:59:21.517 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-23T14:59:21.517 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-23T14:59:21.517 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 236, 2024-04-23T14:59:21.517 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1560159, 2024-04-23T14:59:21.517 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-23T14:59:21.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:21.517 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:21.517 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:21.517 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:21.518 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:21.518 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:59:21.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:21 smithi203 ceph-mon[32800]: pgmap v340: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:59:21.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:21 smithi083 ceph-mon[27867]: pgmap v340: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:59:21.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:21 smithi173 ceph-mon[32443]: pgmap v340: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-23T14:59:22.313 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-23T14:59:22.313 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-23T14:59:22.313 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-23T14:59:22.313 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-23T14:59:22.313 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-23T14:59:22.313 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-23T14:59:22.314 DEBUG:tasks.check_counter:Getting stats from g 2024-04-23T14:59:22.314 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-23T14:59:22.760 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:59:23.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:23 smithi203 ceph-mon[32800]: pgmap v341: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:59:23.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:23 smithi083 ceph-mon[27867]: pgmap v341: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:59:23.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:23 smithi173 ceph-mon[32443]: pgmap v341: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:59:25.163 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:59:25.163 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T14:59:25.163 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 230, 2024-04-23T14:59:25.164 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 219, 2024-04-23T14:59:25.164 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1592080, 2024-04-23T14:59:25.164 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 134944, 2024-04-23T14:59:25.164 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-23T14:59:25.164 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-04-23T14:59:25.164 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.116780712, 2024-04-23T14:59:25.164 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.035699529, 2024-04-23T14:59:25.164 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.105306772, 2024-04-23T14:59:25.164 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.009078920, 2024-04-23T14:59:25.165 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:25.165 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 218, 2024-04-23T14:59:25.165 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.034818920, 2024-04-23T14:59:25.165 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000159719 2024-04-23T14:59:25.165 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:25.165 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:25.165 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:25.165 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:25.166 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:25.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:25.166 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1592080, 2024-04-23T14:59:25.166 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 134944 2024-04-23T14:59:25.166 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:25.166 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T14:59:25.166 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2, 2024-04-23T14:59:25.166 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 168, 2024-04-23T14:59:25.167 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 320, 2024-04-23T14:59:25.167 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 185872, 2024-04-23T14:59:25.167 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-23T14:59:25.167 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-23T14:59:25.167 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.032521963, 2024-04-23T14:59:25.167 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.023439112, 2024-04-23T14:59:25.167 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.003538099, 2024-04-23T14:59:25.167 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T14:59:25.168 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:25.168 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 165, 2024-04-23T14:59:25.168 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.040734980, 2024-04-23T14:59:25.168 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000246878 2024-04-23T14:59:25.168 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:25.168 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:25.168 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:25.168 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:25.169 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:25.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:25.169 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T14:59:25.169 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 185872 2024-04-23T14:59:25.169 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:25.169 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T14:59:25.169 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 4, 2024-04-23T14:59:25.169 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 4, 2024-04-23T14:59:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3024, 2024-04-23T14:59:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 448, 2024-04-23T14:59:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-04-23T14:59:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-23T14:59:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.001689837, 2024-04-23T14:59:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000108853, 2024-04-23T14:59:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.001499979, 2024-04-23T14:59:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T14:59:25.170 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-04-23T14:59:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.001649722, 2024-04-23T14:59:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000549907 2024-04-23T14:59:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:25.171 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-23T14:59:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T14:59:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-23T14:59:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-04-23T14:59:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-23T14:59:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:25.172 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-23T14:59:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-23T14:59:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-23T14:59:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T14:59:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-23T14:59:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T14:59:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-23T14:59:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T14:59:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T14:59:25.173 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T14:59:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T14:59:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T14:59:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-23T14:59:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T14:59:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T14:59:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-23T14:59:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-23T14:59:25.174 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-23T14:59:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-23T14:59:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T14:59:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T14:59:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T14:59:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T14:59:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T14:59:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-23T14:59:25.175 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-23T14:59:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-23T14:59:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T14:59:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T14:59:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-23T14:59:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-23T14:59:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-23T14:59:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-23T14:59:25.176 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T14:59:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-23T14:59:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T14:59:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-23T14:59:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148977, 2024-04-23T14:59:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-04-23T14:59:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-23T14:59:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-23T14:59:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-23T14:59:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-23T14:59:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-23T14:59:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-23T14:59:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-23T14:59:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-23T14:59:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-04-23T14:59:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-04-23T14:59:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T14:59:25.178 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-23T14:59:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-23T14:59:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-23T14:59:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-04-23T14:59:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-04-23T14:59:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-23T14:59:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-23T14:59:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-23T14:59:25.179 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-23T14:59:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T14:59:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-23T14:59:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-23T14:59:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-23T14:59:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 232, 2024-04-23T14:59:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1559639, 2024-04-23T14:59:25.180 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 232, 2024-04-23T14:59:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-23T14:59:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-23T14:59:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 232, 2024-04-23T14:59:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1559639, 2024-04-23T14:59:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-23T14:59:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:25.181 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:25.182 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:25.182 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:59:25.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:25 smithi083 ceph-mon[27867]: pgmap v342: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:59:25.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:25 smithi203 ceph-mon[32800]: pgmap v342: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:59:25.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:25 smithi173 ceph-mon[32443]: pgmap v342: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-23T14:59:26.030 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-23T14:59:26.030 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-23T14:59:26.031 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-23T14:59:26.031 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-23T14:59:26.031 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-23T14:59:26.031 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-23T14:59:26.031 DEBUG:tasks.check_counter:Getting stats from j 2024-04-23T14:59:26.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-23T14:59:26.502 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T14:59:27.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:27 smithi083 ceph-mon[27867]: pgmap v343: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:59:27.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:27 smithi203 ceph-mon[32800]: pgmap v343: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:59:27.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:27 smithi173 ceph-mon[32443]: pgmap v343: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T14:59:28.888 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-04-23T14:59:28.889 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T14:59:28.889 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 0, 2024-04-23T14:59:28.889 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2, 2024-04-23T14:59:28.889 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 0, 2024-04-23T14:59:28.889 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 0, 2024-04-23T14:59:28.889 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-04-23T14:59:28.889 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-04-23T14:59:28.890 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.003512906, 2024-04-23T14:59:28.890 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000000000, 2024-04-23T14:59:28.890 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.000631332, 2024-04-23T14:59:28.890 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T14:59:28.890 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:28.890 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:28.890 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:28.890 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:28.890 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:28.891 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:28.891 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:28.891 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:28.891 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:28.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:28.891 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T14:59:28.891 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T14:59:28.891 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:28.891 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T14:59:28.891 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 237, 2024-04-23T14:59:28.892 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 226, 2024-04-23T14:59:28.892 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1593984, 2024-04-23T14:59:28.892 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 139424, 2024-04-23T14:59:28.892 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-04-23T14:59:28.892 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-04-23T14:59:28.892 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.100930025, 2024-04-23T14:59:28.892 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.033592986, 2024-04-23T14:59:28.892 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.082788479, 2024-04-23T14:59:28.892 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.010738458, 2024-04-23T14:59:28.893 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:28.893 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 225, 2024-04-23T14:59:28.893 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.036845526, 2024-04-23T14:59:28.893 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000163757 2024-04-23T14:59:28.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:28.893 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:28.893 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:28.893 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:28.893 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:28.893 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:28.894 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1593984, 2024-04-23T14:59:28.894 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 139424 2024-04-23T14:59:28.894 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:28.894 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T14:59:28.894 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 6, 2024-04-23T14:59:28.894 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 175, 2024-04-23T14:59:28.894 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3344, 2024-04-23T14:59:28.894 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 187200, 2024-04-23T14:59:28.895 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-04-23T14:59:28.895 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-04-23T14:59:28.895 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.032506867, 2024-04-23T14:59:28.895 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.022921564, 2024-04-23T14:59:28.895 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.004948643, 2024-04-23T14:59:28.895 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T14:59:28.895 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:28.895 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 173, 2024-04-23T14:59:28.895 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.040143953, 2024-04-23T14:59:28.896 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000232045 2024-04-23T14:59:28.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:28.896 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:28.896 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:28.896 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:28.896 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:28.896 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:28.896 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-23T14:59:28.896 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 187200 2024-04-23T14:59:28.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:28.897 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-04-23T14:59:28.897 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-04-23T14:59:28.897 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-04-23T14:59:28.897 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:28.897 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-04-23T14:59:28.897 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-04-23T14:59:28.897 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-04-23T14:59:28.897 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T14:59:28.898 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-04-23T14:59:28.898 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T14:59:28.898 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-04-23T14:59:28.898 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T14:59:28.898 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T14:59:28.898 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T14:59:28.898 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T14:59:28.898 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T14:59:28.898 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-04-23T14:59:28.899 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T14:59:28.899 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T14:59:28.899 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-04-23T14:59:28.899 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-04-23T14:59:28.899 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-04-23T14:59:28.899 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-04-23T14:59:28.899 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T14:59:28.899 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T14:59:28.899 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T14:59:28.900 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T14:59:28.900 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T14:59:28.900 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-04-23T14:59:28.900 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-04-23T14:59:28.900 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-04-23T14:59:28.900 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T14:59:28.900 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T14:59:28.900 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-04-23T14:59:28.900 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-04-23T14:59:28.900 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-04-23T14:59:28.901 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-04-23T14:59:28.901 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T14:59:28.901 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-04-23T14:59:28.901 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T14:59:28.901 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-04-23T14:59:28.901 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 148714, 2024-04-23T14:59:28.901 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 30, 2024-04-23T14:59:28.901 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-04-23T14:59:28.901 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-04-23T14:59:28.902 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-04-23T14:59:28.902 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-04-23T14:59:28.902 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-04-23T14:59:28.902 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-04-23T14:59:28.902 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-04-23T14:59:28.902 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-04-23T14:59:28.902 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-04-23T14:59:28.902 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-04-23T14:59:28.902 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T14:59:28.903 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-04-23T14:59:28.903 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-04-23T14:59:28.903 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-04-23T14:59:28.903 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-04-23T14:59:28.903 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-04-23T14:59:28.903 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-04-23T14:59:28.903 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-04-23T14:59:28.903 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-04-23T14:59:28.903 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-04-23T14:59:28.903 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-04-23T14:59:28.904 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T14:59:28.904 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-04-23T14:59:28.904 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-04-23T14:59:28.904 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-04-23T14:59:28.904 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 239, 2024-04-23T14:59:28.904 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1560549, 2024-04-23T14:59:28.904 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:28.904 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 239, 2024-04-23T14:59:28.904 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-04-23T14:59:28.904 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-04-23T14:59:28.904 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 239, 2024-04-23T14:59:28.905 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1560549, 2024-04-23T14:59:28.905 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-04-23T14:59:28.905 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-04-23T14:59:28.905 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-04-23T14:59:28.905 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-04-23T14:59:28.905 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:28.905 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-04-23T14:59:28.905 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-04-23T14:59:29.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:29 smithi203 ceph-mon[32800]: pgmap v344: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-23T14:59:29.753 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:29 smithi083 ceph-mon[27867]: pgmap v344: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-23T14:59:29.819 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-23T14:59:29.819 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-23T14:59:29.819 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-23T14:59:29.819 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-23T14:59:29.819 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-23T14:59:29.820 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-23T14:59:29.820 DEBUG:tasks.check_counter:Getting stats from b 2024-04-23T14:59:29.820 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-23T14:59:29.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:29 smithi173 ceph-mon[32443]: pgmap v344: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-23T14:59:30.087 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T14:59:31.551 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:31 smithi173 ceph-mon[32443]: pgmap v345: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:59:31.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:31 smithi203 ceph-mon[32800]: pgmap v345: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:59:31.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:31 smithi083 ceph-mon[27867]: pgmap v345: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:59:31.924 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-23T14:59:31.924 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T14:59:31.924 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 2144, 2024-04-23T14:59:31.924 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 1351, 2024-04-23T14:59:31.924 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 2493473, 2024-04-23T14:59:31.925 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 613246, 2024-04-23T14:59:31.925 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 14, 2024-04-23T14:59:31.925 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2024-04-23T14:59:31.925 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.270564314, 2024-04-23T14:59:31.925 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.076188617, 2024-04-23T14:59:31.925 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.116769750, 2024-04-23T14:59:31.925 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.075521877, 2024-04-23T14:59:31.925 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:31.925 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1354, 2024-04-23T14:59:31.926 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.252506045, 2024-04-23T14:59:31.926 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000186488 2024-04-23T14:59:31.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.926 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:31.926 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2193, 2024-04-23T14:59:31.926 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000469157, 2024-04-23T14:59:31.926 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000213 2024-04-23T14:59:31.926 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.926 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 2041264, 2024-04-23T14:59:31.927 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 272736 2024-04-23T14:59:31.927 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.927 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T14:59:31.927 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 2487, 2024-04-23T14:59:31.927 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 3039, 2024-04-23T14:59:31.927 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 963249, 2024-04-23T14:59:31.927 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 3846845, 2024-04-23T14:59:31.927 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 157, 2024-04-23T14:59:31.927 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 7, 2024-04-23T14:59:31.928 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.299274363, 2024-04-23T14:59:31.928 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.085162705, 2024-04-23T14:59:31.928 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.159703094, 2024-04-23T14:59:31.928 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.058825246, 2024-04-23T14:59:31.928 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:31.928 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3037, 2024-04-23T14:59:31.928 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.281377822, 2024-04-23T14:59:31.928 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000092649 2024-04-23T14:59:31.928 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.929 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:31.929 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3824, 2024-04-23T14:59:31.929 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000643871, 2024-04-23T14:59:31.929 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000168 2024-04-23T14:59:31.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.929 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 857280, 2024-04-23T14:59:31.929 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 2042672 2024-04-23T14:59:31.929 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.929 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T14:59:31.930 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 2681, 2024-04-23T14:59:31.930 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 2206, 2024-04-23T14:59:31.930 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 784168, 2024-04-23T14:59:31.930 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 5040346, 2024-04-23T14:59:31.930 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 10, 2024-04-23T14:59:31.930 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 7, 2024-04-23T14:59:31.930 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.198228797, 2024-04-23T14:59:31.930 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.068210776, 2024-04-23T14:59:31.931 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.097859445, 2024-04-23T14:59:31.931 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.042499236, 2024-04-23T14:59:31.931 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:31.931 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2200, 2024-04-23T14:59:31.931 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.476775666, 2024-04-23T14:59:31.931 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000216716 2024-04-23T14:59:31.931 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.931 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:31.931 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3341, 2024-04-23T14:59:31.932 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000612815, 2024-04-23T14:59:31.932 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000183 2024-04-23T14:59:31.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.932 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 43472, 2024-04-23T14:59:31.932 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 4400 2024-04-23T14:59:31.932 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.932 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-23T14:59:31.932 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-04-23T14:59:31.933 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-23T14:59:31.933 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.933 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-04-23T14:59:31.933 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-23T14:59:31.933 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-23T14:59:31.933 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 703, 2024-04-23T14:59:31.933 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.834875718, 2024-04-23T14:59:31.933 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001187589 2024-04-23T14:59:31.933 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:31.934 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.934 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-04-23T14:59:31.934 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-23T14:59:31.934 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-23T14:59:31.934 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 172, 2024-04-23T14:59:31.934 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.024621761, 2024-04-23T14:59:31.934 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000143149 2024-04-23T14:59:31.934 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:31.935 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.935 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-04-23T14:59:31.935 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 957, 2024-04-23T14:59:31.935 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 941, 2024-04-23T14:59:31.935 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-04-23T14:59:31.935 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 941, 2024-04-23T14:59:31.935 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 12.665438566, 2024-04-23T14:59:31.935 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.013459552 2024-04-23T14:59:31.936 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.936 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-04-23T14:59:31.936 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 16, 2024-04-23T14:59:31.936 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-04-23T14:59:31.936 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-04-23T14:59:31.936 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 12, 2024-04-23T14:59:31.936 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-04-23T14:59:31.937 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-04-23T14:59:31.937 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 169, 2024-04-23T14:59:31.937 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 46, 2024-04-23T14:59:31.937 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 36, 2024-04-23T14:59:31.937 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 87, 2024-04-23T14:59:31.937 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 103, 2024-04-23T14:59:31.937 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 253, 2024-04-23T14:59:31.937 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 8, 2024-04-23T14:59:31.937 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 8, 2024-04-23T14:59:31.938 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 5, 2024-04-23T14:59:31.938 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 1520, 2024-04-23T14:59:31.938 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 1411, 2024-04-23T14:59:31.938 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 16, 2024-04-23T14:59:31.938 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 1, 2024-04-23T14:59:31.938 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-04-23T14:59:31.938 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-04-23T14:59:31.938 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 49, 2024-04-23T14:59:31.938 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2024-04-23T14:59:31.938 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-04-23T14:59:31.939 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 72, 2024-04-23T14:59:31.939 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 311, 2024-04-23T14:59:31.939 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 5, 2024-04-23T14:59:31.939 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 6, 2024-04-23T14:59:31.939 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-04-23T14:59:31.939 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 287, 2024-04-23T14:59:31.939 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-04-23T14:59:31.939 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 325, 2024-04-23T14:59:31.939 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 13424230511, 2024-04-23T14:59:31.940 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 6, 2024-04-23T14:59:31.940 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 287, 2024-04-23T14:59:31.940 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-04-23T14:59:31.940 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T14:59:31.940 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-04-23T14:59:31.940 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 158, 2024-04-23T14:59:31.940 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 18, 2024-04-23T14:59:31.940 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 140, 2024-04-23T14:59:31.940 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 129, 2024-04-23T14:59:31.940 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 1, 2024-04-23T14:59:31.941 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 208, 2024-04-23T14:59:31.941 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 421, 2024-04-23T14:59:31.941 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 1, 2024-04-23T14:59:31.941 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T14:59:31.941 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T14:59:31.941 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 338, 2024-04-23T14:59:31.941 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 130, 2024-04-23T14:59:31.941 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 15, 2024-04-23T14:59:31.941 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 344 2024-04-23T14:59:31.942 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.942 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-04-23T14:59:31.942 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-04-23T14:59:31.942 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 0, 2024-04-23T14:59:31.942 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 0, 2024-04-23T14:59:31.942 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 1, 2024-04-23T14:59:31.942 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 76, 2024-04-23T14:59:31.942 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-04-23T14:59:31.942 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-04-23T14:59:31.943 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-04-23T14:59:31.943 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 155, 2024-04-23T14:59:31.943 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 94, 2024-04-23T14:59:31.943 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 61, 2024-04-23T14:59:31.943 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-04-23T14:59:31.943 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-04-23T14:59:31.943 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-04-23T14:59:31.943 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-04-23T14:59:31.944 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-04-23T14:59:31.944 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-04-23T14:59:31.944 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 0, 2024-04-23T14:59:31.944 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 0, 2024-04-23T14:59:31.944 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 18, 2024-04-23T14:59:31.944 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 72, 2024-04-23T14:59:31.944 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-04-23T14:59:31.944 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-04-23T14:59:31.944 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-04-23T14:59:31.945 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-04-23T14:59:31.945 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.945 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-04-23T14:59:31.945 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-04-23T14:59:31.945 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 1069, 2024-04-23T14:59:31.945 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-04-23T14:59:31.945 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-04-23T14:59:31.945 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 1069, 2024-04-23T14:59:31.946 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-04-23T14:59:31.946 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-04-23T14:59:31.946 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 3, 2024-04-23T14:59:31.946 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-04-23T14:59:31.946 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-04-23T14:59:31.946 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 3, 2024-04-23T14:59:31.946 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 2, 2024-04-23T14:59:31.946 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-04-23T14:59:31.946 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-04-23T14:59:31.947 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-04-23T14:59:31.947 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 8198928, 2024-04-23T14:59:31.947 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 0, 2024-04-23T14:59:31.947 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-04-23T14:59:31.947 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 300, 2024-04-23T14:59:31.947 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 11.627628206, 2024-04-23T14:59:31.947 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.038758760 2024-04-23T14:59:31.947 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.947 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 0 2024-04-23T14:59:31.948 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.948 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-04-23T14:59:31.948 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 106, 2024-04-23T14:59:31.948 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 367, 2024-04-23T14:59:31.948 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 261, 2024-04-23T14:59:31.948 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 93, 2024-04-23T14:59:31.948 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 101, 2024-04-23T14:59:31.948 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 8, 2024-04-23T14:59:31.948 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 169, 2024-04-23T14:59:31.948 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 532, 2024-04-23T14:59:31.949 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 363, 2024-04-23T14:59:31.949 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 8, 2024-04-23T14:59:31.949 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 344, 2024-04-23T14:59:31.949 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 336, 2024-04-23T14:59:31.949 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 60060, 2024-04-23T14:59:31.949 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 231800 2024-04-23T14:59:31.949 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.949 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-04-23T14:59:31.949 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 1214, 2024-04-23T14:59:31.950 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 2, 2024-04-23T14:59:31.950 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 957, 2024-04-23T14:59:31.950 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 100, 2024-04-23T14:59:31.950 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 8, 2024-04-23T14:59:31.950 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-04-23T14:59:31.950 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 239, 2024-04-23T14:59:31.950 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.361524310, 2024-04-23T14:59:31.950 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001512654 2024-04-23T14:59:31.950 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.951 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-04-23T14:59:31.951 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 173, 2024-04-23T14:59:31.951 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.063975107, 2024-04-23T14:59:31.951 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000369798 2024-04-23T14:59:31.951 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.951 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-04-23T14:59:31.951 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.951 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.951 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.952 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.952 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-04-23T14:59:31.952 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 61, 2024-04-23T14:59:31.952 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.057151911, 2024-04-23T14:59:31.952 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000936916 2024-04-23T14:59:31.952 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.952 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-04-23T14:59:31.952 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 69, 2024-04-23T14:59:31.953 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.044813051, 2024-04-23T14:59:31.953 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000649464 2024-04-23T14:59:31.953 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.953 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-04-23T14:59:31.954 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.954 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.954 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.954 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.954 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-04-23T14:59:31.954 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1, 2024-04-23T14:59:31.954 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000130836, 2024-04-23T14:59:31.954 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000130836 2024-04-23T14:59:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-04-23T14:59:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-04-23T14:59:31.955 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-04-23T14:59:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-04-23T14:59:31.956 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 11, 2024-04-23T14:59:31.957 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.002412224, 2024-04-23T14:59:31.957 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000219293 2024-04-23T14:59:31.957 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.957 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-04-23T14:59:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 88, 2024-04-23T14:59:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.093673442, 2024-04-23T14:59:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001064470 2024-04-23T14:59:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-04-23T14:59:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.958 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.959 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-04-23T14:59:31.959 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 11, 2024-04-23T14:59:31.959 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.117810023, 2024-04-23T14:59:31.959 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.010710002 2024-04-23T14:59:31.959 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.959 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-04-23T14:59:31.959 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 24, 2024-04-23T14:59:31.959 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.005995581, 2024-04-23T14:59:31.959 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000249815 2024-04-23T14:59:31.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.960 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-04-23T14:59:31.960 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 18, 2024-04-23T14:59:31.960 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.369002759, 2024-04-23T14:59:31.960 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.020500153 2024-04-23T14:59:31.960 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.960 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-04-23T14:59:31.960 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 80, 2024-04-23T14:59:31.960 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 11.254640720, 2024-04-23T14:59:31.961 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.140683009 2024-04-23T14:59:31.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.961 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-04-23T14:59:31.961 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.961 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.961 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.961 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.961 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-04-23T14:59:31.961 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.961 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.962 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.962 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.962 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-04-23T14:59:31.962 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 4, 2024-04-23T14:59:31.962 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.054970303, 2024-04-23T14:59:31.962 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.013742575 2024-04-23T14:59:31.962 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.962 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-04-23T14:59:31.962 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 5, 2024-04-23T14:59:31.963 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.025087145, 2024-04-23T14:59:31.963 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.005017429 2024-04-23T14:59:31.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.963 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-04-23T14:59:31.963 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.963 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.963 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.963 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.963 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-04-23T14:59:31.964 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2024-04-23T14:59:31.964 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.004315747, 2024-04-23T14:59:31.964 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002157873 2024-04-23T14:59:31.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.964 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-04-23T14:59:31.964 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.964 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.964 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.964 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.964 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-04-23T14:59:31.965 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2024-04-23T14:59:31.965 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000633774, 2024-04-23T14:59:31.965 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000316887 2024-04-23T14:59:31.965 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.965 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-04-23T14:59:31.965 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.965 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.965 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.966 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-04-23T14:59:31.966 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 8, 2024-04-23T14:59:31.966 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.019934493, 2024-04-23T14:59:31.966 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002491811 2024-04-23T14:59:31.966 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.966 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-04-23T14:59:31.966 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3, 2024-04-23T14:59:31.966 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.002627361, 2024-04-23T14:59:31.966 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000875787 2024-04-23T14:59:31.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.967 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-04-23T14:59:31.967 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 142, 2024-04-23T14:59:31.967 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.186739779, 2024-04-23T14:59:31.967 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001315068 2024-04-23T14:59:31.967 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.968 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-04-23T14:59:31.968 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-04-23T14:59:31.968 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-04-23T14:59:31.968 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.968 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.968 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.968 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:31.968 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.968 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-04-23T14:59:31.969 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 2, 2024-04-23T14:59:31.969 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 6, 2024-04-23T14:59:31.969 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 4, 2024-04-23T14:59:31.969 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 2, 2024-04-23T14:59:31.969 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-04-23T14:59:31.969 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 29, 2024-04-23T14:59:31.969 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 14, 2024-04-23T14:59:31.969 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 645, 2024-04-23T14:59:31.969 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-04-23T14:59:31.970 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.970 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-23T14:59:31.970 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-23T14:59:31.970 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-23T14:59:31.970 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T14:59:31.970 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-23T14:59:31.970 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T14:59:31.970 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-23T14:59:31.970 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T14:59:31.971 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T14:59:31.971 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T14:59:31.971 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T14:59:31.971 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T14:59:31.971 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-23T14:59:31.971 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T14:59:31.971 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T14:59:31.971 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-23T14:59:31.971 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-23T14:59:31.971 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-23T14:59:31.972 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-23T14:59:31.972 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T14:59:31.972 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T14:59:31.972 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T14:59:31.972 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T14:59:31.972 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T14:59:31.972 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-23T14:59:31.972 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-23T14:59:31.972 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-23T14:59:31.973 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T14:59:31.973 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T14:59:31.973 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-23T14:59:31.973 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-23T14:59:31.973 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-23T14:59:31.973 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-23T14:59:31.973 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T14:59:31.973 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-23T14:59:31.973 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T14:59:31.974 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-23T14:59:31.974 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 325180, 2024-04-23T14:59:31.974 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 208, 2024-04-23T14:59:31.974 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 704, 2024-04-23T14:59:31.974 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 8, 2024-04-23T14:59:31.974 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-23T14:59:31.974 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-23T14:59:31.974 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-23T14:59:31.975 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-23T14:59:31.975 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-23T14:59:31.975 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-23T14:59:31.975 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 8128, 2024-04-23T14:59:31.975 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 169, 2024-04-23T14:59:31.975 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T14:59:31.975 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-23T14:59:31.975 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-23T14:59:31.975 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-23T14:59:31.976 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 553843, 2024-04-23T14:59:31.976 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 6654, 2024-04-23T14:59:31.976 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-23T14:59:31.976 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-23T14:59:31.976 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-23T14:59:31.976 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-23T14:59:31.976 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.976 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-04-23T14:59:31.976 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-04-23T14:59:31.977 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-04-23T14:59:31.977 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 1337, 2024-04-23T14:59:31.977 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 4067775, 2024-04-23T14:59:31.977 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 32, 2024-04-23T14:59:31.977 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 1337, 2024-04-23T14:59:31.977 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-04-23T14:59:31.977 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1337, 2024-04-23T14:59:31.977 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 27.823373304, 2024-04-23T14:59:31.977 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.020810301 2024-04-23T14:59:31.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.978 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-04-23T14:59:31.978 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-04-23T14:59:31.978 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1337, 2024-04-23T14:59:31.978 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 1391 2024-04-23T14:59:31.978 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.978 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 1337, 2024-04-23T14:59:31.978 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 332, 2024-04-23T14:59:31.978 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 1005, 2024-04-23T14:59:31.979 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-04-23T14:59:31.979 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-04-23T14:59:31.979 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 0, 2024-04-23T14:59:31.979 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 11, 2024-04-23T14:59:31.979 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 27, 2024-04-23T14:59:31.979 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 309, 2024-04-23T14:59:31.979 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 36, 2024-04-23T14:59:31.979 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-04-23T14:59:31.979 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-04-23T14:59:31.980 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-04-23T14:59:31.980 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-04-23T14:59:31.980 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 606, 2024-04-23T14:59:31.980 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-04-23T14:59:31.980 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-04-23T14:59:31.980 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-04-23T14:59:31.980 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 305, 2024-04-23T14:59:31.980 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 22, 2024-04-23T14:59:31.980 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-04-23T14:59:31.981 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-04-23T14:59:31.981 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-04-23T14:59:31.981 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-04-23T14:59:31.981 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-04-23T14:59:31.981 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-04-23T14:59:31.981 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T14:59:31.981 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-04-23T14:59:31.981 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-04-23T14:59:31.981 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 1, 2024-04-23T14:59:31.982 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-04-23T14:59:31.982 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-04-23T14:59:31.982 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-04-23T14:59:31.982 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-04-23T14:59:31.982 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-04-23T14:59:31.982 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-04-23T14:59:31.982 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-04-23T14:59:31.982 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-04-23T14:59:31.982 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-04-23T14:59:31.983 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-04-23T14:59:31.983 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-04-23T14:59:31.983 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-04-23T14:59:31.983 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-04-23T14:59:31.983 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-04-23T14:59:31.983 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-04-23T14:59:31.983 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-04-23T14:59:31.983 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 99, 2024-04-23T14:59:31.983 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-04-23T14:59:31.984 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 14, 2024-04-23T14:59:31.984 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 12, 2024-04-23T14:59:31.984 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 12, 2024-04-23T14:59:31.984 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2024-04-23T14:59:31.984 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-04-23T14:59:31.984 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 63, 2024-04-23T14:59:31.984 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 0, 2024-04-23T14:59:31.984 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 11 2024-04-23T14:59:31.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.985 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-04-23T14:59:31.985 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 1, 2024-04-23T14:59:31.985 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-04-23T14:59:31.985 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-04-23T14:59:31.985 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-04-23T14:59:31.985 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.985 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-04-23T14:59:31.985 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2024-04-23T14:59:31.986 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.986 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-04-23T14:59:31.986 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-04-23T14:59:31.986 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 10, 2024-04-23T14:59:31.986 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-04-23T14:59:31.986 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 10, 2024-04-23T14:59:31.986 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 94, 2024-04-23T14:59:31.986 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 94, 2024-04-23T14:59:31.986 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-04-23T14:59:31.987 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.987 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T14:59:31.987 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:31.987 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T14:59:31.987 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:31.987 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 7308, 2024-04-23T14:59:31.987 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 3422687, 2024-04-23T14:59:31.987 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:31.987 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 7308, 2024-04-23T14:59:31.988 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T14:59:31.988 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T14:59:31.988 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 7308, 2024-04-23T14:59:31.988 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 3422687, 2024-04-23T14:59:31.988 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:31.988 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.988 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.988 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.988 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:31.989 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.989 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-04-23T14:59:31.989 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:31.989 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T14:59:31.989 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:31.989 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T14:59:31.989 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T14:59:31.989 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:31.990 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:31.990 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 1337, 2024-04-23T14:59:31.990 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 4077269, 2024-04-23T14:59:31.990 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1337, 2024-04-23T14:59:31.990 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 4077269, 2024-04-23T14:59:31.990 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:31.990 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.990 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.990 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.991 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:31.991 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.991 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-04-23T14:59:31.991 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:31.991 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-04-23T14:59:31.991 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:31.991 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T14:59:31.991 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T14:59:31.992 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:31.992 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:31.992 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 1337, 2024-04-23T14:59:31.992 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 1337, 2024-04-23T14:59:31.992 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1337, 2024-04-23T14:59:31.992 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 1337, 2024-04-23T14:59:31.992 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:31.992 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.992 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.993 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.993 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:31.993 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.993 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-04-23T14:59:31.993 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:31.993 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-23T14:59:31.993 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:31.993 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 94, 2024-04-23T14:59:31.993 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 9494, 2024-04-23T14:59:31.994 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:31.994 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 94, 2024-04-23T14:59:31.994 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T14:59:31.994 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T14:59:31.994 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 9, 2024-04-23T14:59:31.994 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 9494, 2024-04-23T14:59:31.994 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:31.994 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.994 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.995 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.995 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:31.995 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:31.995 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x55f92e44e500": { 2024-04-23T14:59:31.995 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:31.995 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-23T14:59:31.995 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:31.995 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 1069, 2024-04-23T14:59:31.995 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 4004624, 2024-04-23T14:59:31.996 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:31.996 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 1069, 2024-04-23T14:59:31.996 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T14:59:31.996 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T14:59:31.996 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 300, 2024-04-23T14:59:31.996 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 4004624, 2024-04-23T14:59:31.996 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:31.996 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:31.996 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:31.997 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:31.997 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:31.997 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:31.997 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-23T14:59:32.789 INFO:tasks.check_counter:Daemon mds.b mds.exported=72 2024-04-23T14:59:32.790 INFO:tasks.check_counter:Daemon mds.b mds.imported=5 2024-04-23T14:59:32.790 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-23T14:59:32.790 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-23T14:59:32.790 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-23T14:59:32.790 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-04-23T14:59:32.790 DEBUG:tasks.check_counter:Getting stats from e 2024-04-23T14:59:32.790 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-23T14:59:33.074 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T14:59:33.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:33 smithi203 ceph-mon[32800]: pgmap v346: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:59:33.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:33 smithi083 ceph-mon[27867]: pgmap v346: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:59:33.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:33 smithi173 ceph-mon[32443]: pgmap v346: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:59:34.827 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-23T14:59:34.828 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T14:59:34.828 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 2347, 2024-04-23T14:59:34.828 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 2348, 2024-04-23T14:59:34.828 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 720068, 2024-04-23T14:59:34.828 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 746035, 2024-04-23T14:59:34.828 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2024-04-23T14:59:34.829 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 2, 2024-04-23T14:59:34.829 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.189951428, 2024-04-23T14:59:34.829 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.059530133, 2024-04-23T14:59:34.829 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.046631225, 2024-04-23T14:59:34.829 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.069165696, 2024-04-23T14:59:34.829 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:34.829 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2346, 2024-04-23T14:59:34.829 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.076650976, 2024-04-23T14:59:34.830 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000032673 2024-04-23T14:59:34.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.830 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:34.830 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.830 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.830 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.830 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T14:59:34.830 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T14:59:34.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.831 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T14:59:34.831 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 4, 2024-04-23T14:59:34.831 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 175, 2024-04-23T14:59:34.831 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 876, 2024-04-23T14:59:34.831 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 466186, 2024-04-23T14:59:34.831 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2024-04-23T14:59:34.831 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2024-04-23T14:59:34.832 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.009736717, 2024-04-23T14:59:34.832 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.006345561, 2024-04-23T14:59:34.832 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.001438674, 2024-04-23T14:59:34.832 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000053545, 2024-04-23T14:59:34.832 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:34.832 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 172, 2024-04-23T14:59:34.832 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.025622599, 2024-04-23T14:59:34.832 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000148968 2024-04-23T14:59:34.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.833 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:34.833 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.833 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.833 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.833 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T14:59:34.833 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 465488 2024-04-23T14:59:34.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.834 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T14:59:34.834 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 266, 2024-04-23T14:59:34.834 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 247, 2024-04-23T14:59:34.834 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 1644812, 2024-04-23T14:59:34.834 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 143310, 2024-04-23T14:59:34.834 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 3, 2024-04-23T14:59:34.834 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 2, 2024-04-23T14:59:34.834 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.046812909, 2024-04-23T14:59:34.835 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.014766138, 2024-04-23T14:59:34.835 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.035194266, 2024-04-23T14:59:34.835 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.005749079, 2024-04-23T14:59:34.835 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:34.835 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 247, 2024-04-23T14:59:34.835 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.013424534, 2024-04-23T14:59:34.835 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000054350 2024-04-23T14:59:34.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.835 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:34.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.836 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.836 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1644544, 2024-04-23T14:59:34.836 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 142992 2024-04-23T14:59:34.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.836 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-23T14:59:34.836 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-04-23T14:59:34.837 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-23T14:59:34.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.837 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-04-23T14:59:34.837 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-23T14:59:34.837 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-23T14:59:34.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3942, 2024-04-23T14:59:34.837 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.101597790, 2024-04-23T14:59:34.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000025773 2024-04-23T14:59:34.837 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:34.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.838 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-04-23T14:59:34.838 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-23T14:59:34.838 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-23T14:59:34.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.838 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.838 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:34.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.839 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-04-23T14:59:34.839 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 0, 2024-04-23T14:59:34.839 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 0, 2024-04-23T14:59:34.839 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-04-23T14:59:34.839 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.839 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.839 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.840 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-04-23T14:59:34.840 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 0, 2024-04-23T14:59:34.840 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-04-23T14:59:34.840 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-04-23T14:59:34.840 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 0, 2024-04-23T14:59:34.840 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-04-23T14:59:34.840 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-04-23T14:59:34.840 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 21, 2024-04-23T14:59:34.840 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 0, 2024-04-23T14:59:34.841 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 21, 2024-04-23T14:59:34.841 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 0, 2024-04-23T14:59:34.841 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 21, 2024-04-23T14:59:34.841 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 0, 2024-04-23T14:59:34.841 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2024-04-23T14:59:34.841 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2024-04-23T14:59:34.841 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 5, 2024-04-23T14:59:34.841 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 0, 2024-04-23T14:59:34.842 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 0, 2024-04-23T14:59:34.842 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 0, 2024-04-23T14:59:34.842 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 0, 2024-04-23T14:59:34.842 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-04-23T14:59:34.842 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-04-23T14:59:34.842 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-04-23T14:59:34.842 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2024-04-23T14:59:34.843 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-04-23T14:59:34.843 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 0, 2024-04-23T14:59:34.843 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 0, 2024-04-23T14:59:34.843 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 0, 2024-04-23T14:59:34.843 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 0, 2024-04-23T14:59:34.843 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-04-23T14:59:34.843 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2024-04-23T14:59:34.843 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2024-04-23T14:59:34.843 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 0, 2024-04-23T14:59:34.844 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 0, 2024-04-23T14:59:34.844 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 0, 2024-04-23T14:59:34.844 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T14:59:34.844 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-04-23T14:59:34.844 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T14:59:34.844 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-04-23T14:59:34.844 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-04-23T14:59:34.844 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T14:59:34.845 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T14:59:34.845 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2024-04-23T14:59:34.845 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-04-23T14:59:34.845 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T14:59:34.845 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 0, 2024-04-23T14:59:34.845 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T14:59:34.845 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T14:59:34.845 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T14:59:34.845 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 0, 2024-04-23T14:59:34.846 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 0, 2024-04-23T14:59:34.846 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 0, 2024-04-23T14:59:34.846 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 0 2024-04-23T14:59:34.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.846 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-04-23T14:59:34.846 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-04-23T14:59:34.846 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 0, 2024-04-23T14:59:34.846 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 0, 2024-04-23T14:59:34.847 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 0, 2024-04-23T14:59:34.847 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 0, 2024-04-23T14:59:34.847 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-04-23T14:59:34.847 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-04-23T14:59:34.847 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-04-23T14:59:34.847 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 0, 2024-04-23T14:59:34.847 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 0, 2024-04-23T14:59:34.847 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-04-23T14:59:34.847 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-04-23T14:59:34.848 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-04-23T14:59:34.848 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-04-23T14:59:34.848 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-04-23T14:59:34.848 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-04-23T14:59:34.848 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-04-23T14:59:34.848 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 0, 2024-04-23T14:59:34.848 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 0, 2024-04-23T14:59:34.848 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T14:59:34.849 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 0, 2024-04-23T14:59:34.849 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-04-23T14:59:34.849 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-04-23T14:59:34.849 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-04-23T14:59:34.849 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-04-23T14:59:34.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.849 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-04-23T14:59:34.849 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-04-23T14:59:34.849 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 0, 2024-04-23T14:59:34.850 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-04-23T14:59:34.850 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-04-23T14:59:34.850 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 26, 2024-04-23T14:59:34.850 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-04-23T14:59:34.850 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-04-23T14:59:34.850 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 0, 2024-04-23T14:59:34.850 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-04-23T14:59:34.850 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-04-23T14:59:34.851 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 2, 2024-04-23T14:59:34.851 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 2, 2024-04-23T14:59:34.851 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-04-23T14:59:34.851 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-04-23T14:59:34.851 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-04-23T14:59:34.851 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 4250983, 2024-04-23T14:59:34.851 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 4250409, 2024-04-23T14:59:34.851 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-04-23T14:59:34.851 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.852 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.852 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.852 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 26 2024-04-23T14:59:34.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.852 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-04-23T14:59:34.852 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 22, 2024-04-23T14:59:34.852 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 22, 2024-04-23T14:59:34.852 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 0, 2024-04-23T14:59:34.853 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 18, 2024-04-23T14:59:34.853 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 18, 2024-04-23T14:59:34.853 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 0, 2024-04-23T14:59:34.853 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 21, 2024-04-23T14:59:34.853 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 21, 2024-04-23T14:59:34.853 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 0, 2024-04-23T14:59:34.853 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2024-04-23T14:59:34.853 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 0, 2024-04-23T14:59:34.854 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 0, 2024-04-23T14:59:34.854 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 49868, 2024-04-23T14:59:34.854 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 215416 2024-04-23T14:59:34.854 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.854 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-04-23T14:59:34.854 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 0, 2024-04-23T14:59:34.854 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 0, 2024-04-23T14:59:34.854 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 0, 2024-04-23T14:59:34.854 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 0, 2024-04-23T14:59:34.855 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 0, 2024-04-23T14:59:34.855 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-04-23T14:59:34.855 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.855 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.855 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.855 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-04-23T14:59:34.855 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.856 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.856 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.856 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-04-23T14:59:34.856 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.856 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.856 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.856 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.856 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-04-23T14:59:34.857 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.857 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.857 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.857 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.857 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-04-23T14:59:34.857 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.857 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.857 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.858 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-04-23T14:59:34.858 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.858 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.858 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.858 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.858 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-04-23T14:59:34.858 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.858 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.859 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.859 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-04-23T14:59:34.859 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.859 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.859 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.859 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-04-23T14:59:34.860 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.860 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.860 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.860 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-04-23T14:59:34.860 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.860 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.860 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.861 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-04-23T14:59:34.861 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.861 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.861 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.861 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.861 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-04-23T14:59:34.861 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.862 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.862 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.862 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.862 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-04-23T14:59:34.863 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.863 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.863 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.863 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-04-23T14:59:34.863 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.863 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.863 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.864 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-04-23T14:59:34.864 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.864 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.864 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.864 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-04-23T14:59:34.864 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.864 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.864 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.865 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-04-23T14:59:34.865 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.865 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.865 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.865 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-04-23T14:59:34.865 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.865 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.866 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.866 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-04-23T14:59:34.866 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.866 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.866 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.866 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.866 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-04-23T14:59:34.867 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.867 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.867 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.867 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-04-23T14:59:34.867 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.867 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.867 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.868 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-04-23T14:59:34.868 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.868 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.868 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.868 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-04-23T14:59:34.868 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.868 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.869 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.869 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-04-23T14:59:34.869 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.869 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.869 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.869 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-04-23T14:59:34.869 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.870 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.870 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.870 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-04-23T14:59:34.870 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.870 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.870 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.870 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.871 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-04-23T14:59:34.871 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.871 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.871 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.871 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-04-23T14:59:34.871 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.871 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.871 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.872 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-04-23T14:59:34.872 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.872 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.872 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.872 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.872 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-04-23T14:59:34.872 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-04-23T14:59:34.872 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-04-23T14:59:34.873 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.873 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.873 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.873 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:34.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.873 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-04-23T14:59:34.873 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 2, 2024-04-23T14:59:34.873 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 5, 2024-04-23T14:59:34.873 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 3, 2024-04-23T14:59:34.874 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 4, 2024-04-23T14:59:34.874 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-04-23T14:59:34.874 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 0, 2024-04-23T14:59:34.874 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 0, 2024-04-23T14:59:34.874 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 5022, 2024-04-23T14:59:34.874 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-04-23T14:59:34.874 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.874 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-23T14:59:34.875 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-23T14:59:34.875 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-23T14:59:34.875 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T14:59:34.875 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-23T14:59:34.875 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T14:59:34.875 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-23T14:59:34.875 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T14:59:34.875 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T14:59:34.876 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T14:59:34.876 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T14:59:34.876 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T14:59:34.876 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-23T14:59:34.876 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T14:59:34.876 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T14:59:34.876 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-23T14:59:34.876 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-23T14:59:34.876 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-23T14:59:34.877 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-23T14:59:34.877 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T14:59:34.877 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T14:59:34.877 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T14:59:34.877 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T14:59:34.877 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T14:59:34.877 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-23T14:59:34.877 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-23T14:59:34.878 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-23T14:59:34.878 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T14:59:34.878 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T14:59:34.878 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-23T14:59:34.878 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-23T14:59:34.878 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-23T14:59:34.878 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-23T14:59:34.878 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T14:59:34.878 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-23T14:59:34.879 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T14:59:34.879 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-23T14:59:34.879 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 181920, 2024-04-23T14:59:34.879 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 64, 2024-04-23T14:59:34.879 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-23T14:59:34.879 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-23T14:59:34.879 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-23T14:59:34.879 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-23T14:59:34.879 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-23T14:59:34.880 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-23T14:59:34.880 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-23T14:59:34.880 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-23T14:59:34.880 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 8128, 2024-04-23T14:59:34.880 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 169, 2024-04-23T14:59:34.880 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T14:59:34.880 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-23T14:59:34.880 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-23T14:59:34.881 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-23T14:59:34.881 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 96284, 2024-04-23T14:59:34.881 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 729, 2024-04-23T14:59:34.881 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-23T14:59:34.881 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-23T14:59:34.881 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-23T14:59:34.881 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-23T14:59:34.881 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.881 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-04-23T14:59:34.882 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-04-23T14:59:34.882 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-04-23T14:59:34.882 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 2349, 2024-04-23T14:59:34.882 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 21, 2024-04-23T14:59:34.882 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 3, 2024-04-23T14:59:34.882 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 2349, 2024-04-23T14:59:34.882 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-04-23T14:59:34.882 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2349, 2024-04-23T14:59:34.883 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 7.654827929, 2024-04-23T14:59:34.883 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.003258760 2024-04-23T14:59:34.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.883 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-04-23T14:59:34.883 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-04-23T14:59:34.883 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2349, 2024-04-23T14:59:34.883 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 2350 2024-04-23T14:59:34.883 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.884 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 2349, 2024-04-23T14:59:34.884 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 2349, 2024-04-23T14:59:34.884 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 0, 2024-04-23T14:59:34.884 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-04-23T14:59:34.884 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-04-23T14:59:34.884 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 1527, 2024-04-23T14:59:34.884 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 0, 2024-04-23T14:59:34.884 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 820, 2024-04-23T14:59:34.884 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 0, 2024-04-23T14:59:34.885 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 0, 2024-04-23T14:59:34.885 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-04-23T14:59:34.885 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-04-23T14:59:34.885 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-04-23T14:59:34.885 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-04-23T14:59:34.885 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 0, 2024-04-23T14:59:34.885 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-04-23T14:59:34.885 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-04-23T14:59:34.885 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-04-23T14:59:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 1, 2024-04-23T14:59:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 0, 2024-04-23T14:59:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-04-23T14:59:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-04-23T14:59:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-04-23T14:59:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-04-23T14:59:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-04-23T14:59:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-04-23T14:59:34.886 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T14:59:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-04-23T14:59:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-04-23T14:59:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-04-23T14:59:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-04-23T14:59:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-04-23T14:59:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-04-23T14:59:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-04-23T14:59:34.887 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-04-23T14:59:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-04-23T14:59:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-04-23T14:59:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-04-23T14:59:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-04-23T14:59:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-04-23T14:59:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-04-23T14:59:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-04-23T14:59:34.888 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-04-23T14:59:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-04-23T14:59:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-04-23T14:59:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-04-23T14:59:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 99, 2024-04-23T14:59:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-04-23T14:59:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 13, 2024-04-23T14:59:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 4, 2024-04-23T14:59:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 4, 2024-04-23T14:59:34.889 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2024-04-23T14:59:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-04-23T14:59:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 0, 2024-04-23T14:59:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 2, 2024-04-23T14:59:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 0 2024-04-23T14:59:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-04-23T14:59:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 0, 2024-04-23T14:59:34.890 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-04-23T14:59:34.891 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-04-23T14:59:34.891 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-04-23T14:59:34.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.891 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-04-23T14:59:34.891 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2024-04-23T14:59:34.891 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.891 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-04-23T14:59:34.891 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-04-23T14:59:34.891 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T14:59:34.892 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-04-23T14:59:34.892 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 0, 2024-04-23T14:59:34.892 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 0, 2024-04-23T14:59:34.892 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 0, 2024-04-23T14:59:34.892 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-04-23T14:59:34.892 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.892 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T14:59:34.892 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:34.893 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T14:59:34.893 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:34.893 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 2613, 2024-04-23T14:59:34.893 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 2114193, 2024-04-23T14:59:34.893 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:34.893 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 2613, 2024-04-23T14:59:34.893 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T14:59:34.893 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T14:59:34.893 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2613, 2024-04-23T14:59:34.894 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 2114193, 2024-04-23T14:59:34.894 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:34.894 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.894 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.894 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.894 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:34.894 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.894 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-04-23T14:59:34.895 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:34.895 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T14:59:34.895 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:34.895 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T14:59:34.895 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T14:59:34.895 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:34.895 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:34.895 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 2349, 2024-04-23T14:59:34.895 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 56684, 2024-04-23T14:59:34.896 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2349, 2024-04-23T14:59:34.896 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 56684, 2024-04-23T14:59:34.896 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:34.896 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.896 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.896 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.896 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:34.896 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.897 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-04-23T14:59:34.897 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:34.897 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-04-23T14:59:34.897 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:34.897 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T14:59:34.897 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T14:59:34.897 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:34.897 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:34.897 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 2349, 2024-04-23T14:59:34.898 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 2349, 2024-04-23T14:59:34.898 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2349, 2024-04-23T14:59:34.898 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 2349, 2024-04-23T14:59:34.898 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:34.898 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.898 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.898 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.898 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:34.899 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.899 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-04-23T14:59:34.899 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:34.899 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-23T14:59:34.899 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:34.899 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T14:59:34.899 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T14:59:34.899 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:34.900 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:34.900 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T14:59:34.900 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T14:59:34.900 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-04-23T14:59:34.900 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-04-23T14:59:34.900 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:34.900 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.900 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.901 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.901 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:34.901 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:34.901 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x55ee35c6a6e0": { 2024-04-23T14:59:34.901 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:34.901 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-23T14:59:34.901 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:34.901 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T14:59:34.901 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T14:59:34.902 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:34.902 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:34.902 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T14:59:34.902 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T14:59:34.902 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-04-23T14:59:34.902 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-04-23T14:59:34.902 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:34.902 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:34.903 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:34.903 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:34.903 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:34.903 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:34.903 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-23T14:59:35.621 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-23T14:59:35.621 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-23T14:59:35.621 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-23T14:59:35.621 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-23T14:59:35.621 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T14:59:35.621 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-23T14:59:35.621 DEBUG:tasks.check_counter:Getting stats from h 2024-04-23T14:59:35.622 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-23T14:59:35.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:35 smithi203 ceph-mon[32800]: pgmap v347: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:35.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:35 smithi083 ceph-mon[27867]: pgmap v347: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:35.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:35 smithi173 ceph-mon[32443]: pgmap v347: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:35.868 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T14:59:37.582 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-23T14:59:37.582 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T14:59:37.582 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 665, 2024-04-23T14:59:37.582 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 783, 2024-04-23T14:59:37.582 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 212075, 2024-04-23T14:59:37.583 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 817522, 2024-04-23T14:59:37.583 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 39, 2024-04-23T14:59:37.583 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 6, 2024-04-23T14:59:37.583 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.095270956, 2024-04-23T14:59:37.583 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.037572805, 2024-04-23T14:59:37.583 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.049245033, 2024-04-23T14:59:37.583 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.005350013, 2024-04-23T14:59:37.583 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:37.584 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 781, 2024-04-23T14:59:37.584 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.049486926, 2024-04-23T14:59:37.584 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000063363 2024-04-23T14:59:37.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.584 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:37.584 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1298, 2024-04-23T14:59:37.584 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000283501, 2024-04-23T14:59:37.584 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000218 2024-04-23T14:59:37.584 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.585 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 11520, 2024-04-23T14:59:37.585 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 17456 2024-04-23T14:59:37.585 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.585 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T14:59:37.585 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 380, 2024-04-23T14:59:37.585 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 523, 2024-04-23T14:59:37.585 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 1675850, 2024-04-23T14:59:37.585 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 565461, 2024-04-23T14:59:37.585 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 13, 2024-04-23T14:59:37.585 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 5, 2024-04-23T14:59:37.586 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.065408869, 2024-04-23T14:59:37.586 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.023624557, 2024-04-23T14:59:37.586 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.045600101, 2024-04-23T14:59:37.586 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.006656150, 2024-04-23T14:59:37.586 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:37.586 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 523, 2024-04-23T14:59:37.586 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.053762570, 2024-04-23T14:59:37.586 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000102796 2024-04-23T14:59:37.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.587 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:37.587 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 73, 2024-04-23T14:59:37.587 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000014513, 2024-04-23T14:59:37.587 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000198 2024-04-23T14:59:37.587 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.587 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1668672, 2024-04-23T14:59:37.587 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 507920 2024-04-23T14:59:37.588 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.588 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T14:59:37.588 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 509, 2024-04-23T14:59:37.588 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 339, 2024-04-23T14:59:37.588 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 147235, 2024-04-23T14:59:37.588 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 379741, 2024-04-23T14:59:37.588 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 19, 2024-04-23T14:59:37.588 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 6, 2024-04-23T14:59:37.588 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.034197240, 2024-04-23T14:59:37.589 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.011475159, 2024-04-23T14:59:37.589 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.026272434, 2024-04-23T14:59:37.589 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.001234207, 2024-04-23T14:59:37.589 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:37.589 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 336, 2024-04-23T14:59:37.589 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.019618884, 2024-04-23T14:59:37.589 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000058389 2024-04-23T14:59:37.589 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.589 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:37.590 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 699, 2024-04-23T14:59:37.590 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000133129, 2024-04-23T14:59:37.590 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000190 2024-04-23T14:59:37.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.590 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-04-23T14:59:37.590 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 217856 2024-04-23T14:59:37.590 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.590 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-23T14:59:37.590 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2024-04-23T14:59:37.591 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-23T14:59:37.591 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.591 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2024-04-23T14:59:37.591 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-23T14:59:37.591 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-23T14:59:37.591 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 120, 2024-04-23T14:59:37.591 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.006994141, 2024-04-23T14:59:37.591 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000058284 2024-04-23T14:59:37.592 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:37.592 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.592 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2024-04-23T14:59:37.592 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2024-04-23T14:59:37.592 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2024-04-23T14:59:37.592 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2024-04-23T14:59:37.592 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000008456, 2024-04-23T14:59:37.592 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000004228 2024-04-23T14:59:37.593 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:37.593 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.593 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2024-04-23T14:59:37.593 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 24, 2024-04-23T14:59:37.593 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 8, 2024-04-23T14:59:37.593 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2024-04-23T14:59:37.593 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 8, 2024-04-23T14:59:37.593 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.002060018, 2024-04-23T14:59:37.593 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000257502 2024-04-23T14:59:37.594 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.594 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2024-04-23T14:59:37.594 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 16, 2024-04-23T14:59:37.594 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2024-04-23T14:59:37.594 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2024-04-23T14:59:37.594 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 11, 2024-04-23T14:59:37.594 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2024-04-23T14:59:37.594 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2024-04-23T14:59:37.594 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 31, 2024-04-23T14:59:37.595 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 0, 2024-04-23T14:59:37.595 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 31, 2024-04-23T14:59:37.595 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 0, 2024-04-23T14:59:37.595 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 31, 2024-04-23T14:59:37.595 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 0, 2024-04-23T14:59:37.595 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 7, 2024-04-23T14:59:37.595 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 8, 2024-04-23T14:59:37.595 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 6, 2024-04-23T14:59:37.595 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 28, 2024-04-23T14:59:37.596 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 11, 2024-04-23T14:59:37.596 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 10, 2024-04-23T14:59:37.596 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 4, 2024-04-23T14:59:37.596 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2024-04-23T14:59:37.596 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2024-04-23T14:59:37.596 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2024-04-23T14:59:37.596 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2024-04-23T14:59:37.596 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2024-04-23T14:59:37.596 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 0, 2024-04-23T14:59:37.597 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 0, 2024-04-23T14:59:37.597 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 3, 2024-04-23T14:59:37.597 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 5, 2024-04-23T14:59:37.597 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2024-04-23T14:59:37.597 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 45, 2024-04-23T14:59:37.597 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 1, 2024-04-23T14:59:37.597 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 325, 2024-04-23T14:59:37.597 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 13424230511, 2024-04-23T14:59:37.597 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 6, 2024-04-23T14:59:37.598 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 45, 2024-04-23T14:59:37.598 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2024-04-23T14:59:37.598 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T14:59:37.598 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2024-04-23T14:59:37.598 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2024-04-23T14:59:37.598 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T14:59:37.598 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T14:59:37.598 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 45, 2024-04-23T14:59:37.599 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2024-04-23T14:59:37.599 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T14:59:37.599 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 0, 2024-04-23T14:59:37.599 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T14:59:37.599 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T14:59:37.599 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T14:59:37.599 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 8, 2024-04-23T14:59:37.599 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 2, 2024-04-23T14:59:37.599 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 0, 2024-04-23T14:59:37.600 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 2 2024-04-23T14:59:37.600 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.600 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2024-04-23T14:59:37.600 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2024-04-23T14:59:37.600 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 0, 2024-04-23T14:59:37.600 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 0, 2024-04-23T14:59:37.600 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 5, 2024-04-23T14:59:37.600 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 3, 2024-04-23T14:59:37.601 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2024-04-23T14:59:37.601 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2024-04-23T14:59:37.601 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2024-04-23T14:59:37.601 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 0, 2024-04-23T14:59:37.601 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 0, 2024-04-23T14:59:37.601 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2024-04-23T14:59:37.601 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2024-04-23T14:59:37.601 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2024-04-23T14:59:37.601 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2024-04-23T14:59:37.602 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2024-04-23T14:59:37.602 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2024-04-23T14:59:37.602 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2024-04-23T14:59:37.602 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_path": 0, 2024-04-23T14:59:37.602 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_quiesce_inode": 0, 2024-04-23T14:59:37.602 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T14:59:37.602 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 0, 2024-04-23T14:59:37.602 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2024-04-23T14:59:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2024-04-23T14:59:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2024-04-23T14:59:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2024-04-23T14:59:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2024-04-23T14:59:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2024-04-23T14:59:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 26, 2024-04-23T14:59:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2024-04-23T14:59:37.603 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2024-04-23T14:59:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 26, 2024-04-23T14:59:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2024-04-23T14:59:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2024-04-23T14:59:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 2, 2024-04-23T14:59:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2024-04-23T14:59:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2024-04-23T14:59:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 2, 2024-04-23T14:59:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 2, 2024-04-23T14:59:37.604 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2024-04-23T14:59:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2024-04-23T14:59:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2024-04-23T14:59:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 4250983, 2024-04-23T14:59:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 0, 2024-04-23T14:59:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2024-04-23T14:59:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 17, 2024-04-23T14:59:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.060286856, 2024-04-23T14:59:37.605 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.003546285 2024-04-23T14:59:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 0 2024-04-23T14:59:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2024-04-23T14:59:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 32, 2024-04-23T14:59:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 32, 2024-04-23T14:59:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 0, 2024-04-23T14:59:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 28, 2024-04-23T14:59:37.606 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 29, 2024-04-23T14:59:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 1, 2024-04-23T14:59:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 31, 2024-04-23T14:59:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 31, 2024-04-23T14:59:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 0, 2024-04-23T14:59:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 8, 2024-04-23T14:59:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 9, 2024-04-23T14:59:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 1, 2024-04-23T14:59:37.607 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 53596, 2024-04-23T14:59:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 223608 2024-04-23T14:59:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2024-04-23T14:59:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 25, 2024-04-23T14:59:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 0, 2024-04-23T14:59:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 24, 2024-04-23T14:59:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 30, 2024-04-23T14:59:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 0, 2024-04-23T14:59:37.608 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2024-04-23T14:59:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2024-04-23T14:59:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 4, 2024-04-23T14:59:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000675234, 2024-04-23T14:59:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000168808 2024-04-23T14:59:37.609 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2024-04-23T14:59:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2024-04-23T14:59:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.610 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2024-04-23T14:59:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.611 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2024-04-23T14:59:37.612 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.612 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.612 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.612 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2024-04-23T14:59:37.612 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1, 2024-04-23T14:59:37.612 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000104210, 2024-04-23T14:59:37.612 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000104210 2024-04-23T14:59:37.612 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.612 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2024-04-23T14:59:37.613 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.613 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.613 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.613 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2024-04-23T14:59:37.613 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.613 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.613 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.613 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.614 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2024-04-23T14:59:37.614 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.614 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.614 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.614 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.614 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2024-04-23T14:59:37.614 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.614 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.615 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.615 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.615 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2024-04-23T14:59:37.616 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.616 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.616 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.616 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.616 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2024-04-23T14:59:37.616 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.616 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.617 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.617 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2024-04-23T14:59:37.617 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.617 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.617 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.617 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.617 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2024-04-23T14:59:37.618 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.618 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.618 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.618 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.618 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2024-04-23T14:59:37.618 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3, 2024-04-23T14:59:37.618 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.001280574, 2024-04-23T14:59:37.618 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000426858 2024-04-23T14:59:37.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.619 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2024-04-23T14:59:37.619 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.619 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.619 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.619 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.619 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2024-04-23T14:59:37.619 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.620 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.620 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.620 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2024-04-23T14:59:37.620 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.620 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.620 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.620 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.620 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2024-04-23T14:59:37.621 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.621 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.621 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.621 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.621 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2024-04-23T14:59:37.621 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.621 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.621 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.622 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2024-04-23T14:59:37.622 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.622 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.622 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.622 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.622 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2024-04-23T14:59:37.622 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.623 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.623 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.623 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2024-04-23T14:59:37.623 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.623 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.623 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.623 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.623 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2024-04-23T14:59:37.624 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.624 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.624 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.624 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2024-04-23T14:59:37.624 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.624 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.624 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.624 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.625 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2024-04-23T14:59:37.625 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.625 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.625 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.625 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.625 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2024-04-23T14:59:37.625 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.625 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.625 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.626 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2024-04-23T14:59:37.626 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.626 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.626 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.626 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.626 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2024-04-23T14:59:37.626 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2024-04-23T14:59:37.626 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2024-04-23T14:59:37.627 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.627 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.627 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.627 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:37.627 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.627 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2024-04-23T14:59:37.627 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 2, 2024-04-23T14:59:37.627 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 5, 2024-04-23T14:59:37.628 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 3, 2024-04-23T14:59:37.628 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 2, 2024-04-23T14:59:37.628 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2024-04-23T14:59:37.628 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 4, 2024-04-23T14:59:37.628 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 2, 2024-04-23T14:59:37.628 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 644, 2024-04-23T14:59:37.628 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2024-04-23T14:59:37.628 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.629 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-23T14:59:37.629 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-23T14:59:37.629 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-23T14:59:37.629 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T14:59:37.629 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-23T14:59:37.629 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T14:59:37.630 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-23T14:59:37.630 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T14:59:37.630 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T14:59:37.630 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T14:59:37.630 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T14:59:37.630 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T14:59:37.630 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-23T14:59:37.630 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T14:59:37.631 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T14:59:37.631 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-23T14:59:37.631 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-23T14:59:37.631 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-23T14:59:37.631 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-23T14:59:37.631 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T14:59:37.631 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T14:59:37.631 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T14:59:37.632 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T14:59:37.632 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T14:59:37.632 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-23T14:59:37.632 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-23T14:59:37.632 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-23T14:59:37.632 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T14:59:37.632 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T14:59:37.632 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-23T14:59:37.632 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-23T14:59:37.633 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-23T14:59:37.633 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-23T14:59:37.633 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T14:59:37.633 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-23T14:59:37.633 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T14:59:37.633 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-23T14:59:37.633 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 292476, 2024-04-23T14:59:37.634 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 141, 2024-04-23T14:59:37.634 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-23T14:59:37.634 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-23T14:59:37.634 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-23T14:59:37.634 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-23T14:59:37.634 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-23T14:59:37.634 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-23T14:59:37.634 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-23T14:59:37.635 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-23T14:59:37.635 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 8128, 2024-04-23T14:59:37.635 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 169, 2024-04-23T14:59:37.635 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T14:59:37.635 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-23T14:59:37.635 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-23T14:59:37.635 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-23T14:59:37.635 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 142476, 2024-04-23T14:59:37.635 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 1011, 2024-04-23T14:59:37.636 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-23T14:59:37.636 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-23T14:59:37.636 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-23T14:59:37.636 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-23T14:59:37.636 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.636 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2024-04-23T14:59:37.636 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2024-04-23T14:59:37.636 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2024-04-23T14:59:37.637 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 105, 2024-04-23T14:59:37.637 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 69223, 2024-04-23T14:59:37.637 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 33, 2024-04-23T14:59:37.637 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 105, 2024-04-23T14:59:37.637 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2024-04-23T14:59:37.637 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 105, 2024-04-23T14:59:37.637 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 1.127974609, 2024-04-23T14:59:37.637 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.010742615 2024-04-23T14:59:37.637 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.637 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2024-04-23T14:59:37.638 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2024-04-23T14:59:37.638 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 105, 2024-04-23T14:59:37.638 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 130 2024-04-23T14:59:37.638 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.638 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 105, 2024-04-23T14:59:37.638 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 45, 2024-04-23T14:59:37.638 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 60, 2024-04-23T14:59:37.638 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2024-04-23T14:59:37.638 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2024-04-23T14:59:37.639 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 0, 2024-04-23T14:59:37.639 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 10, 2024-04-23T14:59:37.639 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 0, 2024-04-23T14:59:37.639 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 17, 2024-04-23T14:59:37.639 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 11, 2024-04-23T14:59:37.639 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2024-04-23T14:59:37.639 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2024-04-23T14:59:37.639 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2024-04-23T14:59:37.640 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2024-04-23T14:59:37.640 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 6, 2024-04-23T14:59:37.640 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2024-04-23T14:59:37.640 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2024-04-23T14:59:37.640 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2024-04-23T14:59:37.640 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 45, 2024-04-23T14:59:37.640 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 20, 2024-04-23T14:59:37.640 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2024-04-23T14:59:37.640 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2024-04-23T14:59:37.641 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2024-04-23T14:59:37.641 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2024-04-23T14:59:37.641 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2024-04-23T14:59:37.641 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2024-04-23T14:59:37.641 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T14:59:37.641 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2024-04-23T14:59:37.641 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2024-04-23T14:59:37.641 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2024-04-23T14:59:37.641 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2024-04-23T14:59:37.641 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2024-04-23T14:59:37.642 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2024-04-23T14:59:37.642 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2024-04-23T14:59:37.642 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2024-04-23T14:59:37.642 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2024-04-23T14:59:37.642 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2024-04-23T14:59:37.642 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2024-04-23T14:59:37.642 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2024-04-23T14:59:37.642 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2024-04-23T14:59:37.643 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2024-04-23T14:59:37.643 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2024-04-23T14:59:37.643 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2024-04-23T14:59:37.643 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2024-04-23T14:59:37.643 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2024-04-23T14:59:37.643 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2024-04-23T14:59:37.643 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 99, 2024-04-23T14:59:37.643 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2024-04-23T14:59:37.643 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 13, 2024-04-23T14:59:37.644 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 11, 2024-04-23T14:59:37.644 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 11, 2024-04-23T14:59:37.644 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2024-04-23T14:59:37.644 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2024-04-23T14:59:37.644 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 19, 2024-04-23T14:59:37.644 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 0, 2024-04-23T14:59:37.644 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 2 2024-04-23T14:59:37.644 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.644 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2024-04-23T14:59:37.644 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 1, 2024-04-23T14:59:37.645 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2024-04-23T14:59:37.645 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2024-04-23T14:59:37.645 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2024-04-23T14:59:37.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.645 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2024-04-23T14:59:37.645 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2024-04-23T14:59:37.645 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.645 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2024-04-23T14:59:37.645 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2024-04-23T14:59:37.646 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T14:59:37.646 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2024-04-23T14:59:37.646 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 0, 2024-04-23T14:59:37.646 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed_ops": 0, 2024-04-23T14:59:37.646 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 0, 2024-04-23T14:59:37.646 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2024-04-23T14:59:37.646 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.646 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T14:59:37.647 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:37.647 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T14:59:37.647 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:37.647 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 1550, 2024-04-23T14:59:37.647 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 1873757, 2024-04-23T14:59:37.647 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:37.647 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 1550, 2024-04-23T14:59:37.647 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T14:59:37.647 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T14:59:37.648 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 1550, 2024-04-23T14:59:37.648 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 1873757, 2024-04-23T14:59:37.648 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:37.648 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.648 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.648 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.648 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:37.648 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.648 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2024-04-23T14:59:37.649 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:37.649 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T14:59:37.649 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:37.649 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T14:59:37.649 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T14:59:37.649 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:37.649 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:37.649 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 105, 2024-04-23T14:59:37.649 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 69223, 2024-04-23T14:59:37.650 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 105, 2024-04-23T14:59:37.650 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 69223, 2024-04-23T14:59:37.650 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:37.650 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.650 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.650 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.650 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:37.650 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.651 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2024-04-23T14:59:37.651 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:37.651 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2024-04-23T14:59:37.651 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:37.651 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T14:59:37.651 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T14:59:37.651 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:37.651 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:37.651 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 105, 2024-04-23T14:59:37.652 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 105, 2024-04-23T14:59:37.652 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 105, 2024-04-23T14:59:37.652 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 105, 2024-04-23T14:59:37.652 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:37.652 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.652 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.652 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.652 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:37.653 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.653 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2024-04-23T14:59:37.653 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:37.653 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-23T14:59:37.653 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:37.653 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2024-04-23T14:59:37.653 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2024-04-23T14:59:37.653 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:37.653 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:37.654 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T14:59:37.654 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T14:59:37.654 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2024-04-23T14:59:37.654 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2024-04-23T14:59:37.654 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:37.654 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.654 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.654 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.654 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:37.655 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:37.655 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x55cc8e4f4500": { 2024-04-23T14:59:37.655 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:37.655 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2024-04-23T14:59:37.655 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:37.655 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 26, 2024-04-23T14:59:37.655 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 56679, 2024-04-23T14:59:37.655 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:37.655 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 26, 2024-04-23T14:59:37.655 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T14:59:37.656 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T14:59:37.656 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 17, 2024-04-23T14:59:37.656 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 56679, 2024-04-23T14:59:37.656 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:37.656 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:37.656 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:37.656 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:37.656 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:37.656 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:37.656 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-23T14:59:37.657 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:37 smithi173 ceph-mon[32443]: pgmap v348: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:59:37.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:37 smithi203 ceph-mon[32800]: pgmap v348: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:59:37.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:37 smithi083 ceph-mon[27867]: pgmap v348: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:59:38.394 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-23T14:59:38.394 INFO:tasks.check_counter:Daemon mds.h mds.imported=3 2024-04-23T14:59:38.394 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-23T14:59:38.394 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-23T14:59:38.394 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T14:59:38.394 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-23T14:59:38.395 DEBUG:tasks.check_counter:Getting stats from k 2024-04-23T14:59:38.395 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-23T14:59:38.637 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T14:59:39.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:39 smithi203 ceph-mon[32800]: pgmap v349: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:39.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:39 smithi083 ceph-mon[27867]: pgmap v349: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:39.763 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:39 smithi173 ceph-mon[32443]: pgmap v349: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:40.336 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2024-04-23T14:59:40.337 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T14:59:40.337 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 0, 2024-04-23T14:59:40.337 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 2, 2024-04-23T14:59:40.337 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 0, 2024-04-23T14:59:40.337 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 0, 2024-04-23T14:59:40.337 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 4, 2024-04-23T14:59:40.337 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 0, 2024-04-23T14:59:40.337 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.001067606, 2024-04-23T14:59:40.337 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.000000000, 2024-04-23T14:59:40.338 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.000271604, 2024-04-23T14:59:40.338 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T14:59:40.338 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:40.338 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:40.338 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:40.338 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:40.338 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:40.338 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:40.339 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:40.339 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:40.339 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:40.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:40.339 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T14:59:40.339 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T14:59:40.339 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:40.339 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T14:59:40.339 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 241, 2024-04-23T14:59:40.340 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 230, 2024-04-23T14:59:40.340 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 1595072, 2024-04-23T14:59:40.340 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 141984, 2024-04-23T14:59:40.340 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 2, 2024-04-23T14:59:40.340 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 1, 2024-04-23T14:59:40.340 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.046251070, 2024-04-23T14:59:40.340 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.014416955, 2024-04-23T14:59:40.340 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.033799609, 2024-04-23T14:59:40.340 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.005976491, 2024-04-23T14:59:40.341 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:40.341 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 229, 2024-04-23T14:59:40.341 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.012527047, 2024-04-23T14:59:40.341 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000054703 2024-04-23T14:59:40.341 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:40.341 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:40.341 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:40.341 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:40.341 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:40.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:40.342 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1595072, 2024-04-23T14:59:40.342 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 141984 2024-04-23T14:59:40.342 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:40.342 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T14:59:40.342 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 6, 2024-04-23T14:59:40.342 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 178, 2024-04-23T14:59:40.342 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 3344, 2024-04-23T14:59:40.343 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 187728, 2024-04-23T14:59:40.343 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 5, 2024-04-23T14:59:40.343 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 3, 2024-04-23T14:59:40.343 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.011891228, 2024-04-23T14:59:40.343 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.007857409, 2024-04-23T14:59:40.343 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.001631436, 2024-04-23T14:59:40.343 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T14:59:40.343 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:40.343 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 176, 2024-04-23T14:59:40.344 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.034276413, 2024-04-23T14:59:40.344 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000194752 2024-04-23T14:59:40.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:40.344 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:40.344 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:40.344 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:40.344 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:40.344 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:40.345 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-23T14:59:40.345 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 187728 2024-04-23T14:59:40.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:40.345 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2024-04-23T14:59:40.345 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 0, 2024-04-23T14:59:40.345 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2024-04-23T14:59:40.345 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:40.345 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2024-04-23T14:59:40.345 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2024-04-23T14:59:40.346 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2024-04-23T14:59:40.346 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T14:59:40.346 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2024-04-23T14:59:40.346 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T14:59:40.346 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2024-04-23T14:59:40.346 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T14:59:40.346 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T14:59:40.347 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T14:59:40.347 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T14:59:40.347 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T14:59:40.347 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2024-04-23T14:59:40.347 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T14:59:40.347 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T14:59:40.347 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2024-04-23T14:59:40.347 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2024-04-23T14:59:40.347 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2024-04-23T14:59:40.348 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2024-04-23T14:59:40.348 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T14:59:40.348 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T14:59:40.348 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T14:59:40.348 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T14:59:40.348 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T14:59:40.348 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2024-04-23T14:59:40.348 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2024-04-23T14:59:40.348 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2024-04-23T14:59:40.349 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T14:59:40.349 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T14:59:40.349 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2024-04-23T14:59:40.349 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2024-04-23T14:59:40.349 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2024-04-23T14:59:40.349 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2024-04-23T14:59:40.349 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T14:59:40.349 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2024-04-23T14:59:40.349 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T14:59:40.350 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2024-04-23T14:59:40.350 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 148794, 2024-04-23T14:59:40.350 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 32, 2024-04-23T14:59:40.350 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2024-04-23T14:59:40.350 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2024-04-23T14:59:40.350 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2024-04-23T14:59:40.350 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2024-04-23T14:59:40.350 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2024-04-23T14:59:40.351 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2024-04-23T14:59:40.351 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2024-04-23T14:59:40.351 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2024-04-23T14:59:40.351 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 0, 2024-04-23T14:59:40.351 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 0, 2024-04-23T14:59:40.351 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T14:59:40.351 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2024-04-23T14:59:40.351 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2024-04-23T14:59:40.352 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2024-04-23T14:59:40.352 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 928, 2024-04-23T14:59:40.352 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 2, 2024-04-23T14:59:40.352 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2024-04-23T14:59:40.352 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2024-04-23T14:59:40.352 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2024-04-23T14:59:40.352 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2024-04-23T14:59:40.352 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2024-04-23T14:59:40.352 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T14:59:40.353 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2024-04-23T14:59:40.353 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2024-04-23T14:59:40.353 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2024-04-23T14:59:40.353 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 243, 2024-04-23T14:59:40.353 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 1561069, 2024-04-23T14:59:40.353 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:40.353 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 243, 2024-04-23T14:59:40.353 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2024-04-23T14:59:40.353 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2024-04-23T14:59:40.354 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 243, 2024-04-23T14:59:40.354 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 1561069, 2024-04-23T14:59:40.354 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2024-04-23T14:59:40.354 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2024-04-23T14:59:40.354 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2024-04-23T14:59:40.354 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2024-04-23T14:59:40.354 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:40.354 INFO:teuthology.orchestra.run.smithi173.stdout: } 2024-04-23T14:59:40.354 INFO:teuthology.orchestra.run.smithi173.stdout:} 2024-04-23T14:59:41.060 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-23T14:59:41.060 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-23T14:59:41.060 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-23T14:59:41.060 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-23T14:59:41.060 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-23T14:59:41.061 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-23T14:59:41.061 DEBUG:tasks.check_counter:Getting stats from c 2024-04-23T14:59:41.061 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-23T14:59:41.318 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:59:41.686 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:41 smithi203 ceph-mon[32800]: pgmap v350: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:59:41.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:41 smithi083 ceph-mon[27867]: pgmap v350: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:59:41.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:41 smithi173 ceph-mon[32443]: pgmap v350: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:59:43.398 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-04-23T14:59:43.399 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T14:59:43.399 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 1966, 2024-04-23T14:59:43.399 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 1949, 2024-04-23T14:59:43.399 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 6151616, 2024-04-23T14:59:43.399 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 684580, 2024-04-23T14:59:43.400 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 5, 2024-04-23T14:59:43.400 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 1, 2024-04-23T14:59:43.400 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.209945120, 2024-04-23T14:59:43.400 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.065064658, 2024-04-23T14:59:43.400 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.080131902, 2024-04-23T14:59:43.400 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.059451061, 2024-04-23T14:59:43.401 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:43.401 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1948, 2024-04-23T14:59:43.401 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.088419397, 2024-04-23T14:59:43.401 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000045389 2024-04-23T14:59:43.401 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.401 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:43.401 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.401 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.402 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.402 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.402 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 1641680, 2024-04-23T14:59:43.402 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 142704 2024-04-23T14:59:43.402 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.402 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T14:59:43.402 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 6, 2024-04-23T14:59:43.402 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 178, 2024-04-23T14:59:43.403 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 1976, 2024-04-23T14:59:43.403 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 547034, 2024-04-23T14:59:43.403 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 8, 2024-04-23T14:59:43.403 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2024-04-23T14:59:43.403 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.010916229, 2024-04-23T14:59:43.403 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.006698609, 2024-04-23T14:59:43.403 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.002137552, 2024-04-23T14:59:43.403 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000111074, 2024-04-23T14:59:43.404 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:43.404 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 175, 2024-04-23T14:59:43.404 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.035738824, 2024-04-23T14:59:43.404 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000204221 2024-04-23T14:59:43.404 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.404 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:43.404 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.404 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.404 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.404 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.405 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T14:59:43.405 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 545696 2024-04-23T14:59:43.405 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.405 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T14:59:43.405 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 796, 2024-04-23T14:59:43.405 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 795, 2024-04-23T14:59:43.405 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 198142, 2024-04-23T14:59:43.405 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 251993, 2024-04-23T14:59:43.405 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2024-04-23T14:59:43.406 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 3, 2024-04-23T14:59:43.406 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.076297707, 2024-04-23T14:59:43.406 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.026626754, 2024-04-23T14:59:43.406 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.016629316, 2024-04-23T14:59:43.406 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.026229791, 2024-04-23T14:59:43.406 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:43.406 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 794, 2024-04-23T14:59:43.406 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.034147365, 2024-04-23T14:59:43.406 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000043006 2024-04-23T14:59:43.407 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.407 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:43.407 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.407 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.407 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.407 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.407 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-23T14:59:43.407 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T14:59:43.407 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.408 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-04-23T14:59:43.408 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2024-04-23T14:59:43.408 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-04-23T14:59:43.408 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.408 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2024-04-23T14:59:43.408 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-04-23T14:59:43.408 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-04-23T14:59:43.408 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 4143, 2024-04-23T14:59:43.408 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.118943348, 2024-04-23T14:59:43.409 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000028709 2024-04-23T14:59:43.409 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:43.409 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.409 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2024-04-23T14:59:43.409 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-04-23T14:59:43.409 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-04-23T14:59:43.409 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.409 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.409 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.410 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:43.410 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.410 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2024-04-23T14:59:43.410 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 0, 2024-04-23T14:59:43.410 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 0, 2024-04-23T14:59:43.410 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2024-04-23T14:59:43.410 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.410 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.411 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.411 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.411 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2024-04-23T14:59:43.411 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 0, 2024-04-23T14:59:43.411 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2024-04-23T14:59:43.411 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2024-04-23T14:59:43.411 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 0, 2024-04-23T14:59:43.412 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2024-04-23T14:59:43.412 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2024-04-23T14:59:43.412 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 184, 2024-04-23T14:59:43.412 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 0, 2024-04-23T14:59:43.412 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 184, 2024-04-23T14:59:43.412 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 0, 2024-04-23T14:59:43.412 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 184, 2024-04-23T14:59:43.412 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 0, 2024-04-23T14:59:43.413 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 0, 2024-04-23T14:59:43.413 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 0, 2024-04-23T14:59:43.413 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 5, 2024-04-23T14:59:43.413 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 0, 2024-04-23T14:59:43.413 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 0, 2024-04-23T14:59:43.413 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 0, 2024-04-23T14:59:43.413 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 0, 2024-04-23T14:59:43.413 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2024-04-23T14:59:43.414 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2024-04-23T14:59:43.414 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2024-04-23T14:59:43.414 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2024-04-23T14:59:43.414 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2024-04-23T14:59:43.414 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 0, 2024-04-23T14:59:43.414 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 0, 2024-04-23T14:59:43.414 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 0, 2024-04-23T14:59:43.414 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 0, 2024-04-23T14:59:43.414 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2024-04-23T14:59:43.415 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 0, 2024-04-23T14:59:43.415 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2024-04-23T14:59:43.415 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 325, 2024-04-23T14:59:43.415 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 13424230511, 2024-04-23T14:59:43.415 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 6, 2024-04-23T14:59:43.415 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T14:59:43.415 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2024-04-23T14:59:43.415 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T14:59:43.415 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2024-04-23T14:59:43.415 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2024-04-23T14:59:43.416 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T14:59:43.416 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T14:59:43.416 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 0, 2024-04-23T14:59:43.416 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 0, 2024-04-23T14:59:43.416 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T14:59:43.416 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 0, 2024-04-23T14:59:43.416 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T14:59:43.416 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T14:59:43.417 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T14:59:43.417 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 0, 2024-04-23T14:59:43.417 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 0, 2024-04-23T14:59:43.417 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 0, 2024-04-23T14:59:43.417 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 0 2024-04-23T14:59:43.417 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.417 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2024-04-23T14:59:43.417 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 0, 2024-04-23T14:59:43.417 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 0, 2024-04-23T14:59:43.418 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 0, 2024-04-23T14:59:43.418 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 0, 2024-04-23T14:59:43.418 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 0, 2024-04-23T14:59:43.418 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2024-04-23T14:59:43.418 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2024-04-23T14:59:43.418 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2024-04-23T14:59:43.418 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 0, 2024-04-23T14:59:43.418 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 0, 2024-04-23T14:59:43.418 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2024-04-23T14:59:43.419 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2024-04-23T14:59:43.419 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2024-04-23T14:59:43.419 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2024-04-23T14:59:43.419 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2024-04-23T14:59:43.419 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2024-04-23T14:59:43.419 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2024-04-23T14:59:43.419 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_path": 0, 2024-04-23T14:59:43.419 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_inode": 0, 2024-04-23T14:59:43.420 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T14:59:43.420 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 0, 2024-04-23T14:59:43.420 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2024-04-23T14:59:43.420 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2024-04-23T14:59:43.420 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2024-04-23T14:59:43.420 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2024-04-23T14:59:43.420 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.420 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2024-04-23T14:59:43.420 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2024-04-23T14:59:43.421 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 0, 2024-04-23T14:59:43.421 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2024-04-23T14:59:43.421 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2024-04-23T14:59:43.421 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 1069, 2024-04-23T14:59:43.421 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2024-04-23T14:59:43.421 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2024-04-23T14:59:43.421 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 0, 2024-04-23T14:59:43.422 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2024-04-23T14:59:43.422 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2024-04-23T14:59:43.422 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 3, 2024-04-23T14:59:43.422 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 2, 2024-04-23T14:59:43.422 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2024-04-23T14:59:43.422 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2024-04-23T14:59:43.422 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2024-04-23T14:59:43.422 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 8198928, 2024-04-23T14:59:43.422 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 8198816, 2024-04-23T14:59:43.423 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2024-04-23T14:59:43.423 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.423 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.423 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.423 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.423 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 1069 2024-04-23T14:59:43.423 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.423 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2024-04-23T14:59:43.423 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 27, 2024-04-23T14:59:43.424 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 345, 2024-04-23T14:59:43.424 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 318, 2024-04-23T14:59:43.424 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 16, 2024-04-23T14:59:43.424 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 97, 2024-04-23T14:59:43.424 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 81, 2024-04-23T14:59:43.424 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 184, 2024-04-23T14:59:43.424 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 510, 2024-04-23T14:59:43.424 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 326, 2024-04-23T14:59:43.424 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 0, 2024-04-23T14:59:43.425 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 0, 2024-04-23T14:59:43.425 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 0, 2024-04-23T14:59:43.425 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 51676, 2024-04-23T14:59:43.425 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 215416 2024-04-23T14:59:43.425 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.425 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2024-04-23T14:59:43.425 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 0, 2024-04-23T14:59:43.425 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 0, 2024-04-23T14:59:43.425 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 0, 2024-04-23T14:59:43.426 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 0, 2024-04-23T14:59:43.426 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 0, 2024-04-23T14:59:43.426 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2024-04-23T14:59:43.426 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.426 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.426 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.426 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.426 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2024-04-23T14:59:43.427 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.427 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.427 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.427 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.427 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2024-04-23T14:59:43.427 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.427 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.427 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.427 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.428 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2024-04-23T14:59:43.428 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.428 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.428 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.428 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.428 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2024-04-23T14:59:43.428 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.428 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.428 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.429 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.429 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2024-04-23T14:59:43.429 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.429 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.429 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.429 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.429 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2024-04-23T14:59:43.429 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.430 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.430 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.430 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.430 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2024-04-23T14:59:43.430 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.430 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.430 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.430 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.431 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2024-04-23T14:59:43.431 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.431 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.431 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.431 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.431 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2024-04-23T14:59:43.431 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.431 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.432 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.432 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.432 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2024-04-23T14:59:43.432 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.432 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.432 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.432 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.432 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2024-04-23T14:59:43.433 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.433 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.433 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.433 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.434 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2024-04-23T14:59:43.434 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.434 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.434 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.434 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.434 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2024-04-23T14:59:43.434 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.434 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.435 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.435 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.435 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2024-04-23T14:59:43.435 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.435 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.435 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.435 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.435 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2024-04-23T14:59:43.436 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.436 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.436 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.436 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.436 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2024-04-23T14:59:43.436 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.436 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.436 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.437 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.437 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2024-04-23T14:59:43.437 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.437 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.437 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.437 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.437 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2024-04-23T14:59:43.437 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.437 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.438 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.438 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.438 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2024-04-23T14:59:43.438 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.438 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.438 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.438 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.438 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2024-04-23T14:59:43.439 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.439 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.439 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.439 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.439 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2024-04-23T14:59:43.439 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.439 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.439 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.440 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.440 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2024-04-23T14:59:43.440 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.440 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.440 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.440 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.440 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2024-04-23T14:59:43.440 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.441 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.441 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.441 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.441 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2024-04-23T14:59:43.441 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.441 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.441 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.441 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.442 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2024-04-23T14:59:43.442 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.442 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.442 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.442 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.442 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2024-04-23T14:59:43.443 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.443 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.443 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.443 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.443 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2024-04-23T14:59:43.443 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.443 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.443 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.443 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.444 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2024-04-23T14:59:43.444 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.444 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.444 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.444 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.444 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2024-04-23T14:59:43.444 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2024-04-23T14:59:43.444 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2024-04-23T14:59:43.444 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.445 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.445 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.445 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:43.445 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.445 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2024-04-23T14:59:43.445 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 2, 2024-04-23T14:59:43.445 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 6, 2024-04-23T14:59:43.445 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 4, 2024-04-23T14:59:43.446 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 5, 2024-04-23T14:59:43.446 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2024-04-23T14:59:43.446 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 0, 2024-04-23T14:59:43.446 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 0, 2024-04-23T14:59:43.446 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 6130, 2024-04-23T14:59:43.446 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2024-04-23T14:59:43.446 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.446 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-04-23T14:59:43.447 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-04-23T14:59:43.447 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-04-23T14:59:43.447 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T14:59:43.447 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-04-23T14:59:43.447 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T14:59:43.447 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-04-23T14:59:43.447 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T14:59:43.447 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T14:59:43.448 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T14:59:43.448 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T14:59:43.448 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T14:59:43.448 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-04-23T14:59:43.448 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T14:59:43.448 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T14:59:43.448 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-04-23T14:59:43.448 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-04-23T14:59:43.448 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-04-23T14:59:43.449 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-04-23T14:59:43.449 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T14:59:43.449 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T14:59:43.449 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T14:59:43.449 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T14:59:43.449 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T14:59:43.449 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-04-23T14:59:43.449 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-04-23T14:59:43.449 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-04-23T14:59:43.450 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T14:59:43.450 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T14:59:43.450 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-04-23T14:59:43.450 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-04-23T14:59:43.450 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-04-23T14:59:43.450 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-04-23T14:59:43.450 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T14:59:43.450 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-04-23T14:59:43.451 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T14:59:43.451 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-04-23T14:59:43.451 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 190222, 2024-04-23T14:59:43.451 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 74, 2024-04-23T14:59:43.451 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-04-23T14:59:43.451 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-04-23T14:59:43.451 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-04-23T14:59:43.451 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-04-23T14:59:43.451 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-04-23T14:59:43.452 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-04-23T14:59:43.452 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-04-23T14:59:43.452 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-04-23T14:59:43.452 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 8128, 2024-04-23T14:59:43.452 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 169, 2024-04-23T14:59:43.452 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T14:59:43.452 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-04-23T14:59:43.453 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-04-23T14:59:43.453 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-04-23T14:59:43.453 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 242551, 2024-04-23T14:59:43.453 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 1638, 2024-04-23T14:59:43.453 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-04-23T14:59:43.453 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-04-23T14:59:43.453 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-04-23T14:59:43.453 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-04-23T14:59:43.453 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.454 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2024-04-23T14:59:43.454 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2024-04-23T14:59:43.454 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2024-04-23T14:59:43.454 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 2499, 2024-04-23T14:59:43.454 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 26, 2024-04-23T14:59:43.454 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 4, 2024-04-23T14:59:43.454 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 2499, 2024-04-23T14:59:43.455 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2024-04-23T14:59:43.455 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2499, 2024-04-23T14:59:43.455 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 11.316708482, 2024-04-23T14:59:43.455 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.004528494 2024-04-23T14:59:43.455 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.455 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2024-04-23T14:59:43.455 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2024-04-23T14:59:43.455 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2499, 2024-04-23T14:59:43.455 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2500 2024-04-23T14:59:43.456 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.456 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 2499, 2024-04-23T14:59:43.456 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 2499, 2024-04-23T14:59:43.456 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 0, 2024-04-23T14:59:43.456 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2024-04-23T14:59:43.456 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2024-04-23T14:59:43.456 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 1595, 2024-04-23T14:59:43.456 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 0, 2024-04-23T14:59:43.456 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 901, 2024-04-23T14:59:43.457 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 0, 2024-04-23T14:59:43.457 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 0, 2024-04-23T14:59:43.457 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2024-04-23T14:59:43.457 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2024-04-23T14:59:43.457 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2024-04-23T14:59:43.457 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2024-04-23T14:59:43.457 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 0, 2024-04-23T14:59:43.457 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2024-04-23T14:59:43.457 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2024-04-23T14:59:43.458 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2024-04-23T14:59:43.458 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 2, 2024-04-23T14:59:43.458 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 0, 2024-04-23T14:59:43.458 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2024-04-23T14:59:43.458 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2024-04-23T14:59:43.458 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2024-04-23T14:59:43.458 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2024-04-23T14:59:43.458 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2024-04-23T14:59:43.459 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2024-04-23T14:59:43.459 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T14:59:43.459 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2024-04-23T14:59:43.459 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2024-04-23T14:59:43.459 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2024-04-23T14:59:43.459 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2024-04-23T14:59:43.459 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2024-04-23T14:59:43.459 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2024-04-23T14:59:43.459 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2024-04-23T14:59:43.460 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2024-04-23T14:59:43.460 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2024-04-23T14:59:43.460 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2024-04-23T14:59:43.460 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2024-04-23T14:59:43.460 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2024-04-23T14:59:43.460 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2024-04-23T14:59:43.460 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2024-04-23T14:59:43.460 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2024-04-23T14:59:43.460 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2024-04-23T14:59:43.461 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2024-04-23T14:59:43.461 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2024-04-23T14:59:43.461 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2024-04-23T14:59:43.461 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 99, 2024-04-23T14:59:43.461 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2024-04-23T14:59:43.461 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 13, 2024-04-23T14:59:43.461 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 5, 2024-04-23T14:59:43.461 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 5, 2024-04-23T14:59:43.461 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2024-04-23T14:59:43.462 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2024-04-23T14:59:43.462 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 0, 2024-04-23T14:59:43.462 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 2, 2024-04-23T14:59:43.462 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 0 2024-04-23T14:59:43.462 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.462 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2024-04-23T14:59:43.462 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 0, 2024-04-23T14:59:43.463 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2024-04-23T14:59:43.463 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2024-04-23T14:59:43.463 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2024-04-23T14:59:43.463 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.463 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2024-04-23T14:59:43.463 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2024-04-23T14:59:43.463 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.463 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2024-04-23T14:59:43.464 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2024-04-23T14:59:43.464 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T14:59:43.464 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2024-04-23T14:59:43.464 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 0, 2024-04-23T14:59:43.464 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed_ops": 0, 2024-04-23T14:59:43.464 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 0, 2024-04-23T14:59:43.464 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2024-04-23T14:59:43.464 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.464 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T14:59:43.465 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:43.465 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-04-23T14:59:43.465 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:43.465 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 2764, 2024-04-23T14:59:43.465 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 6086535, 2024-04-23T14:59:43.465 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:43.465 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 2764, 2024-04-23T14:59:43.465 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-04-23T14:59:43.465 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-04-23T14:59:43.466 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 2764, 2024-04-23T14:59:43.466 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 6086535, 2024-04-23T14:59:43.466 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:43.466 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.466 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.466 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.466 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:43.466 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.467 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2024-04-23T14:59:43.467 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:43.467 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-04-23T14:59:43.467 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:43.467 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-04-23T14:59:43.467 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-04-23T14:59:43.467 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:43.467 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:43.467 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 2499, 2024-04-23T14:59:43.468 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 4004634, 2024-04-23T14:59:43.468 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 2499, 2024-04-23T14:59:43.468 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 4004634, 2024-04-23T14:59:43.468 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:43.468 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.468 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.468 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.468 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:43.469 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.469 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2024-04-23T14:59:43.469 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:43.469 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2024-04-23T14:59:43.469 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:43.469 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-04-23T14:59:43.469 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-04-23T14:59:43.469 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:43.469 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:43.470 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 2499, 2024-04-23T14:59:43.470 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 2499, 2024-04-23T14:59:43.470 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 2499, 2024-04-23T14:59:43.470 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 2499, 2024-04-23T14:59:43.470 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:43.470 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.470 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.470 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.470 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:43.471 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.471 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2024-04-23T14:59:43.471 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:43.471 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-04-23T14:59:43.471 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:43.471 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-04-23T14:59:43.471 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-04-23T14:59:43.471 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:43.471 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:43.472 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-04-23T14:59:43.472 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-04-23T14:59:43.472 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-04-23T14:59:43.472 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-04-23T14:59:43.472 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:43.472 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.472 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.472 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.472 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:43.473 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:43.473 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x5625e9a226e0": { 2024-04-23T14:59:43.473 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:43.473 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-04-23T14:59:43.473 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:43.473 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-04-23T14:59:43.473 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-04-23T14:59:43.474 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:43.474 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:43.474 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-04-23T14:59:43.474 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-04-23T14:59:43.474 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-04-23T14:59:43.474 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-04-23T14:59:43.474 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:43.474 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:43.474 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:43.475 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:43.475 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:43.475 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:43.475 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-04-23T14:59:43.499 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:43 smithi203 ceph-mon[32800]: pgmap v351: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:59:43.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:43 smithi083 ceph-mon[27867]: pgmap v351: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:59:43.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:43 smithi173 ceph-mon[32443]: pgmap v351: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:59:44.236 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-23T14:59:44.236 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-23T14:59:44.237 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-23T14:59:44.237 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-23T14:59:44.237 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T14:59:44.237 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-23T14:59:44.237 DEBUG:tasks.check_counter:Getting stats from f 2024-04-23T14:59:44.237 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-23T14:59:44.494 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:59:45.702 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:45 smithi203 ceph-mon[32800]: pgmap v352: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:45.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:45 smithi083 ceph-mon[27867]: pgmap v352: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:45.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:45 smithi173 ceph-mon[32443]: pgmap v352: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:46.275 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-04-23T14:59:46.276 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T14:59:46.276 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 748, 2024-04-23T14:59:46.276 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 729, 2024-04-23T14:59:46.276 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 5457987, 2024-04-23T14:59:46.276 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 297236, 2024-04-23T14:59:46.276 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 6, 2024-04-23T14:59:46.276 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 2, 2024-04-23T14:59:46.276 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.096351288, 2024-04-23T14:59:46.277 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.028739800, 2024-04-23T14:59:46.277 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.053135740, 2024-04-23T14:59:46.277 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.018990706, 2024-04-23T14:59:46.277 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:46.277 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 729, 2024-04-23T14:59:46.277 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.049024665, 2024-04-23T14:59:46.277 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000067249 2024-04-23T14:59:46.277 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.278 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:46.278 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.278 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.278 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.278 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.278 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 1645088, 2024-04-23T14:59:46.278 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 144272 2024-04-23T14:59:46.278 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.278 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T14:59:46.279 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 769, 2024-04-23T14:59:46.279 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 941, 2024-04-23T14:59:46.279 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 8581026, 2024-04-23T14:59:46.279 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 711191, 2024-04-23T14:59:46.279 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 7, 2024-04-23T14:59:46.279 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2024-04-23T14:59:46.279 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.085372346, 2024-04-23T14:59:46.279 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.028880900, 2024-04-23T14:59:46.279 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.023296720, 2024-04-23T14:59:46.279 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.024965404, 2024-04-23T14:59:46.280 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:46.280 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 938, 2024-04-23T14:59:46.280 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.063330091, 2024-04-23T14:59:46.280 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000067516 2024-04-23T14:59:46.280 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.280 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:46.280 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.280 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.280 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.280 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.281 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T14:59:46.281 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 467280 2024-04-23T14:59:46.281 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.281 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T14:59:46.281 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 1141, 2024-04-23T14:59:46.281 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 1143, 2024-04-23T14:59:46.281 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 8744421, 2024-04-23T14:59:46.281 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 362896, 2024-04-23T14:59:46.281 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 5, 2024-04-23T14:59:46.282 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 4, 2024-04-23T14:59:46.282 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.129335052, 2024-04-23T14:59:46.282 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.036323935, 2024-04-23T14:59:46.282 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.039604806, 2024-04-23T14:59:46.282 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.042188981, 2024-04-23T14:59:46.282 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:46.282 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1141, 2024-04-23T14:59:46.282 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.051928590, 2024-04-23T14:59:46.282 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000045511 2024-04-23T14:59:46.282 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.283 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:46.283 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.283 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.283 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.283 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.283 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T14:59:46.283 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T14:59:46.283 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.283 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-04-23T14:59:46.284 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2024-04-23T14:59:46.284 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-04-23T14:59:46.284 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.284 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2024-04-23T14:59:46.284 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-04-23T14:59:46.284 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-04-23T14:59:46.284 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 4038, 2024-04-23T14:59:46.284 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.112504570, 2024-04-23T14:59:46.284 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000027861 2024-04-23T14:59:46.284 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:46.285 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.285 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2024-04-23T14:59:46.285 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-04-23T14:59:46.285 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-04-23T14:59:46.285 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.285 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.285 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.285 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:46.285 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.285 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2024-04-23T14:59:46.286 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 0, 2024-04-23T14:59:46.286 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 0, 2024-04-23T14:59:46.286 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2024-04-23T14:59:46.286 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.286 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.286 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.286 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.286 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2024-04-23T14:59:46.286 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 0, 2024-04-23T14:59:46.287 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2024-04-23T14:59:46.287 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2024-04-23T14:59:46.287 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 0, 2024-04-23T14:59:46.287 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 0, 2024-04-23T14:59:46.287 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2024-04-23T14:59:46.287 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 1231, 2024-04-23T14:59:46.287 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 1, 2024-04-23T14:59:46.287 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 1230, 2024-04-23T14:59:46.287 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 0, 2024-04-23T14:59:46.288 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 1230, 2024-04-23T14:59:46.288 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 0, 2024-04-23T14:59:46.288 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 0, 2024-04-23T14:59:46.288 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 0, 2024-04-23T14:59:46.288 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 8, 2024-04-23T14:59:46.288 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 0, 2024-04-23T14:59:46.288 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 0, 2024-04-23T14:59:46.288 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 0, 2024-04-23T14:59:46.288 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 0, 2024-04-23T14:59:46.288 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2024-04-23T14:59:46.289 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2024-04-23T14:59:46.289 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2024-04-23T14:59:46.289 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2024-04-23T14:59:46.289 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2024-04-23T14:59:46.289 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 0, 2024-04-23T14:59:46.289 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 0, 2024-04-23T14:59:46.289 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 0, 2024-04-23T14:59:46.289 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 0, 2024-04-23T14:59:46.289 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2024-04-23T14:59:46.289 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 0, 2024-04-23T14:59:46.290 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2024-04-23T14:59:46.290 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 0, 2024-04-23T14:59:46.290 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 0, 2024-04-23T14:59:46.290 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 0, 2024-04-23T14:59:46.290 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T14:59:46.290 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2024-04-23T14:59:46.290 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T14:59:46.290 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2024-04-23T14:59:46.290 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 0, 2024-04-23T14:59:46.290 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T14:59:46.291 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T14:59:46.291 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 0, 2024-04-23T14:59:46.291 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 0, 2024-04-23T14:59:46.291 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T14:59:46.291 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 0, 2024-04-23T14:59:46.291 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T14:59:46.291 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T14:59:46.291 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T14:59:46.292 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 0, 2024-04-23T14:59:46.292 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 0, 2024-04-23T14:59:46.292 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 0, 2024-04-23T14:59:46.292 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 0 2024-04-23T14:59:46.292 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.292 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2024-04-23T14:59:46.292 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 0, 2024-04-23T14:59:46.292 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 0, 2024-04-23T14:59:46.292 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 0, 2024-04-23T14:59:46.292 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 0, 2024-04-23T14:59:46.293 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 0, 2024-04-23T14:59:46.293 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2024-04-23T14:59:46.293 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2024-04-23T14:59:46.293 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2024-04-23T14:59:46.293 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 0, 2024-04-23T14:59:46.293 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 0, 2024-04-23T14:59:46.293 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2024-04-23T14:59:46.293 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2024-04-23T14:59:46.293 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2024-04-23T14:59:46.294 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2024-04-23T14:59:46.294 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2024-04-23T14:59:46.294 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2024-04-23T14:59:46.294 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2024-04-23T14:59:46.294 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_path": 0, 2024-04-23T14:59:46.294 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_inode": 0, 2024-04-23T14:59:46.294 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T14:59:46.294 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 0, 2024-04-23T14:59:46.294 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2024-04-23T14:59:46.294 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 0, 2024-04-23T14:59:46.295 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2024-04-23T14:59:46.295 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2024-04-23T14:59:46.295 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.295 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2024-04-23T14:59:46.295 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2024-04-23T14:59:46.295 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 0, 2024-04-23T14:59:46.295 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2024-04-23T14:59:46.295 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2024-04-23T14:59:46.295 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 7505, 2024-04-23T14:59:46.295 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2024-04-23T14:59:46.296 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2024-04-23T14:59:46.296 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 0, 2024-04-23T14:59:46.296 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2024-04-23T14:59:46.296 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2024-04-23T14:59:46.296 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 11, 2024-04-23T14:59:46.296 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 2, 2024-04-23T14:59:46.296 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2024-04-23T14:59:46.296 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2024-04-23T14:59:46.296 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2024-04-23T14:59:46.297 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 24658753, 2024-04-23T14:59:46.297 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 24658179, 2024-04-23T14:59:46.297 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2024-04-23T14:59:46.297 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.297 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.297 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.297 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.297 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 7505 2024-04-23T14:59:46.297 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.297 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2024-04-23T14:59:46.298 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 728, 2024-04-23T14:59:46.298 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 1106, 2024-04-23T14:59:46.298 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 378, 2024-04-23T14:59:46.298 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 99, 2024-04-23T14:59:46.298 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 112, 2024-04-23T14:59:46.298 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 13, 2024-04-23T14:59:46.298 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 1231, 2024-04-23T14:59:46.298 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 1270, 2024-04-23T14:59:46.298 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 39, 2024-04-23T14:59:46.299 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 0, 2024-04-23T14:59:46.299 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 0, 2024-04-23T14:59:46.299 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 0, 2024-04-23T14:59:46.299 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 58964, 2024-04-23T14:59:46.299 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 215416 2024-04-23T14:59:46.299 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.299 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2024-04-23T14:59:46.299 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 0, 2024-04-23T14:59:46.299 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 0, 2024-04-23T14:59:46.299 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 0, 2024-04-23T14:59:46.300 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 0, 2024-04-23T14:59:46.300 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 0, 2024-04-23T14:59:46.300 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2024-04-23T14:59:46.300 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.300 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.300 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.300 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.300 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2024-04-23T14:59:46.300 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.301 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.301 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.301 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.301 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2024-04-23T14:59:46.301 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.301 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.301 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.301 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.301 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2024-04-23T14:59:46.301 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.302 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.302 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.302 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.302 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2024-04-23T14:59:46.302 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.302 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.302 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.302 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.302 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2024-04-23T14:59:46.303 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.303 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.303 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.303 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.303 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2024-04-23T14:59:46.303 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.303 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.303 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.303 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.304 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2024-04-23T14:59:46.304 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.304 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.304 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.304 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.304 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2024-04-23T14:59:46.304 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.304 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.304 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.305 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.305 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2024-04-23T14:59:46.305 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.305 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.305 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.305 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.305 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2024-04-23T14:59:46.305 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.305 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.305 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.306 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.306 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2024-04-23T14:59:46.306 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.306 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.307 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.307 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.307 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2024-04-23T14:59:46.307 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.307 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.307 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.307 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.307 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2024-04-23T14:59:46.307 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.308 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.308 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.308 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.308 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2024-04-23T14:59:46.308 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.308 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.308 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.308 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.308 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2024-04-23T14:59:46.309 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.309 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.309 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.309 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.309 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2024-04-23T14:59:46.309 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.309 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.309 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.309 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.310 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2024-04-23T14:59:46.310 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.310 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.310 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.310 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.310 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2024-04-23T14:59:46.310 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.310 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.310 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.311 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.311 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2024-04-23T14:59:46.311 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.311 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.311 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.311 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.311 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2024-04-23T14:59:46.311 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.311 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.311 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.312 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.312 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2024-04-23T14:59:46.312 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.312 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.312 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.312 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.312 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2024-04-23T14:59:46.312 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.313 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.313 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.313 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.313 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2024-04-23T14:59:46.313 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.313 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.313 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.313 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.313 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2024-04-23T14:59:46.313 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.314 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.314 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.314 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.314 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2024-04-23T14:59:46.314 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.314 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.314 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.314 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.314 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2024-04-23T14:59:46.315 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.315 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.315 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.315 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.315 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2024-04-23T14:59:46.315 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.315 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.315 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.315 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.315 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2024-04-23T14:59:46.316 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.316 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.316 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.316 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.316 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2024-04-23T14:59:46.316 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2024-04-23T14:59:46.316 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2024-04-23T14:59:46.316 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.316 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.316 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.317 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:46.317 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.317 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2024-04-23T14:59:46.317 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2024-04-23T14:59:46.317 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 4, 2024-04-23T14:59:46.317 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 3, 2024-04-23T14:59:46.317 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 1, 2024-04-23T14:59:46.317 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2024-04-23T14:59:46.317 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 0, 2024-04-23T14:59:46.318 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 0, 2024-04-23T14:59:46.318 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 2857, 2024-04-23T14:59:46.318 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2024-04-23T14:59:46.318 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.318 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-04-23T14:59:46.318 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-04-23T14:59:46.318 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-04-23T14:59:46.318 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T14:59:46.318 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-04-23T14:59:46.319 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T14:59:46.319 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-04-23T14:59:46.319 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T14:59:46.319 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T14:59:46.319 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T14:59:46.319 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T14:59:46.319 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T14:59:46.319 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-04-23T14:59:46.319 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T14:59:46.320 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T14:59:46.320 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-04-23T14:59:46.320 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-04-23T14:59:46.320 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-04-23T14:59:46.320 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-04-23T14:59:46.320 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T14:59:46.320 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T14:59:46.320 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T14:59:46.320 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T14:59:46.320 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T14:59:46.320 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-04-23T14:59:46.321 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-04-23T14:59:46.321 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-04-23T14:59:46.321 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T14:59:46.321 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T14:59:46.321 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-04-23T14:59:46.321 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-04-23T14:59:46.321 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-04-23T14:59:46.321 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-04-23T14:59:46.321 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T14:59:46.322 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-04-23T14:59:46.322 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T14:59:46.322 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-04-23T14:59:46.322 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 236813, 2024-04-23T14:59:46.322 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 866, 2024-04-23T14:59:46.322 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-04-23T14:59:46.322 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-04-23T14:59:46.322 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-04-23T14:59:46.322 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-04-23T14:59:46.323 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-04-23T14:59:46.323 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-04-23T14:59:46.323 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-04-23T14:59:46.323 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-04-23T14:59:46.323 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 8128, 2024-04-23T14:59:46.323 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 169, 2024-04-23T14:59:46.323 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T14:59:46.323 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-04-23T14:59:46.323 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-04-23T14:59:46.323 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-04-23T14:59:46.324 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 2761685, 2024-04-23T14:59:46.324 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 32257, 2024-04-23T14:59:46.324 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-04-23T14:59:46.324 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-04-23T14:59:46.324 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-04-23T14:59:46.324 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-04-23T14:59:46.324 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.324 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2024-04-23T14:59:46.324 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2024-04-23T14:59:46.325 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2024-04-23T14:59:46.325 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 2389, 2024-04-23T14:59:46.325 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 21, 2024-04-23T14:59:46.325 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 3, 2024-04-23T14:59:46.325 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 2389, 2024-04-23T14:59:46.325 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2024-04-23T14:59:46.325 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2389, 2024-04-23T14:59:46.325 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 43.981866657, 2024-04-23T14:59:46.326 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.018410157 2024-04-23T14:59:46.326 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.326 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2024-04-23T14:59:46.326 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2024-04-23T14:59:46.326 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2389, 2024-04-23T14:59:46.326 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 2390 2024-04-23T14:59:46.326 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.326 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 2389, 2024-04-23T14:59:46.326 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 2389, 2024-04-23T14:59:46.327 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 0, 2024-04-23T14:59:46.327 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2024-04-23T14:59:46.327 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2024-04-23T14:59:46.327 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 1473, 2024-04-23T14:59:46.327 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 0, 2024-04-23T14:59:46.327 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 914, 2024-04-23T14:59:46.327 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 0, 2024-04-23T14:59:46.327 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 0, 2024-04-23T14:59:46.328 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2024-04-23T14:59:46.328 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2024-04-23T14:59:46.328 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2024-04-23T14:59:46.328 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 0, 2024-04-23T14:59:46.328 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 0, 2024-04-23T14:59:46.328 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2024-04-23T14:59:46.328 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2024-04-23T14:59:46.328 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2024-04-23T14:59:46.328 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 1, 2024-04-23T14:59:46.329 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 0, 2024-04-23T14:59:46.329 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2024-04-23T14:59:46.329 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2024-04-23T14:59:46.329 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2024-04-23T14:59:46.329 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2024-04-23T14:59:46.329 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2024-04-23T14:59:46.329 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2024-04-23T14:59:46.329 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T14:59:46.329 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2024-04-23T14:59:46.329 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2024-04-23T14:59:46.330 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 0, 2024-04-23T14:59:46.330 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2024-04-23T14:59:46.330 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2024-04-23T14:59:46.330 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2024-04-23T14:59:46.330 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2024-04-23T14:59:46.330 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2024-04-23T14:59:46.330 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2024-04-23T14:59:46.330 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2024-04-23T14:59:46.331 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2024-04-23T14:59:46.331 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2024-04-23T14:59:46.331 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2024-04-23T14:59:46.331 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2024-04-23T14:59:46.331 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2024-04-23T14:59:46.331 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2024-04-23T14:59:46.331 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2024-04-23T14:59:46.331 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2024-04-23T14:59:46.331 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2024-04-23T14:59:46.331 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 99, 2024-04-23T14:59:46.332 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2024-04-23T14:59:46.332 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 13, 2024-04-23T14:59:46.332 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 7, 2024-04-23T14:59:46.332 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 7, 2024-04-23T14:59:46.332 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2024-04-23T14:59:46.332 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2024-04-23T14:59:46.332 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 0, 2024-04-23T14:59:46.332 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 2, 2024-04-23T14:59:46.332 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 0 2024-04-23T14:59:46.332 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.333 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2024-04-23T14:59:46.333 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 0, 2024-04-23T14:59:46.333 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2024-04-23T14:59:46.333 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2024-04-23T14:59:46.333 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2024-04-23T14:59:46.333 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.333 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2024-04-23T14:59:46.333 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2024-04-23T14:59:46.333 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.333 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2024-04-23T14:59:46.334 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2024-04-23T14:59:46.334 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T14:59:46.334 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2024-04-23T14:59:46.334 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 0, 2024-04-23T14:59:46.334 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed_ops": 0, 2024-04-23T14:59:46.334 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 0, 2024-04-23T14:59:46.334 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2024-04-23T14:59:46.334 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.335 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T14:59:46.335 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:46.335 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-04-23T14:59:46.335 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:46.335 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 2654, 2024-04-23T14:59:46.335 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 22528077, 2024-04-23T14:59:46.335 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:46.335 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 2654, 2024-04-23T14:59:46.335 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-04-23T14:59:46.336 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-04-23T14:59:46.336 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 2654, 2024-04-23T14:59:46.336 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 22528077, 2024-04-23T14:59:46.336 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:46.336 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.336 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.336 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.336 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:46.336 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.336 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2024-04-23T14:59:46.337 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:46.337 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-04-23T14:59:46.337 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:46.337 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-04-23T14:59:46.337 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-04-23T14:59:46.337 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:46.337 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:46.337 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 2389, 2024-04-23T14:59:46.338 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 20466786, 2024-04-23T14:59:46.338 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 2389, 2024-04-23T14:59:46.338 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 20466786, 2024-04-23T14:59:46.338 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:46.338 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.338 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.338 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.338 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:46.338 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.339 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2024-04-23T14:59:46.339 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:46.339 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2024-04-23T14:59:46.339 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:46.339 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-04-23T14:59:46.339 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-04-23T14:59:46.339 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:46.339 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:46.339 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 2389, 2024-04-23T14:59:46.340 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 2389, 2024-04-23T14:59:46.340 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 2389, 2024-04-23T14:59:46.340 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 2389, 2024-04-23T14:59:46.340 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:46.340 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.340 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.340 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.340 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:46.340 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.340 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2024-04-23T14:59:46.341 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:46.341 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-04-23T14:59:46.341 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:46.341 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-04-23T14:59:46.341 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-04-23T14:59:46.341 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:46.341 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:46.341 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-04-23T14:59:46.341 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-04-23T14:59:46.341 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-04-23T14:59:46.342 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-04-23T14:59:46.342 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:46.342 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.342 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.342 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.342 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:46.342 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:46.342 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x5590aed4e6e0": { 2024-04-23T14:59:46.342 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:46.343 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-04-23T14:59:46.343 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:46.343 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-04-23T14:59:46.343 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-04-23T14:59:46.343 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:46.343 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:46.343 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-04-23T14:59:46.343 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-04-23T14:59:46.343 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 0, 2024-04-23T14:59:46.343 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 0, 2024-04-23T14:59:46.344 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:46.344 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:46.344 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:46.344 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:46.344 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:46.344 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:46.344 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-04-23T14:59:47.217 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-23T14:59:47.217 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-23T14:59:47.217 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-23T14:59:47.217 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-23T14:59:47.218 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T14:59:47.218 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-23T14:59:47.218 DEBUG:tasks.check_counter:Getting stats from i 2024-04-23T14:59:47.220 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-23T14:59:47.461 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:59:47.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:47 smithi203 ceph-mon[32800]: pgmap v353: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:59:47.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:47 smithi083 ceph-mon[27867]: pgmap v353: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:59:47.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:47 smithi173 ceph-mon[32443]: pgmap v353: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:59:49.326 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-04-23T14:59:49.327 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T14:59:49.327 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 2044, 2024-04-23T14:59:49.327 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 2164, 2024-04-23T14:59:49.327 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 2086873, 2024-04-23T14:59:49.327 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 9148366, 2024-04-23T14:59:49.327 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 43, 2024-04-23T14:59:49.327 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 5, 2024-04-23T14:59:49.327 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.200208218, 2024-04-23T14:59:49.328 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.069180710, 2024-04-23T14:59:49.328 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.079340137, 2024-04-23T14:59:49.328 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.047811641, 2024-04-23T14:59:49.328 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:49.328 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2163, 2024-04-23T14:59:49.328 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.103760086, 2024-04-23T14:59:49.328 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000047970 2024-04-23T14:59:49.328 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.328 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:49.328 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 41, 2024-04-23T14:59:49.329 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000004399, 2024-04-23T14:59:49.329 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000107 2024-04-23T14:59:49.329 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.329 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 1657200, 2024-04-23T14:59:49.329 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 495840 2024-04-23T14:59:49.329 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.329 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T14:59:49.329 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 11195, 2024-04-23T14:59:49.329 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 14492, 2024-04-23T14:59:49.330 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 3937214, 2024-04-23T14:59:49.330 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 16187463, 2024-04-23T14:59:49.330 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 14, 2024-04-23T14:59:49.330 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 6, 2024-04-23T14:59:49.330 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.816889049, 2024-04-23T14:59:49.330 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.333664756, 2024-04-23T14:59:49.330 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.540445066, 2024-04-23T14:59:49.330 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.088357411, 2024-04-23T14:59:49.331 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:49.331 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 14490, 2024-04-23T14:59:49.331 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1.005615405, 2024-04-23T14:59:49.331 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000069400 2024-04-23T14:59:49.331 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.331 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:49.331 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 14706, 2024-04-23T14:59:49.331 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.002505018, 2024-04-23T14:59:49.332 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000170 2024-04-23T14:59:49.332 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.332 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 2993792, 2024-04-23T14:59:49.332 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 6383840 2024-04-23T14:59:49.332 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.332 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T14:59:49.332 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 4337, 2024-04-23T14:59:49.332 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 4739, 2024-04-23T14:59:49.332 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 1518168, 2024-04-23T14:59:49.333 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 2376149, 2024-04-23T14:59:49.333 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 12, 2024-04-23T14:59:49.333 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 6, 2024-04-23T14:59:49.333 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.275516790, 2024-04-23T14:59:49.333 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.101721408, 2024-04-23T14:59:49.333 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.098604349, 2024-04-23T14:59:49.333 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.062786128, 2024-04-23T14:59:49.333 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:49.333 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 4737, 2024-04-23T14:59:49.334 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.139327990, 2024-04-23T14:59:49.334 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000029412 2024-04-23T14:59:49.334 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.334 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:49.334 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 3229, 2024-04-23T14:59:49.334 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000611895, 2024-04-23T14:59:49.334 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000189 2024-04-23T14:59:49.334 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.334 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T14:59:49.335 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T14:59:49.335 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.335 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-04-23T14:59:49.335 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 1, 2024-04-23T14:59:49.335 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-04-23T14:59:49.335 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.335 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-MDSRank": { 2024-04-23T14:59:49.335 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-04-23T14:59:49.335 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-04-23T14:59:49.336 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 7193, 2024-04-23T14:59:49.336 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 3.467713469, 2024-04-23T14:59:49.336 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000482095 2024-04-23T14:59:49.336 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:49.336 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.336 INFO:teuthology.orchestra.run.smithi203.stdout: "finisher-PurgeQueue": { 2024-04-23T14:59:49.336 INFO:teuthology.orchestra.run.smithi203.stdout: "queue_len": 0, 2024-04-23T14:59:49.336 INFO:teuthology.orchestra.run.smithi203.stdout: "complete_latency": { 2024-04-23T14:59:49.337 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 676, 2024-04-23T14:59:49.337 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.031095269, 2024-04-23T14:59:49.337 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000045998 2024-04-23T14:59:49.337 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:49.337 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.337 INFO:teuthology.orchestra.run.smithi203.stdout: "mds": { 2024-04-23T14:59:49.337 INFO:teuthology.orchestra.run.smithi203.stdout: "request": 1141, 2024-04-23T14:59:49.337 INFO:teuthology.orchestra.run.smithi203.stdout: "reply": 1136, 2024-04-23T14:59:49.337 INFO:teuthology.orchestra.run.smithi203.stdout: "reply_latency": { 2024-04-23T14:59:49.338 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1136, 2024-04-23T14:59:49.338 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 1.846353172, 2024-04-23T14:59:49.338 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001625310 2024-04-23T14:59:49.338 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.338 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_reply": 0, 2024-04-23T14:59:49.338 INFO:teuthology.orchestra.run.smithi203.stdout: "forward": 5, 2024-04-23T14:59:49.338 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_complete": 0, 2024-04-23T14:59:49.338 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_fetch_keys": 0, 2024-04-23T14:59:49.338 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_commit": 20, 2024-04-23T14:59:49.339 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_split": 1, 2024-04-23T14:59:49.339 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_merge": 0, 2024-04-23T14:59:49.339 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes": 886, 2024-04-23T14:59:49.339 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_top": 30, 2024-04-23T14:59:49.339 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_bottom": 584, 2024-04-23T14:59:49.339 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pin_tail": 272, 2024-04-23T14:59:49.339 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_pinned": 843, 2024-04-23T14:59:49.339 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_expired": 8, 2024-04-23T14:59:49.339 INFO:teuthology.orchestra.run.smithi203.stdout: "inodes_with_caps": 396, 2024-04-23T14:59:49.340 INFO:teuthology.orchestra.run.smithi203.stdout: "caps": 396, 2024-04-23T14:59:49.340 INFO:teuthology.orchestra.run.smithi203.stdout: "subtrees": 11, 2024-04-23T14:59:49.340 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse": 1280, 2024-04-23T14:59:49.340 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_hit": 1140, 2024-04-23T14:59:49.340 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_forward": 1, 2024-04-23T14:59:49.340 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_discover": 69, 2024-04-23T14:59:49.340 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_dir_fetch": 0, 2024-04-23T14:59:49.340 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_remote_ino": 0, 2024-04-23T14:59:49.340 INFO:teuthology.orchestra.run.smithi203.stdout: "traverse_lock": 0, 2024-04-23T14:59:49.341 INFO:teuthology.orchestra.run.smithi203.stdout: "load_cent": 0, 2024-04-23T14:59:49.341 INFO:teuthology.orchestra.run.smithi203.stdout: "q": 0, 2024-04-23T14:59:49.341 INFO:teuthology.orchestra.run.smithi203.stdout: "exported": 5, 2024-04-23T14:59:49.341 INFO:teuthology.orchestra.run.smithi203.stdout: "exported_inodes": 6, 2024-04-23T14:59:49.341 INFO:teuthology.orchestra.run.smithi203.stdout: "imported": 69, 2024-04-23T14:59:49.341 INFO:teuthology.orchestra.run.smithi203.stdout: "imported_inodes": 306, 2024-04-23T14:59:49.341 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_dir_fetch": 0, 2024-04-23T14:59:49.341 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_backtrace_fetch": 5682, 2024-04-23T14:59:49.341 INFO:teuthology.orchestra.run.smithi203.stdout: "openino_peer_discover": 0, 2024-04-23T14:59:49.342 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rfiles": 325, 2024-04-23T14:59:49.342 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rbytes": 13424230511, 2024-04-23T14:59:49.342 INFO:teuthology.orchestra.run.smithi203.stdout: "root_rsnaps": 6, 2024-04-23T14:59:49.342 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_fetch": 5682, 2024-04-23T14:59:49.342 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_set_tag": 0, 2024-04-23T14:59:49.342 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T14:59:49.342 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_inotable_repaired": 0, 2024-04-23T14:59:49.342 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_inodes": 1210, 2024-04-23T14:59:49.342 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T14:59:49.343 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_dirfrag_rstats": 1210, 2024-04-23T14:59:49.343 INFO:teuthology.orchestra.run.smithi203.stdout: "scrub_file_inodes": 4472, 2024-04-23T14:59:49.343 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_inode_file_caps": 4, 2024-04-23T14:59:49.343 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_revoke": 311, 2024-04-23T14:59:49.343 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_grant": 3025, 2024-04-23T14:59:49.343 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_trunc": 5, 2024-04-23T14:59:49.343 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T14:59:49.343 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T14:59:49.343 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps": 6280, 2024-04-23T14:59:49.344 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_caps_dirty": 4913, 2024-04-23T14:59:49.344 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_cap_release": 114, 2024-04-23T14:59:49.344 INFO:teuthology.orchestra.run.smithi203.stdout: "process_request_cap_release": 574 2024-04-23T14:59:49.344 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.344 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_cache": { 2024-04-23T14:59:49.344 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update": 0, 2024-04-23T14:59:49.344 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_update_receipt": 0, 2024-04-23T14:59:49.344 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_try_discover": 0, 2024-04-23T14:59:49.344 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_send_discover": 74, 2024-04-23T14:59:49.345 INFO:teuthology.orchestra.run.smithi203.stdout: "dir_handle_discover": 1, 2024-04-23T14:59:49.345 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays": 0, 2024-04-23T14:59:49.345 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_delayed": 0, 2024-04-23T14:59:49.345 INFO:teuthology.orchestra.run.smithi203.stdout: "num_strays_enqueuing": 0, 2024-04-23T14:59:49.345 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_created": 375, 2024-04-23T14:59:49.345 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_enqueued": 375, 2024-04-23T14:59:49.345 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_reintegrated": 0, 2024-04-23T14:59:49.345 INFO:teuthology.orchestra.run.smithi203.stdout: "strays_migrated": 0, 2024-04-23T14:59:49.345 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_processing": 0, 2024-04-23T14:59:49.346 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_enqueued": 0, 2024-04-23T14:59:49.346 INFO:teuthology.orchestra.run.smithi203.stdout: "num_recovering_prioritized": 0, 2024-04-23T14:59:49.346 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_started": 0, 2024-04-23T14:59:49.346 INFO:teuthology.orchestra.run.smithi203.stdout: "recovery_completed": 0, 2024-04-23T14:59:49.346 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_path": 0, 2024-04-23T14:59:49.346 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_quiesce_inode": 0, 2024-04-23T14:59:49.346 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T14:59:49.346 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_exportdir": 5, 2024-04-23T14:59:49.346 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_flush": 0, 2024-04-23T14:59:49.346 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragmentdir": 1, 2024-04-23T14:59:49.347 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_fragstats": 0, 2024-04-23T14:59:49.347 INFO:teuthology.orchestra.run.smithi203.stdout: "ireq_inodestats": 0 2024-04-23T14:59:49.347 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.347 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_log": { 2024-04-23T14:59:49.347 INFO:teuthology.orchestra.run.smithi203.stdout: "evlrg": 0, 2024-04-23T14:59:49.347 INFO:teuthology.orchestra.run.smithi203.stdout: "evadd": 7505, 2024-04-23T14:59:49.347 INFO:teuthology.orchestra.run.smithi203.stdout: "evex": 0, 2024-04-23T14:59:49.347 INFO:teuthology.orchestra.run.smithi203.stdout: "evtrm": 0, 2024-04-23T14:59:49.347 INFO:teuthology.orchestra.run.smithi203.stdout: "ev": 7505, 2024-04-23T14:59:49.348 INFO:teuthology.orchestra.run.smithi203.stdout: "evexg": 0, 2024-04-23T14:59:49.348 INFO:teuthology.orchestra.run.smithi203.stdout: "evexd": 0, 2024-04-23T14:59:49.348 INFO:teuthology.orchestra.run.smithi203.stdout: "segadd": 11, 2024-04-23T14:59:49.348 INFO:teuthology.orchestra.run.smithi203.stdout: "segex": 0, 2024-04-23T14:59:49.348 INFO:teuthology.orchestra.run.smithi203.stdout: "segtrm": 0, 2024-04-23T14:59:49.348 INFO:teuthology.orchestra.run.smithi203.stdout: "seg": 11, 2024-04-23T14:59:49.348 INFO:teuthology.orchestra.run.smithi203.stdout: "segmjr": 2, 2024-04-23T14:59:49.348 INFO:teuthology.orchestra.run.smithi203.stdout: "segexg": 0, 2024-04-23T14:59:49.348 INFO:teuthology.orchestra.run.smithi203.stdout: "segexd": 0, 2024-04-23T14:59:49.348 INFO:teuthology.orchestra.run.smithi203.stdout: "expos": 4194304, 2024-04-23T14:59:49.349 INFO:teuthology.orchestra.run.smithi203.stdout: "wrpos": 24658753, 2024-04-23T14:59:49.349 INFO:teuthology.orchestra.run.smithi203.stdout: "rdpos": 0, 2024-04-23T14:59:49.349 INFO:teuthology.orchestra.run.smithi203.stdout: "jlat": { 2024-04-23T14:59:49.349 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 1683, 2024-04-23T14:59:49.349 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 54.469421841, 2024-04-23T14:59:49.349 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.032364481 2024-04-23T14:59:49.349 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.349 INFO:teuthology.orchestra.run.smithi203.stdout: "replayed": 0 2024-04-23T14:59:49.349 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.349 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_mem": { 2024-04-23T14:59:49.350 INFO:teuthology.orchestra.run.smithi203.stdout: "ino": 735, 2024-04-23T14:59:49.350 INFO:teuthology.orchestra.run.smithi203.stdout: "ino+": 1113, 2024-04-23T14:59:49.350 INFO:teuthology.orchestra.run.smithi203.stdout: "ino-": 378, 2024-04-23T14:59:49.350 INFO:teuthology.orchestra.run.smithi203.stdout: "dir": 106, 2024-04-23T14:59:49.350 INFO:teuthology.orchestra.run.smithi203.stdout: "dir+": 120, 2024-04-23T14:59:49.350 INFO:teuthology.orchestra.run.smithi203.stdout: "dir-": 14, 2024-04-23T14:59:49.350 INFO:teuthology.orchestra.run.smithi203.stdout: "dn": 886, 2024-04-23T14:59:49.350 INFO:teuthology.orchestra.run.smithi203.stdout: "dn+": 1317, 2024-04-23T14:59:49.350 INFO:teuthology.orchestra.run.smithi203.stdout: "dn-": 431, 2024-04-23T14:59:49.351 INFO:teuthology.orchestra.run.smithi203.stdout: "cap": 396, 2024-04-23T14:59:49.351 INFO:teuthology.orchestra.run.smithi203.stdout: "cap+": 776, 2024-04-23T14:59:49.351 INFO:teuthology.orchestra.run.smithi203.stdout: "cap-": 380, 2024-04-23T14:59:49.351 INFO:teuthology.orchestra.run.smithi203.stdout: "rss": 68580, 2024-04-23T14:59:49.351 INFO:teuthology.orchestra.run.smithi203.stdout: "heap": 223608 2024-04-23T14:59:49.351 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.351 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_server": { 2024-04-23T14:59:49.351 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_client_request": 1267, 2024-04-23T14:59:49.351 INFO:teuthology.orchestra.run.smithi203.stdout: "dispatch_server_request": 4, 2024-04-23T14:59:49.351 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_request": 1141, 2024-04-23T14:59:49.352 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_client_session": 21, 2024-04-23T14:59:49.352 INFO:teuthology.orchestra.run.smithi203.stdout: "handle_peer_request": 10, 2024-04-23T14:59:49.352 INFO:teuthology.orchestra.run.smithi203.stdout: "req_create_latency": { 2024-04-23T14:59:49.352 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 526, 2024-04-23T14:59:49.352 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.499409328, 2024-04-23T14:59:49.352 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000949447 2024-04-23T14:59:49.352 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.352 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getattr_latency": { 2024-04-23T14:59:49.352 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 7, 2024-04-23T14:59:49.352 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.005308326, 2024-04-23T14:59:49.353 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000758332 2024-04-23T14:59:49.353 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.353 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getfilelock_latency": { 2024-04-23T14:59:49.353 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.353 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.353 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.353 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.353 INFO:teuthology.orchestra.run.smithi203.stdout: "req_link_latency": { 2024-04-23T14:59:49.353 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.353 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.354 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.354 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.354 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookup_latency": { 2024-04-23T14:59:49.354 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 19, 2024-04-23T14:59:49.354 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.005879310, 2024-04-23T14:59:49.354 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000309437 2024-04-23T14:59:49.354 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.354 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookuphash_latency": { 2024-04-23T14:59:49.354 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.355 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.355 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.355 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.355 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupino_latency": { 2024-04-23T14:59:49.355 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.355 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.355 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.355 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.355 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupname_latency": { 2024-04-23T14:59:49.355 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.356 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.356 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.356 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.356 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupparent_latency": { 2024-04-23T14:59:49.356 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.356 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.356 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.356 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.356 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lookupsnap_latency": { 2024-04-23T14:59:49.356 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.357 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.357 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.357 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.357 INFO:teuthology.orchestra.run.smithi203.stdout: "req_lssnap_latency": { 2024-04-23T14:59:49.357 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.358 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.358 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.358 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.358 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mkdir_latency": { 2024-04-23T14:59:49.358 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 17, 2024-04-23T14:59:49.358 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.016280838, 2024-04-23T14:59:49.358 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000957696 2024-04-23T14:59:49.358 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.358 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mknod_latency": { 2024-04-23T14:59:49.359 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.359 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.359 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.359 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.359 INFO:teuthology.orchestra.run.smithi203.stdout: "req_mksnap_latency": { 2024-04-23T14:59:49.359 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.359 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.359 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.359 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.360 INFO:teuthology.orchestra.run.smithi203.stdout: "req_open_latency": { 2024-04-23T14:59:49.360 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 25, 2024-04-23T14:59:49.360 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.006293936, 2024-04-23T14:59:49.360 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000251757 2024-04-23T14:59:49.360 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.360 INFO:teuthology.orchestra.run.smithi203.stdout: "req_readdir_latency": { 2024-04-23T14:59:49.360 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 141, 2024-04-23T14:59:49.360 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.643873269, 2024-04-23T14:59:49.360 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.004566477 2024-04-23T14:59:49.360 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.361 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rename_latency": { 2024-04-23T14:59:49.361 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 33, 2024-04-23T14:59:49.361 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.054875326, 2024-04-23T14:59:49.361 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001662888 2024-04-23T14:59:49.361 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.361 INFO:teuthology.orchestra.run.smithi203.stdout: "req_renamesnap_latency": { 2024-04-23T14:59:49.361 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.361 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.362 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.362 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.362 INFO:teuthology.orchestra.run.smithi203.stdout: "req_snapdiff_latency": { 2024-04-23T14:59:49.362 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.362 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.362 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.362 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.362 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmdir_latency": { 2024-04-23T14:59:49.362 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 12, 2024-04-23T14:59:49.363 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.094044975, 2024-04-23T14:59:49.363 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.007837081 2024-04-23T14:59:49.363 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.363 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmsnap_latency": { 2024-04-23T14:59:49.363 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.363 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.363 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.363 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.363 INFO:teuthology.orchestra.run.smithi203.stdout: "req_rmxattr_latency": { 2024-04-23T14:59:49.363 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.364 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.364 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.364 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.364 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setattr_latency": { 2024-04-23T14:59:49.364 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 5, 2024-04-23T14:59:49.364 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.027195206, 2024-04-23T14:59:49.364 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.005439041 2024-04-23T14:59:49.364 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.364 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setdirlayout_latency": { 2024-04-23T14:59:49.364 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.365 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.365 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.365 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.365 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setfilelock_latency": { 2024-04-23T14:59:49.365 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 2, 2024-04-23T14:59:49.365 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000429165, 2024-04-23T14:59:49.365 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000214582 2024-04-23T14:59:49.365 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.365 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setlayout_latency": { 2024-04-23T14:59:49.366 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.366 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.366 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.366 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.366 INFO:teuthology.orchestra.run.smithi203.stdout: "req_setxattr_latency": { 2024-04-23T14:59:49.366 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.366 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.366 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.366 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.367 INFO:teuthology.orchestra.run.smithi203.stdout: "req_symlink_latency": { 2024-04-23T14:59:49.367 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 10, 2024-04-23T14:59:49.367 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.006168678, 2024-04-23T14:59:49.367 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000616867 2024-04-23T14:59:49.367 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.367 INFO:teuthology.orchestra.run.smithi203.stdout: "req_unlink_latency": { 2024-04-23T14:59:49.367 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 339, 2024-04-23T14:59:49.367 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.486594815, 2024-04-23T14:59:49.367 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.001435382 2024-04-23T14:59:49.367 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.368 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_revoke_eviction": 0, 2024-04-23T14:59:49.368 INFO:teuthology.orchestra.run.smithi203.stdout: "cap_acquisition_throttle": 0, 2024-04-23T14:59:49.368 INFO:teuthology.orchestra.run.smithi203.stdout: "req_getvxattr_latency": { 2024-04-23T14:59:49.368 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.368 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.368 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.368 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:49.368 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.368 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_sessions": { 2024-04-23T14:59:49.369 INFO:teuthology.orchestra.run.smithi203.stdout: "session_count": 1, 2024-04-23T14:59:49.369 INFO:teuthology.orchestra.run.smithi203.stdout: "session_add": 4, 2024-04-23T14:59:49.369 INFO:teuthology.orchestra.run.smithi203.stdout: "session_remove": 3, 2024-04-23T14:59:49.369 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_open": 1, 2024-04-23T14:59:49.369 INFO:teuthology.orchestra.run.smithi203.stdout: "sessions_stale": 0, 2024-04-23T14:59:49.369 INFO:teuthology.orchestra.run.smithi203.stdout: "total_load": 213, 2024-04-23T14:59:49.369 INFO:teuthology.orchestra.run.smithi203.stdout: "average_load": 213, 2024-04-23T14:59:49.369 INFO:teuthology.orchestra.run.smithi203.stdout: "avg_session_uptime": 2861, 2024-04-23T14:59:49.369 INFO:teuthology.orchestra.run.smithi203.stdout: "mdthresh_evicted": 0 2024-04-23T14:59:49.369 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.370 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-04-23T14:59:49.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-04-23T14:59:49.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-04-23T14:59:49.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T14:59:49.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-04-23T14:59:49.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T14:59:49.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-04-23T14:59:49.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T14:59:49.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T14:59:49.370 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T14:59:49.371 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T14:59:49.371 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T14:59:49.371 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-04-23T14:59:49.371 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T14:59:49.371 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T14:59:49.371 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-04-23T14:59:49.371 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-04-23T14:59:49.371 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-04-23T14:59:49.371 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-04-23T14:59:49.372 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T14:59:49.372 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T14:59:49.372 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T14:59:49.372 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T14:59:49.372 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T14:59:49.372 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-04-23T14:59:49.372 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-04-23T14:59:49.372 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-04-23T14:59:49.372 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T14:59:49.373 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T14:59:49.373 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-04-23T14:59:49.373 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-04-23T14:59:49.373 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-04-23T14:59:49.373 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-04-23T14:59:49.373 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T14:59:49.373 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-04-23T14:59:49.373 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T14:59:49.373 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-04-23T14:59:49.373 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 322594, 2024-04-23T14:59:49.374 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 527, 2024-04-23T14:59:49.374 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 880, 2024-04-23T14:59:49.374 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 10, 2024-04-23T14:59:49.374 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-04-23T14:59:49.374 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-04-23T14:59:49.374 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-04-23T14:59:49.374 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-04-23T14:59:49.374 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-04-23T14:59:49.374 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-04-23T14:59:49.374 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 8128, 2024-04-23T14:59:49.375 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 169, 2024-04-23T14:59:49.375 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T14:59:49.375 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-04-23T14:59:49.375 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-04-23T14:59:49.375 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-04-23T14:59:49.375 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 2691686, 2024-04-23T14:59:49.375 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 32842, 2024-04-23T14:59:49.375 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-04-23T14:59:49.375 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-04-23T14:59:49.375 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-04-23T14:59:49.376 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-04-23T14:59:49.376 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.376 INFO:teuthology.orchestra.run.smithi203.stdout: "objecter": { 2024-04-23T14:59:49.376 INFO:teuthology.orchestra.run.smithi203.stdout: "op_active": 0, 2024-04-23T14:59:49.376 INFO:teuthology.orchestra.run.smithi203.stdout: "op_laggy": 0, 2024-04-23T14:59:49.376 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send": 8004, 2024-04-23T14:59:49.376 INFO:teuthology.orchestra.run.smithi203.stdout: "op_send_bytes": 20795713, 2024-04-23T14:59:49.376 INFO:teuthology.orchestra.run.smithi203.stdout: "op_resend": 28, 2024-04-23T14:59:49.376 INFO:teuthology.orchestra.run.smithi203.stdout: "op_reply": 8004, 2024-04-23T14:59:49.376 INFO:teuthology.orchestra.run.smithi203.stdout: "op_latency": { 2024-04-23T14:59:49.377 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 8004, 2024-04-23T14:59:49.377 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 109.979166053, 2024-04-23T14:59:49.377 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.013740525 2024-04-23T14:59:49.377 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.377 INFO:teuthology.orchestra.run.smithi203.stdout: "op_inflight": 0, 2024-04-23T14:59:49.377 INFO:teuthology.orchestra.run.smithi203.stdout: "oplen_avg": { 2024-04-23T14:59:49.377 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 8004, 2024-04-23T14:59:49.377 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 8244 2024-04-23T14:59:49.377 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.377 INFO:teuthology.orchestra.run.smithi203.stdout: "op": 8004, 2024-04-23T14:59:49.378 INFO:teuthology.orchestra.run.smithi203.stdout: "op_r": 5692, 2024-04-23T14:59:49.378 INFO:teuthology.orchestra.run.smithi203.stdout: "op_w": 2312, 2024-04-23T14:59:49.378 INFO:teuthology.orchestra.run.smithi203.stdout: "op_rmw": 0, 2024-04-23T14:59:49.378 INFO:teuthology.orchestra.run.smithi203.stdout: "op_pg": 0, 2024-04-23T14:59:49.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_stat": 0, 2024-04-23T14:59:49.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_create": 10, 2024-04-23T14:59:49.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_read": 10, 2024-04-23T14:59:49.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_write": 1693, 2024-04-23T14:59:49.378 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writefull": 30, 2024-04-23T14:59:49.379 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_writesame": 0, 2024-04-23T14:59:49.379 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_append": 0, 2024-04-23T14:59:49.379 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_zero": 0, 2024-04-23T14:59:49.379 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_truncate": 1, 2024-04-23T14:59:49.379 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_delete": 391, 2024-04-23T14:59:49.379 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_mapext": 0, 2024-04-23T14:59:49.379 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_sparse_read": 0, 2024-04-23T14:59:49.379 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_clonerange": 0, 2024-04-23T14:59:49.379 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_getxattr": 5682, 2024-04-23T14:59:49.380 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_setxattr": 20, 2024-04-23T14:59:49.380 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_cmpxattr": 0, 2024-04-23T14:59:49.380 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_rmxattr": 0, 2024-04-23T14:59:49.380 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_resetxattrs": 0, 2024-04-23T14:59:49.380 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_call": 0, 2024-04-23T14:59:49.380 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_watch": 0, 2024-04-23T14:59:49.380 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_notify": 0, 2024-04-23T14:59:49.380 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T14:59:49.380 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls": 0, 2024-04-23T14:59:49.381 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_pgls_filter": 0, 2024-04-23T14:59:49.381 INFO:teuthology.orchestra.run.smithi203.stdout: "osdop_other": 5, 2024-04-23T14:59:49.381 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_active": 0, 2024-04-23T14:59:49.381 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_send": 0, 2024-04-23T14:59:49.381 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_resend": 0, 2024-04-23T14:59:49.381 INFO:teuthology.orchestra.run.smithi203.stdout: "linger_ping": 0, 2024-04-23T14:59:49.381 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_active": 0, 2024-04-23T14:59:49.381 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_send": 0, 2024-04-23T14:59:49.382 INFO:teuthology.orchestra.run.smithi203.stdout: "poolop_resend": 0, 2024-04-23T14:59:49.382 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_active": 0, 2024-04-23T14:59:49.382 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_send": 0, 2024-04-23T14:59:49.382 INFO:teuthology.orchestra.run.smithi203.stdout: "poolstat_resend": 0, 2024-04-23T14:59:49.382 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_active": 0, 2024-04-23T14:59:49.382 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_send": 0, 2024-04-23T14:59:49.382 INFO:teuthology.orchestra.run.smithi203.stdout: "statfs_resend": 0, 2024-04-23T14:59:49.382 INFO:teuthology.orchestra.run.smithi203.stdout: "command_active": 0, 2024-04-23T14:59:49.383 INFO:teuthology.orchestra.run.smithi203.stdout: "command_send": 0, 2024-04-23T14:59:49.383 INFO:teuthology.orchestra.run.smithi203.stdout: "command_resend": 0, 2024-04-23T14:59:49.383 INFO:teuthology.orchestra.run.smithi203.stdout: "map_epoch": 99, 2024-04-23T14:59:49.383 INFO:teuthology.orchestra.run.smithi203.stdout: "map_full": 0, 2024-04-23T14:59:49.383 INFO:teuthology.orchestra.run.smithi203.stdout: "map_inc": 13, 2024-04-23T14:59:49.383 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_sessions": 12, 2024-04-23T14:59:49.383 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_open": 12, 2024-04-23T14:59:49.383 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_session_close": 0, 2024-04-23T14:59:49.383 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_laggy": 0, 2024-04-23T14:59:49.383 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_wr": 282, 2024-04-23T14:59:49.384 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_rd": 0, 2024-04-23T14:59:49.384 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_del": 120 2024-04-23T14:59:49.384 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.384 INFO:teuthology.orchestra.run.smithi203.stdout: "oft": { 2024-04-23T14:59:49.384 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_objs": 1, 2024-04-23T14:59:49.384 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_kv_pairs": 0, 2024-04-23T14:59:49.384 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_updates": 0, 2024-04-23T14:59:49.384 INFO:teuthology.orchestra.run.smithi203.stdout: "omap_total_removes": 0 2024-04-23T14:59:49.384 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.385 INFO:teuthology.orchestra.run.smithi203.stdout: "osd-slow-ops": { 2024-04-23T14:59:49.385 INFO:teuthology.orchestra.run.smithi203.stdout: "slow_ops_count": 0 2024-04-23T14:59:49.385 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.385 INFO:teuthology.orchestra.run.smithi203.stdout: "purge_queue": { 2024-04-23T14:59:49.385 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops": 0, 2024-04-23T14:59:49.385 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_ops_high_water": 10, 2024-04-23T14:59:49.385 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing": 0, 2024-04-23T14:59:49.385 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executing_high_water": 10, 2024-04-23T14:59:49.385 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed_ops": 375, 2024-04-23T14:59:49.386 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_executed": 375, 2024-04-23T14:59:49.386 INFO:teuthology.orchestra.run.smithi203.stdout: "pq_item_in_journal": 0 2024-04-23T14:59:49.386 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.386 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T14:59:49.386 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:49.386 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-04-23T14:59:49.386 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:49.386 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 17572, 2024-04-23T14:59:49.386 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 5613488, 2024-04-23T14:59:49.387 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:49.387 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 17572, 2024-04-23T14:59:49.387 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-04-23T14:59:49.387 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-04-23T14:59:49.387 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 17572, 2024-04-23T14:59:49.387 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 5613488, 2024-04-23T14:59:49.387 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:49.387 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.387 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.388 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.388 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:49.388 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.388 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_bytes": { 2024-04-23T14:59:49.388 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:49.388 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-04-23T14:59:49.388 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:49.388 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-04-23T14:59:49.388 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-04-23T14:59:49.389 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:49.389 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:49.389 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 8004, 2024-04-23T14:59:49.389 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 20833588, 2024-04-23T14:59:49.389 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 8004, 2024-04-23T14:59:49.389 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 20833588, 2024-04-23T14:59:49.389 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:49.389 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.389 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.389 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.390 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:49.390 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.390 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-objecter_ops": { 2024-04-23T14:59:49.390 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:49.390 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 1024, 2024-04-23T14:59:49.390 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:49.390 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 0, 2024-04-23T14:59:49.390 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 0, 2024-04-23T14:59:49.390 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:49.391 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 0, 2024-04-23T14:59:49.391 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 8004, 2024-04-23T14:59:49.391 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 8004, 2024-04-23T14:59:49.391 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 8004, 2024-04-23T14:59:49.391 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 8004, 2024-04-23T14:59:49.391 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:49.391 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.391 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.391 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.391 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:49.392 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.392 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle": { 2024-04-23T14:59:49.392 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:49.392 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-04-23T14:59:49.392 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:49.392 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 375, 2024-04-23T14:59:49.392 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 37875, 2024-04-23T14:59:49.392 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:49.392 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 375, 2024-04-23T14:59:49.392 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-04-23T14:59:49.393 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-04-23T14:59:49.393 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 10, 2024-04-23T14:59:49.393 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 37875, 2024-04-23T14:59:49.393 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:49.393 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.393 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.393 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.393 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:49.393 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:49.393 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-write_buf_throttle-0x55eb3de34500": { 2024-04-23T14:59:49.394 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:49.394 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 3758096384, 2024-04-23T14:59:49.394 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:49.394 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 7505, 2024-04-23T14:59:49.394 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 20464449, 2024-04-23T14:59:49.394 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:49.394 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 7505, 2024-04-23T14:59:49.394 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-04-23T14:59:49.394 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-04-23T14:59:49.395 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 1683, 2024-04-23T14:59:49.395 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 20464449, 2024-04-23T14:59:49.395 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:49.395 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:49.395 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:49.395 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:49.395 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:49.395 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:49.395 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-04-23T14:59:49.693 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:49 smithi203 ceph-mon[32800]: pgmap v354: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:49.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:49 smithi083 ceph-mon[27867]: pgmap v354: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:49.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:49 smithi173 ceph-mon[32443]: pgmap v354: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:50.221 INFO:tasks.check_counter:Daemon mds.i mds.exported=5 2024-04-23T14:59:50.221 INFO:tasks.check_counter:Daemon mds.i mds.imported=69 2024-04-23T14:59:50.221 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-23T14:59:50.221 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-23T14:59:50.221 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T14:59:50.221 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=1 2024-04-23T14:59:50.222 DEBUG:tasks.check_counter:Getting stats from l 2024-04-23T14:59:50.222 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0 shell --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-23T14:59:50.465 INFO:teuthology.orchestra.run.smithi203.stderr:Inferring config /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T14:59:51.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:51 smithi203 ceph-mon[32800]: pgmap v355: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:59:51.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:51 smithi083 ceph-mon[27867]: pgmap v355: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:59:51.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:51 smithi173 ceph-mon[32443]: pgmap v355: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T14:59:52.347 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-04-23T14:59:52.347 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T14:59:52.347 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 4, 2024-04-23T14:59:52.347 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 4, 2024-04-23T14:59:52.347 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 3024, 2024-04-23T14:59:52.347 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 448, 2024-04-23T14:59:52.347 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 4, 2024-04-23T14:59:52.348 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 0, 2024-04-23T14:59:52.348 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.001335855, 2024-04-23T14:59:52.348 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.000043975, 2024-04-23T14:59:52.348 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.000513318, 2024-04-23T14:59:52.348 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T14:59:52.348 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:52.348 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 3, 2024-04-23T14:59:52.348 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000674398, 2024-04-23T14:59:52.349 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000224799 2024-04-23T14:59:52.349 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:52.349 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:52.349 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:52.349 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:52.349 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:52.349 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:52.349 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-23T14:59:52.349 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T14:59:52.350 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:52.350 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T14:59:52.350 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 3, 2024-04-23T14:59:52.350 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 181, 2024-04-23T14:59:52.350 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 480, 2024-04-23T14:59:52.350 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 266400, 2024-04-23T14:59:52.350 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 6, 2024-04-23T14:59:52.350 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 3, 2024-04-23T14:59:52.350 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.012254608, 2024-04-23T14:59:52.351 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.008214140, 2024-04-23T14:59:52.351 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.001608716, 2024-04-23T14:59:52.351 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T14:59:52.351 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:52.351 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 177, 2024-04-23T14:59:52.351 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.028065707, 2024-04-23T14:59:52.351 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000158563 2024-04-23T14:59:52.351 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:52.351 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:52.352 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:52.352 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:52.352 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:52.352 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:52.352 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-23T14:59:52.352 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 266400 2024-04-23T14:59:52.352 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:52.352 INFO:teuthology.orchestra.run.smithi203.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T14:59:52.352 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_messages": 244, 2024-04-23T14:59:52.353 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages": 233, 2024-04-23T14:59:52.353 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_bytes": 1595888, 2024-04-23T14:59:52.353 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_bytes": 143904, 2024-04-23T14:59:52.353 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_created_connections": 2, 2024-04-23T14:59:52.353 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_active_connections": 1, 2024-04-23T14:59:52.353 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_total_time": 0.046704981, 2024-04-23T14:59:52.353 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_send_time": 0.014353981, 2024-04-23T14:59:52.353 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_recv_time": 0.034479554, 2024-04-23T14:59:52.353 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_running_fast_dispatch_time": 0.005946328, 2024-04-23T14:59:52.354 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T14:59:52.354 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 232, 2024-04-23T14:59:52.354 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.013085170, 2024-04-23T14:59:52.354 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000056401 2024-04-23T14:59:52.354 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:52.354 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_handle_ack_lat": { 2024-04-23T14:59:52.354 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:52.354 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:52.354 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:52.355 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:52.355 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_recv_encrypted_bytes": 1595888, 2024-04-23T14:59:52.355 INFO:teuthology.orchestra.run.smithi203.stdout: "msgr_send_encrypted_bytes": 143904 2024-04-23T14:59:52.355 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:52.355 INFO:teuthology.orchestra.run.smithi203.stdout: "cct": { 2024-04-23T14:59:52.355 INFO:teuthology.orchestra.run.smithi203.stdout: "total_workers": 0, 2024-04-23T14:59:52.355 INFO:teuthology.orchestra.run.smithi203.stdout: "unhealthy_workers": 0 2024-04-23T14:59:52.355 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:52.355 INFO:teuthology.orchestra.run.smithi203.stdout: "mempool": { 2024-04-23T14:59:52.356 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_bytes": 0, 2024-04-23T14:59:52.356 INFO:teuthology.orchestra.run.smithi203.stdout: "bloom_filter_items": 0, 2024-04-23T14:59:52.356 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T14:59:52.356 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_alloc_items": 0, 2024-04-23T14:59:52.356 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T14:59:52.356 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_data_items": 0, 2024-04-23T14:59:52.356 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T14:59:52.356 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T14:59:52.356 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T14:59:52.357 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T14:59:52.357 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T14:59:52.357 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_other_items": 0, 2024-04-23T14:59:52.357 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T14:59:52.357 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T14:59:52.357 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_bytes": 0, 2024-04-23T14:59:52.357 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_extent_items": 0, 2024-04-23T14:59:52.357 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_bytes": 0, 2024-04-23T14:59:52.357 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_blob_items": 0, 2024-04-23T14:59:52.358 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T14:59:52.358 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T14:59:52.358 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T14:59:52.358 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T14:59:52.358 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T14:59:52.358 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_fsck_items": 0, 2024-04-23T14:59:52.358 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_bytes": 0, 2024-04-23T14:59:52.358 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_txc_items": 0, 2024-04-23T14:59:52.358 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T14:59:52.359 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T14:59:52.359 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_bytes": 0, 2024-04-23T14:59:52.359 INFO:teuthology.orchestra.run.smithi203.stdout: "bluestore_writing_items": 0, 2024-04-23T14:59:52.359 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_bytes": 0, 2024-04-23T14:59:52.359 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_items": 0, 2024-04-23T14:59:52.359 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T14:59:52.359 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_reader_items": 0, 2024-04-23T14:59:52.359 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T14:59:52.359 INFO:teuthology.orchestra.run.smithi203.stdout: "bluefs_file_writer_items": 0, 2024-04-23T14:59:52.360 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_bytes": 148714, 2024-04-23T14:59:52.360 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_anon_items": 30, 2024-04-23T14:59:52.360 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_bytes": 0, 2024-04-23T14:59:52.360 INFO:teuthology.orchestra.run.smithi203.stdout: "buffer_meta_items": 0, 2024-04-23T14:59:52.360 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_bytes": 0, 2024-04-23T14:59:52.360 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_items": 0, 2024-04-23T14:59:52.360 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_bytes": 0, 2024-04-23T14:59:52.360 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_mapbl_items": 0, 2024-04-23T14:59:52.360 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_bytes": 0, 2024-04-23T14:59:52.360 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_pglog_items": 0, 2024-04-23T14:59:52.361 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_bytes": 0, 2024-04-23T14:59:52.361 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_items": 0, 2024-04-23T14:59:52.361 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T14:59:52.361 INFO:teuthology.orchestra.run.smithi203.stdout: "osdmap_mapping_items": 0, 2024-04-23T14:59:52.361 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_bytes": 0, 2024-04-23T14:59:52.361 INFO:teuthology.orchestra.run.smithi203.stdout: "pgmap_items": 0, 2024-04-23T14:59:52.361 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_bytes": 928, 2024-04-23T14:59:52.361 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_co_items": 2, 2024-04-23T14:59:52.361 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_bytes": 0, 2024-04-23T14:59:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_1_items": 0, 2024-04-23T14:59:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_bytes": 0, 2024-04-23T14:59:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: "unittest_2_items": 0 2024-04-23T14:59:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-23T14:59:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T14:59:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: "val": 0, 2024-04-23T14:59:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: "max": 104857600, 2024-04-23T14:59:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: "get_started": 0, 2024-04-23T14:59:52.362 INFO:teuthology.orchestra.run.smithi203.stdout: "get": 247, 2024-04-23T14:59:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: "get_sum": 1561486, 2024-04-23T14:59:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_fail": 0, 2024-04-23T14:59:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: "get_or_fail_success": 247, 2024-04-23T14:59:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: "take": 0, 2024-04-23T14:59:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: "take_sum": 0, 2024-04-23T14:59:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: "put": 247, 2024-04-23T14:59:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: "put_sum": 1561486, 2024-04-23T14:59:52.363 INFO:teuthology.orchestra.run.smithi203.stdout: "wait": { 2024-04-23T14:59:52.364 INFO:teuthology.orchestra.run.smithi203.stdout: "avgcount": 0, 2024-04-23T14:59:52.364 INFO:teuthology.orchestra.run.smithi203.stdout: "sum": 0.000000000, 2024-04-23T14:59:52.364 INFO:teuthology.orchestra.run.smithi203.stdout: "avgtime": 0.000000000 2024-04-23T14:59:52.364 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:52.364 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-23T14:59:52.364 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-04-23T14:59:53.214 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-23T14:59:53.215 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-23T14:59:53.215 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-23T14:59:53.215 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-23T14:59:53.215 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-23T14:59:53.215 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-23T14:59:53.215 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-23T14:59:53.227 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-23T14:59:53.227 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:59:53.227 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:59:53.301 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-23T14:59:53.301 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T14:59:53.353 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:59:53.612 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.612+0000 7f509fe00640 1 -- 172.21.15.83:0/2361075164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5098104df0 msgr2=0x7f5098108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.612+0000 7f509fe00640 1 --2- 172.21.15.83:0/2361075164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5098104df0 0x7f5098108e30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5084009900 tx=0x7f508402a400 comp rx=0 tx=0).stop 2024-04-23T14:59:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.612+0000 7f509fe00640 1 -- 172.21.15.83:0/2361075164 shutdown_connections 2024-04-23T14:59:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.612+0000 7f509fe00640 1 --2- 172.21.15.83:0/2361075164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5098109510 0x7f50981110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.612+0000 7f509fe00640 1 --2- 172.21.15.83:0/2361075164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5098104df0 0x7f5098108e30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.612+0000 7f509fe00640 1 --2- 172.21.15.83:0/2361075164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5098104440 0x7f5098104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.612+0000 7f509fe00640 1 -- 172.21.15.83:0/2361075164 >> 172.21.15.83:0/2361075164 conn(0x7f50980ffcc0 msgr2=0x7f50981020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.613+0000 7f509fe00640 1 -- 172.21.15.83:0/2361075164 shutdown_connections 2024-04-23T14:59:53.613 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.613+0000 7f509fe00640 1 -- 172.21.15.83:0/2361075164 wait complete. 2024-04-23T14:59:53.614 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.614+0000 7f509fe00640 1 Processor -- start 2024-04-23T14:59:53.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.614+0000 7f509fe00640 1 -- start start 2024-04-23T14:59:53.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.614+0000 7f509fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5098104440 0x7f50981a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:53.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.614+0000 7f509fe00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5098109510 0x7f50981a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:53.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.614+0000 7f509fe00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509819a900 0x7f509819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:53.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.614+0000 7f509fe00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5098114580 con 0x7f509819a900 2024-04-23T14:59:53.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.614+0000 7f509fe00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5098114400 con 0x7f5098104440 2024-04-23T14:59:53.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.614+0000 7f509fe00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5098114700 con 0x7f5098109510 2024-04-23T14:59:53.615 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.615+0000 7f509ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5098109510 0x7f50981a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:53.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.615+0000 7f509ca00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5098109510 0x7f50981a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:45568/0 (socket says 172.21.15.83:45568) 2024-04-23T14:59:53.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.615+0000 7f509ca00640 1 -- 172.21.15.83:0/1046423655 learned_addr learned my addr 172.21.15.83:0/1046423655 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:53.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.615+0000 7f509de00640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509819a900 0x7f509819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:53.616 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.615+0000 7f509d400640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5098104440 0x7f50981a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.615+0000 7f509de00640 1 -- 172.21.15.83:0/1046423655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5098109510 msgr2=0x7f50981a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.615+0000 7f509de00640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5098109510 0x7f50981a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.615+0000 7f509de00640 1 -- 172.21.15.83:0/1046423655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5098104440 msgr2=0x7f50981a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.615+0000 7f509de00640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5098104440 0x7f50981a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.615+0000 7f509de00640 1 -- 172.21.15.83:0/1046423655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509819b420 con 0x7f509819a900 2024-04-23T14:59:53.617 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.616+0000 7f509de00640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509819a900 0x7f509819adb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f509000e970 tx=0x7f509000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.616+0000 7f5096000640 1 -- 172.21.15.83:0/1046423655 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509000ccc0 con 0x7f509819a900 2024-04-23T14:59:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.616+0000 7f509fe00640 1 -- 172.21.15.83:0/1046423655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509819b710 con 0x7f509819a900 2024-04-23T14:59:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.616+0000 7f5096000640 1 -- 172.21.15.83:0/1046423655 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5090005020 con 0x7f509819a900 2024-04-23T14:59:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.616+0000 7f509fe00640 1 -- 172.21.15.83:0/1046423655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50981a7960 con 0x7f509819a900 2024-04-23T14:59:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.616+0000 7f5096000640 1 -- 172.21.15.83:0/1046423655 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509001c070 con 0x7f509819a900 2024-04-23T14:59:53.618 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.617+0000 7f509fe00640 1 -- 172.21.15.83:0/1046423655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5098069270 con 0x7f509819a900 2024-04-23T14:59:53.619 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.619+0000 7f5096000640 1 -- 172.21.15.83:0/1046423655 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f50900051d0 con 0x7f509819a900 2024-04-23T14:59:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.621+0000 7f5096000640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f50800a3d20 0x7f50800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.621+0000 7f5096000640 1 -- 172.21.15.83:0/1046423655 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f50900d0b40 con 0x7f509819a900 2024-04-23T14:59:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.621+0000 7f509d400640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f50800a3d20 0x7f50800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:53.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.622+0000 7f509d400640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f50800a3d20 0x7f50800a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f509819ba40 tx=0x7f508c009500 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:53.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.624+0000 7f5096000640 1 -- 172.21.15.83:0/1046423655 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f509009d170 con 0x7f509819a900 2024-04-23T14:59:53.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:53 smithi203 ceph-mon[32800]: pgmap v356: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:59:53.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:53 smithi083 ceph-mon[27867]: pgmap v356: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:59:53.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:53 smithi173 ceph-mon[32443]: pgmap v356: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T14:59:53.830 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.830+0000 7f509fe00640 1 -- 172.21.15.83:0/1046423655 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f509819be30 con 0x7f509819a900 2024-04-23T14:59:53.834 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.835+0000 7f5096000640 1 -- 172.21.15.83:0/1046423655 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23166 (secure 0 0 0) 0x7f509819be30 con 0x7f509819a900 2024-04-23T14:59:53.835 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:59:53.835 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":99,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:59:17.356224+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/622708991":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/1081276647":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/4179157882":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6801/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/2255156114":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:59:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.838+0000 7f509fe00640 1 -- 172.21.15.83:0/1046423655 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f50800a3d20 msgr2=0x7f50800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.838+0000 7f509fe00640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f50800a3d20 0x7f50800a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f509819ba40 tx=0x7f508c009500 comp rx=0 tx=0).stop 2024-04-23T14:59:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.838+0000 7f509fe00640 1 -- 172.21.15.83:0/1046423655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509819a900 msgr2=0x7f509819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.838+0000 7f509fe00640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509819a900 0x7f509819adb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f509000e970 tx=0x7f509000ee30 comp rx=0 tx=0).stop 2024-04-23T14:59:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.838+0000 7f509fe00640 1 -- 172.21.15.83:0/1046423655 shutdown_connections 2024-04-23T14:59:53.838 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.838+0000 7f509fe00640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f50800a3d20 0x7f50800a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.838+0000 7f509fe00640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f509819a900 0x7f509819adb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.838+0000 7f509fe00640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5098109510 0x7f50981a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.838+0000 7f509fe00640 1 --2- 172.21.15.83:0/1046423655 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5098104440 0x7f50981a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.838+0000 7f509fe00640 1 -- 172.21.15.83:0/1046423655 >> 172.21.15.83:0/1046423655 conn(0x7f50980ffcc0 msgr2=0x7f50981010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.838+0000 7f509fe00640 1 -- 172.21.15.83:0/1046423655 shutdown_connections 2024-04-23T14:59:53.839 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:53.838+0000 7f509fe00640 1 -- 172.21.15.83:0/1046423655 wait complete. 2024-04-23T14:59:53.859 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T14:40:41.241992+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T14:45:57.341602+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T14:45:58.654798+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T14:59:53.859 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T14:59:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.068+0000 7f384aa00640 1 -- 172.21.15.83:0/2090325793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f384410e660 msgr2=0x7f384410ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.068+0000 7f384aa00640 1 --2- 172.21.15.83:0/2090325793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f384410e660 0x7f384410ea40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3834009930 tx=0x7f383402a420 comp rx=0 tx=0).stop 2024-04-23T14:59:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.068+0000 7f384aa00640 1 -- 172.21.15.83:0/2090325793 shutdown_connections 2024-04-23T14:59:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.068+0000 7f384aa00640 1 --2- 172.21.15.83:0/2090325793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3844108fd0 0x7f3844115490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.068+0000 7f384aa00640 1 --2- 172.21.15.83:0/2090325793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3844108610 0x7f3844108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.068+0000 7f384aa00640 1 --2- 172.21.15.83:0/2090325793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f384410e660 0x7f384410ea40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.068+0000 7f384aa00640 1 -- 172.21.15.83:0/2090325793 >> 172.21.15.83:0/2090325793 conn(0x7f3844076540 msgr2=0x7f3844078960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.069+0000 7f384aa00640 1 -- 172.21.15.83:0/2090325793 shutdown_connections 2024-04-23T14:59:54.070 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.069+0000 7f384aa00640 1 -- 172.21.15.83:0/2090325793 wait complete. 2024-04-23T14:59:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.069+0000 7f384aa00640 1 Processor -- start 2024-04-23T14:59:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f384aa00640 1 -- start start 2024-04-23T14:59:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f384aa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3844108610 0x7f384419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f384aa00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3844108fd0 0x7f384419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f384aa00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384410e660 0x7f38441a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f384aa00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3844118a30 con 0x7f3844108fd0 2024-04-23T14:59:54.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f384aa00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f38441188b0 con 0x7f384410e660 2024-04-23T14:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f384aa00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f3844118bb0 con 0x7f3844108610 2024-04-23T14:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f3843400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3844108fd0 0x7f384419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f3848a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384410e660 0x7f38441a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f3843e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3844108610 0x7f384419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f3848a00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384410e660 0x7f38441a3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:59104/0 (socket says 172.21.15.83:59104) 2024-04-23T14:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f3848a00640 1 -- 172.21.15.83:0/49641404 learned_addr learned my addr 172.21.15.83:0/49641404 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:54.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.070+0000 7f3843400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3844108fd0 0x7f384419f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33180/0 (socket says 172.21.15.83:33180) 2024-04-23T14:59:54.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.071+0000 7f3848a00640 1 -- 172.21.15.83:0/49641404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3844108610 msgr2=0x7f384419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.071+0000 7f3848a00640 1 --2- 172.21.15.83:0/49641404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3844108610 0x7f384419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.071+0000 7f3848a00640 1 -- 172.21.15.83:0/49641404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3844108fd0 msgr2=0x7f384419f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.071+0000 7f3848a00640 1 --2- 172.21.15.83:0/49641404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3844108fd0 0x7f384419f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.071+0000 7f3848a00640 1 -- 172.21.15.83:0/49641404 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38441a3ec0 con 0x7f384410e660 2024-04-23T14:59:54.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.071+0000 7f3848a00640 1 --2- 172.21.15.83:0/49641404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384410e660 0x7f38441a3790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f383c00efd0 tx=0x7f383c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:54.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.071+0000 7f3840c00640 1 -- 172.21.15.83:0/49641404 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c003b70 con 0x7f384410e660 2024-04-23T14:59:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.071+0000 7f384aa00640 1 -- 172.21.15.83:0/49641404 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38441a41b0 con 0x7f384410e660 2024-04-23T14:59:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.071+0000 7f3840c00640 1 -- 172.21.15.83:0/49641404 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c003d10 con 0x7f384410e660 2024-04-23T14:59:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.071+0000 7f3840c00640 1 -- 172.21.15.83:0/49641404 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0109f0 con 0x7f384410e660 2024-04-23T14:59:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.071+0000 7f384aa00640 1 -- 172.21.15.83:0/49641404 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38441aba40 con 0x7f384410e660 2024-04-23T14:59:54.076 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.072+0000 7f384aa00640 1 -- 172.21.15.83:0/49641404 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3844069270 con 0x7f384410e660 2024-04-23T14:59:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.074+0000 7f3840c00640 1 -- 172.21.15.83:0/49641404 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f383c010b90 con 0x7f384410e660 2024-04-23T14:59:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.076+0000 7f3840c00640 1 --2- 172.21.15.83:0/49641404 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38280a3d20 0x7f38280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.076+0000 7f3840c00640 1 -- 172.21.15.83:0/49641404 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f383c0d1110 con 0x7f384410e660 2024-04-23T14:59:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.076+0000 7f3843e00640 1 --2- 172.21.15.83:0/49641404 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38280a3d20 0x7f38280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:54.077 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.077+0000 7f3843e00640 1 --2- 172.21.15.83:0/49641404 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38280a3d20 0x7f38280a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3834009900 tx=0x7f3834000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:54.082 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.082+0000 7f3840c00640 1 -- 172.21.15.83:0/49641404 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f383c09d740 con 0x7f384410e660 2024-04-23T14:59:54.282 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.283+0000 7f384aa00640 1 -- 172.21.15.83:0/49641404 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3844110b90 con 0x7f384410e660 2024-04-23T14:59:54.283 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.284+0000 7f3840c00640 1 -- 172.21.15.83:0/49641404 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f383c09d040 con 0x7f384410e660 2024-04-23T14:59:54.284 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 1 2024-04-23T14:59:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.286+0000 7f384aa00640 1 -- 172.21.15.83:0/49641404 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38280a3d20 msgr2=0x7f38280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.286+0000 7f384aa00640 1 --2- 172.21.15.83:0/49641404 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38280a3d20 0x7f38280a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3834009900 tx=0x7f3834000bc0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.286+0000 7f384aa00640 1 -- 172.21.15.83:0/49641404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384410e660 msgr2=0x7f38441a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.286+0000 7f384aa00640 1 --2- 172.21.15.83:0/49641404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384410e660 0x7f38441a3790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f383c00efd0 tx=0x7f383c00c6a0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.286+0000 7f384aa00640 1 -- 172.21.15.83:0/49641404 shutdown_connections 2024-04-23T14:59:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.286+0000 7f384aa00640 1 --2- 172.21.15.83:0/49641404 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f384410e660 0x7f38441a3790 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.286+0000 7f384aa00640 1 --2- 172.21.15.83:0/49641404 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f38280a3d20 0x7f38280a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.287 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.286+0000 7f384aa00640 1 --2- 172.21.15.83:0/49641404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3844108fd0 0x7f384419f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.286+0000 7f384aa00640 1 --2- 172.21.15.83:0/49641404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3844108610 0x7f384419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.286+0000 7f384aa00640 1 -- 172.21.15.83:0/49641404 >> 172.21.15.83:0/49641404 conn(0x7f3844076540 msgr2=0x7f3844077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:54.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.287+0000 7f384aa00640 1 -- 172.21.15.83:0/49641404 shutdown_connections 2024-04-23T14:59:54.288 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.287+0000 7f384aa00640 1 -- 172.21.15.83:0/49641404 wait complete. 2024-04-23T14:59:54.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T14:59:54.511 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.511+0000 7f5e5de00640 1 -- 172.21.15.83:0/2680616557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e5810d500 msgr2=0x7f5e5810d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.511+0000 7f5e5de00640 1 --2- 172.21.15.83:0/2680616557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e5810d500 0x7f5e5810d8e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009900 tx=0x7f5e4802a410 comp rx=0 tx=0).stop 2024-04-23T14:59:54.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.511+0000 7f5e5de00640 1 -- 172.21.15.83:0/2680616557 shutdown_connections 2024-04-23T14:59:54.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.511+0000 7f5e5de00640 1 --2- 172.21.15.83:0/2680616557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e58108f40 0x7f5e581153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.511+0000 7f5e5de00640 1 --2- 172.21.15.83:0/2680616557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e58108580 0x7f5e58108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.512 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.511+0000 7f5e5de00640 1 --2- 172.21.15.83:0/2680616557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e5810d500 0x7f5e5810d8e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.511+0000 7f5e5de00640 1 -- 172.21.15.83:0/2680616557 >> 172.21.15.83:0/2680616557 conn(0x7f5e58077e20 msgr2=0x7f5e58106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:54.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.512+0000 7f5e5de00640 1 -- 172.21.15.83:0/2680616557 shutdown_connections 2024-04-23T14:59:54.513 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.512+0000 7f5e5de00640 1 -- 172.21.15.83:0/2680616557 wait complete. 2024-04-23T14:59:54.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.513+0000 7f5e5de00640 1 Processor -- start 2024-04-23T14:59:54.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.513+0000 7f5e5de00640 1 -- start start 2024-04-23T14:59:54.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.513+0000 7f5e5de00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e58108580 0x7f5e5819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:54.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.513+0000 7f5e5de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e58108f40 0x7f5e5819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:54.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.513+0000 7f5e5de00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e5810d500 0x7f5e581a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:54.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.513+0000 7f5e5de00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58118910 con 0x7f5e58108580 2024-04-23T14:59:54.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.513+0000 7f5e5de00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58118790 con 0x7f5e58108f40 2024-04-23T14:59:54.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.513+0000 7f5e5de00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58118a90 con 0x7f5e5810d500 2024-04-23T14:59:54.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.513+0000 7f5e5ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e58108580 0x7f5e5819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:54.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.513+0000 7f5e4fe00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e58108f40 0x7f5e5819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:54.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.513+0000 7f5e5ca00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e58108580 0x7f5e5819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:33200/0 (socket says 172.21.15.83:33200) 2024-04-23T14:59:54.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.514+0000 7f5e5ca00640 1 -- 172.21.15.83:0/2075563759 learned_addr learned my addr 172.21.15.83:0/2075563759 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:54.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.514+0000 7f5e5d400640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e5810d500 0x7f5e581a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:54.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.514+0000 7f5e4fe00640 1 -- 172.21.15.83:0/2075563759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e5810d500 msgr2=0x7f5e581a3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.514+0000 7f5e4fe00640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e5810d500 0x7f5e581a3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.514+0000 7f5e4fe00640 1 -- 172.21.15.83:0/2075563759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e58108580 msgr2=0x7f5e5819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.514+0000 7f5e4fe00640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e58108580 0x7f5e5819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.517 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.514+0000 7f5e4fe00640 1 -- 172.21.15.83:0/2075563759 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e581a3d40 con 0x7f5e58108f40 2024-04-23T14:59:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.514+0000 7f5e4fe00640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e58108f40 0x7f5e5819f290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e9f0 tx=0x7f5e4000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.514+0000 7f5e56000640 1 -- 172.21.15.83:0/2075563759 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000ccc0 con 0x7f5e58108f40 2024-04-23T14:59:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.514+0000 7f5e56000640 1 -- 172.21.15.83:0/2075563759 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e40005020 con 0x7f5e58108f40 2024-04-23T14:59:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.514+0000 7f5e56000640 1 -- 172.21.15.83:0/2075563759 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e400049e0 con 0x7f5e58108f40 2024-04-23T14:59:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.514+0000 7f5e5de00640 1 -- 172.21.15.83:0/2075563759 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e581a4030 con 0x7f5e58108f40 2024-04-23T14:59:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.515+0000 7f5e5de00640 1 -- 172.21.15.83:0/2075563759 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e581ab8c0 con 0x7f5e58108f40 2024-04-23T14:59:54.518 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.516+0000 7f5e5de00640 1 -- 172.21.15.83:0/2075563759 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e58069190 con 0x7f5e58108f40 2024-04-23T14:59:54.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.517+0000 7f5e56000640 1 -- 172.21.15.83:0/2075563759 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5e40005560 con 0x7f5e58108f40 2024-04-23T14:59:54.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.519+0000 7f5e56000640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5e3c0a3da0 0x7f5e3c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:54.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.519+0000 7f5e56000640 1 -- 172.21.15.83:0/2075563759 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f5e400d0750 con 0x7f5e58108f40 2024-04-23T14:59:54.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.519+0000 7f5e5ca00640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5e3c0a3da0 0x7f5e3c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:54.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.520+0000 7f5e5ca00640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5e3c0a3da0 0x7f5e3c0a6260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5e480096c0 tx=0x7f5e48005b50 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:54.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.523+0000 7f5e56000640 1 -- 172.21.15.83:0/2075563759 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e400051d0 con 0x7f5e58108f40 2024-04-23T14:59:54.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:54 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1046423655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:59:54.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:54 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/49641404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:59:54.722 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.723+0000 7f5e5de00640 1 -- 172.21.15.83:0/2075563759 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5e581abb20 con 0x7f5e58108f40 2024-04-23T14:59:54.723 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.723+0000 7f5e56000640 1 -- 172.21.15.83:0/2075563759 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f5e4009cd80 con 0x7f5e58108f40 2024-04-23T14:59:54.724 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 32 2024-04-23T14:59:54.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.726+0000 7f5e5de00640 1 -- 172.21.15.83:0/2075563759 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5e3c0a3da0 msgr2=0x7f5e3c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.726+0000 7f5e5de00640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5e3c0a3da0 0x7f5e3c0a6260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5e480096c0 tx=0x7f5e48005b50 comp rx=0 tx=0).stop 2024-04-23T14:59:54.726 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.726+0000 7f5e5de00640 1 -- 172.21.15.83:0/2075563759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e58108f40 msgr2=0x7f5e5819f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.726+0000 7f5e5de00640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e58108f40 0x7f5e5819f290 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000e9f0 tx=0x7f5e4000eeb0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.726+0000 7f5e5de00640 1 -- 172.21.15.83:0/2075563759 shutdown_connections 2024-04-23T14:59:54.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.726+0000 7f5e5de00640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e5810d500 0x7f5e581a3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.726+0000 7f5e5de00640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e58108f40 0x7f5e5819f290 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.726+0000 7f5e5de00640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5e3c0a3da0 0x7f5e3c0a6260 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.726+0000 7f5e5de00640 1 --2- 172.21.15.83:0/2075563759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e58108580 0x7f5e5819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.726+0000 7f5e5de00640 1 -- 172.21.15.83:0/2075563759 >> 172.21.15.83:0/2075563759 conn(0x7f5e58077e20 msgr2=0x7f5e580782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:54.727 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.726+0000 7f5e5de00640 1 -- 172.21.15.83:0/2075563759 shutdown_connections 2024-04-23T14:59:54.728 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.726+0000 7f5e5de00640 1 -- 172.21.15.83:0/2075563759 wait complete. 2024-04-23T14:59:54.743 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:54 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1046423655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:59:54.743 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:54 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/49641404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:59:54.745 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T14:59:54.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:54 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1046423655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:59:54.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:54 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/49641404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T14:59:54.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.950+0000 7f66a3e00640 1 -- 172.21.15.83:0/3724194437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a410c490 msgr2=0x7f66a410e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.950+0000 7f66a3e00640 1 --2- 172.21.15.83:0/3724194437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a410c490 0x7f66a410e910 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f669c009900 tx=0x7f669c02a480 comp rx=0 tx=0).stop 2024-04-23T14:59:54.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.950+0000 7f66a3e00640 1 -- 172.21.15.83:0/3724194437 shutdown_connections 2024-04-23T14:59:54.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.950+0000 7f66a3e00640 1 --2- 172.21.15.83:0/3724194437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a410c490 0x7f66a410e910 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.950+0000 7f66a3e00640 1 --2- 172.21.15.83:0/3724194437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41039e0 0x7f66a410bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.950+0000 7f66a3e00640 1 --2- 172.21.15.83:0/3724194437 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a41030c0 0x7f66a41034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.950+0000 7f66a3e00640 1 -- 172.21.15.83:0/3724194437 >> 172.21.15.83:0/3724194437 conn(0x7f66a40fc880 msgr2=0x7f66a40feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:54.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.950+0000 7f66a3e00640 1 -- 172.21.15.83:0/3724194437 shutdown_connections 2024-04-23T14:59:54.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.950+0000 7f66a3e00640 1 -- 172.21.15.83:0/3724194437 wait complete. 2024-04-23T14:59:54.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.951+0000 7f66a3e00640 1 Processor -- start 2024-04-23T14:59:54.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.951+0000 7f66a3e00640 1 -- start start 2024-04-23T14:59:54.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a3e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41030c0 0x7f66a41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:54.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a3e00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a41039e0 0x7f66a41a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:54.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a3e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a410c490 0x7f66a419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:54.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a3e00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4112280 con 0x7f66a41039e0 2024-04-23T14:59:54.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a3e00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4112100 con 0x7f66a410c490 2024-04-23T14:59:54.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a3e00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4112400 con 0x7f66a41030c0 2024-04-23T14:59:54.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f669a000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a41039e0 0x7f66a41a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:54.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a2a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41030c0 0x7f66a41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:54.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a410c490 0x7f66a419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a3400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a410c490 0x7f66a419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:47438/0 (socket says 172.21.15.83:47438) 2024-04-23T14:59:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a3400640 1 -- 172.21.15.83:0/899949214 learned_addr learned my addr 172.21.15.83:0/899949214 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a2a00640 1 -- 172.21.15.83:0/899949214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a410c490 msgr2=0x7f66a419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a2a00640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a410c490 0x7f66a419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a2a00640 1 -- 172.21.15.83:0/899949214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a41039e0 msgr2=0x7f66a41a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a2a00640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a41039e0 0x7f66a41a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f66a2a00640 1 -- 172.21.15.83:0/899949214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a419afe0 con 0x7f66a41030c0 2024-04-23T14:59:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.952+0000 7f669a000640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a41039e0 0x7f66a41a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:59:54.955 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.953+0000 7f66a2a00640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41030c0 0x7f66a41a0660 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f669400eb00 tx=0x7f669400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.953+0000 7f669be00640 1 -- 172.21.15.83:0/899949214 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694010430 con 0x7f66a41030c0 2024-04-23T14:59:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.953+0000 7f669be00640 1 -- 172.21.15.83:0/899949214 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6694010ab0 con 0x7f66a41030c0 2024-04-23T14:59:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.953+0000 7f66a3e00640 1 -- 172.21.15.83:0/899949214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a419b2d0 con 0x7f66a41030c0 2024-04-23T14:59:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.953+0000 7f669be00640 1 -- 172.21.15.83:0/899949214 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694017bb0 con 0x7f66a41030c0 2024-04-23T14:59:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.953+0000 7f66a3e00640 1 -- 172.21.15.83:0/899949214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a41a75e0 con 0x7f66a41030c0 2024-04-23T14:59:54.956 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.954+0000 7f66a3e00640 1 -- 172.21.15.83:0/899949214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a4069a10 con 0x7f66a41030c0 2024-04-23T14:59:54.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.956+0000 7f669be00640 1 -- 172.21.15.83:0/899949214 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6694010c60 con 0x7f66a41030c0 2024-04-23T14:59:54.957 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.958+0000 7f669be00640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f66880a3da0 0x7f66880a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.958+0000 7f669be00640 1 -- 172.21.15.83:0/899949214 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f66940d11b0 con 0x7f66a41030c0 2024-04-23T14:59:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.958+0000 7f669a000640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f66880a3da0 0x7f66880a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:54.958 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.959+0000 7f669a000640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f66880a3da0 0x7f66880a6260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f668c0095f0 tx=0x7f668c005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:54.960 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:54.960+0000 7f669be00640 1 -- 172.21.15.83:0/899949214 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f669409d7e0 con 0x7f66a41030c0 2024-04-23T14:59:55.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.164+0000 7f66a3e00640 1 -- 172.21.15.83:0/899949214 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f66a41a77b0 con 0x7f66a41030c0 2024-04-23T14:59:55.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.165+0000 7f669be00640 1 -- 172.21.15.83:0/899949214 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f669409d0e0 con 0x7f66a41030c0 2024-04-23T14:59:55.165 INFO:teuthology.orchestra.run.smithi083.stdout:pg_num: 64 2024-04-23T14:59:55.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.169+0000 7f66a3e00640 1 -- 172.21.15.83:0/899949214 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f66880a3da0 msgr2=0x7f66880a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:55.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.169+0000 7f66a3e00640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f66880a3da0 0x7f66880a6260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f668c0095f0 tx=0x7f668c005f90 comp rx=0 tx=0).stop 2024-04-23T14:59:55.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.169+0000 7f66a3e00640 1 -- 172.21.15.83:0/899949214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41030c0 msgr2=0x7f66a41a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:55.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.169+0000 7f66a3e00640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41030c0 0x7f66a41a0660 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f669400eb00 tx=0x7f669400c2d0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.169+0000 7f66a3e00640 1 -- 172.21.15.83:0/899949214 shutdown_connections 2024-04-23T14:59:55.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.169+0000 7f66a3e00640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a410c490 0x7f66a419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.169+0000 7f66a3e00640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f66880a3da0 0x7f66880a6260 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.169+0000 7f66a3e00640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f66a41039e0 0x7f66a41a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.169+0000 7f66a3e00640 1 --2- 172.21.15.83:0/899949214 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f66a41030c0 0x7f66a41a0660 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.169+0000 7f66a3e00640 1 -- 172.21.15.83:0/899949214 >> 172.21.15.83:0/899949214 conn(0x7f66a40fc880 msgr2=0x7f66a40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:55.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.169+0000 7f66a3e00640 1 -- 172.21.15.83:0/899949214 shutdown_connections 2024-04-23T14:59:55.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.169+0000 7f66a3e00640 1 -- 172.21.15.83:0/899949214 wait complete. 2024-04-23T14:59:55.188 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T14:59:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.392+0000 7fe89f200640 1 -- 172.21.15.83:0/2803296527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a0108580 msgr2=0x7fe8a0108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.392+0000 7fe89f200640 1 --2- 172.21.15.83:0/2803296527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a0108580 0x7fe8a0108a00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe888009930 tx=0x7fe88802a440 comp rx=0 tx=0).stop 2024-04-23T14:59:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.392+0000 7fe89f200640 1 -- 172.21.15.83:0/2803296527 shutdown_connections 2024-04-23T14:59:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.392+0000 7fe89f200640 1 --2- 172.21.15.83:0/2803296527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a0108f40 0x7fe8a01153b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.392+0000 7fe89f200640 1 --2- 172.21.15.83:0/2803296527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a0108580 0x7fe8a0108a00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.392+0000 7fe89f200640 1 --2- 172.21.15.83:0/2803296527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a010d500 0x7fe8a010d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.392+0000 7fe89f200640 1 -- 172.21.15.83:0/2803296527 >> 172.21.15.83:0/2803296527 conn(0x7fe8a0077e20 msgr2=0x7fe8a0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:55.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.392+0000 7fe89f200640 1 -- 172.21.15.83:0/2803296527 shutdown_connections 2024-04-23T14:59:55.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.392+0000 7fe89f200640 1 -- 172.21.15.83:0/2803296527 wait complete. 2024-04-23T14:59:55.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.393+0000 7fe89f200640 1 Processor -- start 2024-04-23T14:59:55.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.393+0000 7fe89f200640 1 -- start start 2024-04-23T14:59:55.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.393+0000 7fe89f200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a0108580 0x7fe8a019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:55.395 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.393+0000 7fe89f200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a0108f40 0x7fe8a019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:55.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89f200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a010d500 0x7fe8a01a3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:55.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89f200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0118900 con 0x7fe8a010d500 2024-04-23T14:59:55.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89f200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0118780 con 0x7fe8a0108580 2024-04-23T14:59:55.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89f200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0118a80 con 0x7fe8a0108f40 2024-04-23T14:59:55.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a0108580 0x7fe8a019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:55.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89e800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a010d500 0x7fe8a01a3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:55.396 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89de00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a0108580 0x7fe8a019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:47446/0 (socket says 172.21.15.83:47446) 2024-04-23T14:59:55.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89de00640 1 -- 172.21.15.83:0/4174088868 learned_addr learned my addr 172.21.15.83:0/4174088868 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:55.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe895400640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a0108f40 0x7fe8a019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:55.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89de00640 1 -- 172.21.15.83:0/4174088868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a0108f40 msgr2=0x7fe8a019f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:55.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89de00640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a0108f40 0x7fe8a019f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89de00640 1 -- 172.21.15.83:0/4174088868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a010d500 msgr2=0x7fe8a01a3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:55.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89de00640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a010d500 0x7fe8a01a3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.394+0000 7fe89de00640 1 -- 172.21.15.83:0/4174088868 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a01a3d50 con 0x7fe8a0108580 2024-04-23T14:59:55.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.395+0000 7fe89de00640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a0108580 0x7fe8a019ed60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe89000d950 tx=0x7fe89000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:55.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.395+0000 7fe897400640 1 -- 172.21.15.83:0/4174088868 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890017020 con 0x7fe8a0108580 2024-04-23T14:59:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.395+0000 7fe89f200640 1 -- 172.21.15.83:0/4174088868 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a01a4040 con 0x7fe8a0108580 2024-04-23T14:59:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.395+0000 7fe897400640 1 -- 172.21.15.83:0/4174088868 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe890009c30 con 0x7fe8a0108580 2024-04-23T14:59:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.395+0000 7fe897400640 1 -- 172.21.15.83:0/4174088868 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89000f6b0 con 0x7fe8a0108580 2024-04-23T14:59:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.395+0000 7fe89f200640 1 -- 172.21.15.83:0/4174088868 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a01ab8d0 con 0x7fe8a0108580 2024-04-23T14:59:55.398 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.396+0000 7fe89f200640 1 -- 172.21.15.83:0/4174088868 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a0069190 con 0x7fe8a0108580 2024-04-23T14:59:55.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.398+0000 7fe897400640 1 -- 172.21.15.83:0/4174088868 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe89001a020 con 0x7fe8a0108580 2024-04-23T14:59:55.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.399+0000 7fe897400640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe8840a3d20 0x7fe8840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:55.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.400+0000 7fe897400640 1 -- 172.21.15.83:0/4174088868 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7fe8900d0710 con 0x7fe8a0108580 2024-04-23T14:59:55.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.400+0000 7fe895400640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe8840a3d20 0x7fe8840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:55.400 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.400+0000 7fe895400640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe8840a3d20 0x7fe8840a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe888006010 tx=0x7fe888000bc0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:55.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.404+0000 7fe897400640 1 -- 172.21.15.83:0/4174088868 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe890021020 con 0x7fe8a0108580 2024-04-23T14:59:55.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.621+0000 7fe89f200640 1 -- 172.21.15.83:0/4174088868 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe8a0110a50 con 0x7fe8a0108580 2024-04-23T14:59:55.621 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.622+0000 7fe897400640 1 -- 172.21.15.83:0/4174088868 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v99) ==== 81+0+943 (secure 0 0 0) 0x7fe890098070 con 0x7fe8a0108580 2024-04-23T14:59:55.622 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/622708991 2024-04-24T14:48:27.715221+0000 2024-04-23T14:59:55.622 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1081276647 2024-04-24T14:48:27.715221+0000 2024-04-23T14:59:55.622 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/4179157882 2024-04-24T14:48:27.715221+0000 2024-04-23T14:59:55.622 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/1964818199 2024-04-24T14:48:27.715221+0000 2024-04-23T14:59:55.622 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3063977912 2024-04-24T14:37:57.064987+0000 2024-04-23T14:59:55.622 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/1987378267 2024-04-24T14:37:57.064987+0000 2024-04-23T14:59:55.622 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/3974620988 2024-04-24T14:37:57.064987+0000 2024-04-23T14:59:55.622 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/313104778 2024-04-24T14:37:23.112444+0000 2024-04-23T14:59:55.623 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/1964818199 2024-04-24T14:48:27.715221+0000 2024-04-23T14:59:55.623 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3743883304 2024-04-24T14:37:23.112444+0000 2024-04-23T14:59:55.623 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/871735489 2024-04-24T14:37:23.112444+0000 2024-04-23T14:59:55.623 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/2255156114 2024-04-24T14:48:27.715221+0000 2024-04-23T14:59:55.623 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/1316070089 2024-04-24T14:37:23.112444+0000 2024-04-23T14:59:55.623 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6801/1316070089 2024-04-24T14:37:23.112444+0000 2024-04-23T14:59:55.623 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:0/3707212986 2024-04-24T14:37:57.064987+0000 2024-04-23T14:59:55.623 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.15.83:6800/3974620988 2024-04-24T14:37:57.064987+0000 2024-04-23T14:59:55.624 INFO:teuthology.orchestra.run.smithi083.stderr:listed 16 entries 2024-04-23T14:59:55.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.624+0000 7fe89f200640 1 -- 172.21.15.83:0/4174088868 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe8840a3d20 msgr2=0x7fe8840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:55.624 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.624+0000 7fe89f200640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe8840a3d20 0x7fe8840a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe888006010 tx=0x7fe888000bc0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.624+0000 7fe89f200640 1 -- 172.21.15.83:0/4174088868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a0108580 msgr2=0x7fe8a019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:55.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.624+0000 7fe89f200640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a0108580 0x7fe8a019ed60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe89000d950 tx=0x7fe89000de10 comp rx=0 tx=0).stop 2024-04-23T14:59:55.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.624+0000 7fe89f200640 1 -- 172.21.15.83:0/4174088868 shutdown_connections 2024-04-23T14:59:55.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.624+0000 7fe89f200640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fe8840a3d20 0x7fe8840a61e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.624+0000 7fe89f200640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8a010d500 0x7fe8a01a3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.624+0000 7fe89f200640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe8a0108f40 0x7fe8a019f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.624+0000 7fe89f200640 1 --2- 172.21.15.83:0/4174088868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe8a0108580 0x7fe8a019ed60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:55.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.624+0000 7fe89f200640 1 -- 172.21.15.83:0/4174088868 >> 172.21.15.83:0/4174088868 conn(0x7fe8a0077e20 msgr2=0x7fe8a0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:55.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.625+0000 7fe89f200640 1 -- 172.21.15.83:0/4174088868 shutdown_connections 2024-04-23T14:59:55.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:55.625+0000 7fe89f200640 1 -- 172.21.15.83:0/4174088868 wait complete. 2024-04-23T14:59:55.643 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-23T14:59:55.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:59:55.643 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-23T14:59:55.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:55 smithi083 ceph-mon[27867]: pgmap v357: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:55.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:55 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/2075563759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:59:55.757 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:55 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/899949214' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:59:55.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:55 smithi173 ceph-mon[32443]: pgmap v357: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:55.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:55 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/2075563759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:59:55.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:55 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/899949214' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:59:55.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:55 smithi203 ceph-mon[32800]: pgmap v357: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-23T14:59:55.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:55 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/2075563759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T14:59:55.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:55 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/899949214' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T14:59:56.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:56 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/4174088868' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T14:59:56.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:56 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/4174088868' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T14:59:56.970 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:56 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/4174088868' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T14:59:57.529 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-23T14:59:57.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:59:57.529 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T14:59:57.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:59:57.605 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-23T14:59:57.717 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T14:59:57.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:59:57.718 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T14:59:57.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:57 smithi083 ceph-mon[27867]: pgmap v358: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:59:57.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T14:59:57.778 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-23T14:59:57.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:57 smithi173 ceph-mon[32443]: pgmap v358: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:59:57.911 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-23T14:59:57.923 INFO:tasks.cephadm:Teardown begin 2024-04-23T14:59:57.924 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T14:59:57.924 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:59:57.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:57 smithi203 ceph-mon[32800]: pgmap v358: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-23T14:59:58.092 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.092+0000 7f7cf1600640 1 -- 172.21.15.83:0/1208581234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cec102240 msgr2=0x7f7cec102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.092+0000 7f7cf1600640 1 --2- 172.21.15.83:0/1208581234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cec102240 0x7f7cec102620 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc009900 tx=0x7f7cdc02a410 comp rx=0 tx=0).stop 2024-04-23T14:59:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.093+0000 7f7cf1600640 1 -- 172.21.15.83:0/1208581234 shutdown_connections 2024-04-23T14:59:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.093+0000 7f7cf1600640 1 --2- 172.21.15.83:0/1208581234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cec107310 0x7f7cec10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.093+0000 7f7cf1600640 1 --2- 172.21.15.83:0/1208581234 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cec102bf0 0x7f7cec106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.093+0000 7f7cf1600640 1 --2- 172.21.15.83:0/1208581234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cec102240 0x7f7cec102620 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.093+0000 7f7cf1600640 1 -- 172.21.15.83:0/1208581234 >> 172.21.15.83:0/1208581234 conn(0x7f7cec0793c0 msgr2=0x7f7cec0797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:58.093 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.093+0000 7f7cf1600640 1 -- 172.21.15.83:0/1208581234 shutdown_connections 2024-04-23T14:59:58.094 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.093+0000 7f7cf1600640 1 -- 172.21.15.83:0/1208581234 wait complete. 2024-04-23T14:59:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7cf1600640 1 Processor -- start 2024-04-23T14:59:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7cf1600640 1 -- start start 2024-04-23T14:59:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7cf1600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cec102240 0x7f7cec1a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7cf1600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cec107310 0x7f7cec1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7cf1600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cec19a920 0x7f7cec19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7cf1600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec112420 con 0x7f7cec19a920 2024-04-23T14:59:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7cf1600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec1122a0 con 0x7f7cec107310 2024-04-23T14:59:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7cf1600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec1125a0 con 0x7f7cec102240 2024-04-23T14:59:58.096 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7ceaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cec102240 0x7f7cec1a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7ceb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cec19a920 0x7f7cec19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7ceaa00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cec102240 0x7f7cec1a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50030/0 (socket says 172.21.15.83:50030) 2024-04-23T14:59:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7ceb400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cec19a920 0x7f7cec19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56602/0 (socket says 172.21.15.83:56602) 2024-04-23T14:59:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.094+0000 7f7ceaa00640 1 -- 172.21.15.83:0/342282318 learned_addr learned my addr 172.21.15.83:0/342282318 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.095+0000 7f7ce2000640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cec107310 0x7f7cec1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.095+0000 7f7ceb400640 1 -- 172.21.15.83:0/342282318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cec102240 msgr2=0x7f7cec1a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.095+0000 7f7ceb400640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cec102240 0x7f7cec1a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.095+0000 7f7ceb400640 1 -- 172.21.15.83:0/342282318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cec107310 msgr2=0x7f7cec1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.095+0000 7f7ceb400640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cec107310 0x7f7cec1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.097 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.095+0000 7f7ceb400640 1 -- 172.21.15.83:0/342282318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec19b440 con 0x7f7cec19a920 2024-04-23T14:59:58.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.095+0000 7f7ceb400640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cec19a920 0x7f7cec19add0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009fd0 tx=0x7f7ce400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:58.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.095+0000 7f7ce3e00640 1 -- 172.21.15.83:0/342282318 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce400ce20 con 0x7f7cec19a920 2024-04-23T14:59:58.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.095+0000 7f7ce3e00640 1 -- 172.21.15.83:0/342282318 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce4005020 con 0x7f7cec19a920 2024-04-23T14:59:58.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.095+0000 7f7cf1600640 1 -- 172.21.15.83:0/342282318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec19b730 con 0x7f7cec19a920 2024-04-23T14:59:58.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.095+0000 7f7ce3e00640 1 -- 172.21.15.83:0/342282318 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4010720 con 0x7f7cec19a920 2024-04-23T14:59:58.098 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.095+0000 7f7cf1600640 1 -- 172.21.15.83:0/342282318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec1a7980 con 0x7f7cec19a920 2024-04-23T14:59:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.096+0000 7f7cf1600640 1 -- 172.21.15.83:0/342282318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec069270 con 0x7f7cec19a920 2024-04-23T14:59:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.099+0000 7f7ce3e00640 1 -- 172.21.15.83:0/342282318 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7ce4005540 con 0x7f7cec19a920 2024-04-23T14:59:58.099 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.100+0000 7f7ce3e00640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7cd00a3d20 0x7f7cd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:58.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.100+0000 7f7ce3e00640 1 -- 172.21.15.83:0/342282318 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f7ce40d0e50 con 0x7f7cec19a920 2024-04-23T14:59:58.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.100+0000 7f7ceaa00640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7cd00a3d20 0x7f7cd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:58.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.101+0000 7f7ceaa00640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7cd00a3d20 0x7f7cd00a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0096c0 tx=0x7f7cdc0046c0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:58.105 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.105+0000 7f7ce3e00640 1 -- 172.21.15.83:0/342282318 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ce409d4e0 con 0x7f7cec19a920 2024-04-23T14:59:58.332 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.333+0000 7f7cf1600640 1 -- 172.21.15.83:0/342282318 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7cec102dd0 con 0x7f7cec19a920 2024-04-23T14:59:58.334 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.334+0000 7f7ce3e00640 1 -- 172.21.15.83:0/342282318 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7f7ce409cde0 con 0x7f7cec19a920 2024-04-23T14:59:58.335 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:59:58.335 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:59:58.335 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:59:58.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.337+0000 7f7cf1600640 1 -- 172.21.15.83:0/342282318 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7cd00a3d20 msgr2=0x7f7cd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.337+0000 7f7cf1600640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7cd00a3d20 0x7f7cd00a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0096c0 tx=0x7f7cdc0046c0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.337+0000 7f7cf1600640 1 -- 172.21.15.83:0/342282318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cec19a920 msgr2=0x7f7cec19add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.337+0000 7f7cf1600640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cec19a920 0x7f7cec19add0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009fd0 tx=0x7f7ce400ef10 comp rx=0 tx=0).stop 2024-04-23T14:59:58.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.337+0000 7f7cf1600640 1 -- 172.21.15.83:0/342282318 shutdown_connections 2024-04-23T14:59:58.337 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.337+0000 7f7cf1600640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f7cd00a3d20 0x7f7cd00a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.337+0000 7f7cf1600640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7cec19a920 0x7f7cec19add0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.337+0000 7f7cf1600640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cec107310 0x7f7cec1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.337+0000 7f7cf1600640 1 --2- 172.21.15.83:0/342282318 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cec102240 0x7f7cec1a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.337+0000 7f7cf1600640 1 -- 172.21.15.83:0/342282318 >> 172.21.15.83:0/342282318 conn(0x7f7cec0793c0 msgr2=0x7f7cec1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:58.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.337+0000 7f7cf1600640 1 -- 172.21.15.83:0/342282318 shutdown_connections 2024-04-23T14:59:58.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.337+0000 7f7cf1600640 1 -- 172.21.15.83:0/342282318 wait complete. 2024-04-23T14:59:58.356 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T14:59:58.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.539+0000 7f5a87200640 1 -- 172.21.15.83:0/2127068272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a88102050 msgr2=0x7f5a88102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.539+0000 7f5a87200640 1 --2- 172.21.15.83:0/2127068272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a88102050 0x7f5a88102430 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009900 tx=0x7f5a7802a410 comp rx=0 tx=0).stop 2024-04-23T14:59:58.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.539+0000 7f5a87200640 1 -- 172.21.15.83:0/2127068272 shutdown_connections 2024-04-23T14:59:58.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.539+0000 7f5a87200640 1 --2- 172.21.15.83:0/2127068272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88107120 0x7f5a8810ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.539 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.539+0000 7f5a87200640 1 --2- 172.21.15.83:0/2127068272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a88102a00 0x7f5a88106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.539+0000 7f5a87200640 1 --2- 172.21.15.83:0/2127068272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a88102050 0x7f5a88102430 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.539+0000 7f5a87200640 1 -- 172.21.15.83:0/2127068272 >> 172.21.15.83:0/2127068272 conn(0x7f5a880fd930 msgr2=0x7f5a880ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.539+0000 7f5a87200640 1 -- 172.21.15.83:0/2127068272 shutdown_connections 2024-04-23T14:59:58.540 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.539+0000 7f5a87200640 1 -- 172.21.15.83:0/2127068272 wait complete. 2024-04-23T14:59:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.540+0000 7f5a87200640 1 Processor -- start 2024-04-23T14:59:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.540+0000 7f5a87200640 1 -- start start 2024-04-23T14:59:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.540+0000 7f5a87200640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a88102050 0x7f5a88198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.540+0000 7f5a87200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88107120 0x7f5a88198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.540+0000 7f5a87200640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a8819cfe0 0x7f5a8819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.540+0000 7f5a87200640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a881121b0 con 0x7f5a8819cfe0 2024-04-23T14:59:58.542 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.540+0000 7f5a87200640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88112030 con 0x7f5a88107120 2024-04-23T14:59:58.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.540+0000 7f5a87200640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88112330 con 0x7f5a88102050 2024-04-23T14:59:58.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.541+0000 7f5a86800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a8819cfe0 0x7f5a8819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:58.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.541+0000 7f5a85e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a88102050 0x7f5a88198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:58.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.541+0000 7f5a85e00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a88102050 0x7f5a88198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50046/0 (socket says 172.21.15.83:50046) 2024-04-23T14:59:58.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.541+0000 7f5a85e00640 1 -- 172.21.15.83:0/809851176 learned_addr learned my addr 172.21.15.83:0/809851176 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:58.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.541+0000 7f5a85400640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88107120 0x7f5a88198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:58.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.541+0000 7f5a85e00640 1 -- 172.21.15.83:0/809851176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88107120 msgr2=0x7f5a88198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.541+0000 7f5a85e00640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88107120 0x7f5a88198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.543 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.541+0000 7f5a85e00640 1 -- 172.21.15.83:0/809851176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a8819cfe0 msgr2=0x7f5a8819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.541+0000 7f5a85e00640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a8819cfe0 0x7f5a8819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.541+0000 7f5a85e00640 1 -- 172.21.15.83:0/809851176 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8819dad0 con 0x7f5a88102050 2024-04-23T14:59:58.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.541+0000 7f5a85400640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88107120 0x7f5a88198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T14:59:58.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.541+0000 7f5a85e00640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a88102050 0x7f5a88198770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5a78006010 tx=0x7f5a78004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:58.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.542+0000 7f5a7ea00640 1 -- 172.21.15.83:0/809851176 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7803f030 con 0x7f5a88102050 2024-04-23T14:59:58.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.542+0000 7f5a7ea00640 1 -- 172.21.15.83:0/809851176 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a78004440 con 0x7f5a88102050 2024-04-23T14:59:58.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.542+0000 7f5a7ea00640 1 -- 172.21.15.83:0/809851176 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a78035070 con 0x7f5a88102050 2024-04-23T14:59:58.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.542+0000 7f5a87200640 1 -- 172.21.15.83:0/809851176 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8819dd60 con 0x7f5a88102050 2024-04-23T14:59:58.544 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.542+0000 7f5a87200640 1 -- 172.21.15.83:0/809851176 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a881a54d0 con 0x7f5a88102050 2024-04-23T14:59:58.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.543+0000 7f5a87200640 1 -- 172.21.15.83:0/809851176 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a88069190 con 0x7f5a88102050 2024-04-23T14:59:58.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.544+0000 7f5a7ea00640 1 -- 172.21.15.83:0/809851176 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5a78042030 con 0x7f5a88102050 2024-04-23T14:59:58.545 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.546+0000 7f5a7ea00640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5a6c0a3da0 0x7f5a6c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:58.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.546+0000 7f5a7ea00640 1 -- 172.21.15.83:0/809851176 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f5a780f0bf0 con 0x7f5a88102050 2024-04-23T14:59:58.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.546+0000 7f5a85400640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5a6c0a3da0 0x7f5a6c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:58.546 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.546+0000 7f5a85400640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5a6c0a3da0 0x7f5a6c0a6260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a88101020 tx=0x7f5a74005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:58.550 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.550+0000 7f5a7ea00640 1 -- 172.21.15.83:0/809851176 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a780bd220 con 0x7f5a88102050 2024-04-23T14:59:58.756 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:58 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/342282318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:59:58.774 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.775+0000 7f5a87200640 1 -- 172.21.15.83:0/809851176 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a88199a70 con 0x7f5a88102050 2024-04-23T14:59:58.776 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.776+0000 7f5a7ea00640 1 -- 172.21.15.83:0/809851176 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11863 (secure 0 0 0) 0x7f5a780bcb20 con 0x7f5a88102050 2024-04-23T14:59:58.776 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:59:58.776 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14388,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/928555402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":928555402},{"type":"v1","addr":"172.21.15.83:6835","nonce":928555402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14457,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/436672914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":436672914},{"type":"v1","addr":"172.21.15.83:6839","nonce":436672914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6841/238596160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6840","nonce":238596160},{"type":"v1","addr":"172.21.15.83:6841","nonce":238596160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24436,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6833/317357729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":317357729},{"type":"v1","addr":"172.21.15.203:6833","nonce":317357729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/39034794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":39034794},{"type":"v1","addr":"172.21.15.173:6833","nonce":39034794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/3773827425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":3773827425},{"type":"v1","addr":"172.21.15.83:6837","nonce":3773827425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T14:45:59.723532+0000","modified":"2024-04-23T14:53:18.715010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24599,"mds_1":24542,"mds_2":24532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24499":{"gid":24499,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6835/2384637189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2384637189},{"type":"v1","addr":"172.21.15.203:6835","nonce":2384637189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6835/2567101723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2567101723},{"type":"v1","addr":"172.21.15.173:6835","nonce":2567101723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.203:6837/258138306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":258138306},{"type":"v1","addr":"172.21.15.203:6837","nonce":258138306}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.203:6839/2628587422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6838","nonce":2628587422},{"type":"v1","addr":"172.21.15.203:6839","nonce":2628587422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.173:6837/3951862333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3951862333},{"type":"v1","addr":"172.21.15.173:6837","nonce":3951862333}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24599":{"gid":24599,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6839/2485025097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2485025097},{"type":"v1","addr":"172.21.15.173:6839","nonce":2485025097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24599,"qdb_cluster":[24532,24542,24599]},"id":1}]} 2024-04-23T14:59:58.776 INFO:teuthology.orchestra.run.smithi083.stderr:dumped fsmap epoch 29 2024-04-23T14:59:58.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.778+0000 7f5a87200640 1 -- 172.21.15.83:0/809851176 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5a6c0a3da0 msgr2=0x7f5a6c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.778 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.778+0000 7f5a87200640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5a6c0a3da0 0x7f5a6c0a6260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a88101020 tx=0x7f5a74005f90 comp rx=0 tx=0).stop 2024-04-23T14:59:58.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.778+0000 7f5a87200640 1 -- 172.21.15.83:0/809851176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a88102050 msgr2=0x7f5a88198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.778+0000 7f5a87200640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a88102050 0x7f5a88198770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5a78006010 tx=0x7f5a78004040 comp rx=0 tx=0).stop 2024-04-23T14:59:58.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.778+0000 7f5a87200640 1 -- 172.21.15.83:0/809851176 shutdown_connections 2024-04-23T14:59:58.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.778+0000 7f5a87200640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f5a6c0a3da0 0x7f5a6c0a6260 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.778+0000 7f5a87200640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a8819cfe0 0x7f5a8819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.778+0000 7f5a87200640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a88107120 0x7f5a88198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.778+0000 7f5a87200640 1 --2- 172.21.15.83:0/809851176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a88102050 0x7f5a88198770 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.778+0000 7f5a87200640 1 -- 172.21.15.83:0/809851176 >> 172.21.15.83:0/809851176 conn(0x7f5a880fd930 msgr2=0x7f5a880fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:58.779 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.778+0000 7f5a87200640 1 -- 172.21.15.83:0/809851176 shutdown_connections 2024-04-23T14:59:58.780 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.779+0000 7f5a87200640 1 -- 172.21.15.83:0/809851176 wait complete. 2024-04-23T14:59:58.797 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T14:59:58.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:58 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/342282318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:59:58.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:58 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/342282318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:59:58.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.982+0000 7f1cbe600640 1 -- 172.21.15.83:0/4109494474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb81084a0 msgr2=0x7f1cb8108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.982+0000 7f1cbe600640 1 --2- 172.21.15.83:0/4109494474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb81084a0 0x7f1cb8108900 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009900 tx=0x7f1ca402a480 comp rx=0 tx=0).stop 2024-04-23T14:59:58.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.982+0000 7f1cbe600640 1 -- 172.21.15.83:0/4109494474 shutdown_connections 2024-04-23T14:59:58.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.982+0000 7f1cbe600640 1 --2- 172.21.15.83:0/4109494474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cb8108e40 0x7f1cb81152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.982 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.982+0000 7f1cbe600640 1 --2- 172.21.15.83:0/4109494474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb81084a0 0x7f1cb8108900 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.982+0000 7f1cbe600640 1 --2- 172.21.15.83:0/4109494474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cb810d3c0 0x7f1cb810d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.982+0000 7f1cbe600640 1 -- 172.21.15.83:0/4109494474 >> 172.21.15.83:0/4109494474 conn(0x7f1cb8077b80 msgr2=0x7f1cb8079fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:58.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.982+0000 7f1cbe600640 1 -- 172.21.15.83:0/4109494474 shutdown_connections 2024-04-23T14:59:58.983 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.982+0000 7f1cbe600640 1 -- 172.21.15.83:0/4109494474 wait complete. 2024-04-23T14:59:58.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.983+0000 7f1cbe600640 1 Processor -- start 2024-04-23T14:59:58.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.983+0000 7f1cbe600640 1 -- start start 2024-04-23T14:59:58.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.984+0000 7f1cbe600640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cb81084a0 0x7f1cb81a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:58.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.984+0000 7f1cbe600640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cb8108e40 0x7f1cb81a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:58.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.984+0000 7f1cbe600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb810d3c0 0x7f1cb819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:58.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.984+0000 7f1cbe600640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8118800 con 0x7f1cb810d3c0 2024-04-23T14:59:58.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.984+0000 7f1cbe600640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8118680 con 0x7f1cb81084a0 2024-04-23T14:59:58.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.984+0000 7f1cbe600640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8118980 con 0x7f1cb8108e40 2024-04-23T14:59:58.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.984+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cb81084a0 0x7f1cb81a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:58.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.984+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cb81084a0 0x7f1cb81a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:47496/0 (socket says 172.21.15.83:47496) 2024-04-23T14:59:58.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.984+0000 7f1cbd200640 1 -- 172.21.15.83:0/135414540 learned_addr learned my addr 172.21.15.83:0/135414540 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:58.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.984+0000 7f1cb7e00640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cb8108e40 0x7f1cb81a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.984+0000 7f1cbdc00640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb810d3c0 0x7f1cb819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.985+0000 7f1cbd200640 1 -- 172.21.15.83:0/135414540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cb8108e40 msgr2=0x7f1cb81a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.985+0000 7f1cbd200640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cb8108e40 0x7f1cb81a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.985+0000 7f1cbd200640 1 -- 172.21.15.83:0/135414540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb810d3c0 msgr2=0x7f1cb819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.985+0000 7f1cbd200640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb810d3c0 0x7f1cb819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.985+0000 7f1cbd200640 1 -- 172.21.15.83:0/135414540 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb819f260 con 0x7f1cb81084a0 2024-04-23T14:59:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.985+0000 7f1cb7e00640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cb8108e40 0x7f1cb81a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:59:58.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.985+0000 7f1cbd200640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cb81084a0 0x7f1cb81a4930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00e9f0 tx=0x7f1cac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.985+0000 7f1cb5600640 1 -- 172.21.15.83:0/135414540 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac00cd80 con 0x7f1cb81084a0 2024-04-23T14:59:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.985+0000 7f1cbe600640 1 -- 172.21.15.83:0/135414540 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb819f550 con 0x7f1cb81084a0 2024-04-23T14:59:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.985+0000 7f1cb5600640 1 -- 172.21.15.83:0/135414540 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cac010910 con 0x7f1cb81084a0 2024-04-23T14:59:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.985+0000 7f1cb5600640 1 -- 172.21.15.83:0/135414540 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0188f0 con 0x7f1cb81084a0 2024-04-23T14:59:58.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.985+0000 7f1cbe600640 1 -- 172.21.15.83:0/135414540 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb81ab7f0 con 0x7f1cb81084a0 2024-04-23T14:59:58.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.986+0000 7f1cbe600640 1 -- 172.21.15.83:0/135414540 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb8069190 con 0x7f1cb81084a0 2024-04-23T14:59:58.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.988+0000 7f1cb5600640 1 -- 172.21.15.83:0/135414540 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1cac010ac0 con 0x7f1cb81084a0 2024-04-23T14:59:58.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.990+0000 7f1cb5600640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1ca00a3d20 0x7f1ca00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:58.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.990+0000 7f1cb5600640 1 -- 172.21.15.83:0/135414540 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f1cac0d0fc0 con 0x7f1cb81084a0 2024-04-23T14:59:58.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.990+0000 7f1cb7e00640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1ca00a3d20 0x7f1ca00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:58.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.991+0000 7f1cb7e00640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1ca00a3d20 0x7f1ca00a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009230 tx=0x7f1ca40058d0 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:58.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:58.992+0000 7f1cb5600640 1 -- 172.21.15.83:0/135414540 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1cac09cfa0 con 0x7f1cb81084a0 2024-04-23T14:59:59.180 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.180+0000 7f1cbe600640 1 -- 172.21.15.83:0/135414540 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cb8110900 con 0x7f1cb81084a0 2024-04-23T14:59:59.182 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.182+0000 7f1cb5600640 1 -- 172.21.15.83:0/135414540 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23166 (secure 0 0 0) 0x7f1cac0a12a0 con 0x7f1cb81084a0 2024-04-23T14:59:59.182 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-23T14:59:59.182 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":99,"fsid":"5f4fe708-017e-11ef-bc93-c7b262605968","created":"2024-04-23T14:36:49.928506+0000","modified":"2024-04-23T14:59:17.356224+0000","last_up_change":"2024-04-23T14:45:01.834303+0000","last_in_change":"2024-04-23T14:44:41.814511+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T14:40:41.241992+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T14:45:57.341602+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T14:45:58.654798+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f0dcc33-b313-4e7b-8cbf-d541c92624bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6803","nonce":1064791722}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6805","nonce":1064791722}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6809","nonce":1064791722}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":1064791722},{"type":"v1","addr":"172.21.15.83:6807","nonce":1064791722}]},"public_addr":"172.21.15.83:6803/1064791722","cluster_addr":"172.21.15.83:6805/1064791722","heartbeat_back_addr":"172.21.15.83:6809/1064791722","heartbeat_front_addr":"172.21.15.83:6807/1064791722","state":["exists","up"]},{"osd":1,"uuid":"50d1440d-148d-4828-8607-63ebd9076ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6801","nonce":548359483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6803","nonce":548359483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6807","nonce":548359483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":548359483},{"type":"v1","addr":"172.21.15.173:6805","nonce":548359483}]},"public_addr":"172.21.15.173:6801/548359483","cluster_addr":"172.21.15.173:6803/548359483","heartbeat_back_addr":"172.21.15.173:6807/548359483","heartbeat_front_addr":"172.21.15.173:6805/548359483","state":["exists","up"]},{"osd":2,"uuid":"f7fd9255-ac0b-47bb-813f-d29f1ffa131b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6801","nonce":4282658379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6803","nonce":4282658379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6807","nonce":4282658379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6804","nonce":4282658379},{"type":"v1","addr":"172.21.15.203:6805","nonce":4282658379}]},"public_addr":"172.21.15.203:6801/4282658379","cluster_addr":"172.21.15.203:6803/4282658379","heartbeat_back_addr":"172.21.15.203:6807/4282658379","heartbeat_front_addr":"172.21.15.203:6805/4282658379","state":["exists","up"]},{"osd":3,"uuid":"8caf9f27-bac9-4ff3-b700-bcca3c250e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6811","nonce":3253964161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6813","nonce":3253964161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6817","nonce":3253964161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3253964161},{"type":"v1","addr":"172.21.15.83:6815","nonce":3253964161}]},"public_addr":"172.21.15.83:6811/3253964161","cluster_addr":"172.21.15.83:6813/3253964161","heartbeat_back_addr":"172.21.15.83:6817/3253964161","heartbeat_front_addr":"172.21.15.83:6815/3253964161","state":["exists","up"]},{"osd":4,"uuid":"07401fed-df44-4869-92ff-72c233457a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6809","nonce":2156601908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6811","nonce":2156601908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6815","nonce":2156601908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":2156601908},{"type":"v1","addr":"172.21.15.173:6813","nonce":2156601908}]},"public_addr":"172.21.15.173:6809/2156601908","cluster_addr":"172.21.15.173:6811/2156601908","heartbeat_back_addr":"172.21.15.173:6815/2156601908","heartbeat_front_addr":"172.21.15.173:6813/2156601908","state":["exists","up"]},{"osd":5,"uuid":"04c3ad4f-1bd5-45fa-a333-26b242b2ba1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6809","nonce":581398636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6810","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6811","nonce":581398636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6814","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6815","nonce":581398636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":581398636},{"type":"v1","addr":"172.21.15.203:6813","nonce":581398636}]},"public_addr":"172.21.15.203:6809/581398636","cluster_addr":"172.21.15.203:6811/581398636","heartbeat_back_addr":"172.21.15.203:6815/581398636","heartbeat_front_addr":"172.21.15.203:6813/581398636","state":["exists","up"]},{"osd":6,"uuid":"835bb798-4cbd-4689-af88-4da9744dd601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6819","nonce":783931631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6821","nonce":783931631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6825","nonce":783931631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":783931631},{"type":"v1","addr":"172.21.15.83:6823","nonce":783931631}]},"public_addr":"172.21.15.83:6819/783931631","cluster_addr":"172.21.15.83:6821/783931631","heartbeat_back_addr":"172.21.15.83:6825/783931631","heartbeat_front_addr":"172.21.15.83:6823/783931631","state":["exists","up"]},{"osd":7,"uuid":"e0ac95e1-5d76-4f47-9483-1420f8c834c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6817","nonce":1174963936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6819","nonce":1174963936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6823","nonce":1174963936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1174963936},{"type":"v1","addr":"172.21.15.173:6821","nonce":1174963936}]},"public_addr":"172.21.15.173:6817/1174963936","cluster_addr":"172.21.15.173:6819/1174963936","heartbeat_back_addr":"172.21.15.173:6823/1174963936","heartbeat_front_addr":"172.21.15.173:6821/1174963936","state":["exists","up"]},{"osd":8,"uuid":"75266f49-b8fb-46c3-aa62-bef7957954dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6816","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6817","nonce":1549180524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6819","nonce":1549180524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6823","nonce":1549180524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6820","nonce":1549180524},{"type":"v1","addr":"172.21.15.203:6821","nonce":1549180524}]},"public_addr":"172.21.15.203:6817/1549180524","cluster_addr":"172.21.15.203:6819/1549180524","heartbeat_back_addr":"172.21.15.203:6823/1549180524","heartbeat_front_addr":"172.21.15.203:6821/1549180524","state":["exists","up"]},{"osd":9,"uuid":"bf897b76-ece1-4df5-8531-ed3aab716d5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6827","nonce":3165262956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6829","nonce":3165262956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6833","nonce":3165262956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3165262956},{"type":"v1","addr":"172.21.15.83:6831","nonce":3165262956}]},"public_addr":"172.21.15.83:6827/3165262956","cluster_addr":"172.21.15.83:6829/3165262956","heartbeat_back_addr":"172.21.15.83:6833/3165262956","heartbeat_front_addr":"172.21.15.83:6831/3165262956","state":["exists","up"]},{"osd":10,"uuid":"1b95d0e9-c1de-4607-b765-2eb61bb91faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6825","nonce":2121565520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6827","nonce":2121565520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6831","nonce":2121565520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":2121565520},{"type":"v1","addr":"172.21.15.173:6829","nonce":2121565520}]},"public_addr":"172.21.15.173:6825/2121565520","cluster_addr":"172.21.15.173:6827/2121565520","heartbeat_back_addr":"172.21.15.173:6831/2121565520","heartbeat_front_addr":"172.21.15.173:6829/2121565520","state":["exists","up"]},{"osd":11,"uuid":"636b2be4-2656-4ec1-89c4-739c158b6221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6825","nonce":2856576923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6827","nonce":2856576923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6831","nonce":2856576923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2856576923},{"type":"v1","addr":"172.21.15.203:6829","nonce":2856576923}]},"public_addr":"172.21.15.203:6825/2856576923","cluster_addr":"172.21.15.203:6827/2856576923","heartbeat_back_addr":"172.21.15.203:6831/2856576923","heartbeat_front_addr":"172.21.15.203:6829/2856576923","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:39:48.783654+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:14.472732+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:40:38.725792+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:12.092908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:41:38.572655+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:04.068932+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:42:38.697211+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:05.783192+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:43:31.305323+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:06.121898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:34.068251+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T14:44:59.900186+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.83:0/622708991":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/1081276647":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/4179157882":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6801/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3063977912":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/1987378267":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6801/3974620988":"2024-04-24T14:37:57.064987+0000","172.21.15.83:0/313104778":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6800/1964818199":"2024-04-24T14:48:27.715221+0000","172.21.15.83:0/3743883304":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/871735489":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/2255156114":"2024-04-24T14:48:27.715221+0000","172.21.15.83:6800/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:6801/1316070089":"2024-04-24T14:37:23.112444+0000","172.21.15.83:0/3707212986":"2024-04-24T14:37:57.064987+0000","172.21.15.83:6800/3974620988":"2024-04-24T14:37:57.064987+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T14:59:59.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.185+0000 7f1cbe600640 1 -- 172.21.15.83:0/135414540 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1ca00a3d20 msgr2=0x7f1ca00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:59.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.185+0000 7f1cbe600640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1ca00a3d20 0x7f1ca00a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009230 tx=0x7f1ca40058d0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.185+0000 7f1cbe600640 1 -- 172.21.15.83:0/135414540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cb81084a0 msgr2=0x7f1cb81a4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:59.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.185+0000 7f1cbe600640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cb81084a0 0x7f1cb81a4930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00e9f0 tx=0x7f1cac00eeb0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.185 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.185+0000 7f1cbe600640 1 -- 172.21.15.83:0/135414540 shutdown_connections 2024-04-23T14:59:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.185+0000 7f1cbe600640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f1ca00a3d20 0x7f1ca00a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.185+0000 7f1cbe600640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cb810d3c0 0x7f1cb819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.185+0000 7f1cbe600640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cb8108e40 0x7f1cb81a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.185+0000 7f1cbe600640 1 --2- 172.21.15.83:0/135414540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1cb81084a0 0x7f1cb81a4930 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.185+0000 7f1cbe600640 1 -- 172.21.15.83:0/135414540 >> 172.21.15.83:0/135414540 conn(0x7f1cb8077b80 msgr2=0x7f1cb8078330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.185+0000 7f1cbe600640 1 -- 172.21.15.83:0/135414540 shutdown_connections 2024-04-23T14:59:59.186 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.185+0000 7f1cbe600640 1 -- 172.21.15.83:0/135414540 wait complete. 2024-04-23T14:59:59.204 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T14:59:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.386+0000 7f962d800640 1 -- 172.21.15.83:0/1212003418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9628101620 msgr2=0x7f962810e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.386+0000 7f962d800640 1 --2- 172.21.15.83:0/1212003418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9628101620 0x7f962810e250 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9614009900 tx=0x7f961402a3f0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.386+0000 7f962d800640 1 -- 172.21.15.83:0/1212003418 shutdown_connections 2024-04-23T14:59:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.386+0000 7f962d800640 1 --2- 172.21.15.83:0/1212003418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f962810e820 0x7f9628110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.386 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.386+0000 7f962d800640 1 --2- 172.21.15.83:0/1212003418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9628101620 0x7f962810e250 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.386+0000 7f962d800640 1 --2- 172.21.15.83:0/1212003418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9628100c70 0x7f9628101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.386+0000 7f962d800640 1 -- 172.21.15.83:0/1212003418 >> 172.21.15.83:0/1212003418 conn(0x7f96280fc880 msgr2=0x7f96280feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.386+0000 7f962d800640 1 -- 172.21.15.83:0/1212003418 shutdown_connections 2024-04-23T14:59:59.387 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.386+0000 7f962d800640 1 -- 172.21.15.83:0/1212003418 wait complete. 2024-04-23T14:59:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.387+0000 7f962d800640 1 Processor -- start 2024-04-23T14:59:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.387+0000 7f962d800640 1 -- start start 2024-04-23T14:59:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.387+0000 7f962d800640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9628100c70 0x7f96281a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.387+0000 7f962d800640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628101620 0x7f96281a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.387+0000 7f962d800640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f962810e820 0x7f962819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:59.389 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.387+0000 7f962d800640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9628114500 con 0x7f962810e820 2024-04-23T14:59:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.387+0000 7f962d800640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7f9628114380 con 0x7f9628100c70 2024-04-23T14:59:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.387+0000 7f962d800640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7f9628114680 con 0x7f9628101620 2024-04-23T14:59:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.388+0000 7f962ce00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f962810e820 0x7f962819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.388+0000 7f9627400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628101620 0x7f96281a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.388+0000 7f9627400640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628101620 0x7f96281a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50078/0 (socket says 172.21.15.83:50078) 2024-04-23T14:59:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.388+0000 7f9627400640 1 -- 172.21.15.83:0/1820449798 learned_addr learned my addr 172.21.15.83:0/1820449798 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.388+0000 7f9627e00640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9628100c70 0x7f96281a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.388+0000 7f962ce00640 1 -- 172.21.15.83:0/1820449798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628101620 msgr2=0x7f96281a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:59.390 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.388+0000 7f962ce00640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628101620 0x7f96281a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.388+0000 7f962ce00640 1 -- 172.21.15.83:0/1820449798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9628100c70 msgr2=0x7f96281a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.388+0000 7f962ce00640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9628100c70 0x7f96281a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.388+0000 7f962ce00640 1 -- 172.21.15.83:0/1820449798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962819afb0 con 0x7f962810e820 2024-04-23T14:59:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.388+0000 7f962ce00640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f962810e820 0x7f962819a7e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9628101bc0 tx=0x7f962000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.389+0000 7f9624c00640 1 -- 172.21.15.83:0/1820449798 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962000cdf0 con 0x7f962810e820 2024-04-23T14:59:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.389+0000 7f9624c00640 1 -- 172.21.15.83:0/1820449798 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9620005020 con 0x7f962810e820 2024-04-23T14:59:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.389+0000 7f9624c00640 1 -- 172.21.15.83:0/1820449798 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9620010670 con 0x7f962810e820 2024-04-23T14:59:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.389+0000 7f962d800640 1 -- 172.21.15.83:0/1820449798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962819b2a0 con 0x7f962810e820 2024-04-23T14:59:59.391 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.389+0000 7f962d800640 1 -- 172.21.15.83:0/1820449798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96281a74c0 con 0x7f962810e820 2024-04-23T14:59:59.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.390+0000 7f962d800640 1 -- 172.21.15.83:0/1820449798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9628069a70 con 0x7f962810e820 2024-04-23T14:59:59.392 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.392+0000 7f9624c00640 1 -- 172.21.15.83:0/1820449798 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f96200054d0 con 0x7f962810e820 2024-04-23T14:59:59.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.393+0000 7f9624c00640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f960c0a3d20 0x7f960c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:59.393 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.394+0000 7f9624c00640 1 -- 172.21.15.83:0/1820449798 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8542+0+0 (secure 0 0 0) 0x7f96200d0e90 con 0x7f962810e820 2024-04-23T14:59:59.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.394+0000 7f9627e00640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f960c0a3d20 0x7f960c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:59.394 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.394+0000 7f9627e00640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f960c0a3d20 0x7f960c0a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f96180095f0 tx=0x7f9618005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:59.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.398+0000 7f9624c00640 1 -- 172.21.15.83:0/1820449798 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f962009d4c0 con 0x7f962810e820 2024-04-23T14:59:59.639 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.639+0000 7f962d800640 1 -- 172.21.15.83:0/1820449798 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9628109920 con 0x7f962810e820 2024-04-23T14:59:59.643 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.643+0000 7f9624c00640 1 -- 172.21.15.83:0/1820449798 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f962009cdc0 con 0x7f962810e820 2024-04-23T14:59:59.644 INFO:teuthology.orchestra.run.smithi083.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T14:59:59.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.648+0000 7f962d800640 1 -- 172.21.15.83:0/1820449798 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f960c0a3d20 msgr2=0x7f960c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:59.648 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.648+0000 7f962d800640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f960c0a3d20 0x7f960c0a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f96180095f0 tx=0x7f9618005f90 comp rx=0 tx=0).stop 2024-04-23T14:59:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.648+0000 7f962d800640 1 -- 172.21.15.83:0/1820449798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f962810e820 msgr2=0x7f962819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.648+0000 7f962d800640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f962810e820 0x7f962819a7e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9628101bc0 tx=0x7f962000ef10 comp rx=0 tx=0).stop 2024-04-23T14:59:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.648+0000 7f962d800640 1 -- 172.21.15.83:0/1820449798 shutdown_connections 2024-04-23T14:59:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.648+0000 7f962d800640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7f960c0a3d20 0x7f960c0a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.648+0000 7f962d800640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f962810e820 0x7f962819a7e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.648+0000 7f962d800640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9628101620 0x7f96281a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.648+0000 7f962d800640 1 --2- 172.21.15.83:0/1820449798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9628100c70 0x7f96281a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.649 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.648+0000 7f962d800640 1 -- 172.21.15.83:0/1820449798 >> 172.21.15.83:0/1820449798 conn(0x7f96280fc880 msgr2=0x7f96280fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:59.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.649+0000 7f962d800640 1 -- 172.21.15.83:0/1820449798 shutdown_connections 2024-04-23T14:59:59.650 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.649+0000 7f962d800640 1 -- 172.21.15.83:0/1820449798 wait complete. 2024-04-23T14:59:59.667 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T14:59:59.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-mon[32443]: pgmap v359: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-23T14:59:59.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/809851176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:59:59.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/135414540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:59:59.826 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-e[62490]: ignoring --setuser ceph since I am not root 2024-04-23T14:59:59.826 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-e[62490]: ignoring --setgroup ceph since I am not root 2024-04-23T14:59:59.826 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-mds[62494]: -- 172.21.15.173:0/3581218852 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56220ed0fa40 con 0x56220ece2800 2024-04-23T14:59:59.826 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-e[62490]: starting mds.e at 2024-04-23T14:59:59.827 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-h[62797]: ignoring --setuser ceph since I am not root 2024-04-23T14:59:59.827 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-h[62797]: ignoring --setgroup ceph since I am not root 2024-04-23T14:59:59.827 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-mds[62801]: -- 172.21.15.173:0/346960248 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5559ee38fa40 con 0x5559ee362800 2024-04-23T14:59:59.827 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-h[62797]: starting mds.h at 2024-04-23T14:59:59.827 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b[63123]: ignoring --setuser ceph since I am not root 2024-04-23T14:59:59.827 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b[63123]: ignoring --setgroup ceph since I am not root 2024-04-23T14:59:59.828 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-mds[63127]: -- 172.21.15.173:0/73921143 <== mon.2 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4fa2c7a40 con 0x55a4fa29a800 2024-04-23T14:59:59.828 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 14:59:59 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b[63123]: starting mds.b at 2024-04-23T14:59:59.853 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.853+0000 7fc5cda00640 1 -- 172.21.15.83:0/1168355769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5c80743a0 msgr2=0x7fc5c8074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.853+0000 7fc5cda00640 1 --2- 172.21.15.83:0/1168355769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5c80743a0 0x7fc5c8074780 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009900 tx=0x7fc5b802a480 comp rx=0 tx=0).stop 2024-04-23T14:59:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.854+0000 7fc5cda00640 1 -- 172.21.15.83:0/1168355769 shutdown_connections 2024-04-23T14:59:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.854+0000 7fc5cda00640 1 --2- 172.21.15.83:0/1168355769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5c8112ff0 0x7fc5c81153e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.854+0000 7fc5cda00640 1 --2- 172.21.15.83:0/1168355769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5c8074cc0 0x7fc5c8112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.854+0000 7fc5cda00640 1 --2- 172.21.15.83:0/1168355769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5c80743a0 0x7fc5c8074780 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.854+0000 7fc5cda00640 1 -- 172.21.15.83:0/1168355769 >> 172.21.15.83:0/1168355769 conn(0x7fc5c8079400 msgr2=0x7fc5c8079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T14:59:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.854+0000 7fc5cda00640 1 -- 172.21.15.83:0/1168355769 shutdown_connections 2024-04-23T14:59:59.854 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.854+0000 7fc5cda00640 1 -- 172.21.15.83:0/1168355769 wait complete. 2024-04-23T14:59:59.856 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5cda00640 1 Processor -- start 2024-04-23T14:59:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5cda00640 1 -- start start 2024-04-23T14:59:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5cda00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5c80743a0 0x7fc5c819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5cda00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5c8074cc0 0x7fc5c819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5cda00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5c8112ff0 0x7fc5c81a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5cda00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8118a40 con 0x7fc5c8074cc0 2024-04-23T14:59:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5cda00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c81188c0 con 0x7fc5c8112ff0 2024-04-23T14:59:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5cda00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8118bc0 con 0x7fc5c80743a0 2024-04-23T14:59:59.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5c6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5c8074cc0 0x7fc5c819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5c7400640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5c8112ff0 0x7fc5c81a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5c6000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5c8074cc0 0x7fc5c819f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56698/0 (socket says 172.21.15.83:56698) 2024-04-23T14:59:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5c6000640 1 -- 172.21.15.83:0/1016238485 learned_addr learned my addr 172.21.15.83:0/1016238485 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T14:59:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.855+0000 7fc5c6a00640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5c80743a0 0x7fc5c819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.856+0000 7fc5c6000640 1 -- 172.21.15.83:0/1016238485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5c80743a0 msgr2=0x7fc5c819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.856+0000 7fc5c6000640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5c80743a0 0x7fc5c819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.856+0000 7fc5c6000640 1 -- 172.21.15.83:0/1016238485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5c8112ff0 msgr2=0x7fc5c81a37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T14:59:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.856+0000 7fc5c6000640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5c8112ff0 0x7fc5c81a37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T14:59:59.858 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.856+0000 7fc5c6000640 1 -- 172.21.15.83:0/1016238485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c81a3ed0 con 0x7fc5c8074cc0 2024-04-23T14:59:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.856+0000 7fc5c6a00640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5c80743a0 0x7fc5c819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T14:59:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.856+0000 7fc5c6000640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5c8074cc0 0x7fc5c819f420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400d950 tx=0x7fc5b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.856+0000 7fc5bf400640 1 -- 172.21.15.83:0/1016238485 <== mon.0 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4017030 con 0x7fc5c8074cc0 2024-04-23T14:59:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.856+0000 7fc5cda00640 1 -- 172.21.15.83:0/1016238485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c81a41c0 con 0x7fc5c8074cc0 2024-04-23T14:59:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.856+0000 7fc5bf400640 1 -- 172.21.15.83:0/1016238485 <== mon.0 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b4004020 con 0x7fc5c8074cc0 2024-04-23T14:59:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.856+0000 7fc5bf400640 1 -- 172.21.15.83:0/1016238485 <== mon.0 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b40047a0 con 0x7fc5c8074cc0 2024-04-23T14:59:59.859 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.856+0000 7fc5cda00640 1 -- 172.21.15.83:0/1016238485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c81aba50 con 0x7fc5c8074cc0 2024-04-23T14:59:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.858+0000 7fc5cda00640 1 -- 172.21.15.83:0/1016238485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c8068e20 con 0x7fc5c8074cc0 2024-04-23T14:59:59.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.860+0000 7fc5bf400640 1 -- 172.21.15.83:0/1016238485 <== mon.0 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc5b40041d0 con 0x7fc5c8074cc0 2024-04-23T14:59:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.861+0000 7fc5bf400640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc5ac0a3d20 0x7fc5ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T14:59:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.861+0000 7fc5bf400640 1 -- 172.21.15.83:0/1016238485 <== mon.0 v2:172.21.15.83:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8756+0+0 (secure 0 0 0) 0x7fc5b40d0250 con 0x7fc5c8074cc0 2024-04-23T14:59:59.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.862+0000 7fc5c6a00640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc5ac0a3d20 0x7fc5ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T14:59:59.862 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.862+0000 7fc5c6a00640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc5ac0a3d20 0x7fc5ac0a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8000c00 tx=0x7fc5b8031510 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T14:59:59.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T14:59:59.866+0000 7fc5bf400640 1 -- 172.21.15.83:0/1016238485 <== mon.0 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5b409c800 con 0x7fc5c8074cc0 2024-04-23T14:59:59.866 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:59 smithi083 ceph-mon[27867]: pgmap v359: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-23T14:59:59.866 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:59 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/809851176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:59:59.866 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:59 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/135414540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:59:59.866 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 14:59:59 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a[27863]: 2024-04-23T14:59:59.641+0000 7f8001a00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T14:59:59.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-mon[32800]: pgmap v359: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-23T14:59:59.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/809851176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T14:59:59.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/135414540' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T14:59:59.971 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c[63252]: ignoring --setuser ceph since I am not root 2024-04-23T14:59:59.972 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c[63252]: ignoring --setgroup ceph since I am not root 2024-04-23T14:59:59.972 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-mds[63256]: -- 172.21.15.203:0/1143377192 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55de038e3a40 con 0x55de038b7000 2024-04-23T14:59:59.972 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c[63252]: starting mds.c at 2024-04-23T14:59:59.972 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f[62931]: ignoring --setuser ceph since I am not root 2024-04-23T14:59:59.972 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f[62931]: ignoring --setgroup ceph since I am not root 2024-04-23T14:59:59.972 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-mds[62935]: -- 172.21.15.203:0/3416106761 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec80e5da40 con 0x55ec80e30c00 2024-04-23T14:59:59.972 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f[62931]: starting mds.f at 2024-04-23T14:59:59.973 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-i[63571]: ignoring --setuser ceph since I am not root 2024-04-23T14:59:59.973 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-i[63571]: ignoring --setgroup ceph since I am not root 2024-04-23T14:59:59.973 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-mds[63575]: -- 172.21.15.203:0/833353522 <== mon.1 v2:172.21.15.203:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564d15fd3a40 con 0x564d15fa7000 2024-04-23T14:59:59.973 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 14:59:59 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-i[63571]: starting mds.i at 2024-04-23T15:00:00.111 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.112+0000 7fc5cda00640 1 -- 172.21.15.83:0/1016238485 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc5c81abd50 con 0x7fc5c8074cc0 2024-04-23T15:00:00.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a[27863]: 2024-04-23T14:59:59.999+0000 7f8004c00700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-23T15:00:00.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a[27863]: 2024-04-23T14:59:59.999+0000 7f8004c00700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T15:00:00.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a[27863]: 2024-04-23T14:59:59.999+0000 7f8004c00700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-04-23T15:00:00.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a[27863]: 2024-04-23T14:59:59.999+0000 7f8004c00700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T15:00:00.257 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a[27863]: 2024-04-23T14:59:59.999+0000 7f8004c00700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-04-23T15:00:00.653 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.654+0000 7fc5bf400640 1 -- 172.21.15.83:0/1016238485 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7fc5b409c100 con 0x7fc5c8074cc0 2024-04-23T15:00:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.657+0000 7fc5cda00640 1 -- 172.21.15.83:0/1016238485 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc5ac0a3d20 msgr2=0x7fc5ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T15:00:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.657+0000 7fc5cda00640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc5ac0a3d20 0x7fc5ac0a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8000c00 tx=0x7fc5b8031510 comp rx=0 tx=0).stop 2024-04-23T15:00:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.657+0000 7fc5cda00640 1 -- 172.21.15.83:0/1016238485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5c8074cc0 msgr2=0x7fc5c819f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T15:00:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.657+0000 7fc5cda00640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5c8074cc0 0x7fc5c819f420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400d950 tx=0x7fc5b400de10 comp rx=0 tx=0).stop 2024-04-23T15:00:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.657+0000 7fc5cda00640 1 -- 172.21.15.83:0/1016238485 shutdown_connections 2024-04-23T15:00:00.657 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.657+0000 7fc5cda00640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5c8112ff0 0x7fc5c81a37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:00.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.657+0000 7fc5cda00640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fc5ac0a3d20 0x7fc5ac0a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:00.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.657+0000 7fc5cda00640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5c8074cc0 0x7fc5c819f420 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:00.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.657+0000 7fc5cda00640 1 --2- 172.21.15.83:0/1016238485 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5c80743a0 0x7fc5c819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:00.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.657+0000 7fc5cda00640 1 -- 172.21.15.83:0/1016238485 >> 172.21.15.83:0/1016238485 conn(0x7fc5c8079400 msgr2=0x7fc5c810ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T15:00:00.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.657+0000 7fc5cda00640 1 -- 172.21.15.83:0/1016238485 shutdown_connections 2024-04-23T15:00:00.658 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.657+0000 7fc5cda00640 1 -- 172.21.15.83:0/1016238485 wait complete. 2024-04-23T15:00:00.677 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T15:00:00.767 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1820449798' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T15:00:00.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-mon[27867]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T15:00:00.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-mon[27867]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T15:00:00.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1820449798' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T15:00:00.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-mon[27867]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T15:00:00.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-mon[27867]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-23T15:00:00.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-mon[27867]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-23T15:00:00.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-mon[27867]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T15:00:00.768 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-mon[27867]: fs cephfs is degraded 2024-04-23T15:00:00.769 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-mon[27867]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T15:00:00.769 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-mon[27867]: fs cephfs is offline because no MDS is active for it. 2024-04-23T15:00:00.769 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:00 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1016238485' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T15:00:00.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:00 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1820449798' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T15:00:00.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:00 smithi173 ceph-mon[32443]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T15:00:00.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:00 smithi173 ceph-mon[32443]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T15:00:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:00 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1820449798' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T15:00:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:00 smithi173 ceph-mon[32443]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T15:00:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:00 smithi173 ceph-mon[32443]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-23T15:00:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:00 smithi173 ceph-mon[32443]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-23T15:00:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:00 smithi173 ceph-mon[32443]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T15:00:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:00 smithi173 ceph-mon[32443]: fs cephfs is degraded 2024-04-23T15:00:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:00 smithi173 ceph-mon[32443]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T15:00:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:00 smithi173 ceph-mon[32443]: fs cephfs is offline because no MDS is active for it. 2024-04-23T15:00:00.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:00 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1016238485' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T15:00:00.860 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.860+0000 7fcc4ac00640 1 -- 172.21.15.83:0/168710990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc44109520 msgr2=0x7fcc441110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T15:00:00.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.860+0000 7fcc4ac00640 1 --2- 172.21.15.83:0/168710990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc44109520 0x7fcc441110f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009be0 tx=0x7fcc3c02a6c0 comp rx=0 tx=0).stop 2024-04-23T15:00:00.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.861+0000 7fcc4ac00640 1 -- 172.21.15.83:0/168710990 shutdown_connections 2024-04-23T15:00:00.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.861+0000 7fcc4ac00640 1 --2- 172.21.15.83:0/168710990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc44109520 0x7fcc441110f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:00.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.861+0000 7fcc4ac00640 1 --2- 172.21.15.83:0/168710990 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc44104e00 0x7fcc44108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:00.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.861+0000 7fcc4ac00640 1 --2- 172.21.15.83:0/168710990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc44104450 0x7fcc44104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:00.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.861+0000 7fcc4ac00640 1 -- 172.21.15.83:0/168710990 >> 172.21.15.83:0/168710990 conn(0x7fcc440ffcd0 msgr2=0x7fcc441020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T15:00:00.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.861+0000 7fcc4ac00640 1 -- 172.21.15.83:0/168710990 shutdown_connections 2024-04-23T15:00:00.861 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.861+0000 7fcc4ac00640 1 -- 172.21.15.83:0/168710990 wait complete. 2024-04-23T15:00:00.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc4ac00640 1 Processor -- start 2024-04-23T15:00:00.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc4ac00640 1 -- start start 2024-04-23T15:00:00.863 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc4ac00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc44104450 0x7fcc441a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T15:00:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc4ac00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc44109520 0x7fcc441a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T15:00:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc4ac00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc4419a8c0 0x7fcc4419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T15:00:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc4ac00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441145c0 con 0x7fcc44109520 2024-04-23T15:00:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc4ac00640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fcc44114440 con 0x7fcc44104450 2024-04-23T15:00:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc4ac00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fcc44114740 con 0x7fcc4419a8c0 2024-04-23T15:00:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc43e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc44104450 0x7fcc441a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T15:00:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc43400640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc44109520 0x7fcc441a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T15:00:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc43e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc44104450 0x7fcc441a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.83:47534/0 (socket says 172.21.15.83:47534) 2024-04-23T15:00:00.864 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc43e00640 1 -- 172.21.15.83:0/432047324 learned_addr learned my addr 172.21.15.83:0/432047324 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T15:00:00.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.862+0000 7fcc48c00640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc4419a8c0 0x7fcc4419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T15:00:00.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.863+0000 7fcc43e00640 1 -- 172.21.15.83:0/432047324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc4419a8c0 msgr2=0x7fcc4419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T15:00:00.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.863+0000 7fcc43e00640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc4419a8c0 0x7fcc4419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:00.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.863+0000 7fcc43e00640 1 -- 172.21.15.83:0/432047324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc44109520 msgr2=0x7fcc441a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T15:00:00.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.863+0000 7fcc43e00640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc44109520 0x7fcc441a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:00.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.863+0000 7fcc43e00640 1 -- 172.21.15.83:0/432047324 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4419b3e0 con 0x7fcc44104450 2024-04-23T15:00:00.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.863+0000 7fcc43400640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc44109520 0x7fcc441a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T15:00:00.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.863+0000 7fcc43e00640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc44104450 0x7fcc441a0710 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e9f0 tx=0x7fcc3400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T15:00:00.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.863+0000 7fcc40c00640 1 -- 172.21.15.83:0/432047324 <== mon.2 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3400cd80 con 0x7fcc44104450 2024-04-23T15:00:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.863+0000 7fcc40c00640 1 -- 172.21.15.83:0/432047324 <== mon.2 v2:172.21.15.173:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc34010910 con 0x7fcc44104450 2024-04-23T15:00:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.863+0000 7fcc4ac00640 1 -- 172.21.15.83:0/432047324 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4419b6d0 con 0x7fcc44104450 2024-04-23T15:00:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.863+0000 7fcc40c00640 1 -- 172.21.15.83:0/432047324 <== mon.2 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34018990 con 0x7fcc44104450 2024-04-23T15:00:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.864+0000 7fcc4ac00640 1 -- 172.21.15.83:0/432047324 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc441a7920 con 0x7fcc44104450 2024-04-23T15:00:00.866 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.865+0000 7fcc4ac00640 1 -- 172.21.15.83:0/432047324 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc44069270 con 0x7fcc44104450 2024-04-23T15:00:00.867 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.867+0000 7fcc40c00640 1 -- 172.21.15.83:0/432047324 <== mon.2 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcc34010ac0 con 0x7fcc44104450 2024-04-23T15:00:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.868+0000 7fcc40c00640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcc280a3d20 0x7fcc280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T15:00:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.868+0000 7fcc40c00640 1 -- 172.21.15.83:0/432047324 <== mon.2 v2:172.21.15.173:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8756+0+0 (secure 0 0 0) 0x7fcc340d0e70 con 0x7fcc44104450 2024-04-23T15:00:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.868+0000 7fcc43400640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcc280a3d20 0x7fcc280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T15:00:00.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.869+0000 7fcc43400640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcc280a3d20 0x7fcc280a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcc300096c0 tx=0x7fcc30009260 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T15:00:00.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:00.870+0000 7fcc40c00640 1 -- 172.21.15.83:0/432047324 <== mon.2 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc3409cd70 con 0x7fcc44104450 2024-04-23T15:00:00.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:00 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1820449798' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T15:00:00.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:00 smithi203 ceph-mon[32800]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T15:00:00.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:00 smithi203 ceph-mon[32800]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T15:00:00.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:00 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1820449798' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T15:00:00.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:00 smithi203 ceph-mon[32800]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T15:00:00.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:00 smithi203 ceph-mon[32800]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-23T15:00:00.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:00 smithi203 ceph-mon[32800]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-23T15:00:00.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:00 smithi203 ceph-mon[32800]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-23T15:00:00.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:00 smithi203 ceph-mon[32800]: fs cephfs is degraded 2024-04-23T15:00:00.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:00 smithi203 ceph-mon[32800]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-23T15:00:00.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:00 smithi203 ceph-mon[32800]: fs cephfs is offline because no MDS is active for it. 2024-04-23T15:00:00.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:00 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1016238485' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T15:00:01.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.074+0000 7fcc4ac00640 1 -- 172.21.15.83:0/432047324 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcc4419bdf0 con 0x7fcc44104450 2024-04-23T15:00:01.664 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.664+0000 7fcc40c00640 1 -- 172.21.15.83:0/432047324 <== mon.2 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) ==== 181+0+0 (secure 0 0 0) 0x7fcc340a1070 con 0x7fcc44104450 2024-04-23T15:00:01.665 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_metadata' removed 2024-04-23T15:00:01.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.669+0000 7fcc4ac00640 1 -- 172.21.15.83:0/432047324 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcc280a3d20 msgr2=0x7fcc280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T15:00:01.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.669+0000 7fcc4ac00640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcc280a3d20 0x7fcc280a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcc300096c0 tx=0x7fcc30009260 comp rx=0 tx=0).stop 2024-04-23T15:00:01.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.669+0000 7fcc4ac00640 1 -- 172.21.15.83:0/432047324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc44104450 msgr2=0x7fcc441a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T15:00:01.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.669+0000 7fcc4ac00640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc44104450 0x7fcc441a0710 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e9f0 tx=0x7fcc3400eeb0 comp rx=0 tx=0).stop 2024-04-23T15:00:01.669 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.669+0000 7fcc4ac00640 1 -- 172.21.15.83:0/432047324 shutdown_connections 2024-04-23T15:00:01.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.669+0000 7fcc4ac00640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc4419a8c0 0x7fcc4419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:01.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.669+0000 7fcc4ac00640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fcc280a3d20 0x7fcc280a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:01.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.669+0000 7fcc4ac00640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc44109520 0x7fcc441a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:01.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.669+0000 7fcc4ac00640 1 --2- 172.21.15.83:0/432047324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc44104450 0x7fcc441a0710 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:01.670 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.669+0000 7fcc4ac00640 1 -- 172.21.15.83:0/432047324 >> 172.21.15.83:0/432047324 conn(0x7fcc440ffcd0 msgr2=0x7fcc44101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T15:00:01.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.670+0000 7fcc4ac00640 1 -- 172.21.15.83:0/432047324 shutdown_connections 2024-04-23T15:00:01.671 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.670+0000 7fcc4ac00640 1 -- 172.21.15.83:0/432047324 wait complete. 2024-04-23T15:00:01.694 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T15:00:01.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: pgmap v361: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2024-04-23T15:00:01.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T15:00:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T15:00:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: Cluster is now healthy 2024-04-23T15:00:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/1016238485' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T15:00:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T15:00:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T15:00:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T15:00:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T15:00:01.826 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T15:00:01.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T15:00:01.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.203:6834/3288270779,v1:172.21.15.203:6835/3288270779] up:boot 2024-04-23T15:00:01.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.173:6834/1047930140,v1:172.21.15.173:6836/1047930140] up:boot 2024-04-23T15:00:01.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.173:6837/3202174592,v1:172.21.15.173:6839/3202174592] up:boot 2024-04-23T15:00:01.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.173:6835/1159444016,v1:172.21.15.173:6838/1159444016] up:boot 2024-04-23T15:00:01.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.203:6836/2186096338,v1:172.21.15.203:6837/2186096338] up:boot 2024-04-23T15:00:01.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: mds.? [v2:172.21.15.203:6838/807387310,v1:172.21.15.203:6839/807387310] up:boot 2024-04-23T15:00:01.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: fsmap 12 up:standby 2024-04-23T15:00:01.827 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T15:00:01.828 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/432047324' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T15:00:01.828 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:01 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T15:00:01.911 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.912+0000 7fb0ae000640 1 -- 172.21.15.83:0/2670966314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a8112dd0 msgr2=0x7fb0a81151c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T15:00:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.912+0000 7fb0ae000640 1 --2- 172.21.15.83:0/2670966314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a8112dd0 0x7fb0a81151c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009ca0 tx=0x7fb0a002a950 comp rx=0 tx=0).stop 2024-04-23T15:00:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.912+0000 7fb0ae000640 1 -- 172.21.15.83:0/2670966314 shutdown_connections 2024-04-23T15:00:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.912+0000 7fb0ae000640 1 --2- 172.21.15.83:0/2670966314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a8112dd0 0x7fb0a81151c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.912+0000 7fb0ae000640 1 --2- 172.21.15.83:0/2670966314 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8074cc0 0x7fb0a8112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.912+0000 7fb0ae000640 1 --2- 172.21.15.83:0/2670966314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a80743a0 0x7fb0a8074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.912+0000 7fb0ae000640 1 -- 172.21.15.83:0/2670966314 >> 172.21.15.83:0/2670966314 conn(0x7fb0a80fd8d0 msgr2=0x7fb0a80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T15:00:01.912 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.912+0000 7fb0ae000640 1 -- 172.21.15.83:0/2670966314 shutdown_connections 2024-04-23T15:00:01.913 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.912+0000 7fb0ae000640 1 -- 172.21.15.83:0/2670966314 wait complete. 2024-04-23T15:00:01.914 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.913+0000 7fb0ae000640 1 Processor -- start 2024-04-23T15:00:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.913+0000 7fb0ae000640 1 -- start start 2024-04-23T15:00:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.913+0000 7fb0ae000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a80743a0 0x7fb0a81a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T15:00:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.913+0000 7fb0ae000640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8074cc0 0x7fb0a81a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T15:00:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.913+0000 7fb0ae000640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a8112dd0 0x7fb0a819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T15:00:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.913+0000 7fb0ae000640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8118790 con 0x7fb0a8112dd0 2024-04-23T15:00:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.913+0000 7fb0ae000640 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8118610 con 0x7fb0a8074cc0 2024-04-23T15:00:01.915 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.913+0000 7fb0ae000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8118910 con 0x7fb0a80743a0 2024-04-23T15:00:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0ad600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a8112dd0 0x7fb0a819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T15:00:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0acc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a80743a0 0x7fb0a81a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T15:00:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0a7e00640 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8074cc0 0x7fb0a81a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T15:00:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0ad600640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a8112dd0 0x7fb0a819eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:56718/0 (socket says 172.21.15.83:56718) 2024-04-23T15:00:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0acc00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a80743a0 0x7fb0a81a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.83:50130/0 (socket says 172.21.15.83:50130) 2024-04-23T15:00:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0ad600640 1 -- 172.21.15.83:0/4140285035 learned_addr learned my addr 172.21.15.83:0/4140285035 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-23T15:00:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0acc00640 1 -- 172.21.15.83:0/4140285035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8074cc0 msgr2=0x7fb0a81a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T15:00:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0acc00640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8074cc0 0x7fb0a81a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:01.916 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0acc00640 1 -- 172.21.15.83:0/4140285035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a8112dd0 msgr2=0x7fb0a819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T15:00:01.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0acc00640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a8112dd0 0x7fb0a819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:01.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0acc00640 1 -- 172.21.15.83:0/4140285035 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a819f270 con 0x7fb0a80743a0 2024-04-23T15:00:01.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0a7e00640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8074cc0 0x7fb0a81a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T15:00:01.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.914+0000 7fb0acc00640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a80743a0 0x7fb0a81a48f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00e9f0 tx=0x7fb09c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T15:00:01.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.915+0000 7fb0a5600640 1 -- 172.21.15.83:0/4140285035 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c00ccc0 con 0x7fb0a80743a0 2024-04-23T15:00:01.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.915+0000 7fb0a5600640 1 -- 172.21.15.83:0/4140285035 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb09c005020 con 0x7fb0a80743a0 2024-04-23T15:00:01.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.915+0000 7fb0a5600640 1 -- 172.21.15.83:0/4140285035 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0049e0 con 0x7fb0a80743a0 2024-04-23T15:00:01.917 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.915+0000 7fb0ae000640 1 -- 172.21.15.83:0/4140285035 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a819f560 con 0x7fb0a80743a0 2024-04-23T15:00:01.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.915+0000 7fb0ae000640 1 -- 172.21.15.83:0/4140285035 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a81ab7b0 con 0x7fb0a80743a0 2024-04-23T15:00:01.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.916+0000 7fb0ae000640 1 -- 172.21.15.83:0/4140285035 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a8068d40 con 0x7fb0a80743a0 2024-04-23T15:00:01.918 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.918+0000 7fb0a5600640 1 -- 172.21.15.83:0/4140285035 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb09c005560 con 0x7fb0a80743a0 2024-04-23T15:00:01.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.919+0000 7fb0a5600640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb0900a3da0 0x7fb0900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T15:00:01.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.920+0000 7fb0a5600640 1 -- 172.21.15.83:0/4140285035 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8202+0+0 (secure 0 0 0) 0x7fb09c0d0600 con 0x7fb0a80743a0 2024-04-23T15:00:01.919 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.920+0000 7fb0a7e00640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb0900a3da0 0x7fb0900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T15:00:01.920 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.920+0000 7fb0a7e00640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb0900a3da0 0x7fb0900a6260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8105ba0 tx=0x7fb094005f90 comp rx=0 tx=0).ready entity=mgr.15114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T15:00:01.923 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:01.924+0000 7fb0a5600640 1 -- 172.21.15.83:0/4140285035 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb09c0051d0 con 0x7fb0a80743a0 2024-04-23T15:00:01.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: pgmap v361: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2024-04-23T15:00:01.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T15:00:01.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T15:00:01.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: Cluster is now healthy 2024-04-23T15:00:01.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/1016238485' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T15:00:01.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T15:00:01.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T15:00:01.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T15:00:01.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T15:00:01.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T15:00:01.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T15:00:01.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.203:6834/3288270779,v1:172.21.15.203:6835/3288270779] up:boot 2024-04-23T15:00:01.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.173:6834/1047930140,v1:172.21.15.173:6836/1047930140] up:boot 2024-04-23T15:00:01.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.173:6837/3202174592,v1:172.21.15.173:6839/3202174592] up:boot 2024-04-23T15:00:01.972 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.173:6835/1159444016,v1:172.21.15.173:6838/1159444016] up:boot 2024-04-23T15:00:01.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.203:6836/2186096338,v1:172.21.15.203:6837/2186096338] up:boot 2024-04-23T15:00:01.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: mds.? [v2:172.21.15.203:6838/807387310,v1:172.21.15.203:6839/807387310] up:boot 2024-04-23T15:00:01.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: fsmap 12 up:standby 2024-04-23T15:00:01.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T15:00:01.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/432047324' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T15:00:01.973 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:01 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T15:00:02.006 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: pgmap v361: 97 pgs: 97 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2024-04-23T15:00:02.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T15:00:02.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T15:00:02.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: Cluster is now healthy 2024-04-23T15:00:02.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/1016238485' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T15:00:02.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T15:00:02.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T15:00:02.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T15:00:02.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T15:00:02.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T15:00:02.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T15:00:02.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.203:6834/3288270779,v1:172.21.15.203:6835/3288270779] up:boot 2024-04-23T15:00:02.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.173:6834/1047930140,v1:172.21.15.173:6836/1047930140] up:boot 2024-04-23T15:00:02.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.173:6837/3202174592,v1:172.21.15.173:6839/3202174592] up:boot 2024-04-23T15:00:02.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.173:6835/1159444016,v1:172.21.15.173:6838/1159444016] up:boot 2024-04-23T15:00:02.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.203:6836/2186096338,v1:172.21.15.203:6837/2186096338] up:boot 2024-04-23T15:00:02.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: mds.? [v2:172.21.15.203:6838/807387310,v1:172.21.15.203:6839/807387310] up:boot 2024-04-23T15:00:02.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: fsmap 12 up:standby 2024-04-23T15:00:02.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T15:00:02.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/432047324' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T15:00:02.009 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:01 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T15:00:02.134 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.134+0000 7fb0ae000640 1 -- 172.21.15.83:0/4140285035 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb0a81052a0 con 0x7fb0a80743a0 2024-04-23T15:00:02.902 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.902+0000 7fb0a5600640 1 -- 172.21.15.83:0/4140285035 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) ==== 169+0+0 (secure 0 0 0) 0x7fb09c09cd80 con 0x7fb0a80743a0 2024-04-23T15:00:02.905 INFO:teuthology.orchestra.run.smithi083.stderr:pool 'cephfs_data' removed 2024-04-23T15:00:02.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.907+0000 7fb0ae000640 1 -- 172.21.15.83:0/4140285035 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb0900a3da0 msgr2=0x7fb0900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T15:00:02.907 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.907+0000 7fb0ae000640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb0900a3da0 0x7fb0900a6260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8105ba0 tx=0x7fb094005f90 comp rx=0 tx=0).stop 2024-04-23T15:00:02.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.907+0000 7fb0ae000640 1 -- 172.21.15.83:0/4140285035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a80743a0 msgr2=0x7fb0a81a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T15:00:02.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.907+0000 7fb0ae000640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a80743a0 0x7fb0a81a48f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00e9f0 tx=0x7fb09c00eeb0 comp rx=0 tx=0).stop 2024-04-23T15:00:02.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.908+0000 7fb0ae000640 1 -- 172.21.15.83:0/4140285035 shutdown_connections 2024-04-23T15:00:02.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.908+0000 7fb0ae000640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.83:6800/1667121447,v1:172.21.15.83:6801/1667121447] conn(0x7fb0900a3da0 0x7fb0900a6260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:02.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.908+0000 7fb0ae000640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a8112dd0 0x7fb0a819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:02.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.908+0000 7fb0ae000640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0a8074cc0 0x7fb0a81a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:02.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.908+0000 7fb0ae000640 1 --2- 172.21.15.83:0/4140285035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a80743a0 0x7fb0a81a48f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T15:00:02.908 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.908+0000 7fb0ae000640 1 -- 172.21.15.83:0/4140285035 >> 172.21.15.83:0/4140285035 conn(0x7fb0a80fd8d0 msgr2=0x7fb0a80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T15:00:02.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.908+0000 7fb0ae000640 1 -- 172.21.15.83:0/4140285035 shutdown_connections 2024-04-23T15:00:02.909 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-23T15:00:02.908+0000 7fb0ae000640 1 -- 172.21.15.83:0/4140285035 wait complete. 2024-04-23T15:00:02.935 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:02 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T15:00:02.935 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:02 smithi083 ceph-mon[27867]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T15:00:02.935 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:02 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T15:00:02.936 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:02 smithi083 ceph-mon[27867]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T15:00:02.936 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:02 smithi083 ceph-mon[27867]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T15:00:02.936 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:02 smithi083 ceph-mon[27867]: from='client.? 172.21.15.83:0/4140285035' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T15:00:02.936 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:02 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T15:00:02.939 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T15:00:02.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:02 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T15:00:02.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:02 smithi203 ceph-mon[32800]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T15:00:02.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:02 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T15:00:02.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:02 smithi203 ceph-mon[32800]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T15:00:02.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:02 smithi203 ceph-mon[32800]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T15:00:02.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:02 smithi203 ceph-mon[32800]: from='client.? 172.21.15.83:0/4140285035' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T15:00:02.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:02 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T15:00:03.015 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T15:00:03.040 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:02 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T15:00:03.040 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:02 smithi173 ceph-mon[32443]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T15:00:03.040 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:02 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T15:00:03.040 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:02 smithi173 ceph-mon[32443]: from='mgr.15114 172.21.15.83:0/1378772287' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T15:00:03.041 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:02 smithi173 ceph-mon[32443]: from='mgr.15114 ' entity='mgr.x' 2024-04-23T15:00:03.041 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:02 smithi173 ceph-mon[32443]: from='client.? 172.21.15.83:0/4140285035' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T15:00:03.041 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:02 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T15:00:03.046 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T15:00:03.075 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-23T15:00:03.075 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T15:00:03.120 DEBUG:teuthology.orchestra.run.smithi173:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T15:00:03.136 DEBUG:teuthology.orchestra.run.smithi203:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T15:00:03.152 INFO:tasks.cephadm:Stopping all daemons... 2024-04-23T15:00:03.153 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-23T15:00:03.153 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.a 2024-04-23T15:00:03.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:03 smithi083 systemd[1]: Stopping Ceph mon.a for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:00:03.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:03 smithi203 ceph-mon[32800]: pgmap v363: 65 pgs: 65 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T15:00:03.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:03 smithi203 ceph-mon[32800]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T15:00:03.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:03 smithi203 ceph-mon[32800]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T15:00:04.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:03 smithi083 ceph-mon[27867]: pgmap v363: 65 pgs: 65 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T15:00:04.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:03 smithi083 ceph-mon[27867]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T15:00:04.007 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:03 smithi083 ceph-mon[27867]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T15:00:04.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:03 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a[27863]: 2024-04-23T15:00:03.811+0000 7f8008800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:00:04.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:03 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a[27863]: 2024-04-23T15:00:03.811+0000 7f8008800700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-23T15:00:04.008 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:03 smithi083 podman[95485]: 2024-04-23 15:00:03.896448406 +0000 UTC m=+0.257007144 container died 4c2a91033a2a8cd38abaf6838701787456d7fa3872bce62a29bcbb48325afea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:00:04.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:03 smithi173 ceph-mon[32443]: pgmap v363: 65 pgs: 65 active+clean; 13 GiB data, 29 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T15:00:04.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:03 smithi173 ceph-mon[32443]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T15:00:04.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:03 smithi173 ceph-mon[32443]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T15:00:04.506 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:04 smithi083 podman[95485]: 2024-04-23 15:00:04.342669024 +0000 UTC m=+0.703227755 container cleanup 4c2a91033a2a8cd38abaf6838701787456d7fa3872bce62a29bcbb48325afea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-23T15:00:04.507 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:04 smithi083 bash[95485]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a 2024-04-23T15:00:05.256 INFO:journalctl@ceph.mon.a.smithi083.stdout:Apr 23 15:00:04 smithi083 podman[95496]: 2024-04-23 15:00:04.845482465 +0000 UTC m=+0.930596560 container remove 4c2a91033a2a8cd38abaf6838701787456d7fa3872bce62a29bcbb48325afea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-a, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T15:00:05.367 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.a.service' 2024-04-23T15:00:05.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:00:05.536 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-23T15:00:05.536 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-23T15:00:05.536 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.b 2024-04-23T15:00:05.825 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:05 smithi173 systemd[1]: Stopping Ceph mon.b for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:00:07.075 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:06 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-b[32439]: 2024-04-23T15:00:06.817+0000 7f014ed4a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:00:07.076 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:06 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-b[32439]: 2024-04-23T15:00:06.817+0000 7f014ed4a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-23T15:00:07.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:07 smithi173 podman[68681]: 2024-04-23 15:00:07.085751754 +0000 UTC m=+1.229387451 container died 8d63442d7a983b5adb33ebc441672241952ed59deede7989fd04063a00e0d59e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-23T15:00:11.575 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:11 smithi173 podman[68681]: 2024-04-23 15:00:11.097550597 +0000 UTC m=+5.241186296 container cleanup 8d63442d7a983b5adb33ebc441672241952ed59deede7989fd04063a00e0d59e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-23T15:00:11.576 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:11 smithi173 bash[68681]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-b 2024-04-23T15:00:12.013 INFO:journalctl@ceph.mon.b.smithi173.stdout:Apr 23 15:00:11 smithi173 podman[68693]: 2024-04-23 15:00:11.683046725 +0000 UTC m=+4.593567885 container remove 8d63442d7a983b5adb33ebc441672241952ed59deede7989fd04063a00e0d59e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2) 2024-04-23T15:00:12.031 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.b.service' 2024-04-23T15:00:12.102 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:00:12.102 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-23T15:00:12.103 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-23T15:00:12.103 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.c 2024-04-23T15:00:12.721 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:12 smithi203 systemd[1]: Stopping Ceph mon.c for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:00:14.701 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:14 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-c[32796]: 2024-04-23T15:00:14.407+0000 7fedc3787700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:00:14.702 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:14 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-c[32796]: 2024-04-23T15:00:14.407+0000 7fedc3787700 -1 mon.c@1(probing) e3 *** Got Signal Terminated *** 2024-04-23T15:00:14.971 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:14 smithi203 podman[69333]: 2024-04-23 15:00:14.702257789 +0000 UTC m=+1.616502494 container died 06fa1ec7928659783b41e449b5525419181bc1b0f99e0c41f71e32d17c3e413c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-23T15:00:21.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:20 smithi203 podman[69333]: 2024-04-23 15:00:20.821227467 +0000 UTC m=+7.735472148 container cleanup 06fa1ec7928659783b41e449b5525419181bc1b0f99e0c41f71e32d17c3e413c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T15:00:21.221 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:20 smithi203 bash[69333]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-c 2024-04-23T15:00:24.785 INFO:journalctl@ceph.mon.c.smithi203.stdout:Apr 23 15:00:24 smithi203 podman[69351]: 2024-04-23 15:00:24.524938636 +0000 UTC m=+9.826210258 container remove 06fa1ec7928659783b41e449b5525419181bc1b0f99e0c41f71e32d17c3e413c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-23T15:00:24.799 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mon.c.service' 2024-04-23T15:00:24.904 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:00:24.905 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-23T15:00:24.905 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-23T15:00:24.905 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.x 2024-04-23T15:00:25.256 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 15:00:25 smithi083 systemd[1]: Stopping Ceph mgr.x for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:00:28.007 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 15:00:27 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T15:00:27.680+0000 7f0b71800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:00:28.007 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 15:00:27 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x[28097]: 2024-04-23T15:00:27.680+0000 7f0b71800700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-23T15:00:28.007 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 15:00:27 smithi083 podman[95596]: 2024-04-23 15:00:27.775678256 +0000 UTC m=+2.471299138 container died 00a122261f10fc638711ef69ea53cf03db57490734f4692e215ace2d337b161e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-23T15:00:28.506 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 15:00:28 smithi083 podman[95596]: 2024-04-23 15:00:28.275579486 +0000 UTC m=+2.971200402 container cleanup 00a122261f10fc638711ef69ea53cf03db57490734f4692e215ace2d337b161e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416) 2024-04-23T15:00:28.507 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 15:00:28 smithi083 bash[95596]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x 2024-04-23T15:00:29.006 INFO:journalctl@ceph.mgr.x.smithi083.stdout:Apr 23 15:00:28 smithi083 podman[95610]: 2024-04-23 15:00:28.544187788 +0000 UTC m=+0.746801940 container remove 00a122261f10fc638711ef69ea53cf03db57490734f4692e215ace2d337b161e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d44fddd910b5e94da0f2357af4be37d7bd495ae0, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T15:00:29.062 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-23T15:00:29.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:00:29.171 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-23T15:00:29.171 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-23T15:00:29.171 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.y 2024-04-23T15:00:29.575 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 15:00:29 smithi173 systemd[1]: Stopping Ceph mgr.y for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:00:30.075 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 15:00:29 smithi173 podman[68804]: 2024-04-23 15:00:29.823987152 +0000 UTC m=+0.457502047 container died 0c1434fbabb195a7562e692c0158e201be8b9f5a340e3734edf3a29a79ed5ef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-23T15:00:30.522 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 15:00:30 smithi173 podman[68804]: 2024-04-23 15:00:30.27110464 +0000 UTC m=+0.904619541 container cleanup 0c1434fbabb195a7562e692c0158e201be8b9f5a340e3734edf3a29a79ed5ef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True) 2024-04-23T15:00:30.522 INFO:journalctl@ceph.mgr.y.smithi173.stdout:Apr 23 15:00:30 smithi173 bash[68804]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-y 2024-04-23T15:00:30.728 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-23T15:00:30.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:00:30.764 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-23T15:00:30.764 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-23T15:00:30.764 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.z 2024-04-23T15:00:31.221 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 15:00:30 smithi203 systemd[1]: Stopping Ceph mgr.z for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:00:32.971 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 15:00:32 smithi203 podman[69471]: 2024-04-23 15:00:32.52779492 +0000 UTC m=+1.562053041 container died 96b8fc56554b16714cd7d4d69c92541cd57fbd01194d34884786150cc1b40ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416) 2024-04-23T15:00:35.221 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 15:00:34 smithi203 podman[69471]: 2024-04-23 15:00:34.972970885 +0000 UTC m=+4.007229007 container cleanup 96b8fc56554b16714cd7d4d69c92541cd57fbd01194d34884786150cc1b40ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T15:00:35.221 INFO:journalctl@ceph.mgr.z.smithi203.stdout:Apr 23 15:00:34 smithi203 bash[69471]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mgr-z 2024-04-23T15:00:35.454 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-23T15:00:35.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:00:35.488 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-23T15:00:35.488 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-23T15:00:35.488 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.0 2024-04-23T15:00:36.006 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:35 smithi083 systemd[1]: Stopping Ceph osd.0 for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:00:36.507 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:36 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0[42503]: 2024-04-23T15:00:36.063+0000 7f02fc800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:00:36.507 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:36 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0[42503]: 2024-04-23T15:00:36.063+0000 7f02fc800700 -1 osd.0 102 *** Got signal Terminated *** 2024-04-23T15:00:36.507 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:36 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0[42503]: 2024-04-23T15:00:36.063+0000 7f02fc800700 -1 osd.0 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T15:00:42.257 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:42 smithi083 podman[95728]: 2024-04-23 15:00:42.029487769 +0000 UTC m=+6.108816379 container died f638f6fc65b630eaef2dcda1bd8f6a7bd2ade534990a9e64ca58264366063ac2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-23T15:00:42.743 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:42 smithi083 podman[95728]: 2024-04-23 15:00:42.483282505 +0000 UTC m=+6.562611112 container cleanup f638f6fc65b630eaef2dcda1bd8f6a7bd2ade534990a9e64ca58264366063ac2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-23T15:00:42.744 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:42 smithi083 bash[95728]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0 2024-04-23T15:00:43.006 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:42 smithi083 podman[95748]: 2024-04-23 15:00:42.743029144 +0000 UTC m=+0.698564338 container remove f638f6fc65b630eaef2dcda1bd8f6a7bd2ade534990a9e64ca58264366063ac2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-23T15:00:43.985 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:43 smithi083 podman[95839]: 2024-04-23 15:00:43.514362832 +0000 UTC m=+0.032917472 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T15:00:43.985 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:43 smithi083 podman[95839]: 2024-04-23 15:00:43.644081548 +0000 UTC m=+0.162636213 container create 0ba139702d0cb358984e029ff7dfa56dcefac3ceed662bc0e973b1ec3878ab0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T15:00:43.985 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:43 smithi083 podman[95839]: 2024-04-23 15:00:43.984209489 +0000 UTC m=+0.502764171 container init 0ba139702d0cb358984e029ff7dfa56dcefac3ceed662bc0e973b1ec3878ab0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T15:00:44.256 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:43 smithi083 podman[95839]: 2024-04-23 15:00:43.995146216 +0000 UTC m=+0.513700858 container start 0ba139702d0cb358984e029ff7dfa56dcefac3ceed662bc0e973b1ec3878ab0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-23T15:00:44.257 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:44 smithi083 podman[95839]: 2024-04-23 15:00:44.113919303 +0000 UTC m=+0.632473991 container attach 0ba139702d0cb358984e029ff7dfa56dcefac3ceed662bc0e973b1ec3878ab0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-23T15:00:45.221 INFO:journalctl@ceph.osd.0.smithi083.stdout:Apr 23 15:00:44 smithi083 podman[95839]: 2024-04-23 15:00:44.865952317 +0000 UTC m=+1.384506992 container died 0ba139702d0cb358984e029ff7dfa56dcefac3ceed662bc0e973b1ec3878ab0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T15:00:45.660 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.0.service' 2024-04-23T15:00:45.767 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:00:45.768 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-23T15:00:45.768 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-23T15:00:45.768 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.1 2024-04-23T15:00:46.075 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:45 smithi173 systemd[1]: Stopping Ceph osd.1 for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:00:46.575 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:46 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1[37018]: 2024-04-23T15:00:46.082+0000 7f8993aff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:00:46.576 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:46 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1[37018]: 2024-04-23T15:00:46.082+0000 7f8993aff700 -1 osd.1 102 *** Got signal Terminated *** 2024-04-23T15:00:46.576 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:46 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1[37018]: 2024-04-23T15:00:46.082+0000 7f8993aff700 -1 osd.1 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T15:00:52.068 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:51 smithi173 podman[68933]: 2024-04-23 15:00:51.787699914 +0000 UTC m=+5.826777456 container died 36ee470295f34d3a1ea97a9ab7de460c833a73d6e67c47158a7068bd6bc806d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True) 2024-04-23T15:00:52.325 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:52 smithi173 podman[68933]: 2024-04-23 15:00:52.159365314 +0000 UTC m=+6.198442851 container cleanup 36ee470295f34d3a1ea97a9ab7de460c833a73d6e67c47158a7068bd6bc806d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-23T15:00:52.325 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:52 smithi173 bash[68933]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1 2024-04-23T15:00:52.785 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:52 smithi173 podman[68951]: 2024-04-23 15:00:52.477908683 +0000 UTC m=+0.684588916 container remove 36ee470295f34d3a1ea97a9ab7de460c833a73d6e67c47158a7068bd6bc806d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-23T15:00:53.075 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:52 smithi173 podman[69041]: 2024-04-23 15:00:52.68891278 +0000 UTC m=+0.017975262 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T15:00:53.076 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:52 smithi173 podman[69041]: 2024-04-23 15:00:52.82904722 +0000 UTC m=+0.158109692 container create 59fa12a2fbab7be238ffe2b5afe4ebb8472405b964682e6ab29bbd0164453675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_CLEAN=True) 2024-04-23T15:00:53.348 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:53 smithi173 podman[69041]: 2024-04-23 15:00:53.091774092 +0000 UTC m=+0.420836564 container init 59fa12a2fbab7be238ffe2b5afe4ebb8472405b964682e6ab29bbd0164453675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-23T15:00:53.348 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:53 smithi173 podman[69041]: 2024-04-23 15:00:53.095037581 +0000 UTC m=+0.424100058 container start 59fa12a2fbab7be238ffe2b5afe4ebb8472405b964682e6ab29bbd0164453675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-23T15:00:53.348 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:53 smithi173 podman[69041]: 2024-04-23 15:00:53.216703243 +0000 UTC m=+0.545765731 container attach 59fa12a2fbab7be238ffe2b5afe4ebb8472405b964682e6ab29bbd0164453675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-23T15:00:53.729 INFO:journalctl@ceph.osd.1.smithi173.stdout:Apr 23 15:00:53 smithi173 podman[69041]: 2024-04-23 15:00:53.416974361 +0000 UTC m=+0.746036828 container died 59fa12a2fbab7be238ffe2b5afe4ebb8472405b964682e6ab29bbd0164453675 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-23T15:00:54.056 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.1.service' 2024-04-23T15:00:54.093 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:00:54.093 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-23T15:00:54.094 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-23T15:00:54.094 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.2 2024-04-23T15:00:54.412 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:00:54 smithi203 systemd[1]: Stopping Ceph osd.2 for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:00:54.721 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:00:54 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2[37959]: 2024-04-23T15:00:54.410+0000 7f68809a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:00:54.721 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:00:54 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2[37959]: 2024-04-23T15:00:54.410+0000 7f68809a4700 -1 osd.2 102 *** Got signal Terminated *** 2024-04-23T15:00:54.721 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:00:54 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2[37959]: 2024-04-23T15:00:54.410+0000 7f68809a4700 -1 osd.2 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T15:01:00.438 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:01:00 smithi203 podman[69596]: 2024-04-23 15:01:00.098015069 +0000 UTC m=+5.815898060 container died 2aa7ac21b3cbd66e6a514fb1c499e6e65f33292f6d3fb8443ad4b48154a39475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-23T15:01:00.721 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:01:00 smithi203 podman[69596]: 2024-04-23 15:01:00.529724024 +0000 UTC m=+6.247607012 container cleanup 2aa7ac21b3cbd66e6a514fb1c499e6e65f33292f6d3fb8443ad4b48154a39475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-23T15:01:00.721 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:01:00 smithi203 bash[69596]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2 2024-04-23T15:01:01.146 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:01:00 smithi203 podman[69625]: 2024-04-23 15:01:00.749112618 +0000 UTC m=+0.637958685 container remove 2aa7ac21b3cbd66e6a514fb1c499e6e65f33292f6d3fb8443ad4b48154a39475 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T15:01:01.421 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:01:01 smithi203 podman[69715]: 2024-04-23 15:01:01.049835611 +0000 UTC m=+0.018138594 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T15:01:01.422 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:01:01 smithi203 podman[69715]: 2024-04-23 15:01:01.247333535 +0000 UTC m=+0.215636500 container create 33b94e210cdf659a7da1f33300a26e482d9df4a4e453587e8d3a367b210cb483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-23T15:01:01.721 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:01:01 smithi203 podman[69715]: 2024-04-23 15:01:01.587917761 +0000 UTC m=+0.556220726 container init 33b94e210cdf659a7da1f33300a26e482d9df4a4e453587e8d3a367b210cb483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-23T15:01:01.721 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:01:01 smithi203 podman[69715]: 2024-04-23 15:01:01.591073485 +0000 UTC m=+0.559376453 container start 33b94e210cdf659a7da1f33300a26e482d9df4a4e453587e8d3a367b210cb483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T15:01:02.221 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:01:01 smithi203 podman[69715]: 2024-04-23 15:01:01.730647638 +0000 UTC m=+0.698950606 container attach 33b94e210cdf659a7da1f33300a26e482d9df4a4e453587e8d3a367b210cb483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, maintainer=Guillaume Abrioux ) 2024-04-23T15:01:02.221 INFO:journalctl@ceph.osd.2.smithi203.stdout:Apr 23 15:01:01 smithi203 podman[69715]: 2024-04-23 15:01:01.927983241 +0000 UTC m=+0.896286198 container died 33b94e210cdf659a7da1f33300a26e482d9df4a4e453587e8d3a367b210cb483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T15:01:02.600 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.2.service' 2024-04-23T15:01:02.640 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:01:02.640 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-23T15:01:02.641 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-23T15:01:02.641 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.3 2024-04-23T15:01:03.007 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:02 smithi083 systemd[1]: Stopping Ceph osd.3 for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:01:03.507 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:03 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-3[48977]: 2024-04-23T15:01:03.251+0000 7effc6000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:01:03.507 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:03 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-3[48977]: 2024-04-23T15:01:03.251+0000 7effc6000700 -1 osd.3 102 *** Got signal Terminated *** 2024-04-23T15:01:03.507 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:03 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-3[48977]: 2024-04-23T15:01:03.251+0000 7effc6000700 -1 osd.3 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T15:01:09.412 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:09 smithi083 podman[95959]: 2024-04-23 15:01:09.050183457 +0000 UTC m=+5.941237727 container died eeda600cd284cb1a30191f65ca3f465f2f0a14bf45800b74820046ccdcd559f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-23T15:01:09.756 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:09 smithi083 podman[95959]: 2024-04-23 15:01:09.546133733 +0000 UTC m=+6.437188013 container cleanup eeda600cd284cb1a30191f65ca3f465f2f0a14bf45800b74820046ccdcd559f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416) 2024-04-23T15:01:09.757 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:09 smithi083 bash[95959]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-3 2024-04-23T15:01:10.256 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:09 smithi083 podman[95980]: 2024-04-23 15:01:09.789054332 +0000 UTC m=+0.708019295 container remove eeda600cd284cb1a30191f65ca3f465f2f0a14bf45800b74820046ccdcd559f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:01:10.873 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:10 smithi083 podman[96066]: 2024-04-23 15:01:10.523304013 +0000 UTC m=+0.033480018 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T15:01:10.873 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:10 smithi083 podman[96066]: 2024-04-23 15:01:10.69863342 +0000 UTC m=+0.208809462 container create 25d469173f6b75dfccb7f7ec068a8b302d33a5941abf6934d56e7a523602918a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T15:01:11.256 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:11 smithi083 podman[96066]: 2024-04-23 15:01:11.002425449 +0000 UTC m=+0.512601498 container init 25d469173f6b75dfccb7f7ec068a8b302d33a5941abf6934d56e7a523602918a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-23T15:01:11.257 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:11 smithi083 podman[96066]: 2024-04-23 15:01:11.013572844 +0000 UTC m=+0.523748892 container start 25d469173f6b75dfccb7f7ec068a8b302d33a5941abf6934d56e7a523602918a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-23T15:01:11.257 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:11 smithi083 podman[96066]: 2024-04-23 15:01:11.117884581 +0000 UTC m=+0.628060680 container attach 25d469173f6b75dfccb7f7ec068a8b302d33a5941abf6934d56e7a523602918a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T15:01:11.756 INFO:journalctl@ceph.osd.3.smithi083.stdout:Apr 23 15:01:11 smithi083 podman[96066]: 2024-04-23 15:01:11.464028304 +0000 UTC m=+0.974204342 container died 25d469173f6b75dfccb7f7ec068a8b302d33a5941abf6934d56e7a523602918a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-23T15:01:12.304 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.3.service' 2024-04-23T15:01:12.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:01:12.402 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-23T15:01:12.402 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-23T15:01:12.402 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.4 2024-04-23T15:01:12.726 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:12 smithi173 systemd[1]: Stopping Ceph osd.4 for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:01:13.075 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:12 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4[43975]: 2024-04-23T15:01:12.725+0000 7fa120271700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:01:13.075 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:12 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4[43975]: 2024-04-23T15:01:12.725+0000 7fa120271700 -1 osd.4 102 *** Got signal Terminated *** 2024-04-23T15:01:13.075 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:12 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4[43975]: 2024-04-23T15:01:12.725+0000 7fa120271700 -1 osd.4 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T15:01:18.575 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:18 smithi173 podman[69174]: 2024-04-23 15:01:18.281435453 +0000 UTC m=+5.688342787 container died 580aec3dc3588ce8bb002d638fbbed08de2585d9df01167d8ba8db851980b02c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd) 2024-04-23T15:01:19.044 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:18 smithi173 podman[69174]: 2024-04-23 15:01:18.826379708 +0000 UTC m=+6.233287083 container cleanup 580aec3dc3588ce8bb002d638fbbed08de2585d9df01167d8ba8db851980b02c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-23T15:01:19.044 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:18 smithi173 bash[69174]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4 2024-04-23T15:01:19.325 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:19 smithi173 podman[69194]: 2024-04-23 15:01:19.04493973 +0000 UTC m=+0.761200850 container remove 580aec3dc3588ce8bb002d638fbbed08de2585d9df01167d8ba8db851980b02c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:01:19.718 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:19 smithi173 podman[69284]: 2024-04-23 15:01:19.346378364 +0000 UTC m=+0.017643920 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T15:01:19.718 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:19 smithi173 podman[69284]: 2024-04-23 15:01:19.462657797 +0000 UTC m=+0.133923346 container create a7d5c862d13bb5a52ccc59be370756a4caf90c16aa6a633a28e151d0e72b68c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-23T15:01:19.718 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:19 smithi173 podman[69284]: 2024-04-23 15:01:19.71845524 +0000 UTC m=+0.389720789 container init a7d5c862d13bb5a52ccc59be370756a4caf90c16aa6a633a28e151d0e72b68c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-23T15:01:20.015 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:19 smithi173 podman[69284]: 2024-04-23 15:01:19.721602768 +0000 UTC m=+0.392868317 container start a7d5c862d13bb5a52ccc59be370756a4caf90c16aa6a633a28e151d0e72b68c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T15:01:20.016 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:19 smithi173 podman[69284]: 2024-04-23 15:01:19.807427642 +0000 UTC m=+0.478693200 container attach a7d5c862d13bb5a52ccc59be370756a4caf90c16aa6a633a28e151d0e72b68c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T15:01:20.302 INFO:journalctl@ceph.osd.4.smithi173.stdout:Apr 23 15:01:20 smithi173 podman[69284]: 2024-04-23 15:01:20.016188732 +0000 UTC m=+0.687454276 container died a7d5c862d13bb5a52ccc59be370756a4caf90c16aa6a633a28e151d0e72b68c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-04-23T15:01:20.623 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.4.service' 2024-04-23T15:01:20.660 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:01:20.660 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-23T15:01:20.660 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-23T15:01:20.661 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.5 2024-04-23T15:01:20.971 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:20 smithi203 systemd[1]: Stopping Ceph osd.5 for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:01:21.471 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:20 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5[44480]: 2024-04-23T15:01:20.987+0000 7f83eeda2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:01:21.471 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:20 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5[44480]: 2024-04-23T15:01:20.987+0000 7f83eeda2700 -1 osd.5 102 *** Got signal Terminated *** 2024-04-23T15:01:21.471 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:20 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5[44480]: 2024-04-23T15:01:20.987+0000 7f83eeda2700 -1 osd.5 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T15:01:26.939 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:26 smithi203 podman[69837]: 2024-04-23 15:01:26.637503437 +0000 UTC m=+5.780216461 container died 22e8caf1f34e5016a30e1210a90e1fb271163d7094770db8153279996338af62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-23T15:01:27.221 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:27 smithi203 podman[69837]: 2024-04-23 15:01:27.0470289 +0000 UTC m=+6.189741923 container cleanup 22e8caf1f34e5016a30e1210a90e1fb271163d7094770db8153279996338af62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-23T15:01:27.221 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:27 smithi203 bash[69837]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5 2024-04-23T15:01:27.656 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:27 smithi203 podman[69854]: 2024-04-23 15:01:27.29879237 +0000 UTC m=+0.657218025 container remove 22e8caf1f34e5016a30e1210a90e1fb271163d7094770db8153279996338af62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-23T15:01:27.952 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:27 smithi203 podman[69943]: 2024-04-23 15:01:27.559150578 +0000 UTC m=+0.018009794 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T15:01:27.953 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:27 smithi203 podman[69943]: 2024-04-23 15:01:27.691637922 +0000 UTC m=+0.150497168 container create 6a0bdc19cf3bf86b8ba9ee09b33f6eaa3cc7332c8cfde50f798ecf2167741482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-23T15:01:28.220 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:27 smithi203 podman[69943]: 2024-04-23 15:01:27.952966164 +0000 UTC m=+0.411825376 container init 6a0bdc19cf3bf86b8ba9ee09b33f6eaa3cc7332c8cfde50f798ecf2167741482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-23T15:01:28.221 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:27 smithi203 podman[69943]: 2024-04-23 15:01:27.956358278 +0000 UTC m=+0.415217482 container start 6a0bdc19cf3bf86b8ba9ee09b33f6eaa3cc7332c8cfde50f798ecf2167741482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T15:01:28.221 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:28 smithi203 podman[69943]: 2024-04-23 15:01:28.044682249 +0000 UTC m=+0.503541462 container attach 6a0bdc19cf3bf86b8ba9ee09b33f6eaa3cc7332c8cfde50f798ecf2167741482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:01:28.521 INFO:journalctl@ceph.osd.5.smithi203.stdout:Apr 23 15:01:28 smithi203 podman[69943]: 2024-04-23 15:01:28.248066072 +0000 UTC m=+0.706925269 container died 6a0bdc19cf3bf86b8ba9ee09b33f6eaa3cc7332c8cfde50f798ecf2167741482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T15:01:28.883 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.5.service' 2024-04-23T15:01:28.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:01:28.921 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-23T15:01:28.921 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-23T15:01:28.921 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.6 2024-04-23T15:01:29.257 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:29 smithi083 systemd[1]: Stopping Ceph osd.6 for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:01:30.007 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:29 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-6[55526]: 2024-04-23T15:01:29.506+0000 7fcc11e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:01:30.007 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:29 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-6[55526]: 2024-04-23T15:01:29.506+0000 7fcc11e00700 -1 osd.6 102 *** Got signal Terminated *** 2024-04-23T15:01:30.007 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:29 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-6[55526]: 2024-04-23T15:01:29.506+0000 7fcc11e00700 -1 osd.6 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T15:01:35.625 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:35 smithi083 podman[96182]: 2024-04-23 15:01:35.308136507 +0000 UTC m=+5.986050729 container died 56a52a582e43021d19e91dcd9cb6a1e140dc2511220959855d4ada970b1ae792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-23T15:01:36.006 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:35 smithi083 podman[96182]: 2024-04-23 15:01:35.766754606 +0000 UTC m=+6.444668844 container cleanup 56a52a582e43021d19e91dcd9cb6a1e140dc2511220959855d4ada970b1ae792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T15:01:36.006 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:35 smithi083 bash[96182]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-6 2024-04-23T15:01:36.506 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:36 smithi083 podman[96200]: 2024-04-23 15:01:36.009568549 +0000 UTC m=+0.690711618 container remove 56a52a582e43021d19e91dcd9cb6a1e140dc2511220959855d4ada970b1ae792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd) 2024-04-23T15:01:37.157 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:36 smithi083 podman[96288]: 2024-04-23 15:01:36.730013285 +0000 UTC m=+0.040166947 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T15:01:37.157 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:36 smithi083 podman[96288]: 2024-04-23 15:01:36.860558035 +0000 UTC m=+0.170711735 container create 82bc7c23cecac08ccbad45930cd6560a12467a86bc7b6a1c08bc14299cb4d6d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T15:01:37.157 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:37 smithi083 podman[96288]: 2024-04-23 15:01:37.156092587 +0000 UTC m=+0.466246254 container init 82bc7c23cecac08ccbad45930cd6560a12467a86bc7b6a1c08bc14299cb4d6d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-23T15:01:37.506 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:37 smithi083 podman[96288]: 2024-04-23 15:01:37.166984145 +0000 UTC m=+0.477137799 container start 82bc7c23cecac08ccbad45930cd6560a12467a86bc7b6a1c08bc14299cb4d6d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-23T15:01:37.507 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:37 smithi083 podman[96288]: 2024-04-23 15:01:37.279750165 +0000 UTC m=+0.589903831 container attach 82bc7c23cecac08ccbad45930cd6560a12467a86bc7b6a1c08bc14299cb4d6d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-23T15:01:37.974 INFO:journalctl@ceph.osd.6.smithi083.stdout:Apr 23 15:01:37 smithi083 podman[96288]: 2024-04-23 15:01:37.617918158 +0000 UTC m=+0.928071811 container died 82bc7c23cecac08ccbad45930cd6560a12467a86bc7b6a1c08bc14299cb4d6d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd) 2024-04-23T15:01:38.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.6.service' 2024-04-23T15:01:38.504 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:01:38.504 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-23T15:01:38.505 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-23T15:01:38.505 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.7 2024-04-23T15:01:38.806 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:38 smithi173 systemd[1]: Stopping Ceph osd.7 for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:01:39.068 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:38 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7[50825]: 2024-04-23T15:01:38.806+0000 7faee7959700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:01:39.068 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:38 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7[50825]: 2024-04-23T15:01:38.806+0000 7faee7959700 -1 osd.7 102 *** Got signal Terminated *** 2024-04-23T15:01:39.068 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:38 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7[50825]: 2024-04-23T15:01:38.806+0000 7faee7959700 -1 osd.7 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T15:01:44.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:43 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:43.836+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:44.598 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:44 smithi173 podman[69389]: 2024-04-23 15:01:44.34561946 +0000 UTC m=+5.652543538 container died bc0668cb881c81f9f00387750201ef7d453468a86d0d301d5a82f0426c9fca99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:01:44.974 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:44 smithi173 podman[69389]: 2024-04-23 15:01:44.68955369 +0000 UTC m=+5.996477767 container cleanup bc0668cb881c81f9f00387750201ef7d453468a86d0d301d5a82f0426c9fca99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_BRANCH=HEAD, ceph=True) 2024-04-23T15:01:44.974 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:44 smithi173 bash[69389]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7 2024-04-23T15:01:45.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:44 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:44.856+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:45.291 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:44 smithi173 podman[69417]: 2024-04-23 15:01:44.974889346 +0000 UTC m=+0.633339137 container remove bc0668cb881c81f9f00387750201ef7d453468a86d0d301d5a82f0426c9fca99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True) 2024-04-23T15:01:45.292 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:45 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:45.103+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:45.570 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:45 smithi173 podman[69513]: 2024-04-23 15:01:45.292590801 +0000 UTC m=+0.114064478 container create f1f8888a1dd6fdce2bab3794eeae43baa0a770d073dd018966b1aaf797f5629f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-23T15:01:45.570 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:45 smithi173 podman[69513]: 2024-04-23 15:01:45.196769184 +0000 UTC m=+0.018242879 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T15:01:45.570 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:45 smithi173 podman[69513]: 2024-04-23 15:01:45.494984794 +0000 UTC m=+0.316458471 container init f1f8888a1dd6fdce2bab3794eeae43baa0a770d073dd018966b1aaf797f5629f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:01:45.570 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:45 smithi173 podman[69513]: 2024-04-23 15:01:45.498053782 +0000 UTC m=+0.319527460 container start f1f8888a1dd6fdce2bab3794eeae43baa0a770d073dd018966b1aaf797f5629f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T15:01:45.571 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:45 smithi173 podman[69513]: 2024-04-23 15:01:45.570087879 +0000 UTC m=+0.391561572 container attach f1f8888a1dd6fdce2bab3794eeae43baa0a770d073dd018966b1aaf797f5629f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-23T15:01:45.825 INFO:journalctl@ceph.osd.7.smithi173.stdout:Apr 23 15:01:45 smithi173 podman[69513]: 2024-04-23 15:01:45.789552301 +0000 UTC m=+0.611025983 container died f1f8888a1dd6fdce2bab3794eeae43baa0a770d073dd018966b1aaf797f5629f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd) 2024-04-23T15:01:45.968 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:45 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:45.706+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:46.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:45 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:45.834+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:46.221 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:46 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8[51492]: 2024-04-23T15:01:46.060+0000 7f79e2ec0700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.137520+0000 front 2024-04-23T15:00:37.137781+0000 (oldest deadline 2024-04-23T15:01:45.337848+0000) 2024-04-23T15:01:46.374 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:46 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:46.116+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:46.397 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.7.service' 2024-04-23T15:01:46.437 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:01:46.437 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-23T15:01:46.437 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-23T15:01:46.437 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.8 2024-04-23T15:01:46.721 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:46 smithi203 systemd[1]: Stopping Ceph osd.8 for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:01:47.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:46 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:46.705+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:47.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:46 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:46.813+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:47.221 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:46 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8[51492]: 2024-04-23T15:01:46.802+0000 7f79ea8d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:01:47.221 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:46 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8[51492]: 2024-04-23T15:01:46.802+0000 7f79ea8d4700 -1 osd.8 102 *** Got signal Terminated *** 2024-04-23T15:01:47.221 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:46 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8[51492]: 2024-04-23T15:01:46.802+0000 7f79ea8d4700 -1 osd.8 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T15:01:47.221 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:47 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8[51492]: 2024-04-23T15:01:47.042+0000 7f79e2ec0700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.137520+0000 front 2024-04-23T15:00:37.137781+0000 (oldest deadline 2024-04-23T15:01:45.337848+0000) 2024-04-23T15:01:47.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:47 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:47.075+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:48.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:47 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:47.729+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:48.065 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:47 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:47.808+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:48.470 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:48 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8[51492]: 2024-04-23T15:01:48.065+0000 7f79e2ec0700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.137520+0000 front 2024-04-23T15:00:37.137781+0000 (oldest deadline 2024-04-23T15:01:45.337848+0000) 2024-04-23T15:01:48.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:48 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:48.125+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:49.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:48 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:48.688+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:49.221 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:49 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8[51492]: 2024-04-23T15:01:49.020+0000 7f79e2ec0700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.137520+0000 front 2024-04-23T15:00:37.137781+0000 (oldest deadline 2024-04-23T15:01:45.337848+0000) 2024-04-23T15:01:49.222 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:48 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:48.827+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:49.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:49 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:49.110+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:50.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:49 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:49.700+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:50.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:49 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:49.845+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:50.221 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:50 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8[51492]: 2024-04-23T15:01:50.022+0000 7f79e2ec0700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.137520+0000 front 2024-04-23T15:00:37.137781+0000 (oldest deadline 2024-04-23T15:01:45.337848+0000) 2024-04-23T15:01:50.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:50 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:50.143+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:51.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:50 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:50.691+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:51.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:50 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:50.893+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:51.221 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:50 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8[51492]: 2024-04-23T15:01:50.995+0000 7f79e2ec0700 -1 osd.8 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.137520+0000 front 2024-04-23T15:00:37.137781+0000 (oldest deadline 2024-04-23T15:01:45.337848+0000) 2024-04-23T15:01:51.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:51 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:51.146+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:52.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:51 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:51.655+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:52.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:51 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:51.846+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:52.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:52 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:52.128+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:52.720 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:52 smithi203 podman[70065]: 2024-04-23 15:01:52.464284967 +0000 UTC m=+5.828988819 container died df94c99918484deb93f6c97bb5fa144b25aff723c7c0faff3f3c99228333cc3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T15:01:53.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:52 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:52.647+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:53.153 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:52 smithi203 podman[70065]: 2024-04-23 15:01:52.860615818 +0000 UTC m=+6.225319661 container cleanup df94c99918484deb93f6c97bb5fa144b25aff723c7c0faff3f3c99228333cc3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-23T15:01:53.153 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:52 smithi203 bash[70065]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8 2024-04-23T15:01:53.153 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:52 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:52.819+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:53.154 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:52 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:52.819+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:01:53.471 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:53 smithi203 podman[70085]: 2024-04-23 15:01:53.153686418 +0000 UTC m=+0.689097486 container remove df94c99918484deb93f6c97bb5fa144b25aff723c7c0faff3f3c99228333cc3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-23T15:01:53.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:53 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:53.140+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:53.732 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:53 smithi203 podman[70174]: 2024-04-23 15:01:53.38495992 +0000 UTC m=+0.018474099 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T15:01:53.733 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:53 smithi203 podman[70174]: 2024-04-23 15:01:53.488443084 +0000 UTC m=+0.121957252 container create 54c971929003911d78142ca0891d2e9848241a6903f1520e61b17b7b65af8b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-23T15:01:54.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:53 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:53.663+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:54.028 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:53 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:53.769+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:54.028 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:53 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:53.769+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:01:54.028 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:53 smithi203 podman[70174]: 2024-04-23 15:01:53.733238756 +0000 UTC m=+0.366752933 container init 54c971929003911d78142ca0891d2e9848241a6903f1520e61b17b7b65af8b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-23T15:01:54.028 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:53 smithi203 podman[70174]: 2024-04-23 15:01:53.736713669 +0000 UTC m=+0.370227837 container start 54c971929003911d78142ca0891d2e9848241a6903f1520e61b17b7b65af8b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-23T15:01:54.028 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:53 smithi203 podman[70174]: 2024-04-23 15:01:53.816187607 +0000 UTC m=+0.449701783 container attach 54c971929003911d78142ca0891d2e9848241a6903f1520e61b17b7b65af8b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd) 2024-04-23T15:01:54.301 INFO:journalctl@ceph.osd.8.smithi203.stdout:Apr 23 15:01:54 smithi203 podman[70174]: 2024-04-23 15:01:54.02893993 +0000 UTC m=+0.662454098 container died 54c971929003911d78142ca0891d2e9848241a6903f1520e61b17b7b65af8b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-23T15:01:54.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:54 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:54.127+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:54.704 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.8.service' 2024-04-23T15:01:54.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:01:54.743 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-23T15:01:54.743 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-23T15:01:54.743 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.9 2024-04-23T15:01:54.971 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:54 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:54.761+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:54.971 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:54 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:54.761+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:01:55.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:54 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:54.664+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:55.007 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:54 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:54.664+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.234029+0000 front 2024-04-23T15:00:48.233898+0000 (oldest deadline 2024-04-23T15:01:54.134058+0000) 2024-04-23T15:01:55.007 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:54 smithi083 systemd[1]: Stopping Ceph osd.9 for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:01:55.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:55 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:55.129+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:55.706 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:55 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:55.352+0000 7f8bf6a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:01:55.706 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:55 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:55.352+0000 7f8bf6a00700 -1 osd.9 102 *** Got signal Terminated *** 2024-04-23T15:01:55.707 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:55 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:55.352+0000 7f8bf6a00700 -1 osd.9 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T15:01:56.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:55 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:55.705+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:56.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:55 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:55.705+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.234029+0000 front 2024-04-23T15:00:48.233898+0000 (oldest deadline 2024-04-23T15:01:54.134058+0000) 2024-04-23T15:01:56.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:55 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:55.808+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:56.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:55 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:55.808+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:01:56.456 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:56 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:56.136+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:57.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:56 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:56.673+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:57.007 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:56 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:56.673+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.234029+0000 front 2024-04-23T15:00:48.233898+0000 (oldest deadline 2024-04-23T15:01:54.134058+0000) 2024-04-23T15:01:57.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:56 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:56.841+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:57.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:56 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:56.841+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:01:57.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:57 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:57.153+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:57.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:57 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:57.153+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:01:58.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:57 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:57.719+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:58.006 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:57 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:57.719+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.234029+0000 front 2024-04-23T15:00:48.233898+0000 (oldest deadline 2024-04-23T15:01:54.134058+0000) 2024-04-23T15:01:58.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:57 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:57.863+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:58.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:57 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:57.863+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:01:58.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:58 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:58.199+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:58.576 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:58 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:58.199+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:01:59.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:58 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:58.886+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:01:59.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:58 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:58.886+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:01:59.256 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:58 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:58.759+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:01:59.257 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:58 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:58.759+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.234029+0000 front 2024-04-23T15:00:48.233898+0000 (oldest deadline 2024-04-23T15:01:54.134058+0000) 2024-04-23T15:01:59.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:59 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:59.199+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:01:59.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:01:59 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:01:59.199+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:02:00.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:59 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:59.908+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:00.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:01:59 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:01:59.908+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:00.256 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:59 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:59.767+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:37.131224+0000 front 2024-04-23T15:00:37.131330+0000 (oldest deadline 2024-04-23T15:01:45.331993+0000) 2024-04-23T15:02:00.256 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:01:59 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9[62113]: 2024-04-23T15:01:59.767+0000 7f8bed200700 -1 osd.9 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.234029+0000 front 2024-04-23T15:00:48.233898+0000 (oldest deadline 2024-04-23T15:01:54.134058+0000) 2024-04-23T15:02:00.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:00 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:00.154+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:02:00.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:00 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:00.154+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:02:01.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:00 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:00.922+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:01.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:00 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:00.922+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:01.441 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:02:01 smithi083 podman[96393]: 2024-04-23 15:02:01.128343735 +0000 UTC m=+5.946765123 container died ac994596b8b683803e78fad8b1c8bff83447b5034a32611de4cb98d09816670b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-04-23T15:02:01.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:01 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:01.119+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:02:01.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:01 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:01.119+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:02:01.756 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:02:01 smithi083 podman[96393]: 2024-04-23 15:02:01.560645562 +0000 UTC m=+6.379067000 container cleanup ac994596b8b683803e78fad8b1c8bff83447b5034a32611de4cb98d09816670b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-23T15:02:01.756 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:02:01 smithi083 bash[96393]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9 2024-04-23T15:02:02.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:01 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:01.925+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:02.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:01 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:01.925+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:02.256 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:02:01 smithi083 podman[96413]: 2024-04-23 15:02:01.861895692 +0000 UTC m=+0.714605705 container remove ac994596b8b683803e78fad8b1c8bff83447b5034a32611de4cb98d09816670b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-23T15:02:02.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:02 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:02.159+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:02:02.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:02 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:02.159+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:02:03.007 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:02:02 smithi083 podman[96505]: 2024-04-23 15:02:02.615626425 +0000 UTC m=+0.032275172 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T15:02:03.007 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:02:02 smithi083 podman[96505]: 2024-04-23 15:02:02.737882744 +0000 UTC m=+0.154531527 container create 32c23068ebe6680fe9ada5065ec0946ed44c8a301ca5425a827098759fee202c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T15:02:03.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:02 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:02.970+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:03.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:02 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:02.970+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:03.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:02 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:02.970+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:03.488 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:02:03 smithi083 podman[96505]: 2024-04-23 15:02:03.0333623 +0000 UTC m=+0.450011090 container init 32c23068ebe6680fe9ada5065ec0946ed44c8a301ca5425a827098759fee202c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-23T15:02:03.489 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:02:03 smithi083 podman[96505]: 2024-04-23 15:02:03.04398198 +0000 UTC m=+0.460630773 container start 32c23068ebe6680fe9ada5065ec0946ed44c8a301ca5425a827098759fee202c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-23T15:02:03.489 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:02:03 smithi083 podman[96505]: 2024-04-23 15:02:03.131881011 +0000 UTC m=+0.548529821 container attach 32c23068ebe6680fe9ada5065ec0946ed44c8a301ca5425a827098759fee202c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:02:03.489 INFO:journalctl@ceph.osd.9.smithi083.stdout:Apr 23 15:02:03 smithi083 podman[96505]: 2024-04-23 15:02:03.488266831 +0000 UTC m=+0.904915619 container died 32c23068ebe6680fe9ada5065ec0946ed44c8a301ca5425a827098759fee202c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-23T15:02:03.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:03 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:03.171+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:02:03.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:03 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:03.171+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:02:03.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:03 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:03.171+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:57.087576+0000 front 2024-04-23T15:00:57.087544+0000 (oldest deadline 2024-04-23T15:02:02.987712+0000) 2024-04-23T15:02:04.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:03 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:03.950+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:04.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:03 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:03.950+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:04.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:03 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:03.950+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:04.270 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.9.service' 2024-04-23T15:02:04.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:02:04.361 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-23T15:02:04.361 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-23T15:02:04.361 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.10 2024-04-23T15:02:04.387 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:04 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:04.135+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:02:04.387 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:04 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:04.135+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:02:04.387 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:04 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:04.135+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:57.087576+0000 front 2024-04-23T15:00:57.087544+0000 (oldest deadline 2024-04-23T15:02:02.987712+0000) 2024-04-23T15:02:04.677 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:04 smithi173 systemd[1]: Stopping Ceph osd.10 for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:02:05.075 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:04 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:04.676+0000 7f2c61709700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:02:05.075 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:04 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:04.676+0000 7f2c61709700 -1 osd.10 102 *** Got signal Terminated *** 2024-04-23T15:02:05.076 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:04 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:04.676+0000 7f2c61709700 -1 osd.10 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T15:02:05.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:04 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:04.939+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:05.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:04 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:04.939+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:05.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:04 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:04.939+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:05.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:05 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:05.166+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:02:05.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:05 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:05.166+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:02:05.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:05 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:05.166+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:57.087576+0000 front 2024-04-23T15:00:57.087544+0000 (oldest deadline 2024-04-23T15:02:02.987712+0000) 2024-04-23T15:02:06.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:05 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:05.913+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:06.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:05 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:05.913+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:06.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:05 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:05.913+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:06.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:06 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:06.156+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:02:06.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:06 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:06.156+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:02:06.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:06 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:06.156+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:57.087576+0000 front 2024-04-23T15:00:57.087544+0000 (oldest deadline 2024-04-23T15:02:02.987712+0000) 2024-04-23T15:02:07.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:06 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:06.907+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:07.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:06 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:06.907+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:07.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:06 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:06.907+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:07.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:07 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:07.187+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:02:07.576 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:07 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:07.187+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:02:07.576 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:07 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:07.187+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:57.087576+0000 front 2024-04-23T15:00:57.087544+0000 (oldest deadline 2024-04-23T15:02:02.987712+0000) 2024-04-23T15:02:08.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:07 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:07.901+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:08.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:07 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:07.901+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:08.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:07 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:07.901+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:08.476 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:08 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:08.207+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:02:08.477 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:08 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:08.207+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:02:08.477 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:08 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:08.207+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:57.087576+0000 front 2024-04-23T15:00:57.087544+0000 (oldest deadline 2024-04-23T15:02:02.987712+0000) 2024-04-23T15:02:09.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:08 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:08.876+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:09.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:08 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:08.876+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:09.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:08 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:08.876+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:09.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:09 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:09.244+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:40.884290+0000 front 2024-04-23T15:00:40.884414+0000 (oldest deadline 2024-04-23T15:01:44.384197+0000) 2024-04-23T15:02:09.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:09 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:09.244+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:45.985753+0000 front 2024-04-23T15:00:45.985713+0000 (oldest deadline 2024-04-23T15:01:56.586317+0000) 2024-04-23T15:02:09.575 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:09 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10[57601]: 2024-04-23T15:02:09.244+0000 7f2c59cf5700 -1 osd.10 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:57.087576+0000 front 2024-04-23T15:00:57.087544+0000 (oldest deadline 2024-04-23T15:02:02.987712+0000) 2024-04-23T15:02:10.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:09 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:09.912+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:10.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:09 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:09.912+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:10.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:09 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:09.912+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:10.494 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:10 smithi173 podman[69620]: 2024-04-23 15:02:10.222152378 +0000 UTC m=+5.662197060 container died 3d36e1d4b6698e032f1a2b8bf442cb719e3dc4e5b4bc7dd323da61c222d89faa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:02:10.825 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:10 smithi173 podman[69620]: 2024-04-23 15:02:10.593955686 +0000 UTC m=+6.034000370 container cleanup 3d36e1d4b6698e032f1a2b8bf442cb719e3dc4e5b4bc7dd323da61c222d89faa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T15:02:10.825 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:10 smithi173 bash[69620]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10 2024-04-23T15:02:11.198 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:10 smithi173 podman[69649]: 2024-04-23 15:02:10.845617255 +0000 UTC m=+0.619942707 container remove 3d36e1d4b6698e032f1a2b8bf442cb719e3dc4e5b4bc7dd323da61c222d89faa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-23T15:02:11.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:10 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:10.881+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:11.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:10 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:10.881+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:11.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:10 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:10.881+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:11.465 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:11 smithi173 podman[69738]: 2024-04-23 15:02:11.101768636 +0000 UTC m=+0.017416075 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T15:02:11.466 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:11 smithi173 podman[69738]: 2024-04-23 15:02:11.213404494 +0000 UTC m=+0.129051906 container create bc2f35aa940239c7db1207502c23f7a794fb9df18ca79487b909f12478ecf223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-23T15:02:11.466 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:11 smithi173 podman[69738]: 2024-04-23 15:02:11.466165266 +0000 UTC m=+0.381812681 container init bc2f35aa940239c7db1207502c23f7a794fb9df18ca79487b909f12478ecf223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-23T15:02:11.763 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:11 smithi173 podman[69738]: 2024-04-23 15:02:11.469460557 +0000 UTC m=+0.385107970 container start bc2f35aa940239c7db1207502c23f7a794fb9df18ca79487b909f12478ecf223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:02:11.763 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:11 smithi173 podman[69738]: 2024-04-23 15:02:11.557886531 +0000 UTC m=+0.473533952 container attach bc2f35aa940239c7db1207502c23f7a794fb9df18ca79487b909f12478ecf223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-23T15:02:12.075 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:11 smithi173 podman[69738]: 2024-04-23 15:02:11.764229465 +0000 UTC m=+0.679876902 container died bc2f35aa940239c7db1207502c23f7a794fb9df18ca79487b909f12478ecf223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-23T15:02:12.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:11 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:11.907+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:12.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:11 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:11.907+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:12.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:11 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:11.907+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:12.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.10.service' 2024-04-23T15:02:12.424 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:12 smithi173 podman[69764]: 2024-04-23 15:02:12.336674486 +0000 UTC m=+0.567427951 container remove bc2f35aa940239c7db1207502c23f7a794fb9df18ca79487b909f12478ecf223 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-23T15:02:12.424 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:12 smithi173 systemd[1]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-23T15:02:12.424 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:12 smithi173 systemd[1]: Stopped Ceph osd.10 for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T15:02:12.424 INFO:journalctl@ceph.osd.10.smithi173.stdout:Apr 23 15:02:12 smithi173 systemd[1]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.10.service: Consumed 3min 17.166s CPU time. 2024-04-23T15:02:12.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:02:12.441 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-23T15:02:12.441 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-23T15:02:12.441 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.11 2024-04-23T15:02:12.722 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:12 smithi203 systemd[1]: Stopping Ceph osd.11 for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:02:13.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:12 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:12.763+0000 7f84c12e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:02:13.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:12 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:12.763+0000 7f84c12e0700 -1 osd.11 102 *** Got signal Terminated *** 2024-04-23T15:02:13.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:12 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:12.763+0000 7f84c12e0700 -1 osd.11 102 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T15:02:13.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:12 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:12.909+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:13.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:12 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:12.909+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:13.222 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:12 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:12.909+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:14.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:13 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:13.878+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:14.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:13 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:13.878+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:14.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:13 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:13.878+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:15.220 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:14 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:14.897+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:15.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:14 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:14.897+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:15.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:14 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:14.897+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:16.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:15 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:15.933+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:16.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:15 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:15.933+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:16.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:15 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:15.933+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:17.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:16 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:16.899+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.83:6806 osd.0 since back 2024-04-23T15:00:38.136658+0000 front 2024-04-23T15:00:38.136774+0000 (oldest deadline 2024-04-23T15:01:42.836349+0000) 2024-04-23T15:02:17.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:16 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:16.899+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.173:6804 osd.1 since back 2024-04-23T15:00:48.038269+0000 front 2024-04-23T15:00:48.038291+0000 (oldest deadline 2024-04-23T15:01:52.138242+0000) 2024-04-23T15:02:17.221 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:16 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11[58109]: 2024-04-23T15:02:16.899+0000 7f84b98cc700 -1 osd.11 102 heartbeat_check: no reply from 172.21.15.203:6804 osd.2 since back 2024-04-23T15:00:59.139964+0000 front 2024-04-23T15:00:59.139891+0000 (oldest deadline 2024-04-23T15:02:02.540524+0000) 2024-04-23T15:02:18.648 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:18 smithi203 podman[70292]: 2024-04-23 15:02:18.296306646 +0000 UTC m=+5.663051681 container died 54e31043b974b9113b8044826e62743223fa76ab8a4b1cb870e8c649391d753e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:02:18.971 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:18 smithi203 podman[70292]: 2024-04-23 15:02:18.747578631 +0000 UTC m=+6.114323673 container cleanup 54e31043b974b9113b8044826e62743223fa76ab8a4b1cb870e8c649391d753e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-04-23T15:02:18.971 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:18 smithi203 bash[70292]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11 2024-04-23T15:02:19.362 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:19 smithi203 podman[70309]: 2024-04-23 15:02:19.015634657 +0000 UTC m=+0.713875951 container remove 54e31043b974b9113b8044826e62743223fa76ab8a4b1cb870e8c649391d753e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-23T15:02:19.678 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:19 smithi203 podman[70401]: 2024-04-23 15:02:19.265626477 +0000 UTC m=+0.018130689 image pull 76e77f280c74312ca1add9b6019a4a921a516f91795d85bae00a0c8ca9b1c62e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd 2024-04-23T15:02:19.678 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:19 smithi203 podman[70401]: 2024-04-23 15:02:19.374825925 +0000 UTC m=+0.127330164 container create f68fb053386ebd5b59a55a73ed93e8283b0770e94429a29120d2231ac19040a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-23T15:02:19.678 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:19 smithi203 podman[70401]: 2024-04-23 15:02:19.594625042 +0000 UTC m=+0.347129238 container init f68fb053386ebd5b59a55a73ed93e8283b0770e94429a29120d2231ac19040a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T15:02:19.678 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:19 smithi203 podman[70401]: 2024-04-23 15:02:19.597780848 +0000 UTC m=+0.350285051 container start f68fb053386ebd5b59a55a73ed93e8283b0770e94429a29120d2231ac19040a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T15:02:19.971 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:19 smithi203 podman[70401]: 2024-04-23 15:02:19.677771645 +0000 UTC m=+0.430275849 container attach f68fb053386ebd5b59a55a73ed93e8283b0770e94429a29120d2231ac19040a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2) 2024-04-23T15:02:19.971 INFO:journalctl@ceph.osd.11.smithi203.stdout:Apr 23 15:02:19 smithi203 podman[70401]: 2024-04-23 15:02:19.894072593 +0000 UTC m=+0.646576793 container died f68fb053386ebd5b59a55a73ed93e8283b0770e94429a29120d2231ac19040a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-23T15:02:20.475 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@osd.11.service' 2024-04-23T15:02:20.512 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:02:20.512 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-23T15:02:20.513 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-23T15:02:20.513 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.a 2024-04-23T15:02:21.007 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 23 15:02:20 smithi083 systemd[1]: Stopping Ceph mds.a for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:02:21.507 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 23 15:02:21 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-a[68786]: 2024-04-23T15:02:21.118+0000 7efef6800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:02:21.507 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 23 15:02:21 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-a[68786]: 2024-04-23T15:02:21.118+0000 7efef6800700 -1 mds.a *** got signal Terminated *** 2024-04-23T15:02:23.176 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 23 15:02:22 smithi083 podman[96632]: 2024-04-23 15:02:22.898821853 +0000 UTC m=+1.936686302 container died 9d9eb3f563f24986f03a3913f612af7b42c9371afb0137ffc1ba574307d697e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-23T15:02:23.481 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 23 15:02:23 smithi083 podman[96632]: 2024-04-23 15:02:23.304075116 +0000 UTC m=+2.341939568 container cleanup 9d9eb3f563f24986f03a3913f612af7b42c9371afb0137ffc1ba574307d697e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0) 2024-04-23T15:02:23.481 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 23 15:02:23 smithi083 bash[96632]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-a 2024-04-23T15:02:23.756 INFO:journalctl@ceph.mds.a.smithi083.stdout:Apr 23 15:02:23 smithi083 podman[96645]: 2024-04-23 15:02:23.480012088 +0000 UTC m=+0.565442919 container remove 9d9eb3f563f24986f03a3913f612af7b42c9371afb0137ffc1ba574307d697e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-23T15:02:23.962 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.a.service' 2024-04-23T15:02:24.065 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:02:24.069 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-23T15:02:24.069 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-23T15:02:24.070 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.d 2024-04-23T15:02:24.256 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 15:02:24 smithi083 systemd[1]: Stopping Ceph mds.d for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:02:25.006 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 15:02:24 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-d[67741]: 2024-04-23T15:02:24.621+0000 7f9d23a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:02:25.007 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 15:02:24 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-d[67741]: 2024-04-23T15:02:24.621+0000 7f9d23a00700 -1 mds.d *** got signal Terminated *** 2024-04-23T15:02:28.256 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 15:02:27 smithi083 podman[96753]: 2024-04-23 15:02:27.989325013 +0000 UTC m=+3.496774045 container died 024fcdd4320a1f571b7be5cd2be02000aef0117433f60df63169da95bc1ef262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-d, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-23T15:02:28.603 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 15:02:28 smithi083 podman[96753]: 2024-04-23 15:02:28.390781365 +0000 UTC m=+3.898230412 container cleanup 024fcdd4320a1f571b7be5cd2be02000aef0117433f60df63169da95bc1ef262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-23T15:02:28.603 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 15:02:28 smithi083 bash[96753]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-d 2024-04-23T15:02:29.006 INFO:journalctl@ceph.mds.d.smithi083.stdout:Apr 23 15:02:28 smithi083 podman[96767]: 2024-04-23 15:02:28.602483309 +0000 UTC m=+0.596621469 container remove 024fcdd4320a1f571b7be5cd2be02000aef0117433f60df63169da95bc1ef262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:02:29.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.d.service' 2024-04-23T15:02:29.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:02:29.242 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-23T15:02:29.242 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-23T15:02:29.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.g 2024-04-23T15:02:29.506 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 15:02:29 smithi083 systemd[1]: Stopping Ceph mds.g for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:02:30.256 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 15:02:29 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-g[68385]: 2024-04-23T15:02:29.773+0000 7f8029c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:02:30.257 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 15:02:29 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-g[68385]: 2024-04-23T15:02:29.773+0000 7f8029c00700 -1 mds.g *** got signal Terminated *** 2024-04-23T15:02:31.073 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 15:02:30 smithi083 podman[96872]: 2024-04-23 15:02:30.773813506 +0000 UTC m=+1.133152406 container died 8c789337abcfe3b8ada6b0e3015b42a228fd936b06244e71b4bb367ee8eb0cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-23T15:02:31.470 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 15:02:31 smithi083 podman[96872]: 2024-04-23 15:02:31.185885974 +0000 UTC m=+1.545224914 container cleanup 8c789337abcfe3b8ada6b0e3015b42a228fd936b06244e71b4bb367ee8eb0cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:02:31.470 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 15:02:31 smithi083 bash[96872]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-g 2024-04-23T15:02:31.756 INFO:journalctl@ceph.mds.g.smithi083.stdout:Apr 23 15:02:31 smithi083 podman[96886]: 2024-04-23 15:02:31.470139691 +0000 UTC m=+0.680471349 container remove 8c789337abcfe3b8ada6b0e3015b42a228fd936b06244e71b4bb367ee8eb0cdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True) 2024-04-23T15:02:31.993 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.g.service' 2024-04-23T15:02:32.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:02:32.094 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-23T15:02:32.094 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-23T15:02:32.095 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.j 2024-04-23T15:02:32.721 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 15:02:32 smithi083 systemd[1]: Stopping Ceph mds.j for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:02:33.006 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 15:02:32 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-j[67394]: 2024-04-23T15:02:32.719+0000 7faa21e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:02:33.006 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 15:02:32 smithi083 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-j[67394]: 2024-04-23T15:02:32.719+0000 7faa21e00700 -1 mds.j *** got signal Terminated *** 2024-04-23T15:02:36.207 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 15:02:35 smithi083 podman[96992]: 2024-04-23 15:02:35.92925599 +0000 UTC m=+3.341334094 container died 6f4fa56f68af72204b9b4e90a2b325f8b9038addd3e38318ace42368d5a1053a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-23T15:02:36.491 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 15:02:36 smithi083 podman[96992]: 2024-04-23 15:02:36.322747341 +0000 UTC m=+3.734825459 container cleanup 6f4fa56f68af72204b9b4e90a2b325f8b9038addd3e38318ace42368d5a1053a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-23T15:02:36.491 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 15:02:36 smithi083 bash[96992]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-j 2024-04-23T15:02:36.756 INFO:journalctl@ceph.mds.j.smithi083.stdout:Apr 23 15:02:36 smithi083 podman[97006]: 2024-04-23 15:02:36.490083097 +0000 UTC m=+0.545587586 container remove 6f4fa56f68af72204b9b4e90a2b325f8b9038addd3e38318ace42368d5a1053a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-j, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-23T15:02:36.969 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.j.service' 2024-04-23T15:02:37.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:02:37.071 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-23T15:02:37.071 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-23T15:02:37.072 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.b 2024-04-23T15:02:37.354 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 15:02:37 smithi173 systemd[1]: Stopping Ceph mds.b for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:02:37.825 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 15:02:37 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b[63123]: 2024-04-23T15:02:37.354+0000 7f7b3d71d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:02:37.825 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 15:02:37 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b[63123]: 2024-04-23T15:02:37.354+0000 7f7b3d71d700 -1 mds.b *** got signal Terminated *** 2024-04-23T15:02:40.044 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 15:02:39 smithi173 podman[69843]: 2024-04-23 15:02:39.711383106 +0000 UTC m=+2.443548943 container died e01408b8ab43d9300d4385bb5d32f071bfb3db63512cd8e6cb73151898726338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T15:02:40.045 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 15:02:40 smithi173 podman[69843]: 2024-04-23 15:02:40.045336445 +0000 UTC m=+2.777502284 container cleanup e01408b8ab43d9300d4385bb5d32f071bfb3db63512cd8e6cb73151898726338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b, ceph=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-23T15:02:40.325 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 15:02:40 smithi173 bash[69843]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b 2024-04-23T15:02:40.325 INFO:journalctl@ceph.mds.b.smithi173.stdout:Apr 23 15:02:40 smithi173 podman[69857]: 2024-04-23 15:02:40.263555946 +0000 UTC m=+0.546641662 container remove e01408b8ab43d9300d4385bb5d32f071bfb3db63512cd8e6cb73151898726338 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd) 2024-04-23T15:02:40.467 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.b.service' 2024-04-23T15:02:40.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:02:40.499 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-23T15:02:40.500 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-23T15:02:40.500 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.e 2024-04-23T15:02:40.816 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 15:02:40 smithi173 systemd[1]: Stopping Ceph mds.e for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:02:41.075 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 15:02:40 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-e[62490]: 2024-04-23T15:02:40.816+0000 7f6a2acf1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:02:41.075 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 15:02:40 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-e[62490]: 2024-04-23T15:02:40.816+0000 7f6a2acf1700 -1 mds.e *** got signal Terminated *** 2024-04-23T15:02:43.992 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 15:02:43 smithi173 podman[69964]: 2024-04-23 15:02:43.712846494 +0000 UTC m=+2.991612959 container died 59862320b8af9157b7af44d6a0612237fb8ca5289e93d311ab631a1bacc1b1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-23T15:02:44.325 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 15:02:44 smithi173 podman[69964]: 2024-04-23 15:02:44.083322605 +0000 UTC m=+3.362089068 container cleanup 59862320b8af9157b7af44d6a0612237fb8ca5289e93d311ab631a1bacc1b1b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-23T15:02:44.325 INFO:journalctl@ceph.mds.e.smithi173.stdout:Apr 23 15:02:44 smithi173 bash[69964]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-e 2024-04-23T15:02:44.516 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.e.service' 2024-04-23T15:02:44.553 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:02:44.553 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-23T15:02:44.553 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-23T15:02:44.553 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.h 2024-04-23T15:02:45.075 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 15:02:44 smithi173 systemd[1]: Stopping Ceph mds.h for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:02:45.075 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 15:02:44 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-h[62797]: 2024-04-23T15:02:44.878+0000 7fcac8547700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:02:45.075 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 15:02:44 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-h[62797]: 2024-04-23T15:02:44.878+0000 7fcac8547700 -1 mds.h *** got signal Terminated *** 2024-04-23T15:02:47.988 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 15:02:47 smithi173 podman[70099]: 2024-04-23 15:02:47.710532361 +0000 UTC m=+2.936508942 container died df4755f5aa57ec98459237851e4ebe0a9904a38e39b3cec6be112da7340ddbcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T15:02:48.325 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 15:02:48 smithi173 podman[70099]: 2024-04-23 15:02:48.079044249 +0000 UTC m=+3.305020817 container cleanup df4755f5aa57ec98459237851e4ebe0a9904a38e39b3cec6be112da7340ddbcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T15:02:48.325 INFO:journalctl@ceph.mds.h.smithi173.stdout:Apr 23 15:02:48 smithi173 bash[70099]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-h 2024-04-23T15:02:48.510 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.h.service' 2024-04-23T15:02:48.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:02:48.542 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-23T15:02:48.542 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-23T15:02:48.542 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.k 2024-04-23T15:02:49.075 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 15:02:48 smithi173 systemd[1]: Stopping Ceph mds.k for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:02:49.076 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 15:02:48 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-k[62181]: 2024-04-23T15:02:48.832+0000 7efef8296700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:02:49.076 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 15:02:48 smithi173 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-k[62181]: 2024-04-23T15:02:48.832+0000 7efef8296700 -1 mds.k *** got signal Terminated *** 2024-04-23T15:02:52.325 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 15:02:52 smithi173 podman[70218]: 2024-04-23 15:02:52.072192272 +0000 UTC m=+3.317034008 container died 6db9430fe8edb98a89e013f30b798a6b292ed1f3908e2549c06a358188f50437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T15:02:52.719 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 15:02:52 smithi173 podman[70218]: 2024-04-23 15:02:52.451881329 +0000 UTC m=+3.696723054 container cleanup 6db9430fe8edb98a89e013f30b798a6b292ed1f3908e2549c06a358188f50437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-k, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T15:02:52.719 INFO:journalctl@ceph.mds.k.smithi173.stdout:Apr 23 15:02:52 smithi173 bash[70218]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-k 2024-04-23T15:02:52.878 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.k.service' 2024-04-23T15:02:52.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:02:52.908 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-23T15:02:52.908 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-23T15:02:52.909 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.c 2024-04-23T15:02:53.206 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 15:02:52 smithi203 systemd[1]: Stopping Ceph mds.c for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:02:53.470 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 15:02:53 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c[63252]: 2024-04-23T15:02:53.206+0000 7f148ea13700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:02:53.471 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 15:02:53 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c[63252]: 2024-04-23T15:02:53.206+0000 7f148ea13700 -1 mds.c *** got signal Terminated *** 2024-04-23T15:02:55.971 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 15:02:55 smithi203 podman[70506]: 2024-04-23 15:02:55.715088036 +0000 UTC m=+2.610426116 container died b61521142c823471ba8a98eea9a401f7cd1fdf8194eb06799ce1b7aa0a944c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-23T15:02:56.257 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 15:02:56 smithi203 podman[70506]: 2024-04-23 15:02:56.056279763 +0000 UTC m=+2.951617855 container cleanup b61521142c823471ba8a98eea9a401f7cd1fdf8194eb06799ce1b7aa0a944c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-23T15:02:56.257 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 15:02:56 smithi203 bash[70506]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c 2024-04-23T15:02:56.485 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.c.service' 2024-04-23T15:02:56.507 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 15:02:56 smithi203 podman[70532]: 2024-04-23 15:02:56.257463331 +0000 UTC m=+0.536655647 container remove b61521142c823471ba8a98eea9a401f7cd1fdf8194eb06799ce1b7aa0a944c42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2) 2024-04-23T15:02:56.507 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 15:02:56 smithi203 systemd[1]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.c.service: Deactivated successfully. 2024-04-23T15:02:56.507 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 15:02:56 smithi203 systemd[1]: Stopped Ceph mds.c for 5f4fe708-017e-11ef-bc93-c7b262605968. 2024-04-23T15:02:56.507 INFO:journalctl@ceph.mds.c.smithi203.stdout:Apr 23 15:02:56 smithi203 systemd[1]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.c.service: Consumed 2.524s CPU time. 2024-04-23T15:02:56.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:02:56.521 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-23T15:02:56.521 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-23T15:02:56.521 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.f 2024-04-23T15:02:56.852 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 15:02:56 smithi203 systemd[1]: Stopping Ceph mds.f for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:02:57.220 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 15:02:56 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f[62931]: 2024-04-23T15:02:56.850+0000 7f476676d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:02:57.221 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 15:02:56 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f[62931]: 2024-04-23T15:02:56.850+0000 7f476676d700 -1 mds.f *** got signal Terminated *** 2024-04-23T15:02:59.971 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 15:02:59 smithi203 podman[70637]: 2024-04-23 15:02:59.710015867 +0000 UTC m=+2.974650777 container died 8108ce5b96073d0971db2d759c6a87ede89ae9102688f08fe07927d2e453bfb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-23T15:03:00.471 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 15:03:00 smithi203 podman[70637]: 2024-04-23 15:03:00.052538465 +0000 UTC m=+3.317173373 container cleanup 8108ce5b96073d0971db2d759c6a87ede89ae9102688f08fe07927d2e453bfb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-23T15:03:00.471 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 15:03:00 smithi203 bash[70637]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f 2024-04-23T15:03:00.471 INFO:journalctl@ceph.mds.f.smithi203.stdout:Apr 23 15:03:00 smithi203 podman[70650]: 2024-04-23 15:03:00.278972107 +0000 UTC m=+0.569137023 container remove 8108ce5b96073d0971db2d759c6a87ede89ae9102688f08fe07927d2e453bfb0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T15:03:00.532 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.f.service' 2024-04-23T15:03:00.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:03:00.564 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-23T15:03:00.564 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-23T15:03:00.564 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.i 2024-04-23T15:03:00.897 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 15:03:00 smithi203 systemd[1]: Stopping Ceph mds.i for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:03:01.220 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 15:03:00 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-i[63571]: 2024-04-23T15:03:00.896+0000 7f0e1afa1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:03:01.221 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 15:03:00 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-i[63571]: 2024-04-23T15:03:00.896+0000 7f0e1afa1700 -1 mds.i *** got signal Terminated *** 2024-04-23T15:03:03.971 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 15:03:03 smithi203 podman[70757]: 2024-04-23 15:03:03.717027067 +0000 UTC m=+2.931557163 container died e1125a3808be299cbd7e5b7cb1ba48ecdeb41d797b1985838fe9965f581df481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-23T15:03:04.356 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 15:03:04 smithi203 podman[70757]: 2024-04-23 15:03:04.114657099 +0000 UTC m=+3.329187195 container cleanup e1125a3808be299cbd7e5b7cb1ba48ecdeb41d797b1985838fe9965f581df481 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-23T15:03:04.357 INFO:journalctl@ceph.mds.i.smithi203.stdout:Apr 23 15:03:04 smithi203 bash[70757]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-i 2024-04-23T15:03:04.510 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.i.service' 2024-04-23T15:03:04.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:03:04.541 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-23T15:03:04.542 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-23T15:03:04.542 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.l 2024-04-23T15:03:04.971 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 15:03:04 smithi203 systemd[1]: Stopping Ceph mds.l for 5f4fe708-017e-11ef-bc93-c7b262605968... 2024-04-23T15:03:04.971 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 15:03:04 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-l[62622]: 2024-04-23T15:03:04.866+0000 7f7874ac1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T15:03:04.971 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 15:03:04 smithi203 ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-l[62622]: 2024-04-23T15:03:04.866+0000 7f7874ac1700 -1 mds.l *** got signal Terminated *** 2024-04-23T15:03:09.220 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 15:03:08 smithi203 podman[70876]: 2024-04-23 15:03:08.874897338 +0000 UTC m=+4.114790276 container died 976d5bb48ff328ff67081bfdc8dd722a7d3c11f274731a75e3d3bf0d9b0f88bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T15:03:09.527 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 15:03:09 smithi203 podman[70876]: 2024-04-23 15:03:09.326886269 +0000 UTC m=+4.566779196 container cleanup 976d5bb48ff328ff67081bfdc8dd722a7d3c11f274731a75e3d3bf0d9b0f88bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af2350ee15fec7863cae8207e1de3b5c3baf4b211768c50629a9f2fdb9a2f3cd, name=ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240416.193735-3-d44fddd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-23T15:03:09.527 INFO:journalctl@ceph.mds.l.smithi203.stdout:Apr 23 15:03:09 smithi203 bash[70876]: ceph-5f4fe708-017e-11ef-bc93-c7b262605968-mds-l 2024-04-23T15:03:09.752 DEBUG:teuthology.orchestra.run.smithi203:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5f4fe708-017e-11ef-bc93-c7b262605968@mds.l.service' 2024-04-23T15:03:09.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T15:03:09.784 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-23T15:03:09.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T15:03:10.094 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T15:03:15.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T15:03:15.228 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T15:03:17.762 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T15:03:17.953 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T15:03:20.377 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T15:03:20.469 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T15:03:20.500 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T15:03:20.526 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-23T15:03:20.528 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569/remote/smithi083/crash 2024-04-23T15:03:20.529 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/crash -- . 2024-04-23T15:03:20.619 INFO:teuthology.orchestra.run.smithi083.stderr:tar: /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T15:03:20.620 INFO:teuthology.orchestra.run.smithi083.stderr:tar: Error is not recoverable: exiting now 2024-04-23T15:03:20.624 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569/remote/smithi173/crash 2024-04-23T15:03:20.624 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/crash -- . 2024-04-23T15:03:20.668 INFO:teuthology.orchestra.run.smithi173.stderr:tar: /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T15:03:20.668 INFO:teuthology.orchestra.run.smithi173.stderr:tar: Error is not recoverable: exiting now 2024-04-23T15:03:20.670 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569/remote/smithi203/crash 2024-04-23T15:03:20.671 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/crash -- . 2024-04-23T15:03:20.708 INFO:teuthology.orchestra.run.smithi203.stderr:tar: /var/lib/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T15:03:20.708 INFO:teuthology.orchestra.run.smithi203.stderr:tar: Error is not recoverable: exiting now 2024-04-23T15:03:20.710 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-23T15:03:20.710 DEBUG:teuthology.orchestra.run.smithi083:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-23T15:03:20.856 INFO:teuthology.orchestra.run.smithi083.stdout:2024-04-23T14:52:48.966177+0000 mon.a (mon.0) 1190 : cluster [WRN] Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-23T15:03:20.857 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-23T15:03:20.857 INFO:tasks.cephadm:Compressing logs... 2024-04-23T15:03:20.857 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T15:03:20.862 DEBUG:teuthology.orchestra.run.smithi173:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T15:03:20.864 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T15:03:20.888 INFO:teuthology.orchestra.run.smithi203.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T15:03:20.889 INFO:teuthology.orchestra.run.smithi203.stderr:: No such file or directory 2024-04-23T15:03:20.889 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.log 2024-04-23T15:03:20.889 INFO:teuthology.orchestra.run.smithi203.stderr:gzip/var/log/ceph/cephadm.log: -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-23T15:03:20.890 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-23T15:03:20.890 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T15:03:20.890 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-23T15:03:20.896 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-23T15:03:20.896 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-23T15:03:20.896 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-23T15:03:20.896 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-23T15:03:20.896 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T15:03:20.896 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-23T15:03:20.896 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-23T15:03:20.896 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T15:03:20.897 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-23T15:03:20.898 INFO:teuthology.orchestra.run.smithi173.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T15:03:20.899 INFO:teuthology.orchestra.run.smithi173.stderr:: No such file or directory 2024-04-23T15:03:20.899 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.log 2024-04-23T15:03:20.899 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T15:03:20.899 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-23T15:03:20.899 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-23T15:03:20.899 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-23T15:03:20.899 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-23T15:03:20.900 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-23T15:03:20.900 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-23T15:03:20.900 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-23T15:03:20.900 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T15:03:20.900 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-23T15:03:20.900 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T15:03:20.900 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-23T15:03:20.900 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-23T15:03:20.900 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mon.b.log: /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-23T15:03:20.931 INFO:teuthology.orchestra.run.smithi083.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T15:03:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:: No such file or directory 2024-04-23T15:03:20.932 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.log 2024-04-23T15:03:20.933 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T15:03:20.934 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-23T15:03:20.935 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-23T15:03:20.936 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-23T15:03:20.937 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-23T15:03:20.937 INFO:teuthology.orchestra.run.smithi083.stderr: 87.2% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T15:03:20.938 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-23T15:03:20.939 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-23T15:03:20.940 INFO:teuthology.orchestra.run.smithi083.stderr: 94.4% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-23T15:03:20.941 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-23T15:03:20.941 INFO:teuthology.orchestra.run.smithi083.stderr: 94.4% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-23T15:03:20.941 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T15:03:20.942 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-23T15:03:20.943 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T15:03:20.944 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-23T15:03:20.945 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-23T15:03:20.948 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.d.log: /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.4% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-23T15:03:20.950 INFO:teuthology.orchestra.run.smithi083.stderr: 94.4% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-23T15:03:20.953 INFO:teuthology.orchestra.run.smithi173.stderr: 83.9% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T15:03:20.971 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.l.log: 83.8% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T15:03:20.973 INFO:teuthology.orchestra.run.smithi173.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T15:03:20.973 INFO:teuthology.orchestra.run.smithi173.stderr: 90.3% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T15:03:20.973 INFO:teuthology.orchestra.run.smithi173.stderr: 87.2% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T15:03:20.973 INFO:teuthology.orchestra.run.smithi203.stderr: 90.3% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T15:03:20.995 INFO:teuthology.orchestra.run.smithi083.stderr: 90.1% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T15:03:20.998 INFO:teuthology.orchestra.run.smithi203.stderr: 94.6% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-23T15:03:21.008 INFO:teuthology.orchestra.run.smithi083.stderr: 84.2% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T15:03:21.050 INFO:teuthology.orchestra.run.smithi173.stderr: 94.4% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-23T15:03:21.061 INFO:teuthology.orchestra.run.smithi173.stderr: 92.0% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-23T15:03:21.074 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T15:03:21.101 INFO:teuthology.orchestra.run.smithi173.stderr: 91.0% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T15:03:21.107 INFO:teuthology.orchestra.run.smithi173.stderr: 92.8% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-23T15:03:21.134 INFO:teuthology.orchestra.run.smithi083.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T15:03:21.163 INFO:teuthology.orchestra.run.smithi203.stderr: 87.2% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T15:03:21.167 INFO:teuthology.orchestra.run.smithi203.stderr: 92.1% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-23T15:03:21.188 INFO:teuthology.orchestra.run.smithi203.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T15:03:21.461 INFO:teuthology.orchestra.run.smithi173.stderr: 91.8% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-23T15:03:21.544 INFO:teuthology.orchestra.run.smithi203.stderr: 91.0% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T15:03:21.559 INFO:teuthology.orchestra.run.smithi203.stderr: 94.6% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-23T15:03:21.791 INFO:teuthology.orchestra.run.smithi083.stderr: 90.4% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-23T15:03:21.826 INFO:teuthology.orchestra.run.smithi203.stderr: 91.7% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-23T15:03:21.931 INFO:teuthology.orchestra.run.smithi203.stderr: 95.6% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-23T15:03:23.613 INFO:teuthology.orchestra.run.smithi173.stderr: 93.3% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-23T15:03:24.327 INFO:teuthology.orchestra.run.smithi083.stderr: 90.3% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-23T15:03:30.681 INFO:teuthology.orchestra.run.smithi203.stderr: 94.1% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-23T15:04:10.217 INFO:teuthology.orchestra.run.smithi173.stderr: 92.2% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-23T15:04:38.491 INFO:teuthology.orchestra.run.smithi203.stderr: 92.2% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-23T15:04:45.126 INFO:teuthology.orchestra.run.smithi203.stderr: 92.2% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-23T15:04:47.308 INFO:teuthology.orchestra.run.smithi203.stderr: 92.3% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-23T15:04:54.649 INFO:teuthology.orchestra.run.smithi203.stderr: 92.3% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-23T15:04:54.651 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-04-23T15:04:54.651 INFO:teuthology.orchestra.run.smithi203.stderr:real 1m33.772s 2024-04-23T15:04:54.651 INFO:teuthology.orchestra.run.smithi203.stderr:user 3m50.822s 2024-04-23T15:04:54.651 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m7.590s 2024-04-23T15:04:54.747 INFO:teuthology.orchestra.run.smithi173.stderr: 92.2% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-23T15:05:10.191 INFO:teuthology.orchestra.run.smithi173.stderr: 92.3% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-23T15:05:15.644 INFO:teuthology.orchestra.run.smithi173.stderr: 92.1% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-23T15:05:15.646 INFO:teuthology.orchestra.run.smithi173.stderr: 2024-04-23T15:05:15.647 INFO:teuthology.orchestra.run.smithi173.stderr:real 1m54.767s 2024-04-23T15:05:15.647 INFO:teuthology.orchestra.run.smithi173.stderr:user 3m52.162s 2024-04-23T15:05:15.647 INFO:teuthology.orchestra.run.smithi173.stderr:sys 0m7.747s 2024-04-23T15:05:42.343 INFO:teuthology.orchestra.run.smithi083.stderr: 92.4% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-23T15:06:15.978 INFO:teuthology.orchestra.run.smithi083.stderr: 92.2% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-23T15:06:20.821 INFO:teuthology.orchestra.run.smithi083.stderr: 92.2% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-23T15:06:25.119 INFO:teuthology.orchestra.run.smithi083.stderr: 92.1% -- replaced with /var/log/ceph/5f4fe708-017e-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-23T15:06:25.122 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-23T15:06:25.122 INFO:teuthology.orchestra.run.smithi083.stderr:real 3m4.222s 2024-04-23T15:06:25.122 INFO:teuthology.orchestra.run.smithi083.stderr:user 5m42.150s 2024-04-23T15:06:25.122 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m36.660s 2024-04-23T15:06:25.124 INFO:tasks.cephadm:Archiving logs... 2024-04-23T15:06:25.124 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569/remote/smithi083/log 2024-04-23T15:06:25.126 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T15:07:02.635 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569/remote/smithi173/log 2024-04-23T15:07:02.636 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T15:07:40.557 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569/remote/smithi203/log 2024-04-23T15:07:40.558 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T15:08:17.878 INFO:tasks.cephadm:Removing cluster... 2024-04-23T15:08:17.878 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 --force 2024-04-23T15:08:18.196 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T15:08:19.438 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 --force 2024-04-23T15:08:19.630 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T15:08:20.161 DEBUG:teuthology.orchestra.run.smithi203:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5f4fe708-017e-11ef-bc93-c7b262605968 --force 2024-04-23T15:08:20.353 INFO:teuthology.orchestra.run.smithi203.stdout:Deleting cluster with fsid: 5f4fe708-017e-11ef-bc93-c7b262605968 2024-04-23T15:08:20.894 INFO:tasks.cephadm:Removing cephadm ... 2024-04-23T15:08:20.894 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T15:08:20.952 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T15:08:20.970 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T15:08:20.988 INFO:tasks.cephadm:Teardown complete 2024-04-23T15:08:20.989 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T15:08:21.001 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T15:08:21.001 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T15:08:21.008 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T15:08:21.013 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T15:08:21.183 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T15:08:21.193 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T15:08:21.194 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T15:08:21.202 DEBUG:teuthology.orchestra.run.smithi173:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T15:08:21.204 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T15:08:21.218 INFO:teuthology.orchestra.run.smithi203.stderr:bash: line 1: ntpq: command not found 2024-04-23T15:08:21.219 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 1: ntpq: command not found 2024-04-23T15:08:21.230 INFO:teuthology.orchestra.run.smithi203.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T15:08:21.230 INFO:teuthology.orchestra.run.smithi203.stdout:=============================================================================== 2024-04-23T15:08:21.230 INFO:teuthology.orchestra.run.smithi203.stdout:^- hv01.front.sepia.ceph.com 2 6 377 16 +810us[ +817us] +/- 593ms 2024-04-23T15:08:21.230 INFO:teuthology.orchestra.run.smithi203.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -587us[ -580us] +/- 58ms 2024-04-23T15:08:21.231 INFO:teuthology.orchestra.run.smithi203.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +689us[ +689us] +/- 70ms 2024-04-23T15:08:21.231 INFO:teuthology.orchestra.run.smithi203.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T15:08:21.240 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T15:08:21.240 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2024-04-23T15:08:21.240 INFO:teuthology.orchestra.run.smithi173.stdout:^- hv01.front.sepia.ceph.com 2 6 377 16 +801us[ +801us] +/- 593ms 2024-04-23T15:08:21.240 INFO:teuthology.orchestra.run.smithi173.stdout:^* hv02.front.sepia.ceph.com 3 6 377 17 -594us[ -600us] +/- 58ms 2024-04-23T15:08:21.241 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +692us[ +686us] +/- 70ms 2024-04-23T15:08:21.241 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T15:08:21.248 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-23T15:08:21.259 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T15:08:21.259 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-23T15:08:21.259 INFO:teuthology.orchestra.run.smithi083.stdout:^- hv01.front.sepia.ceph.com 2 7 377 14 +889us[ +891us] +/- 593ms 2024-04-23T15:08:21.260 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -550us[ -548us] +/- 58ms 2024-04-23T15:08:21.260 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +726us[ +728us] +/- 70ms 2024-04-23T15:08:21.260 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T15:08:21.261 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T15:08:21.274 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T15:08:21.275 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T15:08:21.287 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T15:08:21.523 DEBUG:teuthology.orchestra.run.smithi173:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T15:08:21.605 DEBUG:teuthology.orchestra.run.smithi203:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T15:08:21.722 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T15:08:21.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T15:08:21.816 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T15:08:21.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T15:08:21.848 DEBUG:teuthology.orchestra.run.smithi203:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T15:08:21.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T15:08:21.876 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T15:08:21.888 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T15:08:21.902 INFO:teuthology.task.internal:Duration was 2603.506206 seconds 2024-04-23T15:08:21.903 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T15:08:21.914 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T15:08:21.914 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T15:08:21.923 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T15:08:21.926 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T15:08:21.977 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T15:08:22.001 INFO:teuthology.orchestra.run.smithi203.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T15:08:22.107 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T15:08:22.613 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T15:08:22.613 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-04-23T15:08:22.613 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T15:08:22.674 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi173.front.sepia.ceph.com 2024-04-23T15:08:22.675 DEBUG:teuthology.orchestra.run.smithi173:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T15:08:22.711 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-04-23T15:08:22.711 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T15:08:22.737 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T15:08:22.738 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T15:08:22.746 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T15:08:22.754 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T15:08:22.831 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T15:08:22.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T15:08:22.837 DEBUG:teuthology.orchestra.run.smithi173:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T15:08:22.840 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T15:08:23.330 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T15:08:23.350 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T15:08:23.350 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T15:08:23.452 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T15:08:23.483 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T15:08:23.517 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T15:08:23.530 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T15:08:23.538 DEBUG:teuthology.orchestra.run.smithi173:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T15:08:23.540 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T15:08:23.566 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = core 2024-04-23T15:08:23.582 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-04-23T15:08:23.652 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-04-23T15:08:23.699 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T15:08:23.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T15:08:23.750 DEBUG:teuthology.orchestra.run.smithi173:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T15:08:23.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T15:08:23.765 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T15:08:23.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T15:08:23.780 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T15:08:23.792 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T15:08:23.793 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569/remote/smithi083 2024-04-23T15:08:23.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T15:08:23.898 DEBUG:teuthology.misc:Transferring archived files from smithi173:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569/remote/smithi173 2024-04-23T15:08:23.898 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T15:08:23.936 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569/remote/smithi203 2024-04-23T15:08:23.936 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T15:08:23.975 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T15:08:23.975 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T15:08:23.983 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T15:08:23.985 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T15:08:24.034 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T15:08:24.048 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T15:08:24.048 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T15:08:24.062 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T15:08:24.063 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T15:08:24.070 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T15:08:24.072 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T15:08:24.085 INFO:teuthology.orchestra.run.smithi173.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 15:08 /home/ubuntu/cephtest 2024-04-23T15:08:24.089 INFO:teuthology.orchestra.run.smithi203.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 15:08 /home/ubuntu/cephtest 2024-04-23T15:08:24.119 INFO:teuthology.orchestra.run.smithi083.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 15:08 /home/ubuntu/cephtest 2024-04-23T15:08:24.122 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T15:08:24.135 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T15:08:24.167 INFO:teuthology.nuke:Checking targets against current locks 2024-04-23T15:08:24.196 DEBUG:teuthology.nuke:shortname: smithi083 2024-04-23T15:08:24.196 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T15:08:24.224 DEBUG:teuthology.nuke:shortname: smithi173 2024-04-23T15:08:24.224 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T15:08:24.229 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:35.710731', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOJKqACtIvRJOSpWkZ2GCQaksijfrwGP67CDNXeRoTCGUvgv2WtsWa5yRAoa/rR6n1D8NUQx0we9O6O7w5S9njE='} 2024-04-23T15:08:24.250 DEBUG:teuthology.nuke:shortname: smithi203 2024-04-23T15:08:24.250 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T15:08:24.255 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:35.712293', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIxtn2o/mrPu1GucuiHWSWJIR0rGPngFs0WNIXdLVpxCwpbhetQOBVJ57z8Ciy4t0NJyG82VYYdcnsAIs5qASq8='} 2024-04-23T15:08:24.287 INFO:teuthology.orchestra.console.smithi083:Power off 2024-04-23T15:08:24.287 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T15:08:24.299 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-23_14:04:26-fs-wip-rishabh-testing-20240416.193735-3-testing-default-smithi/7669569', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 14:05:35.713353', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEk+B8YSfkKV1bLDhgD+UbEpqvPgJV4Nd6crtUgg+mmFFCWmDQXiT52cTXhTVeien6aQYUmTYsGz7GVQv1m4vKQ='} 2024-04-23T15:08:24.311 DEBUG:teuthology.orchestra.console.smithi083:power off output: Chassis Power Control: Down/Off 2024-04-23T15:08:24.311 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T15:08:24.332 INFO:teuthology.orchestra.console.smithi173:Power off 2024-04-23T15:08:24.333 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T15:08:24.350 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-23T15:08:24.355 DEBUG:teuthology.orchestra.console.smithi173:power off output: Chassis Power Control: Down/Off 2024-04-23T15:08:24.355 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T15:08:24.378 INFO:teuthology.orchestra.console.smithi203:Power off 2024-04-23T15:08:24.378 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T15:08:24.388 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-04-23T15:08:24.401 DEBUG:teuthology.orchestra.console.smithi203:power off output: Chassis Power Control: Down/Off 2024-04-23T15:08:24.401 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T15:08:24.426 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2024-04-23T15:08:28.351 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T15:08:28.389 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T15:08:28.428 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T15:08:28.466 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-23T15:08:28.499 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2024-04-23T15:08:28.538 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is on 2024-04-23T15:08:32.470 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T15:08:32.500 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T15:08:32.539 DEBUG:teuthology.orchestra.console.smithi203:pexpect command: ipmitool -H smithi203.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T15:08:32.582 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is on 2024-04-23T15:08:32.610 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2024-04-23T15:08:32.650 DEBUG:teuthology.orchestra.console.smithi203:check power output: Chassis Power is off 2024-04-23T15:08:32.711 INFO:teuthology.orchestra.console.smithi173:Power off completed 2024-04-23T15:08:32.751 INFO:teuthology.orchestra.console.smithi203:Power off completed 2024-04-23T15:08:36.586 DEBUG:teuthology.orchestra.console.smithi083:pexpect command: ipmitool -H smithi083.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T15:08:36.697 DEBUG:teuthology.orchestra.console.smithi083:check power output: Chassis Power is off 2024-04-23T15:08:36.798 INFO:teuthology.orchestra.console.smithi083:Power off completed 2024-04-23T15:08:36.905 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/ffsb}} duration: 2603.506205558777 failure_reason: 'Command failed (workunit test suites/ffsb.sh) on smithi083 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d44fddd910b5e94da0f2357af4be37d7bd495ae0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=0fd6da3dc7c344d199b1a9e1c0a3971b status: fail success: false 2024-04-23T15:08:36.906 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T15:08:36.993 INFO:teuthology.run:FAIL